Eight-section brocade action recognition auxiliary device based on multi-modal data fusion

Abstract

The utility model discloses an eight-segment brocade motion identification auxiliary device based on multi-modal data fusion, which comprises a wearing bandage, an integrated module, a power supply module, a feedback module and an image analysis module, the wearing bandage is provided with a magic tape, a mounting cavity and a buffer layer are arranged in the wearing bandage, and the integrated module comprises a nine-axis attitude sensor, a data processing unit and a Bluetooth unit. The motion data can be collected, processed and sent to an external terminal, the image analysis module is composed of a camera and a processor and can capture motion images, position joint points and calculate the similarity with standard motion, the power supply module is a bendable lithium polymer battery, and the feedback module is provided with a vibration unit and a terminal graphical interface and achieves tactile and visual feedback. And the data processing unit and the image analysis module synchronize data through Bluetooth and fuse the data to generate a comprehensive evaluation result. In use, the device can assist a user in accurately identifying eight-segment brocade actions, provides comprehensive action evaluation through multi-modal data fusion, helps the user to correct the actions in time, and improves the exercise effect.

Description

Technical Field

[0001] This utility model relates to the field of sports assistive device technology, specifically to an assistive device for recognizing Baduanjin movements based on multimodal data fusion. Background Technology

[0002] As society ages, more and more elderly people are paying attention to physical exercise, and health exercises such as Baduanjin are suitable for them. However, it is inconvenient for the elderly to concentrate on learning, and their movements vary greatly, making it difficult to achieve the expected results. Existing smart bracelets rely on three-axis accelerometers to detect human vibrations and record the number of runs, but they cannot detect the curved movement trajectory of health exercises, nor can they verify the degree of exercise in traditional health exercises such as Baduanjin.

[0003] Chinese patent CN105797353B discloses a monitoring device for a Baduanjin training bracelet, including a microcontroller, a heart rate and blood oxygen sensing unit, a position sensing unit, a storage unit, a button unit, a display unit, a communication interface unit, a music unit, and a charging unit.

[0004] Chinese patent CN208287003U discloses a device for collecting movement trajectory data for Baduanjin (Eight Pieces of Brocade) training. The device includes a strip-shaped strap for fixing to the body, and a microcontroller unit (MCU), a six-axis sensor, a FLASH memory, a heart rate sensor, a blood oxygen sensor, a system power supply, a charging circuit, and a communication interface disposed inside the strap. The six-axis sensor transmits the collected movement trajectory data to the MCU as a signal. The heart rate sensor and blood oxygen sensor transmit the collected heart rate data and blood oxygen content data to the MCU as signals, respectively. The movement trajectory data, heart rate data, and blood oxygen content data are processed by the MCU and stored in the FLASH memory. The system power supply and charging circuit are connected to the power supply terminal and charging terminal of the MCU, respectively.

[0005] In practical use, although the Chinese patent with publication number CN105797353B introduces image comparison, it does not solve the problem of zero sensor drift. Although the Chinese patent with publication number CN208287003U can monitor motion details, it lacks visual data supplementation.

[0006] Therefore, there is a need for an auxiliary device for recognizing Baduanjin movements based on multimodal data fusion to solve the above problems. Utility Model Content

[0007] Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this utility model provides an auxiliary device for recognizing Baduanjin movements based on multimodal data fusion. This solves the problems mentioned in the background regarding existing Baduanjin devices, such as the inability to accurately detect the curved motion trajectory of the exercise movements, the inability to assess the degree of Baduanjin practice, and the lack of sensor zero-drift issues in patent CN105797353B and the absence of supplementary visual data in patent CN208287003U. This device achieves more accurate recognition of Baduanjin movements in the elderly and more effective assessment of the degree of practice, while simultaneously resolving the sensor zero-drift problem and supplementing visual data.

[0009] Technical solution

[0010] To achieve the above objectives, this utility model provides the following technical solution: an auxiliary device for recognizing Baduanjin movements based on multimodal data fusion, comprising a wear strap, an integrated module, a power supply module, a feedback module, and an image analysis module. The two ends of the wear strap are respectively fixed to the two ends of the mounting cavity. The integrated module and the power supply module are encapsulated in the mounting cavity. The feedback module includes a vibration unit and a terminal graphical interface. The vibration unit is encapsulated in the mounting cavity, and the terminal graphical interface is embedded in the image analysis module. The integrated module and the vibration unit are electrically connected to the power supply module.

[0011] Furthermore, the wearable strap has a two-section structure, which is fixedly connected by Velcro.

[0012] Furthermore, the integrated module includes a nine-axis attitude sensor, a data processing unit, and a Bluetooth unit. The nine-axis attitude sensor, data processing unit, Bluetooth unit, and vibration unit are connected sequentially via electrical signals and are each electrically connected to a power supply module.

[0013] Furthermore, the image analysis module includes a camera and a processor, which are fixed together by a housing. The terminal graphical interface is embedded in the housing. The camera, processor, and terminal graphical interface are connected in sequence by electrical signals, and the Bluetooth unit is connected to the processor by Bluetooth signals.

[0014] Furthermore, the mounting cavity has a shock-absorbing buffer layer installed on the surface facing the Velcro, and a gap is provided on the side of the mounting cavity, where a breathable mesh layer is installed.

[0015] Furthermore, the power supply module is a flexible lithium polymer battery.

[0016] Furthermore, the inner shock-absorbing layer of the wearable strap is made of silicone material with a hardness of 30A.

[0017] Beneficial effects

[0018] This utility model provides an auxiliary device for recognizing movements in Baduanjin (Eight Pieces of Brocade) based on multimodal data fusion, which has the following beneficial effects:

[0019] 1. In practical use, the integration of sensor data and visual data overcomes the limitations of a single data source and solves the problems of sensor zero drift and lack of visual data supplementation; it can accurately identify actions, detect action deviations in real time and provide feedback to help users correct their actions in a timely manner.

[0020] 2. In actual use, the device uses elastic and breathable straps, combined with a visual interface, making it easy to use and lowering the barrier to entry. It is suitable for users of different body types, especially for elderly people to exercise at home, and can effectively improve the effect of Baduanjin exercises.

[0021] 3. In actual use, by setting a breathable mesh layer, the heat dissipation problem of the installation cavity can be effectively solved. At the same time, the breathable mesh layer can buffer the impact when the installation cavity comes into hard contact with an object due to a fall. Using a breathable mesh layer instead of other rigid mesh can also effectively prevent the mesh from falling off after a hard impact and exposing sharp parts that could cause damage. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of the bandage-wearing part in this utility model;

[0023] Figure 2 This is a schematic diagram of the image analysis module structure in this utility model;

[0024] Figure 3 This is a schematic diagram illustrating the principle of an embodiment of the present utility model.

[0025] In the diagram: 1. Wearing straps; 11. Velcro; 12. Mounting cavity; 13. Breathable mesh layer; 14. Shock-absorbing buffer layer; 2. Integrated module; 21. Nine-axis attitude sensor; 22. Data processing unit; 23. Bluetooth unit; 3. Image analysis module; 31. Camera; 32. Processor; 4. Power supply module; 51. Vibration unit; 52. Terminal graphical interface. Detailed Implementation

[0026] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. Based on the embodiments of the present utility model, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present utility model.

[0027] In the description of this utility model, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this utility model and simplifying the description, and are not intended to indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this utility model.

[0028] Please see Figure 1-3 The present invention provides a technical solution: the two ends of the wear strap 1 are respectively fixed to the two ends of the mounting cavity 12, the integrated module 2 and the power supply module 4 are encapsulated in the mounting cavity 12, the feedback module 5 includes a vibration unit 51 and a terminal graphical interface 52, the vibration unit 51 is encapsulated in the mounting cavity 12, the terminal graphical interface 52 is embedded in the image analysis module 3, and the integrated module 2 and the vibration unit 51 are electrically connected to the power supply module 4 respectively;

[0029] The wearing strap 1 has a two-section structure, which is fixedly connected by Velcro 11.

[0030] The integrated module 2 includes a nine-axis attitude sensor 21, a data processing unit 22, and a Bluetooth unit 23. The nine-axis attitude sensor 21, the data processing unit 22, the Bluetooth unit 23, and the vibration unit 51 are connected in sequence via electrical signals and are respectively connected to the power supply module 4.

[0031] Integrated module 2 includes a nine-axis attitude sensor 21, a data processing unit 22, and a Bluetooth unit 23. The Bluetooth unit 23 sends motion data to an external terminal.

[0032] The image analysis module 3 includes a camera 31 and a processor 32, which are fixed together by a housing. The camera 31 captures motion images, and the processor 32 uses image analysis algorithms to locate key points and calculate the similarity to standard motion.

[0033] Feedback module 5 includes a vibration unit 51 and a terminal graphical interface 52; the terminal graphical interface 52 is embedded in the housing, and the camera 31, processor 32, and terminal graphical interface 52 are sequentially connected via electrical signals. Bluetooth unit 23 is connected to processor 32 via Bluetooth signals. Power supply module 4 is a flexible lithium polymer battery.

[0034] The mounting cavity 12 has a shock-absorbing buffer layer 14 installed on the surface facing the Velcro 11. The mounting cavity 12 has a gap on its side, and a breathable mesh layer 13 is installed in the gap.

[0035] By setting a breathable mesh layer, the heat dissipation problem of the installation cavity can be effectively solved. At the same time, the breathable mesh layer can buffer the impact when the installation cavity comes into hard contact with an object due to a fall. Using a breathable mesh layer instead of other rigid mesh can also effectively prevent the mesh from falling off after a hard impact and exposing sharp parts that could cause damage.

[0036] The vibration unit 51 triggers tactile prompts based on the motion recognition results, and the terminal graphical interface 52 displays the camera 31 image, joint point markings, similarity scores, and motion correction prompts in real time; the data processing unit 22 and the image analysis module 3 achieve data synchronization through the Bluetooth unit 23.

[0037] During use, the nine-axis attitude sensor 21 can collect three-axis acceleration, three-axis angular velocity, and three-axis magnetic force data of the user's movements in real time; the data processing unit 22 performs Kalman filtering noise reduction, local coordinate system transformation, and time window segmentation on the raw data collected by the nine-axis attitude sensor 21 in sequence to generate motion data; the Bluetooth unit 23 sends the motion data generated by the data processing unit 22 to an external terminal for user viewing and subsequent analysis; the camera 31 in the image analysis module 3 captures motion images, and the processor 32 locates joint points and calculates the similarity with standard movements through image analysis algorithms; the power supply module 4 is electrically connected to the integrated module 2 to provide stable power support for the entire device; the vibration unit 51 in the feedback module 5 triggers tactile prompts based on the motion recognition results, allowing the user to perceive the correctness of their movements through touch in real time and make timely adjustments; the terminal graphical interface 52 displays the camera 31 image, joint point markings, similarity scores, and motion correction prompts in real time.

[0038] The wearing strap 1 is made of elastic fabric, which allows the wearing strap 1 to stretch and deform to a certain extent according to the shape and movement of the user's body parts.

[0039] The similarity calculation of image analysis module 3 includes: extracting the angle features of the shoulder, elbow, hip and knee in the user's action image, and using a dynamic weighted algorithm to calculate the error value between it and the standard action. The similarity calculation part of image analysis module 3 provides key data for action evaluation by extracting the angle features of the shoulder, elbow, hip and knee, which are key joints of the Baduanjin movement, in the user's action image, and comprehensively judges the difference between the user's action and the standard action from multiple dimensions.

[0040] The Kalman filter noise reduction parameters of the data processing unit 22 are dynamically adjusted according to the zero drift characteristics of the nine-axis sensor. The Kalman filter noise reduction parameters can be dynamically adjusted according to the zero drift characteristics of the nine-axis sensor, which allows the Kalman filter to be optimized in real time according to the actual zero drift of the nine-axis sensor. This enables more accurate noise reduction processing of the raw data collected by the nine-axis sensor, which includes three-axis acceleration, three-axis angular velocity and three-axis magnetic force, and effectively eliminates the error caused by zero drift.

[0041] The inner shock-absorbing layer 14 of the wearable strap 1 is made of silicone with a hardness of 30A. This silicone material effectively cushions the vibrations and impacts generated by body movements in components such as the nine-axis attitude sensor 21 within the integrated module 2 during Baduanjin training, preventing loosening or damage and ensuring normal operation. Furthermore, the softness and suitable hardness of the silicone material allow it to conform to the skin, providing a comfortable wearing experience.

[0042] The processor 32 of the image analysis module 3 is configured to locate human joints using the Mediapipe algorithm and construct a posture framework using the OpenPose algorithm. The posture framework is used to calculate the joint angle error values ​​between the user's action image and the standard action. The processor 32 of the image analysis module 3 uses the Mediapipe algorithm to accurately locate joints such as the shoulder, elbow, hip, and knee of the human body, providing key basic information for action analysis. The posture framework constructed using the OpenPose algorithm can comprehensively and systematically present the human body's action posture. Based on this posture framework, the processor 32 can accurately calculate the joint angle error values ​​between the user's action image and the standard action, and achieve accurate evaluation of the standardization of the user's Baduanjin movements by quantifying the error, thereby providing a reliable basis for the feedback module 5.

[0043] As one embodiment of this utility model: when using an Eight-Section Brocade movement recognition auxiliary device based on multimodal data fusion:

[0044] 1. First, the user puts the wearable strap 1 on a suitable part of the body, such as the arm or leg, and adjusts the tightness to a comfortable level using the Velcro 11 at both ends. Because the strap is made of elastic fabric and covered with a breathable mesh layer 13, it not only adapts to changes in body movement but also keeps the skin dry and comfortable;

[0045] 2. After the device is turned on, the power supply module 4 begins to supply power to various components such as the integrated attitude sensor module and the image analysis module 3. The nine-axis attitude sensor 21 immediately begins to collect real-time data on the user's three-axis acceleration, three-axis angular velocity, and three-axis magnetic force. This raw data is transmitted to the data processing unit 22. The data processing unit 22 performs Kalman filtering noise reduction, local coordinate system transformation, and time window segmentation on the raw data according to a preset program to generate optimized motion feature data. The Kalman filtering noise reduction parameters are dynamically adjusted according to the zero-drift characteristics of the nine-axis sensor to ensure the accuracy of the data.

[0046] 3. Simultaneously, the camera 31 of the image analysis module 3 captures images of the user's Baduanjin movements at a certain frame rate. The processor 32 uses the Mediapipe algorithm to locate the human joints, and then combines it with the OpenPose algorithm to construct a posture framework. It then calculates the angle features of the shoulder, elbow, hip, and knee in the user's movement image, and performs weighted error calculation with the corresponding features of the standard movement image to obtain a similarity score.

[0047] 4. The motion feature data generated by the sensor module and the similarity score obtained by the image analysis module 3 are synchronized to an external terminal via the Bluetooth transmission unit. The data processing unit 22 and the image analysis module 3 also synchronize data via the Bluetooth transmission unit, fusing sensor feature data and image angle features to generate a comprehensive motion evaluation result;

[0048] 5. Feedback module 5 starts working based on the comprehensive movement evaluation results. If the movement is not standardized, the vibration unit 51 on the inside of the strap will promptly trigger a tactile prompt, allowing the user to notice the error. The terminal graphical interface 52 displays the camera 31's image, joint point markings, and similarity score in real time, and provides specific correction suggestions in the movement correction prompt area, such as prompting the user that "the shoulder angle needs to be opened by another 5°" and "the knees should be kept slightly bent." Based on this feedback information, the user adjusts their movements in real time, gradually making the Baduanjin movements more standardized and standard.

[0049] Specifically, throughout the entire practice process, the device continuously collects, analyzes, and provides feedback on data, forming a complete closed loop for motion optimization, helping users to continuously improve the effectiveness of Baduanjin practice.

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0051] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal data fusion-based movement recognition auxiliary device for Baduanjin (a traditional Chinese exercise), comprising a wearable strap (1), an integrated module (2), a power supply module (4), a feedback module (5), and an image analysis module (3), characterized in that: The two ends of the wearing strap (1) are respectively fixed to the two ends of the mounting cavity (12). The integrated module (2) and the power supply module (4) are encapsulated in the mounting cavity (12). The feedback module (5) includes a vibration unit (51) and a terminal graphical interface (52). The vibration unit (51) is encapsulated in the mounting cavity (12). The terminal graphical interface (52) is embedded in the image analysis module (3). The integrated module (2) and the vibration unit (51) are electrically connected to the power supply module (4).

2. The Eight-Section Brocade Movement Recognition Assistive Device based on Multimodal Data Fusion according to claim 1, characterized in that: The wearing strap (1) has a two-section structure, which is fixedly connected by Velcro (11).

3. The Eight-Section Brocade Movement Recognition Assistive Device based on Multimodal Data Fusion according to claim 1, characterized in that: The integrated module (2) includes a nine-axis attitude sensor (21), a data processing unit (22) and a Bluetooth unit (23). The nine-axis attitude sensor (21), the data processing unit (22), the Bluetooth unit (23) and the vibration unit (51) are connected in sequence by electrical signals and are respectively connected to the power supply module (4).

4. The Eight-Section Brocade Movement Recognition Assistive Device based on Multimodal Data Fusion according to claim 1, characterized in that: The image analysis module (3) includes a camera (31) and a processor (32). The camera (31) and the processor (32) are fixed together by a housing. The terminal graphical interface (52) is embedded in the housing. The camera (31), the processor (32) and the terminal graphical interface (52) are connected in sequence by electrical signals. The Bluetooth unit (23) is connected to the processor (32) by Bluetooth signals.

5. The Eight-Section Brocade Movement Recognition Assistive Device based on Multimodal Data Fusion according to claim 1, characterized in that: The mounting cavity (12) has a shock-absorbing buffer layer (14) installed on the surface facing the Velcro (11). The mounting cavity (12) has a gap on its side, and a breathable mesh layer (13) is installed in the gap.

6. The Eight-Section Brocade Movement Recognition Assistive Device based on Multimodal Data Fusion according to claim 1, characterized in that: The power supply module (4) is a flexible lithium polymer battery.

7. The Eight-Section Brocade Movement Recognition Assistive Device based on Multimodal Data Fusion according to claim 1, characterized in that: The inner shock-absorbing layer (14) of the wear strap (1) is made of silicone material with a hardness of 30A.

Images