A gait machine interaction system and method based on gesture recognition control

Abstract

The application discloses a kind of based on posture recognition control's walk machine interaction system and method.It relates to fitness equipment technical field, system includes posture recognition APP's smart phone and walk machine installed with.APP is in real time by camera Whole body image acquisition, identify user posture action, according to preset rule generates power on, power off, acceleration, deceleration and mode switching etc. Instruction and send to walk machine control unit.When recognizing that user leaves tread or falls down, automatically power off, and start motor brake fast stop.It can also be switched to game control mode, map leg lifting action as game character's advance / stop, map left and right hand lifting action as left turn / right turn, map single arm or double arm waving action as attack operation, realize somatosensory interaction.The application does not need to hold remote controller, walk machine can be controlled by natural posture, and has safety protection and entertainment function.

Description

Technical Field

[0001] This invention relates to the field of fitness equipment technology, and more specifically, to an interactive system and control method for controlling a walking machine based on human posture. Background Technology

[0002] Traditional walking machines (or treadmills) typically rely on physical buttons, touchscreens, or separate remote controls for operation. Users must directly touch the control interface to adjust the speed or stop the machine, which has several drawbacks: First, reaching in during exercise can easily cause a shift in the body's center of gravity, increasing the risk of loss of balance and falls; second, the remote control needs to be held or carried, making it easy to lose or forget; third, traditional walking machines lack the ability to actively sense the user's status, so the machine often continues to run even if the user falls or accidentally leaves, potentially causing secondary injuries; fourth, the simple act of walking can be monotonous, and the user experience needs improvement. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a walking machine interactive system and method based on posture recognition control. The system captures the user's full-body posture in real time through the smartphone camera, wirelessly controls the operation of the walking machine after recognizing specific posture movements, and integrates motion-sensing game functions to enhance the fun and safety of exercise.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A walking machine interaction system based on posture recognition control includes:

[0006] A smartphone with a gesture recognition application installed.

[0007] The treadmill is equipped with a drive motor and control unit.

[0008] The smartphone and the treadmill establish a two-way wireless communication connection via Bluetooth or other wireless communication chips.

[0009] The smartphone is placed 1 to 3 meters in front of the treadmill, and the smartphone's camera field of view can completely cover the entire body of the user standing on the treadmill;

[0010] The posture recognition application is configured to: capture the user's full-body image in real time through the mobile phone camera, recognize the user's human posture and movements, generate control commands according to preset motion mapping rules, and send them to the control unit of the walking machine through the wireless communication connection;

[0011] The control commands include at least: power-on command, power-off command, acceleration command, and deceleration command.

[0012] Furthermore, the control commands also include mode switching commands.

[0013] Furthermore, the preset motion mapping rules in the posture recognition application include: generating a power-on command when a user is detected standing on the treadmill and raising one arm; generating a stop command when both arms are raised; generating an acceleration command when a user is detected raising their right hand; generating a deceleration command when a user is detected raising their left hand; and automatically generating a power-off command when a user is detected leaving the treadmill's tread area or falling due to abnormal body posture.

[0014] Furthermore, the smartphone simultaneously performs three functions: image acquisition, posture recognition algorithm operation, and wireless command transmission; after receiving the power-off command, the control unit of the walking machine not only stops the motor but also activates the electromagnetic brake to quickly stop the walking belt.

[0015] Furthermore, the posture recognition application also includes a virtual game module; when the posture recognition application switches the system to game control mode based on preset specific limb movements, the user's lower or upper limb posture movements are mapped to character operation commands in the virtual game; wherein:

[0016] When the system detects that the user is raising their left or right leg, it maps to a forward command for the game character; when the system detects that the user has stopped raising their legs (i.e., both legs are standing still), it maps to a stop command for the game character.

[0017] When the system detects that the user has raised their right hand, it maps this action to a right turn command for the game character.

[0018] When the system detects that the user is raising their left hand, it maps this action to a leftward turn command for the game character.

[0019] When the system detects that the user is making a gesture of waving both arms, waving one arm up and down, or waving one arm left and right, it maps it to an attack command from the game character.

[0020] Meanwhile, the user leaving or falling recognition function maintains the highest priority in game control mode and can still generate shutdown commands.

[0021] Furthermore, when generating acceleration commands, the posture recognition application continuously generates multiple acceleration commands based on the duration of the user holding the right hand up, so as to achieve a step-by-step speed increase; when generating deceleration commands, it continuously generates multiple deceleration commands based on the duration of the user holding the left hand up, so as to achieve a step-by-step speed decrease.

[0022] Furthermore, the treadmill includes a control unit and a Bluetooth receiver module. The control unit is configured to receive and execute power-on, power-off, acceleration, deceleration, and mode switching commands sent from a smartphone, and simultaneously cut off the motor power and activate the electromagnetic brake when a power-off command is received.

[0023] A treadmill interaction method based on posture recognition control, applied to any of the above-mentioned systems, includes the following steps:

[0024] Deployment steps: Place a smartphone with the posture recognition app installed 1-3 meters in front of the treadmill so that the phone's camera can capture the user's whole body and establish a wireless communication connection between the phone and the treadmill.

[0025] Data Acquisition and Recognition Steps: The mobile phone camera acquires images in real time, and the posture recognition APP extracts the coordinates of key points of the human skeleton from the images and recognizes the user's posture and movements.

[0026] Command generation and transmission steps: Based on the recognized posture and movement, corresponding control commands are generated according to preset mapping rules and sent to the treadmill through a wireless communication link;

[0027] Execution steps: The control unit of the treadmill receives and executes the control command.

[0028] Furthermore, in the acquisition and recognition steps, the difference in vertical coordinates between the wrist key point and the shoulder key point is used to determine whether the user is raising their left or right hand; the rate of change of the vertical position of the hip key point and the coordinate change of the head key point are used to determine whether the user has left or fallen.

[0029] Furthermore, the method also includes a mode switching step: when the user is detected to have crossed their hands in front of their chest and held for 1 second, switching between device control mode and game control mode; in game control mode:

[0030] The user's leg-lifting motion is mapped to the game character's forward and stop commands;

[0031] The user's right hand raising gesture is mapped to a right turn command;

[0032] The user's left hand raising gesture is mapped to a left turn command;

[0033] The user's single-arm or double-arm waving motions are mapped as attack commands.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] This invention eliminates the need for a handheld remote or touch panel; all operations of the treadmill can be completed through natural postures such as raising hands or arms, avoiding distractions and imbalances caused by searching for remotes or buttons during exercise. Utilizing the user's existing smartphone as the sensing and control terminal eliminates the need for additional dedicated sensors, cameras, or remotes, reducing overall system costs. The system can detect the user's posture when leaving the treadmill or falling in real time and automatically send a shutdown command within a very short time. Simultaneously, the treadmill control unit activates the electromagnetic brake to quickly stop the walking belt, effectively preventing secondary injuries. Through a motion-sensing game module, the user's walking movements are mapped to the operation commands of a game character, transforming monotonous aerobic exercise into an immersive entertainment experience and increasing the user's willingness to persist in exercising. Continuous motion detection enables step-by-step speed adjustment, allowing users to precisely control the speed change based on the duration of the held movement, providing a continuous and natural operating feel. Attached Figure Description

[0036] Figure 1 This is a schematic diagram showing the overall composition and deployment location of the system of the present invention;

[0037] Figure 2 This is a block diagram of the action mapping logic of the posture recognition APP of the present invention;

[0038] Figure 3 This is an overall flowchart of the interaction method of the present invention;

[0039] Figure 4 This is a schematic diagram of action mapping in the game control mode of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1: System Composition and Deployment Location

[0042] like Figure 1 As shown, this embodiment provides a walking machine interaction system based on posture recognition control.

[0043] The system consists of a smartphone and a treadmill. The smartphone is a commercially available, standard smartphone, integrating a camera, processor, and Bluetooth communication module. A posture recognition application (hereinafter referred to as "APP") is installed on the smartphone. The treadmill is an electric treadmill equipped with a drive motor, integrating a Bluetooth receiver module, a control unit, a drive motor, and an electromagnetic brake. The control unit is connected to the drive motor and is used to control the motor's start, stop, and speed. The Bluetooth receiver module is connected to the control unit and is used to receive external wireless commands. The electromagnetic brake is mechanically connected to the treadmill's drive wheel; the control unit can activate the electromagnetic brake upon receiving a shutdown command.

[0044] In terms of connectivity, a two-way wireless communication connection is established between the smartphone's Bluetooth communication module and the treadmill's Bluetooth receiver module. This connection can use the Bluetooth protocol or other wireless communication chip protocols, such as a 2.4GHz proprietary protocol or Wi-Fi Direct.

[0045] Regarding deployment location relationships, such as Figure 1 As shown, the smartphone is fixed in front of the treadmill using a desktop stand or tripod, at a distance of 1 to 3 meters from the front edge of the treadmill. At the same time, the angle of the smartphone camera is adjusted so that it is horizontally facing the treadmill, ensuring that when the user is standing on the treadmill, the smartphone camera's field of view can completely cover the user's entire body, from the top of the head to the ankles, in the frame.

[0046] In this system, the smartphone performs three functions simultaneously: First, image acquisition—acquiring a video stream containing the user in real time via the camera; second, posture recognition algorithm execution—the processor runs the posture recognition algorithm built into the app to extract key points of the human skeleton from each frame of the image; and third, wireless command transmission—generating control commands based on the recognition results and sending them to the treadmill via the Bluetooth communication module.

[0047] In addition to the conventional motor speed control function, the control unit of the walking machine also has an emergency braking function. When the control unit receives a shutdown command via the Bluetooth receiver module, it performs two operations simultaneously: first, it cuts off the power supply to the drive motor; second, it activates the electromagnetic brake to quickly stop the walking belt.

[0048] Example 2: Action Mapping Logic of a Pose Recognition App

[0049] like Figure 2 As shown in the figure, this embodiment describes in detail the action mapping logic inside the posture recognition APP.

[0050] The app is configured to capture real-time images of the user's full body using the phone's camera, recognize the user's posture and movements, generate control commands based on preset motion mapping rules, and send them to the treadmill's control unit via wireless communication. The control commands include at least: power-on command, power-off command, acceleration command, and deceleration command.

[0051] Specifically, such as Figure 2 As shown, the app first captures a real-time image of the user's entire body using the phone's camera, then performs human posture recognition and key point extraction. After extraction, the app categorizes and judges the user's posture movements and generates corresponding control commands based on the following preset motion mapping rules:

[0052] First, when the system detects that a user is standing on the treadmill and raises one arm, it generates a power-on command; when the user raises both arms, it generates a power-off command.

[0053] Second, when the system detects that the user is raising their right hand upwards, it generates an acceleration command.

[0054] Third, when the system detects that the user is raising their left hand upwards, it generates a deceleration command.

[0055] Fourth, when the system detects that the user has crossed their hands in front of their chest and held them for 1 second, it generates a mode switching command, meaning that the control command also includes a mode switching command.

[0056] Fifth, when the system detects that a user has left the treadmill's tread area or that the user has fallen due to abnormal body posture, it automatically generates a shutdown command.

[0057] After the above control commands are generated, the APP sends them to the treadmill control unit via Bluetooth, and the treadmill performs the corresponding operations.

[0058] Example 3: Overall Flow of the Interaction Method

[0059] like Figure 3 As shown, this embodiment provides a walking machine interaction method based on posture recognition control. This method is applied to the above system and includes deployment steps, data acquisition and recognition steps, instruction generation and transmission steps, and execution steps.

[0060] Deployment steps: such as Figure 3 As shown, first, place a smartphone with a posture recognition app installed 1 to 3 meters in front of the treadmill; then, adjust the angle of the phone's camera so that it can capture the user's entire body; finally, establish a Bluetooth wireless communication connection between the phone and the treadmill.

[0061] Data Acquisition and Recognition Steps: After deployment, the phone's camera acquires images in real time. The posture recognition app extracts the coordinates of key points on the human skeleton from the images and identifies the user's posture and movements. In this step, the difference in vertical coordinates between the wrist and shoulder key points is used to determine whether the user is raising their left or right hand; the rate of change of the vertical position of the hip key point and the coordinate changes of the head key point are used to determine whether the user has left or fallen.

[0062] Command generation and transmission steps: Based on the identified posture and movement, determine whether a certain posture or movement matches according to preset mapping rules. If a match is successful, the corresponding control command is generated and sent to the treadmill via wireless communication link; if no preset movement is matched, return to continue acquiring images.

[0063] Execution steps: After receiving the command, the control unit of the treadmill performs corresponding operations such as powering on, powering off, accelerating, decelerating, and switching modes.

[0064] In addition, such as Figure 3 As shown, the method also includes a mode switching step: when the user is detected to cross their hands in front of their chest and hold them for 1 second, the method switches between device control mode and game control mode.

[0065] Example 4: Action Mapping in Game Control Mode

[0066] like Figure 4 As shown, this embodiment describes a specific implementation method in the game control mode.

[0067] The posture recognition app also includes a virtual game module. When the posture recognition app switches the system to game control mode based on a preset specific action (i.e., crossing hands in front of the chest and holding for 1 second), the user's leg or upper limb posture movements are mapped to character operation commands in the virtual game. Simultaneously, the user leaving or falling recognition function maintains the highest priority in game control mode and can still generate a shutdown command.

[0068] Specifically, such as Figure 4 As shown, in game control mode:

[0069] When the app detects that the user raises their left or right leg (i.e., the vertical coordinate of the left knee joint key point is higher than the left hip key point, or the vertical coordinate of the right knee joint key point is higher than the right hip key point), it maps the posture to the "forward" command in the virtual game, and the game character continues to move forward.

[0070] When the app detects that the user has stopped lifting their legs (i.e. both legs are in a standing position without being lifted, and the key points of the left and right knee joints are not higher than the corresponding key points of the hip), it is mapped to the "stop" command in the virtual game, and the game character stops moving forward.

[0071] When the app detects that the user raises their right hand (i.e., the Y-coordinate of the right wrist key point is less than the Y-coordinate of the right shoulder key point), it maps this posture to the "turn right" command in the virtual game, and the game character changes direction to the right.

[0072] When the app detects that the user raises their left hand (i.e., the Y-coordinate of the left wrist key point is less than the Y-coordinate of the left shoulder key point), it maps this posture to the "turn left" command in the virtual game, and the game character changes direction to the left.

[0073] When the app detects that the user makes a gesture of waving both arms, waving one arm up and down, or waving one arm left and right (i.e., the wrist key point moves back and forth relative to the elbow or shoulder), it maps the gesture to an "attack / shoot" command in the virtual game, and the game character performs the attack action.

[0074] like Figure 4 As shown, in game control mode, the treadmill's operating speed is not always constant, but is limited to a preset maximum speed threshold. Specifically, users can adjust the treadmill speed using acceleration or deceleration commands, but the adjustment range is limited to between 0 and the maximum speed limit, which is a preset safety value (e.g., 3.5 km / h). When a user attempts to accelerate beyond this speed limit, the treadmill control unit will ignore the excess acceleration command, maintaining the speed at the maximum limit. The combination of continuous walking and intermittent posture operations creates an experience of playing motion-sensing games while exercising.

[0075] Meanwhile, in game control mode, although the speed adjustment range is limited by height, the two safety functions—user leaving detection and fall detection—always maintain the highest priority. The safety monitoring thread in the APP runs independently of the game logic. Once the APP detects that a user has left the treadmill or fallen, it will immediately interrupt the game and send a shutdown command, regardless of the current game state. The treadmill control unit will then execute the electromagnetic braking shutdown procedure.

[0076] Example 5: Specific Implementation of Stepped Speed ​​Regulation

[0077] This embodiment further illustrates the step-speed regulation mechanism.

[0078] The posture recognition application generates multiple acceleration commands continuously based on the duration the user holds their raised right hand, thus achieving a gradual increase in speed. Specifically, the app sets a timer variable to record the start time of the raised right hand action. When the raised right hand action is detected for the first time, the current system timestamp is recorded, and the first acceleration command (e.g., +0.1 km / h) is immediately sent. Subsequently, as long as the raised right hand action continues, the app sends an acceleration command every 0.3 seconds. When the user lowers their right hand, the timer is reset to zero. By holding the raised right hand for 0.6 seconds, the user can obtain an acceleration of +0.2 km / h; by holding it for 0.9 seconds, they can obtain an acceleration of +0.3 km / h, and so on.

[0079] Similarly, when generating deceleration commands, the posture recognition application continuously generates multiple deceleration commands based on the duration of the user's left-hand raising action to achieve a step-by-step speed reduction. The deceleration process adopts a completely symmetrical implementation method, using another timer variable to record the start time of the left-hand raising action, and sending a deceleration command (-0.1 km / h) every 0.3 seconds until the speed drops to zero.

[0080] Example 6: Specific Algorithm for Key Point Coordinate Calculation

[0081] This embodiment illustrates the specific implementation method of extracting key point coordinates in the acquisition and recognition steps.

[0082] The app integrates a lightweight human pose estimation model. This model is optimized for mobile devices and can perform real-time inference on ordinary smartphones.

[0083] When determining whether to raise the left or right hand, the app calculates the vertical coordinate difference between key points on the wrist and shoulder. Taking raising the right hand as an example, the app compares the Y-coordinate of the right wrist with the Y-coordinate of the right shoulder. Since the positive Y-axis points downwards in the image coordinate system, "the Y-coordinate of the right wrist being less than the Y-coordinate of the right shoulder" indicates that the wrist is higher than the shoulder. Simultaneously, the app compares the Y-coordinate relationships between the right wrist and right elbow, and between the right elbow and right shoulder, to ensure that the arm is raised upwards rather than extended forward. The logic for raising the left hand is symmetrical.

[0084] To determine whether a user has left or fallen, the system calculates the rate of change in the vertical position of the hip keypoint and the coordinate change of the head keypoint. Fall detection utilizes the vertical position change of the hip keypoint (the midpoint between the left and right hips): the app maintains a 5-frame historical queue of hip Y-coordinates. It calculates the difference between the current frame's hip Y-coordinate and the oldest frame's Y-coordinate in the queue. When this difference exceeds 30% of the screen height, it simultaneously checks if the head keypoint's Y-coordinate exceeds 80% of the screen height (i.e., the head is close to the ground). If both conditions are met, a fall-related shutdown is triggered.

[0085] The departure detection is achieved by monitoring whether the vertical coordinates of key points on the hip are continuously outside the bottom of the screen: if the hip Y coordinate is greater than the screen height minus 20 pixels for 2 consecutive seconds (i.e., close to or beyond the bottom boundary of the screen), it is determined that the user has left the camera's field of view, that is, left the treadmill tread.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A walking machine interactive system based on posture recognition control, characterized in that, include: A smartphone with a gesture recognition application installed. The treadmill is equipped with a drive motor and control unit. The smartphone and the treadmill establish a two-way wireless communication connection via Bluetooth or other wireless communication chips. The smartphone is placed 1 to 3 meters in front of the treadmill, and the smartphone's camera field of view can completely cover the entire body of the user standing on the treadmill; The posture recognition application is configured to: capture the user's full-body image in real time through the mobile phone camera, recognize the user's human posture and movements, generate control commands according to preset motion mapping rules, and send them to the control unit of the walking machine through the wireless communication connection; The control commands include at least: power-on command, power-off command, acceleration command, and deceleration command.

2. The treadmill interactive system based on posture recognition control according to claim 1, characterized in that, The control commands also include mode switching commands.

3. The treadmill interactive system based on posture recognition control according to claim 1, characterized in that, The preset action mapping rules in the pose recognition application include: When the machine detects that a user is standing on the treadmill and raises one arm, it generates a power-on command; when the user raises both arms, it generates a power-off command. When the system detects that the user is raising their right hand, it generates an acceleration command. When the system detects that the user is raising their left hand upwards, it generates a deceleration command. When the machine detects that a user has left the treadmill's treadmill area or that the user has fallen due to abnormal posture, it automatically generates a shutdown command.

4. The treadmill interactive system based on posture recognition control according to claim 1, characterized in that, The smartphone simultaneously performs three functions: image acquisition, posture recognition algorithm operation, and wireless command transmission; after receiving the power-off command, the control unit of the walking machine not only stops the motor but also activates the electromagnetic brake to quickly stop the walking belt.

5. The treadmill interactive system based on posture recognition control according to claim 1, characterized in that, The posture recognition application also includes a virtual game module; when the posture recognition application switches the system to game control mode based on preset specific limb movements, the user's lower or upper limb posture movements are mapped to character operation commands in the virtual game; wherein: When the system detects that the user is raising their left or right leg, it maps to a forward command for the game character; when the system detects that the user has stopped raising their legs (i.e., both legs are standing still), it maps to a stop command for the game character. When the system detects that the user has raised their right hand, it maps this action to a right turn command for the game character. When the system detects that the user is raising their left hand, it maps this action to a leftward turn command for the game character. When the system detects that the user is making a gesture of waving both arms, waving one arm up and down, or waving one arm left and right, it maps it to an attack command from the game character. Meanwhile, the user leaving or falling recognition function maintains the highest priority in game control mode and can still generate shutdown commands.

6. The treadmill interactive system based on posture recognition control according to claim 1, characterized in that, When generating acceleration commands, the posture recognition application continuously generates multiple acceleration commands based on the duration of the user's right-hand raising motion to achieve a step-by-step speed increase; when generating deceleration commands, it continuously generates multiple deceleration commands based on the duration of the user's left-hand raising motion to achieve a step-by-step speed decrease.

7. The treadmill interactive system based on posture recognition control according to any one of claims 1 to 6, characterized in that, The device includes a control unit and a Bluetooth receiving module. The control unit is configured to receive and execute power-on, power-off, acceleration, deceleration, and mode switching commands sent from a smartphone, and to simultaneously cut off the motor power and activate the electromagnetic brake when a power-off command is received.

8. An interaction method for a walking machine interaction system based on posture recognition control as described in any one of claims 1 to 6, characterized in that, Includes the following steps: Deployment steps: Place a smartphone with the posture recognition app installed 1-3 meters in front of the treadmill so that the phone's camera can capture the user's whole body and establish a wireless communication connection between the phone and the treadmill. Data Acquisition and Recognition Steps: The mobile phone camera acquires images in real time, and the posture recognition APP extracts the coordinates of key points of the human skeleton from the images and recognizes the user's posture and movements. Command generation and transmission steps: Based on the recognized posture and movement, corresponding control commands are generated according to preset mapping rules and sent to the treadmill through a wireless communication link; Execution steps: The control unit of the treadmill receives and executes the control command.

9. The treadmill interaction method based on posture recognition control according to claim 8, characterized in that, In the acquisition and recognition steps, the difference in vertical coordinates between the wrist key point and the shoulder key point is used to determine whether the user is raising their left or right hand; the speed of change of the vertical position of the hip key point and the coordinate change of the head key point are used to determine whether the user has left or fallen.

10. The treadmill interaction method based on posture recognition control according to claim 8, characterized in that, It also includes a mode switching step: when the system detects that the user has crossed their arms in front of their chest and held the position for 1 second, it switches between device control mode and game control mode; in game control mode: The user's leg-lifting motion is mapped to the game character's forward and stop commands; The user's right hand raising gesture is mapped to a right turn command; The user's left hand raising gesture is mapped to a left turn command; The user's single-arm or double-arm waving motions are mapped as attack commands.

Images