Intelligent wheelchair dynamic gesture recognition control system

Abstract

The invention discloses a dynamic gesture recognition control system for an intelligent wheelchair, and relates to the technical field of intelligent wheelchairs, the system comprises the following components: an environment sensing module which is composed of a laser radar and an ultrasonic sensor and is used for collecting information of a surrounding environment in real time and constructing an environment map; meanwhile, the position and the shape of the obstacle and the distance information between the obstacle and the wheelchair are accurately recognized; by introducing the dynamic gesture recognition technology, not only can the current static gesture action of the user be recognized, but also the following action intention of the user can be predicted based on parameters such as the speed, the angle and the acceleration of the gesture, so that the intelligent wheelchair can make a response in advance, closer and smoother interaction with the gesture operation of the user is achieved, and in addition, the user experience is improved. In combination with the information of the environment sensing module, the intelligent wheelchair can more accurately understand the intention of the user and perform reasonable obstacle avoidance operation in a complex environment, so that the overall use experience and safety are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent wheelchair technology, specifically to an intelligent wheelchair dynamic gesture recognition control system. Background Technology

[0002] In today's society, with the rapid development of technology, intelligent assistive devices are playing an increasingly important role in people's daily lives. Especially against the backdrop of an aging population and the growing demand for mobility assistive devices among people with disabilities, intelligent wheelchairs, as a key assistive mobility tool, have received widespread attention for their technological research and development and application. Intelligent wheelchairs not only need to have basic mobility functions, but also need to achieve breakthroughs in interactive control, environmental perception, and intelligent collaboration to meet the diverse and personalized needs of users.

[0003] However, existing smart wheelchairs still have many shortcomings in terms of interactive control. First, the gesture recognition function of traditional smart wheelchairs is mostly limited to recognizing current static gestures and lacks the ability to predict the user's subsequent intentions. This results in the interaction between the user and the wheelchair being less intelligent and smooth, significantly reducing the user experience. Second, the environmental perception capability is limited. Traditional smart wheelchairs struggle to obtain detailed information about the surrounding environment in real time and accurately. When facing obstacles, they cannot perform reasonable obstacle avoidance operations based on environmental conditions and user intentions, posing certain safety risks. Furthermore, the coordination between the wheelchair's motion control, the user's gesture operations, and environmental changes is insufficient, making it difficult to guarantee safe and efficient mobility in complex environments. These problems limit the widespread application and further development of smart wheelchairs.

[0004] To address the aforementioned issues, it is necessary to optimize the existing intelligent wheelchair dynamic gesture recognition control system. By utilizing environmental perception, gesture trajectory prediction, and intelligent collaborative control functions, the system can accurately predict the user's intentions and adjust the wheelchair's movement path in real time. Therefore, developing an intelligent wheelchair dynamic gesture recognition control system that can comprehensively achieve the above features is of great significance. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an intelligent wheelchair dynamic gesture recognition control system. This system integrates devices such as LiDAR and ultrasonic sensors to achieve real-time perception and accurate reconstruction of the surrounding environment. Simultaneously, it utilizes machine learning algorithms to monitor and predict the user's gesture trajectory in real time, thereby enabling the system to anticipate the user's intentions. Based on this, the system combines environmental perception information with the predicted user gesture trajectory through an intelligent collaborative processing module to generate corresponding processing instructions, achieving real-time adjustment of the wheelchair's movement path. Furthermore, the system also possesses a motion intention re-prediction and adjustment function, enabling timely adjustments to the wheelchair's movement strategy based on changes in the user's gesture trajectory during obstacle avoidance, ensuring safe and efficient movement in complex environments.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an intelligent wheelchair dynamic gesture recognition control system, which includes the following components:

[0007] Environmental perception module: Composed of lidar and ultrasonic sensors, it collects information about the surrounding environment in real time and builds an environmental map, while accurately identifying the location, shape and distance of obstacles to the wheelchair.

[0008] Gesture trajectory monitoring and analysis module: Monitors the user's gesture trajectory in real time, and predicts the user's action intention based on the speed, angle and acceleration parameters of the gesture, combined with historical gesture trajectory data, through a gesture action intention analysis model;

[0009] Intelligent collaborative processing module: Based on the specific attributes of obstacles detected by the environmental perception module and the user's gesture trajectory predicted by the gesture trajectory monitoring and analysis module, intelligent collaborative processing is performed to generate corresponding processing instructions through pre-set rules and algorithms;

[0010] Wheelchair motion control module: Receives obstacle avoidance gestures from the user that conform to the processing instructions, and adjusts the wheelchair's movement path in real time by combining data provided by the environmental perception module. At the same time, during obstacle avoidance, it continuously monitors the user's gesture trajectory and dynamically adjusts the wheelchair's speed and direction according to the user's real-time operation intentions and the dynamic changes in the environment through control algorithms.

[0011] Action Intent Re-prediction and Adjustment Module: During obstacle avoidance, if a change in the user's gesture trajectory is detected, the new gesture trajectory data is analyzed through a re-prediction mechanism to re-predict the user's action intent, and the new action intent is converted into a command and sent to the wheelchair motion control module to adjust the wheelchair motion strategy.

[0012] Furthermore, the gesture trajectory monitoring and analysis module combines historical gesture trajectory data and predicts the user's gesture intent through a gesture intention analysis model, the formula of which is: in, Given the sequence of hand gesture velocity vectors from the start time to time t. and acceleration vector sequence In this case, the intention of the gesture is I j The probability, I j β is the j-th type of gesture intention, including but not limited to forward, backward, and turning, where j is the total number of gesture intention categories. jk These are parameters related to the j-th gesture intention and the k-th feature function, where s is the number of feature functions used to describe the gesture velocity and acceleration sequence. It is the k-th sequence of gesture velocity vectors and acceleration vector sequence The feature function values ​​extracted include, but are not limited to, the rate of change of velocity and the peak acceleration.

[0013] Furthermore, the intelligent collaborative processing module integrates obstacle information acquired by the environmental perception module and user gesture intentions predicted by the gesture trajectory monitoring and analysis module. It uses a data association algorithm to determine the relationship between obstacle information and user gestures. Based on different obstacle situations and user gesture intentions, it formulates corresponding collaborative processing rules. According to the collaborative processing rules and data fusion results, it formulates collaborative processing decisions through a comprehensive decision formula and generates corresponding control commands. The control commands include wheelchair speed control commands, direction control commands, and prompt information commands, and sends these commands to the wheelchair motion control module and corresponding prompting devices.

[0014] Furthermore, the intelligent collaborative processing module uses a data association algorithm to determine the relationship between obstacle information and user gestures. The algorithm formula is as follows: Among them, C ij C represents the correlation between the i-th obstacle information and the j-th user gesture action. ij The higher the value, the stronger the correlation between the two factors. α and β are the weighting coefficients for distance and time factors, respectively, and are adjusted according to the actual situation to balance the influence of distance and time on the correlation. Represents the position vector of the i-th obstacle. The position vector when the j-th user's gesture action occurs The Euclidean distance between them, t i and t j These are the times when obstacle information is acquired and when the user's gesture occurs, respectively. i -t j| represents the time difference between the two, n is the number of obstacles currently detected, and m is the number of user gestures currently recognized.

[0015] Furthermore, the intelligent collaborative processing module formulates collaborative processing decisions using a comprehensive decision formula, which is as follows: Among them, D action The final collaborative processing decision is used to guide the wheelchair's subsequent actions. K is the number of preset collaborative processing rules, including but not limited to processing rules under different obstacle situations, gesture intentions, and combinations of environmental conditions. ω k This is the weight coefficient of the Kth collaborative processing rule, determined based on the rule's importance and applicable scenario factors. It is the Kth collaborative processing rule function, using the obstacle information vector. User gesture intention I gesture and environmental status information E env As input, output the corresponding decision value under this rule.

[0016] Furthermore, the wheelchair motion control module dynamically adjusts the wheelchair's speed and direction through a control algorithm, with the speed adjustment formula being: v control =v target ·(1-δ·S judge ), where v control This is the actual controlled speed of the wheelchair after safety adjustments, v target The target speed is set based on the user's gesture intent and ideal conditions. δ is an adjustment coefficient for the effect of safe distance on speed, used to control the degree of speed reduction when an unsafe distance is detected. S judge This is the result of the safety distance judgment; the direction adjustment formula is: Δθ control =θ target +∈·(θ obstacle -θ wheel ), where Δθ control It is the amount of change in the wheelchair's directional control angle after adjustment, θ target It is the target direction angle set according to the user's gesture intention, ∈ is the influence coefficient of the obstacle direction on the wheelchair's direction adjustment, used to control the degree of influence of the obstacle direction on the wheelchair's steering during obstacle avoidance, θ obstacle θ is the angle of the obstacle relative to the wheelchair. wheel It is the current angle of the wheelchair's direction of travel.

[0017] Furthermore, the motion intent re-prediction and adjustment module monitors changes in the gesture trajectory in real time. When the change in the position, speed, or acceleration parameters of the gesture trajectory exceeds a preset threshold, it is determined that the gesture trajectory has changed. When a change in the gesture trajectory is detected, a re-prediction method is immediately initiated. This method uses the same model capable of processing time-series data as the gesture motion intent prediction method, inputs new historical gesture trajectory data and current gesture trajectory data, and quickly re-predicts the user's motion intent through data learning and analysis. Based on the re-predicted user motion intent, the motion intent re-prediction and adjustment module generates corresponding adjustment instructions and sends them to the wheelchair motion control module. The wheelchair motion control module re-plans the wheelchair's motion path and control parameters according to the adjustment instructions, thereby achieving timely adjustment of the wheelchair motion strategy.

[0018] Furthermore, based on the re-predicted user action intent, the action intent re-prediction and adjustment module generates corresponding adjustment instructions, the formula for calculating the adjustment amount being: Where, ΔS adjust This is the adjustment amount for the motion strategy, used to adjust the wheelchair's movement. ζ is the adjustment coefficient, set according to the wheelchair's response characteristics and safety requirements. It is the position vector of the wheelchair motion trajectory expected based on the reassessed gesture intention. It is the current position vector of the wheelchair's motion trajectory, Ω(Ψ(I predicted )) is the policy adjustment function, which adjusts the predicted gesture intent probability Ψ(I) based on the re-evaluation. predicted The adjustment amount is corrected.

[0019] Furthermore, the Ψ(I) predocted ) is the re-evaluated probability of the predicted gesture intent, calculated using the following formula: Specifically, before the change in gesture trajectory, based on the gesture velocity vector sequence and acceleration vector sequence The intention of the gesture is I j The probability P old (I j ),Right now When a change in gesture trajectory is detected, a new gesture position change vector appears. Then, the gesture intention analysis model is used again to extract new gesture velocity and acceleration parameters, as well as new gesture feature functions h. k The value of I is obtained by inputting the new data into the gesture intent probability prediction model, and the gesture intent under the new gesture change is calculated as I. j probability By using the intention re-evaluation formula, P is balanced using the weighting coefficient γ of the old intention probability. old (Ij )and A larger γ indicates a greater reference to the probability of the old intention, while a smaller γ indicates a greater emphasis on the probability of the new intention. A weighted sum is then used to arrive at the re-evaluated gesture intention as I. j The probability Ψ(I) predicted ).

[0020] Compared with existing technologies, this intelligent wheelchair dynamic gesture recognition control system has the following advantages:

[0021] I. This invention introduces dynamic gesture recognition technology, which can not only recognize the user's current static gestures, but also predict the user's next action intentions based on parameters such as the speed, angle, and acceleration of the gestures. This allows the smart wheelchair to respond in advance, thereby achieving a closer and smoother interaction with the user's gesture operations. In addition, combined with information from the environmental perception module, the smart wheelchair can more accurately understand the user's intentions and make reasonable obstacle avoidance operations in complex environments, improving the overall user experience and safety.

[0022] Second, this invention utilizes the high-precision obstacle detection capability of the environmental perception module, combined with the intelligent collaborative processing module and real-time monitoring and analysis of user gesture trajectories, to quickly react when an obstacle is detected and generate corresponding obstacle avoidance commands. This real-time environmental perception and intelligent collaborative processing mechanism enables the intelligent wheelchair to dynamically adjust its movement strategy according to different environmental conditions and user operating intentions, effectively avoiding safety hazards caused by insufficient environmental perception or misunderstanding of user intentions.

[0023] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0025] Figure 1 A schematic diagram of a dynamic gesture recognition control system for an intelligent wheelchair.

[0026] Figure 2 This is a flowchart of an intelligent wheelchair dynamic gesture recognition control system. Detailed Implementation

[0027] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0028] Example 1

[0029] In a general hospital, a patient with a leg injury needs to move between wards, examination rooms and corridors using a smart wheelchair. The hospital environment is complex, with obstacles such as beds, medical equipment, medical staff and other patients, and the space is relatively narrow, which requires a high degree of precision and safety in the wheelchair operation.

[0030] The multi-line LiDAR on the smart wheelchair continuously scans the surrounding environment at a frequency of 15Hz, emitting a large number of laser beams per second. By receiving reflected signals and calculating time differences, it obtains three-dimensional spatial information of surrounding objects, constructing a high-precision environmental map. Meanwhile, eight ultrasonic sensors evenly distributed around the wheelchair emit ultrasonic pulses every 0.1 seconds to detect nearby obstacles in real time. When the patient is about to go from the ward to the examination room, as they pass a corner in the corridor, the LiDAR scan data shows that there is a set of moving medical carts ahead. The system uses point cloud processing technology to accurately identify the position, size, and direction of movement of the medical carts. At the same time, the ultrasonic sensors also sense the approaching carts, further confirming the existence of obstacles. The system fuses and processes this information to provide a more reliable basis for subsequent decision-making.

[0031] The patient sits in a wheelchair, and a depth camera in front of them captures their hand gestures in real time at 30 frames per second. The depth camera not only acquires color images of the gestures but also accurately measures their position in three-dimensional space. When the patient wants to move forward, they make a forward pushing gesture. The system monitors the speed, direction, and acceleration of the gesture in real time, and combines this with previously recorded historical gesture data, utilizing machine learning algorithms... The system analyzes and determines that the patient's current gesture indicates an intention to move forward. To improve the accuracy of the prediction, the system also considers factors such as the duration and amplitude of the gesture.

[0032] Combining the information from the medical cart detected by the environmental perception module and the patient's forward movement intention predicted by the gesture trajectory monitoring and analysis module, the system quickly conducts a risk assessment. It determines that if the patient continues to move forward, there is a high probability of colliding with the medical cart. Therefore, the system immediately issues a warning message through voice prompts and the display screen on the wheelchair, informing the patient that there is an obstacle in front of them, and suggesting that the patient make a lateral sliding gesture to guide the wheelchair to go around it. At the same time, the system also calculates an appropriate obstacle avoidance time window based on the distance and speed of the obstacle, providing the patient with more reasonable operation suggestions.

[0033] When the patient makes a sideways sliding gesture to avoid an obstacle, the system uses environmental perception data provided by the environmental perception module, including the location, shape, and distance of obstacles, as well as the surrounding spatial layout, to adjust the wheelchair's movement path in real time using a path planning algorithm. During obstacle avoidance, the system continuously monitors changes in the patient's gestures and the dynamics of the surrounding environment. Through sensors such as encoders and gyroscopes mounted on the wheelchair, it acquires the wheelchair's position, speed, and direction information in real time. Based on this information, the system uses advanced control algorithms to dynamically adjust the wheelchair's speed and direction. The speed adjustment formula is: v control =v target ·(1-δ·S judge ), where v control This is the actual controlled speed of the wheelchair after safety adjustments, v target The target speed is set based on the user's gesture intent and ideal conditions. δ is an adjustment coefficient for the effect of safe distance on speed, used to control the degree of speed reduction when an unsafe distance is detected. S judge This is the result of the safety distance judgment; the direction adjustment formula is: Δθ control =θ target +∈·(θ obstacle -θ wheel ), where Δθ control It is the amount of change in the wheelchair's directional control angle after adjustment, θ target It is the target direction angle set according to the user's gesture intention, ∈ is the influence coefficient of the obstacle direction on the wheelchair's direction adjustment, used to control the degree of influence of the obstacle direction on the wheelchair's steering during obstacle avoidance, θ obstacle θ is the angle of the obstacle relative to the wheelchair. wheel It refers to the current direction and angle of the wheelchair's movement. For example, when passing through a narrow passageway, the system will automatically reduce the wheelchair's speed to 60% of its original speed to ensure safe passage. At the same time, the system will also precisely adjust the wheelchair's direction of movement based on subtle changes in the patient's hand gestures, enabling the wheelchair to avoid obstacles more flexibly.

[0034] During obstacle avoidance, the patient may find that the space on the other side is more suitable for detour, thus changing their gesture intention. The system monitors changes in the gesture trajectory in real time. When it detects significant changes in parameters such as the speed, direction, or acceleration of the gesture, it immediately activates the re-prediction mechanism. The system quickly analyzes the new gesture data, combines it with current environmental information and previous gesture history data, and re-predicts the patient's action intention. Based on the new intention, the system promptly adjusts the wheelchair's motion strategy, replans the obstacle avoidance path, and sends new control commands to the wheelchair motion control module, enabling the wheelchair to safely avoid obstacles along the new path. During the adjustment process, the system also informs the patient of the current operating status through voice prompts, enhancing the user experience and sense of security.

[0035] Example 2

[0036] In a large shopping mall and its surrounding area, an elderly person with limited mobility wants to use a smart wheelchair for shopping and leisure activities. The mall has facilities such as shelves, display stands, and elevators, while the outside streets have obstacles such as pedestrians, green belts, and parked vehicles, making the environment quite complex and changeable.

[0037] Inside the shopping mall, multi-line LiDAR continuously scans the surrounding environment at a frequency of 12Hz. The emitted laser beams have a wide coverage area and can quickly acquire 3D point cloud data of objects such as shelves and display stands, thereby constructing a detailed and accurate environmental map. At the same time, six ultrasonic sensors evenly distributed around the wheelchair emit an ultrasonic pulse every 0.15 seconds to monitor nearby obstacles in real time. When an elderly person drives their wheelchair to the elevator, the LiDAR scan data shows that there is a row of shelves displaying goods ahead. The system uses point cloud processing and analysis technology to not only accurately identify the location and shape of the shelves, but also to determine the approximate height and density of the goods on the shelves. On the street outside the mall, LiDAR and ultrasonic sensors also work together. LiDAR can detect green belts and parked vehicles at a distance, while ultrasonic sensors accurately detect nearby obstacles such as pedestrians approaching the wheelchair. The system integrates internal and external environmental information to provide comprehensive environmental data support for subsequent decision-making.

[0038] A depth camera mounted in front of the wheelchair tracks the elderly person's hand gestures in real time at 25 frames per second. The depth camera boasts high resolution and excellent low-light performance, clearly capturing subtle changes in the elderly person's gestures while accurately measuring their position and posture in three-dimensional space. When an elderly person wants to go to a store, they make a gesture pointing in that direction. The system monitors parameters such as the gesture's speed, angle, acceleration, and shape changes in real time. Through analysis and processing of multiple consecutive frames of gesture images, combined with previously recorded historical gesture data, including the elderly person's gesture habits and movement patterns in different scenarios, a machine learning model is applied. The system analyzes and determines that the elderly person's current gesture indicates an intention to move in a specified direction. To improve the accuracy of the prediction, the system also considers factors such as the fluency of the elderly person's gesture and the duration of pauses.

[0039] Based on the information about the shelf ahead provided by the environmental perception module and the movement intention predicted by the gesture trajectory monitoring and analysis module, the system first conducts a risk assessment, determining the probability of a collision with the shelf if movement continues in the current direction and immediately issuing a warning. Through clear voice prompts and a small display screen on the wheelchair armrest, the system informs the elderly person of the shelf's presence in large font and high contrast, and suggests appropriate obstacle avoidance gestures. Simultaneously, based on the specific characteristics of the obstacle, such as the shelf's length, width, and relative position to the wheelchair, combined with the elderly person's gesture habits and the wheelchair's movement performance, the system formulates multiple possible obstacle avoidance paths and speed control strategies. Finally, a comprehensive decision-making formula is used to... Select the most suitable strategy from these options to provide precise guidance for subsequent wheelchair motion control.

[0040] Based on the decisions made by the intelligent collaborative processing module, the system uses motion control algorithms to adjust the wheelchair's movement path and speed in real time. The speed adjustment formula is: v control =v target ·(1-δ·S judge ), where v control This is the actual controlled speed of the wheelchair after safety adjustments, v target The target speed is set based on the user's gesture intent and ideal conditions. δ is an adjustment coefficient for the effect of safe distance on speed, used to control the degree of speed reduction when an unsafe distance is detected. S judge This is the result of the safety distance judgment; the direction adjustment formula is: Δθ control =θ target +∈·(θ obstacle -θ wheel ), where Δθ control It is the amount of change in the wheelchair's directional control angle after adjustment, θ target It is the target direction angle set according to the user's gesture intention, ∈ is the influence coefficient of the obstacle direction on the wheelchair's direction adjustment, used to control the degree of influence of the obstacle direction on the wheelchair's steering during obstacle avoidance, θ obstacle θ is the angle of the obstacle relative to the wheelchair. wheel It is the current direction and angle of the wheelchair. During obstacle avoidance, the system continuously monitors the elderly person's hand gestures and the dynamics of the surrounding environment. Through sensors such as high-precision encoders, gyroscopes and accelerometers installed on the wheelchair, it obtains the wheelchair's position, speed, direction and attitude information in real time.

[0041] Based on this information, the system dynamically adjusts the wheelchair's speed and direction. For example, when encountering a pedestrian, the system automatically reduces the wheelchair's speed to 40% of its original speed and adjusts its direction appropriately to avoid collisions. At the same time, when passing through narrow passages, the system controls the wheelchair's movement more precisely based on the width and shape of the passage, keeping the wheelchair's movement accuracy within a small range to ensure safe passage. In addition, the system also adjusts the wheelchair's movement in real time based on subtle changes in the elderly person's gestures, such as slight swaying or pauses, making the wheelchair's movement more in line with the elderly person's operating intentions.

[0042] During obstacle avoidance, if an elderly person changes their direction of movement upon seeing other shops of interest, the system monitors the changes in their gesture trajectory in real time. When significant changes are detected in parameters such as the speed, direction, acceleration, and shape of the gesture, a re-prediction mechanism is immediately activated. The system quickly analyzes the new gesture data, combining it with current environmental information, including the location and movement of surrounding obstacles, the layout of the shopping mall, and previously recorded gesture habits of the elderly person. Using reinforcement learning algorithms, the system re-predicts the elderly person's intentions. Based on the new intentions, the system promptly adjusts the wheelchair's movement strategy, replans the obstacle avoidance path, and sends new control commands to the wheelchair motion control module. The formula for calculating the adjustment amount is as follows: Where, ΔS adjust This is the adjustment amount for the motion strategy, used to adjust the wheelchair's movement. ζ is the adjustment coefficient, set according to the wheelchair's response characteristics and safety requirements. It is the position vector of the wheelchair motion trajectory expected based on the reassessed gesture intention. It is the current position vector of the wheelchair's motion trajectory, Ω(Ψ(I predicted )) is the policy adjustment function, which adjusts the predicted gesture intent probability Ψ(I) based on the re-evaluation. predicted The system will adjust the amount of movement and provide feedback on the current operation status and new movement path to the elderly through voice prompts and display screen during the adjustment process. This will enhance the elderly’s understanding and control of wheelchair movement and improve the safety and comfort of use.

[0043] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An intelligent wheelchair dynamic gesture recognition control system, characterized in that, The system includes the following components: Environmental perception module: Composed of lidar and ultrasonic sensors, it collects information about the surrounding environment in real time and builds an environmental map, while accurately identifying the location, shape and distance of obstacles to the wheelchair. Gesture trajectory monitoring and analysis module: Monitors the user's gesture trajectory in real time, and predicts the user's action intention based on the speed, angle and acceleration parameters of the gesture, combined with historical gesture trajectory data, through a gesture action intention analysis model; Intelligent collaborative processing module: Based on the specific attributes of obstacles detected by the environmental perception module and the user's gesture trajectory predicted by the gesture trajectory monitoring and analysis module, intelligent collaborative processing is performed to generate corresponding processing instructions through pre-set rules and algorithms; Wheelchair motion control module: Receives obstacle avoidance gestures from the user that conform to the processing instructions, and adjusts the wheelchair's movement path in real time by combining data provided by the environmental perception module. At the same time, during obstacle avoidance, it continuously monitors the user's gesture trajectory and dynamically adjusts the wheelchair's speed and direction according to the user's real-time operation intentions and the dynamic changes in the environment through control algorithms. Action Intent Re-prediction and Adjustment Module: During obstacle avoidance, if a change in the user's gesture trajectory is detected, the new gesture trajectory data is analyzed through a re-prediction mechanism to re-predict the user's action intent, and the new action intent is converted into a command and sent to the wheelchair motion control module to adjust the wheelchair motion strategy.

2. The intelligent wheelchair dynamic gesture recognition control system according to claim 1, characterized in that, The gesture trajectory monitoring and analysis module combines historical gesture trajectory data and predicts the user's gesture intent through a gesture intent analysis model. The model formula is as follows: in, Given the sequence of hand gesture velocity vectors from the start time to time t. and acceleration vector sequence In this case, the intention of the gesture is I j The probability, I j β is the j-th type of gesture intention, including but not limited to forward, backward, and turning, where j is the total number of gesture intention categories. jk These are parameters related to the j-th gesture intention and the k-th feature function, where s is the number of feature functions used to describe the gesture velocity and acceleration sequence. It is the k-th sequence of gesture velocity vectors and acceleration vector sequence The feature function values ​​extracted include, but are not limited to, the rate of change of velocity and the peak acceleration.

3. The intelligent wheelchair dynamic gesture recognition control system according to claim 1, characterized in that, The intelligent collaborative processing module integrates obstacle information acquired by the environmental perception module and user gesture intentions predicted by the gesture trajectory monitoring and analysis module. It uses a data association algorithm to determine the relationship between obstacle information and user gestures. Based on different obstacle situations and user gesture intentions, it formulates corresponding collaborative processing rules. According to the collaborative processing rules and data fusion results, it formulates collaborative processing decisions through a comprehensive decision formula and generates corresponding control commands. The control commands include wheelchair speed control commands, direction control commands, and prompt information commands, and sends these commands to the wheelchair motion control module and corresponding prompting devices.

4. The intelligent wheelchair dynamic gesture recognition control system according to claim 3, characterized in that, The intelligent collaborative processing module uses a data association algorithm to determine the relationship between obstacle information and user gestures. The algorithm formula is as follows: Among them, C ij C represents the correlation between the i-th obstacle information and the j-th user gesture action. ij The higher the value, the stronger the correlation between the two factors. α and β are the weighting coefficients for distance and time factors, respectively, and are adjusted according to the actual situation to balance the influence of distance and time on the correlation. Represents the position vector of the i-th obstacle. The position vector when the j-th user's gesture action occurs The Euclidean distance between them, t i and t j These are the times when obstacle information is acquired and when the user's gesture occurs, respectively. i -t j | represents the time difference between the two, n is the number of obstacles currently detected, and m is the number of user gestures currently recognized.

5. The intelligent wheelchair dynamic gesture recognition control system according to claim 3, characterized in that, The intelligent collaborative processing module formulates collaborative processing decisions through a comprehensive decision formula, which is as follows: Among them, D action The final collaborative processing decision is used to guide the wheelchair's subsequent actions. K is the number of preset collaborative processing rules, including but not limited to processing rules under different obstacle situations, gesture intentions, and combinations of environmental conditions. ω k This is the weight coefficient of the Kth collaborative processing rule, determined based on the rule's importance and applicable scenario factors. It is the Kth collaborative processing rule function, using the obstacle information vector. User gesture intention I gesture and environmental status information E env As input, output the corresponding decision value under this rule.

6. The intelligent wheelchair dynamic gesture recognition control system according to claim 1, characterized in that, The wheelchair motion control module dynamically adjusts the wheelchair's speed and direction through a control algorithm. The speed adjustment formula is: v control =v target ·(1-δ·S judge ), where v control This is the actual controlled speed of the wheelchair after safety adjustments, v target The target speed is set based on the user's gesture intent and ideal conditions. δ is an adjustment coefficient for the effect of safe distance on speed, used to control the degree of speed reduction when an unsafe distance is detected. S judge This is the result of the safety distance judgment; the direction adjustment formula is: Δθ control =θ target +∈·(θ obstacle -θ wheel ), where Δθ control It is the amount of change in the wheelchair's directional control angle after adjustment, θ target It is the target direction angle set according to the user's gesture intention, ∈ is the influence coefficient of the obstacle direction on the wheelchair's direction adjustment, used to control the degree of influence of the obstacle direction on the wheelchair's steering during obstacle avoidance, θ obstacle θ is the angle of the obstacle relative to the wheelchair. wheel It is the current angle of the wheelchair's direction of travel.

7. The intelligent wheelchair dynamic gesture recognition control system according to claim 1, characterized in that, The motion intent re-prediction and adjustment module monitors changes in the gesture trajectory in real time. When the change in the position, speed, or acceleration parameters of the gesture trajectory exceeds a preset threshold, it is determined that the gesture trajectory has changed. When a change in the gesture trajectory is detected, a re-prediction method is immediately initiated. This method uses the same model capable of processing time-series data as the gesture motion intent prediction method. New historical gesture trajectory data and current gesture trajectory data are input. Through learning and analysis of the data, the user's motion intent is quickly re-predicted. Based on the re-predicted user motion intent, the motion intent re-prediction and adjustment module generates corresponding adjustment instructions and sends them to the wheelchair motion control module. The wheelchair motion control module re-plans the wheelchair's motion path and control parameters according to the adjustment instructions, thereby achieving timely adjustment of the wheelchair motion strategy.

8. The intelligent wheelchair dynamic gesture recognition control system according to claim 1, characterized in that, The action intent re-prediction and adjustment module generates corresponding adjustment instructions based on the re-predicted user action intent. The adjustment amount is calculated using the following formula: Where, ΔS adjust This is the adjustment amount for the motion strategy, used to adjust the wheelchair's movement. ζ is the adjustment coefficient, set according to the wheelchair's response characteristics and safety requirements. It is the position vector of the wheelchair motion trajectory expected based on the reassessed gesture intention. It is the current position vector of the wheelchair's motion trajectory, Ω(Ψ(I predicted )) is the policy adjustment function, which adjusts the predicted gesture intent probability Ψ(I) based on the re-evaluation. predicted The adjustment amount is corrected.

9. The intelligent wheelchair dynamic gesture recognition control system according to claim 8, characterized in that, The Ψ(I predicted ) is the re-evaluated probability of the predicted gesture intent, calculated using the following formula: Specifically, before the change in gesture trajectory, based on the gesture velocity vector sequence and acceleration vector sequence The intention of the gesture is I j The probability P old (I j ),Right now When a change in gesture trajectory is detected, a new gesture position change vector appears. Then, the gesture intention analysis model is used again to extract new gesture velocity and acceleration parameters, as well as new gesture feature functions h. k The value of I is obtained by inputting the new data into the gesture intent probability prediction model, and the gesture intent under the new gesture change is calculated as I. j probability By using the intention re-evaluation formula, P is balanced using the weighting coefficient γ of the old intention probability. old (I j )and A larger γ indicates a greater reference to the probability of the old intention, while a smaller γ indicates a greater emphasis on the probability of the new intention. A weighted sum is then used to arrive at the re-evaluated gesture intention as I. j The probability Ψ(I) predicted ).

Images