Intelligent wheelchair control method based on multi-sensor information fusion

Abstract

The application discloses a kind of intelligent wheelchair control methods based on multi-sensing information fusion, first, collect Hall rocker voltage signal, the distance between wheelchair and obstacle, obstacle image and wheelchair inertia data;Then, data are processed, the position offset of Hall rocker, the included angle between the advancing direction of wheelchair and the center of obstacle, road inclination angle are obtained;Based on fuzzy control, the distance between wheelchair and obstacle, the included angle between the advancing direction of wheelchair and the center of obstacle, road inclination angle are fused, and the comprehensive environment coefficient is obtained;Finally, according to the position offset of wheelchair, the running state of wheelchair is judged, the information coupling of the gear information, position offset, comprehensive environment coefficient of wheelchair is carried out, and the expected speed of left and right wheels of wheelchair under different running states is calculated.The method fully extracts environmental information, and realizes the full coupling of environmental information, wheelchair running state and control signal, ensures the safety and reliability of wheelchair running, and realizes the intelligent control of wheelchair.

Description

Technical Field

[0001] This invention belongs to the field of intelligent wheelchair control technology, specifically an intelligent wheelchair control method based on multi-sensor information fusion. Background Technology

[0002] Wheelchairs, as a common assistive device for mobility and travel for the elderly and people with limb dysfunction, not only improve their quality of life but also reduce the burden on caregivers. Existing electric wheelchairs are mainly operated manually, using joysticks, buttons, and other methods for human-computer interaction. During use, users need to adapt to their surroundings and operate the wheelchairs accordingly, resulting in a relatively low level of automation.

[0003] Smart wheelchairs can detect environmental information through sensors, and the controller, based on the analysis and processing of this information, issues control signals to control the wheelchair, effectively preventing dangers and making its operation more reliable and safe. However, existing smart wheelchairs use relatively simple sensors that cannot comprehensively reflect environmental information, and the coupling between the wheelchair's operating status and environmental information is low. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to provide an intelligent wheelchair control method based on multi-sensor information fusion.

[0005] The present invention solves the aforementioned technical problem by adopting the following technical solution:

[0006] A method for controlling an intelligent wheelchair based on multi-sensor information fusion, characterized by comprising the following steps:

[0007] Step 1: Collect data, including Hall effect sensor voltage signals, distance between the wheelchair and the obstacle, obstacle image, and wheelchair inertial data;

[0008] Step 2: Process the data to obtain the position offset of the Hall effect joystick, the angle between the wheelchair's forward direction and the center of the obstacle, and the road surface tilt angle;

[0009] Step 3: Based on fuzzy control, the distance between the wheelchair and the obstacle, the angle between the wheelchair's direction of travel and the center of the obstacle, and the road surface tilt angle are fused to obtain a comprehensive environmental coefficient;

[0010] Step 4: Calculate the expected speeds of the left and right wheels of the wheelchair under different operating conditions;

[0011] (1) When both |ΔX| and |ΔY| are less than or equal to the error threshold ε, it is considered that the Hall effect lever was accidentally touched by a human, and the wheelchair does not move.

[0012] (2) When |ΔY| > c|ΔX|, the wheelchair is in a straight running state. If ΔY > ε, the wheelchair is moving straight forward, and the expected speeds of the left and right wheels of the wheelchair are both: V l = V r = w·ΔY·K·m; if ΔY < -ε, the wheelchair is moving straight backward, and the expected speeds of the left and right wheels of the wheelchair are both: V l = V r = w·ΔY·K·m;

[0013] (3) When |ΔY| < 1 / c|ΔX|, the wheelchair is in a stationary turning state. If ΔX > ε, the wheelchair is turning right in place, and the expected speeds of the left and right wheels are respectively: V l = w·ΔX·K·m, V r = w·-ΔX·K·m; if ΔX < -ε, the wheelchair is turning left in place, and the expected speeds of the left and right wheels are respectively: V l = w·ΔX·K·m, V r = w·-ΔX·K·m;

[0014] (4) When (-1 / c)ΔX < ΔY < -cΔX or 1 / cΔX < ΔY < cΔX, and both |ΔX| and |ΔY| are greater than the error threshold ε, the wheelchair is in a turning while moving forward state, and the expected speeds of the left and right wheels are respectively: V l = w·(ΔY + b·ΔX)·K·m, V l = w·(ΔY - b·ΔX)·K·m;

[0015] (5) When cΔX < ΔY < 1 / cΔX or -cΔX < ΔY < (-1 / c)ΔX, and both |ΔX| and |ΔY| are greater than the error threshold ε, the wheelchair is in a turning while moving backward state, and the expected speeds of the left and right wheels are respectively: V l = w·(ΔY + b·ΔX)·K·m, V l = w·(ΔY - b·ΔX)·K·m;

[0016] Among them, (ΔX, ΔY) is the position offset of the Hall rocker, c is a constant, V l is the expected speed of the left wheel of the wheelchair, V r is the expected speed of the right wheel of the wheelchair, K is the wheelchair gear coefficient, w is the proportionality coefficient, m is the comprehensive environment coefficient, b is a constant; in the turning while moving forward and turning while moving backward, ΔY + b·ΔX is the command value of the left wheel, and ΔY - b·ΔX is the command value of the right wheel. When the command value is positive, the motor rotates forward, and when the command value is negative, the motor rotates backward.

[0017] Furthermore, the angle θ between the forward direction of the wheelchair and the center of the obstacle is calculated by the following formula:

[0018]

[0019] In the formula, a′ is the width of the obstacle image, and b′ is the distance from the center point of the obstacle in the image to the straight line along the direction of the wheelchair's movement. This refers to the camera's imaging angle range.

[0020] Furthermore, the formula for calculating the road surface inclination angle γ is as follows:

[0021]

[0022] In the formula, as well as Let x, y, and z be the components of the fused wheelchair posture at time k, respectively, along the x, y, and z axes. To incorporate the physical aspects of the wheelchair posture.

[0023] Furthermore, the wheelchair inertial data includes the acceleration from the accelerometer and the angular velocity from the gyroscope. The wheelchair attitude is calculated based on the acceleration and angular velocity, respectively, and the calculated attitudes are fused to obtain the fused wheelchair attitude as shown in the following formula.

[0024] q est (k)=σq r (k)+(1-σ)q ω (k) (17)

[0025] In the formula, q est (k) represents the fused wheelchair posture at time k, q r (k) represents the wheelchair posture at time k obtained from the acceleration calculation, q ω (k) represents the wheelchair posture at time k obtained from the angular velocity calculation, and σ∈(0,1) is the weighting factor.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] This invention first collects voltage signals from a Hall effect joystick, the distance between the wheelchair and an obstacle, obstacle images, and wheelchair inertial data using multiple sensors to obtain environmental information including the distance between the wheelchair and the obstacle, the angle between the wheelchair's forward direction and the obstacle's center, and the road surface tilt angle. Then, based on this environmental information, the wheelchair's speed setting, and its operating status, fuzzy logic control is used to fuse this information, resulting in a comprehensive environmental coefficient reflecting the wheelchair's operating environment. Finally, based on the comprehensive environmental coefficient and the wheelchair's operating status, a differential speed optimization model for the left and right wheels is established, thereby obtaining the desired speeds for the left and right wheels. This method fully extracts environmental information and achieves full coupling between environmental information, wheelchair operating status, and control signals, maximizing the safety and reliability of the wheelchair during operation and enhancing its intelligence. Attached Figure Description

[0028] Figure 1 It is the overall flowchart;

[0029] Figure 2 It is a two-dimensional voltage diagram of a Hall effect rocker.

[0030] Figure 3 Diagram of a pinhole imaging model for a CCD camera;

[0031] Figure 4 This is a flowchart of environmental information fusion based on fuzzy logic control. Detailed Implementation

[0032] Specific embodiments are given below with reference to the accompanying drawings. These specific embodiments are only used to describe the technical solution of the present invention in detail, and are not intended to limit the scope of protection of this application.

[0033] This invention provides an intelligent wheelchair control method based on multi-sensor information fusion (hereinafter referred to as the method, see below). Figures 1-4 The process includes the following steps:

[0034] Step 1: The voltage signal of the Hall joystick is collected by the Hall sensor, the distance between the wheelchair and the obstacle is collected by the ultrasonic sensor, the obstacle image is collected by the visual sensor, and the inertial data of the wheelchair is collected by the IMU sensor, including the acceleration of the accelerometer and the angular velocity of the gyroscope.

[0035] The second step is data processing, which includes obtaining the position offset (ΔX, ΔY) of the Hall joystick based on the voltage signal of the Hall joystick, filtering the distance between the wheelchair and the obstacle, processing the obstacle image to obtain the angle θ between the wheelchair's forward direction and the center of the obstacle, and calculating the road tilt angle γ based on the data collected by the IMU sensor.

[0036] Hall effect joysticks use Hall sensors to detect their position. The Hall sensors sense the joystick's position based on magnetic field strength. The Hall sensors are mounted on the joystick; movement of the joystick causes changes in the magnetic field strength, generating different voltage signals. The voltage signals from the Hall effect joystick are then converted from analog to digital signals using an analog-to-digital converter. Based on the lateral and longitudinal characteristics of the Hall effect joystick, the voltage signals are split into X and Y signals, which serve as the horizontal and vertical coordinates representing the joystick's position, respectively. Therefore, the operating plane of the Hall effect joystick can be considered a two-dimensional plane. Figure 2 As shown, the voltage horizontal axis ranges from 0 to 5V, with 0 to 2.5V indicating the wheelchair turning left and 2.5 to 5V indicating the wheelchair turning right; the voltage vertical axis also ranges from 0 to 5V, with 0 to 2.5V indicating the wheelchair moving backward and 2.5 to 5V indicating the wheelchair moving forward; the position offset of the Hall effect lever is represented by (ΔX, ΔY).

[0037] This embodiment uses Kalman filtering to remove distance noise between the wheelchair and obstacles. The Kalman filter system equations consist of two parts: a state transition equation and an observation equation. The specific formulas are as follows:

[0038] x k =Ax k-1 +Bu k-1 +w k (1)

[0039] z k =Hx k +v k (2)

[0040] In the formula: x k-1 x k Let A represent the system state variables at times k-1 and k, respectively, and let B represent the system model parameters. k-1 w is the control variable of the system at time k-1. k Let H be the system process noise at time k, and H be the observation matrix. k The system observation noise at time k; z k The system observation value at time k is the measurement value of the ultrasonic sensor at time k.

[0041] System state prediction:

[0042]

[0043] In the formula: Let be the prior estimate of the system state at time k. This is the estimated system state at time k-1. These are the estimated values ​​of the control variables for the system at time k-1;

[0044] Calculate the covariance matrix between the estimated value and the true value:

[0045]

[0046] In the formula: Let P be the prediction covariance at time k. k-1 Let Q be the system state covariance at time k-1, and let Q be the system process noise covariance matrix.

[0047] System observation prediction:

[0048]

[0049] In the formula: This represents the predicted output value at time k.

[0050] Kalman Gain Update:

[0051]

[0052] Where: K k R is the Kalman gain at time k. k Let be the observation noise variance-covariance matrix at time k;

[0053] Filter information term update:

[0054]

[0055] Where: δ k The filtered information term at time k;

[0056] System status update:

[0057]

[0058] In the formula, The system state estimate at time k;

[0059] Covariance matrix update:

[0060]

[0061] In the formula: P k Let I be the estimated value of the covariance matrix at time k, and let I be the identity matrix.

[0062] This embodiment uses a monocular vision sensor, i.e., a CCD camera, to acquire images of obstacles; according to Figure 3 The pinhole imaging model of the CCD camera shown simplifies the camera position to point O, with point T being the center point of the obstacle. It is assumed that the imaging angle range of the CCD camera is... If the width is 'a', the distance from the center point of the obstacle to the straight line along the wheelchair's direction of travel is 'b', then the width of the image captured by the CCD camera is 'a', the position of the obstacle's center point in the image is 'T', and the distance from the position of the obstacle's center point in the image to the straight line along the wheelchair's direction of travel is 'b'. Therefore, the angle θ between the wheelchair's direction of travel and the obstacle's center point is calculated by the following formula:

[0063]

[0064] Since accurate attitude estimation is difficult to achieve using measurements from a single sensor, this embodiment fuses the attitude calculated from gyroscope and accelerometer data. This fusion is achieved using an all-pass filter composed of low-pass and high-pass filters to obtain a more accurate attitude estimate. Specifically, the angular velocity measured by the gyroscope includes components such as self-drift bias and random noise, and can be expressed as:

[0065] ω=ω t +fω +n ω (11)

[0066] In the formula: ω is the measured value of angular velocity, ω t f is the true value of angular velocity. ω n is the temperature drift component of angular velocity. ω The noise component of the angular velocity;

[0067] Attitude calculation is performed using quaternions. Quaternions represent attitude through hypercomplex numbers, consisting of four elements, and their expression is as follows:

[0068] q = q0 + q x i+q y j+q z s (12)

[0069] In the formula: q is a quaternion, i, j, s represent the imaginary part, q0, q x q y q z All are real numbers, q0 is the real part, q x q y q z Represents the components on the x, y, and z axes;

[0070] The quaternion differential has the following relationship with angular velocity:

[0071]

[0072] In the formula: ω x ω y and ω z These are the three-axis components of the angular velocity measurement values;

[0073] By integrating equation (13), the wheelchair posture expressed by quaternions can be obtained.

[0074] Similar to the principle of gyroscope attitude calculation, the acceleration measured by the accelerometer can be expressed as:

[0075] r = r t +r v +f r +n r (14)

[0076] In the formula: r is the measured acceleration value, r t For the target acceleration, r v f is the component of motion acceleration (0 when stationary). r For the temperature drift component of acceleration, n r The noise component represents acceleration;

[0077] When the wheelchair is stationary, the accelerometer data only represents gravitational acceleration. In this case, under coordinate-to-matrix mapping, the acceleration *r* in the wheelchair coordinate system and the gravitational acceleration *g* in the geographic coordinate system are different. n The following relationships exist:

[0078]

[0079]

[0080] In the formula: r x r y r z These are the triaxial accelerations measured by accelerometers. Let g be the rotation matrix, and g be the gravitational constant.

[0081] According to Equation (15), only gravitational acceleration is a useful component in attitude calculation. The wheelchair attitude based on quaternions can be calculated through the gravitational acceleration component.

[0082] Attitude fusion primarily involves fusing the wheelchair attitude calculated from gyroscope and accelerometer measurements. This is achieved through an all-pass filter consisting of low-pass and high-pass filters. The fused wheelchair attitude is represented as follows:

[0083] q est (k)=σq r (k)+(1-σ)q ω (k) (17)

[0084] In the formula: q est (k) represents the fused wheelchair posture at time k, q r (k) represents the wheelchair posture at time k obtained from the acceleration calculation, q ω (k) represents the wheelchair posture at time k obtained by angular velocity calculation; σ∈(0,1) is a weighting factor, the value of which depends on the wheelchair motion state. When the wheelchair is at a high speed, the proportion of interference components in the accelerometer increases, and the wheelchair posture obtained by acceleration calculation is unreliable. Therefore, σ should be as small as possible.

[0085] Obtain the fused pose q based on quaternions estAfter (k), Euler angles can be calculated using quaternions. Euler angles are represented by three angles: roll, yaw, and pitch. In this embodiment, attitude angles are defined by rotating around a fixed axis in the positive direction. Specifically, the yaw angle is obtained by rotating the other two axes of the initial coordinate system around the fixed axis z, the pitch angle is obtained by rotating the other two axes of the coordinate system after one rotation around the fixed axis y, and the roll angle is obtained by rotating the other two axes of the coordinate system after two rotations around the fixed axis x. Here, the x-axis points in the direction of wheelchair movement, the z-axis points to the ground, and the y-axis follows the right-hand rule. Therefore, when the wheelchair pitch angle is known, the road tilt angle γ can be obtained, and its calculation formula is as follows:

[0086]

[0087] In the formula, as well as Let x, y, and z be the components of the fused wheelchair posture at time k, respectively, along the x, y, and z axes. To incorporate the real parts of the wheelchair posture;

[0088] Step 3: Combine the distance between the wheelchair and the obstacle, the angle between the wheelchair's direction of travel and the center of the obstacle, and the road surface inclination angle to obtain the comprehensive environmental coefficient;

[0089] Fuzzy logic control is used to fuse the distance between the wheelchair and the obstacle, the angle between the wheelchair's direction of travel and the center of the obstacle, and the road surface inclination angle to obtain a comprehensive environmental coefficient. This coefficient describes the overall environment in which the wheelchair operates. The comprehensive environmental coefficient is denoted by m, and its value ranges from [0,1]. The larger the comprehensive environmental coefficient, the less it affects the wheelchair's speed, and the safer the operation. The specific steps are as follows:

[0090] (1) Determine the structure of the fuzzy controller

[0091] The fuzzy controller is a multi-input single-output structure. The input variables include the distance d between the wheelchair and the obstacle, the angle θ between the wheelchair's forward direction and the center of the obstacle, and the road surface tilt angle γ. The output variable is the comprehensive environmental coefficient m.

[0092] (2) Variable fuzzification

[0093] Variable fuzzification is the process of mapping the magnitudes of the input and output variables of a fuzzy controller to fuzzy linguistic variables. This mapping process uses a triangular membership function. Considering the wheelchair's operating environment and sensor performance, this embodiment sets the fuzzy segmentation number of the distance d between the wheelchair and the obstacle to 3, with fuzzy linguistic variables {near, medium, far} and a universe of discourse [0, 6m]. The fuzzy segmentation number of the angle θ between the wheelchair's forward direction and the obstacle's center is set to 5, with fuzzy linguistic variables {negative small, negative large, zero, positive small, positive large}. The universe of discourse is ±90 degrees, which is approximately [-1.5, 1.5] radians. The angle between the obstacle and the wheelchair is defined as positive when the obstacle is on the right side of the wheelchair and negative otherwise. The fuzzy segmentation number of the road slope angle γ is set to 5, and its fuzzy linguistic variables are {negative large, negative small, zero, positive small, positive large}, with a universe of discourse of ±90 degrees. According to the definition of the Euler angle coordinate system, when the road slope angle is positive, the wheelchair is in an uphill state. The fuzzy segmentation number of the comprehensive environmental coefficient m is set to 5, and its fuzzy linguistic variables are {zero, small, medium, large, relatively large}, with a universe of discourse of [0, 1].

[0094] (3) Establish a fuzzy rule base

[0095] The fuzzy control rules in this embodiment are formulated using a "perception-result" approach. This involves sensing environmental information through various sensors, analyzing the information, and determining the magnitude of the comprehensive environmental coefficient *m*. When formulating the fuzzy control rules, the influence of environmental factors must be considered throughout the wheelchair's operation to ensure its speed remains within a safe range. For example, the control rules state: when γ is large, d is close, and θ is small, the wheelchair's operating environment is considered harsh, and its speed needs to be reduced to maintain a safe operating speed; therefore, the comprehensive environmental coefficient should be smaller. When γ is small, d is far, and θ is large, the wheelchair's operating environment is considered good, and its speed is primarily controlled by the Hall effect sensor signal, making it less affected by the environment; therefore, the comprehensive environmental coefficient should be larger. Based on the fuzzy rules, the fuzzy relationship is obtained as follows:

[0096]

[0097] In the formula, R t Let represent the fuzzy relation corresponding to the t-th fuzzy rule, and n be the number of fuzzy rules;

[0098] (4) Fuzzy reasoning

[0099] Fuzzy inference of the comprehensive environmental coefficient is performed based on the established fuzzy rule base. The commonly used inference method is the min-max method, which interprets the fuzzy implication relationship of A and B and C→D as a direct product of A×B×C×D, then:

[0100] R=A and B andC→D=A×B×C×D

[0101] or

[0102] μ R (γ,d,θ,m)=μ A (γ)∩μ B (d)∩μ C (θ)∩μ D (m)

[0103] (5) Fuzzy decision-making

[0104] The fuzzy output obtained through fuzzy inference is a fuzzy linguistic variable and cannot be directly used as a control variable. Therefore, the fuzzy output needs to be defuzzified. This embodiment uses the centroid method for defuzzification to obtain the specific value of the comprehensive environmental coefficient, which is used to evaluate the degree of influence of the environment on the omnidirectional differential speed operation of the wheelchair. The process is as follows:

[0105] Given d, θ, γ, and all fuzzy rules, the preliminary reasoning result is as follows:

[0106]

[0107] In the formula, This represents the t-th preliminary reasoning result. The t-th fuzzy linguistic variable represents the road surface inclination angle γ. The t-th fuzzy linguistic variable represents the distance d between the wheelchair and the obstacle. The t-th fuzzy linguistic variable represents the angle θ between the wheelchair's direction of travel and the center of the obstacle. This represents the t-th fuzzy linguistic variable representing the overall environmental coefficient m;

[0108] Output fuzzy set Depend on The combined formula is as follows:

[0109]

[0110] Output fuzzy inference set The "centroid" is the centroid of the region enclosed by the membership function curve. Its corresponding coordinates are the accurate values ​​of the comprehensive environmental coefficient after defuzzification. The calculation formula is:

[0111]

[0112] In the formula, This represents the membership value of the q-th subset of the output fuzzy set, m. q Let x be the x-coordinate of a point on the membership function of the q-th subset, and N represent the number of subsets in the output fuzzy set.

[0113] Step 4: Determine the wheelchair running state based on the wheelchair position offset, perform information coupling on the wheelchair gear information, position offset, and comprehensive environment coefficient, and calculate the expected speeds of the left and right wheels of the wheelchair in each running state;

[0114] (1) Set the error thresholds for the horizontal and vertical position offsets ΔX and ΔY of the Hall rocker to be both ε. When both |ΔX| and |ΔY| are less than or equal to the error threshold ε, it is considered that the Hall rocker is accidentally touched by a person, and the wheelchair does not move at this time;

[0115] (2) When |ΔY| > c|ΔX|, the wheelchair running state is straight. If ΔY > ε, the wheelchair is moving straight forward. At this time, the expected speeds of the left and right wheels of the wheelchair are both: V l = V r = w·ΔY·K·m; if ΔY < -ε, the wheelchair is moving straight backward. At this time, the expected speeds of the left and right wheels of the wheelchair are both: V l = V r = w·ΔY·K·m; where c is a constant, and the value range is [5, 10]; V l is the expected speed of the left wheel of the wheelchair, V r is the expected speed of the right wheel of the wheelchair, K is the wheelchair gear coefficient, w is the proportionality coefficient. The proportionality coefficient is used to map the expected speed to the actual speed range of the motor. At this time, whether ΔX is equal to 0 or not, it will be ignored. The expected speed is positive, and the motor rotates forward, otherwise it rotates backward;

[0116] (3) When |ΔY| < 1 / c|ΔX|, the wheelchair running state is turning in place. If ΔX > ε, the wheelchair is turning right in place. At this time, the expected speeds of the left and right wheels are respectively: V l = w·ΔX·K·m, V r = w·-ΔX·K·m; if ΔX < -ε, the wheelchair is turning left in place. At this time, the expected speeds of the left and right wheels are respectively: V l = w·ΔX·K·m, V r = w·-ΔX·K·m; When turning in place, regardless of the value of ΔY, it will be ignored. The magnitudes of the expected speeds of the left and right wheels are equal, and the directions are opposite;

[0117] (4) When (-1 / c)ΔX < ΔY < -cΔX or 1 / cΔX < ΔY < cΔX, and both |ΔX| and |ΔY| are greater than the error threshold ε, the wheelchair running state is turning while moving forward. The differential method is used to achieve turning while moving forward. The expected speeds of the left and right wheels are respectively: V l = w·(ΔY + b·ΔX)·K·m, V l=w·(ΔY-b·ΔX)·K·m; where ΔY+b·ΔX is the command value of the left wheel, and ΔY-b·ΔX is the command value of the right wheel. When the command value is positive, the motor rotates forward; when the command value is negative, the motor rotates in reverse. The same applies when turning in reverse. b is a constant with a value range of [0,1].

[0118] (5) When cΔX < ΔY < 1 / cΔX or -cΔX < ΔY < (-1 / c)ΔX, and both |ΔX| and |ΔY| are greater than the error threshold ε, the wheelchair is in a backward turning state. The differential speed method is also used to achieve backward turning. The expected speeds of the left and right wheels are respectively: V l =w·(ΔY+b·ΔX)·K·m, V l =w·(ΔY-b·ΔX)·K·m.

[0119] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A method for controlling an intelligent wheelchair based on multi-sensor information fusion, characterized in that, Includes the following steps: Step 1: Collect data, including Hall effect sensor voltage signals, distance between the wheelchair and the obstacle, obstacle image, and wheelchair inertial data; Step 2: Process the data to obtain the position offset of the Hall effect joystick, the angle between the wheelchair's forward direction and the center of the obstacle, and the road surface tilt angle; Step 3: Based on fuzzy control, the distance between the wheelchair and the obstacle, the angle between the wheelchair's direction of travel and the center of the obstacle, and the road surface tilt angle are fused to obtain the comprehensive environmental coefficient; Step 4: Calculate the expected speeds of the left and right wheels of the wheelchair under different operating conditions; (1) When both |ΔX| and |ΔY| are less than or equal to the error threshold ε, it is considered that the Hall effect lever was accidentally touched by a human, and the wheelchair does not move. (2) When |ΔY|>c|ΔX|, the wheelchair is moving in a straight line. If ΔY>ε, the wheelchair is moving forward in a straight line. The expected speeds of the left and right wheels of the wheelchair are both: V l =V r = w·ΔY·K·m; If ΔY < -ε, then the wheelchair is moving in a straight line backward, and the expected speed of the left and right wheels of the wheelchair is: V l =V r =w·ΔY·K·m; (3) When |ΔY| < 1 / c|ΔX|, the wheelchair is turning in place. If ΔX > ε, the wheelchair is turning right in place. The expected speeds of the left and right wheels are respectively: V l =w·ΔX·K·m,V r = w·-ΔX·K·m; If ΔX<-ε, then the wheelchair will turn left in place, and the expected speeds of the left and right wheels are respectively: V l =w·ΔX·K·m,V r =w·-ΔX·K·m; (4) When (-1 / c)ΔX < ΔY < -cΔX or 1 / cΔX < ΔY < cΔX, and both |ΔX| and |ΔY| are greater than the error threshold ε, the wheelchair running state is turning while moving forward, and the expected speeds of the left and right wheels are respectively: V l = w·(ΔY + b·ΔX)·K·m, V l = w·(ΔY - b·ΔX)·K·m; (5) When cΔX < ΔY < 1 / cΔX or -cΔX < ΔY < (-1 / c)ΔX, and both |ΔX| and |ΔY| are greater than the error threshold ε, the wheelchair is in a turning while reversing state, and the desired speeds of the left and right wheels are respectively: V l = w·(ΔY + b·ΔX)·K·m, V l = w·(ΔY - b·ΔX)·K·m; Where (ΔX, ΔY) is the position offset of the Hall effect rocker, c is a constant, and V l V is the desired speed of the wheelchair's left wheelchair. r ΔY is the desired speed of the right wheelchair, K is the wheelchair gear ratio, w is the proportional coefficient, m is the comprehensive environmental coefficient, and b is a constant. When turning in forward and backward movements, ΔY+b·ΔX is the command value of the left wheel and ΔY-b·ΔX is the command value of the right wheel. A positive command value means the motor rotates forward, and a negative command value means the motor rotates in reverse.

2. The intelligent wheelchair control method based on multi-sensor information fusion according to claim 1, characterized in that, The angle θ between the wheelchair's forward direction and the center of the obstacle is calculated by the following formula: In the formula, a′ is the width of the obstacle image, and b′ is the distance from the center point of the obstacle in the image to the straight line along the direction of the wheelchair's movement. This refers to the camera's imaging angle range.

3. The intelligent wheelchair control method based on multi-sensor information fusion according to claim 1 or 2, characterized in that, The formula for calculating the road surface inclination angle γ is as follows: In the formula, as well as Let x, y, and z be the components of the fused wheelchair posture at time k, respectively, along the x, y, and z axes. To incorporate the physical aspects of the wheelchair posture.

4. The intelligent wheelchair control method based on multi-sensor information fusion according to claim 3, characterized in that, Wheelchair inertial data includes acceleration from accelerometers and angular velocity from gyroscopes. The wheelchair attitude is calculated based on acceleration and angular velocity, respectively, and the calculated attitudes are fused to obtain the fused wheelchair attitude as shown in the following formula. q est (k)=σq r (k)+(1-σ)q ω (k) (17) In the formula, q est (k) represents the fused wheelchair posture at time k, q r (k) represents the wheelchair posture at time k obtained from the acceleration calculation, q ω (k) represents the wheelchair posture at time k obtained from the angular velocity calculation, and σ∈(0,1) is the weighting factor.

Images