A method for intelligent wheelchair control based on multi-source information fusion

Abstract

A multi-source information fusion-based intelligent wheelchair control method belongs to the field of data recognition and intelligent wheelchair control. The method involves: constructing a correspondence between hand gestures and wheelchair control directions, and pre-setting a recognition framework; processing the acquired multi-source information to obtain the basic probability allocation result of the evidence corresponding to each single-source information; determining the absolute importance of the evidence corresponding to each single-source information in the fusion process; determining the relative importance of the evidence corresponding to each single-source information in the fusion process; determining the comprehensive importance of the evidence based on the obtained relative and absolute importance; reallocating the fusion weights of the evidence corresponding to each single-source information based on the comprehensive importance of the evidence corresponding to each single-source information to obtain weighted new evidence; applying DS evidence fusion to obtain the final fused hand gesture recognition result; and controlling the wheelchair behavior based on the hand gesture recognition result to achieve intelligent wheelchair control based on multi-source information fusion.

Description

Technical Field

[0001] This invention belongs to the field of data recognition and intelligent wheelchair control, and relates to an intelligent wheelchair control method that integrates multi-source information. Background Technology

[0002] As an important assistive mobility tool for the elderly and disabled, intelligent wheelchairs are receiving increasing attention for their intelligent and user-friendly design. Traditional intelligent wheelchair control methods mainly rely on joysticks, head posture control, or brain-computer interfaces. However, these control methods have some limitations. For example, joystick control requires users to have good upper limb manipulation skills, which can be difficult for users with limited upper limb strength; head posture control can easily cause neck fatigue; and brain-computer interface control requires high technological costs, making large-scale adoption difficult.

[0003] Gesture control, as an emerging human-computer interaction method, offers advantages such as intuitiveness and non-contact operation, providing new possibilities for the control of intelligent wheelchairs. However, existing gesture-based wheelchair control methods mostly rely on a single information source, resulting in low reliability and limited recognition dimensions, making it difficult to meet the needs of stable control in everyday environments. Therefore, there is an urgent need for an intelligent wheelchair control solution that can overcome the limitations of a single information source to improve the accuracy and robustness of gesture recognition. Summary of the Invention

[0004] To overcome the limitations of controlling a wheelchair from a single information source, the present invention aims to provide a multi-source information fusion intelligent wheelchair control method. This method acquires multi-source information for recognizing hand gestures, establishes a correspondence between hand gestures and wheelchair control directions, recognizes hand gestures by fusing multi-source information, and controls wheelchair behavior based on the recognition results, thereby improving the control accuracy and efficiency of the intelligent wheelchair.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] This invention discloses a multi-source information fusion-based intelligent wheelchair control method. The method acquires multi-source information for recognizing hand gestures, constructs a correspondence between hand gestures and wheelchair control directions, and presets a recognition framework. It processes the acquired multi-source information to obtain a basic probability allocation result for the evidence corresponding to each single-source information. Based on this basic probability allocation result, it determines the absolute importance of the evidence corresponding to each single-source information in the fusion process. Based on the basic probability allocation result, it determines the relative importance of the evidence corresponding to each single-source information in the fusion process. Based on the relative and absolute importance, it determines the comprehensive importance of the evidence. Based on the comprehensive importance of the evidence corresponding to each single-source information, it reassigns fusion weights to each piece of evidence, obtaining weighted new evidence. It applies DS evidence fusion to obtain the final fused hand gesture recognition result, and controls the wheelchair's behavior based on the hand gesture recognition result, thus achieving intelligent wheelchair control based on multi-source information fusion. This invention can improve the accuracy of intelligent wheelchair control by fusing multi-source information.

[0007] The present invention discloses a multi-source information fusion intelligent wheelchair control method, comprising the following steps:

[0008] Step 1: Pre-set a recognition framework consisting of five hand gestures: palm forward, palm backward, palm left, palm right, and fist clench. Acquire three types of multi-source information for hand gesture recognition: electromyography (EMG) signals, acceleration signals, and angle signals from the arm. Determine the hand gesture category within the recognition framework based on the multi-source information. The five pre-set hand gestures correspond to five wheelchair control behaviors. Construct a correspondence between hand gestures and wheelchair control behaviors based on the recognition framework and multi-source information: palm forward represents forward wheelchair movement, palm backward represents backward wheelchair movement, palm left represents left turn, palm right represents right turn, and fist clench represents stopping the wheelchair.

[0009] The preset recognition framework consists of five gesture actions. ,in Represents the palm turning forward, Represents the palm turning backward, Represents the palm turning to the left, Represents the palm turning to the right. The hand gesture represents a clenched fist. Electromyography (EMG) signals S1 from the arm are acquired using an EMG sensor, while acceleration signals S2 and angle signals S3 are acquired using an accelerometer and tilt sensor. These signals are used as multi-source information for recognizing hand gestures. Based on the recognition framework and the multi-source information, a correspondence between hand gestures and wheelchair control directions is constructed. Specifically, five hand gestures correspond to five wheelchair control actions: a forward hand gesture represents forward movement, a backward hand gesture represents backward movement, a leftward hand gesture represents leftward turn, a rightward hand gesture represents rightward turn, and a clenched fist gesture represents stopping the wheelchair.

[0010] Step 2: Process the multi-source information obtained in Step 1 to obtain the basic probability allocation results of the evidence corresponding to each single-source information.

[0011] As a preferred option, the specific implementation method of step two is as follows:

[0012] Step 2.1: Construct three backpropagation neural network models. The inputs of the three neural network models are electromyography signal S1, acceleration signal S2, and angle signal S3, respectively. The output layer function of each neural network model is the Softmax function. The output value of the Softmax function is between 0 and 1. The output category is the five hand gestures in the recognition framework of Step 1, namely, palm forward, palm backward, palm left, palm right, and fist.

[0013] Step 2.2: Use the multi-source information for recognizing gestures to train the three neural network models constructed in Step 2.1, and obtain the three trained neural network models;

[0014] Step 2.3: Input the real-time multi-source information into the neural network model trained in Step 2.2. The value of each output layer of the neural network model is the basic probability allocation result of the evidence corresponding to each single-source information. The basic probability allocation of the evidence corresponding to each single-source information is expressed as follows: m i ( ), m i ( ), m i ( ), m i ( ), m i ( ),in i =1, 2, 3, m 1 represents the evidence corresponding to the single-source electromyographic signal S1. m 2 represents the evidence corresponding to the single-source acceleration signal S2. m3 represents the evidence corresponding to the single-source angle signal S3; m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability allocation.

[0015] Step 3: Based on the basic probability allocation results of the evidence corresponding to each single source information obtained in Step 2, determine the absolute importance of the evidence corresponding to each single source information in the fusion process.

[0016] Step 3.1: Within the gesture recognition action framework In the middle, the evidence corresponding to each single-source signal m i The certainty of Cer (Certainty) is represented as:

[0017] (1)

[0018] In equation (1), i = 1, 2, 3, representing the electromyographic signal S1, acceleration signal S2, and angle signal S3 of the arm, respectively; k = 1, 2, 3, 4, 5, representing the hand gesture of turning the palm forward. Turn your palm back Turn your palm to the left Turn your palm to the right and clenched fist ;

[0019] The certainty of the evidence corresponding to each single-source signal is calculated according to equation (1).

[0020] Step 3.2: Calculate the absolute importance (Abs) of the evidence corresponding to each single-source signal in the fusion process according to equation (2):

[0021] (2)

[0022] In equation (2), i = 1, 2, 3, representing the electromyographic signal S1, acceleration signal S2, and angle signal S3 of the arm, respectively. The higher the certainty of each single-source signal, the more important the evidence is in the fusion process, and the higher the weight assigned to it in the fusion process.

[0023] Step 4: Based on the basic probability allocation results of the evidence corresponding to each single source information obtained in Step 2, determine the relative importance of the evidence corresponding to each single source information in the fusion process.

[0024] Step 4.1: To facilitate the determination of the conflict coefficient between pieces of evidence, within the gesture recognition action framework... In the middle, the definition m 1 and m 2 consists of two sets of evidence. m The focal length of 1 is , m The focal length of 2 is Then the two sets of evidence m 1 and m The conflict coefficient between 2 is expressed as:

[0025] (3)

[0026] The conflict coefficient between the evidences is calculated according to equation (3).

[0027] Step 4.2: Define and solve the maximum mean difference (MMD) distance between the evidence, and construct the MMD distance matrix between different pieces of evidence based on the obtained MMD distance. Normalize the matrix. Define and solve the MMD conflict coefficient based on the conflict coefficient and the normalized MMD distance in Step 4.1, and construct the MMD conflict coefficient matrix.

[0028] evidence m 1 and m The MMD distance of 2 is expressed as:

[0029] (4)

[0030] In equation (4), p represents evidence. m The probability vector of 1 joule, where q represents evidence.m The probability vector of the two focal elements, K represents the Gaussian radial basis function matrix, and the formulas for solving p and q are given in formula (5), and the formula for solving K is given in formula (6):

[0031] (5)

[0032] (6)

[0033] In equation (6), k The solution formula for the Gaussian radial basis function is as follows:

[0034] (7)

[0035] In equation (7), x and y Represents a gesture recognition action framework elements in For ease of calculation, Perform one-hot encoding. Represents the bandwidth parameter, take .

[0036] The MMD distance matrix between different pieces of evidence is constructed as follows:

[0037] (8)

[0038] Find the maximum value in the MMD distance matrix The MMD distance matrix is ​​normalized by dividing each element of the MMD distance matrix by 1 / 2. The normalized MMD distance matrix is ​​obtained as shown in equation (9):

[0039] (9)

[0040] Based on the conflict coefficient and the normalized MMD distance definition from step 4.1, the MMD conflict coefficient is calculated and expressed as follows:

[0041] (10)

[0042] In equation (10), Representative evidence m 1 and m A conflict coefficient of 2 Representative evidence m 1 and m 2. Normalized MMD distance;

[0043] The MMD conflict coefficient matrix is ​​constructed according to formula (10) as follows:

[0044] (11)

[0045] Step 4.3: Normalize the MMD conflict coefficient matrix from Step 4.2, and define and solve the comprehensive influence matrix among the various pieces of evidence;

[0046] Find the maximum value in the MMD conflict coefficient matrix In step 4.2, each element of the MMD conflict coefficient matrix is ​​divided by the maximum value to normalize the MMD conflict coefficient matrix, which is represented as follows:

[0047] (12)

[0048] Define the overall influence matrix T among the various pieces of evidence as follows:

[0049] (13)

[0050] In equation (13), I is a 3-by-3 identity matrix;

[0051] Step 4.4: Calculate the row sum and column sum of the comprehensive influence matrix in Step 4.3 to determine the importance of each piece of evidence in the fusion process;

[0052] After obtaining the comprehensive influence matrix T from step 4.3, the elements in T can be represented as follows: t fg , f represents the number of rows, f = 1, 2, 3 g Represents the column number. g = 1, 2, 3, let r f For the comprehensive influence matrix T, the th f The sum of the rows, representing the first row. f The degree of overall influence of the line on other factors r f The solution formula is:

[0053] (14)

[0054] make c g For the matrix of the first g The sum of the columns, representing the first column. g The degree of influence of the column on other factors c g The solution formula is:

[0055] (15)

[0056] Based on the row sum of the comprehensive influence matrix T rf and column sum c g Determine the importance of each piece of evidence in the fusion process. ID (Importance Degree) is represented as:

[0057] (16)

[0058] Step 4.5: Based on the importance obtained in Step 4.4, determine the relative importance of each piece of evidence in the fusion process;

[0059] Calculate the relative evidence corresponding to each single-source signal during the fusion process. Rel (Relative) Importance is expressed as:

[0060] (17)

[0061] In equation (15), i = 1, 2, 3, representing the electromyographic signal S1, acceleration signal S2, and angle signal S3 of the arm, respectively. The higher the importance of each single-source signal, the more important the evidence is in the fusion process, and the higher the weight assigned to it in the fusion process.

[0062] Step 5: Based on the absolute importance of the evidence corresponding to the single-source information determined in Step 3 in the fusion process, and the relative importance of the evidence corresponding to the single-source information determined in Step 4 in the fusion process, determine the comprehensive importance of the evidence corresponding to the single-source information in the fusion process.

[0063] Based on the absolute importance of the fusion obtained in step 3.2 and the relative importance obtained in step 4.5, the synthesis of evidence is determined according to equation (18). Com (Comprehensive) Importance:

[0064] (18)

[0065] Step Six: Based on the overall importance of the evidence corresponding to the single-source information obtained in Step Five, reassign the fusion weight to the evidence corresponding to each single-source information to obtain the new weighted evidence.

[0066] Based on the overall importance of the evidence corresponding to the single-source information obtained in step five, the fusion weights are reassigned to the evidence corresponding to each single-source information. The formula for calculating the weights is as follows:

[0067] (19)

[0068] Based on the weight assigned to each piece of evidence, a new piece of evidence is obtained by weighted averaging, expressed as:

[0069] (20)

[0070] New evidence was obtained regarding the gestures and actions. The basic probability assignment result at time is expressed as:

[0071] (twenty one)

[0072] (twenty two)

[0073] (twenty three)

[0074] (twenty four)

[0075] (25);

[0076] According to equations (21) to (25), the fusion weights are reassigned to the evidence corresponding to each single source information to obtain the new weighted evidence.

[0077] Step 7: Apply the DS evidence fusion rule to fuse the weighted new evidence obtained in Step 6. After fusion, the basic probability allocation results of the five gesture actions in the recognition framework of Step 1 are obtained. The gesture action corresponding to the maximum value of the basic probability allocation is the recognition result. Control the wheelchair behavior according to the recognition result, that is, realize intelligent wheelchair control based on multi-source information fusion.

[0078] Step 7.1: To facilitate the presentation of DS evidence fusion rules, within the gesture recognition action framework... In the middle, the definition m 1 and m 2 consists of two sets of evidence. m The focal length of 1 is , m The focal length of 2 is Then the DS fusion rule for the two sets of evidence is expressed as:

[0079] (26)

[0080] The weighted new evidence is fused twice, and the DS evidence fusion rule is used to fuse the new evidence to obtain the basic probability allocation results of the corresponding gesture actions.

[0081] Step 7.2: Traverse the basic probability allocation results of all gesture recognition actions in Step 7.1, determine the gesture action corresponding to the maximum value of the basic probability allocation, and map the corresponding wheelchair control behavior according to the correspondence between the gesture action and the wheelchair control direction constructed in Step 1, thus realizing intelligent wheelchair control based on multi-source information fusion.

[0082] Beneficial effects:

[0083] 1. The intelligent wheelchair control method based on multi-source information fusion disclosed in this invention pre-sets a gesture recognition framework. Under the gesture recognition framework, the absolute importance of evidence in the fusion process is obtained by solving the certainty of the evidence corresponding to single-source information. The relative importance of evidence in the fusion process is obtained by constructing the MMD conflict coefficient, solving the comprehensive influence matrix, and solving the importance. The comprehensive importance of evidence is determined based on the relative and absolute importance of evidence. The fusion weight of evidence is determined based on the comprehensive importance. The new evidence after weighted averaging is solved. The DS evidence theory is applied to fuse the new evidence. The wheelchair movement is controlled according to the fusion recognition result to achieve precise and intelligent control of wheelchair behavior.

[0084] 2. The intelligent wheelchair control method based on multi-source information fusion disclosed in this invention pre-defines five hand gestures corresponding to five wheelchair control actions: a forward palm flip represents forward movement, a backward palm flip represents backward movement, a left palm flip represents left turn, a right palm flip represents right turn, and a clenched fist represents stopping. Complex path planning is achieved through combinations of these five hand gestures, constructing a complete and efficient wheelchair control system. These five hand gestures have significant differences, fundamentally reducing the risk of misidentification, and the "clenched fist" gesture is set as a stop command that conforms to human instinct, providing safety redundancy. The pre-defined five hand gestures and their corresponding wheelchair control actions achieve a balance between simplicity and functional completeness in intelligent wheelchair control.

[0085] 3. The intelligent wheelchair control method based on multi-source information fusion disclosed in this invention applies a neural network model to solve for the basic probability allocation results of evidence corresponding to single-source information. The neural network model can be pre-trained using prior information, accurately and efficiently capturing and recognizing nonlinear gesture patterns contained in the information during intelligent wheelchair control, without relying on manually set fixed threshold rules. The Softmax output layer transforms the network recognition results into a normalized probability distribution from 0 to 1. Based on the probability distribution, the confidence level of each gesture is intuitively quantified, improving the accuracy and efficiency of multi-source information fusion in recognizing gesture information, thereby improving the accuracy and robustness of intelligent wheelchair control decisions in complex real-world environments. Attached Figure Description

[0086] Figure 1 This is a flowchart of an intelligent wheelchair control method based on multi-source information fusion according to the present invention.

[0087] Figure 2 This is a flowchart for solving absolute importance.

[0088] Figure 3 This is a flowchart for solving relative importance.

[0089] Figure 4 It is the fusion recognition result of the DS fusion rules.

[0090] Figure 5 It is the result of the fusion identification of literature methods.

[0091] Figure 6 It is the result of fusion recognition of existing patents.

[0092] Figure 7 This is the fusion recognition result of the present invention. Detailed Implementation

[0093] To better illustrate the purpose and advantages of this invention, the following description, in conjunction with the accompanying drawings and examples, further explains the invention and compares it with the original DS evidence theory and methods, existing literature, and existing patents.

[0094] like Figure 1 As shown in the figure; this embodiment discloses a multi-source information fusion intelligent wheelchair control method, the specific implementation steps of which are as follows:

[0095] Step 1: Preset a recognition framework consisting of five gestures. ,in Represents the palm turning forward, Represents the palm turning backward, Represents the palm turning to the left, Represents the palm turning to the right. The hand gesture represents a clenched fist. Electromyography (EMG) signals S1 from the arm are acquired using an EMG sensor, while acceleration signals S2 and angle signals S3 are acquired using an accelerometer and tilt sensor. These signals are used as multi-source information for recognizing hand gestures. Based on the recognition framework and the multi-source information, a correspondence between hand gestures and wheelchair control directions is constructed. Specifically, five hand gestures correspond to five wheelchair control actions: a forward hand gesture represents forward movement, a backward hand gesture represents backward movement, a leftward hand gesture represents leftward turn, a rightward hand gesture represents rightward turn, and a clenched fist gesture represents stopping the wheelchair.

[0096] Step 2: Process the multi-source information obtained in Step 1 to obtain the basic probability allocation results of the evidence corresponding to each single-source information.

[0097] Step 2.1: Construct three backpropagation neural network models. The inputs of the three backpropagation neural networks are electromyography signals, acceleration signals, and angle signals, respectively. The output layer function of each network model is the Softmax function. The output value of the Softmax function is between 0 and 1. The output category is the five hand gestures in the recognition framework in Step 1, namely, palm forward, palm backward, palm left, palm right, and fist.

[0098] Step 2.2: Apply the historical multi-source information of the recognized gestures to train the three backpropagation neural network models constructed in Step 2.1, and obtain the trained three backpropagation neural network models.

[0099] Step 2.3: Input the real-time multi-source information into the backpropagation network model trained in Step 2.2. The value of each output layer of the backpropagation network model is the basic probability assignment result of the evidence corresponding to each single-source information. The basic probability assignment of the evidence corresponding to each single-source information is expressed as follows: m i ( ), m i ( ), m i ( ), m i ( ), m i ( ),in i =1, 2, 3, m 1 represents the evidence corresponding to the single-source electromyographic signal S1. m 2 represents the evidence corresponding to the single-source acceleration signal S2. m 3 represents the evidence corresponding to the single-source angle signal S3; m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability allocation.

[0100] The multi-source information in step one consists of three types (electromyography signal S1 from the arm, acceleration signal S2, and angle signal S3). Each single-source information corresponds to a set of evidence. The basic probability distribution of the evidence corresponding to each single-source signal (S1, S2, and S3) is expressed as follows: m i ( i =1, 2, 3), where, m 1 represents the evidence corresponding to the single-source signal S1. m 2 represents the evidence corresponding to the single-source signal S2. m 3 represents the evidence corresponding to the single-source signal S3; the hand gestures in step one include... 、 、 、 , Five types, then the basic probability allocation of the evidence corresponding to each single-source signal is expressed as: m i ( ), m i ( ), m i ( ), m i ( ), m i ( The basic probability allocation results of the evidence corresponding to each single-source signal are shown in Table 1.

[0101] Table 1. Basic probability distribution of evidence corresponding to single-source signals

[0102]

[0103] Step 3: Based on the basic probability allocation results of the evidence corresponding to each single source information obtained in Step 2, determine the absolute importance of the evidence corresponding to each single source information in the fusion process.

[0104] like Figure 2 As shown, the process of determining the absolute importance of evidence is as follows:

[0105] Step 3.1: Calculate the certainty of the evidence corresponding to each single-source signal;

[0106] In gesture recognition action framework In the process, the evidence corresponding to each single-source signal is solved. mi The certainty (Cer) is expressed as follows:

[0107]

[0108]

[0109]

[0110] Step 3.2: Calculate the absolute importance of the evidence corresponding to each single-source signal in the fusion process;

[0111] Calculate the absolute value of the evidence corresponding to each single-source signal during the fusion process. Abs (Absolute) Importance, respectively:

[0112]

[0113]

[0114]

[0115] Step 4: Based on the basic probability allocation results of the evidence corresponding to each single source information obtained in Step 2, determine the relative importance of the evidence corresponding to each single source information in the fusion process.

[0116] like Figure 3 As shown, the process of determining the relative importance of evidence is as follows:

[0117] Step 4.1: Calculate the conflict coefficient between the pieces of evidence;

[0118] In gesture recognition action framework In the middle, the conflict coefficient between the two sets of evidence is:

[0119] , ,

[0120] Step 4.2: Define and solve the maximum mean difference (MMD) distance between the evidence, and construct the MMD distance matrix between different pieces of evidence based on the solved MMD distance. Normalize the matrix, and then define and solve the MMD conflict coefficient based on the conflict coefficient and normalized MMD distance in Step 4.1, and construct the MMD conflict coefficient matrix.

[0121] The MMD distance matrix between different pieces of evidence is obtained by solving:

[0122]

[0123] Then find the maximum value in the MMD distance matrix. Then, the MMD distance matrix is ​​normalized, that is, each element of the MMD distance matrix is ​​divided by 1 / 2. The normalized MMD distance matrix is ​​obtained, and is expressed as:

[0124]

[0125] Then, based on the conflict coefficients from step 4.1 and the normalized MMD distance definition, the MMD conflict coefficients are solved. The resulting MMD conflict coefficient matrix is:

[0126]

[0127] Step 4.3: Normalize the MMD conflict coefficient matrix from Step 4.2, and define and solve the comprehensive influence matrix among the various pieces of evidence;

[0128] Find the maximum value in the MMD conflict coefficient matrix In step 4.2, each element of the MMD conflict coefficient matrix is ​​divided by the maximum value to normalize the MMD conflict coefficient matrix, which is represented as follows:

[0129]

[0130] Define the overall influence matrix T among the various pieces of evidence as follows:

[0131]

[0132] Step 4.4: Calculate the row sum and column sum of the comprehensive influence matrix in Step 4.3 to determine the importance of each piece of evidence in the fusion process;

[0133] The row sums and column sums of the comprehensive influence matrix were calculated, and the results are shown in Table 2.

[0134] Table 2. Row sum and column sum of the comprehensive influence matrix

[0135]

[0136] The importance of each piece of evidence in the fusion process is determined based on the row sums and column sums of the comprehensive influence matrix T. ID (Importance Degree), namely:

[0137] , ,

[0138] Step 4.5: Based on the importance obtained in Step 4.4, determine the relative importance of each piece of evidence in the fusion process;

[0139] Calculate the relative (Rel) importance of the evidence corresponding to each single-source signal in the fusion process, as follows:

[0140]

[0141]

[0142]

[0143] Step 5: Based on the absolute importance of the evidence corresponding to the single-source information determined in Step 3, and the relative importance of the evidence corresponding to the single-source information determined in Step 4, determine the overall importance of the evidence corresponding to the single-source information in the fusion process. The overall importance of each piece of evidence is as follows:

[0144]

[0145]

[0146]

[0147] Step Six: Based on the overall importance of the evidence corresponding to the single-source information obtained in Step Five, reassign the fusion weights to the evidence corresponding to each single-source information. The weights for each piece of evidence are as follows:

[0148] , ,

[0149] Based on the weights assigned to each piece of evidence, a new piece of evidence is obtained through a weighted average. This yields the basic probability allocation of the new evidence for the given actions and gestures, denoted as follows:

[0150] , , , ,

[0151] Step 7: Apply the DS evidence fusion rule to fuse the weighted new evidence obtained in Step 6. After fusion, the basic probability allocation results of the five gesture actions in the recognition framework of Step 1 are obtained. The gesture action corresponding to the maximum value of the basic probability allocation is the recognition result. Control the wheelchair behavior according to the recognition result, that is, realize intelligent wheelchair control based on multi-source information fusion.

[0152] Step 7.1: The weighted new evidence is fused twice, and the DS fusion rule is used to fuse the new evidence to obtain the basic probability allocation of the corresponding gesture, as follows:

[0153] , , , ,

[0154] Step 7.2: Traverse the basic probability distributions of all gesture recognition actions in Step 7.1, and determine the gesture action corresponding to the maximum value of the basic probability distribution. (Turning the palm to the left) is the wheelchair control gesture, meaning that this gesture controls the wheelchair to turn to the left.

[0155] The fusion recognition results obtained by applying DS fusion rules, literature methods, and existing patents are shown in Table 3. Figures 4 to 7 As shown.

[0156] Table 3. Fusion recognition results of different methods

[0157]

[0158] Analysis of Table 3 shows that the DS evidence theory, after fusing evidence, cannot recognize hand gestures, and therefore cannot control wheelchair behavior; in contrast, the hand gesture recognition result of the patent is a clenched fist, which controls the left turn of the wheelchair. Analysis of Table 1 in step one indicates that only… m 2. Supports the gesture of clenching a fist. The other two pieces of evidence do not support the occurrence of the clenched fist gesture, indicating that the fusion result is paradoxical and inconsistent with the actual situation.

[0159] Only the methods in the comparative literature and this patent identify the hand turning to the left, i.e., controlling the wheelchair to turn left. A careful comparison of the basic probability distributions of the methods in the comparative literature and this patent reveals that the method proposed in this patent... In contrast to the methods proposed in the literature As can be seen from the comparison, the proposed method has a basic probability distribution that is 0.2034 higher than that of the comparative literature. The method proposed in this patent has higher recognition accuracy and can achieve more precise intelligent control of wheelchairs.

[0160] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent wheelchair control based on multi-source information fusion, characterized in that: Comprising the following steps: Step one: preset a recognition framework composed of five gesture actions, the five gesture actions are respectively palm forward, palm backward, palm left, palm right and clenched fist; obtain three kinds of multi-source information for recognizing gesture actions, the three kinds of multi-source information are respectively electromyographic signals of arms, acceleration signals and angle signals; determine the gesture action category in the recognition framework according to the multi-source information; preset that the five gesture actions correspond to five behaviors of controlling the wheelchair respectively; construct the corresponding relationship between gesture actions and behaviors of controlling the wheelchair according to the recognition framework and the multi-source information, that is, palm forward represents controlling the wheelchair to move forward, palm backward represents controlling the wheelchair to move backward, palm left represents controlling the wheelchair to turn left, palm right represents controlling the wheelchair to turn right, and clenched fist represents controlling the wheelchair to stop; Step two: process the multi-source information obtained in step one to obtain the basic probability assignment result of the evidence corresponding to each single-source information; Step three: determine the absolute importance of the evidence corresponding to each single-source information in the fusion process according to the basic probability assignment result of the evidence corresponding to each single-source information obtained in step two; Step four: determine the relative importance of the evidence corresponding to each single-source information in the fusion process according to the basic probability assignment result of the evidence corresponding to each single-source information obtained in step two; Step five: determine the comprehensive importance of the evidence corresponding to each single-source information in the fusion process according to the absolute importance of the evidence corresponding to each single-source information in the fusion process determined in step three and the relative importance of the evidence corresponding to each single-source information in the fusion process determined in step four; Step six: reassign the fusion weight to the evidence corresponding to each single-source information according to the comprehensive importance of the evidence corresponding to each single-source information obtained in step five to obtain new evidence after weighting; Step seven: apply the DS evidence fusion rule to fuse the new evidence after weighting obtained in step six to obtain the basic probability assignment result of the five gesture actions in the recognition framework after fusion, and the gesture action corresponding to the maximum basic probability assignment value is the recognition result, and the behavior of the wheelchair is controlled according to the recognition result, that is, the intelligent wheelchair control based on multi-source information fusion is realized.

2. The intelligent wheelchair control method of multi-source information fusion according to claim 1, characterized in that: The implementation method of step one is: The preset recognition framework composed of five gesture actions is wherein represents palm turning forward, represents palm turning backward, represents palm turning left, represents palm turning right, represents clenched fist; the electromyographic signal S1 of the arm is acquired according to the electromyographic sensor, the acceleration signal S2 and the angle signal S3 are acquired according to the acceleration inclination sensor, and the electromyographic signal S1, the acceleration signal S2 and the angle signal S3 are taken as multi-source information for recognizing gesture actions; the corresponding relationship between gesture actions and wheelchair control behaviors is constructed according to the recognition framework and the multi-source information, that is, the five gesture actions correspond to five behaviors of controlling the wheelchair respectively, the palm turning forward represents controlling the wheelchair to move forward, the palm turning backward represents controlling the wheelchair to move backward, the palm turning left represents controlling the wheelchair to turn left, the palm turning right represents controlling the wheelchair to turn right, and the clenched fist represents controlling the wheelchair to stop.

3. The intelligent wheelchair control method of claim 2, wherein: The specific implementation method of step two is: Step 2.1: construct three back propagation neural network models, the input ends of the three neural network models are electromyographic signals S1, acceleration signals S2 and angle signals S3 respectively; the output layer function of each neural network model is a Softmax function, the output value range of the Softmax function is between 0 and 1, and the output category is the five gesture actions in the recognition framework of step one, which are respectively palm forward, palm backward, palm left, palm right and clenched fist; Step 2.2: apply the multi-source information for recognizing gesture actions to train the three neural network models constructed in step 2.1 to obtain the trained three neural network models; Step 2.3: Input the real-time multi-source information into the neural network model trained in Step 2.

2. The value of each output layer of the neural network model is the basic probability allocation result of the evidence corresponding to each single-source information. The basic probability allocation of the evidence corresponding to each single-source information is expressed as follows: m i ( ), m i ( ), m i ( ), m i ( ), m i ( ),in i =1, 2, 3, m 1 represents the evidence corresponding to the single-source electromyographic signal S1. m 2 represents the evidence corresponding to the single-source acceleration signal S2. m 3 represents the evidence corresponding to the single-source angle signal S3; m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability distribution, m i ( ) indicates in the evidence m i Lowering gesture The basic probability allocation.

4. The intelligent wheelchair control method of multi-source information fusion according to claim 3, characterized in that: The specific implementation method of step three is: Step 3.1: In the gesture recognition action framework the certainty Cer of each single-source signal corresponding evidence m i is represented as: （1） In formula (1), i = 1, 2, 3, respectively represent the myoelectric signal S1, the acceleration signal S2 and the angle signal S3 of the arm; k = 1, 2, 3, 4, 5, respectively represent the gesture actions of palm forward , palm backward , palm left , palm right and clenched fist ; Calculate the certainty of the evidence corresponding to each single-source signal according to formula (1); Step 3.2: Calculate the absolute importance of the evidence corresponding to each single-source signal in the fusion process according to formula (2) Abs : （2） In formula (2), i = 1, 2, 3, respectively represent the myoelectric signal S1, the acceleration signal S2 and the angle signal S3 of the arm; the higher the certainty of each single-source signal, the more important the evidence in the fusion process, and the higher the weight allocated in the fusion process.

5. The intelligent wheelchair control method of claim 4, wherein: The specific implementation method of step four is: Step 4.1: To give the conflict coefficient between two pieces of evidence conveniently, in the gesture recognition action framework , define m 1 and m 2 as two groups of evidence, m 1, the focal element of , m 2, the focal element of , then the conflict coefficient between two groups of evidence m 1 and m 2 is expressed as: （3） Solve the conflict coefficient between the evidences according to formula (3); Step 4.2: define and solve the maximum mean difference (MMD) distance between evidences, construct the MMD distance matrix between different evidences according to the solved MMD distance, normalize the MMD distance matrix, define and solve the MMD conflict coefficient according to the conflict coefficient of step 4.1 and the normalized MMD distance, and construct the MMD conflict coefficient matrix; evidence m 1 and m The MMD distance for 2 is expressed as: （4） In formula (4), p represents evidence m 1 probability vector of focus element, q represents evidence m 2 probability vector of focus element, K represents Gaussian radial basis function matrix, the solving formula of p and q is shown in formula (5), and the solving formula of K is shown in formula (6): （5） （6） In formula (6), k representing a Gaussian radial basis function, the solution formula is: （7） In formula (7), x and y represent a gesture recognition action framework elements in , will one-hot encoding, represent a wideband parameter, take ; The MMD distance matrix between different evidences is constructed and represented as: （8） Seeking the maximum value in the MMD distance matrix , each element of the MMD distance matrix is divided by , to obtain the normalized MMD distance matrix, as shown in equation (9): （9） The MMD conflict coefficient is defined and solved according to the conflict coefficient of step 4.1 and the normalized MMD distance, and represented as: （10） In formula (10), representative evidence m 1 and m 2 conflict coefficients, representative evidence m 1 and m 2 normalized MMD distances; The MMD conflict coefficient matrix is constructed according to formula (10): （11） Step 4.3: normalize the MMD conflict coefficient matrix in step 4.2, define and solve the comprehensive influence matrix between each evidence; Finding the maximum value in the MMD conflict coefficient matrix Step 4.2: Divide each element in the MMD conflict coefficient matrix by the maximum value, and normalize the MMD conflict coefficient matrix, denoted as: （12） The comprehensive influence matrix T between each evidence is defined and represented as: （13） In formula (13), I is a 3 by 3 unit matrix; Step 4.4: calculate the row sum and column sum of the comprehensive influence matrix in step 4.3 to determine the importance of each evidence in the fusion process; After the comprehensive influence matrix T of step 4.3 is obtained, the elements in T are expressed as t fg , f representing the row number, f = 1, 2, 3, g representing the column number, g = 1, 2, 3, let r f be the sum of the first f row in the comprehensive influence matrix T, representing the total influence degree of the first f row to other factors, r f The solution formula is: （14） Let c g The sum of the elements in the first g column of the matrix represents the total impact of the first g column on other factors, c g The solution formula is: （15） According to the rows of the combined influence matrix T r f and the columns of T c g determining the importance of each piece of evidence in the fusion process ID is represented as: （16） Step 4.5: Determine the relative importance of each piece of evidence in the fusion process ID , determining the relative importance of each piece of evidence in the fusion process; Calculate the relative evidence corresponding to each single-source signal during the fusion process. Rel Importance is expressed as: （17） In formula (15), i = 1, 2, 3, respectively represent the myoelectric signal S1, the acceleration signal S2 and the angle signal S3 of the arm; the higher the importance of each single-source signal, the more important the evidence in the fusion process, and the higher the weight allocated in the fusion process.

6. The intelligent wheelchair control method of multi-source information fusion according to claim 5, characterized in that: In step five, Based on the absolute importance of fusion obtained from step 3.2 and the relative importance obtained from step 4.5, the integrated importance of the evidence is determined according to equation (18) Com Importance: （18）。 7. The intelligent wheelchair control method of claim 6, wherein: The specific implementation method in step six is: According to the comprehensive importance of the evidence corresponding to the single-source information obtained in step five, the fusion weight of each evidence corresponding to the single-source information is redistributed, and the weight solving formula is: （19） According to the weight assigned to each evidence, the new evidence is obtained by weighted average, and represented as: （20） New evidence is obtained in the action gesture respectively for the basic probability assignment results, denoted as: （21） （22） （23） （24） （25）； According to formula (21) to formula (25), the fusion weight of each evidence corresponding to the single-source information is redistributed, and the weighted new evidence is obtained.

8. The intelligent wheelchair control method of claim 7, wherein: The implementation method in step seven is: Step 7.1: To give the DS evidence fusion rule conveniently, in the gesture recognition action framework , define m 1 and m 2 as two groups of evidence, m 1 has the focus element , m 2 has the focus element , then the DS fusion rule of the two groups of evidence is expressed as: （26） Fuse the weighted new evidence twice by itself, fuse the new evidence using the DS evidence fusion rule, and obtain the basic probability assignment result corresponding to the gesture action; Step 7.2: traverse the basic probability assignment result of all gesture recognition actions in step 7.1 to determine the gesture action corresponding to the maximum basic probability assignment value, and map the corresponding relationship between gesture actions and wheelchair control directions according to the gesture action and wheelchair control direction corresponding relationship constructed in step one to obtain the corresponding control wheelchair behavior, that is, to realize the intelligent wheelchair control based on multi-source information fusion.

Images