Artificial limb control method based on bio-electricity signals and electronic equipment

Abstract

The invention relates to the field of intelligent control, in particular to a bioelectrical signal-based artificial limb control method and electronic equipment, and the bioelectrical signal-based artificial limb control method comprises the following steps: collecting a first motion intention signal of a target user; performing data processing on the first motion intention signal to obtain first control information; adjusting the first control information according to the obtained target interaction information of the target user to obtain second control information; signal processing is carried out according to the second control information, target control information is obtained, and the target control information is used for controlling the artificial limb of the target user. According to the method, the motion intention signal of the target user is collected, data processing and dynamic adjustment are carried out, the target control information used for controlling the artificial limb is generated, it is ensured that the artificial limb can accurately reflect the real motion intention of the user, the real-time interaction requirement is met, and the operation precision and the user experience are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, specifically to a prosthetic limb control method and electronic device based on bioelectric signals. Background Technology

[0002] Electroencephalography (EEG) is a technique that records the synchronous activity of a group of neurons in the brain using scalp electrodes. It can reflect motor imagery and intentions and is a commonly used non-invasive brain-computer interface (BCI) method. BCIs directly control external devices by acquiring brain signals. EEG is widely used in the BCI field due to its safety, cost-effectiveness, and ease of use. Traditional prosthetic control mainly relies on electromyography (EMG), which controls prosthetic movements through electrical activity generated by muscle contractions. However, surface EMG signals suffer from instability, such as changes in skin impedance and electrode misalignment, affecting the user's long-term experience.

[0003] In recent years, prosthetic control technology has pursued flexible and precise control with multiple degrees of freedom, and myokinetic interfaces have emerged as a new prosthetic control method. This interface achieves high control accuracy by implanting magnets within muscles and using external sensors to track the magnets' displacement. However, this technology also has limitations such as invasiveness, poor scalability, signal stability issues, and single-channel information. Implanting magnets requires surgery, increasing the risk of infection and tissue damage, and hinders rapid deployment among different patients. In the long term, the magnets may migrate within the body, affecting signal stability. Furthermore, the myokinetic interface only utilizes muscle displacement signals and cannot obtain higher-level motion intention information.

[0004] Current brain-computer interfaces are mostly used for static or simple motion control, lacking the ability to control complex movements in real time. Overall, existing technologies lack comprehensive support for natural language processing, intent recognition, and multimodal input, resulting in a less intuitive and intelligent interactive experience. Summary of the Invention

[0005] One objective of this invention is to provide a prosthetic limb control method and electronic device based on bioelectric signals, which addresses the technical problems of high invasiveness, poor scalability, and unstable and single-signal characteristics in traditional prosthetic limb control methods.

[0006] In a first aspect, embodiments of the present invention provide a prosthetic limb control method based on bioelectrical signals, applied to a prosthetic limb control system, the method comprising: Collect the target user's initial movement intention signal; The first motion intention signal is processed to obtain the first control information; The first control information is adjusted based on the target user's target interaction information to obtain the second control information; Signal processing is performed based on the second control information to obtain target control information, which is used to control the prosthesis of the target user.

[0007] In a second aspect, an electronic device is provided, the electronic device including a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor, when executing the one or more computer programs, causing the electronic device to implement the bioelectric signal-based prosthetic control method as described in the first aspect.

[0008] In a third aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the bioelectric signal-based prosthetic control method as described in the first aspect.

[0009] In the embodiments of the above-mentioned prosthetic limb control method, electronic device and storage medium based on bioelectric signals, a first motion intention signal of the target user is first acquired; then, the first motion intention signal is processed to obtain first control information; then, the first control information is adjusted according to the target user's target interaction information to obtain second control information; finally, the second control information is processed to obtain target control information, which is used to control the target user's prosthetic limb. This embodiment uses a first motion intention signal to ensure that the acquired signal accurately reflects the user's true motion intention. Further data processing of the first motion intention signal generates first control information, which can initially extract and convert the user's motion intention into operable control commands. Furthermore, by introducing target interaction information, the first control information obtained solely based on the first motion intention signal is adjusted, thereby filtering or correcting noise or ambiguity in the first motion intention signal. This makes the generated second control information and target control information more accurately reflect the user's true intention, better adapting to the user's real-time needs and interactive environment. This allows for precise control of the target user's prosthesis, improving the prosthesis's operational accuracy and user experience, enhancing response flexibility and adaptability. Moreover, the introduction of target interaction information enables the system to integrate user intentions from more dimensions, thereby improving the system's ability to operate stably in complex or disruptive environments. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the structure of a prosthetic control system according to an embodiment of the present invention; Figure 2 This is a software architecture diagram of a prosthetic control system in an embodiment of the present invention; Figure 3 This is a flowchart illustrating a prosthetic limb control method based on bioelectric signals in an embodiment of the present invention. Figure 4 This is a schematic diagram of the structure of a prosthetic control device based on bioelectric signals in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0013] It should be noted that, unless otherwise specified, the various features in the embodiments of this invention can be combined with each other, all of which are within the protection scope of this invention. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this invention do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0014] Please see Figure 1 , Figure 1 This is a schematic diagram of the prosthetic limb control system. Figure 1 In the prosthetic control system 10, there is an electronic device 20, which includes at least one processor 201 and a memory 202.

[0015] Among them, electronic devices 20 can be multi-channel EEG head-mounted devices (EEG electrode caps or headbands), VR headsets, eye-tracking devices, microphones, servers, network devices (such as routers and switches), programmable logic controllers (PLCs), monitoring and management devices, etc., and are not limited to one specific device.

[0016] The processor 201 is configured to support the prosthetic control system in performing the corresponding functions of the bioelectric signal-based prosthetic control method in the above-described method embodiments. The processor 201 can be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0017] Specifically, the processor 201 may include a transmitting card, a receiving card, and a driver chip.

[0018] At the software level, such as Figure 2 As shown, Figure 2 The software architecture diagram of the prosthetic limb control system, as shown in Figure 2, includes an EEG acquisition module 30, a signal preprocessing and decoding module 40, a multimodal interaction module 50 (including VR headset, eye tracking, and voice control sub-modules), a motion mapping control module 60, and a prosthetic limb feedback module 70. The structure and function of each module are as follows, and they work together to complete the hand motion control task.

[0019] The EEG acquisition module 30 is used to acquire weak EEG signals related to motor intentions in real time from the surface of the user's scalp, especially in the motor cortex areas (such as near C3, C4, and CZ). These signals are traces of neural activity generated when the user imagines hand movements. It is the original signal source for the entire control process. The raw, unprocessed EEG data stream is then passed to the signal preprocessing and decoding module 40.

[0020] The signal preprocessing and decoding module 40 includes sub-modules for filtering, denoising, feature extraction, and pattern recognition / classification, and may use machine learning models (such as SVM, LSTM, CNN, etc.) for decoding. Specifically, it receives raw EEG data from the EEG acquisition module 30, processes and decodes it, outputs first control information, and may transmit some decoding results or state information to the multimodal interaction module.

[0021] Specifically, the signal preprocessing and decoding module 40 performs filtering (removing power frequency interference, high-frequency noise, and low-frequency drift), artifact removal (such as eye movement and muscle activity interference), and rereference operations on the raw data from the EEG acquisition module 30 to improve signal quality. It extracts features related to motor intention (such as power changes in specific frequency bands, i.e., changes in sensored motor rhythm), and then uses a trained decoding model to map these features into specific, preliminary intention commands (e.g., "Imagine left hand bending" -> "Type 1 intention", "Imagine right hand extending" -> "Type 2 intention"), outputting the first control information.

[0022] The multimodal interaction module 50 includes a VR headset submodule, an eye-tracking submodule, and a voice control submodule. It acquires and analyzes data from hardware such as the VR headset, eye-tracking device, and microphone. The analyzed target interaction information (such as "the user looked at the 'stop' button" or "the user said 'grip'") is then passed to the motion mapping control module 60 to adjust the first control information.

[0023] Specifically, the multimodal interaction module 50 is responsible for collecting and processing non-EEG signal interaction information from the user. This information can supplement, modify, or replace the motor intention. That is, it adjusts the first control information based on the target interaction information. For example, if the EEG decodes "bend your finger," but eye tracking shows that the user is looking at a virtual button that says "open your hand," the system may prioritize the eye tracking information and generate the "open your hand" command, thus generating the second control information. For example, the user can confirm or change the intention decoded by the EEG by looking at a virtual button; or, when the EEG signal quality is poor, the user can rely entirely on voice commands to control the prosthesis.

[0024] Specifically, the motion mapping control module 60 maps the adjusted second control information (such as "bend finger", "open hand", "stop") to specific prosthetic motor control parameters (such as the rotation angle, speed, force, etc. of each finger joint) to generate the final target control information.

[0025] Specifically, the prosthetic feedback module 70 is responsible for acquiring data from sensors (such as force sensors, tactile sensors, and position sensors) installed on the prosthesis and feeding this information back to the user to enhance control accuracy and perception.

[0026] The memory 202 is used to store program code and common storage components, including graph databases, vector databases, MySQL, Redis, and MQ, to ensure data storage and management. The memory 202 may include volatile memory (VM), such as random access memory (RAM); it may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may include a combination of the above types of memory.

[0027] Therefore, through this multi-module, multi-modal collaboration, the system can more robustly, flexibly, and naturally understand the user's true intentions and precisely control the prosthesis to complete complex hand movement tasks.

[0028] See Figure 3 , Figure 3 A flowchart illustrating a bioelectric signal-based prosthetic control method provided in an embodiment of the present invention includes the following steps: S10, Collect the target user's first motion intention signal.

[0029] The aforementioned first motor intention signal is the electrical activity of nerves and / or muscles that accompanies the user's intention to move. The first motor intention signal is generated when the user "imagines" moving their arm or fingers, or prepares to actually move.

[0030] The first motor intention signal may include, but is not limited to, electroencephalogram (EEG) and electromyography (EMG).

[0031] Electroencephalography (EEG) is a non-invasive method for monitoring brain activity. It uses electrodes placed on the scalp to record the weak bioelectrical signals generated when groups of neurons in the cerebral cortex are synchronously active.

[0032] Electromyography (EMG) records the electrical activity of muscles during contraction and relaxation. Muscle contraction is triggered by motor nerve signals, so EMG can reflect motor intentions to some extent, especially signals from residual limb muscles.

[0033] Before acquiring the target user's initial motor intention signal, the target user must wear an EEG headgear (such as an electrode cap or headband). Technicians, using the international 10-20 system, apply conductive gel to key areas such as C3, C4, and CZ, as well as the reference and ground electrodes, and install Ag / AgCl electrodes, ensuring good contact. Impedance checks may also be performed to ensure that the impedance of each channel is within a reasonable range (e.g., <5kΩ). If EMG is used, an sEMG electrode array is installed on the corresponding muscle surface (or implantation site) of the residual limb, again ensuring good contact.

[0034] In the process of acquiring the target user's first motor intention signal, the system includes a high-input-impedance EEG amplifier and an analog-to-digital converter (ADC) to amplify and digitize the weak EEG voltage signal. When the user "imagines" moving their arm or fingers (e.g., imagining clenching their left fist), specific EEG signal changes occur in the relevant areas of the motor cortex (e.g., decreased SMR in the C3 region). Simultaneously, if the user has a residual limb, the imagined movement may trigger weak muscle contractions in the residual limb, generating EMG signals. These weak bioelectrical signals are captured by electrodes and transmitted through wires to the high-input-impedance amplifier. The amplifier amplifies the signals, and the ADC converts the analog signals into digital signals. The system records the multi-channel digital signals (first motor intention signal) in real time at a set sampling rate (e.g., 500Hz).

[0035] For example, an amputee user wants to control his prosthetic limb to grasp a cup. He is told to "imagine grasping it with your left hand." While he is imagining this, the EEG acquisition module records a significant decrease in SMR signal intensity (ERD phenomenon) around 10Hz near the C3 electrode on the left side of his head, while the signal in the C4 region on the right side may increase (ERS phenomenon, or remain unchanged). Simultaneously, RP signals related to pre-movement preparation may also be recorded; this pattern of change represents the first intention to move signal.

[0036] For example, if the same user uses the remaining muscles in the stump to control the movement, he may slightly contract a muscle near the elbow joint. The sEMG acquisition module will detect the weak electrical signal pattern generated by this muscle, which is associated with the "grabbing" action he wants to perform.

[0037] Optionally, eye-tracking technology can be used for users with high-level amputations, insufficient muscle signals in the residual limb, or poor EEG signal quality. For example, users can select prosthetic movements by looking at different icons on the screen, or confirm commands through specific eye movement patterns (such as gaze + blink). Head posture (detected via IMU sensors) can be used for simpler controls, such as nodding to switch modes and shaking to cancel operations.

[0038] As can be seen, this embodiment can obtain stable, reliable and accurate bioelectrical signal input that reflects the user's movement intention by collecting signals from the target user.

[0039] S20. Perform data processing on the first motion intention signal to obtain first control information.

[0040] Data processing may include, but is not limited to, filtering, noise reduction, and decoding.

[0041] The first control information is the preliminary intent representation obtained after decoding. The first control information is usually a discrete command (such as "left", "right", "grab", "release") or a low-dimensional continuous value (such as a single numerical value representing the intent to go left or right).

[0042] For example, if the system recognizes an intention to perform a left-hand action, the first control information output is "left". This information represents the user's most direct interpretation of the intention based on a single signal source (such as EEG), but it may not be accurate enough or needs to be optimized by combining other information.

[0043] The specific implementation process of S20 can be referred to the descriptions of S201-S203, and will not be repeated here.

[0044] As can be seen, in this embodiment, the original first motion intention signal is converted into control information corresponding to the actual action through data processing, which improves the accuracy and response speed of the prosthetic control method based on bioelectric signals.

[0045] S30. Adjust the first control information according to the target user's target interaction information to obtain the second control information.

[0046] Among them, target interaction information refers to contextual information, specific goals, or instructions obtained through multimodal interaction devices such as VR, eye tracking, and voice, which supplement or clarify the user's intentions.

[0047] The second control information refers to the more precise, specific, and directly usable instructions that drive the prosthesis to perform actions, generated by integrating the first control information and the target interaction information.

[0048] The specific implementation process of S30 can be found in the descriptions of S301-S303, and will not be repeated here.

[0049] As can be seen, the adjustment process in this embodiment can adapt to the user's personalized needs, improve the system's response accuracy, enhance the user experience, and make the prosthetic operation more natural and intuitive.

[0050] S40. Signal processing is performed based on the second control information to obtain target control information, which is used to control the prosthesis of the target user.

[0051] Signal processing refers to a series of mathematical operations, transformations, and mapping operations performed on the second control information, with the aim of converting it into a form suitable for driving the prosthetic actuator.

[0052] Among them, the target control information is the output of the signal processing stage. It is a specific, formatted signal or dataset that contains all the details required to drive the prosthesis to perform a specific action.

[0053] For example, for a prosthetic hand, it might include the target angle of each finger joint, the desired speed of movement, and the required torque or current intensity; for a prosthetic leg, it might include the target angles and angular velocities of the hip, knee, and ankle joints. This information is directly used to control motors, hydraulic systems, or other drive components.

[0054] As can be seen, this embodiment uses signal processing techniques such as mapping, transformation, and formatting to ensure that the user's well-thought-out and interactively confirmed intentions can be accurately transmitted to the physical drive system of the prosthesis, thereby achieving smooth and accurate human-computer interaction and prosthesis control.

[0055] This embodiment uses a first motion intention signal to ensure that the acquired signal accurately reflects the user's true motion intention. Further data processing of the first motion intention signal generates first control information, which can initially extract and convert the user's motion intention into operable control commands. Furthermore, by introducing target interaction information, the first control information obtained solely based on the first motion intention signal is adjusted, thereby filtering or correcting noise or ambiguity in the first motion intention signal. This makes the generated second control information and target control information more accurately reflect the user's true intention, better adapting to the user's real-time needs and interactive environment. This allows for precise control of the target user's prosthesis, improving the prosthesis's operational accuracy and user experience, enhancing response flexibility and adaptability. Moreover, the introduction of target interaction information enables the system to integrate user intentions from more dimensions, thereby improving the system's ability to operate stably in complex or disruptive environments.

[0056] S201. In one embodiment, the step of processing the first motion intention signal to obtain first control information includes: filtering the first motion intention signal to obtain filtered first motion intention information; denoising the filtered first motion intention information to obtain second motion intention information; and decoding the second motion intention information to obtain first control information.

[0057] Filtering refers to removing unwanted frequency components from a signal. Bioelectrical signals typically contain information relevant to movement intent only within a specific frequency range, while other frequency ranges may contain noise or irrelevant physiological signals.

[0058] Specifically, the filtering process can include two steps: one is to use a 0.5~50Hz bandpass filter to retain the main EEG frequency band and remove DC drift and high-frequency electromyography noise; the other is to apply an adaptive algorithm in parallel to suppress power frequency interference.

[0059] Specifically, while performing bandpass filtering, an adaptive algorithm is activated to specifically suppress power frequency interference (e.g., 50Hz in China, 60Hz in North America). The adaptive algorithm is typically based on the principles of adaptive filtering, such as the Least Mean Square (LMS) algorithm. It estimates the characteristics of the interference signal (such as frequency, amplitude, and phase) and generates an inverse "out-of-phase" signal, subtracting it from the original signal to cancel out the interference. Because the power grid frequency may fluctuate slightly, the adaptive algorithm can dynamically track and suppress interference.

[0060] Among them, the filtered first motion intention information refers to the version of the signal that has been filtered to remove low-frequency drift and high-frequency electromyography interference, and retains the signal in the range of 0.5-50Hz. This signal is "cleaner" than the original signal, but may still contain other types of noise (such as power line interference, electrooculography artifacts, etc.).

[0061] Denoising refers to removing non-targeted, random, or systematic interference from a signal. Denoising includes removing specific artifacts, such as electrooculogram (EOG) artifacts and electrocardiogram (ECG) artifacts.

[0062] Specifically, denoising techniques also include Independent Component Analysis (ICA), which can isolate and remove independent signal sources caused by eye movements or heart activity; or using methods such as wavelet transform for time-frequency domain denoising; and sometimes combining hardware-level filtering (such as the 50Hz notch filter mentioned in the disclosure document).

[0063] For example, when a user blinks, obvious artifacts are generated in the EEG signal. By analyzing the signals of specially placed electrodes that record the potential eye movements (EOG), or by using the ICA algorithm, these artifact components caused by blinking can be identified and removed, thereby purifying the EEG signal related to movement intention.

[0064] The second motion intent information refers to the motion intent signal that has been further purified after noise reduction processing. The second motion intent information is closer to the pure intent signal of the target user than the filtered signal, but it has not yet been converted into specific control commands.

[0065] Decoding refers to mapping the purified second motion intent information (usually frequency band power, time domain waveform characteristics, etc.) into specific intent categories or instructions that can be understood by the control system.

[0066] Decoding typically involves feature extraction (extracting statistical properties related to intent from the signal, such as power spectral density, event-related synchronization / desynchronization ERD / ERS) and pattern recognition (using machine learning algorithms, such as linear discriminant analysis (LDA), support vector machine (SVM), artificial neural network (ANN), etc., to match the extracted features with predefined intent categories).

[0067] Specifically, decoding is typically performed using machine learning models (such as Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), neural networks, etc.). These machine learning models need to be trained on the user's signal data beforehand to learn how to associate signal features with specific motion intentions.

[0068] The specific process of decoding the second running intent information to obtain the first control information is described in S202, and will not be repeated here.

[0069] For example, suppose the original EEG signal contains, in addition to the weak beta wave (approximately 20Hz) generated at the C3 electrode when the user imagines raising their hand, a slowly rising DC drift (<0.1Hz, possibly caused by electrode dryness) and a high-frequency spike (>100Hz) caused by the user's slight blinking. After a 0.5-50Hz bandpass filter, the DC drift is significantly reduced, and the high-frequency blinking spike is removed. Although the 50Hz power line interference may still be present (assuming the filter edge is not steep enough or the interference is strong), the parallel-running adaptive algorithm will detect this 50Hz sine wave and generate an inverted 50Hz signal to cancel it out. The final filtered signal more clearly highlights the 20Hz beta wave activity. Although the filtered signal removes the drift and high-frequency electromyography, it may still contain small spikes caused by the user's involuntary blinking (electroocular artifacts). Further denoising is performed by observing the signal changes at the forehead electrode and removing them from the signals of all channels using ICA or other methods. The resulting second motion intention information will have a clearer and more stable 20Hz beta wave activity, almost unaffected by interference such as blinking. The denoised second motion intention information (mainly timing data from the C3 electrode) is divided into 200ms windows. Within each window, the average power in the 13-30Hz (beta wave) band is calculated. If this power value is significantly lower than the baseline power value in the user's relaxed state, the decoding model (assuming a trained LDA model) determines that the user is imagining a left-hand movement and outputs the first control information: "Move left." The "Move left" instruction represents the system's interpretation of the user's intention.

[0070] As can be seen, in this embodiment, interference outside the main frequency band is first removed by filtering, then specific artifacts and residual interference are removed by denoising, and finally the purified signal is converted into a preliminary intention representation by decoding. This gradually improves the signal quality and the clarity of the intention, ensuring high signal quality and accuracy. This enables the system to accurately respond to the user's motion intention, improving the control precision of the device and the user experience.

[0071] S202. In one embodiment, the step of decoding the second running intention information to obtain the first control information includes: extracting features from the second running intention information to obtain a feature vector corresponding to each time window in the second running intention information; performing feature fusion processing on the feature vector corresponding to each time window to obtain a target feature vector; predicting the target feature vector according to a preset classification model to obtain the predicted intention category and corresponding confidence level output by the preset classification model; and determining the first control information when the confidence level is greater than or equal to a preset confidence level, wherein the first control information includes the predicted intention category and the corresponding confidence level.

[0072] In the process of feature extraction of the second running intention information, all features extracted within each time window (e.g., C3 wave power, C4 wave power, event-related synchronization / desynchronization ERD / ERS features, etc.) are organized into a vector, which represents the "digital fingerprint" of the user's movement intention within that time window, i.e., the feature vector.

[0073] Feature fusion refers to the process of combining features from different sources, types, or time points to form a more comprehensive and discriminative integrated feature representation. The purpose of feature fusion is to improve the robustness and accuracy of the model by utilizing multi-source information.

[0074] The feature fusion processing methods may include, but are not limited to: concatenated fusion, weighted summation, and model-based fusion.

[0075] Specifically, concatenation fusion simply connects different feature vectors together to form a longer vector. For example, it connects the feature vectors of the left hand and the feature vectors of the right hand; weighted summation assigns weights to each feature based on its importance and then sums them; model-based fusion uses a specific machine learning model (such as a neural network) to learn how to best combine these features.

[0076] The target feature vector refers to the feature vector after fusion processing, which better represents the user's current motion intention state. The target feature vector will be used as input for subsequent model predictions.

[0077] In this context, a pre-trained classification model refers to a machine learning model that is pre-trained to classify input feature vectors into predefined categories. Common classification models include Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), and neural networks.

[0078] Among them, the predicted intent category refers to the user's most likely movement intent determined by the model based on the input feature vector, such as "move left", "move right", "grab", "relax", etc.

[0079] Confidence level refers to the degree to which a model is certain of its prediction results. It is usually a value between 0 and 1. The closer it is to 1, the more confident the model is in its prediction.

[0080] The pre-set reliability refers to a pre-set threshold (e.g., 0.7 or 70%) used to judge the reliability of the model's predictions.

[0081] In practice, the target feature vector is input into a pre-defined classification model. Based on the learned patterns, the model outputs one or more possible intent categories, along with the confidence score for each category. For example, the model might output: "Shift left" (confidence score 0.85), "Grab" (confidence score 0.05), and "Relax" (confidence score 0.10).

[0082] Furthermore, check whether the highest confidence value in the predicted category is greater than or equal to the preset confidence threshold. For example, if the preset confidence is 0.8 and the confidence of "left shift" is 0.85, then the condition is met.

[0083] Furthermore, if the highest confidence level meets the threshold condition, the predicted intent category is determined to be the user's true intent, and this is used as the first control information. Simultaneously, the corresponding confidence level is also output for reference by subsequent modules. For example, the first control information is determined as: "Shift left" (confidence level 0.85).

[0084] In this context, a pre-trained regression model refers to a machine learning model that is trained in advance to predict continuous numerical values. Common regression models include linear regression, support vector regression (SVR), and neural network regression.

[0085] The target value refers to a continuous value predicted by the regression model, which can be directly mapped to a certain continuous control dimension of the prosthesis.

[0086] Optionally, except when the confidence level is greater than or equal to a preset confidence level, if the highest confidence level of the preset classification model is lower than the preset confidence level threshold (e.g., the highest confidence level is only 0.65), it indicates that the model is not sufficiently certain about the user's intent, the user's intent may be ambiguous, or the signal quality may be poor. Therefore, the target feature vector is input into a preset regression model, which is trained to predict a continuous value rather than a discrete category.

[0087] Furthermore, the pre-defined regression model outputs a target value based on the input feature vector. This target value can represent: the angle change of the prosthetic joint, the opening and closing degree of the prosthetic end effector, the speed or force of prosthetic movement, etc. For example, the predicted value might be "-15 degrees," indicating that the prosthesis should rotate 15 degrees to the left; or "0.7," indicating that the prosthesis should perform a 70% grasping force. Therefore, this predicted target value is directly used as the first control information. In this case, the control information is no longer a discrete intention label, but an instruction that can directly drive the prosthesis to perform fine, continuous movements.

[0088] As can be seen, in this embodiment, user intent is extracted through meticulous feature engineering and intelligent model selection; the classification model is used to handle explicit and discrete intents, while the regression model serves as a supplement to handle fuzzy intents or for continuous control. Furthermore, the robustness of the system is increased through a confidence mechanism to ensure that different strategies are adopted when there is uncertainty.

[0089] S203. In one embodiment, after performing feature fusion processing on the feature vectors corresponding to each time window to obtain a target feature vector, the method further includes: inputting the target feature vector into a preset regression model for prediction to obtain a target value, wherein the target value is the first control information.

[0090] Optionally, the step of decoding the second running intention information to obtain the first control information includes: extracting features from the second running intention information to obtain a feature vector corresponding to each time window in the second running intention information; performing feature fusion processing on the feature vector corresponding to each time window to obtain a target feature vector; and inputting the target feature vector into a preset regression model for prediction to obtain a target value, wherein the target value is the first control information.

[0091] The target value refers to a continuous value predicted by the regression model, which can be directly mapped to a certain continuous control dimension of the prosthesis.

[0092] The pre-defined regression model outputs a target value based on the input feature vector. This target value can represent: the angle change of the prosthetic joint, the opening and closing degree of the prosthetic end effector, the speed or force of prosthetic movement, etc. For example, the predicted value might be "-15 degrees," indicating that the prosthesis should rotate 15 degrees to the left; or "0.7," indicating that the prosthesis should perform a 70% grasping force. Therefore, this predicted target value is directly used as the first control information. In this case, the control information is no longer a discrete intention label, but an instruction that can directly drive the prosthesis to perform fine, continuous movements.

[0093] As can be seen, in this embodiment, the target feature vector is input into a preset regression model for prediction, and the target value is obtained as the first control information. This avoids the errors and uncertainties that may exist in traditional control methods, and achieves precise control and high adaptability.

[0094] S301. In one embodiment, before adjusting the first control information according to the acquired target interaction information of the target user to obtain the second control information, the method further includes: acquiring first interaction information, the first interaction information being obtained by a first interaction device in the prosthetic control system; and / or acquiring second interaction information, the second interaction information being obtained by a second interaction device in the prosthetic control system; and / or acquiring third interaction information, the second interaction information being obtained by a second interaction device in the prosthetic control system; and generating target interaction information of the target user according to the first interaction information and / or the second interaction information and / or the third interaction information.

[0095] Interactive information refers to information generated by the interaction device between the user and the prosthetic control system, which is used to guide the movement of the prosthesis.

[0096] The first interactive device can be a VR head-mounted display device; the second interactive device can be an eye-tracking device within the VR head-mounted display device (or an independent infrared eye tracker can be selected); and the third interactive device can be a device that collects data via a microphone installed on the head-mounted display or prosthesis.

[0097] The first interactive information refers to the visual feedback, interactive data, and prompts obtained from the user in the virtual environment through the first interactive device (VR head-mounted display). During the acquisition of this first interactive information, a continuous data stream is obtained from the VR head-mounted display, including displayed content and environmental perception results. For example, the computer vision algorithm built into the head-mounted display analyzes the camera feed in real time, identifies objects within the user's field of vision (such as cups or books), and uses these identification results as the first interactive information.

[0098] The second interactive information refers to the user's gaze direction and gaze target obtained through the second interactive device (eye-tracking device).

[0099] For example, even if the EEG signal contains the intention to grasp both the left and right objects, if eye tracking shows that the user keeps looking at the right object for more than 2 seconds, then the second interaction information clearly points to the right object.

[0100] Among them, third interactive information refers to the acquisition of user voice commands or voice content through a third interactive device (microphone acquisition device).

[0101] For example, a user might say commands such as "grab the cup," "stop," or "switch mode." The voice recognition result serves as a third form of interactive information.

[0102] In the process of generating target interaction information for the target user based on the first interaction information and / or the second interaction information and / or the third interaction information, a preset fusion algorithm (such as weighted decision, rule logic, Bayesian network, etc.) is used to comprehensively judge the first interaction information, the second interaction information, and the third interaction information, and output the target interaction information. The target interaction information is typically a structured instruction that includes the action type, target object, execution parameters, etc. For example: "Grab the water cup marked in blue, located 0.5 meters in front of the user, with moderate grasping force."

[0103] Specifically, the above rule logic may include, but is not limited to: Rule 1: If a clear voice command is detected (such as "grab the cup"), the command is executed first, ignoring the current EEG and eye movement information (unless the voice command is "cancel"); Rule 2: If there is no voice command, but the eye movement continuously focuses on an object for more than a threshold time (such as 2 seconds), and the EEG decoding result is "grab", then the target interaction information "grab [the object being looked at]" is generated; Rule 3: If there is no voice command, and the eye movement does not have a clear focus, but the EEG decoding result has a high confidence level (such as >90%) showing "grab", then the target interaction information "execute the grab action" is generated (this may require further confirmation from the user or be combined with the default target in the VR environment); Rule 4: If all information is vague or has low confidence, the system may prompt the user to provide more specific instructions (such as "please say the object you want to grab" or "please look at the target").

[0104] Alternatively, if a VR headset and built-in eye tracking are not used, the system's multimodal interaction can be achieved in other ways. For example, augmented reality (AR) can be used instead of VR, that is, transparent AR glasses can be used instead of a closed VR headset. This can provide users with information prompts and eye tracking functions without obstructing their real field of vision, making it more suitable for everyday use. Alternatively, a head-mounted display device can be completely avoided, and a virtual training environment can be presented on an environment projected onto a tablet / computer, combined with an external eye tracker to achieve similar functions. Alternatively, in the absence of an eye tracker, a camera can be mounted on the prosthesis, and computer vision algorithms can be used to infer the target by recognizing the user's facial orientation or the direction of their remaining limbs. For example, by detecting the user's head orientation and the direction of their remaining fingers, the object they are paying attention to can be inferred, essentially replacing direct eye tracking with external visual cues. Although the accuracy and timeliness are not as good as eye trackers, it is feasible under certain conditions.

[0105] Optionally, when voice input is inconvenient, alternative input methods can be introduced to allow users to issue discrete commands. For example, several touch buttons can be installed on the prosthesis or armrest, allowing the user to switch modes by touching different buttons with their unaffected hand. Alternatively, the movement of the residual limb can be utilized: some commercial prostheses allow users to quickly flex and extend the muscles of the residual limb twice to switch modes, which is essentially a "gesture" command. Similarly, this system can be set up with similar triggers, such as the user blinking twice quickly or opening their mouth to make a certain expression, which the camera recognizes as a specific command to replace the voice function.

[0106] As can be seen, this embodiment integrates multiple interaction methods such as VR, eye tracking, and voice, and uses a fusion algorithm to integrate scattered and potentially ambiguous information into clear and specific high-level instructions, which greatly improves the naturalness, accuracy, and reliability of prosthetic control.

[0107] S302. In one embodiment, adjusting the first control information based on the target interaction information of the target user to obtain the second control information includes: parsing the target interaction information to obtain at least one type of interaction information; sequentially fusing and judging each type of interaction information in the first control information and the at least one type of interaction information to obtain at least one type of third control information; and integrating each type of third control information in the at least one type of third control information to obtain the second control information.

[0108] Among them, at least one type of interactive information refers to the specific information type parsed from the target interactive information, such as visual, speech, or attention information, that is, the individual elements or dimensions that constitute the target interactive information after parsing.

[0109] For example, from the target interaction information "grab the red cup", we can parse the following: Interaction information A: Action type - grab; Interaction information B: Target object - red cup; Interaction information C: Priority - high (if confirmed by voice).

[0110] The third control information is an intermediate control signal generated through fusion judgment, which further refines and adjusts the user's operation instructions.

[0111] In the process of sequentially fusing and judging each type of interaction information among the first control information and the at least one type of interaction information to obtain at least one third control information, each type of parsed interaction information is individually combined with the first control information and evaluated or decided according to preset rules or algorithms to generate a temporary control suggestion adjusted for the specific interaction information. For each type of parsed interaction information, a fusion judgment will be performed once.

[0112] Specifically, the process involves retrieving the first control information (e.g., "execute the grasping action"); retrieving the currently processed interaction information (e.g., "focus on target - red cup"); and generating a third control information based on preset rules (e.g., "if the target is a red cup, then set the target coordinates of the grasping action to the coordinates of the red cup") or preset algorithms (e.g., weighted calculation, Bayesian inference, considering the confidence or importance of the interaction information). This third control information is the result of incorporating the influence of the current interaction information into the first control information. For example, the third control information after incorporating "focus on target - red cup" might be: "execute the grasping action on the object at coordinates (1.2, 0.5, 0.8)". If "voice command - pick up" is also incorporated, the third control information might also include the instruction "pick up and raise it to a certain height".

[0113] Integration refers to the process of merging multiple third-party control information into a final, unified second-party control information.

[0114] For example, integrating the third control information 1, 2, and 3: Third control information 1: "Grab and lift"; Third control information 2: "Grab the red cup (coordinates already set)"; Third control information 3: "Grab the red cup, no path planning adjustment required." During integration, the action type "grab and lift," the target object "red cup (coordinates already set)," and the path information "no path planning adjustment required" are combined. Therefore, the second control information is "Execute the grabbing action on the red cup at coordinates (1.2, 0.5, 0.8), grab and lift, no additional path planning required." For a detailed description of how the second control information is obtained by integrating each of the at least one third control information, please refer to S303; the description will not be repeated here.

[0115] As can be seen, this embodiment can precisely process complex interactive information from different modalities through parsing, multi-round fusion judgment and integration, and deeply combine it with EEG intention to ensure that the system makes the correct response in complex interactive environments, that is, to generate second control information that conforms to the user's true intention and the current situation, thereby improving user experience and system efficiency.

[0116] S303. In one embodiment, integrating each of the at least one third control information to obtain the second control information includes: determining the priority corresponding to each third control information according to a preset fusion rule; determining the second control information according to the priority, wherein the second control information is the third control information with the highest priority.

[0117] Among them, preset fusion rules refer to a set of pre-defined logic or conditions used to guide the system on how to evaluate and compare control information from different sources or types, and to assign priorities to them. Preset fusion rules can be customized according to application scenarios, user preferences, or task requirements, and are not limited to a single specific rule here.

[0118] Priority refers to the numerical value or level of importance or applicability of a third piece of control information relative to other information. Information with higher priority is more likely to be selected as the final second piece of control information.

[0119] For example, suppose the following two third control messages are obtained through the aforementioned process: Third control message A: Based on the fusion of EEG intent and visual information, grasp the red cup on the table in front; Third control message B: Based on voice input, execute "Stop all actions". Assume the preset fusion rules are: Rule 1: All voice commands containing "Stop" or "Emergency" instructions have the highest priority; Rule 2: In the absence of an emergency instruction, intents combined with visual information are given priority. Therefore, according to Rule 1, third control message B (voice "Stop") is assigned the highest priority. Third control message A is assigned a lower priority. Since third control message B has the highest priority, the system determines the second control message to be "Stop all actions".

[0120] As can be seen, by introducing a priority ranking mechanism based on preset fusion rules in this embodiment, clear and predictable decisions can be made in complex scenarios of multimodal information fusion. This further ensures that in the event of information conflict or redundancy, the most critical, applicable, or clearly expressed user intent can be selected as the final control command according to a predetermined strategy, thereby improving the robustness, security, and user satisfaction of the control method. In one embodiment, the step of processing the signal based on the second control information to obtain the target control information includes: obtaining the target control information based on the second control information, a preset control mode database, and a preset kinematic model; or, obtaining the target control information based on the second control information and the preset kinematic model.

[0121] The preset control mode database stores various standard action sequences and their corresponding parameters (such as gripping patterns and motion trajectories). When processing discrete instructions, the system searches for and matches the most suitable mode from this database.

[0122] The pre-defined kinematic model is a mathematical model that describes how the joints of a prosthesis move to achieve changes in the position and posture of the end effector (such as a hand). It is used to convert high-level commands or low-level control signals into specific joint angles or velocities.

[0123] In the process of obtaining the target control information based on the second control information, the preset control mode database, and the preset kinematic model, the system searches for the mode corresponding to the second control information in the preset control mode database and obtains the corresponding mode parameters. The kinematic model calculates the motion trajectory of each joint based on the mode parameters, thereby generating specific joint control signals (target control information). This process is applicable to scenarios that require calling preset standard actions (such as grasping and swinging).

[0124] For example, the second control information is to grasp a fragile object. The "gentle grip" mode is searched in the preset control mode database to obtain the corresponding mode parameters: finger closing speed = 5cm / s, maximum force = 20g. The kinematic model calculates the motion trajectory of each joint based on the mode parameters. The motion trajectory is "joint 1 rotates 10°, joint 2 rotates 5°". Therefore, the target control information is obtained: "joint 1 = 10° / s, joint 2 = 5° / s".

[0125] In the process of obtaining the target control information based on the second control information and the preset kinematic model, the kinematic model calculates the corresponding motion parameters based on the second control information and outputs the motion parameters as the target control information. This process is suitable for scenarios that require direct calculation of joint motion (such as free trajectory control).

[0126] For example, the second control information is for the hand to move 5cm to the left. The kinematic model calculates the joint angles based on the target position. The joint angles are: joint 1 = 30°, joint 2 = 20°. Therefore, the target control information is output as "joint 1 = 30° / s, joint 2 = 20° / s".

[0127] As can be seen, this embodiment achieves efficient, precise, and intelligent movement of the prosthesis through hierarchical control. Furthermore, by combining a preset database and kinematic model, it can flexibly handle standard movements and free trajectory tasks. At the same time, the fusion of second control information enhances the adaptability and safety of the control.

[0128] Optionally, the second control information is parsed to determine the instruction type of the second control information, the instruction type including a first instruction type and a second instruction type; if the second control information is the first instruction type, the target control information is obtained according to the second control information, the preset control mode database and the preset kinematic model; or, if the second control information is the second instruction type, the target control information is obtained according to the second control information and the preset kinematic model.

[0129] The instruction type is a classification of the second control information content. Based on the amount of information it contains and the control requirements, it can be divided into different types.

[0130] The first type of instruction can be discrete instructions, which are those that represent a specific, instantaneous action. They typically have clear start and end points and execute a complete sequence of actions.

[0131] For example, a user might explicitly express an intention to "grab" or "release" through EEG classification, or issue a "stop" command via voice. These types of commands require the system to match a predefined, complete action pattern for execution.

[0132] The second type of instruction can be a continuous control instruction, which is a control signal that requires continuous adjustment and smooth change. They usually do not have a clear end point (unless interrupted by another instruction), but adjust the output parameters (such as speed and position) in real time according to the strength or pattern of the signal.

[0133] For example, when a user continuously imagines a certain movement (such as imagining raising an arm), the resulting changes in the intensity of the EEG signals are decoded into control signals, which are used to continuously adjust the speed or position of the prosthetic joints.

[0134] The preset control mode database stores various standard action sequences and their corresponding parameters (such as gripping patterns and motion trajectories). When processing discrete instructions, the system searches for and matches the most suitable mode from this database.

[0135] The pre-defined kinematic model is a mathematical model that describes how the joints of a prosthesis move to achieve changes in the position and posture of the end effector (such as a hand). It is used to convert high-level commands or low-level control signals into specific joint angles or velocities.

[0136] In the process of parsing the second control information and determining the instruction type of the second control information, the judgment is made based on the characteristics of the signal (such as the duration and change pattern of the signal) or the content of the information (such as whether it contains explicit action words or parameters). The judgment logic can be as follows: if the information represents a specific action command (such as "grab" or "release"), it is classified as the first instruction type (discrete instruction). If the information represents a continuously changing intensity or pattern (such as a continuous motion image signal), it is classified as the second instruction type (continuous control).

[0137] In the process of obtaining the target control information based on the second control information, a preset control mode database, and a preset kinematic model, if the second control information is of the first instruction type, the following steps are taken: First, the content of the discrete instruction (e.g., "grasp") and other information that may be combined are used to search for a matching action sequence in the preset control mode database. For example, for a cylindrical cup, the "cup grip" mode is selected. Then, based on the matched action mode, a series of joint angle or velocity commands required to execute the mode are calculated using the preset kinematic model. These commands are typically a time series defining the complete motion process from the initial state to the final state. Finally, the generated target control information sequence is sent to the prosthesis's drive system, causing it to execute the action according to the predetermined pattern.

[0138] For example, the second control information is "grasp" (and the eye movement has selected a cylindrical cup). The system matches the "cup grip" pattern and calculates the joint angle sequence of the fingers closing to the appropriate position and the thumb forming a ring around the cylinder with the other fingers. Therefore, the target control information is [joint angle sequence T1, T2, …, Tn]; the prosthetic finger moves according to this sequence to complete the grasp.

[0139] In the implementation process of obtaining the target control information based on the second control information and the preset kinematic model, if the second control information is of the second command type, a predefined pattern is no longer searched. Instead, the continuous input signal is directly processed. The decoded signal strength or pattern is converted into the velocity or position change of the prosthetic joint in real time through a preset mapping relationship (which may be a simple proportional relationship or a complex nonlinear mapping). Therefore, the target control information is no longer a predefined sequence, but a continuously updated value representing the current desired velocity or position change. It needs to continuously receive input signals, continuously calculate and output new control values. The currently calculated control value is sent to the prosthetic drive system, causing the joint to make small, continuous adjustments according to the value.

[0140] For example, the second control information is a continuous motion visualization signal (intensity changes indicate the degree of imagined lifting). The EEG decoder outputs an upward lifting intention intensity value (such as a number between 0 and 1). The system maps in real time: intensity value * maximum permissible speed = current elbow joint speed, therefore the target control information is [current elbow joint speed V]. When the user's visualization intensity increases, V increases, and the elbow joint accelerates upward; when the visualization intensity decreases, V decreases, and the elbow joint decelerates; when the visualization reverses, V becomes negative, and the elbow joint moves in the opposite direction.

[0141] As can be seen, this embodiment uses two fundamentally different strategies, namely matching pattern library or real-time mapping, to generate target control information based on the different types of instructions, thereby achieving precise and flexible control of the prosthesis.

[0142] In one embodiment, after processing the signal according to the second control information to obtain the target control information, the method further includes: receiving feedback information from the target user, the feedback information being information collected after controlling the target user's prosthesis according to the target control information; and determining whether to calibrate the target control information based on the feedback information.

[0143] Feedback information refers to the information collected by sensors installed on the prosthesis itself or the user's residual limb after the prosthesis performs actions according to the target control information, as well as information about the status and effect of the prosthesis received by the user through senses such as vision and hearing.

[0144] The feedback information can be tactile feedback information, visual feedback information, or visual feedback information, etc.

[0145] Specifically, tactile feedback information refers to information detected by sensors (such as pressure sensors, vibration motors, and magnetic actuators) on the prosthesis or residual limb. For example, it includes the gripping force measured by sensors when the prosthesis grasps an object, whether the fingers are in contact with the object, and which finger is making contact. This information is transmitted to the user, allowing them to "feel" the tactile state of the prosthesis.

[0146] Specifically, visual feedback information refers to the virtual or real image of the prosthesis that the user sees through a VR / AR headset, observing whether its movements meet expectations (such as whether the fingers fully grasp the target). Alternatively, it can be status information such as grip strength and current mode seen on the AR interface or on the prosthesis's LED lights.

[0147] Specifically, visual feedback information refers to the prompts or voice announcements that users hear, such as a successful capture prompt, a low battery warning, or a mode switching prompt.

[0148] Among them, calibration involves adjusting control parameters based on feedback information to make the prosthetic limb movements more precise.

[0149] For example, the original control information was: 60% finger closure, speed 15° / s. After calibration, the new control information is: 70% finger closure, speed 12° / s (for a more stable grip).

[0150] As can be seen, this embodiment uses closed-loop feedback to make the control of the prosthesis more precise.

[0151] In one embodiment, the feedback information is compared with the target control information to obtain a similarity value; if the similarity value is greater than or equal to a preset threshold, the target control information continues to be executed; or, if the similarity value is less than the preset threshold, the target control information is calibrated according to the feedback information to obtain calibration information, which is used to control the prosthesis of the target user.

[0152] The comparison process involves comparing feedback information (actual state) with target control information (desired state) and calculating the differences or similarities between them. The comparison process can be a simple logical judgment (such as whether there is contact) or a complex numerical comparison (such as the difference between actual grip strength and desired grip strength).

[0153] The similarity value refers to the result of the comparison process, which quantitatively represents the degree of matching between the feedback information and the target control information. The higher the value, the more the actual execution matches expectations; the lower the value, the greater the deviation.

[0154] For example, if the target control information is "hold stably with 0.5N", but the feedback information shows that the actual grip force is 0.4N and the fingers are in contact with the object, then the similarity value may be calculated based on the grip force deviation and the contact state. If the deviation is small, the similarity value may be high; if the deviation is large or the fingers are not in contact, the similarity value will be low.

[0155] The preset threshold is a pre-defined judgment standard used to distinguish between acceptable deviations and deviations that need correction. The preset threshold can be dynamically adjusted according to task requirements, user preferences, or system performance; it is not a single, fixed threshold.

[0156] The calibration information refers to the adjustment amount or new instruction calculated based on feedback information when the similarity value falls below a threshold. This adjustment is used to correct the original target control information. It guides the prosthesis to make fine adjustments to better match the user's intentions or adapt to changes in the environment.

[0157] Specifically, if the similarity value is greater than or equal to a preset threshold, it indicates that the actual execution effect is basically consistent with the expectation, and the error is within an acceptable range. The system determines that the current control is effective and continues to execute the current target control information without intervention. For example, when a user grabs a cup, they visually see the cup being firmly grasped and tactilely feel a moderate grip force; the similarity value is high, and the system maintains the current grip force control.

[0158] Specifically, if the similarity value is less than a preset threshold, it indicates a significant deviation that needs adjustment. The target control information is calibrated based on feedback, and the cause of the deviation is analyzed. For example, if feedback indicates insufficient grip strength (actual 0.3N, target 0.5N), the calibration information might be "increase grip strength output by 0.2N." If feedback indicates the finger is not in contact with the object but the target is to make contact, the calibration information might be "continue closing the finger." If visual feedback indicates the finger opening angle is not as expected, the calibration information might be "adjust the finger angle by X degrees." Therefore, the adjustment amount or new instruction is encapsulated as calibration information. Finally, the calibration information is sent to the prosthesis's drive system to execute corrective actions. For example, the prosthesis increases grip strength until the grip strength reported by the tactile sensor reaches or approaches the target value, or visual feedback indicates stable grasping.

[0159] As can be seen, by comparing with the original control intention, this embodiment can determine the execution deviation and perform intelligent calibration when necessary to correct the control command, which not only improves the accuracy of control but also reduces the user's frustration of "blind operation".

[0160] In one embodiment, determining whether to calibrate the target control information based on the feedback information includes: generating contact information based on the feedback information; querying a preset action database based on the contact information to obtain a first target instruction corresponding to the contact information; calibrating the target control information based on the first target instruction to obtain calibration information, wherein the calibration information is used to control the prosthesis of the target user.

[0161] Among them, contact information is feature data extracted from feedback information that describes the contact state between the prosthesis and an object or environment.

[0162] For example, feedback information: pressure 3N, sliding speed 2cm / s, generates contact information: "grip slip"; feedback information: collision force 20N, generates contact information: "hard object collision".

[0163] The preset action database refers to a database that stores standard action sequences and their parameters, which is used to match contact information and generate response instructions.

[0164] For example, action template 1: "grasp and slip", which leads to the first target instruction: increase the finger closing speed by 10% and the force by 2N; action template 2: "collision with a hard object", which leads to the first target instruction: stop moving forward, retreat 5cm, and reduce speed by 30%.

[0165] The first target instruction refers to the standard response instruction retrieved from the database based on the contact information. For example, if the contact information is "grasp and slip", the corresponding first target instruction is "closing speed +10%, force +2N".

[0166] The calibration information consists of optimized control parameters generated by fusing the first target command with the original target control information.

[0167] For example, the original target control information is: finger closes at 30°, force is 5N; the first target instruction is: closing speed +10%, force +2N; the calibration information is: finger closes at 30°, force is 7N, closing speed is increased by 10%.

[0168] Therefore, in this embodiment, for example, when a user uses a robotic arm to grasp a slippery cup, the sensor detects insufficient pressure (3N) and slippage at the rim of the cup (1cm / s), and the feedback information obtained is: pressure 3N, slippage speed 1cm / s. If the pressure is lower than the threshold (5N) and slippage occurs, the contact information obtained is "grasp slippage". The system queries the preset action database, and the database contains the first target instruction corresponding to the "grasp slippage" template, which is "closing speed +10%, force +2N". The original control information is: finger closes 40°, force 5N, closing speed 5cm / s. Calibration is performed, and the calibration logic is: force adjustment: 5N + 2N = 7N; speed adjustment: 5cm / s × 1.1 = 5.5cm / s. Therefore, the calibration information is: finger closes 40°, force 7N, closing speed 5.5cm / s. Finally, the prosthetic limb grasps the cup again, and the sensor feedback is 6N pressure and no slippage, and the similarity value meets the standard (e.g., 0.9), so the calibration is successful.

[0169] As can be seen, this embodiment avoids real-time calculation by setting templates in the action database, which is suitable for scenarios with high real-time requirements. Moreover, the action database can be dynamically updated, supporting the rapid integration of new scenarios.

[0170] In one embodiment, determining whether to calibrate the target control information based on the feedback information includes: checking whether the feedback information meets a preset standard indicator; if it does, continuing to execute the target control information; or, if it does not, adjusting the feedback information according to the preset standard indicator to obtain calibration information, the calibration information being used to control the prosthesis of the target user.

[0171] Among them, the preset standard indicators refer to the pre-set quantitative standards used to evaluate the performance of prostheses, which include performance parameters in multiple dimensions.

[0172] For example, force indicators: gripping force range (e.g., 2-5N); positional accuracy: the deviation threshold between the end effector and the target position (e.g., ±3mm); response time: the time limit from the issuance of the command to the completion of the action (e.g., 0.5s); stability indicators: the vibration amplitude limit during the action, etc.

[0173] In the process of verifying whether the feedback information meets the preset standard indicators, the raw data of different sensors are converted into a unified dimension, and key features corresponding to the standard indicators (such as maximum pressure value and final position coordinates) are selected; the feature data are compared with the standard threshold item by item; this item-by-item comparison can be that if a single indicator fails to meet the standard, the whole is judged as failing, or multiple indicators are comprehensively scored, and if they are lower than the threshold, they are judged as failing.

[0174] In the process of adjusting the feedback information according to the preset standard indicators to obtain calibration information if the target is not met, multiple adjustment strategies can be used. These strategies can be parameter compensation, i.e., linear compensation for deviation parameters (e.g., increasing the motor current proportionally when the force is insufficient); or mode switching, i.e., switching the control mode when the target is not met (e.g., switching from position control to force control); or incremental adjustment, i.e., using a PID algorithm for gradual adjustment.

[0175] Optionally, a safety mechanism can be added during the adjustment process. This safety mechanism can be a maximum adjustment range limit, i.e., a single adjustment cannot exceed 20% of the original value; or an abnormal interruption, i.e., the action is stopped immediately when a danger signal (such as excessive resistance) is detected.

[0176] As can be seen, this embodiment significantly improves the reliability and usability of the prosthetic system through quantitative evaluation and closed-loop control.

[0177] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0178] As another aspect of the embodiments of this application, this application provides a prosthetic limb control device based on bioelectric signals. The prosthetic limb control device based on bioelectric signals can be a software module, which includes several instructions stored in a memory. A processor can access the memory, call the instructions, and execute them to complete the prosthetic limb control method based on bioelectric signals described in the above embodiments.

[0179] See Figure 4 , Figure 4 This is a schematic diagram of a prosthetic limb control device based on bioelectric signals provided in an embodiment of this application. Figure 4 As shown, the bioelectric signal-based prosthetic control device 400 includes: Acquisition unit 401 is used to acquire the first motion intention signal of the target user; Processing unit 402 is used to process the first motion intention signal to obtain first control information; The adjustment unit 403 is used to adjust the first control information according to the target interaction information of the target user to obtain the second control information; The processing unit 402 is further configured to perform signal processing based on the second control information to obtain target control information, which is used to control the prosthesis of the target user.

[0180] This embodiment uses a first motion intention signal to ensure that the acquired signal accurately reflects the user's true motion intention. Further data processing of the first motion intention signal generates first control information, which can initially extract and convert the user's motion intention into operable control commands. Furthermore, by introducing target interaction information, the first control information obtained solely based on the first motion intention signal is adjusted, thereby filtering or correcting noise or ambiguity in the first motion intention signal. This makes the generated second control information and target control information more accurately reflect the user's true intention, better adapting to the user's real-time needs and interactive environment. This allows for precise control of the target user's prosthesis, improving the prosthesis's operational accuracy and user experience, enhancing response flexibility and adaptability. Moreover, the introduction of target interaction information enables the system to integrate user intentions from more dimensions, thereby improving the system's ability to operate stably in complex or disruptive environments.

[0181] In one embodiment, in the process of processing the first motion intention signal to obtain the first control information, the processing unit 402 is further configured to: filter the first motion intention signal to obtain filtered first motion intention information; denoise the filtered first motion intention information to obtain second motion intention information; and decode the second motion intention information to obtain the first control information.

[0182] In one embodiment, in the process of decoding the second running intent information to obtain the first control information, the processing unit 402 is further configured to: extract features from the second running intent information to obtain feature vectors corresponding to each time window in the second running intent information; perform feature fusion processing on the feature vectors corresponding to each time window to obtain target feature vectors; predict the target feature vectors according to a preset classification model to obtain the predicted intent category and corresponding confidence level output by the preset classification model; and determine the first control information when the confidence level is greater than or equal to a preset confidence level, wherein the first control information includes the predicted intent category and the corresponding confidence level.

[0183] In one embodiment, after performing feature fusion processing on the feature vectors corresponding to each time window to obtain a target feature vector, the processing unit 402 is further configured to: input the target feature vector into a preset regression model for prediction to obtain a target value, wherein the target value is the first control information.

[0184] In one embodiment, before adjusting the first control information based on the acquired target interaction information of the target user to obtain the second control information, the processing unit 402 is further configured to: acquire first interaction information, the first interaction information being obtained by a first interaction device in the prosthetic control system; and / or acquire second interaction information, the second interaction information being obtained by a second interaction device in the prosthetic control system; and / or acquire third interaction information, the second interaction information being obtained by a second interaction device in the prosthetic control system; and generate the target interaction information of the target user based on the first interaction information and / or the second interaction information and / or the third interaction information.

[0185] In one embodiment, in adjusting the first control information according to the acquired target user's target interaction information to obtain the second control information, the adjustment unit 403 is further configured to: parse the target interaction information to obtain at least one type of interaction information; sequentially fuse and judge each type of interaction information in the first control information and the at least one type of interaction information to obtain at least one type of third control information; and integrate each type of third control information in the at least one type of third control information to obtain the second control information.

[0186] In one embodiment, in the process of integrating each of the at least one third control information to obtain the second control information, the adjustment unit 403 is further configured to: determine the priority corresponding to each third control information according to a preset fusion rule; and determine the second control information according to the priority, wherein the second control information is the third control information with the highest priority.

[0187] In one embodiment, in the step of processing the signal according to the second control information to obtain the target control information, the processing unit 402 is further configured to: obtain the target control information according to the second control information, the preset control mode database and the preset kinematic model; or, obtain the target control information according to the second control information and the preset kinematic model.

[0188] In one embodiment, after processing the signal according to the second control information to obtain the target control information, the processing unit 402 is further configured to: receive feedback information from the target user, the feedback information being information collected after controlling the target user's prosthesis according to the target control information; and determine whether to calibrate the target control information based on the feedback information.

[0189] In one embodiment, in determining whether to calibrate the target control information based on the feedback information, the processing unit 402 is further configured to: compare the feedback information with the target control information to obtain a similarity value; if the similarity value is greater than or equal to a preset threshold, then continue to execute the target control information; or, if the similarity value is less than the preset threshold, then calibrate the target control information based on the feedback information to obtain calibration information, the calibration information being used to control the prosthesis of the target user.

[0190] In one embodiment, in determining whether to calibrate the target control information based on the feedback information, the processing unit 402 is further configured to: generate contact information based on the feedback information; query a preset action database based on the contact information to obtain a first target instruction corresponding to the contact information; calibrate the target control information based on the first target instruction to obtain calibration information, wherein the calibration information is used to control the prosthesis of the target user.

[0191] In one embodiment, in determining whether to calibrate the target control information based on the feedback information, the processing unit 402 is further configured to: check whether the feedback information meets a preset standard indicator; if it does, continue to execute the target control information; or, if it does not, adjust the feedback information according to the preset standard indicator to obtain calibration information, the calibration information being used to control the prosthesis of the target user.

[0192] It should be noted that the above-described bioelectric signal-based prosthetic control device can execute the bioelectric signal-based prosthetic control method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the bioelectric signal-based prosthetic control device can be found in the bioelectric signal-based prosthetic control method provided in the embodiments of this application.

[0193] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the bioelectric signal-based prosthetic control method as described in the foregoing embodiments.

[0194] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0195] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for controlling a prosthesis based on bioelectrical signals, characterized in that, Applied to a prosthetic control system, the method includes: Collect the target user's initial movement intention signal; The first motion intention signal is processed to obtain the first control information; The first control information is adjusted based on the target user's target interaction information to obtain the second control information; Signal processing is performed based on the second control information to obtain target control information, which is used to control the prosthesis of the target user.

2. The method according to claim 1, characterized in that, The step of processing the first motion intention signal to obtain the first control information includes: The first motion intention signal is filtered to obtain the filtered first motion intention information; The filtered first motion intent information is denoised to obtain the second motion intent information; The second execution intent information is decoded to obtain the first control information.

3. The method according to claim 2, characterized in that, The decoding process of the second operational intent information to obtain the first control information includes: Feature extraction is performed on the second running intention information to obtain the feature vector corresponding to each time window in the second running intention information; The feature vectors corresponding to each time window are subjected to feature fusion processing to obtain the target feature vector; The target feature vector is predicted according to a preset classification model to obtain the predicted intent category and corresponding confidence level output by the preset classification model; If the confidence level is greater than or equal to a preset confidence level, the first control information is determined, and the first control information includes the predicted intent category and the corresponding confidence level.

4. The method according to claim 2, characterized in that, After performing feature fusion processing on the feature vectors corresponding to each time window to obtain the target feature vector, the method further includes: The target feature vector is input into a preset regression model for prediction to obtain a target value, which is the first control information.

5. The method according to claim 1, characterized in that, Before adjusting the first control information based on the acquired target user's target interaction information to obtain the second control information, the method further includes: Acquire first interactive information, which is obtained from a first interactive device in the prosthetic control system; and / or, Acquire second interactive information, which is obtained from a second interactive device in the prosthetic control system; and / or, The third interactive information is obtained, wherein the second interactive information is obtained by the second interactive device in the prosthetic control system; Based on the first interaction information and / or the second interaction information and / or the third interaction information, the target interaction information of the target user is generated.

6. The method according to claim 1, characterized in that, The step of adjusting the first control information based on the acquired target user's target interaction information to obtain the second control information includes: The target interaction information is parsed to obtain at least one type of interaction information; The first control information and each of the at least one interaction information are sequentially fused and judged to obtain at least one third control information; The second control information is obtained by integrating each of the at least one third control information.

7. The method according to claim 6, characterized in that, The process of integrating each of the at least one third control information to obtain the second control information includes: According to the preset fusion rules, the priority of each third control information is determined; Based on the priority, the second control information is determined, and the second control information is the third control information with the highest priority.

8. The method according to claim 1, characterized in that, The step of processing the signal according to the second control information to obtain the target control information includes: The target control information is obtained based on the second control information, the preset control mode database, and the preset kinematic model; or, The target control information is obtained based on the second control information and the preset kinematic model.

9. The method according to claim 1, characterized in that, After processing the signal according to the second control information to obtain the target control information, the method further includes: The system receives feedback information from the target user, which is information collected after controlling the target user's prosthesis according to the target control information. Based on the feedback information, determine whether to calibrate the target control information.

10. The method according to claim 9, characterized in that, The step of determining whether to calibrate the target control information based on the feedback information includes: The feedback information is compared with the target control information to obtain a similarity value; If the similarity value is greater than or equal to a preset threshold, then the target control information continues to be executed; or, If the similarity value is less than the preset threshold, the target control information is calibrated according to the feedback information to obtain calibration information, which is used to control the prosthesis of the target user.

11. The method according to claim 9, characterized in that, The step of determining whether to calibrate the target control information based on the feedback information includes: Based on the feedback information, contact information is generated; Based on the contact information, a query is performed in a preset action database to obtain the first target instruction corresponding to the contact information; The target control information is calibrated according to the first target instruction to obtain calibration information, which is used to control the prosthesis of the target user.

12. The method according to claim 9, characterized in that, The step of determining whether to calibrate the target control information based on the feedback information includes: Verify whether the feedback information meets the preset standard indicators; If the target is achieved, then continue executing the aforementioned target control information; or, If the target is not met, the feedback information is adjusted according to the preset standard indicators to obtain calibration information, which is used to control the prosthesis of the target user.

13. An electronic device, characterized in that, The device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the electronic device to implement the bioelectric signal-based prosthetic control method as described in any one of claims 1-12 when executing the one or more computer programs.

Images