An intelligent under-knee prosthesis adaptive collaborative control system and control method

Abstract

This invention discloses an intelligent under-knee prosthesis adaptive collaborative control system and method, belonging to the technical field of intelligent under-knee prosthesis adaptive collaborative control systems. It acquires RGB-D image signals, plantar force signals, receiver interface force signals, IMU signals, sEMG signals, and fNIRS signals through a multimodal signal acquisition unit. The original multimodal signals are preprocessed, synchronized, and uniformly divided into control cycles to generate a unified control dataset. A fusion recognition model performs complex terrain environment recognition, motion intent decoding, and collaborative perception decision-making based on the dataset. The collaborative control unit, combined with real-time feedback signals, calculates ankle joint impedance adjustment parameters and receiver interface force adjustment parameters, generates collaborative control commands, and drives the two types of adjustment mechanisms to achieve adaptive adjustment. This overcomes key problems such as prosthesis dynamic adaptation lag in complex terrain environments, improving prosthesis wearing comfort, functional transmission efficiency, and motion coordination.

Description

Technical Field

[0001] This invention relates to the field of intelligent under-knee prosthesis adaptive collaborative control system technology, and in particular to an under-knee prosthesis adaptive collaborative control system and method based on multimodal perception and with feedback adjustment capability. Background Technology

[0002] Below-knee prostheses, also known as lower leg prostheses or transtibial prostheses, are important rehabilitation aids for patients with below-knee amputations to regain their walking ability. A below-knee prosthesis typically consists of a socket, a connecting support structure, and an ankle joint and prosthetic foot. The socket is the cavity that directly contacts and contains the residual limb; the connecting support structure is the structural part that connects the socket to the prosthetic foot; the ankle joint and prosthetic foot are located at the lowest point of the prosthesis and are mainly used to simulate ankle and foot function.

[0003] Traditional below-knee prostheses often feature fixed or limitedly adjustable sockets, with passive or semi-active ankle structures. This makes it difficult to achieve coordinated adaptive adjustment between the ankle joint and the socket, and also hinders the dynamic switching of movement patterns during walking on different terrains, easily leading to residual limb pain and skin damage. Furthermore, insufficient ankle adjustment can cause gait instability and increased compensatory energy consumption. The stress distribution at the traditional socket interface and ankle movement characteristics are difficult to adaptively adjust to complex terrain and gait changes, and the lack of a dynamic control mechanism based on information feedback results in a lag in the dynamic adaptation of below-knee prostheses, affecting wearing comfort, functional transmission efficiency, and motor coordination. Therefore, developing adaptive sockets and their control technologies for complex environments, overcoming key issues such as lag in dynamic adaptation, and improving wearing comfort, functional transmission efficiency, and motor coordination have become crucial problems urgently needing to be solved in this field.

[0004] Intelligent under-knee prostheses achieve dynamic adaptation through adaptive collaborative control by establishing a collaborative control system. Currently, control methods for under-knee prostheses mainly include joint torque compensation control, motion trajectory following control, and intention control. Joint torque compensation control is typically achieved through impedance control based on finite state machines; motion trajectory following control enables the prosthesis to follow a predetermined kinematic trajectory, and some microprocessor-based prostheses adjust joint damping and power output by collecting gait-related kinematic information to achieve adaptive gait regulation; intention control uses biosignals for low-level control, with existing research often utilizing electromyographic signals to adjust parameters such as joint impedance. However, these methods are mostly focused on single-joint or single-modal control, and still suffer from insufficient environmental perception, delayed mode switching, and difficulties in multi-system coordination in complex terrain environments. Therefore, there is an urgent need to establish a collaborative control system for under-knee prostheses that integrates multi-source sensory information and possesses feedback adjustment capabilities. Summary of the Invention

[0005] Existing collaborative control systems and methods still suffer from drawbacks such as insufficient environmental perception, delayed mode switching, and difficulties in multi-system coordination in complex terrain environments. In view of at least one of the above technical problems, this invention provides an intelligent under-knee prosthesis adaptive collaborative control system and method. Through multimodal signal acquisition and fusion, it achieves complex terrain environment recognition and motion intent decoding, and accordingly completes collaborative perception fusion output, hierarchical control network decision-making, parameter calculation, and control execution, forming a feedback closed-loop adaptive adjustment mechanism. This enables the socket fit state and ankle joint dynamic output to adjust collaboratively in real time according to changes in terrain and motion intent, reducing the risk of interface overpressure and gait instability, and improving wearing comfort and walking stability. The specific technical solution is as follows: An intelligent below-knee prosthesis adaptive collaborative control method includes the following steps: S1: Acquire multimodal raw signals, perform time synchronization, and preprocess, synchronize, and uniformly divide the acquired multimodal raw signals into control cycles to construct a unified control dataset; the multimodal raw signals include RGB-D image signals, plantar force signals, receiver interface force signals, IMU signals, electromyography signals, and fNIRS signals; the unified control dataset includes terrain recognition dataset and intent recognition dataset; S2, Complex Terrain Environment Recognition: Based on the complex terrain environment recognition dataset, RGB-D environmental point cloud features and plantar force features are extracted. Spatial feature encoding is performed on the RGB-D environmental point cloud features, and temporal feature encoding is performed on the plantar force features. The encoding results are adaptively weighted and fused based on an attention mechanism. The fused features are input into the terrain recognition model, and the output is the terrain category, terrain geometric feature parameters, and recognition confidence. Motion intent decoding extracts plantar force features, receptive cavity interface force features, IMU features, sEMG features, and fNIRS features from the intent recognition dataset. Based on the ReliefF weight and mutual information fusion algorithm, the multimodal features are evaluated and filtered to construct a target feature subset. After temporal encoding and adaptive weighted fusion, the subset is input into the motion intent decoding model to obtain the motion intent decoding result. S3, Collaborative perception fusion: The complex terrain recognition results and motion intent decoding results obtained in step S2 are used to construct a joint feature set and map it to a unified joint feature space to obtain a joint representation. Collaborative decision-making is performed based on the joint representation to generate a control state determination result. S4, Layered control network decision-making and parameter calculation: The control state judgment result obtained in step S3 is used to generate the current control target and determine the current control mode. Based on the current control mode, under the constraints of stability, comfort and socket interface force, the ankle joint impedance parameters and socket interface force adjustment parameters are optimized and calculated in real time. According to the calculated parameters, the prosthetic actuator is driven to complete the action output, and the socket interface force, plantar force and IMU feedback signals are acquired to correct the target parameters, update the control parameters and correct the closed-loop control.

[0006] In step S1 of some embodiments of this disclosure, the acquired multimodal raw signals are preprocessed, specifically as follows: The system performs the following steps on RGB-D image signals: invalid depth removal, median filtering denoising, bilateral filtering smoothing, depth range normalization, and environmental point cloud construction. For plantar force signals and receiver interface force signals, it performs invalid signal removal, low-pass filtering denoising, baseline drift correction, pressure signal smoothing, signal normalization, gait cycle segmentation, and abnormal segment removal. For IMU signals, it performs low-pass filtering denoising, stationary segment zero-bias estimation, drift correction, attitude calculation, Z-score normalization, and sliding time window segmentation. For electromyography signals, it performs band-pass filtering denoising, power frequency interference suppression, full-wave rectification, envelope extraction, MVC normalization, and sliding time window segmentation. For fNIRS signals, it performs SNR threshold channel trimming, light intensity to light density conversion, sliding window artifact detection, wavelet filtering artifact correction, band-pass filtering, and hemoglobin concentration change conversion. Based on camera and IMU extrinsic parameter calibration results, the environmental point cloud constructed from RGB-D image signals is mapped to a unified world coordinate system, completing multimodal spatial alignment and unified control dataset construction.

[0007] In step S2 of some embodiments of this disclosure, the identification of complex terrain environment includes: RANSAC plane fitting was used to extract slope, PCA normal estimation was used to extract roughness, elevation maps were used to extract height differences, edge detection was used to extract step and obstacle boundaries, and connected component analysis was used to extract walkable areas to obtain RGB-D environmental point cloud features. Temporal statistical analysis was used to extract average pressure, peak detection was used to extract pressure peaks, variance statistical analysis was used to extract pressure distribution variance, COP calculation was used to extract pressure center trajectory, and phase segmentation was used to extract temporal duration to obtain plantar force features. PointNet++ was used to encode the spatial features of the RGB-D environmental point cloud features, and 1D-CNN and GRU networks were used to encode the temporal features of the plantar force features. A multi-head attention mechanism was used to perform cross-modal adaptive weighted fusion of the encoded RGB-D environmental point cloud features and plantar force features. The terrain categories included at least one of flat ground, uphill, downhill, upstairs stairs, and downstairs stairs.

[0008] In step S2 of some embodiments of this disclosure, motion intent decoding includes: Plantar force characteristics include mean pressure, peak pressure, pressure distribution variance, pressure center trajectory, and phase duration; receiver interface force characteristics include mean interface force, interface force fluctuation, maximum interface force, peak-to-peak value, effective interface force, and interface force change rate; IMU characteristics include mean square value, standard deviation, mean, peak-to-peak value, mean square frequency, and frequency variance; sEMG characteristics include wavelength, zero-crossing rate, spectral entropy, variance, root square, median frequency, skewness, and mean; fNIRS characteristics include mean, standard deviation, energy, peak value, kurtosis, and skewness; multi-scale residual convolution is employed. The convolutional network and attention-gated recurrent unit perform temporal encoding of sEMG features and fNIRS features. The temporal convolutional network and multi-head self-attention encoder perform temporal encoding of IMU features, receiver cavity interface force features and plantar force features. Based on the attention mechanism, the encoded multimodal features are adaptively weighted and fused. The fused features are input into the motion intention decoding model and output the current motion intention category and the intention prediction advance time. The motion intention category includes at least one of flat ground walking, uphill, downhill, up stairs, down stairs, starting and stopping.

[0009] In step S3 of some embodiments of this disclosure, specifically: A joint feature set is formed by combining terrain category, terrain geometric feature parameters, motion intention category, gait stage, and corresponding confidence level. Canonical correlation analysis is used to establish the correlation mapping relationship between terrain environment features and motion intention features, mapping them to a unified joint feature space to achieve heterogeneous feature alignment and obtain joint representation. Based on a gated adaptive weighting mechanism combining spatial and temporal features, the joint representation is fused to obtain a fused representation. The XGBoost collaborative fusion decision model is used to supervise the training and collaborative decision-making of the fused representation, generating a control state determination result that includes terrain category, motion intention, gait stage, and control state. Among them, the plantar force signal is shared in the terrain environment recognition link and the motion intention decoding link, and participates as a common constraint information in the fusion weight calculation and anti-shake criterion for control mode switching, so as to reduce the risk of short-term misjudgment and improve the stability of control mode switching.

[0010] In step S4 of some embodiments of this disclosure, the hierarchical control network decision-making and parameter calculation specifically involve: Task planning layer: A random forest model based on adaptive feature fusion is used to filter features and dynamically assign weights to the fused representations, determine the current control conditions and control modes, and use continuous consistency judgment and anti-jitter rules to manage control mode switching. Optimization adjustment layer: The peak value, rate of change and distribution non-uniformity of the socket interface force are used to characterize the comfort index, and the correlation characteristics of plantar force fluctuation and force ratio are used to characterize the stability index. Under the constraints of stability, comfort and socket interface force, a multi-objective collaborative optimization strategy based on deep reinforcement learning is used to optimize and adjust the ankle joint impedance parameters and socket interface force adjustment parameters in real time, and feasible domain and rate of change constraints are applied to the output parameters. Execution drive layer: Generates receiving cavity adjustment commands and ankle joint control commands based on the calculated parameters, drives the receiving cavity interface force detection and adjustment mechanism and the ankle joint impedance adjustment mechanism to implement closed-loop adjustment, acquires receiving cavity interface force, plantar force and IMU feedback signals, corrects target parameters, updates control parameters and performs closed-loop control correction; Among them, the target output of ankle joint impedance is parameterized by equivalent ankle joint angle, equivalent damping and feedforward compensation term to achieve adaptive adjustment of ankle joint dynamics; the adjustment of the receiving cavity interface force is based on the difference between the mean statistical value of the interface force and the target value to achieve real-time control of the receiving cavity interface force.

[0011] An intelligent below-knee prosthesis adaptive collaborative control system, the specific structure of which includes: The multimodal signal acquisition and unified control dataset construction module is used to simultaneously acquire RGB-D image signals, plantar force signals, receiver cavity interface force signals, IMU signals, sEMG signals and fNIRS signals, and preprocess, time synchronize and align and divide the raw signals of each modality into a unified control dataset that includes terrain recognition dataset and intent recognition dataset. The collaborative perception fusion module is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to perform complex terrain environment recognition and motion intent decoding based on the unified control dataset, and to perform joint feature construction, unified feature space mapping, adaptive weighted fusion and collaborative decision-making on the output results of the two to generate control state determination results. The hierarchical control and feedback update module is connected to the collaborative perception fusion module. Based on the control state determination result, it performs hierarchical control network decision-making and real-time calculation of target parameters, generates collaborative control commands, and drives the receiving cavity interface force detection and adjustment mechanism and the ankle joint impedance adjustment mechanism to complete collaborative adjustment. At the same time, it performs parameter updates and closed-loop control correction based on feedback signals.

[0012] In some embodiments of this disclosure, the multimodal signal acquisition and unified control dataset construction module specifically includes: A depth camera, positioned anterior to the prosthesis socket and facing the direction of travel, is used to acquire RGB-D image signals of the terrain environment ahead; a plantar force sensor, positioned at the bottom of the prosthesis footplate, is used to acquire plantar force signals; a socket interface force sensor, positioned at the contact area between the inner wall of the prosthesis socket and the residual limb, is used to acquire socket interface force signals; an IMU sensor, positioned at the ankle joint drive and transmission mechanism, is used to acquire angular velocity and acceleration signals at the ankle joint and for ankle joint angle calculation; a sEMG sensor, positioned at the target muscle group on the residual limb surface, is used to acquire surface electromyography signals; and an fNIRS sensor, positioned on the corresponding motor cortex area on the human scalp surface, is used to detect changes in oxyhemoglobin and deoxyhemoglobin concentrations. The unified control dataset construction unit is used to preprocess the multimodal raw signals, perform hardware-triggered synchronization and timestamp alignment, use the foot force ground contact event as a common time anchor point to compensate for residual delay, and perform window division and association alignment according to the unified control cycle. The RGB-D image signal and foot force signal are written into the terrain recognition dataset, and the sEMG signal, fNIRS signal, receiving cavity interface force signal, foot force signal and IMU signal are written into the intent recognition dataset to construct the unified control dataset.

[0013] In some embodiments of this disclosure, the collaborative perception fusion module specifically includes: The system includes a complex terrain environment recognition unit, a motion intent decoding unit, a collaborative perception fusion unit, and a collaborative perception path. The complex terrain environment recognition unit is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to receive terrain recognition dataset data, extract RGB-D environmental point cloud features and foot force features, and output terrain category, terrain geometric feature parameters, and recognition confidence after spatial feature encoding, temporal feature encoding, and multi-head attention adaptive weighted fusion. The motion intent decoding unit is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to receive the intent recognition dataset data, extract multimodal features, and output the motion intent category and intent prediction advance time after feature filtering, temporal encoding and adaptive weighted fusion. The collaborative perception fusion unit is connected to the complex terrain environment recognition unit and the motion intention decoding unit respectively. It is used to perform joint feature construction, unified joint feature space mapping, adaptive weighted fusion and collaborative decision-making on the complex terrain environment recognition results and motion intention decoding results, generate control state determination results, and use foot force signal cross-link sharing to perform anti-shake processing on control mode switching. The collaborative sensing path is used to transmit the control state determination result to the hierarchical control and feedback update module, forming an adaptive collaborative control link under the human-machine environment interaction mechanism.

[0014] In some embodiments of this disclosure, the hierarchical control and feedback update module specifically includes: The system comprises a hierarchical control network decision-making and target parameter real-time calculation unit, a receiver cavity interface force adjustment unit, an ankle joint impedance adjustment unit, an ankle joint drive and transmission unit, and a feedback and parameter update unit. The hierarchical control network decision-making and target parameter real-time calculation unit is used to determine the control conditions and control modes at the task planning layer using a random forest model based on adaptive feature fusion, at the optimization adjustment layer using a multi-objective collaborative optimization strategy based on deep reinforcement learning to calculate the ankle joint impedance parameters and receiver cavity interface force adjustment parameters, and at the execution drive layer generating and outputting collaborative control commands. The receiving cavity interface force adjustment unit is used to receive the receiving cavity interface force adjustment parameters and realize the real-time control of the receiving cavity interface force through the receiving cavity interface force detection and adjustment mechanism. The ankle joint impedance adjustment unit is used to receive ankle joint impedance parameters and realize adaptive adjustment of ankle joint dynamic parameters through the ankle joint impedance adjustment mechanism. The ankle joint drive and transmission unit is used to drive the prosthetic footplate to complete the action output according to the coordinated control command; The feedback transmission and parameter update unit is used to collect and transmit the receiving cavity interface force, plantar force and IMU feedback signals. Based on the receiving cavity interface force feedback, it updates the comfort-related statistics, based on the plantar force feedback, it updates the support stability characteristics, and based on the IMU feedback, it updates the motion trend characteristics. It continuously updates the feature weights of the task planning layer and the strategy parameters of the optimization adjustment layer, forming a closed-loop adjustment mechanism that can adapt to different terrains and different motion intentions.

[0015] Compared with existing technologies, the above-mentioned intelligent below-knee prosthesis adaptive collaborative control system and control method have the following beneficial effects: 1. The multimodal signal acquisition unit acquires RGB-D image signals, plantar force signals, receiver interface force signals, IMU signals, sEMG signals, and fNIRS signals. The raw multimodal signals are preprocessed, synchronized, and uniformly divided into control cycles to generate a unified control dataset. A fusion recognition model uses this dataset to perform complex terrain environment recognition, motion intent decoding, and collaborative perception decision-making. The collaborative control unit combines real-time feedback signals to calculate ankle joint impedance adjustment parameters and receiver interface force adjustment parameters, generating collaborative control commands to drive the two types of adjustment mechanisms to achieve adaptive adjustment, meeting the stability and comfort requirements of the prosthesis under different terrains and movement modes. 2. By preprocessing, synchronizing, and dividing the multimodal raw signals into a unified control cycle, a unified control dataset is constructed, enabling complex terrain environment recognition and motion intent decoding to be performed under a unified control cycle, thereby improving the consistency and temporal correspondence of control input information; 3. By constructing joint features, unifying joint feature space mapping, adaptive weighted fusion and collaborative decision-making on the results of complex terrain environment recognition and motion intent decoding, control state determination results are generated, thereby improving the accuracy and stability of control mode determination; 4. By using a hierarchical control network to make decisions and calculate target parameters in real time, the ankle joint impedance parameters and the socket interface force adjustment parameters are calculated, executed, and closed-loop corrected in a coordinated manner, thereby achieving coordinated control of the socket and ankle joint and improving the walking stability, control smoothness, and wearing comfort of the under-knee prosthesis under complex working conditions. 5. By forming a closed-loop structure at the system level, from multimodal perception and collaborative decision-making to execution control and feedback updates, the overall coordination, engineering feasibility, and application reliability of the system can be improved. Attached Figure Description

[0016] Figure 1 The overall flowchart of the adaptive collaborative control method for below-knee prostheses is shown below. Figure 2 Flowchart for preprocessing multimodal raw signals and constructing a unified control dataset; Figure 3 Flowchart for identifying complex terrain environments; Figure 4 Flowchart for decoding motion intent; Figure 5 A flowchart for collaborative perception fusion; Figure 6 Flowchart for real-time calculation of decision-making and target parameters in a hierarchical control network; Figure 7 Flowchart of an adaptive and collaborative control system for below-knee prostheses; Figure 8 A schematic diagram illustrating the arrangement of multimodal sensors and the synergistic relationship between the adjustment of the receiving cavity interface force and the ankle joint impedance adjustment; Figure 9 This is a schematic diagram of the overall structure of a below-knee prosthesis. The numbers in the diagram are as follows: 1. Prosthetic socket; 2. Socket interface force detection and adjustment mechanism; 3. Ankle joint impedance adjustment mechanism; 4. Ankle joint drive and transmission mechanism; 5. Prosthetic footplate. Detailed Implementation

[0017] To better understand the purpose, structure, and function of this invention, the technical solutions in the embodiments of this invention will be clearly and completely described below. It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit this application. The terms "comprising" and "having" and any variations thereof in this application are open-ended and intended to cover non-exclusive inclusion. The serial numbers assigned to components herein are merely for distinguishing the described objects and have no sequential or technical meaning. The term "connection" in this disclosure, unless otherwise specifically stated, includes both direct and indirect "connections".

[0018] As shown in the attached diagram. Figures 1 to 9 As shown, this embodiment provides an adaptive collaborative control system and method for below-knee prostheses. Through synchronous acquisition and fusion of multimodal signals, it achieves recognition of complex terrain environments and decoding of movement intentions, thereby forming a collaborative adaptive adjustment of the receptive cavity interface force and ankle joint impedance, as well as feedback closed-loop correction. To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] like Figure 1 As shown, the adaptive cooperative control method for below-knee prostheses provided in this embodiment operates cyclically within a unified control cycle, including: S1 acquires multimodal raw signals, performs time synchronization, and preprocesses, aligns, and divides the acquired multimodal raw signals into unified control cycles to construct a unified control dataset. The multimodal raw signals include RGB-D image signals, plantar force signals, receptor interface force signals, IMU signals, electromyography signals (sEMG signals), and fNIRS signals. The unified control dataset includes a terrain recognition dataset and an intent recognition dataset. Multimodal raw signals are acquired and time-synchronized. The raw signals of each modality within the same control cycle are aligned and combined to form a unified control dataset. The multimodal raw signals include RGB-D image signals, plantar force signals, receptor interface force signals, IMU signals, electromyography signals, and fNIRS signals; among which, the plantar force signal is shared between complex terrain environment recognition and motion intent decoding.

[0020] Synchronization Alignment and Periodic Encapsulation: A combination of hardware-triggered synchronization and timestamp synchronization is used to achieve multimodal time consistency. The foot force signal ground contact event is used as a common time anchor to compensate for residual delays between key modes, thus forming periodically aligned data within a unified control cycle. Let the unified control cycle be... , No. Each control cycle corresponds to a time interval that can be analyzed as follows: Equation 1: <Formula 1>; Periodic characteristics of high sampling rate signals are obtained by using a sliding time window within the control period. The window length and step size are obtained by calibration and determined by experiment.

[0021] Preprocessing: Filtering, denoising, drift correction, amplitude normalization, and outlier suppression are performed on each modal signal; zero bias is estimated based on the IMU stationary segment and drift correction is performed on the IMU signal; bias compensation is performed on the plantar force signal and the receiver cavity interface force signal, and bias is updated based on the plantar force ground contact and ground lift events.

[0022] S2, Complex Terrain Recognition, extracts RGB-D environmental point cloud features and plantar force features based on a terrain recognition dataset. It performs spatial feature encoding on the RGB-D environmental point cloud features and temporal feature encoding on the plantar force features. Based on an attention mechanism, it performs adaptive weighted fusion of the encoding results, inputs the fused features into the terrain recognition model, and outputs the terrain category, terrain geometric feature parameters, and recognition confidence. Motion intent decoding extracts plantar force features, receptive cavity interface force features, IMU features, sEMG features, and fNIRS features from the intent recognition dataset. Based on the ReliefF weight and mutual information fusion algorithm, the multimodal features are evaluated and filtered to construct a target feature subset. After temporal encoding and adaptive weighted fusion, the subset is input into the motion intent decoding model to obtain the motion intent decoding result. S3, Collaborative perception fusion: The complex terrain recognition results and motion intent decoding results obtained in step S2 are used to construct a joint feature set and map it to a unified joint feature space to obtain a joint representation. Collaborative decision-making is performed based on the joint representation to generate a control state determination result. S4, Layered control network decision-making and parameter calculation: Based on the control state determination result obtained in step S3, the current control target is generated, the current control mode is determined, and based on the current control mode, under the constraints of stability, comfort and socket interface force, the ankle joint impedance parameters and socket interface force adjustment parameters are optimized and calculated in real time. According to the calculated parameters, the prosthetic actuator is driven to complete the action output, and the socket interface force, plantar force and IMU feedback signals are obtained to perform target parameter correction, control parameter update and closed-loop control correction.

[0023] like Figure 7 As shown in the figure, this embodiment provides an intelligent below-knee prosthesis adaptive collaborative control system, including: The multimodal signal acquisition and unified control dataset construction module is used to simultaneously acquire RGB-D image signals, plantar force signals, receiver cavity interface force signals, IMU signals, sEMG signals and fNIRS signals, and preprocess, time synchronize and align and divide the raw signals of each modality into a unified control dataset that includes terrain recognition dataset and intent recognition dataset. The collaborative perception fusion module is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to perform complex terrain environment recognition and motion intent decoding based on the unified control dataset, and to perform joint feature construction, unified feature space mapping, adaptive weighted fusion and collaborative decision-making on the output results of the two to generate control state determination results. The hierarchical control and feedback update module is connected to the collaborative perception fusion module. Based on the control state determination result, it performs hierarchical control network decision-making and real-time calculation of target parameters, generates collaborative control commands, and drives the receiving cavity interface force detection and adjustment mechanism 2 and the ankle joint impedance adjustment mechanism 3 to complete collaborative adjustment. At the same time, it performs parameter updates and closed-loop control correction based on feedback signals.

[0024] Sensor arrangement as follows Figure 8 As shown: The RGB-D camera is fixed to the outside of the prosthesis support segment and faces the direction of travel; the plantar force acquisition unit is set inside the insole, covering the forefoot, midfoot, and hindfoot areas; the receiver cavity interface force sensor is set on the inner wall of the receiver cavity and corresponds to the contact area of ​​the residual limb; the IMU is fixed near the prosthesis support segment or ankle joint module; the electromyography acquisition unit is attached to the target muscle group area on the surface of the residual limb; the fNIRS acquisition unit is worn on the head and covers the area related to lower limb movement.

[0025] This embodiment further discloses the unified control dataset definition: [The following is a description of the first...] The alignment data within the control cycle is encapsulated into a unified control dataset. The dataset is divided into terrain recognition dataset, intent recognition dataset, and feedback dataset. The system state variables from the previous control cycle are also written into the dataset, which can be parsed as Equation 2: <Formula 2>; Among them, RGB-D image signals and plantar force signals are written into the terrain recognition dataset. Electromyography (EMG) signals, fNIRS signals, receptor interface force signals, plantar force signals, and IMU signals are written into the intent recognition dataset. The receiving cavity interface force, foot force, and IMU feedback signal are written into the feedback dataset. ; This refers to the system state quantity output by the control module for the previous control cycle.

[0026] like Figure 2As shown, to construct the unified control dataset, the RGB-D, plantar force, receiver interface force, IMU, electromyography (sEMG), and fNIRS signals are preprocessed by filtering and denoising, drift correction, and amplitude normalization, respectively. Then, time consistency is established by hardware triggering and timestamp alignment, and the plantar force ground contact event is used as a common time anchor point to compensate for residual time delay. Furthermore, the point cloud / environment representation is mapped to the unified world coordinate system based on camera and IMU extrinsic parameter calibration and combined with gravity direction constraints. Finally, the point cloud features are associated and aligned with temporal features such as plantar force according to the unified control cycle time window to complete the construction of the unified control dataset.

[0027] like Figure 3 As shown in the upper part, the RGB-D image signal and foot force signal in the terrain recognition dataset are input into the complex terrain environment recognition pathway, and the environment category, environment parameters and terrain confidence are output.

[0028] Environmental information preprocessing: Invalid depth removal, ground area extraction, and walkable area extraction are performed on the RGB-D image signal; stability is determined by combining the IMU signal and the inter-frame changes of the RGB-D image. Let the IMU angular velocity amplitude be... The RGB-D inter-frame difference is The threshold is Then the stability criterion can be analyzed as follows: Equation 3: stable k =1(∥ω k ∥<ω th ∧ ΔI k th Formula 3; when At the same time, maintain the environmental output of the previous control cycle and reduce the terrain confidence, and update the recognition results after stabilization.

[0029] Coordinate mapping and alignment: When the steady state is satisfied, a point cloud or height map is constructed as an environmental representation, and based on the camera and IMU extrinsic calibration results, combined with gravity direction constraints, it is mapped to a unified world coordinate system; then it is aligned with the foot force signal according to the time window of the unified control dataset.

[0030] Environmental geometric feature extraction: Based on the environmental representation, extract environmental geometric features, including at least slope, step edges, terrain height difference and normal vector.

[0031] Foot force feature extraction: Extract contact timing features and force change features from the foot force signal to characterize landing / lifting events and changes in support stability.

[0032] ​Complex terrain environment recognition model: Spatial encoding of environmental geometric features and temporal encoding of foot force features; Cross-modal feature alignment and fusion of spatial and temporal encoding results based on a multi-head attention-based dynamic weighted fusion mechanism to obtain fused environmental features; and Output complex terrain environment recognition results based on fused environmental features. Environment categories include at least one of flat ground, uphill, downhill, up stairs, and down stairs; environmental parameters include at least estimated slope angle, estimated step height, and estimated step depth, and terrain confidence is output.

[0033] Furthermore, the complex terrain environment recognition model can use DeepLabV3+ as the backbone network for semantic segmentation of environmental regions, and combine it with a fusion classification regression head to realize environmental category determination and environmental parameter regression calculation.

[0034] like Figure 3 As shown in the lower part, the electromyographic signals, fNIRS signals, receptive cavity interface force signals, plantar force signals and IMU signals in the intent recognition dataset are input into the motion intent decoding path, and the motion intent and intent confidence are output.

[0035] Multimodal preprocessing: Filtering, denoising, drift correction, amplitude normalization, and sliding time window segmentation are performed on each modal signal; to improve robustness, independent component analysis can be used to suppress noise components, and variational mode decomposition can be used to obtain multi-scale characterization.

[0036] Intent feature extraction: Intent features include at least muscle activation features, cerebral blood oxygenation change features, IMU posture and angular velocity features, plantar force distribution and force timing features, and receptive cavity interface force distribution and force timing features; among them, the peak value, rate of change and distribution non-uniformity of the receptive cavity interface force are used to characterize comfort-related states.

[0037] Feature selection: The intent features are evaluated for importance using a feature selection method based on the fusion of ReliefF weights and mutual information relevance, and a subset of target features is selected under redundancy constraints. An importance scoring method can be used, specifically analyzed as shown in Equation 4 below: <Formula 4>; in The weights are ReliefF. The mutual information correlation between features and intent labels. As a measure of feature redundancy, This is a weighting factor.

[0038] Intent decoding model: The target feature subset is input into the intent decoding model to identify motion intent and obtain the motion intent decoding result; the intent decoding model outputs the motion intent and intent confidence, whereby the motion intent includes at least one of the following: walking on flat ground, uphill, downhill, climbing stairs, descending stairs, starting, and stopping. The intent decoding model can be implemented using a gated recurrent network, and the intent probability distribution is output through a classification layer, with the highest probability used as the intent confidence.

[0039] Furthermore, the intent decoding model sets a prediction time window and outputs the motion intent prediction result for the next control cycle at the end of the current control cycle, which is used to provide advance for control mode switching and reduce switching lag.

[0040] like Figure 4 As shown in the upper part, the complex terrain environment recognition output and motion intention decoding output are input into the fusion recognition model. The fusion recognition model adaptively calculates the fusion weights and outputs a fusion state vector for collaborative control. The fusion state vector includes at least the environment category, environment parameters, terrain confidence, motion intention and intention confidence, and is combined with the system state variables of the previous control cycle for subsequent parameter calculation.

[0041] The fusion recognition model uses canonical correlation analysis to establish the correlation mapping relationship between terrain environment features and motion intention features, and constructs a joint feature space. Canonical correlation analysis can be expressed as maximizing the correlation after two projections, which can be specifically analyzed as Equation 5 below: <Formula 5>; in This is a terrain environment feature vector. This is the feature vector of the motion intention.

[0042] Within the joint feature space, a dynamic weighting mechanism is used to adaptively calculate the fusion weights and output the fusion state vector. The fusion weights can be gated, as shown in Equation 6 below: <Formula 6>; in For the representation of joint space, This is the Sigmoid function.

[0043] Foot force signal sharing: Foot force signals are shared in the complex terrain environment recognition link and the motion intention decoding link, and participate in the fusion weight calculation as common constraint information; further, the foot force signals are used as common constraint information for the anti-shake criterion of control mode switching, thereby providing consistent contact evidence when switching terrain and intention, reducing the risk of short-term misjudgment and improving the stability of control mode switching.

[0044] like Figure 4As shown in the lower part, the system enters a hierarchical control network based on the fused state vector, completing the cyclical updates of the task planning layer, optimization and adjustment layer, and execution drive layer. This generates collaborative control commands, which then drive the receiver interface force adjustment mechanism and the ankle joint impedance adjustment mechanism to implement closed-loop regulation. The receiver interface force, plantar force, and IMU feedback signals are collected and transmitted back via a feedback acquisition module for parameter updates in the next control cycle.

[0045] Task planning layer: Adaptive feature fusion random forest model is used to filter features and dynamically assign weights to the fused state vector, and output control condition judgment results and control mode switching decisions; continuous consistent judgment and anti-jitter rules are used to reduce jitter caused by frequent switching in a short period of time.

[0046] The optimization adjustment layer employs a multi-objective collaborative optimization strategy based on deep reinforcement learning for real-time parameter calculation, outputting ankle joint impedance adjustment parameters and socket interface force adjustment parameters, and generating collaborative control commands. Comfort indicators are characterized by the peak value, rate of change, and distribution non-uniformity of the socket interface force, while stability indicators are characterized by plantar force fluctuations and force ratio correlation characteristics. Feasibility domains and rate of change constraints are imposed on the output parameters.

[0047] The execution drive layer generates receiver cavity adjustment commands and ankle joint control commands based on the coordinated control commands, and performs closed-loop adjustment of the receiver cavity interface force adjustment mechanism and the ankle joint impedance adjustment mechanism. It acquires and transmits receiver cavity interface force, plantar force, and IMU feedback signals for use as mode switching criteria and parameter calculation updates in the next control cycle.

[0048] The target output of ankle joint impedance can be analyzed as follows: Equation 7: <Formula 7>; in For equivalent stiffness parameters, For equivalent damping parameters, For parameters related to the target ankle angle trajectory, This is a feedforward compensation term.

[0049] An example of receiving cavity interface force adjustment can be analyzed as follows: Equation 8: <Formula 8>; in This is a statistical measure of the mean interfacial force. For the target value, To adjust the parameters.

[0050] The receiving cavity interface force feedback signal, foot force feedback signal and IMU feedback signal are written into the feedback dataset of the unified control dataset and sent back to the control module to update the fused state vector and hierarchical control network control parameters, thereby driving the control condition discrimination, control mode switching decision and parameter calculation update of the next control cycle.

[0051] The feedback update includes: updating comfort-related statistics and distribution correction terms based on the force feedback of the receiving cavity interface; updating support stability-related characteristics and contact timing statistics based on the plantar force feedback; and updating stability judgment criteria and motion trend characteristics based on IMU feedback. The feedback signal is used to update the feature weights of the task planning layer and the strategy parameters or equivalent control parameters of the optimization adjustment layer, thereby forming a closed-loop adjustment mechanism that can continuously adapt to different terrains and different motion intentions.

[0052] like Figure 5 As shown, the overall process of the adaptive collaborative control system for the under-knee prosthesis in this application includes multimodal signal synchronous acquisition, unified control dataset construction, terrain environment recognition, motion intent decoding, collaborative perception fusion, hierarchical control decision-making and parameter calculation, execution drive, and feedback transmission and parameter update.

[0053] Combination Figure 6 This can further illustrate the multimodal sensor arrangement of the system and the synergistic relationship between "receptor cavity interface force adjustment - ankle joint impedance adjustment": On the one hand, the system integrates fNIRS, sEMG, IMU, plantar force and receptor cavity interface force to output motion intent decoding results; on the other hand, it integrates depth camera (RGB-D) and plantar force to output terrain environment recognition results, and then integrates the results of the two channels in the collaborative perception channel to drive adaptive collaborative control.

[0054] The multimodal signal acquisition and feedback module is used to acquire RGB-D image signals, plantar force signals, receiver interface force signals, IMU signals, electromyography signals and fNIRS signals, and to acquire receiver interface force, plantar force and IMU feedback signals. The modules synchronize the signals in time to construct a unified control dataset and transmit the feedback signals back.

[0055] The multimodal signal acquisition and feedback module includes a unified control dataset construction unit, which is used to perform time synchronization and data alignment of multimodal signals to form a unified control dataset, and write RGB-D image signals and plantar force signals into the terrain recognition dataset, write electromyography signals, fNIRS signals, receiver interface force signals, plantar force signals and IMU signals into the intent recognition dataset, and write the system state variables of the previous control cycle output by the control module, and write the receiver interface force, plantar force and IMU feedback signals into the feedback dataset and transmit them back.

[0056] The control module is connected to the multimodal signal acquisition and feedback module and is used to perform complex terrain environment recognition, motion intent decoding and collaborative perception fusion based on a unified control dataset, perform hierarchical control network decision-making and parameter calculation to generate collaborative control commands, and update parameters based on the feedback signal.

[0057] The control module includes a terrain environment recognition unit, a motion intent decoding unit, and a collaborative perception fusion unit; the collaborative perception fusion unit outputs a fused state vector and sends it to the hierarchical control network decision and parameter calculation unit; the foot force signal is shared in the terrain environment recognition unit and the motion intent decoding unit.

[0058] The control module includes a hierarchical control network decision and parameter calculation unit and an execution drive unit. The hierarchical control network decision and parameter calculation unit outputs ankle joint impedance adjustment parameters and socket interface force adjustment parameters and generates coordinated control commands. The execution drive unit converts the coordinated control commands into drive signals for the execution adjustment module to drive the ankle joint impedance adjustment mechanism and socket interface force adjustment mechanism, and realizes closed-loop adjustment and parameter update based on the feedback signal.

[0059] The adjustment module is connected to the control module and includes a socket interface force adjustment mechanism and an ankle joint impedance adjustment mechanism; the positions of the prosthetic socket 1, the socket interface force detection and adjustment mechanism 2, the ankle joint impedance adjustment mechanism 3, the ankle joint drive and transmission mechanism 4, and the prosthetic footplate 5 are as follows: Figure 9 As shown, the receiving cavity interface force detection and adjustment mechanism 2 and the ankle joint impedance adjustment mechanism 3 are used to perform receiving cavity interface force adjustment and ankle joint impedance adjustment respectively according to the coordinated control command.

[0060] By sharing foot force signals across links and organizing the unified control dataset into different domains, this embodiment reduces misjudgments and jitters during control mode switching, and improves wearing comfort and walking stability in complex terrain and multiple sports modes.

[0061] The adaptive collaborative control method and system for under-knee prostheses provided in this application are divided into two levels: a collaborative control method and a supporting collaborative control system. The collaborative control method completes multimodal synchronous construction of a unified control dataset, complex terrain environment recognition, motion intent decoding, collaborative perception fusion output, hierarchical control network decision-making and real-time parameter calculation, actuator collaborative adjustment, and feedback closed-loop correction within a unified control cycle. The supporting collaborative control system is installed on the under-knee prosthesis body and socket structure, and consists of a multimodal sensor layer, a multimodal signal acquisition and feedback module, a control module, and an execution adjustment module. These two systems work together to determine complex terrain and human motion intent and to determine the control strategy.

[0062] The collaborative control method includes: acquiring and preprocessing multimodal signals and constructing a unified control dataset; identifying complex terrain environments based on RGB-D image signals and plantar force signals; decoding motion intent based on electromyography signals, fNIRS signals, receptor interface force signals, plantar force signals, and IMU signals; dynamically weighting and fusing the complex terrain environment identification results and motion intent decoding results to obtain a fused state vector; completing control condition discrimination, control mode switching decisions, and parameter calculation in a hierarchical control network; outputting ankle joint impedance adjustment parameters and receptor interface force adjustment parameters and executing them in a closed loop; and performing closed-loop correction and parameter updates based on receptor interface force, plantar force, and IMU feedback.

[0063] Among them, the complex terrain environment recognition outputs the environment category, environment parameters and terrain confidence; the motion intent decoding outputs the motion intent and intent confidence; the collaborative perception fusion adaptively weights the two types of results and outputs the fusion state vector, which is used to drive the hierarchical control network update.

[0064] The hierarchical control network comprises a task planning layer, an optimization and adjustment layer, and an execution driving layer: the task planning layer uses an adaptive feature fusion random forest model to determine the control conditions and manage the switching of control modes; the optimization and adjustment layer uses a multi-objective collaborative optimization model based on deep reinforcement learning to solve for the ankle joint impedance adjustment parameters and the socket interface force adjustment parameters; the execution driving layer implements closed-loop control on the ankle joint impedance adjustment mechanism and the socket interface force adjustment mechanism respectively and completes parameter updates.

[0065] The collaborative control system acquires terrain information, force information, and human intention information through a sensor layer; constructs a unified control dataset and transmits feedback signals through a multimodal signal acquisition and feedback module; outputs a fused state vector and generates collaborative control commands through a control module; performs receiver interface force adjustment and ankle joint impedance adjustment through an execution adjustment module; and achieves closed-loop correction and parameter updates through a feedback path, thereby enabling collaborative adjustment in flat, sloping, and stair environments, improving wearing comfort and walking stability.

[0066] Compared to control methods that rely solely on threshold triggers from a single sensor, this application unifies complex terrain environment recognition and motion intent decoding into a single control dataset. Furthermore, it enhances control robustness in complex scenarios and reduces the risks of interface overpressure and gait instability through cross-link sharing of plantar force signals, collaborative perception fusion, and hierarchical control network decision-making and parameter calculation. Its advantages include at least the following: (1) To achieve coordinated adaptive adjustment of the receiving cavity interface force and ankle joint impedance under complex terrain; (2) Improve adaptability and recognition stability to complex terrain and individual differences; (3) Reduce the probability of false switching and improve control smoothness through fusion and hierarchical decision-making; (4) Achieve an adaptive balance between comfort and stability through multi-objective collaborative optimization.

[0067] It is understood that the above description is only for illustrating specific embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the scope of disclosure of this application.

Claims

1. A method for adaptive synergic control of intelligent knee prostheses, characterized in that, Includes the following steps: S1: Acquire multimodal raw signals, perform time synchronization, and preprocess, synchronize, and uniformly divide the acquired multimodal raw signals into control cycles to construct a unified control dataset; the multimodal raw signals include RGB-D image signals, plantar force signals, receiver interface force signals, IMU signals, electromyography signals, and fNIRS signals; the unified control dataset includes terrain recognition dataset and intent recognition dataset; S2, Complex Terrain Environment Recognition: Based on the complex terrain environment recognition dataset, RGB-D environmental point cloud features and plantar force features are extracted. Spatial feature encoding is performed on the RGB-D environmental point cloud features, and temporal feature encoding is performed on the plantar force features. The encoding results are adaptively weighted and fused based on an attention mechanism. The fused features are input into the terrain recognition model, and the output is the terrain category, terrain geometric feature parameters and recognition confidence. Motion intent decoding extracts plantar force features, receptive cavity interface force features, IMU features, sEMG features, and fNIRS features from the intent recognition dataset. Based on the ReliefF weight and mutual information fusion algorithm, the multimodal features are evaluated and filtered to construct a target feature subset. After temporal encoding and adaptive weighted fusion, the subset is input into the motion intent decoding model to obtain the motion intent decoding result. S3, Collaborative perception fusion: The complex terrain environment recognition results and motion intention decoding results obtained in step S2 are used to construct a joint feature set and map it to a unified joint feature space to obtain a joint representation. Collaborative decision-making is performed based on the joint representation to generate a control state determination result. S4, Layered control network decision-making and parameter calculation: The control state judgment result obtained in step S3 is used to generate the current control target and determine the current control mode. Based on the current control mode, under the constraints of stability, comfort and socket interface force, the ankle joint impedance parameters and socket interface force adjustment parameters are optimized and calculated in real time. According to the calculated parameters, the prosthetic actuator is driven to complete the action output, and the socket interface force, plantar force and IMU feedback signals are acquired to correct the target parameters, update the control parameters and correct the closed-loop control.

2. The adaptive co-operative control method of the intelligent below-knee prosthesis according to claim 1, characterized in that, In step S1, the acquired multimodal raw signals are preprocessed, specifically as follows: The system performs invalid depth removal, median filtering denoising, bilateral filtering smoothing, depth range normalization, and environmental point cloud construction on RGB-D image signals. It also performs invalid signal removal, low-pass filtering denoising, baseline drift correction, pressure signal smoothing, signal normalization, gait cycle segmentation, and abnormal segment removal on plantar force signals and receiver interface force signals. For IMU signals, it performs low-pass filtering denoising, zero-bias estimation of stationary segments, drift correction, attitude calculation, Z-score normalization, and sliding time window segmentation. For sEMG signals, it performs band-pass filtering denoising, power frequency interference suppression, full-wave rectification, envelope extraction, MVC normalization, and sliding time window segmentation. For fNIRS signals, it performs SNR threshold channel trimming, light intensity to light density conversion, sliding window artifact detection, wavelet filtering artifact correction, band-pass filtering, and hemoglobin concentration change conversion. Based on camera and IMU extrinsic parameter calibration results, the environmental point cloud constructed from RGB-D image signals is mapped to a unified world coordinate system, completing multimodal spatial alignment and unified control dataset construction.

3. The adaptive co-operative control method of the intelligent below-knee prosthesis according to claim 1, characterized in that, In step S2, the identification of complex terrain environment includes: RANSAC plane fitting was used to extract slope, PCA normal estimation was used to extract surface roughness, elevation maps were used to extract height differences, edge detection was used to extract step and obstacle boundaries, and connected component analysis was used to extract walkable areas to obtain RGB-D environmental point cloud features. Temporal statistical analysis was used to extract average pressure, peak detection was used to extract pressure peaks, variance statistical analysis was used to extract pressure distribution variance, COP calculation was used to extract pressure center trajectory, and phase segmentation was used to extract temporal duration to obtain plantar force features. PointNet++ was used to perform spatial feature encoding on the RGB-D environmental point cloud features, and 1D-CNN and GRU networks were used to perform temporal feature encoding on the plantar force features. A multi-head attention mechanism was used to perform cross-modal adaptive weighted fusion of the encoded RGB-D environmental point cloud features and plantar force features. The terrain categories included at least one of flat land, uphill, downhill, upstairs stairs, and downstairs stairs.

4. The adaptive co-operative control method of the intelligent below-knee prosthesis according to claim 1, characterized in that, In step S2, specifically: Plantar force characteristics include mean pressure, peak pressure, pressure distribution variance, pressure center trajectory, and phase duration. The interface force characteristics of the receiving cavity include average interface force, interface force fluctuation, maximum interface force, peak-to-peak value, effective interface force, and interface force change rate. IMU features include mean square value, standard deviation, mean, peak-to-peak value, mean square frequency, and frequency variance; sEMG features include wavelength, zero-crossing rate, spectral entropy, variance, root square, median frequency, skewness, and mean; fNIRS features include mean, standard deviation, energy, peak value, kurtosis, and skewness. A multi-scale residual convolutional network with attention-gated recurrent units is used for temporal encoding of sEMG and fNIRS features. A temporal convolutional network with a multi-head self-attention encoder is used for temporal encoding of IMU features, receiver cavity interface force features, and plantar force features. An attention mechanism is used to adaptively weight and fuse the encoded multimodal features. The fused features are then input into the motion intent decoding model, which outputs the current motion intent category and the intent prediction lead time. The motion intent category includes at least one of walking on flat ground, going uphill, going downhill, going up stairs, going down stairs, starting, and stopping.

5. The intelligent below-knee prosthesis adaptive collaborative control method according to claim 1, characterized in that, In step S3, specifically: A joint feature set is formed by combining terrain category, terrain geometric feature parameters, motion intention category, gait stage, and corresponding confidence level. Canonical correlation analysis is used to establish the correlation mapping relationship between terrain environment features and motion intention features, mapping them to a unified joint feature space to achieve heterogeneous feature alignment and obtain joint representation. Based on a gated adaptive weighting mechanism combining spatial and temporal features, the joint representation is fused to obtain a fused representation. The XGBoost collaborative fusion decision model is used to supervise the training and collaborative decision-making of the fused representation, generating a control state determination result that includes terrain category, motion intention, gait stage, and control state. Among them, the plantar force signal is shared in the terrain environment recognition link and the motion intention decoding link, and participates as a common constraint information in the fusion weight calculation and anti-shake criterion for control mode switching, so as to reduce the risk of short-term misjudgment and improve the stability of control mode switching.

6. The intelligent below-knee prosthesis adaptive collaborative control method according to claim 1, characterized in that, In step S4, the hierarchical control network decision-making and parameter calculation specifically involve: Task planning layer: A random forest model based on adaptive feature fusion is used to filter features and dynamically assign weights to the fused representations, determine the current control conditions and control modes, and use continuous consistency judgment and anti-jitter rules to manage control mode switching. Optimization adjustment layer: The peak value, rate of change and distribution non-uniformity of the socket interface force are used to characterize the comfort index, and the correlation characteristics of plantar force fluctuation and force ratio are used to characterize the stability index. Under the constraints of stability, comfort and socket interface force, a multi-objective collaborative optimization strategy based on deep reinforcement learning is used to optimize and adjust the ankle joint impedance parameters and socket interface force adjustment parameters in real time, and feasible domain and rate of change constraints are applied to the output parameters. Execution drive layer: Generates receiving cavity adjustment command and ankle joint control command based on the calculation parameters, drives the receiving cavity interface force detection and adjustment mechanism (2) and ankle joint impedance adjustment mechanism (3) to implement closed-loop adjustment, obtains receiving cavity interface force, plantar force and IMU feedback signal, corrects target parameters, updates control parameters and performs closed-loop control correction; Among them, the target output of ankle joint impedance is parameterized by equivalent ankle joint angle, equivalent damping and feedforward compensation term to achieve adaptive adjustment of ankle joint dynamics. The interface force of the receiving cavity is adjusted based on the difference between the average statistical value of the interface force and the target value, so as to achieve real-time control of the interface force of the receiving cavity.

7. An intelligent under-knee prosthesis adaptive collaborative control system, characterized in that, include: The multimodal signal acquisition and unified control dataset construction module is used to simultaneously acquire RGB-D image signals, plantar force signals, receiver cavity interface force signals, IMU signals, sEMG signals and fNIRS signals, and preprocess, time synchronize and align and divide the raw signals of each modality into a unified control dataset that includes terrain recognition dataset and intent recognition dataset. The collaborative perception fusion module is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to perform complex terrain environment recognition and motion intent decoding based on the unified control dataset, and to perform joint feature construction, unified feature space mapping, adaptive weighted fusion and collaborative decision-making on the output results of the two to generate control state determination results. The hierarchical control and feedback update module is connected to the collaborative perception fusion module. Based on the control state determination result, it performs hierarchical control network decision-making and real-time calculation of target parameters, generates collaborative control commands, and drives the receiving cavity interface force detection and adjustment mechanism (2) and ankle joint impedance adjustment mechanism (3) to complete collaborative adjustment. At the same time, it performs parameter updates and closed-loop control correction based on feedback signals.

8. The intelligent below-knee prosthesis adaptive collaborative control system according to claim 7, characterized in that, The multimodal signal acquisition and unified control dataset construction module includes a multimodal signal acquisition unit and a unified control dataset construction unit; The multimodal signal acquisition unit includes: a depth camera, located on the front side of the prosthesis socket (1) and facing the direction of travel, for acquiring RGB-D image signals of the terrain environment in front; a plantar force sensor, located at the bottom of the prosthesis footplate (5), for acquiring plantar force signals; a socket interface force sensor, located in the contact area between the inner wall of the prosthesis socket (1) and the residual limb, for acquiring socket interface force signals; an IMU sensor, located at the ankle joint drive and transmission mechanism (4), for acquiring angular velocity and acceleration signals at the ankle joint and for ankle joint angle calculation; an sEMG sensor, located at the target muscle group on the surface of the residual limb, for acquiring surface electromyography signals; and an fNIRS sensor, located on the corresponding motor cortex area on the surface of the human scalp, for detecting changes in the concentration of oxyhemoglobin and deoxyhemoglobin. The unified control dataset construction unit is used to preprocess the multimodal raw signals, perform hardware-triggered synchronization and timestamp alignment, use the foot force ground contact event as a common time anchor point to compensate for residual delay, and perform window division and association alignment according to the unified control cycle. The RGB-D image signal and foot force signal are written into the terrain recognition dataset, and the sEMG signal, fNIRS signal, receiving cavity interface force signal, foot force signal and IMU signal are written into the intent recognition dataset to construct the unified control dataset.

9. The intelligent below-knee prosthesis adaptive collaborative control system according to claim 7, characterized in that, The collaborative perception fusion module includes a complex terrain environment recognition unit, a motion intent decoding unit, a collaborative perception fusion unit, and a collaborative perception path; The complex terrain environment recognition unit is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to receive terrain recognition dataset data, extract RGB-D environmental point cloud features and foot force features, and output terrain category, terrain geometric feature parameters and recognition confidence after spatial feature encoding, temporal feature encoding and multi-head attention adaptive weighted fusion. The motion intent decoding unit is connected to the multimodal signal acquisition and unified control dataset construction module. It is used to receive the intent recognition dataset data, extract multimodal features, and output the motion intent category and intent prediction advance time after feature filtering, temporal encoding and adaptive weighted fusion. The collaborative perception fusion unit is connected to the complex terrain environment recognition unit and the motion intention decoding unit respectively. It is used to perform joint feature construction, unified joint feature space mapping, adaptive weighted fusion and collaborative decision-making on the complex terrain environment recognition results and motion intention decoding results, generate control state determination results, and use foot force signal cross-link sharing to perform anti-shake processing on control mode switching. The collaborative sensing path is used to transmit the control state determination result to the hierarchical control and feedback update module, forming an adaptive collaborative control link under the human-machine environment interaction mechanism.

10. The intelligent below-knee prosthesis adaptive collaborative control system according to claim 7, characterized in that, The hierarchical control and feedback update module includes a hierarchical control network decision-making and target parameter real-time calculation unit, a receiving cavity interface force adjustment unit, an ankle joint impedance adjustment unit, an ankle joint drive and transmission unit, and a feedback transmission and parameter update unit. The hierarchical control network decision and target parameter real-time calculation unit is used to determine the control conditions and control modes in the task planning layer using a random forest model based on adaptive feature fusion, and to calculate the ankle joint impedance parameters and the receiving cavity interface force adjustment parameters in the optimization and adjustment layer using a multi-objective collaborative optimization strategy based on deep reinforcement learning. In the execution and driving layer, it generates and outputs collaborative control commands. The receiving cavity interface force adjustment unit is used to receive the receiving cavity interface force adjustment parameters and realize the real-time control of the receiving cavity interface force through the receiving cavity interface force detection and adjustment mechanism (2); The ankle joint impedance adjustment unit is used to receive ankle joint impedance parameters and realize adaptive adjustment of ankle joint dynamic parameters through the ankle joint impedance adjustment mechanism (3). The ankle joint drive and transmission unit is used to drive the prosthetic footplate (5) to complete the action output according to the coordinated control command; The feedback transmission and parameter update unit is used to collect and transmit the receiving cavity interface force, plantar force and IMU feedback signals. Based on the receiving cavity interface force feedback, it updates the comfort-related statistics, based on the plantar force feedback, it updates the support stability characteristics, and based on the IMU feedback, it updates the motion trend characteristics. It continuously updates the feature weights of the task planning layer and the strategy parameters of the optimization adjustment layer, forming a closed-loop adjustment mechanism that can adapt to different terrains and different motion intentions.

Images