Upper limb dual-mode wearable robot based on multi-sensor fusion and control method

Abstract

The invention discloses an upper limb dual-mode wearable robot based on multi-sensor fusion and a control method. The robot comprises a load-bearing rack; a shoulder joint assisting exoskeleton mechanism comprises a hydraulic actuator, a four-bar mechanism, an outward swinging support, an outward swinging rotating shaft, a thrust bearing, a shoulder adjustable support and a fixed connecting rod. The replaceable arm comprises an exoskeleton form arm and an outer limb form arm, and switching between an exoskeleton enhancing mode and an outer limb expanding mode is achieved by adjusting the connecting mode through a fixed connecting rod. The corresponding control method of the robot comprises the steps that data of a big arm IMU, a force sensor and a depth camera are fused, through LSTM time sequence coding and visual target detection, four working conditions of overhead supporting, carrying, outer limb weight carrying and free movement are intelligently recognized, and an actuator is controlled in a self-adaptive mode. According to the invention, human body assistance and limb expansion dual-mode integration of the same wearable platform is realized, and the wearable platform has the advantages of high power density, high intelligence and multi-task adaptation.

Description

Technical Field

[0001] This invention relates to the field of wearable exoskeleton equipment technology, and in particular to an upper limb dual-mode wearable robot and control method based on multi-sensor fusion. Background Technology

[0002] Most assistive robots take the form of exoskeletons and exlimbs. Exoskeleton robots are intelligent mechanical devices worn on the human body to provide assistance and enhancement. Currently, industrial exoskeletons are commonly used in tasks such as equipment handling, individual combat, and emergency rescue. These tasks often involve carrying heavy loads or transporting wounded personnel, requiring rapid transfer adaptability across multiple scenarios and working conditions. Exlimb robots are additional mechanical limb devices that can work collaboratively with the human body, extending the wearer's limbs and thus expanding their work capabilities. They are used to solve scenarios requiring multi-person collaboration. However, current assistive exoskeletons or exlimbs still suffer from limited auxiliary purposes. Modality switching algorithms for dual-mode upper limb wearable robots are still lacking in the field of wearable exoskeletons, and control over mode switching across multiple working conditions remains insufficient. Therefore, current wearable exoskeletons cannot effectively solve the problems of task mode switching and task control switching across multiple working conditions.

[0003] In summary, existing assistive augmentation robots have not perfectly solved the problems of weak shock resistance of the human body under heavy load conditions and multi-person collaborative work, and the limited range of working conditions. There is an urgent need for a new type of wearable upper limb augmentation robot designed for expandable lower limb exoskeleton robots, which can achieve high load lifting and multi-person collaborative industrial scenarios, and can realize the dual functions of human body augmentation and limb extension, providing a unified solution for complex working environments. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art, propose an upper limb dual-mode wearable robot and control method based on multi-sensor fusion, propose dual-mode transformation of exoskeleton and exoskeleton, develop a control method for mode regulation based on multi-sensor data, and design an adaptive carrying system and a quick-release mechanism.

[0005] This invention provides the following technical solution: a dual-mode wearable robot for the upper limbs based on multi-sensor fusion, comprising: The load-bearing frame includes carbon fiber plates that come into contact with the human body and aluminum alloy plates for load-bearing, as well as the arrangement of soft bags and shoulder straps that are tied to the human body, enabling the installation and arrangement of various functional modules and components. The quick-release front load hook includes a fixing piece mounted on the frame and a front load hook with a groove. A telescopic hook is installed at the end, and a telescopic rope is installed on the front load hook for the wearer to lift heavy objects in front of the body, distributing the heavy load of the whole body to the front and back sides, and realizing the front load function. The quick-release rear load-bearing platform includes a lower connecting support, a flip-up hook, a curved back plate, and a flip-up load-bearing platform. The lower connecting support is used to connect the load-bearing frame, and the flip-up hook can realize the storage, lowering, and quick disassembly of the flip-up load-bearing platform, realizing the rear load-bearing function and the function of carrying the wounded. The shoulder joint-assisted exoskeleton consists of a hydraulic actuator, an external swing support, an external swing rotation axis, an adjustable shoulder support, and a fixed link. The hydraulic actuator provides active assistance to the sagittal plane degree of freedom of the upper limb exoskeleton, while the external swing support and external swing rotation axis ensure passive degrees of freedom of lateral swing and rotation. The adjustable shoulder support can be adjusted according to the wearer's shoulder width, and the fixed link can enable the switching between the exoskeleton and the exolimb structure. The replaceable arm includes a connecting semicircular plate, an exoskeleton-shaped arm, and an exolimb-shaped arm. The connection method is adjusted through the aforementioned fixed connecting rod, and quick replacement is achieved through the sliding groove of the connecting semicircular plate. This allows for the use of an exoskeleton-shaped assistive arm and an exolimb-shaped assistive arm.

[0006] As a preferred option, the upper limb dual-mode wearable robot is bilaterally symmetrical, with a single-sided assist mechanism including three degrees of freedom in the shoulder area. The primary force-output sagittal plane degree of freedom utilizes a high power-to-weight ratio electro-hydraulic actuator. The robot can switch between dual modes depending on the working environment, enabling either arm enhancement or upper limb extension. When faced with heavy load handling or lifting scenarios, the robot switches to human enhancement mode, with the sagittal plane primarily contributing force to effectively elevate the wearer's upper arm. When facing overhead tasks requiring multi-person collaboration, the robot switches to limb extension mode, with multiple degrees of freedom in the single-sided assist mechanism working together to assist the wearer in tasks such as tool delivery and object lifting.

[0007] Preferably, the four-bar linkage satisfies the following kinematic relationship to control the lifting angle of the replaceable arm. : , , ,in, As an auxiliary angle, The distance between the lower electro-hydraulic actuator support, The distance between the upper and lower fulcrums. For the length of the electro-hydraulic actuator, The angle between the shoulder joint connector and the electro-hydraulic actuator. and The sum is a constant. These are the fixed geometric parameters of the four-bar linkage.

[0008] Preferably, the adjustable shoulder support has multiple adjustment positions to accommodate wearers with different shoulder widths.

[0009] Preferably, in the quick-release rear load-bearing platform, the lower end of the curved back plate is hinged to a shaft on the lower connecting support via a flip-plate hook, allowing the curved back plate to be adjusted in attitude relative to the lower connecting support; the tiltable load-bearing platform is a rotating component, detachably connected to the flip-plate hook via bolts and nuts, and the tiltable load-bearing platform and the curved back plate are connected by a steel cable; the steel cable is used to limit the rotation stroke of the tiltable load-bearing platform, keeping its rotation range relative to the lower connecting support stably maintained between 0° and 90°; when the tiltable load-bearing platform... When rotated to the 90° working position, the steel cable is taut to maintain the platform's posture. At the same time, the mechanical limiting structure of the flip-up hook prevents it from slipping off the lower connecting support, ensuring the structural load-bearing stability. When it is not necessary to place heavy objects behind the wearer, the flip-up hook, the flip-up load-bearing platform, and the steel cable can be removed from the lower connecting support and the curved back plate, leaving only the curved back plate as the back support component of the exoskeleton. The curved back plate adopts an ergonomic curved surface design, and its curved contour is highly adapted to the natural physiological curve of the human back.

[0010] Preferably, the replaceable arm is quickly connected to the shoulder joint assistive exoskeleton mechanism via a semi-circular connecting plate; the semi-circular connecting plate is provided with a sliding groove and is rigidly connected to the telescopic forearm or the upper arm support fixing component by bolts and nuts.

[0011] Secondly, the present invention provides a control method for the above-mentioned upper limb dual-mode wearable robot, comprising: Step 1: After starting the device, acquire inertial measurement unit (IMU) data vectors from the wearer's upper arm region. and force sensor data vector at the upper arm strap And acquire the red-green-blue (RGB) image stream from the depth camera; among which, For IMU in Triaxial acceleration data collected at all times. IMU's Three axes; For force sensors in The three-axis interaction force at any moment These are force sensors. Three-axis This represents the magnitude of the torque. Step Two: In the main control module, the data collected by each sensor is fused using a fusion algorithm. The IMU data vector at each moment is concatenated with the force sensor data vector to form a fused feature vector. ,in, ; Step 3: Install a Long Short-Term Memory (LSTM) network in the main control module to fuse feature vectors. The data is input into an LSTM cell, where the temporal features of the fused data are encoded through a forget gate, an input gate, and cell state updates. The LSTM cell state is then updated accordingly. The calculations for the forget gate and input gate are as follows: , , , ,in, , These are the activation vectors for the forget gate and the input gate, respectively; This is the Sigmoid activation function, used to control the opening and closing of the door; , , This is the weight matrix of the output gate; , , This is the bias vector for the output gate; Represents the Hadamard product function; Step 4: Calculate the hidden state at the current moment using the output gate. : , ,in, The output gate activation vector; , These are the weight matrix and bias vector of the output gate; Step 5: Retrieve the last hidden state of the LSTM within the time window. Inputting it into a fully connected layer and a Softmax function yields a set of preset working conditions. probability distribution And combined with the RGB data stream from the depth camera to identify the exoskeleton or exolimb category Make weighted decisions: , ,in, , These are the weight matrix and bias vector of the classification layer, respectively. To identify the category of exoskeleton or extremity using RGB data stream from a depth camera; Step Six: Based on the final working conditions Generate control commands to adjust the output or displacement of the hydraulic actuators to achieve robot actions adapted to the current working conditions.

[0012] Preferably, the preset working condition set include: Exoskeleton-assisted overhead support conditions: When For exoskeletons, when the IMU and force sensor data identify that the upper arm angle is ≥70° and maintains the posture, the hydraulic actuator is controlled to extend to the preset holding position and maintain a constant output force. Exoskeleton-assisted lifting conditions: When When the exoskeleton is used and the IMU and force sensor data identify that the upper arm angle is ≤15° and there is a continuous upward trend, the hydraulic actuator is controlled to perform closed-loop position control to drive the replaceable arm to rise to the preset angle. External limb lifting of heavy objects: when When it is an extremity, it takes priority over other working conditions determined by LSTM, and controls the extremity shape arm to perform lifting operations based on the depth camera and natural language commands; Human body free movement working conditions: when When the exoskeleton is in use and the IMU and force sensor data identify that the boom angle is between 15° and 70° and moves irregularly, the hydraulic actuator is controlled to be in a zero-pressure following state.

[0013] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described control method.

[0014] The beneficial effects of this invention are: (1) The dual-mode wearable robot and control method based on multi-sensor fusion provided by this invention proposes that the same wearable robot configuration can realize dual-mode conversion between exoskeleton robot and exolimb robot. The dual-mode wearable robot integrates the advantages of exoskeleton robot in assisting and enhancing the human body and exolimb robot in extending the human body limbs. It has multiple working condition combinations such as exoskeleton-assisted overhead support, exoskeleton-assisted lifting, exolimb heavy object lifting, and free human movement. At the same time, it adopts electro-hydraulic actuators in the sagittal plane, which has extremely high assistance capability and upper limit of motion capability, and has the characteristics of high power density, high anti-interference and high burst.

[0015] (2) The dual-mode wearable robot for upper limbs provided by this invention introduces a hybrid control algorithm based on LSTM long short-term memory network, which integrates time-series perception and environmental perception. It establishes a judgment mechanism that uses an exoskeleton or exoskeleton based on the environment fed back by the depth camera, and judges the assistive effect based on the time-series information provided by the IMU and force sensor. This method can achieve mode switching between the exoskeleton and exoskeleton in the dual-mode wearable robot, and can also determine the force required by the electro-hydraulic actuator, providing human-friendly assistive efficiency that meets ergonomic requirements.

[0016] (3) The dual-mode wearable robot for upper limbs based on multi-sensor fusion provided by the present invention adopts a four-bar linkage to optimize the transmission structure of the upper limbs and match the motion trajectory of the human shoulder joint to achieve seamless coordination between mechanical action and human upper limb movement, effectively reducing the constraint and interference of mechanical transmission on human limb activities. At the same time, the robot breaks through the layout limitations of traditional hydraulic actuators, integrates the core hydraulic actuator in the back area of ​​the human body, avoids the core range of upper limb activity, reduces the collision and friction between mechanical parts and the human body from the spatial layout, greatly improves the wearing comfort, simplifies the redundancy of the overall structure, makes the layout of each component more compact and reasonable, significantly improves the integration of the equipment, and facilitates the wearer's daily activities and work operations; in addition, by optimizing the appearance line design, the mechanical feel is weakened, the overall aesthetics of the machine are enhanced, breaking the limitations of the bulky and abrupt form of traditional auxiliary robots, and taking into account both practicality and wearing experience.

[0017] (4) The present invention provides a complete carrying system for the upper limb dual-mode wearable robot based on multi-sensor fusion. It is designed for expandable lower limb exoskeletons and integrates three major modules: front load-bearing hook, rear load-bearing platform, and human body frame. It fully covers typical load-bearing needs and supports various application scenarios such as hook lifting boxes, backpack carrying, and carrying wounded persons, significantly improving the wearer's load adaptability. The front load-bearing hook, rear load-bearing platform, and human body frame are all made into quick-release modules, which can be quickly installed when in use and stored or disassembled when not needed. Each component can be quickly installed and disassembled, the manufacturing process is simple, the reliability is high, and it can meet various working conditions. It is convenient to use and can be quickly switched according to different working conditions.

[0018] (5) The upper limb dual-mode wearable robot based on multi-sensor fusion provided by this invention is designed with human-machine engineering as the guide, adopts a biomimetic curved surface structure, and is modeled according to the curve of the human chest. Considering the pressure problem caused by the traditional rigid flat plate structure when carrying the wounded, a curved back plate is designed. The structural curve design of the shoulder hinge point is analyzed to match the exoskeleton shape of the arm and shoulder joint that matches the natural posture of the human arm, so as to ensure the stability of the upper limb assistance without interfering with the flexible movement of the arm.

[0019] The features and advantages of the present invention will be described in detail through embodiments and in conjunction with the accompanying drawings. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the upper limb dual-mode wearable robot structure of the present invention; Figure 2 This is a schematic diagram of the dual-mode replaceable robotic arm of the present invention, wherein the left figure shows the exoskeleton form and the right figure shows the exolimb form; Figure 3 This is a schematic diagram of the principle of the sagittal plane-assisted four-bar linkage of the present invention; Figure 4The diagram shows the exoskeleton and exolimb wearing structure of the present invention, from left to right: normal form (backplate retracted), carrying heavy load form (backplate lowered), carrying wounded person form (backplate removed). Figure 5 This is a schematic diagram of the multi-condition structure of the upper limb dual-mode wearable robot of the present invention; Figure 6 This invention provides the control algorithm framework for the upper limb dual-mode wearable robot. Figure 7 The diagram shows the load-bearing frame and quick-release front load-bearing hook structure of the present invention. From left to right, they are: the initial state of the exolimb, the horizontal operation state of the exolimb, the initial state of the exoskeleton, and the state of the exoskeleton assisted overhead support. The figure includes: the load-bearing frame M1, the quick-release front load-bearing hook M2, the quick-release rear load-bearing platform M3, the shoulder joint assist exoskeleton M4, the replaceable arm M5, and the quick-release waist belt M6. Detailed Implementation

[0021] The technical solution of the present invention will be further described below through embodiments and in conjunction with the accompanying drawings.

[0022] A dual-mode wearable robot based on multi-sensor fusion includes the aforementioned load-bearing frame M1, a quick-release front load-bearing hook M2, a quick-release rear load-bearing platform M3, a shoulder joint assist exoskeleton M4, a replaceable arm M5, and a quick-release waist belt M6, as shown below. Figure 1 As shown.

[0023] The load-bearing frame M1 includes a carbon fiber back plate and an aluminum alloy plate, which are rigidly connected to form a load-bearing frame for contact with the human body and for the arrangement of components.

[0024] The quick-release front load-bearing hook M2 consists of a fixed slide and an adjustable bracket, which together form the quick-release front load-bearing hook for loading the front of the body.

[0025] The quick-release rear load-bearing platform M3 comprises a lower connecting support, a flip-plate hook, a curved back plate, a tiltable load-bearing platform, and steel cables. The lower end of the curved back plate is hinged to a shaft on the lower connecting support via the flip-plate hook, allowing the curved back plate to be adjusted in attitude relative to the lower connecting support. The tiltable load-bearing platform is a rotating component, detachably connected to the flip-plate hook via bolts and nuts. The tiltable load-bearing platform and the curved back plate are connected by steel cables. The steel cables limit the rotational stroke of the tiltable load-bearing platform, keeping its rotation range relative to the lower connecting support stably between 0° and 90°. When the tiltable load-bearing platform rotates to the 90° working position, the steel cables are taut to maintain the platform's attitude, while the mechanical limiting structure of the flip-plate hook prevents it from slipping relative to the lower connecting support, ensuring structural load-bearing stability. Additionally, when there is no need to place heavy objects behind the wearer, the flip-up hook, the flip-up load-bearing platform, and the steel cable can be detached from the lower connecting support and the curved backplate, leaving only the curved backplate as the back support component of the exoskeleton. The curved backplate adopts an ergonomic curved surface design, and its curved contour is highly adapted to the natural physiological curve of the human back. When carrying injured persons, this curved structure can distribute the force on the wearer's chest area, effectively reducing the compression force on the chest, while improving the fit and comfort of carrying for a long time.

[0026] The shoulder joint assistive exoskeleton mechanism M4 includes a hydraulic actuator, an upper actuator support, an adjustable shoulder support, an outward swing support, an outward swing rotation shaft, a thrust bearing, a shaft end pressure ring, a fixing link, and a lower actuator support. The lower end of the hydraulic actuator is fixed to the lower actuator support and rigidly connected to an aluminum alloy plate. The upper end is connected to the adjustable shoulder support and fixed by a shaft and set screws. The rigid connection between the adjustable shoulder support and the upper actuator support allows for active freedom of movement of the shoulder joint assistive exoskeleton mechanism. The adjustable shoulder support and the outward swing support are fixed with bolts and nuts. The adjustable shoulder support has three adjustable ranges to accommodate wearers with different shoulder widths. The outward swing support and the outward swing rotation shaft are connected by a pin, with a step at one end and a retaining spring at the other. The outward swing rotation shaft is coaxially connected to the thrust bearing, and its lower side is rigidly connected to the outward swing rotation shaft by a shaft end pressure ring.

[0027] The replaceable arm structure M5 includes a semi-circular connecting plate, a large arm support fixing component, an exoskeleton-shaped arm, a telescopic large arm, a telescopic forearm, and a claw connecting plate. The exoskeleton-shaped structure consists of a semi-circular connecting plate, a telescopic forearm, a claw connecting plate, and a claw connecting plate. The exoskeleton-shaped structure consists of a semi-circular connecting plate, a large arm support fixing component, and an exoskeleton-shaped arm. The semi-circular connecting plate and the telescopic forearm are connected by a sliding groove and then rigidly connected with bolts and nuts.

[0028] The M6 ​​quick-release waist belt includes a waist belt and a liner. The liner is rigidly connected to the waist belt and carbon fiber plate by bolts and nuts to enhance the connection strength. A quick-release interface for the upper limb exoskeleton frame is provided at the top of the waist belt, and a quick-release interface for the lower limb exoskeleton is provided on the side, thus enabling quick assembly, disassembly, and storage of the upper and lower limb exoskeletons.

[0029] This invention discloses a dual-mode switchable arm for a dual-mode wearable robot based on multi-sensor fusion. This arm is adaptable to different working conditions and can autonomously switch between different configurations. The invention provides a quick-change shoulder joint design that can autonomously switch between, but is not limited to, exoskeletons and exoskeletal arms, making it suitable for various working conditions. A detailed structural diagram is shown below. Figure 2 As shown, the dual-mode arm is in the form of... Figure 3 As shown.

[0030] This invention presents a four-bar linkage design for a dual-mode wearable robot based on multi-sensor fusion, enabling sagittal plane assistance from an electro-hydraulic actuator. The sagittal plane assistance is achieved by actuating the push rod of the electro-hydraulic actuator. A simplified diagram of the mechanism is shown below. Figure 4 As shown. The distance between the lower electro-hydraulic actuator support and the support is denoted as... The distance between the upper and lower fulcrums is denoted as The length of the electro-hydraulic actuator is denoted as The angle between the shoulder joint connector and the electro-hydraulic actuator is denoted as... The angle at which the arm is raised can be varied and recorded as... ,in It is a constant value.

[0031] The assist efficiency provided by the electro-hydraulic actuator is approximated to the ratio of its lever elongation, thus allowing control over the length of the electro-hydraulic actuator in a four-bar linkage. To control the angle of the adjustable arm. Introducing auxiliary angles The function is as follows: , , , .

[0032] In this invention, the mechanism can be transformed into a quick-release load-bearing platform, enabling multiple functions such as individual soldier marching and combat, carrying loads on the back, and carrying wounded soldiers. The specific solution is as follows: In the described individual combat posture, with the retractable load-bearing platform stowed, the exoskeleton resembles a normal backpack, without interfering with the wearer's solo mission execution. In the rear-mounted load-bearing posture, the retractable load-bearing platform's position is adjusted via a flip-up hook, and connected to the curved back panel using steel cables. This configuration adapts to the needs of box-like transport during combat. In the wounded soldier carrying posture, the flip-up hook and retractable load-bearing platform are removed, leaving only the curved back panel. This curved back panel features an ergonomic curved surface design, its contour perfectly matching the natural physiological curve of the human back. When carrying a wounded soldier, this curved structure disperses the force on the wearer's chest area, effectively reducing chest compression and improving the fit between the exoskeleton and the body, as well as comfort during prolonged carrying. Figure 5 As shown, this demonstrates the equipment's versatility, allowing it to adapt to different working conditions and effectively solving the problem that current exoskeletons are only designed for a single working condition.

[0033] This invention discloses a dual-mode control switching method for a dual-mode wearable robot based on multi-sensor fusion, characterized in that it employs the aforementioned dual-mode wearable robot based on multi-sensor fusion, and the method includes: Step 1: After starting the device, record the IMU data vector installed on the upper arm of the human body. The function is as follows: ; Record force sensor data under the upper arm strap for human body The function is as follows: ; The camera data, consisting of RGB image streams, is fed into the host computer; among which... A vector of IMU data, For IMU in Triaxial acceleration data collected at all times. IMU's Three axes; This is the data vector of the force sensor. For force sensors in The three-axis interaction force at any moment These are force sensors. Three-axis This represents the magnitude of the torque. Step 2: In the main control module, the data collected by each sensor is fused using a fusion algorithm, as shown below: Will The function concatenates data vectors collected by multiple sensors at different times, as follows: ,in, ; Step 3: Install an LSTM (Long Short-Term Neural Network) in the main control module. The data is input into an LSTM unit to encode the temporal features of the fused data. Let the dimension of the LSTM hidden layer be . The state update function is input into the LSTM forget gate, and the function is as follows: ; The fused input information from multiple sensors is stored in the input gate of the LSTM, as shown in the following function: , ; Then, by updating the state of the gate cells, the information from long-term memory and the current input is fused together. The function is as follows: , in, , These are the activation vectors for the forget gate and the input gate, respectively; This is the Sigmoid activation function, used to control the opening and closing of the door; , , This is the weight matrix of the output gate; , , This is the bias vector for the output gate; Represents the Hadamard product function; Step 4: In the main control module, the RGB streams from the depth camera are fused to determine the human condition, and the final output is represented by an output gate. The function is as follows: , ,in, The output gate activation vector; , These are the weight matrix and bias vector of the output gate; Step 5: Retrieve the last hidden state of the LSTM within the time window. The action intent, output through multi-sensor data fusion, is placed into a fully connected layer and a Softmax function to determine the probability of belonging to a pre-defined work condition mode. Four work conditions are defined as follows: exoskeleton-assisted overhead support, exoskeleton-assisted lifting, exoskeleton-assisted heavy object lifting, and free human movement, forming a work condition set. Let the probability of each operating condition mode be denoted as . For the working conditions after commissioning The function to be calculated is: , , in, , These are the weight matrix and bias vector of the classification layer, respectively. To identify the category of exoskeleton or extremity using RGB data stream from a depth camera; Step Six: At this point, use the calculated highest probability. This refers to the currently selected working condition, and the algorithm logic is as follows: Figure 6 As shown.

[0034] In this invention, the mechanism can transform between the exoskeleton and the exolimb, enabling multiple functions such as the exolimb assisting the user in extending their limbs to perform corresponding operations and the exoskeleton assisting the user's upper arm in providing support. The specific solution is as follows: In the exoskeleton-assisted overhead support configuration, the RGB stream captured by the depth camera is used by a target detection algorithm to determine that the scenario does not require external limbs. Classified as an exoskeleton, and further determined by combining IMU sensor data and force sensor data, the upper arm is at an angle greater than 70° and maintains this posture for an extended period. The assist efficiency is then categorized as an exoskeleton-assisted overhead support condition. At this point, the electro-hydraulic actuator extends the hydraulic rod to approximately 85° of the exoskeleton arm and maintains this posture, meaning the electro-hydraulic actuator output is constant.

[0035] In the exoskeleton-assisted lifting scenario, the RGB stream captured by the depth camera is determined by the target detection algorithm to belong to a scene where external limbs are not required. Classified as an exoskeleton, and further determined by combining IMU sensor data and force sensor data, the upper arm is at an angle of less than 15° and has a continuous upward lifting trend. The assistance efficiency is then categorized as an exoskeleton-assisted lifting condition. At this point, the electro-hydraulic actuator extends the hydraulic rod from its current position to approximately a 45° position on the exoskeleton arm and maintains this posture. In other words, the electro-hydraulic actuator uses angle information collected by the encoder for position control until the preset angle is reached.

[0036] In the scenario of lifting heavy objects using external limbs, the RGB stream captured by the depth camera is used by a target detection algorithm to determine if the scene requires external limbs. The object is classified as an external limb, which has a higher priority than other work condition classifications determined by LSTM. Target detection is performed, and the lifting and lowering of the external limbs are controlled by a depth camera and natural language to complete the task of lifting heavy objects using the external limbs.

[0037] Under the condition of free human movement, the RGB stream captured by the depth camera is determined by the target detection algorithm to belong to a scene where no external limbs are required. Classified as an exoskeleton, and further analyzed using IMU sensor data and force sensors to determine that the upper arm is at an angle greater than 15° but less than 70° and is continuously and irregularly raising or holding its position, this is categorized as a free-movement human condition. In this state, the electro-hydraulic actuator is not energized, the cylinder damping force is low, and free extension and retraction are possible. Figure 7 As shown.

[0038] The above embodiments are illustrative of the present invention and are not intended to limit the present invention. Any simple modifications to the present invention are within the scope of protection of the present invention.

Claims

1. A dual-mode wearable robot for upper limbs based on multi-sensor fusion, characterized in that, include: A load-bearing frame for wearing on the human torso; A shoulder joint-assisted exoskeleton mechanism, installed in the shoulder area of ​​the load-bearing frame, includes a hydraulic actuator, a four-bar linkage, an external swing support, an external swing rotation shaft, a thrust bearing, an adjustable shoulder support, and a fixed link. The four-bar linkage connects the output end of the hydraulic actuator to the replaceable arm, converting the linear motion of the hydraulic actuator into the lifting motion of the replaceable arm. The external swing support and external swing rotation shaft provide passive lateral swing freedom, while the thrust bearing provides passive spin freedom. The adjustable shoulder support allows for adjustment of the distance according to the wearer's shoulder width. The replaceable arm is detachably mounted on the shoulder joint assistive exoskeleton mechanism, including an exoskeleton-shaped arm and an exolimb-shaped arm. The exoskeleton-shaped arm is used to conform to the wearer's arm to transmit assistance, and the exolimb-shaped arm is used to perform tasks independently of the arm. The replaceable arm is connected to the shoulder joint-assisted exoskeleton mechanism via the fixed link: by adjusting the connection method of the fixed link and installing the exoskeleton-shaped arm or the exoskeleton-shaped arm accordingly, the robot can switch between the exoskeleton enhancement mode and the exoskeleton extension mode.

2. The dual-mode upper limb wearable robot based on multi-sensor fusion according to claim 1, characterized in that, The hydraulic actuator is integrated into the back area of ​​the load-bearing frame. Its cylinder end is fixedly connected to the load-bearing frame through the lower actuator support, and its piston rod end is connected to the four-bar linkage through the upper actuator support.

3. The upper limb dual-mode wearable robot based on multi-sensor fusion according to claim 1, characterized in that, The four-bar linkage satisfies the following kinematic relationship to control the lifting angle of the replaceable arm. : , , ,in, As an auxiliary angle, The distance between the lower electro-hydraulic actuator support, The distance between the upper and lower fulcrums. For the length of the electro-hydraulic actuator, The angle between the shoulder joint connector and the electro-hydraulic actuator. and The sum is a constant. These are the fixed geometric parameters of the four-bar linkage.

4. The dual-mode upper limb wearable robot based on multi-sensor fusion according to claim 1, characterized in that, The adjustable shoulder support has multiple adjustment positions to accommodate wearers with different shoulder widths.

5. The upper limb dual-mode wearable robot based on multi-sensor fusion according to claim 1, characterized in that, Also includes: A quick-release front load-bearing hook, comprising a fixed slide and an adjustable bracket, is installed on the front side of the load-bearing frame for hanging the front load of the carrier; The quick-release rear load-bearing platform includes a lower connecting support, a flip-plate hook, a curved back plate, and a flip-up load-bearing platform; the lower end of the curved back plate is hinged to the lower connecting support via the flip-plate hook, and the flip-up load-bearing platform is detachably connected to the flip-plate hook and connected to the curved back plate via a steel cable to limit the rotation stroke; the curved back plate has a curved surface configuration adapted to the physiological curve of the human back.

6. The upper limb dual-mode wearable robot based on multi-sensor fusion according to claim 1, characterized in that, The replaceable arm is quickly connected to the shoulder joint assist exoskeleton mechanism via a semi-circular connecting plate; the semi-circular connecting plate is provided with a sliding groove and is rigidly connected to the telescopic forearm or the upper arm support fixing component by bolts and nuts.

7. A control method for the upper limb dual-mode wearable robot according to any one of claims 1 to 6, characterized in that, Includes the following steps: Step 1: After activating the device, collect IMU data vectors from the wearer's upper arm region. and force sensor data vector at the upper arm strap And acquire RGB image streams from the depth camera; where, For IMU in Triaxial acceleration data collected at all times. IMU's Three axes; For force sensors in The three-axis interaction force at any moment These are force sensors. Three-axis This represents the magnitude of the torque. Step Two: In the main control module, the data collected by each sensor is fused using a fusion algorithm. The IMU data vector at each moment is concatenated with the force sensor data vector to form a fused feature vector. ,in, ; Step 3: An LSTM (Long Short-Term Memory) network is integrated into the main control module to fuse feature vectors. The data is input into an LSTM cell, where the temporal features of the fused data are encoded through a forget gate, an input gate, and cell state updates. The LSTM cell state is then updated accordingly. The calculations for the forget gate and input gate are as follows: , , , ,in, , These are the activation vectors for the forget gate and the input gate, respectively; This is the Sigmoid activation function, used to control the opening and closing of the door; , , This is the weight matrix of the output gate; , , This is the bias vector for the output gate; Represents the Hadamard product function; Step 4: Calculate the hidden state at the current moment using the output gate. : , ,in, The output gate activation vector; , These are the weight matrix and bias vector of the output gate; Step 5: Retrieve the last hidden state of the LSTM within the time window. Inputting it into a fully connected layer and a Softmax function yields a set of preset working conditions. probability distribution And combined with the RGB data stream from the depth camera to identify the exoskeleton or exolimb category Make weighted decisions: , ,in, , These are the weight matrix and bias vector of the classification layer, respectively. To identify the category of exoskeleton or extremity using RGB data stream from a depth camera; Step Six: Based on the final working conditions Generate control commands to adjust the output or displacement of the hydraulic actuators to achieve robot actions adapted to the current working conditions.

8. The control method according to claim 7, characterized in that, The preset working condition set include: Exoskeleton-assisted overhead support conditions: When For exoskeletons, when the IMU and force sensor data identify that the upper arm angle is ≥70° and maintains the posture, the hydraulic actuator is controlled to extend to the preset holding position and maintain a constant output force. Exoskeleton-assisted lifting conditions: When When the exoskeleton is used and the IMU and force sensor data identify that the upper arm angle is ≤15° and there is a continuous upward trend, the hydraulic actuator is controlled to perform closed-loop position control to drive the replaceable arm to rise to the preset angle. External limb lifting of heavy objects: when When it is an extremity, it takes priority over other working conditions determined by LSTM, and controls the extremity shape arm to perform lifting operations based on the depth camera and natural language commands; Human body free movement working conditions: when When the exoskeleton is in use and the IMU and force sensor data identify that the boom angle is between 15° and 70° and moves irregularly, the hydraulic actuator is controlled to be in a zero-pressure following state.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the control method described in claim 7 or 8.

Images