An ar-based wearable upper limb assistance exoskeleton control method and system

Abstract

The application discloses a kind of based on AR's wearable upper limb power-assisted exoskeleton control method and system, it is related to exoskeleton application technical field, the three-dimensional space data and image data of the working area of automobile chassis are acquired;Depth image after conversion of three-dimensional space data and image data are input neural network CNN model, and feature information target component image is extracted;Target component image is compared with reference image under normal state, and target component working state is judged;Target component three-dimensional space data is converted into equipment display coordinate system, and standard virtual information library is superimposed on target component based on visual SLAM technology;EMG signal and shoulder joint angle are acquired, and muscle fatigue degree is calculated;Upper limb power-assisted exoskeleton is combined according to standard virtual information library and executes operating step, and according to virtual information and muscle fatigue degree, operating sequence and action posture are adjusted in real time.The application can effectively avoid excessive fatigue, motion injury and other problems by AR technology combined with exoskeleton.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of exoskeleton applications, and particularly relates to a wearable upper limb assisting exoskeleton control method and system based on AR. BACKGROUND

[0002] When assembling and checking and repairing automobile chassis parts, the workers often need to hold heavy objects for a long time in an uncomfortable posture. Since the human body can only hold heavy objects for a limited time, the workers need to suspend work many times, which leads to low work efficiency. In addition, in the process of holding heavy objects for a long time, the workers are prone to muscle strain and joint pain, and there is a certain risk of work accidents.

[0003] In order to solve the above problems existing in the process of mechanical repair, some schemes combining AR technology exist in the prior art, for example: Chinese invention patent CN112085223A discloses an induction system and method for mechanical repair, which comprises AR glasses, an external input controller and a server. The AR glasses are used to collect image information. The server performs deep training according to the image information to obtain a training model. The external input controller is used to send control information to the AR glasses. Chinese invention patent CN113570732A discloses a shield maintenance auxiliary method based on AR technology, which comprises a three-dimensional tracking registration unit, a virtual-real fusion display unit, a man-machine interaction unit and a maintenance auxiliary management unit. The AR technology is combined with the shield maintenance device. After the maintenance personnel enter the maintenance scene, they do not need to blindly observe fault information and refer to various maintenance manuals and maintenance information.

[0004] Although the above scheme improves the maintenance efficiency and accuracy to some extent, it does not solve the muscle fatigue problem caused by direct operation of the operator. SUMMARY

[0005] In view of the deficiencies existing in the prior art, the purpose of the present application is to provide a wearable upper limb assisting exoskeleton control method and system based on AR, which can monitor the body state (such as muscle fatigue degree, joint angle, etc.) of the worker in real time by combining AR technology with an exoskeleton, and prompt the worker to adjust the posture or action in time, so as to effectively avoid problems such as excessive fatigue and sports injury.

[0006] In order to achieve the above purpose, the present application is realized by the following technical scheme:

[0007] In a first aspect, an embodiment of the present application provides a wearable upper limb assisting exoskeleton control method based on AR, comprising:

[0008] acquiring three-dimensional space data and image data of an automobile chassis working area;

[0009] The converted depth image and image data of the three-dimensional space data are input into a neural network CNN model to extract feature information of a target component image; the target component image is compared with a reference image in a normal state to determine the working state of the target component; and when the working state of the component is abnormal, the accurate position of the target component in the image is obtained based on the neural network CNN model;

[0010] The three-dimensional space data of the target component is converted into a device display coordinate system, and a preset standard virtual information library is superimposed above the target component based on visual SLAM technology;

[0011] An EMG signal and a shoulder joint angle are obtained, and muscle fatigue degree is calculated.

[0012] The upper limb assisting exoskeleton is combined to perform an operation step according to the standard virtual information library, and the operation sequence and action posture are adjusted in real time according to the virtual information and the muscle fatigue degree.

[0013] As a further implementation manner, the three-dimensional space data is obtained in the following manner:

[0014] A LiDAR depth sensor is used to scan the working area of the automobile chassis to obtain the three-dimensional space data of the target component in real time, and three-dimensional point cloud data is generated.

[0015] As a further implementation manner, the image data is obtained in the following manner:

[0016] The image data is obtained based on an HMD head-mounted display device.

[0017] As a further implementation manner, the three-dimensional space data of the target component is converted into a device display coordinate system based on a coordinate conversion algorithm.

[0018] As a further implementation manner, the EMG signal is obtained by an electromyography sensor, and the shoulder joint angle is obtained by a rotary encoder.

[0019] The EMG signal and the shoulder joint angle are transmitted to an MCU microcontroller after signal processing, the MCU microcontroller calculates the muscle fatigue degree, and the muscle fatigue degree is transmitted to an AR system interface through a wireless transmission mode.

[0020] As a further implementation manner, the muscle fatigue degree is estimated by calculating the spectrum of the EMG signal, wherein the average frequency is used as a fatigue index.

[0021] As a further implementation manner, the activation intensity of the muscle is represented by calculating the absolute value of the electromyography signal and time-averaging it.

[0022] As a further implementation manner, the support force of the upper limb assisting exoskeleton is a dynamic function based on muscle load and working posture; and the support force of the upper limb assisting exoskeleton is adjusted according to the muscle fatigue degree.

[0023] As a further implementation, real-time feedback is provided to a virtual environment in an HMD head-mounted display device according to muscle state, the feedback including: posture correction and feedback, real-time motion guidance.

[0024] In a second aspect, the embodiments of the present application also provide an AR-based wearable upper limb assisting exoskeleton system, comprising:

[0025] The upper limb assisting exoskeleton is mounted with an electromyography sensor, a rotary encoder and an MCU microcontroller, the MCU microcontroller is used to calculate the muscle fatigue degree according to the EMG signal acquired by the electromyography sensor and the shoulder joint angle acquired by the rotary encoder, and transmit the muscle fatigue degree to an HMD head-mounted display device;

[0026] A LiDAR depth sensor is used to scan the working area of the automobile chassis to acquire three-dimensional space data;

[0027] An HMD head-mounted display device is used to acquire image data, combine the image data and the three-dimensional space data and input the combined data into a neural network model, and generate operation instructions after processing.

[0028] The beneficial effects of the present application are as follows:

[0029] The present application acquires real-time data from the electromyography sensor, the LiDAR depth sensor and the rotary encoder, processes the EMG signal, calculates the muscle activation and fatigue degree, etc.; adjusts the support force of the exoskeleton according to the real-time feedback, and provides adjustment suggestions for posture and motion through the AR system; the exoskeleton adjusts the support force according to the electromyography signal and the fatigue degree, and provides instant motion guidance through the AR interface; therefore, through the combination of AR and exoskeleton, the health monitoring and motion evaluation functions can be realized, the body state (such as muscle fatigue degree, joint angle, etc.) is displayed in real time, and the posture or motion is adjusted in time, so as to effectively avoid the problems of excessive fatigue, motion injury, etc. BRIEF DESCRIPTION OF DRAWINGS

[0030] The drawings accompanying the specification of the present application form a part thereof, serve to provide further understanding of the present application, and together with the description of the exemplary embodiments of the present application and the explanation thereof serve to explain the present application, and do not constitute an improper limitation of the present application.

[0031] Figure 1 is a flowchart of the present application according to one or more embodiments;

[0032] Figure 2 is a system block diagram of the present application according to one or more embodiments. DETAILED DESCRIPTION

[0033] It should be noted that the following detailed description is illustrative only and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0034] Embodiment 1

[0035] The embodiment provides an AR-based control method for a wearable upper-limb assisting exoskeleton. The HMD head-mounted display device (AR system) is combined with the upper-limb assisting exoskeleton, virtual elements can be accurately positioned in the three-dimensional space of the real world, and intuitive guidance is provided for the operating personnel. In this way, the operating personnel can obtain auxiliary information in the virtual level without leaving the actual working environment.

[0036] Specifically, as shown in Figure 1 , the method comprises the following steps:

[0037] S1: The operating personnel wears the upper-limb assisting exoskeleton.

[0038] S2: The LiDAR depth sensor emits a laser pulse and receives a reflected signal, scans the automobile chassis working area, obtains three-dimensional space data of each component of the automobile chassis working area in real time, and generates accurate three-dimensional point cloud data; at the same time, the camera of the HMD head-mounted display device shoots an image of the automobile chassis working area.

[0039] The three-dimensional point cloud data is converted into a two-dimensional depth image, and the conversion process is as follows:

[0040] (1) Align the camera coordinate system with the LiDAR coordinate system:

[0041] The three-dimensional point cloud coordinates of the LiDAR are converted into coordinates in the camera coordinate system through an external parameter matrix composed of a rotation matrix R and a translation vector t.

[0042] The formula for converting the point cloud data in the LiDAR coordinate system to the camera coordinate system is:

[0043] P Camera =R·P LiDAR +t (1)

[0044] In formula (1), P LiDAR =[x LiDAR , y LiDAR , z LiDAR ] T represents a three-dimensional point coordinate in the LiDAR coordinate system; P Camera =[x Camera , y Camera , z Camera ] Trepresents the three-dimensional point coordinates in the camera coordinate system; the rotation matrix R represents the rotation relationship from the LiDAR coordinate system to the camera coordinate system, and the rotation matrix is a 3x3 orthogonal matrix, wherein each column is the direction of the coordinate axis after rotation; in the calibration process, the rotation matrix is obtained by aligning the camera and the LiDAR and using the checkerboard calibration board. The translation vector t represents the translation offset from the origin of the LiDAR coordinate system to the origin of the camera coordinate system, which is obtained through the camera calibration process, assuming that the positions of the camera and the LiDAR are known, the translation vector can be calculated by the spatial distance between the two.

[0045] (2) Projecting to image plane:

[0046] The aligned three-dimensional point cloud is projected onto the image plane of the camera using the camera intrinsic matrix, and the x and y coordinates of each three-dimensional point are mapped to the pixel position of the image, while the z coordinate determines the depth value.

[0047] (3) Generating depth image:

[0048] The z value (depth) of each pixel is stored in the depth image, which is a two-dimensional matrix, where each element represents the depth information of the corresponding pixel.

[0049] S3: Combine the above two-dimensional depth image with the image data obtained by the HMD head-mounted display device as input image (composite image) into the convolutional neural network CNN model, the convolutional neural network CNN model extracts local features (such as edges, textures, shapes, etc.) and depth information (such as spatial structure, distance, etc.) in the composite image; use the CNN model to detect target components on the depth map or RGB-D image, compare the target components in the image with the reference image, and determine whether the target components are in normal working condition. If the target component is abnormal, the CNN can locate the exact position of the target component and interface with the three-dimensional point cloud data to further analyze its spatial position and state. Convert the three-dimensional spatial data of the target component to the device display coordinate system, and superimpose the preset standard virtual information library on the target component based on the visual SLAM technology.

[0050] Further, the depth image is combined with the RGB image obtained from the camera in a direct superposition (feature level fusion) manner to form a composite image containing RGB and depth information - RGB-D image: each pixel contains color information (R, G, B) and depth information (D), and the depth image is usually added as the fourth channel to the RGB image. Each pixel of such an image contains the color and depth information of the object, enabling the CNN to simultaneously learn the color features and spatial depth features of the object.

[0051] This embodiment takes a screw as a target component for detailed description:

[0052] The screw two-dimensional depth image is combined with the RGB image, input into a convolutional neural network (CNN) model, local features and depth information of the screw composite image are extracted, the CNN model is used for screw detection on the depth image or RGB-D image, the screw in the image is compared with the screw in the reference image in the normal state, and it is judged whether the screw is in the normal working state. If the position of the screw relative to other components changes or more threads are exposed, it is judged that the screw may be loose; at this time, the neural network CNN model generates a bounding box to mark the accurate position of the screw in the image; in the AR system, a coordinate conversion algorithm is used to convert the three-dimensional space data of the screw into the display coordinate system of the device, and a visual SLAM technology is used to accurately and stably superimpose a preset standard virtual information library above the screw. For the screw, the preset standard virtual information library refers to the loose repair steps, including: the loose repair step "please use a cross screwdriver to tighten the screw", the repair tool suggestion "use an X type cross screwdriver", the operation prompt "insert the screwdriver into the screw and rotate clockwise", and the virtual guide "tighten the screw to the standard of 10 N·m".

[0053] In the embodiment, the AR system uses a coordinate conversion algorithm to convert the three-dimensional space data of the target component into the display coordinate system of the device, and accurately and stably superimposes a preset standard virtual information library above the target component through a visual SLAM technology, and then the upper limb assisting exoskeleton transmits the fatigue state to the AR system interface through a wireless transmission mode; in the above process, in order to ensure that different information provided by the AR system and the upper limb assisting exoskeleton can be simultaneously and accurately displayed in the HMD, it is necessary to coordinate the two types of information through an information fusion and unified display mechanism.

[0054] Specifically, an information hierarchical management method is adopted, different data sources are displayed in layers, the virtual target (such as a screw image, a tool suggestion, etc.) provided by the AR system can be displayed in the main layer of the HMD screen, and the sensor feedback (such as a fatigue prompt of an electromyographic signal or a posture suggestion) of the exoskeleton can be displayed in the secondary information layer. These information can be displayed through transparent, semi-transparent graphics or dynamic pop-up windows, avoiding interference with the line of sight of the worker, while ensuring that necessary information can be displayed at any time.

[0055] S4: The electromyographic sensor on the upper limb assisting exoskeleton acquires an EMG signal, the rotary encoder acquires a shoulder joint angle, the EMG signal and the shoulder joint angle are transmitted to the MCU microcontroller after signal processing, the MCU microcontroller calculates the muscle fatigue degree in real time, and transmits the muscle fatigue degree to the AR system interface through a wireless transmission mode (such as Bluetooth).

[0056] Further, EMG signals need to be processed to calculate muscle load and fatigue level; convert EMG signals to muscle activity level or force output.

[0057] where the muscle activation is expressed as:

[0058]

[0059] In equation (2), A(t) is the muscle activation at time t, EMG(τ) is the EMG signal in time period [t-T, t], and T is the time length of the integration window (e.g., a short window of 50ms to 200ms) that determines the smoothness of the calculation.

[0060] Equation (2) represents the muscle activation intensity by calculating the absolute value of the EMG signal and time-averaging it; a higher value indicates a higher activation state of the muscle, and vice versa.

[0061] As the work time extends, the muscle will fatigue, causing the frequency characteristics of the EMG signal to change. Generally, the low-frequency component (e.g., 20-50Hz) of the EMG signal will increase, while the high-frequency component (e.g., 100-500Hz) will decrease. By calculating the frequency spectrum of the EMG signal, the muscle fatigue level can be estimated. A common fatigue index is the mean frequency, which is calculated as follows:

[0062]

[0063] In equation (3), P(f) is the power spectral density at frequency f (calculated by Fourier transform), f max is the highest frequency of the EMG signal.

[0064] A decrease in MNF indicates an increase in muscle fatigue. If the MNF value decreases too quickly, the upper limb assist exoskeleton can issue a rest prompt or increase the support force based on this signal.

[0065] Further, for a continuous-time signal x(t), its Fourier transform is defined as:

[0066]

[0067] In equation (4), X(f) is the complex representation of the signal at frequency f (including amplitude and phase information), and x(t) is the time-domain signal; e -j2πft is the complex exponential function used to decompose the frequency components of the signal.

[0068] The power spectral density P(f) is the square of the energy of the frequency-domain signal obtained by Fourier transform:

[0069] P(f) = |X(f)|2 (5)

[0070] In formula (5), |X(f)| is the amplitude of X(f).

[0071] S5: Real-time adjustment of operation sequence and action posture according to virtual information of HMD head-mounted display device and muscle fatigue degree, upper limb assistive exoskeleton provides mechanical support and posture monitoring, at the same time, HMD head-mounted display device adjusts the display position and angle of virtual graphics in real time, ensures that the position and state of virtual target components can be clearly seen, avoids fatigue or error caused by improper posture, improves the precision and efficiency of operation.

[0072] The upper limb assistive exoskeleton of the embodiment is used to reduce the physical burden of the operator, so its support force needs to be adjusted in real time according to the muscle load, posture and action, and the control strategy includes dynamic support force adjustment based on electromyographic signals.

[0073] The support force of the exoskeleton is a dynamic function based on muscle load and working posture, expressed as:

[0074]

[0075] In formula (6), F exo is the support force provided by the exoskeleton, k is the gain coefficient of the exoskeleton, which determines the degree of response of the exoskeleton to muscle load; θ is the angle between the operator and the support point of the exoskeleton (the angle of the shoulder).

[0076] When the muscle activation increases, the exoskeleton needs to provide greater support force. In addition, the angle parameter takes into account the posture of the operator, ensuring that the support force can be adjusted according to the working angle.

[0077] As the working time is prolonged, the operator may appear fatigue. The exoskeleton can adjust its response according to the fatigue EMG signal. For example, when fatigue is detected, the support force can be adjusted using the following formula:

[0078]

[0079] In formula (7), F exo_adjusted is the exoskeleton support force after fatigue adjustment, MNF is the current average frequency (indicating the degree of fatigue), MNF0 is the initial average frequency (indicating the state without fatigue); α is the adjustment coefficient, controlling the influence of fatigue on the support force.

[0080] Formula (7) shows that as fatigue increases, the exoskeleton will provide more support force to help the operator reduce muscle load and prolong working time.

[0081] In the AR system, EMG signals can be used to provide real-time feedback on the muscle activity of the worker, helping them adjust their posture and movements. The virtual environment in the AR system can provide real-time feedback based on the muscle state of the worker, guiding them to adjust their behavior.

[0082] 1) Posture correction and feedback:

[0083] The AR system uses a posture correction algorithm to ensure that the worker's posture meets the requirements, which can use the following calculation method:

[0084] Error(t) = ||θ ideal -θ real (t)|| (8)

[0085] In equation (8), Error(t) is the error between the current posture of the worker and the ideal posture, θ ideal is the ideal target posture (angle of the shoulder), and θ real (t) is the actual posture angle of the worker at a certain time.

[0086] This error can be used to drive the virtual feedback of the AR system, such as displaying a virtual alignment line or a guide prompt, prompting the worker to adjust their posture.

[0087] 2) Real-time movement guidance

[0088] By combining EMG signals with the AR system, real-time movement guidance can be achieved. For example, when the worker's posture is detected to deviate from the ideal state, the AR system can provide real-time correction through virtual task markers or guide lines:

[0089] Guide_adjustment(t) = β·(θ ideal -θ real (t)) (9)

[0090] In equation (9), Guide_adjustment(t) is the real-time movement adjustment prompt of the AR system; β is the adjustment factor, which determines the strength of the prompt.

[0091] This embodiment is based on AR technology, which helps workers locate maintenance components or installation positions through visual guidance and virtual markers, improves operation accuracy, reduces errors, and improves work efficiency; combined with AR technology, real-time operation guidance and learning prompts can be obtained through the AR interface without complex training courses; quickly master the operation points through virtual demonstration, thereby reducing training costs and shortening adaptation time. AR combined with exoskeletons can achieve health monitoring and movement evaluation functions, providing real-time display of body status (such as muscle fatigue, joint angle, etc.), and timely prompts to adjust posture or movements, thereby effectively avoiding excessive fatigue, sports injuries, and other problems.

[0092] Embodiment 2:

[0093] The embodiment provides an AR-based wearable upper limb assisting exoskeleton system, as shown in the figure, comprising an upper limb assisting exoskeleton, a LiDAR depth sensor and an HMD head-mounted display device, wherein the upper limb assisting exoskeleton is installed with an electromyography sensor, a rotary encoder and an MCU microcontroller. Figure 2

[0094] The electromyography sensor is a sensor for measuring and recording muscle electrical activity, and can detect the electrical signal generated when the muscle contracts; can reflect the working state, load and fatigue degree of the muscle. The rotary encoder is used for directly measuring the angle θ change of the shoulder joint. The MCU microcontroller is used for calculating the muscle fatigue degree according to the EMG signal obtained by the electromyography sensor and the shoulder joint angle obtained by the rotary encoder, and transmitting the muscle fatigue degree to the HMD head-mounted display device.

[0095] The LiDAR depth sensor is used for scanning the working environment and collecting three-dimensional space data of each part of the automobile chassis in real time. The HMD head-mounted display device is a display worn on the head, which can directly see the image of the virtual world based on AR technology, while maintaining the interaction with the real world. AR can be superimposed in the exoskeleton through graphics, guiding the workers to accurately find the parts (target parts) that need to be repaired or installed, and displaying the best operation path in real time.

[0096] The graphic superposition refers to a technology of superimposing virtual graphic information into the real environment in real time, which enables the workers to see important guiding information such as the position, state and operation path of the parts that need to be repaired or installed through the augmented reality interface in the field of vision. This technology is not simply presenting information on the display screen, but through the combination of the head-mounted display device (HMD) and the exoskeleton, it can accurately position the virtual elements in the three-dimensional space of the real world, thereby providing intuitive guidance for the workers; in this way, the workers can obtain auxiliary information in the virtual level without leaving the actual working environment.

[0097] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for controlling an AR-based wearable upper limb assistive exoskeleton, characterized in that, include: Acquire three-dimensional spatial data and image data of the working area of ​​the vehicle chassis; The depth image and image data after the 3D spatial data are converted are input into a neural network (CNN) model to extract feature information; The target component image is compared with a reference image under normal conditions to determine the working status of the target component; when the working status of the component is abnormal, the precise location of the target component in the image is obtained based on a neural network (CNN) model. The three-dimensional spatial data of the target component is converted into the device display coordinate system, and a preset standard virtual information library is superimposed on the target component based on visual SLAM technology. EMG signals and shoulder joint angles are acquired to calculate muscle fatigue levels. The EMG signals are acquired by an electromyography sensor, and the shoulder joint angles are acquired by a rotary encoder. After signal processing, the EMG signals and shoulder joint angles are transmitted to an MCU microcontroller. The MCU microcontroller calculates the degree of muscle fatigue and transmits the degree of muscle fatigue to the AR system interface wirelessly. Muscle fatigue is estimated by calculating the spectrum of EMG signals, where the average frequency is used as a fatigue index; the activation intensity of muscles is represented by calculating the absolute value of the electromyographic signal and averaging it over time. The upper limb-assisted exoskeleton executes operation steps according to a standard virtual information database, and adjusts the operation sequence and movement posture in real time based on virtual information and muscle fatigue levels. The supporting force of the upper limb-assisted exoskeleton is a dynamic function based on muscle load and working posture, expressed as: ； in, F exo The support is provided by the exoskeleton. k It is the gain coefficient of the exoskeleton. A ( t ) is in time t The muscle activation level at any given moment, where θ is the angle between the worker and the exoskeleton support point; Adjust the support force of the upper limb assistive exoskeleton according to the level of muscle fatigue; when fatigue is detected, adjust the support force using the following formula: ； in, F exo_adjusted It is the exoskeleton support force after fatigue adjustment. MNF It is the current average frequency. MNF 0 is the initial average frequency; α is the adjustment coefficient, which controls the effect of fatigue on the support force.

2. The AR-based wearable upper limb assistive exoskeleton control method according to claim 1, characterized in that, The method for acquiring the three-dimensional spatial data is as follows: A LiDAR depth sensor is used to scan the working area of ​​the car chassis to acquire three-dimensional spatial data of the target component in real time and generate three-dimensional point cloud data.

3. The AR-based wearable upper limb assistive exoskeleton control method according to claim 1, characterized in that, The image data is acquired in the following way: Acquired from HMD head-mounted display devices.

4. The AR-based wearable upper limb assistive exoskeleton control method according to claim 1, characterized in that, The three-dimensional spatial data of the target component is converted into the device display coordinate system based on the coordinate transformation algorithm.

5. The AR-based wearable upper limb assistive exoskeleton control method according to claim 1, characterized in that, The system provides real-time feedback to the virtual environment in the HMD head-mounted display based on muscle status. This feedback includes posture correction and feedback, and real-time motion guidance.

6. An AR-based wearable upper limb assistive exoskeleton system, characterized in that, include: The upper limb assistive exoskeleton is equipped with an electromyography (EMG) sensor, a rotary encoder, and an MCU microcontroller. The MCU microcontroller is used to calculate the degree of muscle fatigue based on the EMG signal obtained by the EMG sensor and the shoulder joint angle obtained by the rotary encoder, and transmits the degree of muscle fatigue to the HMD head-mounted display device. Muscle fatigue is estimated by calculating the spectrum of EMG signals, where the average frequency is used as a fatigue index; the activation intensity of muscles is represented by calculating the absolute value of the electromyographic signal and averaging it over time. LiDAR depth sensors are used to scan the working area of ​​a car chassis to obtain three-dimensional spatial data. HMD head-mounted display devices are used to acquire image data, combine the image data and three-dimensional spatial data, input them into a neural network model, and generate operation commands after processing. The upper limb-assisted exoskeleton executes operation steps according to a standard virtual information database, and adjusts the operation sequence and movement posture in real time based on virtual information and muscle fatigue levels. The supporting force of the upper limb-assisted exoskeleton is a dynamic function based on muscle load and working posture, expressed as: ； in, F exo The support is provided by the exoskeleton. k It is the gain coefficient of the exoskeleton. A ( t ) is in time t The muscle activation level at any given moment, where θ is the angle between the worker and the exoskeleton support point; Adjust the support force of the upper limb assistive exoskeleton according to the level of muscle fatigue; when fatigue is detected, adjust the support force using the following formula: ； in, F exo_adjusted It is the exoskeleton support force after fatigue adjustment. MNF It is the current average frequency. MNF 0 is the initial average frequency; α is the adjustment coefficient, which controls the effect of fatigue on the support force.

Images