An astronaut in-cabin operation and display control system for dual-arm on-orbit operation in space

Abstract

The invention relates to an astronaut in-cabin operation and display control system for on-orbit operation of dual-arms in space, belonging to the technical field of on-orbit operation of dual-arms in space; the system comprises a multi-mode operation control module, a dynamic information management module and a real-time simulation and display module; the mode operation control module comprises a working mode selection module, a ground operation management module, an autonomous task management module and a human-computer interaction operation module; the dynamic information management module further comprises a mechanical arm dynamic bus information management module and a mechanical arm information processing module; the real-time simulation and display module further comprises a mechanical arm simulation training module and a mechanical arm motion parameter and image display module; the human-computer interaction operation module comprises a panel operation module, a rotation handle operation module and a translation handle operation module; the invention realizes the management of multi-point access of dual-arms in space through dynamic information flow management, autonomous task management and multi-data source fusion human-computer interaction design, and provides astronauts with multi-mode operation of the space mechanical arm in the cabin.

Description

Technical Field

[0001] The present invention belongs to the technical field of space dual-arm on-orbit operation, and relates to an astronaut in-cabin operation and display control system for space dual-arm on-orbit operation. Background Art

[0002] The space manipulators on the space station are responsible for cabin transfer and docking, hovering vehicle capture and auxiliary docking, supporting astronauts' extravehicular activities, extravehicular cargo handling and status inspection, exposure platform experimental payload care, and optical platform care. The space station is equipped with two sets of manipulators, a large manipulator in the core cabin (referred to as the big arm, with a 10-meter armspan), and a small manipulator in the experimental cabin (referred to as the small arm, with a 5-meter armspan). The big arm performs large-load and large-range space operations, and the small arm performs high-precision and fine operations. The two sets of manipulators can move to different positions in different cabins of the space station, and can complete flexible transfers, expanding the reachable space of the manipulators.

[0003] Currently, the system cross-section for on-orbit operation of space robotic arms is single and the logic is backward. It does not integrate dynamic information and autonomous tasks, has poor human-computer interaction, and has a single operation mode. The above defects urgently need to be improved to meet the needs of further complex operations. Summary of the invention

[0004] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and propose an astronaut in-cabin operation and display and control system for on-orbit operation of dual-arm space arms, which realizes the management of multi-point access of dual-arm space arms through dynamic information flow management, autonomous task management and multi-data source fusion human-computer interaction design, and provides astronauts with multi-mode operation of space robotic arms in the cabin.

[0005] The solution of the present invention is:

[0006] An astronaut cabin operation and display control system for dual-arm on-orbit operation in space, including a multi-mode operation control module, a dynamic information management module and a real-time simulation and display module; the mode operation control module includes a working mode selection module, a ground operation management module, an autonomous task management module, and a human-computer interaction operation module; the dynamic information management module includes a mechanical arm dynamic bus information management module and a mechanical arm information processing module; the real-time simulation and display module includes a mechanical arm simulation training module and a mechanical arm motion parameter and image display module;

[0007] Working mode selection module: receiving the manipulator control data from the ground operation center; selecting the operation mode of the manipulator, which includes ground operation mode, autonomous operation mode, on-orbit operation mode and on-orbit simulation training operation mode; when the ground operation mode is selected, extracting the ground control data from the manipulator control data, and sending the ground control data to the ground operation management module; when the autonomous operation mode is selected, extracting the autonomous control data from the manipulator control data, and sending the autonomous control data to the autonomous task management module; when the on-orbit operation mode is selected, issuing work instructions to the human-computer interaction operation module;

[0008] Human-machine interactive operation module: receives the working instructions from the working mode selection module and turns on the machine; inputs the robot arm motion control instructions and sends the robot arm motion control instructions to the robot arm dynamic bus information management module; receives the six-dimensional force data from the robot arm information processing module and sets the damping feedback of the operation module according to the six-dimensional force data;

[0009] Ground operation management unit: receives ground control data from the working mode selection module, extracts the specified manipulator information, manipulator working mode information, teleoperation information and manipulator motion operation information from the ground control data, generates manipulator motion control instructions, and sends the manipulator motion control instructions to the manipulator dynamic bus information management module;

[0010] Autonomous task management module: receives autonomous control data from the working mode selection module; calls pre-stored typical autonomous tasks according to autonomous control data or on-orbit control data, generates robot arm motion control instructions, and sends the robot arm motion control instructions to the robot arm dynamic bus information management module; each step of the autonomous task operation process is transmitted to the panel operation module for astronauts to monitor the autonomous task execution process on-orbit;

[0011] The robot arm dynamic bus information management module: receives the robot arm motion control instructions from the human-machine interaction operation module; or receives the robot arm motion control instructions from the ground operation management unit; or receives the robot arm motion control instructions from the autonomous task management module; sends the robot arm motion control instructions to the corresponding robot arm through the bus, operates it to complete the corresponding movement, and generates the real video image data of the robot arm; at the same time, sends the robot arm motion control instructions to the robot arm simulation training module; receives the telemetry data and posture measurement data of the moving robot arm through the bus, and sends the telemetry data and posture measurement data to the robot arm information processing module;

[0012] Robotic arm information processing module: receives telemetry data and posture measurement data from the robot arm dynamic bus information management module; sends the telemetry data to the ground operation center for monitoring; at the same time, sends the telemetry data to the robot arm simulation training module for three-dimensional simulation of the robot arm; sends the posture measurement data to the robot arm motion parameter and image display module; parses the six-dimensional force data based on the telemetry data and posture measurement data, and sends the six-dimensional force data to the human-computer interaction operation module;

[0013] Robotic arm simulation training module: receives telemetry data from the robotic arm information processing module, simulates the robotic arm 3D model to the position of the telemetry data, and displays the motion state of the robotic arm; receives the robotic arm motion control instructions from the robotic arm dynamic bus information management module, drives the robotic arm 3D model for real-time simulation according to the robotic arm motion control instructions, generates the robotic arm simulation video and image data, and sends the robotic arm simulation video and image data to the robotic arm motion parameter and image display module for display; at the same time, sends the robotic arm simulation video and image data to the ground operation center;

[0014] Robotic arm motion parameter and image display module: receives the simulation video and image data of the robotic arm three-dimensional model from the robotic arm simulation training module, and displays the simulation situation; receives the posture measurement data from the robotic arm information processing module; obtains the real video image data of the robotic arm movement; displays the real motion situation of the robotic arm through the posture measurement data and the real video image data of the robotic arm movement.

[0015] In the above-mentioned astronaut in-cabin operation and display control system for on-orbit dual-arm space operation, the robotic arm control data includes ground control data and autonomous control data.

[0016] In the above-mentioned astronaut in-cabin operation and display and control system for space dual-arm on-orbit operation, the robotic arm working mode information includes single-arm independent work, multi-arm combined work, and multi-arm collaborative work; the remote operation information includes ground handle remote operation and ground command remote operation.

[0017] In the above-mentioned astronaut in-cabin operation and display and control system for on-orbit dual-arm space operation, the typical autonomous tasks include pre-programmed movement of the robotic arm, visual closed-loop movement of the robotic arm, autonomous cabin transfer of the robotic arm, and emergency return of the robotic arm; and the typical autonomous tasks are dynamically updated in real time through annotation.

[0018] In the above-mentioned astronaut in-cabin operation and display control system for on-orbit operation of dual-arms in space, the human-computer interaction operation module includes a panel operation module, a rotation handle operation module, and a translation handle operation module; the human-computer interaction operation module inputs the robotic arm motion control instructions through one or more of the panel operation module, the rotation handle operation module, and the translation handle operation module.

[0019] In the above-mentioned astronaut in-cabin operation and display and control system for on-orbit operation of dual-arms in space, the panel operation module receives the astronaut's operation instructions for the robotic arm. The astronaut performs all robotic arm operations such as robotic arm motion parameter setting, robotic arm motion operation, end capture and release, typical autonomous task operation, robotic arm stand-alone management, translation and rotation handle parameter setting management in the cabin through shortcut hard keys and graphic page soft keys. The robotic arm control instructions are generated on-orbit. All robotic arm control instructions are sent to the robotic arm dynamic bus information management module only after verification and secondary confirmation. On the other hand, the operation process information is also synchronously displayed in the panel operation module to provide prompt information for the astronauts. The process information is also sent to the ground operation center as robotic arm telemetry data. The ground operation center can mirror and reproduce the astronauts' on-orbit operation actions for monitoring and evaluation.

[0020] In the above-mentioned astronaut in-cabin operation and display control system for on-orbit dual-arm space operation, the motion parameters of the robotic arm include load, speed, and acceleration gear.

[0021] In the above-mentioned astronaut in-cabin operation and display and control system for on-orbit operation of dual-arms in space, the rotating handle operation module provides a direct operation method for astronauts. By watching the video image of the robotic arm, the astronauts directly operate the rotating handle to adjust the pitch, yaw and rotation direction posture of the robotic arm end, adjust the forward and reverse rotation posture of the single joint, and adjust the pitch and yaw posture of the camera gimbal. The rotating handle module receives the astronaut's operation of the handle, and generates control data for controlling the posture of the robotic arm end, single joint, and camera gimbal according to the set speed and load gear, and sends it to the robotic arm dynamic bus information management module.

[0022] In the above-mentioned astronaut in-cabin operation and display and control system for on-orbit operation of dual-arms in space, the translation handle operation module provides a direct operation method for astronauts. By watching the video image of the robotic arm, the astronauts directly operate the translation handle to adjust the position of the end of the robotic arm in the front and back, left and right, and up and down directions. The translation handle module receives the astronaut's operation of the handle, and generates control data for controlling the position of the end of the robotic arm according to the set speed and load gear, and sends it to the robotic arm dynamic bus information management module.

[0023] In the above-mentioned astronaut in-cabin operation and display and control system for on-orbit dual-arm space operation, the rotation handle operation module and the translation handle operation module both receive the six-dimensional force data sent by the robotic arm information processing module, and superimpose the force feedback information on the handle damping force to provide a smooth force feedback feeling for the astronaut operation.

[0024] The beneficial effects of the present invention compared with the prior art are:

[0025] (1) The manipulator operation and display control system of the present invention is composed of an on-orbit operating console with man-machine interaction functions such as a force feedback handle and a liquid crystal display panel, an intelligent display with visual images and telemetry information, a simulation notebook with a manipulator simulation verification function, and a dynamically switchable manipulator bus system, etc. It has a multi-operation mode redundant backup function, can provide on-orbit training, and can provide man-machine interaction operations such as visual feedback, force feedback, and auditory feedback;

[0026] (2) The robot arm operation and display control system of the present invention is shared by the upper arm and the lower arm, providing a unified operation process for astronauts, which is simple and convenient, and has the autonomous operation function of performing complex collaborative work, reducing the operational instability caused by people in the loop;

[0027] (3) The present invention provides a communication network design based on the integration of space and ground. When astronauts are operating or training on orbit, information such as the status of the robotic arm and the operating status of the astronauts are transmitted to the ground in real time. The ground can restore the on-orbit scene and the visual scene of the robotic arm to conduct effective monitoring and timely intervention.

[0028] (4) The robotic arm of the present invention can "crawl" between modules while in orbit, and the dual robotic arms have multiple on-orbit working modes. Collaborative work and combined work have strict requirements on the operation timing. Through dynamic bus information management, the robotic arm can flexibly adapt to variable bus topology structures. Astronauts can operate the large arms, small arms and combined arms in different modules without any switching feeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the in-cabin operation and display control system of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below in conjunction with the embodiments.

[0031] The present invention provides an astronaut in-cabin operation and display and control system for on-orbit operation of dual-arms in space. A unified in-cabin operation and display and control system is designed for astronauts to operate multiple robotic arms, which can provide visual feedback and force feedback to increase the sense of presence; it can operate in multiple modes such as independent work, combined work, collaborative work, and autonomous work for different objects such as large arms and small arms; it can also be used for on-orbit training of robotic arm operation, and a parameter graphical display page matching it is designed.

[0032] Astronaut cabin operation and display control system, such as Figure 1 As shown, it specifically includes a multi-mode operation control module, a dynamic information management module and a real-time simulation and display module; the mode operation control module includes a working mode selection module, a ground operation management module, an autonomous task management module, and a human-computer interaction operation module; the dynamic information management module includes a robotic arm dynamic bus information management module and a robotic arm information processing module; the real-time simulation and display module includes a robotic arm simulation training module and a robotic arm motion parameter and image display module.

[0033] Working mode selection module: Receives the manipulator control data from the ground operation center; the manipulator control data includes ground control data and autonomous control data. Selects the operation mode of the manipulator, which includes ground operation mode, autonomous operation mode, on-orbit operation mode, and on-orbit simulation training operation mode. When the ground operation mode is selected, the ground control data is extracted from the manipulator control data, and the ground control data is sent to the ground operation management module; when the autonomous operation mode is selected, the autonomous control data is extracted from the manipulator control data, and the autonomous control data is sent to the autonomous task management module; when the on-orbit operation mode is selected, a work instruction is issued to the human-computer interaction operation module.

[0034] Human-computer interaction operation module: receives the working instructions from the working mode selection module and turns on the machine; inputs the robot arm motion control instructions and sends the robot arm motion control instructions to the robot arm dynamic bus information management module; receives the six-dimensional force data from the robot arm information processing module, and sets the damping feedback of the operation module according to the six-dimensional force data.

[0035] Ground operation management unit: receives ground control data from the working mode selection module, extracts the specified manipulator information, manipulator working mode information, teleoperation information and manipulator motion operation information from the ground control data, generates manipulator motion control instructions, and sends the manipulator motion control instructions to the manipulator dynamic bus information management module. The manipulator working mode information includes single-arm independent work, multi-arm combined work, and multi-arm collaborative work; the teleoperation information includes ground handle teleoperation and ground command teleoperation.

[0036] Autonomous task management module: receives autonomous control data from the working mode selection module; calls pre-stored typical autonomous tasks according to autonomous control data or on-orbit control data, generates robotic arm motion control instructions, and sends robotic arm motion control instructions to the robotic arm dynamic bus information management module; each step of the autonomous task operation process is transmitted to the panel operation module for astronauts to monitor the autonomous task execution process on orbit.

[0037] Typical autonomous tasks include pre-programmed movement of the robot arm, visual closed-loop movement of the robot arm, autonomous compartment transfer of the robot arm, and emergency return of the robot arm; and typical autonomous tasks are dynamically updated in real time through annotation.

[0038] Robotic arm dynamic bus information management module: receives the robotic arm motion control instructions from the human-computer interaction operation module; or receives the robotic arm motion control instructions from the ground operation management unit; or receives the robotic arm motion control instructions from the autonomous task management module; sends the robotic arm motion control instructions to the corresponding robotic arm through the bus, operates it to complete the corresponding movement, and generates real video image data of the robotic arm; at the same time, sends the robotic arm motion control instructions to the robotic arm simulation training module; receives the telemetry data and posture measurement data of the moving robotic arm through the bus, and sends the telemetry data and posture measurement data to the robotic arm information processing module.

[0039] Robotic arm information processing module: receives telemetry data and posture measurement data from the robotic arm dynamic bus information management module; sends the telemetry data to the ground operation center for monitoring; at the same time, sends the telemetry data to the robotic arm simulation training module for use in three-dimensional simulation of the robotic arm; sends the posture measurement data to the robotic arm motion parameter and image display module; parses the six-dimensional force data based on the telemetry data and posture measurement data, and sends the six-dimensional force data to the human-computer interaction operation module.

[0040] Robotic arm simulation training module: receives telemetry data from the robotic arm information processing module, simulates the robotic arm three-dimensional model to the position of the telemetry data, and displays the motion state of the robotic arm; receives the robotic arm motion control instructions from the robotic arm dynamic bus information management module, drives the robotic arm three-dimensional model for real-time simulation according to the robotic arm motion control instructions, generates the robotic arm simulation video and image data, and sends the robotic arm simulation video and image data to the robotic arm motion parameter and image display module for display; at the same time, sends the robotic arm simulation video and image data to the ground operation center.

[0041] Robotic arm motion parameter and image display module: receives the simulation video and image data of the robotic arm three-dimensional model from the robotic arm simulation training module, and displays the simulation situation; receives the posture measurement data from the robotic arm information processing module; obtains the real video image data of the robotic arm movement; displays the real motion situation of the robotic arm through the posture measurement data and the real video image data of the robotic arm movement.

[0042] In the present invention, the human-computer interaction operation module includes a panel operation module, a rotation handle operation module, and a translation handle operation module; the human-computer interaction operation module inputs the robot arm motion control instructions through one or more of the panel operation module, the rotation handle operation module, and the translation handle operation module.

[0043] The panel operation module receives the astronauts' operation instructions for the manipulator. The astronauts use the quick hard keys and graphic page soft keys to perform all manipulator operations in the cabin, such as manipulator motion parameter settings, manipulator motion operations, end capture and release, typical autonomous mission operations, manipulator stand-alone management, and translation and rotation handle parameter settings management. The manipulator control instructions are generated on-orbit. All manipulator control instructions are sent to the manipulator dynamic bus information management module only after verification and secondary confirmation. On the other hand, the operation process information is also synchronously displayed in the panel operation module to provide prompt information for the astronauts. The process information is also sent to the ground operation center as the manipulator telemetry data. The ground operation center can mirror the astronauts' on-orbit operation actions for monitoring and evaluation. Among them, the manipulator motion parameters include load, speed, and acceleration gear.

[0044] The handle rotation operation module provides a direct operation method for astronauts. By watching the video images of the robotic arm, the astronauts directly operate the handle to adjust the pitch, yaw and rotation direction of the robotic arm end, adjust the forward and reverse rotation posture of the single joint, and adjust the pitch and yaw posture of the camera gimbal. The handle rotation module receives the astronaut's operation of the handle, and generates control data for controlling the robotic arm end, single joint, and camera gimbal posture according to the set speed and load gear, and sends it to the robotic arm dynamic bus information management module.

[0045] The translation handle operation module provides a direct operation method for astronauts. By watching the video image of the robotic arm, the astronauts directly operate the translation handle to adjust the position of the end of the robotic arm in the front and back, left and right, and up and down directions. The translation handle module receives the astronaut's operation of the handle, and generates control data for the position of the end of the robotic arm according to the set speed and load gear, and sends it to the robotic arm dynamic bus information management module.

[0046] Both the rotation handle operation module and the translation handle operation module receive the six-dimensional force data sent by the robotic arm information processing module, and superimpose the force feedback information on the handle damping force to provide a smooth force feedback feeling for astronauts' operation.

[0047] The present invention is applicable to four operation modes: ground operation, on-orbit operation, autonomous operation and on-orbit simulation training operation; it includes four types of information flows: operation control data flow, video and telemetry data flow, simulation training data flow, and posture data flow;

[0048] Operation Mode:

[0049] The operation and display control system supports the coordinated operation between astronauts and the ground center. Four operation modes are set for this purpose. Astronauts can flexibly operate multiple robotic arms in orbit. The four operation modes include ground operation, on-orbit operation, autonomous operation and on-orbit simulation training operation mode. The four operation modes can be carried out in coordination.

[0050] Ground operation mode: In this operation mode, the robot arm control command is issued by the ground operation center. The astronauts can coordinate and determine the robot arm movement status by monitoring the real-time movement data, fault monitoring data and video image information of the robot arm. In this mode, the astronauts can press the robot arm movement temporary key and the robot arm movement stop key, or switch the operation mode and take over the robot arm operation in orbit. In addition, in this mode, the astronauts' other buttons and handle action inputs are not responded to.

[0051] On-orbit operation mode: In this operation mode, astronauts are provided with various human-computer interaction operations such as buttons, touch screens, and handles. By monitoring the real-time motion data, fault monitoring data, and video image information of the robotic arm, astronauts can select any robotic arm, operate the end movement of the robotic arm, joint rotation, etc., and perform seven-degree-of-freedom motion operations of the robotic arm. In particular, astronauts can make minor adjustments to the position of the end of the robotic arm in three directions and the posture of three angles by operating the end point on the touch screen to improve the accuracy of the robotic arm in place; they can also use the force feedback handle to provide astronauts with a sense of presence, perform large-scale transfer operations on the robotic arm, and adjust the position and posture to increase operational safety. In this mode, astronauts' on-orbit operation instructions can be transmitted to the ground operation center in real time. The ground operation center can mirror and reproduce the steps of the astronauts' on-orbit operations in real time, and the ground can take over the operation of the robotic arm at any time.

[0052] Autonomous operation mode: In this operation mode, the astronaut activates the corresponding autonomous task button on the operation panel. The system makes judgments according to the set steps and key conditions, and activates the corresponding robotic arm to autonomously complete the corresponding task, such as pre-programmed movement, visual closed-loop movement, autonomous compartment transfer of the robotic arm, emergency return of the robotic arm, etc. During the autonomous task, the system receives information from the attitude and orbit control subsystem in real time, and issues spacecraft jet prohibition or permission instructions to ensure that the position and attitude of the spacecraft remain normal during the movement of the robotic arm. After the mission is started, the astronaut can monitor the real-time motion data, fault monitoring data and video image information of the robotic arm, monitor the completion of the mission, and intervene when necessary. This mode reduces the human uncertainty brought by astronauts in the operation loop and is suitable for the operation of known fixed tasks. In this mode, the telemetry and video images of the robotic arm can be transmitted to the ground operation center in real time. The ground operation center can monitor the deviation of the robotic arm movement from the preset movement, and the ground can take over the operation of the robotic arm at any time.

[0053] On-orbit simulation training operation mode: In this operation mode, the manipulator control command is sent to the manipulator simulation training computer. The simulation computer will select and drive the corresponding manipulator 3D simulation model to move according to the command, generate a virtual scene to realize the virtual operation and training of astronauts, and the display will display the relevant telemetry data, fault monitoring data and video images of the simulated manipulator in real time for astronauts to judge the operation status of the simulated operation. The manipulator simulation computer sends a simulated six-dimensional force data packet to the operating handle to simulate the handle force feedback under virtual training, so as to provide astronauts with a sense of presence in operation. In this mode, the operation data of the astronauts' simulated training and the manipulator simulation data will be transmitted to the ground operation center, so that the ground can understand the astronauts' operation training in real time; at the same time, the telemetry and image data of the real manipulator are still transmitted to the ground operation center to monitor the on-orbit status of the manipulator in real time. The ground can switch the operation mode at any time to take over the operation of the manipulator. When the ground takeover is completed and switched back to the on-orbit simulation training operation mode, the astronauts can continue the last training or restart the training.

[0054] Information flow: Four types of information flow: operation control data flow, video and telemetry data flow, simulation training data flow, and posture data flow.

[0055] Operation control data flow: The operation control flow can be generated on-orbit from the panel, buttons, and handles through the astronaut human-computer interaction operation interface; the ground operation center can also send control data, which is forwarded to the multi-mode operation control module by the digital management system through the measurement and control channel; after the multi-mode operation control module parses the robot arm motion control instructions, it sends them to the dynamic information management module for processing. The dynamic information management module automatically identifies the operated robot arm and the bus where the robot arm is currently located, and automatically identifies the fixed end and free end of the robot arm, and sends the operation control instructions to the designated robot arm on the corresponding bus. According to the corresponding robot arm combination, collaborative or independent working mode settings, the corresponding robot arm receives the instructions and responds to produce real actions. At the same time, all the data of the astronaut's operation on orbit will be sent to the ground operation center in real time through the telemetry channel, so that the ground operation center can view the process of the astronaut operating the robot arm on orbit in real time.

[0056] Video and telemetry data stream: On the one hand, the camera on the robotic arm transmits the video stream to the real-time simulation and display module through Ethernet, providing astronauts with intuitive operation display images. The video stream is also transmitted to the ground through the measurement and control network, and the ground operation center displays the image synchronously. On the other hand, the dynamic information management module polls and judges the robotic arm bus communication interruption signal to determine the online status of the upper and lower arms on the dynamic bus, and regularly determines the source of the robotic arm telemetry data. The robotic arm telemetry data is then sent to the multi-mode operation control module, which sends the data to the digital tube system and transmits it to the ground operation center through the measurement and control network for display. It is also sent to the robotic arm simulation training module to drive the digital robotic arm for simulation display. After processing, the key robotic arm motion posture data is extracted and sent to the robotic arm motion parameters and image display module for real-time display, providing astronauts with a real-time monitoring page.

[0057] Simulation training data stream: This data stream is generated in the on-orbit simulation training operation mode. In the on-orbit simulation training operation mode, astronauts generate robot arm simulation training operation instructions through buttons, touch screens, and handle operations. The instructions are not sent to the real robot arm, but are sent to the measurement and control network through the Ethernet interface and forwarded to the robot arm simulation training module. The robot arm simulation training module will drive the three-dimensional simulation model of the robot arm to move according to the instructions, and generate a virtual scene to realize the astronaut's virtual operation. In conjunction with the movement scene of the robot arm, the robot arm simulation training module simultaneously generates robot arm simulation video images and robot arm simulation training data. On the one hand, they are sent to the robot arm motion parameters and image display module through Ethernet for astronauts to monitor their status during on-orbit operation training; on the other hand, the robot arm simulation training data and robot arm simulation video image data will be transmitted to the ground operation center through the measurement and control network, which can evaluate the astronauts' on-orbit training effect.

[0058] Posture data flow: In the autonomous operation mode, the posture data of the robot arm camera is solved by the dynamic information management module. When entering the visual closed-loop mode, the robot arm information processing module sends the solved real-time posture measurement data to the corresponding robot arm, and controls the robot arm to further perform visual servo movement until the movement reaches the predetermined distance. The robot arm exits the visual servo control mode, and the dynamic information management module no longer solves the posture data of the robot arm camera, but only transmits it as telemetry data to the ground control center through the measurement and control network.

[0059] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. An astronaut in-cabin operation and display control system for dual-arm on-orbit space operation, characterized in that: It includes a multi-mode operation control module, a dynamic information management module and a real-time simulation and display module; the mode operation control module includes a working mode selection module, a ground operation management module, an autonomous task management module, and a human-computer interaction operation module; the dynamic information management module includes a mechanical arm dynamic bus information management module and a mechanical arm information processing module; the real-time simulation and display module includes a mechanical arm simulation training module and a mechanical arm motion parameter and image display module; Working mode selection module: receives the manipulator control data from the ground operation center; selects the operation mode of the manipulator, which includes ground operation mode, autonomous operation mode, on-orbit operation mode and on-orbit simulation training operation mode; When the ground operation mode is selected, the ground control data is extracted from the manipulator control data, and the ground control data is sent to the ground operation management module; when the autonomous operation mode is selected, the autonomous control data is extracted from the manipulator control data, and the autonomous control data is sent to the autonomous task management module; When the on-orbit operation mode is selected, a work instruction is issued to the human-machine interaction operation module; Human-computer interaction operation module: receives the working instructions from the working mode selection module and turns on the machine; Inputting the robot arm motion control instruction and sending the robot arm motion control instruction to the robot arm dynamic bus information management module; Receive six-dimensional force data from the information processing module of the robot arm, and set the damping feedback of the operation module according to the six-dimensional force data; Ground operation management unit: receives ground control data from the working mode selection module, extracts the specified manipulator information, manipulator working mode information, teleoperation information and manipulator motion operation information from the ground control data, generates manipulator motion control instructions, and sends the manipulator motion control instructions to the manipulator dynamic bus information management module; Autonomous task management module: receives autonomous control data from the working mode selection module; calls pre-stored typical autonomous tasks according to autonomous control data or on-orbit control data, generates robot arm motion control instructions, and sends the robot arm motion control instructions to the robot arm dynamic bus information management module; Each step of the autonomous mission operation process is transmitted to the panel operation module so that astronauts can monitor the autonomous mission execution process on orbit; The robot arm dynamic bus information management module: receives the robot arm motion control instructions from the human-machine interaction operation module; or receives the robot arm motion control instructions from the ground operation management unit; or receives the robot arm motion control instructions from the autonomous task management module; sends the robot arm motion control instructions to the corresponding robot arm through the bus, operates it to complete the corresponding movement, and generates the real video image data of the robot arm; at the same time, sends the robot arm motion control instructions to the robot arm simulation training module; receives the telemetry data and posture measurement data of the moving robot arm through the bus, and sends the telemetry data and posture measurement data to the robot arm information processing module; Robotic arm information processing module: receives telemetry data and posture measurement data from the robot arm dynamic bus information management module; sends the telemetry data to the ground operation center for monitoring; at the same time, sends the telemetry data to the robot arm simulation training module for three-dimensional simulation of the robot arm; sends the posture measurement data to the robot arm motion parameter and image display module; parses the six-dimensional force data based on the telemetry data and posture measurement data, and sends the six-dimensional force data to the human-computer interaction operation module; Robotic arm simulation training module: receives telemetry data from the robotic arm information processing module, simulates the robotic arm 3D model to the position of the telemetry data, and displays the motion state of the robotic arm; receives the robotic arm motion control instructions from the robotic arm dynamic bus information management module, drives the robotic arm 3D model for real-time simulation according to the robotic arm motion control instructions, generates the robotic arm simulation video and image data, and sends the robotic arm simulation video and image data to the robotic arm motion parameter and image display module for display; at the same time, sends the robotic arm simulation video and image data to the ground operation center; Robotic arm motion parameter and image display module: receives the simulation video and image data of the robotic arm three-dimensional model from the robotic arm simulation training module, and displays the simulation situation; receives the posture measurement data from the robotic arm information processing module; obtains the real video image data of the robotic arm movement; displays the real motion situation of the robotic arm through the posture measurement data and the real video image data of the robotic arm movement.

2. The astronaut in-cabin operation and display control system for dual-arm on-orbit space operation according to claim 1, characterized in that: The manipulator control data includes ground control data and autonomous control data.

3. The astronaut in-cabin operation and display control system for dual-arm on-orbit operation in space according to claim 1, characterized in that: The robot arm working mode information includes single-arm independent working, multi-arm combined working, and multi-arm coordinated working; the remote operation information includes ground handle remote operation and ground command remote operation.

4. The astronaut in-cabin operation and display control system for dual-arm on-orbit operation in space according to claim 1, characterized in that: The typical autonomous tasks include pre-programmed movement of the robot arm, visual closed-loop movement of the robot arm, autonomous compartment transfer of the robot arm, and emergency return of the robot arm; and the typical autonomous tasks are dynamically updated in real time through annotation.

5. The astronaut in-cabin operation and display control system for dual-arm on-orbit operation in space according to claim 1, characterized in that: The human-computer interaction operation module includes a panel operation module, a rotation handle operation module, and a translation handle operation module; the human-computer interaction operation module inputs the robot arm motion control instruction through one or more of the panel operation module, the rotation handle operation module, and the translation handle operation module.

6. The astronaut in-cabin operation and display control system for dual-arm on-orbit space operation according to claim 5, characterized in that: The panel operation module receives the astronauts' operating instructions for the robotic arm. The astronauts use shortcut hard keys and graphic page soft keys to set robotic arm motion parameters, perform robotic arm motion operations, end capture and release, typical autonomous task operations, single-machine management of the robotic arm, and set and manage the parameters of the translation and rotation handles in the cabin. The robotic arm control instructions are generated on orbit. All robotic arm control instructions are sent to the robotic arm dynamic bus information management module only after verification and secondary confirmation. On the other hand, the operation process information is also synchronously displayed in the panel operation module to provide prompt information for the astronauts. The process information is also sent to the ground operation center as robotic arm telemetry data. The ground operation center can mirror the astronauts' on-orbit operation actions for monitoring and evaluation.

7. The astronaut in-cabin operation and display control system for dual-arm on-orbit space operation according to claim 6, characterized in that: The motion parameters of the robot arm include load, speed, and acceleration gear.

8. The astronaut in-cabin operation and display control system for dual-arm on-orbit space operation according to claim 5, characterized in that: The handle rotation operation module provides a direct operation method for astronauts. By watching the video image of the robotic arm, the astronauts directly operate the handle to adjust the pitch, yaw, and rotation direction of the robotic arm end, adjust the forward and reverse rotation posture of the single joint, and adjust the pitch and yaw posture of the camera gimbal. The handle rotation module receives the astronaut's operation of the handle, and generates control data for controlling the attitude of the robotic arm end, single joint, and camera gimbal according to the set speed and load gear, and sends it to the robotic arm dynamic bus information management module.

9. The astronaut in-cabin operation and display control system for dual-arm on-orbit space operation according to claim 8, characterized in that: The translation handle operation module provides a direct operation method for astronauts. By watching the video image of the robotic arm, the astronauts directly operate the translation handle to adjust the position of the end of the robotic arm in the front and back, left and right, and up and down directions. The translation handle module receives the astronaut's operation of the handle, and generates control data for controlling the position of the end of the robotic arm according to the set speed and load gear, and sends it to the robotic arm dynamic bus information management module.

10. The astronaut in-cabin operation and display control system for dual-arm on-orbit space operation according to claim 9, characterized in that: The rotation handle operation module and the translation handle operation module both receive the six-dimensional force data sent by the robotic arm information processing module, and superimpose the force feedback information on the handle damping force to provide a smooth force feedback feeling for the astronaut's operation.

Images