Interaction method and system of space intelligent flying robot in microgravity environment

Abstract

The invention provides an interaction method and system of a space intelligent flying robot in a microgravity environment, and belongs to the field of man-machine interaction. The invention aims to solve the problems of large task understanding deviation and low execution efficiency in a complex hardware equipment and software management system of the existing robot interaction system in the microgravity environment. The system comprises a standby mode, an autonomous flight mode, an autonomous mapping mode and a holder shooting mode, and three interaction methods of a touch screen interface, voice interaction and fuselage lamp language are fused; the touch screen interface can receive complex task instructions and timely feed back interaction information; the voice interaction enables an operator to quickly issue an instruction in a non-contact manner; a machine body lamp can feed back the motion state of the robot in real time, and safety and reliability guarantee are effectively improved; the three interaction modes complement one another, so that the man-machine interaction efficiency is remarkably improved, and the operation convenience and the task execution reliability are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to an interaction method and system for a space intelligent flying robot in a microgravity environment. Background Art

[0002] With the continued advancement of my country's manned space program and deep space exploration program, scientific research experiments in microgravity environments are becoming increasingly complex, characterized by their diversity, long duration, and high dynamics. As a new generation of autonomous unmanned systems developed for microgravity environments, space intelligent flying robots, integrating autonomous navigation, intelligent perception, and human-machine interaction capabilities, can overcome the physiological limitations of astronauts and autonomously perform tasks such as scientific experiment equipment inspections, experimental progress monitoring, and dynamic data collection. They have become a crucial piece of equipment for enhancing the effectiveness of space science experiments. Because the reliability of data collection and the efficiency of human-machine interaction directly determine the effectiveness of these tasks, breakthroughs in data collection and human-machine interaction software technologies are crucial to improving the efficiency and reliability of space science experiments.

[0003] As the interface between the robot and the outside world, the data acquisition and human-computer interaction software primarily addresses three key technologies: high-precision environmental perception under the interference of floating objects in microgravity; intelligent data acquisition based on heterogeneous sensor fusion; and human-computer interaction and precise intention recognition in complex acoustic and optical fields. This software understands the astronaut's operational intent through a multi-channel interactive interface (gesture recognition, voice recognition, touch screen or physical buttons, etc.), and then coordinates subsystems such as navigation control, environmental perception, and data acquisition to complete closed-loop control of the entire process, from command parsing and path planning to mission execution. During data acquisition, the software uses onboard sensors such as cameras and microphones to obtain real-time status or test data from scientific experimental equipment, storing it locally or uploading it to a server to ensure the accuracy and reliability of the collected data.

[0004] Current interactive systems used in microgravity environments typically employ centralized or distributed architectures. These systems generally rely on complex hardware and multi-layered software management systems, making them susceptible to interference from dynamic floating objects, unstable communication lines, and misunderstandings of interactive commands. This can lead to data collection interruptions, data transmission delays, and even mission execution errors. Furthermore, existing software systems often fail to fully consider the characteristics of equipment operation and personnel activities in microgravity during their design. Consequently, the system's sustained stability and overall reliability face severe challenges during long-term test missions.

[0005] To address these technical challenges, this paper proposes an innovative data acquisition and interaction software system. Based on a space intelligent flight robot platform, this system accurately understands astronauts' intentions through multi-channel interaction, implements high-precision obstacle avoidance navigation control based on an autonomous navigation control module, and simultaneously utilizes high-precision sensors such as onboard high-definition cameras and microphones for high-quality data acquisition. Designed with intelligence, modularity, high computational efficiency, and high reliability in mind, this software system aims to enhance the performance of scientific research and experimental missions in microgravity environments and provide a solid technical foundation for future deep space exploration missions. Summary of the Invention

[0006] The technical problems to be solved by the present invention are:

[0007] In order to solve the problems of large task understanding deviation and low execution efficiency in the existing robot interaction system in microgravity environment due to its complex hardware equipment and software management system.

[0008] The present invention is to solve the above technical problems using the following technical solutions:

[0009] The present invention provides an interaction method for a space intelligent flying robot in a microgravity environment, comprising the following steps:

[0010] S100, when the flying robot is in standby mode, collecting data information through the image data acquisition module, and implementing emergency braking of the robot through the emergency control module, including braking and emergency stopping;

[0011] S200: When the flying robot is in autonomous flight mode, the multifunctional interaction module receives a target point execution instruction and a set body posture issued by the user. The flying robot uses the computing module to autonomously plan and avoid obstacles to reach the target point based on data collected by the image data acquisition module and surrounding environment data collected by the onboard sensor, and then switches to the set body posture.

[0012] S300, when the flying robot is in the mapping mode, the flying robot maintains the current posture or the set posture, performs initial positioning by scanning the visual tags, and then performs mapping, and displays the mapping on the display module using the multifunctional interaction module;

[0013] S400. When the flying robot is in the pan-tilt shooting mode, including the wall-sticking shooting mode and the humanoid tracking mode, in the wall-sticking shooting mode, the image data acquisition module is used to detect whether there is a wall around the flying robot body. If there is no wall, a no-wall warning is issued. When there is a wall, the flying robot executes the wall-sticking action and turns on the photo taking and video recording functions of the image data acquisition module to achieve fixed-point shooting and video recording in the state of adsorption to a fixed wall surface; in the humanoid tracking mode, the humanoid tracking function is turned on and tracking information and the position of the target feature point are continuously sent to the transport module. After calculation, the posture control command is obtained and sent to the motion module. The motion module changes the body posture so that the tracked object remains in the center of the shooting screen.

[0014] Furthermore, in step S100, the environmental data collected by the image data acquisition module needs to be converted from the camera coordinate system to the world coordinate system:

[0015]

[0016] in, is the transformation moment from the camera coordinate system to the world coordinate system, is the rotation matrix, is the translation vector describing the camera position, and Represents the point coordinates in camera coordinates and world coordinates respectively.

[0017] Furthermore, in step S200, when performing an autonomous flight mission, the robot first constructs a three-dimensional map based on the SLAM algorithm. The SLAM backend uses a least squares model based on multi-sensor information to minimize the projection error:

[0018]

[0019] in, are the robot pose parameters, represents the pixel coordinates of the jth feature point in the i-th frame image, n represents the total number of image frames, and m represents the total number of feature points. represents the camera projection model with intrinsic distortion correction, is the world coordinate of the jth map point;

[0020] Next, the A* algorithm is used to search for an optimal collision-free path within the constructed map, starting from the current position and ending at preset point B. During flight, the flying robot continuously scans the area ahead using its sensors. Once an obstacle is detected, the dynamic obstacle avoidance algorithm in the computing module is immediately activated until it reaches preset point B and switches to the set posture.

[0021] Target tracking uses the artificial potential field method, and the gravitational potential field is:

[0022]

[0023] in, is the gravitational coefficient, is the Euclidean distance between the current position and the target point, and the repulsive potential field is:

[0024]

[0025] in, is the repulsion coefficient, is the distance between the robot and the obstacle, is the obstacle influence range threshold; the force on the robot is the vector sum of the attraction and repulsion, and the motor output is adjusted by the PID controller:

[0026]

[0027] in, is the position error, are the proportional, integral, and differential coefficients.

[0028] Furthermore, in step S300, when the flying robot receives the autonomous mapping instruction through the multi-functional interactive module, the waiting interface of the display module displays the feedback information of ready to scan the code, and then scans the visual label three times through the image data acquisition module. When the display module interface displays the ready feedback information, it again feeds back the initial positioning success; during this process, the flying robot delays for 3 seconds to build the map; if it is not canceled during the delay period, it continues to execute autonomous mapping and displays the completion percentage and a thumbnail of the current mapping process. After the mapping is completed; when the user sends a map search instruction through the multi-functional interactive module, the server is accessed according to the image ID to obtain the corresponding image file.

[0029] Furthermore, after the flying robot receives the wall-sticking shooting instruction through the multi-functional interactive module, when there is no wall around the flying robot, a prompt message is issued on the display module, requiring the flying robot to be placed near the wall; when there is a wall around the flying robot, the flying robot gradually sticks to the wall, and periodically feedbacks the wall-sticking status during this process until it successfully sticks to the wall.

[0030] Furthermore, in step S400, in the humanoid tracking mode, the flying robot continuously transmits the tracking distance, up / down / left / right offset, and the position of the target feature point at a rate of 10 frames / s. During this process, the tracking parameters are set on the display module, and the tracking parameters include the tracking distance and the uniform position of the center coordinates of the tracked object in the captured image.

[0031] Target tracking uses the Kalman filter model:

[0032]

[0033] in, and is the state vector, For external control input, is the actual measurement value at time k, is the prediction equation, is the observation equation, and are process noise and observation noise, respectively.

[0034] Furthermore, in step S100 to step S400, a voice interaction function is also included, and interaction with the flying robot is performed by giving voice commands.

[0035] Furthermore, it also includes the body light language function. Through the programmable full-color LED light group, the color and flashing frequency of each LED light are defined, and this is encoded to form light language to display the current working status and movement direction of the flying robot.

[0036] An interactive system for a space intelligent flying robot in a microgravity environment has program modules corresponding to the above steps, and executes the steps of the above-mentioned interactive method for a space intelligent flying robot in a microgravity environment when running.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] ① Modular Design: Utilizing a layered and modular design philosophy, the system primarily encompasses core modules for task scheduling, navigation and perception, environmental interaction, data processing and storage, and user interface. This system design decouples functionality between modules, integrating them through a unified standard interface. This improves system scalability and maintainability, laying the foundation for subsequent functional upgrades and technological iterations.

[0039] ② Intelligent task scheduling and optimization: Through the flying robot's operating mode switching strategy based on the voice command table, the system can realize intelligent adjustment of the task execution sequence and optimize resource allocation in real time according to dynamic factors such as task requirements, robot spatial position, current power level, task priority and system resource status, thereby improving the efficiency and reliability of task execution.

[0040] ③ Optimized mapping capabilities integrating multi-source sensing: Integrating multi-modal data such as visual sensors (depth cameras, binocular cameras), temperature and humidity sensors, lidar, and IMU inertial units, and achieving high-precision environmental perception in microgravity environments through multi-source SLAM back-end least squares optimization. Combined with deep learning and reinforcement learning algorithms, the system can intelligently analyze environmental information to ensure that the robot can achieve autonomous obstacle avoidance, precise positioning, and efficient navigation in complex environments.

[0041] ④ Multi-channel integrated human-machine interaction: This system integrates three interaction modes: touchscreen interface, voice interaction, and on-body lighting. The touchscreen interface receives complex task instructions and provides instant feedback; voice interaction allows operators to quickly and contactlessly issue commands; and on-body lighting provides real-time feedback on the robot's motion status, effectively enhancing safety and reliability. These three interaction modes complement each other, significantly improving human-machine interaction efficiency while also enhancing operational convenience and task execution reliability.

[0042] In summary, this paper proposes a highly efficient, reliable, and secure data acquisition and human-computer interaction software system designed for microgravity environments. Through intelligent task scheduling and perception and navigation mechanisms, this system ensures that intelligent flying robots can efficiently and safely complete various tasks in complex microgravity environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of an interaction method for a space intelligent flying robot in a microgravity environment according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart of the multifunctional interactive module for wall shooting in an embodiment of the present invention;

[0045] Figure 3 This is a diagram of the standby mode interface of the multifunctional interactive module in an embodiment of the present invention;

[0046] Figure 4 This is a diagram of the autonomous flight mode of the multifunctional interactive module in an embodiment of the present invention;

[0047] Figure 5 This is a diagram of the autonomous mapping mode of the multifunctional interactive module in an embodiment of the present invention;

[0048] Figure 6 This is a diagram of the multi-functional interactive module pan-tilt shooting mode in an embodiment of the present invention;

[0049] Figure 7 This is a diagram of the wall-mounted shooting mode of the multi-functional interactive module in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] Specific implementation plan 1: Combined Figures 1 to 7 As shown, the present invention provides an interaction method for a space intelligent flying robot in a microgravity environment, comprising the following steps:

[0052] The image data acquisition module collects data information, including image information, through high-definition video and standard-definition video, including static pictures and dynamic videos;

[0053] Combine Figure 3 As shown, after the multi-function interactive module is turned on, it automatically enters the standby mode, and the blank area of ​​the interface displays the image captured by the camera. The environmental data collected by the visual camera needs to be converted from the camera coordinate system to the world coordinate system:

[0054]

[0055] in, is the rotation matrix, is the translation vector describing the camera position, is the transformation moment from the camera coordinate system to the world coordinate system, and Represent the point coordinates in camera coordinates and world coordinates respectively;

[0056] "HD Recording" primarily improves clarity, resulting in larger video files; "SD Recording" primarily extends recording time; the "Photo" button is used to capture still images at maximum resolution; the "Video List" is used to view video files recorded by the aforementioned "HD Recording" and "SD Recording" functions, and supports playback and deletion; the "Picture List" is used to view still images taken by the aforementioned "Photo" function, and supports gallery browsing, zooming in, and deleting.

[0057] The robot's emergency braking is achieved through the emergency control module. When the display module triggers the brake command, the flying robot stops all movement operations. When the display module triggers the emergency stop command, the underlying control algorithm becomes ineffective and stops providing drive signals to the fan. On the display module, the button changes to a "resume work" button. Clicking the button again restores the underlying control algorithm's control over the hardware. Both levels of emergency stop control commands can be triggered by button clicks and voice commands, ensuring the safety of the flying robot and the operator in the event of an emergency.

[0058] The multifunctional interactive module obtains user instructions and passes them to the calculation module. The calculation module recognizes the instructions and then constructs a three-dimensional map and plans the path according to the instructions. The constructed three-dimensional map is displayed in real time on the display module to realize the mapping mode. The path planning instructions are passed to the motion module to realize the underlying motion control and realize the autonomous flight mode. At the same time, the multifunctional interactive module displays the information during and after the calculation through the display module or voice.

[0059] Combine Figure 4As shown in the figure, after the flying robot receives a command to execute a target point via voice or touch signal channel of the display module, the robot first uses the SLAM algorithm to collect real-time data of the surrounding environment using multiple sensors such as visual cameras to build a three-dimensional map. The SLAM backend uses a least squares model based on multi-sensor information to minimize projection errors:

[0060]

[0061] in, are the robot pose parameters, represents the pixel coordinates of the jth feature point in the i-th frame image, n represents the total number of image frames, and m represents the total number of feature points. represents the camera projection model with intrinsic distortion correction, is the world coordinate of the jth map point;

[0062] Next, the A* algorithm is used to search for a collision-free optimal path in the constructed map, starting from the current position and ending at the preset point B. During flight, the robot continuously scans the area ahead through sensors. Once an obstacle is detected, the dynamic obstacle avoidance algorithm is immediately activated until it reaches the preset point B and switches to the set body posture. The "X Warehouse", "A Warehouse" and "M Warehouse" buttons in the upper part of the figure are the destination areas pre-set in the configuration file. After clicking, two specific destination names, "B Section C Quadrant" and "B Section D Quadrant", appear. Each name corresponds to a set of location coordinates and body posture information. Click the corresponding button to set the destination location and body posture. When the user says "Go to" + the location name (for example, "Go to A Warehouse") or clicks the function button of the corresponding destination, its XYZ coordinates and body posture parameters Q1, Q2, Q3, W (x, y, z) are queried. The multi-function interactive module receives the execution progress percentage message and displays it in a pop-up status bar below the main interface. During the execution process, click the button on the lower right side of the screen, and the multi-function interactive module sends a brake / emergency stop command to the server.

[0063] Target tracking uses the artificial potential field method, and the gravitational potential field is:

[0064]

[0065] in, is the gravitational coefficient, is the Euclidean distance between the current position and the target point, and the repulsive potential field is:

[0066]

[0067] in, is the repulsion coefficient, is the distance between the robot and the obstacle, is the obstacle influence range threshold; the force on the robot is the vector sum of the attraction and repulsion, and the motor output is adjusted by the PID controller:

[0068]

[0069] in, is the position error, are the proportional, integral, and differential coefficients;

[0070] When building a 3D map, the flying robot can automatically maintain its current posture and position, making it easier for the operator to use it as a stable recording device. The operator can control the start and stop of recording and taking photos through voice commands. The user sends a mapping preparation instruction to the operation module through the multi-functional interaction module. When the feedback message "Not Ready" is received, a status bar pops up in the display module interface, and a voice prompt "Initializing, please wait..." is given. When the feedback message "Ready to scan visual tags" is received, a status bar pops up and a voice prompt "Please aim the camera at visual tag No. ①" is given. When the feedback message "Scanned code No. ①" is received, a status bar pops up and a voice prompt "Visual tag No. ① has been found, please aim the camera at visual tag No. ②" is given. When the feedback message "Scanned code No. ②" is received, a status bar pops up and a voice prompt "Visual tag No. ① has been found, please aim the camera at visual tag No. ②" is given. When feedback information is given, a status bar pops up and a voice prompt "Visual tag No. ② has been found, please aim the camera at visual tag No. ③" is given. When the "Ready" feedback information is received, a status bar pops up and a voice prompt "Initial positioning is successful, mapping is about to begin, please release the flying robot" is given. After a delay of 3 seconds, a mapping start instruction is sent to the operation module. If the end button is clicked during the 3-second delay, mapping is canceled and the system returns to standby mode. The multi-functional interaction module receives the execution progress percentage message and pops up a status bar below the main interface of the display module. During the execution process, the push message displays a thumbnail of the current mapping process on the display module. When the user sends a map search instruction through the multi-functional interaction module, the server is accessed according to the image ID to obtain the corresponding image file.

[0071] When the user sends the pan-tilt shooting command to the computing module through the voice or touch signal channel of the multi-function interaction module, including wall-sticking shooting and human tracking modes, in the wall-sticking shooting mode, it has the environment detection function. When there is no wall around the body, the multi-function interaction module will issue a wall-free warning; when there is a wall around the body, the motion module will execute the wall-sticking action, and the image data acquisition module will perform photo and video recording functions, realizing fixed-point shooting and recording in the state of adsorption to a fixed wall; in the human tracking mode, the human tracking function is turned on, and the tracking information and the position of the target feature points are continuously sent, and the tracked object is kept in the center of the shooting screen by changing the body posture;

[0072] In order to achieve the robot's equilibrium state in a microgravity environment, the adsorption force F吸附力 The balance conditions in a microgravity environment must be met:

[0073]

[0074] in, is the robot quality, is the microgravity acceleration, is the friction coefficient, is the impact force of the ambient airflow on the robot;

[0075] Combine Figure 6 As shown, in the humanoid tracking mode, when the humanoid tracking instruction is sent to the operation module through the multifunctional interaction module, the "humanoid tracking" button on the right side of the display module is displayed, and the humanoid tracking start / pause / continue / end instruction is sent to the operation module through the multifunctional interaction module; in the humanoid tracking state, the voice instruction is received at any time, and the offset set in the voice instruction is sent. Click the "Camera Position Adjustment" in the display module, and the command button appears to send the corresponding instruction; click the corresponding button to send the translation and rotation offset to the motion module, and the offset includes translation and rotation; click the function button on the right side of the screen on the display module to switch the "humanoid tracking" switch, and the switch status is displayed on the display module; when the " When the "Human Tracking" switch is on, the tracking distance, up / down / left / right offset, and target feature point position are continuously sent at the fastest possible speed (ideally at 10 frames per second or above). The sending method is similar to the autonomous tracking mode. When "Human Tracking" is turned on, click the "Parameter Adjustment" button on the interface of the display module to set specific tracking parameters. When changing the "Tracking Distance" option, the XYZ coordinates of the aircraft are changed to move closer to or farther away from the tracking target, and the aircraft posture remains unchanged. When clicking the "Up", "Down", "Left", and "Right" buttons, the XYZ coordinates of the aircraft remain unchanged, and the position of the center coordinates of the tracked object in the shooting picture is adjusted by changing the aircraft posture.

[0076] Target tracking uses the Kalman filter model:

[0077]

[0078] in, and is the state vector, For external control input, is the actual measurement value at time k, is the prediction equation, is the observation equation, and are process noise and observation noise respectively;

[0079] like Figure 7As shown, in the wall-sticking shooting mode, the flying robot receives the execution instruction through the voice or touch signal channel of the multifunctional interaction module, and then autonomously explores and records the three-dimensional map; after switching to the wall-sticking shooting mode, the multifunctional interaction module sends the wall-sticking mode start instruction to the computing module, including start / pause / continue / end instructions; the multifunctional interaction module receives the execution status message of the wall-sticking action, and pops up a status bar below the main interface of the display module. If the status feedback received is 3 (no wall), the wall-sticking operation cannot be performed. The display module interface pops up a prompt message "There is no wall behind the screen, please place the machine near the wall" and makes a voice broadcast; after the computing module receives the instruction to find the wall, it sends a ready state with a progress of 0%. The multifunctional interaction module prompts "The body will move to the back of the screen and attach" and makes a voice broadcast; after a delay of 3 seconds, the computing module automatically sends a start command. If the emergency stop button is clicked during the delay, the wall-sticking is cancelled, the computing module automatically sends an end command, and returns to the standby mode; after the delay, the computing module starts the wall-sticking servo algorithm based on A* The algorithm generates a wall-adhering motion trajectory and sends it to the motion module to adjust the flying robot's translation speed, rotation angle, and vertical distance from the wall in real time. At a cycle of 500ms, it sends wall-adhering status data including the current progress percentage, real-time attitude angle, actual distance to the wall, etc. to the storage module via the UDP protocol. When the calculation module determines that the wall-adhering progress has reached 100% through the trajectory odometry or visual optical flow algorithm and is confirmed by the in-place detection algorithm, the display module displays "Wall Adherence Successful" and announces it by voice, and enters the shooting ready mode at the same time, waiting for the next instruction.

[0080] The application software proposed in this invention is developed based on the Android environment and can be adapted and run smoothly on devices equipped with the Hongmeng operating system. The program version number is 1.0.0, and its core functions focus on realizing touch screen interaction and voice interaction functions:

[0081] ① Touch screen interaction

[0082] To meet the needs of intelligent flying robots to execute complex commands, a multifunctional interactive module for touch-screen operation was designed. With its intuitive and easy-to-understand interface layout and simple and efficient operating logic, the system enables operators to easily use the various functions of the flying robot. The touch-screen interface not only covers a variety of functions such as mode switching, process start and stop, photo / video control, and system parameter configuration, but also can display the communication status with the onboard navigation system and bone conduction headphones in real time, ensuring instant synchronization of information.

[0083] ② Voice interaction

[0084] In order to meet the astronauts' needs for contactless operation of intelligent flying robots, an innovative voice interaction system based on bone conduction headphones (also a multi-functional interaction module) was designed; the system can receive astronauts' voice commands through bone conduction headphones or the built-in microphone of the fuselage, parse the command intent through voice recognition algorithms, and convert them into corresponding operation commands; as feedback on the voice commands, the system will highlight the corresponding operation command button on the touch screen while executing the voice commands, so as to improve the reliability and convenience of the interaction.

[0085] Table 1 shows the list of voice commands supported by the system. The fine mode enables the robot to provide more precise spatial positioning. In the fast mode, the robot increases its movement speed as much as possible to achieve rapid movement while meeting basic safety conditions.

[0086] Table 1 Voice command table

[0087]

[0088] The human-machine interface consists of a touch screen interface and the body light language. The touch screen interface is the primary human-machine interaction method. On the one hand, it realizes two-way interaction with the operator. On the other hand, it also serves as a visual feedback platform for voice interaction, displaying the voice interaction status in real time:

[0089] ① Touch screen interface design

[0090] In terms of the user-friendly design of the human-computer interaction interface, the software arranges function buttons, selection boxes, and direction key controls on the touch screen interface, aiming to assist the operator in quickly controlling the interactive content and ensuring the simple and efficient execution of operating instructions; at the same time, the touch screen interface can reflect the operating status of the onboard navigation system in real time, providing the operator with intuitive system status monitoring, so that they can quickly issue control commands according to actual needs.

[0091] ② Body lighting design

[0092] To address the difficulty of communicating the robot's position and behavior during human-robot collaborative operations, the robot is equipped with a programmable, full-color LED light system. The operator can flexibly define the color and flashing frequency of each LED, creating a coded light message that accurately displays the robot's current operating status and direction of movement. For example, a solid red LED with a rapidly flashing purple tail LED indicates the robot is being forced to descend due to low battery and requires emergency attention. A blinking blue light with a yellow running light on the left side clearly indicates the robot is in standby mode and moving to the left. This light message system effectively ensures operator safety during on-orbit operations and significantly improves the efficiency of human-robot collaborative operations by reducing communication costs.

[0093] Specific implementation scheme 2: The present invention provides an interactive system for a space intelligent flying robot in a microgravity environment. The system has a program module corresponding to the above steps, and executes the steps in the above-mentioned interactive method for a space intelligent flying robot in a microgravity environment during operation.

[0094] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0095] Although the present invention is disclosed as above, the scope of protection disclosed by the present invention is not limited thereto. Those skilled in the art of the present invention may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for interacting with a space intelligent flying robot in a microgravity environment, characterized in that: The following steps are involved: S100, when the flying robot is in standby mode, collecting data information through the image data acquisition module, and implementing emergency braking of the robot through the emergency control module, including braking and emergency stopping; S200: When the flying robot is in autonomous flight mode, the multifunctional interaction module receives a target point execution instruction and a set body posture issued by the user. The flying robot uses the computing module to autonomously plan and avoid obstacles to reach the target point based on data collected by the image data acquisition module and surrounding environment data collected by the onboard sensor, and then switches to the set body posture. S300, when the flying robot is in the mapping mode, the flying robot maintains the current posture or the set posture, performs initial positioning by scanning the visual tags, and then performs mapping, and displays the mapping on the display module using the multifunctional interaction module; S400. When the flying robot is in the pan-tilt shooting mode, including the wall-sticking shooting mode and the humanoid tracking mode, in the wall-sticking shooting mode, the image data acquisition module is used to detect whether there is a wall around the flying robot body. If there is no wall, a no-wall warning is issued. When there is a wall, the flying robot executes the wall-sticking action and turns on the photo taking and video recording functions of the image data acquisition module to achieve fixed-point shooting and video recording in the state of adsorption to a fixed wall surface; in the humanoid tracking mode, the humanoid tracking function is turned on and tracking information and the position of the target feature point are continuously sent to the transport module. After calculation, the posture control command is obtained and sent to the motion module. The motion module changes the body posture so that the tracked object remains in the center of the shooting screen.

2. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 1, characterized in that: In step S100, the environmental data collected by the image data acquisition module needs to be converted from the camera coordinate system to the world coordinate system: ； in, is the transformation moment from the camera coordinate system to the world coordinate system, is the rotation matrix, is the translation vector describing the camera position, and Represents the point coordinates in camera coordinates and world coordinates respectively.

3. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 2, characterized in that: In step S200, when performing an autonomous flight mission, the robot first constructs a three-dimensional map based on the SLAM algorithm. The SLAM backend uses a least squares model based on multi-sensor information to minimize the projection error: ； in, are the robot pose parameters, represents the pixel coordinates of the jth feature point in the i-th frame image, n represents the total number of image frames, and m represents the total number of feature points. represents the camera projection model with intrinsic distortion correction, is the world coordinate of the jth map point; Next, the A* algorithm is used to search for an optimal collision-free path within the constructed map, starting from the current position and ending at preset point B. During flight, the flying robot continuously scans the area ahead using its sensors. Once an obstacle is detected, the dynamic obstacle avoidance algorithm in the computing module is immediately activated until it reaches preset point B and switches to the set posture. Target tracking uses the artificial potential field method, and the gravitational potential field is: ； in, is the gravitational coefficient, is the Euclidean distance between the current position and the target point, and the repulsive potential field is: ； in, is the repulsion coefficient, is the distance between the robot and the obstacle, is the obstacle influence range threshold; the force on the robot is the vector sum of the attraction and repulsion, and the motor output is adjusted by the PID controller: ； in, is the position error, are the proportional, integral, and differential coefficients.

4. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 3, characterized in that: In step S300, when the flying robot receives the autonomous mapping instruction through the multi-functional interactive module, the waiting interface of the display module displays the feedback information of ready to scan the code, and then scans the visual label three times through the image data acquisition module. When the display module interface displays the ready feedback information, it again feedbacks that the initial positioning is successful; during this process, the flying robot delays for 3 seconds to build the map; if it is not canceled during the delay period, it continues to execute autonomous mapping and displays the completion percentage and a thumbnail of the current mapping process. After the mapping is completed; when the user sends a map search instruction through the multi-functional interactive module, the server is accessed according to the image ID to obtain the corresponding image file.

5. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 4, characterized in that: After the flying robot receives the wall-sticking shooting instruction through the multi-functional interactive module, when there is no wall around the flying robot, a prompt message is issued on the display module, requiring the flying robot to be placed near the wall; when there is a wall around the flying robot, the flying robot gradually sticks to the wall, and periodically feedbacks the wall-sticking status during this process until it succeeds in sticking to the wall.

6. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 5, characterized in that: In step S400, in humanoid tracking mode, the flying robot continuously transmits the tracking distance, vertical / lateral offset, and target feature point position at a rate of 10 frames per second. During this process, the tracking parameters are set on the display module. The tracking parameters include the tracking distance and the uniform position of the center coordinates of the tracked object in the captured image. Target tracking uses the Kalman filter model: ； in, and is the state vector, For external control input, is the actual measurement value at time k, is the prediction equation, is the observation equation, and are process noise and observation noise, respectively.

7. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 6, characterized in that: In step S100 to step S400, a voice interaction function is also included, in which voice commands are given to interact with the flying robot.

8. The method for interacting with a space intelligent flying robot in a microgravity environment according to claim 7, characterized in that: It also includes a body light language function, which uses a programmable full-color LED light group to define the color and flashing frequency of each LED light, and thus encodes the light language to display the current working status and movement direction of the flying robot.

9. An interactive system for a space intelligent flying robot in a microgravity environment, characterized by: The system has a program module corresponding to the steps described in any one of claims 1 to 8, and executes the steps in the above-mentioned method for interacting with a space intelligent flying robot in a microgravity environment when running.

Images