An interactive method and system for a space intelligent flying robot in a microgravity environment

Abstract

The application provides an interactive method and system of a space intelligent flying robot in a microgravity environment, and belongs to the field of artificial interaction. In order to solve the problem of large task understanding deviation and low execution efficiency of the existing robot interaction system in the microgravity environment due to the complex hardware equipment and software management system, the application comprises a standby mode, an autonomous flight mode, an autonomous mapping mode and a cloud platform shooting mode, and combines a touch screen interface, voice interaction and body light language as three interactive methods. The touch screen interface can receive complex task instructions and instantaneously feedback interactive information. The voice interaction enables the operator to quickly issue instructions in a non-contact manner. The body light language can instantaneously feedback the motion state of the robot, effectively improving the safety and reliability guarantee. The three interactive modes are complementary to each other, which not only significantly improves the human-computer interaction efficiency, but also enhances the convenience of operation and the reliability of task execution.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and more specifically, to an interaction method and system for a space intelligent flying robot in a microgravity environment. Background Technology

[0002] With the continuous advancement of my country's manned space program and deep space exploration plan, scientific research experiments in microgravity environments are exhibiting complex characteristics such as diversification, long cycles, and high dynamism. As a new generation of autonomous unmanned systems developed for microgravity environments, space intelligent flying robots, by integrating autonomous navigation, intelligent perception, and human-computer interaction capabilities, can overcome the physiological limitations of astronauts and autonomously perform tasks such as inspecting scientific experimental equipment, monitoring experimental progress, and dynamically acquiring data. They have become important equipment for improving the efficiency of space science experiments. Since the reliability of data acquisition and the efficiency of human-computer interaction directly determine the actual effectiveness of these tasks, breakthroughs in data acquisition and human-computer interaction software technologies are key to improving the efficiency and reliability of space science experiments.

[0003] As the interface for robots to interact with the outside world, the data acquisition and human-computer interaction software primarily addresses three key technologies: high-precision environmental perception under microgravity floating object interference; intelligent data acquisition based on heterogeneous sensor fusion; and human-computer interaction and accurate intent recognition under complex sound and light fields. This software can understand the astronaut's operational intentions through multi-channel interactive interfaces (gesture recognition, voice recognition, touchscreen, or physical buttons, etc.), and then coordinate subsystems such as navigation control, environmental perception, and data acquisition to complete a closed-loop control process from command parsing and path planning to mission execution. During data acquisition, the software uses onboard cameras, microphones, and other sensors to acquire the status of scientific experimental equipment or experimental data in real time, storing it locally or uploading it to a server, thereby ensuring the accuracy and reliability of the acquired data.

[0004] Current interactive systems used in microgravity environments typically employ centralized or distributed architectures. These systems generally rely on complex hardware and multi-layered software management systems, making them susceptible to interference from dynamic floating objects, unstable communication lines, and misunderstandings of interactive commands. This can lead to data acquisition interruptions, data transmission delays, and even task execution errors. Furthermore, existing software systems often fail to fully consider the characteristics of equipment operation and personnel activities in microgravity environments during their design. Therefore, the continuous stability and overall reliability of these systems face severe challenges during long-term experimental tasks.

[0005] To address the aforementioned technical challenges, this invention proposes an innovative data acquisition and interaction software system. Based on a space intelligent flight robot platform, this system accurately understands astronaut intentions through multi-channel interaction and achieves high-precision obstacle avoidance and navigation control based on an autonomous navigation control module. Simultaneously, it utilizes onboard high-definition cameras, microphones, and other high-precision sensors for high-quality data acquisition. This software system is designed with intelligence, modularity, high computational efficiency, and high reliability in mind, aiming to improve the execution efficiency of scientific research experiments in microgravity environments and provide solid technical support for future deep space exploration missions. Summary of the Invention

[0006] The technical problem to be solved by this invention is:

[0007] To address the issues of large task comprehension bias and low execution efficiency in existing robot interaction systems under microgravity environments due to their complex hardware and software management systems.

[0008] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0009] This invention provides an interaction method for a space intelligent flying robot in a microgravity environment, comprising the following steps:

[0010] S100: When the flying robot is in standby mode, it collects data information through the image data acquisition module and realizes emergency braking of the robot through the emergency control module, including braking and emergency stop.

[0011] S200. When the flying robot is in autonomous flight mode, it obtains the target point execution command and set body posture issued by the user through the multi-functional interaction module. The flying robot uses the computing module to autonomously plan and avoid obstacles to reach the target point based on the data collected by the image data acquisition module and the surrounding environment data collected by the airborne sensors, and then switches to the set body posture.

[0012] S300 When the flying robot is in mapping mode, the flying robot maintains its current or set pose, performs initial localization by scanning visual tags, then performs mapping, and displays the map on the display module using the multi-functional interactive module.

[0013] S400: When the flying robot is in gimbal shooting mode, including wall-mounted shooting mode and human tracking mode, in wall-mounted shooting mode, the image data acquisition module detects whether there is a wall around the flying robot. If there is no wall, a wall-no warning is issued. When there is a wall, the robot performs a wall-mounted action and activates the image data acquisition module's photo and video recording functions to achieve fixed-point shooting and video recording while attached to a fixed wall. In human tracking mode, the human tracking function is activated, and tracking information and target feature point positions are continuously sent to the transport module. After calculation, attitude control commands are obtained and sent to the motion module. The motion module changes the robot's attitude to keep the tracked object in the center of the shooting frame.

[0014] Furthermore, in step S100, the environmental data acquired by the image data acquisition module needs to be transformed from the camera coordinate system to the world coordinate system:

[0015]

[0016] in, The transformation moments from the camera coordinate system to the world coordinate system are... For rotation matrix, The translation vector that describes the camera position. and These represent the point coordinates in camera coordinates and world coordinates, respectively.

[0017] Furthermore, in step S200, when performing the autonomous flight mission, the robot first constructs a 3D map based on the SLAM algorithm. The SLAM backend uses a least squares model based on multi-sensor information to minimize the projection error.

[0018]

[0019] in, For robot pose parameters, Let represent the pixel coordinates of the j-th feature point in the i-th frame of the image, n represent the total number of frames, and m represent the total number of feature points. This represents a camera projection model with intrinsic distortion correction. Let j be the world coordinates of the j-th map point;

[0020] Next, using the A* algorithm, starting from the current position and ending at the preset point B, a collision-free optimal path is searched in the constructed map. During flight, the flying robot continuously scans the area in front through sensors. Once an obstacle is detected, the dynamic obstacle avoidance algorithm in the computing module is immediately activated until it moves to the preset point B and switches to the set body posture.

[0021] Target tracking employs the artificial potential field method, with the gravitational potential field being:

[0022]

[0023] in, The gravitational coefficient, Let Euclidean distance be the current position and the target point, and the repulsive potential field be:

[0024]

[0025] in, The repulsion coefficient, The distance between the robot and the obstacle. The threshold value represents the area affected by the obstacle; the force acting on the robot is the vector sum of attractive and repulsive forces, and the motor output is adjusted via a PID controller.

[0026]

[0027] in, For position error, These are the proportional, integral, and differential coefficients.

[0028] Further, in step S300, after the flying robot receives the autonomous mapping instruction through the multi-functional interactive module, the waiting interface of the display module shows the preparation for scanning feedback information. Then, the visual label is scanned three times through the image data acquisition module. After the display module interface shows the ready feedback information, the initial positioning is reported again. During this process, the flying robot delays for 3 seconds before mapping. If the delay is not canceled, the autonomous mapping continues and the completion percentage and a thumbnail of the current mapping process are displayed. After the mapping is completed, when the user sends a map search instruction through the multi-functional interactive module, the corresponding image file is obtained by accessing the server based on the image ID.

[0029] Furthermore, after the flying robot receives the wall-attaching shooting instruction through the multi-functional interactive module, when there are no walls around the flying robot, the display module issues a prompt message indicating that the flying robot needs to be placed near a wall; when there are walls around the flying robot, the flying robot gradually attaches to the wall, and during this process, it periodically provides feedback on the wall-attaching status until the wall-attaching is successful.

[0030] Furthermore, in step S400, in humanoid tracking mode, the flying robot continuously transmits tracking distance, vertical / horizontal offset, and target feature point position at a speed of 10 frames / s; during this process, the tracking parameters are set by the display module, including the tracking distance and the position of the tracking object's center coordinates in the captured image.

[0031] Target tracking uses a Kalman filter model:

[0032]

[0033] in, and For state vectors, For external control input, The actual measured value at time k. For the prediction equation, For the observation equation, and These are process noise and observation noise, respectively.

[0034] Furthermore, steps S100-S400 also include a voice interaction function, allowing interaction with the flying robot by giving voice commands.

[0035] Furthermore, it also includes a fuselage light signal function, which uses programmable full-color LED light groups to define the color and flashing frequency of each LED light, thereby encoding a light signal to display the current working status and direction of movement of the flying robot.

[0036] An interactive system for a space intelligent flight robot in a microgravity environment, the system having program modules corresponding to the above steps, and executing the steps in the above-described interactive method for a space intelligent flight robot in a microgravity environment during runtime.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] ① Modular Design: The system adopts a layered and modular design concept, mainly including task scheduling, navigation and perception, environmental interaction, data processing and storage, and the core user interface module. The system design achieves functional decoupling between modules, and each module is integrated through a unified standard interface, which improves the system's scalability and maintainability, laying the foundation for subsequent functional upgrades and technology iterations.

[0039] ② Intelligent task scheduling and optimization: Through a flight robot operation mode switching strategy based on a voice command table, the system can intelligently adjust the task execution sequence in real time according to dynamic factors such as task requirements, robot spatial location, current battery level, task priority, and system resource status, thereby optimizing resource allocation and improving the efficiency and reliability of task execution.

[0040] ③ Optimized mapping capability integrating multi-source sensors: Integrating multimodal data from visual sensors (depth camera, binocular camera), temperature and humidity sensors, LiDAR, IMU inertial unit, etc., the system achieves high-precision environmental perception in microgravity environment through multi-source SLAM backend least squares optimization. Combined with deep learning and reinforcement learning algorithms, the system can intelligently analyze environmental information to ensure that the robot can achieve autonomous obstacle avoidance, accurate positioning and efficient navigation in complex environments.

[0041] ④ Multi-channel integrated human-machine interaction: It integrates three interaction modes: touch screen interface, voice interaction, and body light signals. The touch screen interface can receive complex task instructions and provide real-time feedback; voice interaction allows operators to quickly issue instructions without contact; and body light signals can provide real-time feedback on the robot's movement status, effectively improving safety and reliability. These three interaction modes complement each other, significantly improving human-machine interaction efficiency and enhancing operational convenience and task execution reliability.

[0042] In summary, this invention proposes a highly efficient, reliable, and safe data acquisition and human-computer interaction software system design suitable for microgravity environments. Through intelligent task scheduling and perception-navigation mechanisms, it ensures that the intelligent flying robot can efficiently and safely complete various tasks in complex microgravity environments. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating an interaction method for a space intelligent flight robot in a microgravity environment, as described in an embodiment of the present invention.

[0044] Figure 2 This is a flowchart of the wall-mounted shooting process of the multifunctional interactive module in an embodiment of the present invention;

[0045] Figure 3 This is a diagram of the standby mode interface of the multi-functional interactive module in an embodiment of the present invention;

[0046] Figure 4 This is a diagram of the autonomous flight mode of the multi-functional interactive module in an embodiment of the present invention;

[0047] Figure 5 This is a diagram of the autonomous mapping mode of the multifunctional interactive module in an embodiment of the present invention;

[0048] Figure 6 This is a diagram of the gimbal shooting mode of the multi-functional interactive module in an embodiment of the present invention;

[0049] Figure 7 This is a diagram of the wall-mounted shooting mode of the multi-functional interactive module in an embodiment of the present invention. Detailed Implementation

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Specific Implementation Plan 1: Combining Figures 1 to 7 As shown, this invention provides an interaction method for a space intelligent flying robot in a microgravity environment, comprising the following steps:

[0052] Data information is collected through the image data acquisition module. The collected information includes image information, which is acquired through high-definition and standard-definition video recording, including still images and dynamic videos.

[0053] Combination Figure 3 As shown, after the multi-functional interactive module is powered on, it automatically enters standby mode, and the image captured by the camera is displayed in the blank area of ​​the interface. The environmental data collected by the visual camera needs to be converted from the camera coordinate system to the world coordinate system.

[0054]

[0055] in, For rotation matrix, The translation vector that describes the camera position. The transformation moments from the camera coordinate system to the world coordinate system are... and These represent the point coordinates in camera coordinates and world coordinates, respectively.

[0056] "HD Recording" primarily improves clarity, resulting in larger video files; "SD Recording" primarily extends recording time; the "Take Photo" button is used to capture still images at maximum resolution; the "Video List" is used to view video files recorded by the aforementioned "HD Recording" and "SD Recording" functions, and supports playback and deletion functions; the "Image List" is used to view still images taken by the aforementioned "Take Photo" function, and supports gallery browsing, zooming, and deletion functions;

[0057] Emergency braking of the robot is achieved through the emergency control module. After the display module triggers the braking command, the flying robot stops all movement operations. After the display module triggers the emergency stop command, the underlying control algorithm fails completely and stops providing drive signals to the fan. On the display module, the button changes to a "Resume Operation" button. Clicking the button again restores the underlying control algorithm's control over the hardware. Both levels of emergency stop control commands can be triggered by button clicks and voice commands, ensuring the safety of the flying robot and operators as much as possible in the event of an accident.

[0058] The multi-functional interactive module acquires user commands and transmits them to the computing module. The computing module recognizes the commands and then constructs a 3D map and plans a path based on the commands. The constructed 3D map is displayed in real time on the display module to realize the mapping mode. The path planning commands are transmitted to the motion module to realize the underlying motion control and realize the autonomous flight mode. At the same time, the multi-functional interactive module displays the information during and after the calculation through the display module or voice.

[0059] Combination Figure 4As shown, after receiving a command to execute a command to a target point via voice or the touch signal channel of the display module, the flying robot first uses multiple sensors, such as a vision camera, to collect real-time data on the surrounding environment based on the SLAM algorithm, constructing a 3D map. The SLAM backend uses a least squares model based on multi-sensor information to minimize projection error.

[0060]

[0061] in, For robot pose parameters, Let represent the pixel coordinates of the j-th feature point in the i-th frame of the image, n represent the total number of frames, and m represent the total number of feature points. This represents a camera projection model with intrinsic distortion correction. Let j be the world coordinates of the j-th map point;

[0062] Next, using the A* algorithm, starting from the current position and ending at a preset point B, a collision-free optimal path is searched in the constructed map. During flight, the robot continuously scans the area ahead using sensors. Once an obstacle is detected, a dynamic obstacle avoidance algorithm is immediately activated until it reaches the preset point B and switches to the set body posture. The "X Warehouse," "A Warehouse," and "M Warehouse" buttons at the top of the diagram represent destination areas preset in the configuration file. Clicking these buttons displays two specific destination names: "B Segment C Quadrant" and "B Segment D Quadrant." Each name corresponds to a set of position coordinates and body posture information. Clicking the corresponding button sets the destination location and body posture. Users can say "Go" + location name (e.g., "Go to Warehouse A") or click the corresponding destination function button to query its XYZ coordinates and body posture parameters Q1, Q2, Q3, W (x, y, z). After the axis rotation and the real part of the quaternion are processed, the data is sent to the display module for display. The multi-functional interactive module receives the execution progress percentage message and displays it in the status bar at the bottom of the main interface. During execution, clicking the button on the lower right side of the screen will send a brake / emergency stop command to the server.

[0063] Target tracking employs the artificial potential field method, with the gravitational potential field being:

[0064]

[0065] in, The gravitational coefficient, Let Euclidean distance be the current position and the target point, and the repulsive potential field be:

[0066]

[0067] in, The repulsion coefficient, The distance between the robot and the obstacle. The threshold value represents the area affected by the obstacle; the force acting on the robot is the vector sum of attractive and repulsive forces, and the motor output is adjusted via a PID controller.

[0068]

[0069] in, For position error, These are the proportional, integral, and differential coefficients;

[0070] During 3D mapping, the flying robot can automatically maintain its current attitude and position, making it easy for operators to use as a stable recording device. Operators can control the start and stop of recording and taking photos via voice commands. Users send mapping preparation commands to the computing module through the multi-functional interactive module. When a "not ready" feedback message is received, a status bar pops up on the display module interface with a voice prompt "Initializing, please wait...". When a "preparing to scan visual tags" feedback message is received, a status bar pops up with a voice prompt "Please point the camera at visual tag ①". When a "scanned tag ①" feedback message is received, a status bar pops up with a voice prompt "Visual tag ① found, please point the camera at visual tag ②". When a "scanned tag ②" feedback message is received... When feedback is received, a status bar pops up with a voice prompt, "Visual tag #2 has been found. Please point the camera at visual tag #3." When the "Ready" feedback is received, a status bar pops up with a voice prompt, "Initial positioning successful. Mapping will begin soon. Please release the flying robot." After a 3-second delay, a start mapping command is sent to the computing module. If the end button is clicked during the 3-second delay, mapping is canceled, and the system returns to standby mode. The multi-functional interaction module receives the execution progress percentage message and displays it in a status bar below the main interface of the display module. During execution, push messages are displayed in the display module showing a thumbnail of the current mapping process. When the user sends a map search command through the multi-functional interaction module, the system accesses the server to obtain the corresponding image file based on the image ID.

[0071] When the user sends gimbal shooting commands to the computing module through the voice or touch signal channel of the multi-functional interactive module, including wall-mounted shooting and human tracking modes, the system has an environmental detection function. When there are no walls around the device, the multi-functional interactive module will issue a wall-less warning. When there are walls around the device, the motion module will perform a wall-mounted action, and the image data acquisition module will take pictures and record videos, achieving fixed-point shooting and video recording while attached to a fixed wall. In human tracking mode, the human tracking function is activated, and tracking information and the position of target feature points are continuously sent. The tracking object is kept in the center of the shooting frame by changing the posture of the device.

[0072] To achieve the robot's equilibrium in a microgravity environment, the adsorption force F吸附力 The equilibrium conditions under microgravity must be met:

[0073]

[0074] in, For robot quality, For microgravity acceleration, The coefficient of friction, The impact of ambient airflow on the robot;

[0075] Combination Figure 6 As shown, in human tracking mode, when a human tracking command is sent to the computing module via the multi-functional interaction module, a "Human Tracking" button is displayed on the right side of the display module. The multi-functional interaction module can then send start / pause / continue / end commands for human tracking to the computing module. During human tracking, voice commands are received at any time, and the offset set in the voice command is sent. Clicking "Camera Position Adjustment" on the display module displays a command button to send the corresponding command. Clicking the corresponding button sends translation and rotation offsets to the motion module; the offsets include translation and rotation. Clicking the function button on the right side of the screen on the display module toggles the "Human Tracking" switch, and the switch status is displayed on the display module. When "Human Tracking" is active, the human tracking mode is activated. When the "Human Tracking" switch is on, it continuously transmits tracking distance, vertical / horizontal offset, and target feature point position at the fastest possible speed (ideally 10 frames per second or more). The transmission method is similar to the autonomous tracking mode. When "Human Tracking" is on, you can set specific tracking parameters by clicking the "Parameter Adjustment" button on the interface in the display module. When changing the "Following Distance" option, you can approach or move away from the target by changing the XYZ coordinates of the camera body, while keeping the camera body posture unchanged. When you click the "Up," "Down," "Left," or "Right" buttons, the XYZ coordinates of the camera body remain unchanged, while the position of the center coordinates of the tracked object in the shooting frame is achieved by changing the body posture.

[0076] Target tracking uses a Kalman filter model:

[0077]

[0078] in, and For state vectors, For external control input, The actual measured value at time k. For the prediction equation, For the observation equation, and These are process noise and observation noise, respectively.

[0079] like Figure 7As shown, in the wall-attaching shooting mode, the flying robot receives execution commands through the voice or touch signal channel of the multi-functional interaction module, autonomously explores and records a 3D map; after switching to the wall-attaching shooting mode, the multi-functional interaction module sends a command to the computing module to start the wall-attaching mode, including start / pause / continue / end commands; the multi-functional interaction module receives the wall-attaching action execution status message and displays it in a status bar at the bottom of the main interface of the display module. If the received status feedback is 3 (no wall), the wall-attaching operation cannot be performed, and the display module interface displays the prompt message "There is no wall behind the screen, please place the robot near a wall," and makes a voice announcement; after receiving the command to find the wall, the computing module sends a ready status, progress 0%, and the multi-functional interaction module prompts "The robot will move to attach to the back of the screen," and makes a voice announcement; after a 3-second delay, the computing module automatically sends a start command. If the emergency stop button is clicked during the delay, the wall-attaching is canceled, the computing module automatically sends an end command, and returns to the standby mode; after the delay ends, the computing module starts the wall-attaching servo algorithm, based on A* The algorithm generates a wall-hugging trajectory and sends it to the motion module to adjust the robot's translational speed, rotation angle, and vertical distance from the wall in real time. Every 500ms, it sends wall-hugging status data, including the current progress percentage, real-time attitude angle, and actual distance from the wall, to the storage module via UDP protocol. When the calculation module determines that the wall-hugging progress has reached 100% using a trajectory odometry or visual optical flow algorithm, and this is confirmed by the positioning detection algorithm, the display module displays "Wall-hugging successful" and announces it via voice. Simultaneously, it enters the shooting-ready mode, awaiting the next instruction.

[0080] The application software proposed in this invention is developed based on the Android environment and can be adapted to and run smoothly on devices equipped with the HarmonyOS operating system. The program version number is 1.0.0, and its core functions focus on realizing touch screen interaction and voice interaction.

[0081] ① Touchscreen interaction

[0082] To meet the needs of intelligent flying robots in executing complex commands, a multi-functional interactive module designed for touchscreen operation was developed. With its intuitive and easy-to-understand interface layout and simple and efficient operation logic, the system enables operators to easily use the various functions of the flying robot. The touchscreen interface not only covers diverse functions such as mode switching, process start and stop, photo / video control, and system parameter configuration, but also displays the communication status with the onboard navigation system and bone conduction headphones in real time, ensuring instant information synchronization.

[0083] ② Voice interaction

[0084] To meet astronauts' needs for contactless operation of intelligent flying robots, an innovative voice interaction system based on bone conduction headphones (also a multi-functional interaction module) was designed. This system can receive astronauts' voice commands through bone conduction headphones or the microphone built into the aircraft, analyze the intent of the commands through voice recognition algorithms, and convert them into corresponding operation commands. As feedback to the voice commands, the system will highlight the corresponding operation command button on the touch screen while executing the voice commands to improve the reliability and convenience of the interaction.

[0085] The list of voice commands supported by the system is shown in Table 1. Among them, the fine mode enables the robot to provide more accurate spatial positioning, while the fast mode maximizes the robot's movement speed to achieve rapid movement while meeting basic safety conditions.

[0086] Table 1 Voice Command Table

[0087]

[0088] The human-computer interface consists of two parts: a touchscreen interface and body-mounted indicator lights. The touchscreen interface serves as the primary human-computer interaction method, enabling two-way interaction with the operator and also acting as a visual feedback platform for voice interaction, displaying the voice interaction status in real time.

[0089] ① Touchscreen interface design

[0090] In terms of user-friendly human-computer interaction design, the software incorporates function buttons, selection boxes, and directional controls on the touchscreen interface to help operators quickly grasp the interactive content and ensure simple and efficient execution of operation commands. At the same time, the touchscreen interface can reflect the real-time operation status of the airborne navigation system, providing operators with intuitive system status monitoring and facilitating them to quickly issue control commands according to actual needs.

[0091] ② Design of fuselage lights

[0092] To address the challenge of communicating the pose and behavior of flying robots during human-robot collaborative operations, the flying robot is equipped with a programmable full-color LED light array as its fuselage lighting signal system. Operators can flexibly define the color and flashing frequency of each LED, encoding them into signals to accurately indicate the robot's current operating status and direction of movement. For example, when an operator sees the robot displaying a solid red light with a rapidly flashing purple tail LED, they can immediately determine that the robot is undergoing a forced descent due to low battery and requires immediate attention. Conversely, a blue breathing flashing light combined with a flowing yellow light on the left clearly indicates that the robot is in standby mode and moving to the left. This lighting signal design effectively ensures the safety of operators during on-orbit operations while significantly improving the efficiency of human-robot collaborative operations by reducing communication costs.

[0093] Specific Implementation Scheme 2: The present invention provides an interactive system for a space intelligent flight robot in a microgravity environment. The system has a program module corresponding to the above steps, and executes the steps in the above-mentioned interactive method for a space intelligent flight robot in a microgravity environment when running.

[0094] The other combinations and connections in this implementation scheme are the same as in Specific Implementation Scheme 1.

[0095] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. An interaction method for a space intelligent flying robot in a microgravity environment, characterized in that, Includes the following steps: S100: When the flying robot is in standby mode, it collects data information through the image data acquisition module and realizes emergency braking of the robot through the emergency control module, including braking and emergency stop. S200. When the flying robot is in autonomous flight mode, it obtains the target point execution command and set body posture issued by the user through the multi-functional interaction module. The flying robot uses the computing module to autonomously plan and avoid obstacles to reach the target point based on the data collected by the image data acquisition module and the surrounding environment data collected by the airborne sensors, and then switches to the set body posture. When performing autonomous flight missions, the robot first constructs a 3D map based on the SLAM algorithm. The SLAM backend uses a least squares model based on multi-sensor information to minimize projection error. in, For robot pose parameters, Let represent the pixel coordinates of the j-th feature point in the i-th frame of the image, n represent the total number of frames, and m represent the total number of feature points. This represents a camera projection model with intrinsic distortion correction. Let j be the world coordinates of the j-th map point; The transformation moments from the camera coordinate system to the world coordinate system; Next, using the A* algorithm, starting from the current position and ending at the preset point B, a collision-free optimal path is searched in the constructed map. During flight, the flying robot continuously scans the area in front through sensors. Once an obstacle is detected, the dynamic obstacle avoidance algorithm in the computing module is immediately activated until it moves to the preset point B and switches to the set body posture. Target tracking employs the artificial potential field method, with the gravitational potential field being: in, The gravitational coefficient, Let Euclidean distance be the current position and the target point, and the repulsive potential field be: in, The repulsion coefficient, The distance between the robot and the obstacle. The threshold value represents the area affected by the obstacle; the force acting on the robot is the vector sum of attractive and repulsive forces, and the motor output is adjusted via a PID controller. in, For position error, These are the proportional, integral, and differential coefficients; S300 When the flying robot is in mapping mode, the flying robot maintains its current or set pose, performs initial localization by scanning visual tags, then performs mapping, and displays the map on the display module using the multi-functional interactive module. S400: When the flying robot is in gimbal shooting mode, including wall-mounted shooting mode and human tracking mode, in wall-mounted shooting mode, the image data acquisition module detects whether there is a wall around the flying robot. If there is no wall, a wall-no warning is issued. When there is a wall, the robot performs a wall-mounted action and activates the image data acquisition module's photo and video recording functions to achieve fixed-point shooting and video recording while attached to a fixed wall. In human tracking mode, the human tracking function is activated, and tracking information and target feature point positions are continuously sent to the transport module. After calculation, attitude control commands are obtained and sent to the motion module. The motion module changes the robot's attitude to keep the tracked object in the center of the shooting frame.

2. The interaction method for a space intelligent flight robot in a microgravity environment according to claim 1, characterized in that: In step S100, the environmental data acquired by the image data acquisition module needs to be transformed from the camera coordinate system to the world coordinate system: in, The transformation moments from the camera coordinate system to the world coordinate system are... Let be a rotation matrix. The translation vector that describes the camera position. and These represent the point coordinates in camera coordinates and world coordinates, respectively.

3. The interaction method for a space intelligent flight robot in a microgravity environment according to claim 2, characterized in that: In step S300, after the flying robot receives the autonomous mapping command through the multi-functional interaction module, the waiting interface of the display module shows the preparation for scanning feedback information. Then, the visual label is scanned three times through the image data acquisition module. After the display module shows the ready feedback information, the initial positioning is reported as successful again. During this process, the flying robot delays for 3 seconds before mapping. If the delay is not canceled, the autonomous mapping continues and the completion percentage and a thumbnail of the current mapping process are displayed. After the mapping is completed, when the user sends a map search command through the multi-functional interaction module, the corresponding image file is obtained by accessing the server based on the image ID.

4. The interaction method for a space intelligent flight robot in a microgravity environment according to claim 3, characterized in that: After the flying robot receives the wall-attaching shooting command through the multi-functional interactive module, when there are no walls around the flying robot, the display module will issue a prompt message indicating that the flying robot needs to be placed near a wall; when there are walls around the flying robot, the flying robot will gradually attach to the wall, and during this process, it will periodically provide feedback on the wall-attaching status until it successfully attaches to the wall.

5. The interaction method for a space intelligent flight robot in a microgravity environment according to claim 4, characterized in that: In step S400, in humanoid tracking mode, the flying robot continuously transmits tracking distance, vertical / horizontal offset, and target feature point position at a speed of 10 frames / s; during this process, the tracking parameters are set by the display module, including the tracking distance and the position of the tracking object's center coordinates in the captured image. Target tracking uses a Kalman filter model: in, and For state vectors, For external control input, The actual measured value at time k. For the prediction equation, For the observation equation, and These are process noise and observation noise, respectively.

6. The interaction method for a space intelligent flight robot in a microgravity environment according to claim 5, characterized in that: Steps S100-S400 also include a voice interaction function, which allows users to interact with the flying robot by giving voice commands.

7. The interaction method for a space intelligent flight robot in a microgravity environment according to claim 6, characterized in that: It also includes a fuselage light signal function, which uses programmable full-color LED light groups to define the color and flashing frequency of each LED light, thereby encoding a light signal to display the current working status and direction of movement of the flying robot.

8. An interactive system for a space intelligent flight robot in a microgravity environment, characterized in that: The system has a program module corresponding to the steps described in any one of claims 1-7, and executes the steps in the above-described interaction method for a space intelligent flying robot in a microgravity environment when it is run.

Images