Insect robot automatic obstacle avoidance and trajectory recording system and method

Abstract

The application discloses an automatic obstacle avoidance and trajectory recording system and method for an insect robot, and belongs to the fields of electronic information technology, computer science technology and biological science technology.The application provides an automatic obstacle avoidance and trajectory recording method and system for an insect robot based on a new generation of information technology, including integrated technology, sensor technology, wireless communication, software technology and the like.In terms of effect, the system can realize real-time sensing of the environment around the insect robot, automatically avoid obstacles in the environment, and record the motion trajectory of the insect robot.In addition, experimenters can monitor and analyze the motion of the insect robot through the system, so that the accuracy and efficiency of experiments are improved.In the long run, the application provides a new idea for the intelligent development of the insect robot.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of electronic information technology, computer science and technology and biological science technology, and particularly relates to an automatic obstacle avoidance and trajectory recording system and method for an insect robot. BACKGROUND

[0002] Insect robots have great application potential in disaster rescue, industrial detection, military reconnaissance, scientific investigation and special engineering fields due to their excellent movement ability, high flexibility and strong concealment, especially in scenes where personnel are difficult to reach or there are safety hazards such as earthquake debris rescue, dangerous environment survey and closed space exploration, insect robots can effectively reduce the risk of personnel and significantly improve the work efficiency. By controlling the movement behavior of insects through electric stimulation and other methods, insect robots have the advantages of low energy consumption, strong load capacity and good environmental adaptability. At present, domestic and foreign scholars mainly focus on the stimulation method, movement control and behavior analysis of insect robots.

[0003] In the prior art, the control method of the insect robot mainly depends on manual operation. The experimenters manually send control instructions by observing the movement state of the insect. This method has the following problems: first, the real-time performance and accuracy of manual control are poor, and it is difficult to deal with unexpected situations in complex environments; second, the insect robot lacks self-perception ability of the environment and cannot effectively avoid obstacles; third, the real-time movement trajectory recording and analysis method is less used in insect robots, which is not conducive to the optimization of control strategies. In order to improve the environmental perception ability of the insect robot, some researchers try to carry various sensors on the back of the insect. However, due to the small size and limited load capacity of the insect, traditional depth cameras, laser radars and other devices are too large and heavy to be practically applied. In addition, most researches only focus on the implementation of a single function, and lack a comprehensive solution that organically combines environmental perception, autonomous obstacle avoidance and trajectory recording functions.

[0004] Therefore, it is necessary to study an intelligent control system for an insect robot that integrates environmental perception, autonomous obstacle avoidance and trajectory recording functions. The system should have a lightweight environmental perception scheme, an intelligent obstacle avoidance decision mechanism, an accurate real-time trajectory recording function and a friendly human-computer interaction interface to improve the movement control ability of the insect robot in complex environments and the experimental analysis efficiency. SUMMARY

[0005] In view of the above technical problems in the prior art, the present application provides an automatic obstacle avoidance and trajectory recording system and method for an insect robot, which is reasonable in design, overcomes the shortcomings of the prior art and has good effects.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] The application discloses an automatic obstacle avoidance and trajectory recording system for an insect robot, which comprises an environment perception module, a depth estimation module, an obstacle avoidance decision module, a wireless communication module, a control signal generation module, an image acquisition module, a motion trajectory recording module and a host computer control display unit module.

[0008] The environment perception module is configured to acquire a first visual angle image stream of the insect robot through a monocular camera installed on a microcontroller on the back of the insect, and record environment information.

[0009] The depth estimation module is configured to extract the image stream of the environment perception module, generate a real-time depth map by using a deep learning algorithm, and obtain relative distance information between the insect robot and obstacles.

[0010] The obstacle avoidance decision module is configured to formulate an obstacle avoidance strategy through a depth value according to the depth map generated by the depth estimation module, and generate corresponding control instructions.

[0011] The wireless communication module is configured to realize wireless data transmission between the modules of the system, and ensure real-time transmission of instructions and data.

[0012] The control signal generation module is configured to receive corresponding instructions transmitted by the wireless communication module, and generate corresponding control stimulation signals. The control signal generation module is connected with four output ports of the microcontroller, and each output port is connected to a microelectrode implanted in the insect in advance.

[0013] The image acquisition module is configured to use an industrial camera to capture motion images of the insect robot, and transmit image data to the motion trajectory recording module.

[0014] The motion trajectory recording module is configured to process images provided by the image acquisition module, record motion trajectories based on a target tracking algorithm, and mark different control instructions with corresponding identifiers on the trajectories.

[0015] The host computer control display unit module is configured to provide a man-machine interactive interface, display pictures of real-time video streams, depth maps and motion trajectories, realize bidirectional data transmission between the microcontroller and the host computer by using a communication protocol, support switching between manual control and automatic obstacle avoidance functions, and set picture, data and waveform parameter storage options.

[0016] Further, the environment perception module accesses WiFi by configuring a microcontroller on the back of the insect robot, and uses a camera to acquire image pictures, so that the environment perception capability of the insect robot is enhanced.

[0017] The microcontroller is arranged on the back of the insect robot, and a row of pins is led out from output ports of the microcontroller and inserted into a row of female pins fixed on the back of the insect.

[0018] The nymph is fixed to the back of the insect by electrodes made of 0.16mm stainless steel needles with a length of 3-8mm. The nymph is connected to its corresponding stimulation site by a lead wire.

[0019] Furthermore, the depth estimation module adopts the Depth Anything V2 depth estimation model and visualizes the depth map using the INFERNO color mapping scheme. The depth value reflects the distance of the obstacle from the insect robot, and the larger the depth value, the closer the obstacle is.

[0020] Furthermore, the obstacle avoidance decision module divides the depth map into three regions: center, left, and right. It extracts the minimum depth value of the center region to determine obstacles ahead and calculates the average depth value of the left and right regions to determine the turning direction. When the minimum depth value of the center region is greater than a set threshold, the system determines that there is an obstacle ahead and selects to turn in the direction with the larger depth value by comparing the difference between the average depth values ​​of the left and right regions.

[0021] Furthermore, the wireless communication module enables bidirectional communication between the microcontroller and the host computer based on the HTTP protocol, transmits video streams and control commands via WiFi, sets a 5-second communication timeout protection mechanism, and adds time interval control when sending commands to ensure control stability.

[0022] Furthermore, the stimulation signal generated by the control signal generation module is a square wave pulse signal with adjustable high and low level time and pulse number. The high and low level time is 10ms, one pulse period is 20ms, and the number of pulses is adjustable, thereby realizing the forward and turning control of the insect robot.

[0023] Furthermore, the image acquisition module uses an industrial camera to acquire images, implements image processing using the Python programming language, obtains raw image data from the industrial camera through the MVSDK interface, and after image processing and buffer management, converts the data into a format that OpenCV can process.

[0024] Furthermore, the motion trajectory recording module uses the KCF algorithm to achieve target tracking. The initial tracking target is determined by selecting with the mouse, and the system automatically draws a yellow bounding box to mark the target position. The motion path is recorded with a red trajectory line, and different shapes and colors of markers are used on the trajectory to represent different control commands.

[0025] Furthermore, the host computer control and display unit module is developed based on the PyQt5 framework. It uses multi-threading to process different algorithms to improve the system response speed, displays video streams, depth maps, and motion trajectories in real time, and provides switching options for manual control and automatic obstacle avoidance functions. It also supports recording interactive interface options.

[0026] Furthermore, this invention also mentions a method for automatic obstacle avoidance and trajectory recording of an insect robot. This method employs the automatic obstacle avoidance and trajectory recording system for insect robots described above, and specifically includes the following steps:

[0027] Step 1: The experimenter connects the slave device of the system to the microelectrode pre-implanted in the insect, initializes the microcontroller and camera, configures the camera parameters and establishes a WiFi connection, and acquires first-person view images of the insect through the camera interface. The image data is transmitted to the host computer for display via the wireless communication module.

[0028] Step 2: For the first-person video stream, use the Depth Anything V2 depth estimation model to generate a depth map, and use the INFERNO color mapping scheme for visualization. The depth values ​​in the depth map reflect the relative distance between obstacles and the insect robot.

[0029] Step 3: The obstacle avoidance decision module divides the depth map into three regions: center, left, and right. It extracts the minimum depth value of the center region to determine obstacles ahead, calculates the average depth value of the left and right regions to determine the turning direction, and generates obstacle avoidance control commands based on the depth value comparison results.

[0030] Step 4: Control commands are sent to the microcontroller via the HTTP protocol. The control signal generation module selects the corresponding output port according to the received command and generates a pulse signal with a high and low level time of 10ms. The pulse signal is applied to different positions in the insect's body through microelectrodes to realize basic movement control such as left turn, right turn, and forward movement. The response of the insect to the stimulus is monitored. When the response is weakened, the user is prompted to adjust the stimulus parameters or change the stimulus waveform. The stimulus fatigue is reduced by increasing the number of pulses.

[0031] Step 5: The industrial camera captures the overall motion of the insect robot in real time through the MVSDK interface. After preprocessing, the image data is transmitted to the trajectory recording module, where the KCF algorithm is used for target tracking and the target position information is updated in real time.

[0032] Step 6: The trajectory recording module draws a yellow bounding box on the image to mark the current position, records the movement trajectory with red lines, and marks the position where the control command is applied on the trajectory. It establishes a proportional mapping relationship between the insect's actual position and the display interface. Each time a control command is sent, different marking symbols are used on the trajectory to record the stimulus position, which is convenient for analyzing the control effect.

[0033] Step 7: The host computer display control unit provides two control modes: manual and automatic. In manual mode, the experimenter can send control commands through the interface buttons. In automatic mode, the system automatically generates obstacle avoidance commands based on depth information to ensure the safe movement of the insect robot.

[0034] Step 8: The first-person view image, depth map, motion trajectory, control commands and stimulus parameters of the insect robot can be saved and recorded for researchers to view and analyze, providing a basis for optimizing control strategies.

[0035] The beneficial technical effects of this invention are as follows:

[0036] This invention achieves real-time environmental perception for insect robots by combining a lightweight monocular camera with a depth estimation algorithm, overcoming the limitations of traditional depth sensors that are large and heavy. Based on depth information, an intelligent obstacle avoidance decision-making mechanism enables the insect robot to autonomously avoid obstacles. The system also integrates precise trajectory recording, allowing researchers to analyze and optimize the control effect. Furthermore, multi-threaded parallel processing and a user-friendly human-machine interface enhance the system's real-time performance and ease of use. This invention provides a complete intelligent control solution for insect robots, improving their application capabilities in complex environments.

[0037] This invention proposes a method and system for automatic obstacle avoidance and trajectory recording of insect robots based on next-generation information technologies (including integration technology, sensor technology, wireless communication, software technology, etc.). In terms of effectiveness, it enables real-time perception of the insect robot's surrounding environment, automatic obstacle avoidance, and recording of the insect robot's movement trajectory. Researchers can use this system to monitor and analyze the insect robot's movement, thereby improving the accuracy and efficiency of experiments. In the long term, it provides a new approach for the intelligent development of insect robots. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating an automatic obstacle avoidance and trajectory recording method for an insect robot according to the present invention.

[0039] Figure 2 This is a schematic diagram of the overall structure of an automatic obstacle avoidance and trajectory recording system and method for an insect robot according to the present invention.

[0040] Figure 3 This is a schematic diagram of the motion trajectory recording module of the present invention.

[0041] Figure 4 This is a schematic diagram of the host computer display control unit module of the present invention.

[0042] In the diagram: 1-Host computer control and display unit module; 2-Network data transmission; 3-Industrial camera; 4-Camera bracket; 5-Environmental obstacles; 6-Electrodes implanted in insect body parts; 7-Microcontroller; 8-Madagascar cockroach insect carrier; 9-Insect back fixation cradle; 10-Experimental ground platform; 11-Communication transmission signal; 12-Wireless communication module; 13-Trajectory recording display screen; 14-Forward command identifier; 15-Left turn command identifier; 16-Insect robot movement trajectory; 17-Right turn command identifier; 18-Host computer real-time video stream Display screen; 19 - Real-time depth map display screen of host computer; 20 - Real-time trajectory recording screen of host computer; 21 - Left turn control button; 22 - Forward control button; 23 - Right turn control button; 24 - Video storage switch button; 25 - Information storage switch button; 26 - Depth control switch button; 27 - Stimulus signal high level duration setting option; 28 - Stimulus signal low level duration setting option; 29 - Stimulus signal pulse count setting option; 30 - Minimum depth value of the central area; 31 - Average depth value of the left area; 32 - Average depth value of the right area. Detailed Implementation

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0044] Example 1:

[0045] like Figure 1 , Figure 2 As shown, an automatic obstacle avoidance and trajectory recording system for an insect robot includes an environmental perception module, a depth estimation module, an obstacle avoidance decision module, a wireless communication module, a control signal generation module, an image acquisition module, a motion trajectory recording module, and a host computer control and display unit module.

[0046] The depth estimation module, obstacle avoidance decision-making module, wireless communication module, image acquisition module, motion trajectory recording module, and upper computer control and display unit module are integrated into the host unit; the environmental perception module and control signal generation module are integrated into the slave unit. The slave unit is a microcontroller carried on the back of the insect robot, which can be easily disassembled and operated outside of experimental periods. A mounting plate 9 is fixed to the insect's back, and electrodes 6, made of 0.16mm stainless steel needles with a length of 3-8mm, are implanted into the insect's body. The mounting plate is connected to its corresponding stimulation site via leads. The needles are led out from the slave unit's output port and inserted into the mounting plate to generate and transmit stimulation signals to the stimulation site.

[0047] The microcontroller 7 uses a XIAO ESP32S3 manufactured by Seeed Studio and is powered by a 3.7V rechargeable polymer lithium battery.

[0048] The environmental perception module uses an OV2640 camera with a maximum resolution of 1600×1200. It is connected to the microcontroller via an I2C interface to collect real-time first-person perspective images of the insect. The camera is fixed to the front of the insect's back with a field of view of 60 degrees, which can clearly capture environmental information in front of it.

[0049] The depth estimation module uses the Depth Anything V2 depth estimation model, which is lightweight and can perform real-time inference on a GPU. The model input is a monocular RGB image and the output is a depth map, which is visualized using the INFERNO color mapping scheme, where dark colors represent far distances and light colors represent near distances, and the depth value ranges from 0 to 10.

[0050] The obstacle avoidance decision module divides the depth map into three regions: center, left, and right. It formulates an obstacle avoidance strategy by comparing the depth values ​​of each region. The center region (1 / 3 width × 1 / 3 height) is used to detect obstacles 5 in the environment ahead, while the left and right regions (each 1 / 2 width) are used to determine the turning direction. When the minimum depth value of the center region is greater than the obstacle detection threshold, the system determines that there is an obstacle ahead; when the difference in the average depth values ​​of the left and right regions is greater than the turning detection threshold, the system chooses to turn in the direction with the larger depth value.

[0051] The wireless communication module 12 implements data transmission between various modules of the system based on the HTTP protocol, transmits video streams and control commands via WiFi, sets a 5-second communication timeout protection mechanism, and adds a 3.5-second time interval control when sending commands to ensure the stability of the communication transmission signal 11. It supports a disconnection reconnection mechanism and can automatically restore the communication connection.

[0052] The stimulation signal generated by the control signal generation module is a square wave pulse signal with adjustable high and low level time and pulse number. The high and low level time is 10ms, the pulse period is 20ms, and the pulse number can be adjusted according to the state of the insect robot. It is connected to microelectrodes pre-implanted in the insect body through four output ports to realize basic motion control such as left turn, right turn, and forward movement. In addition, a grounding pin is led out and implanted in the insect body.

[0053] The image acquisition module uses a Medvision industrial camera 3. The camera and the computer communicate via Ethernet data transmission 2. The original image data with a resolution of 1280×720 is acquired through the MVSDK interface using the Python programming language. After image processing and buffer management, the data is converted into a format that can be processed by OpenCV. The industrial camera is fixed above the experimental platform by a camera bracket 4 to capture the overall movement of the insect robot.

[0054] The motion trajectory recording module, as shown below Figure 3As shown, the trajectory recording display screen 13 shows the movement trajectory 16 of the insect robot. The KCF algorithm is used for target tracking. The initial tracking target is determined by selecting it with the mouse, and the system automatically draws a yellow bounding box to mark the target position. The movement trajectory is recorded with red lines. Different shapes and colors of markers are used on the trajectory to represent different control commands: a yellow triangle represents a forward command identifier 14, a blue cross represents a left turn command identifier 15, and a green diamond represents a right turn command identifier 17.

[0055] The host computer control and display unit module 1 is as follows: Figure 4 As shown, the system is designed based on the PyQt5 framework and employs multi-threading to process different algorithms to improve system response speed. The interface layout uses a three-screen side-by-side display: the host computer's real-time video stream display (18), the host computer's real-time depth map display (19), and the host computer's real-time trajectory recording display (20). It provides manual control and depth control switch buttons (26) to switch between automatic obstacle avoidance functions, supports data recording and image saving, and includes a depth information display area that displays the minimum depth value in front and the average depth value in the left and right areas in real time. The manual control options include left turn control button (21), forward control button (22), and right turn control button (23), and also include options for setting the duration of the high-level stimulus signal (27), the duration of the low-level stimulus signal (28), and the number of stimulus signal pulses (29). Adjusting different settings triggers corresponding control commands to achieve left / right turns and acceleration commands. The data recording and image saving functions are implemented by setting the video storage switch button 24 and the information storage switch button 25. Information recording can record the minimum depth value 30 of the central area, the average depth 31 of the left area, and the average depth value 32 of the right area in real time. Image saving can store the three real-time images so that the experimenters can view, analyze and optimize the control strategy.

[0056] Example 2:

[0057] Based on Embodiment 1 above, the present invention also provides a method for automatic obstacle avoidance and trajectory recording of an insect robot, using a Madagascar cockroach insect carrier 8 as an example for specific illustration, which specifically includes the following steps:

[0058] Step 1: The experimenter connects the slave device of the system to the microelectrode pre-implanted in the body of a Madagascar cockroach, places it on the experimental ground platform 10, initializes the microcontroller and camera, configures the camera parameters and establishes a WiFi connection, and collects first-person perspective images of the insect through the camera interface. The image data is transmitted to the host computer for display via the wireless communication module.

[0059] Step 2: For the first-person video stream, call the Depth Anything V2 depth estimation model to generate a depth map, and use the INFERNO color mapping scheme for visualization. The depth values ​​in the depth map reflect the relative distance between obstacles and the cockroach robot.

[0060] Step 3: The obstacle avoidance decision module divides the depth map into three regions: center, left, and right. It extracts the minimum depth value of the center region to determine obstacles ahead, calculates the average depth value of the left and right regions to determine the turning direction, and generates obstacle avoidance control commands based on the depth value comparison results.

[0061] Step 4: Control commands are sent to the microcontroller via the HTTP protocol. The control signal generation module selects the corresponding output port according to the received command and generates a pulse signal with a high and low level time of 10ms. This pulse signal is applied to different positions inside the insect through microelectrodes to achieve basic movement control such as left turn, right turn, and forward movement. The response of the cockroach to the stimulus is monitored. When the response is weakened, the experimenter is prompted to adjust the stimulus parameters or change the stimulus waveform. The stimulus fatigue is reduced by increasing the number of pulses.

[0062] Step 5: The industrial camera captures the overall motion of the insect robot in real time through the MVSDK interface. After preprocessing, the image data is transmitted to the trajectory recording module, where the KCF algorithm is used for target tracking and the target position information is updated in real time.

[0063] Step 6: The trajectory recording module draws a yellow bounding box on the image to mark the current position, records the movement trajectory with red lines, and marks the position where the control command is applied on the trajectory. It establishes a proportional mapping relationship between the insect's actual position and the display interface. Each time a control command is sent, different marking symbols are used on the trajectory to record the stimulus position, which is convenient for analyzing the control effect.

[0064] Step 7: The host computer display control unit provides two control modes: manual and automatic. In manual mode, the experimenter can send control commands through the interface buttons. In automatic mode, the system automatically generates obstacle avoidance commands based on depth information to ensure that the cockroach robot automatically avoids obstacles in the environment.

[0065] Step 8: The first-person view image, depth map, motion trajectory, control commands and stimulation parameters of the cockroach robot can be saved and recorded for researchers to view and analyze, providing a basis for optimizing control strategies.

[0066] This specific implementation method can be widely used in various types of insect robot control research, and can also be applied to more animal robots for further behavioral analysis and exploration. This embodiment provides a method and system for automatic obstacle avoidance and trajectory recording of insect robots based on new-generation information technologies (including integration technology, sensor technology, wireless communication, software technology, etc.), providing a fundamental method for the development of animal robot technology, promoting the application and development of biorobot technology in various complex environments, and has broad application space and development potential.

[0067] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. An insect robot automatic obstacle avoidance and trajectory recording method, characterized in that: Specifically includes the following steps: Step 1: The experimenter connects the system from the machine with the microelectrode implanted in the insect body in advance, initializes the microcontroller and monocular camera, configures the camera parameters and establishes a WiFi connection with the microcontroller, collects the first perspective image of the insect through the monocular camera interface, and transmits the image data to the host computer display through the wireless communication module; Step 2: For the first perspective video stream, call the Depth Anything V2 depth estimation model to generate a depth map, and use the INFERNO color mapping scheme for visualization processing. The depth value in the depth map reflects the relative distance between the obstacle and the insect robot; Step 3: Divide the depth map into three regions: center, left and right, through the obstacle avoidance decision module. Extract the minimum depth value in the center region to judge the front obstacle, calculate the average depth value in the left and right regions to determine the turning direction, and generate obstacle avoidance control instructions according to the depth value comparison threshold; Step 4: The control instructions are sent to the microcontroller through the HTTP protocol, and the control signal generation module selects the corresponding output port according to the received instructions to generate a pulse signal with a high-low level time of 10ms. Through the microelectrode acting on different positions in the insect body, basic motion control including left turn, right turn and forward movement is realized. Monitor the response of the insect to the stimulus. When the response is found to be weakened, prompt the user to adjust the stimulation parameters or replace the stimulation waveform. Reduce stimulation fatigue through the incremental pulse number strategy; Step 5: The industrial camera collects the overall motion picture of the insect robot in real time through the MVSDK interface. The image data is transmitted to the trajectory recording module after preprocessing, and the KCF algorithm is used for target tracking to update the target position information in real time; Step 6: Draw a yellow bounding box on the image to mark the current position through the trajectory recording module, record the motion trajectory with a red line, and mark the action position of the control instruction on the trajectory. Establish the proportional mapping relationship between the actual position of the insect and the display interface. Each time the control instruction is sent, use different markers to record the stimulation position on the trajectory to facilitate the analysis of the control effect; Step 7: Provide manual and automatic control modes through the host computer display control unit. In manual mode, the experimenter can send control instructions through the interface buttons. In automatic mode, the system automatically generates obstacle avoidance instructions based on depth information to ensure safe movement of the insect robot; Step 8: Save the information including the first perspective image of the insect robot, the depth map, the motion trajectory, the control instruction and the stimulation parameters for the experimenter to view and analyze, and provide a basis for optimizing the control strategy.

Images