A multimodal perception rat robot autonomous navigation system and method

Abstract

The present invention discloses a multimodal perception rat robot autonomous navigation system and method, comprising a biological rat, a wearable electronic control backpack, and a remote PC host computer. Four pairs of stimulation electrodes are implanted in the skull of the biological rat, with the implantation sites being the left and right sides of the medial forebrain bundle and the left and right sides of the ventroposterior medial nucleus of the rat's brain nerve center. The remote PC host computer and the wearable electronic control backpack are connected via wireless communication. The wearable electronic control backpack acquires the biological rat's motion state and environmental information and transmits it to the host computer. After receiving the information data, the host computer generates serialized control instructions and transmits them to the wearable electronic control backpack. The electrical stimulator module converts the control instructions into electrical stimulation instructions and controls the stimulation electrodes to perform corresponding electrical stimulation, thereby guiding the movement of the rat robot. The present invention can realize target search and autonomous navigation of the rat robot in complex and unknown environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of control of novel bio-robots, and in particular relates to a multi-modal perception rat robot autonomous navigation system and method. Background Art

[0002] Animal robots are a new type of robot that uses animals as control objects. Based on neurobiological principles and brain-computer interface technology, they stimulate relevant neural regions in the animal's brain in a specific manner, controlling the animal's movements in the desired manner. Furthermore, because they retain animal instincts, animal robots exhibit significant advantages in environmental adaptability, concealment, and mobility, making them valuable as specialized robots in fields such as counterterrorism and reconnaissance. Compared to traditional mechanical robots, animal robots effectively address key technical challenges such as energy supply and motion control stability, significantly reducing system costs.

[0003] Existing animal robots include rat robots, beetle robots, pigeon robots, cockroach robots, and gecko robots. The rat robot, among others, is designed by implanting bifurcated electrodes made of nickel-chromium alloy in the rat's medial forebrain bundle and barrel area of ​​the somatosensory cortex. By electrically stimulating different brain regions, the rat, after a certain level of training, can be guided to complete three standard movements: left and right turning and forward movement. The experimenter can then control the rat robot to achieve motion control in complex external environments. Patent publication number CN109394208A discloses a biorobotic system comprising an implantable neural stimulation circuit system and an EEG signal acquisition system.

[0004] Currently, the navigation of rat robots still relies heavily on manual control by the experimenter. This involves combining environmental information with the robot's motion state to determine and determine the robot's trajectory, and then providing sequenced control instructions to guide the robot through its movements. Among existing technologies for autonomous navigation of rat robots, patent publication number CN115056235A discloses a rat search and rescue robot based on a UWB human presence sensor and a positioning module.

[0005] However, the above technical solutions lack environmental perception capabilities. The rat robot, which only has a positioning device and a UWB human presence sensor, finds it difficult to achieve target search and autonomous navigation processes in areas with unknown environments. Summary of the Invention

[0006] The present invention provides a multimodal perception rat robot autonomous navigation system and method, which enables the rat robot to complete specific target search and autonomous navigation processes when facing an actual complex environment.

[0007] A multimodal perception rat robot autonomous navigation system includes a biological rat body, a wearable electronic control backpack, and a remote PC host computer. The biological rat body has four pairs of stimulation electrodes implanted in its skull. The reward stimulation implantation sites are located on the left and right sides of the medial forebrain bundle of the rat brain nerve center, and the steering stimulation implantation sites are located on the left and right sides of the ventral posteromedial nucleus. The wearable electronic control backpack is fastened to the back of the biological rat body and connected to the interfaces of the four pairs of stimulation electrodes via an electrical stimulator module.

[0008] The remote PC host computer and the wearable electronic control backpack are connected by wireless communication; the wearable electronic control backpack is used to obtain the motion state information of the biological rat body and the environmental information of the motion process, and send it to the remote PC host computer;

[0009] An autonomous navigation control model is built in the remote PC host computer. After receiving data on motion state information and environmental information, the autonomous navigation control model generates serialized control instructions and sends them to the wearable electronic control backpack. The electrical stimulator module converts the control instructions into electrical stimulation instructions and applies electrical stimulation to the corresponding electrodes, thereby guiding the biological rat body to perform actions corresponding to the control instructions.

[0010] The wearable electronic control backpack securely connects the biological rat to the wearable electronic control backpack through its wearable structure. Connecting wires are inserted into the rat's head, where a wire-to-board connector is implanted on a PCB, establishing an electrical connection between the wearable electronic control backpack and the rat's central nervous system. The working rat robot uses a wireless communication module in the control backpack to establish a wireless signal connection with a remote control platform.

[0011] Furthermore, the wearable electronic control backpack comprises an electrical stimulator module, an embedded microprocessor module, an inertial sensor, a video module, an audio module, an infrared sensor module and a Wi-Fi communication module;

[0012] Among them, the inertial sensor is used to extract the rat's movement status information; the video module, audio module, and infrared sensor module are used to extract environmental information of the rat's movement process; the Wi-Fi communication module is used to realize wireless communication between the remote PC host and the wearable electronic control backpack; the embedded microprocessor is used to obtain information from the inertial sensor, video module, audio module, and infrared sensor module, and transmit it to the remote PC host through the Wi-Fi communication module. At the same time, it receives control instructions from the remote PC host, converts the control instructions into pre-set electrical stimulation instructions through the electrical stimulation module, and transmits them to the corresponding stimulation electrodes via wires, thereby building a two-way connection channel from the remote PC host to the rat's central nervous system.

[0013] Furthermore, the autonomous navigation control model uses environmental information as control input and motion state information as feedback detection to complete closed-loop control; after processing the characteristic data of environmental information and motion state information, the autonomous navigation control model generates serialized control instructions.

[0014] Furthermore, the serialized control instructions include five types, namely task start instructions, forward movement instructions, steering control instructions, search movement instructions and task end instructions; the above five movement instructions are all implemented through the multi-channel electrical stimulation module carried on the wearable electronic control backpack, with self-set different sites of the rat brain nerve center and different stimulation instruction intervals to achieve a combined stimulation mode, and ultimately guide and control the rat robot to complete the target search task in the field.

[0015] Since the intensity of the electrical stimulation instructions of the rat brain nerve center is mainly related to the current stimulation intensity of the rat brain nerve center in a single time and the time interval between two electrical stimulation instructions, that is, the frequency of the electrical stimulation instructions. During a single stimulation, excessive stimulation current may have the risk of damaging the rat brain nerves. Therefore, the current intensity of a single stimulation instruction is set by the experimenter and remains unchanged during subsequent experiments. Therefore, the above-mentioned combined stimulation mode is based on three combinations of adjusting the combination of stimulation instructions, namely, the combination between reward stimuli, the combination between steering stimuli, and the combination between reward stimulation and steering stimulation. By changing the time sequence of different stimulation instructions, a rat robot serialized control instruction set is established, which can control the rat robot to complete different motion tasks.

[0016] The task start instruction is realized by combining multiple reward instructions. The corresponding electrical stimulation is: applying multiple consecutive T min The reward stimulus at the interval between the stimulation instructions caused the rats to be quickly excited and out of the resting state;

[0017] The forward motion instruction is based on the data from the inertial sensor and guides the rat to achieve linear motion through a combination of reward instructions. The corresponding electrical stimulation is specifically: continuous application of reward stimulation, monitoring the target angle of the rat's movement with the inertial sensor on the rat's head, and applying reward stimulation under the set deflection threshold condition to ensure that the rat moves forward within the range;

[0018] The steering motion instruction is based on data from the inertial sensor and consists of a reward instruction and a steering instruction. The corresponding electrical stimulation is specifically as follows: the inertial sensor calculates the heading angle using a Kalman filter, and uses the deviation between the actual installation orientation and the rat's head to correct the actual movement direction of the rat robot's head. Based on the rat robot's head movement intention, a single or multiple steering stimulation combinations are applied. If the rat's head shows an intention to deflect in the set direction, a reward stimulation is given.

[0019] The search movement instructions consist only of reward instructions. Relying on the sniffing characteristics of the rat body in a low-speed movement state, the rat robot realizes autonomous target search in the field. The support vector machine is used to classify the rat's movement speed into fast, medium and slow speeds. The time interval of the reward instructions is adjusted according to different speed gears. The corresponding electrical stimulation is specifically: when the rat is in a low-speed movement state, multiple time intervals of T are given. min When the rats were in a high-speed movement state, the frequency of reward stimulation was reduced and only T max The longest interval of reward stimulation, that is, maintaining the lowest motor activity of the rat robot;

[0020] The task end instruction does not contain any stimulation instruction and only represents the end of the exploration process.

[0021] The reward stimulation refers to stimulating the left and right sides of the medial forebrain bundle of the rat's brain nerve center simultaneously, reducing the movement bias caused by the imbalance of charge in the brain area; the reward stimulation is used as a reward for the rat robot to complete the movement preset by the experimenter, and the functional electrical stimulation of the rat's brain nerves is solidified into the corresponding movement behavior of the biological body;

[0022] During the training process, reward stimulation is used to induce the rat's biological body to move forward. After the rat makes feedback of forward movement, reward stimulation is continued to be applied, forming a positive feedback training model of reward stimulation-rat forward movement.

[0023] The steering stimulation refers to stimulating the left and right sides of the ventroposterior medial nucleus of the rat brain nerve center;

[0024] During the training process, turning stimulation is used to induce the rat's biological body to make a turning movement. After the rat shows the correct intention to turn in the direction, a reward stimulus is applied, forming a positive feedback training model of preset action-turning stimulation-correct turning-reward stimulation; after the rat shows the wrong turning direction, the reward stimulus in that direction is immediately stopped.

[0025] As a preferred embodiment of the present invention, in the process of constructing the rat robot, adult male SD rats are selected as the biological body, and the implantation of homemade double-strand electrodes is achieved through craniotomy. A total of 8 electrodes and 4 pairs of electrodes are implanted in the rat head, which are respectively located on the left and right sides of the ventral posteromedial nucleus of the rat's biological brain central nervous system and the left and right sides of the medial forebrain bundle of the rat's biological brain central nervous system. In order to fix the rat head interface, a wire-to-board connector is used, which is placed on a small PCB, and the implanted head electrode interface and the wire-to-board connector are connected by spot welding technology. After the operation, the encapsulation of the implanted PCB is achieved with dental cement, and the rat scalp incision is fixed to the dental cement. At the same time, a fixture design is adopted to install the inertial sensor on the fixture, and the fixation of the inertial sensor is completed by relying on dental cement.

[0026] As a preferred solution of the present invention, in the wearable electronic control backpack, the power module is powered by a flat lithium battery, and an LDO step-down chip is used to generate on-board digital power and analog power, and a DCDC boost chip is used to power the stimulation power system; in the electrical stimulation module, the output stimulation voltage is regulated by a DAC chip, and a bidirectional stimulation amplification output and a multi-channel channel selector built with an operational amplifier are used to realize the construction of an on-board multi-channel electrical stimulation module, and the electrical stimulation output interface is connected to the electrode implantation interface of the rat head through a homemade wire; the inertial sensor, video module, and infrared sensor module use dedicated modules, and the audio module uses a dual-channel microphone, supplemented by an audio codec chip to realize sound signal acquisition.

[0027] The entire system is designed to wirelessly connect a PC's Wi-Fi communication module to a wearable electronic control backpack's Wi-Fi communication module. An embedded microprocessor module acquires information from multiple sensors and transmits the data to a remote PC host computer via Wi-Fi communication technology. Wireless communication control commands issued by the host computer are then transmitted via Wi-Fi to the wearable electronic control backpack's embedded microprocessor module. The electrical stimulation module converts these control commands into pre-set electrical stimulation commands, which are then transmitted via wires to the neural stimulation pathways in the rat robot's brain, establishing a bidirectional connection between the PC and the rat's central nervous system. Wi-Fi communication technology enables long-distance autonomous navigation of the rat robot, increasing its motion control range from 10 meters using Bluetooth communication to 100 meters. Furthermore, the Wi-Fi communication relay solution enables truly long-distance autonomous navigation and control of the rat robot.

[0028] As a preferred embodiment of the present invention, since rats' backs typically exhibit specific undulations during locomotion, such as the back's jerking motion during normal movement and the back's significant bending or 90-degree stance during abnormal activity, this solution addresses the instability of the inertial sensor, infrared sensor, and camera during locomotion. This solution utilizes a custom-designed mechanical fixture to stably secure the inertial sensor to the rat's head and the infrared sensor and camera to the upper PCB of the wearable electronic control backpack.

[0029] The inertial sensor is clamped to a wire connected to a wire-to-board connector on the PCB on the rat robot's head. A two-layer fixture, designed to meet the dimensions of the small PCB carrying the inertial sensor, is designed. Four screw holes are located around the back of the fixture to connect the small PCB carrying the inertial sensor. This small PCB is also connected to the wearable electronic control backpack using wires.

[0030] The main body of the wearable electronic control backpack is composed of two layers of PCBs. Fixing holes are punched on the four corners between the two layers of PCBs, and nylon studs and nylon nuts are used to achieve reliable connection and fixation of the two layers of PCBs.

[0031] The above-mentioned sensor fixing bracket can be divided into a camera fixing bracket and an infrared auxiliary fixing bracket.

[0032] The camera mounting bracket is based on a longitudinal mounting bracket mounted in the mounting holes. This bracket features a groove structure that accommodates the image sensor. This is combined with a matching back cover structure, and small studs and nuts provide a rigid connection between the back cover and the groove, ensuring a stable press fit for the camera.

[0033] The auxiliary fixation of the infrared sensor is based on the longitudinal fixing frame installed on the fixing hole. The infrared sensor itself is inserted into the vertical pin header welded on the corresponding hole on the upper PCB to achieve electrical connection and partial fixation between the infrared sensor and the upper PCB.

[0034] As a preferred solution, the present invention uses a nine-axis inertial sensor to capture the biological motion characteristics of the rat robot during its mission. This sensor, combined with a machine learning classification algorithm, enables classification of the rat's motion and detection of dead zones. Furthermore, audio, video, and infrared data are used to monitor the robot's surroundings in real time. This, combined with corresponding feature extraction algorithms, enables multimodal environmental information extraction.

[0035] During the autonomous target search and navigation process of the rat robot, audio information, unlike video and infrared information, is not limited by the sensor's field of view. It can monitor sound signals in all directions and has greater application value for detecting target objects in unknown environments.

[0036] The nine-axis inertial sensor mounted near the rat robot's head will employ a Kalman filter algorithm to calculate its attitude angles. Based on the sensor's mounting location and the actual position of the rat's head, the raw data is transformed through rotation and translation to determine the actual changes in the rat's head.

[0037] At the same time, the present invention provides a movement speed classification model based on the actual movement speed model of rats, with a three-axis accelerometer as the data source, and uses the head three-axis accelerometer data as input to realize the low, medium and high classification of rat movement speed by support vector machine (SVM).

[0038] In addition, the present invention provides a motion area dead zone recognition model based on SVM to address the dead zone situations that rats may encounter when actually performing detection tasks. It uses the nine-axis motion data and calculated posture angles in a 400ms period as the data source, extracts time domain features such as mean, standard deviation, energy, absolute change, kurtosis, maximum value and minimum value, and uses the recursive elimination method to reduce the number of selected features, which has the effect of optimizing the detection success rate of the rat robot's autonomous navigation process.

[0039] The dual-channel stereo microphones mounted on the upper PCB are used to collect sound information from the Rat Robot's environment. The microphones convert the acoustic signals into electrical signals, which are then converted from analog to digital using an audio codec. The relative position of the Rat Robot and the target object can be determined based on the strength of the left and right audio signals. If the target possesses specific audio characteristics, the host computer can perform targeted audio signal feature extraction to determine if the target is real or fake.

[0040] The above-mentioned video signal is obtained through a camera, and an embedded microprocessor is used to capture real-time image data. The optical signal can be converted into an electrical signal through a CMOS sensor. After video stream encoding, the single-frame image information can be converted into a video stream signal, which is pushed to the PC host computer through a wireless transmission module. The host computer decodes the video stream and restores the single-frame image information.

[0041] Images typically contain diverse environmental information, and corresponding feature extraction methods can be set for the target object, such as a facial recognition module or object recognition module. Due to the bumpy state of the rat's back during movement, the image sensor can often only recognize partial, incomplete, and discontinuous target information when acquiring a specific target. Therefore, partial feature recognition is used for the image information, such as completing the recognition of prominent facial features such as the mouth and nose of a person, and then combining the recognition information of each individual area to achieve face recognition;

[0042] The above-mentioned infrared sensor can realize the collection of infrared signals in the environment. Selecting infrared sensors arranged in an array form can realize the mapping of infrared signals from the external environment to the sensor sensing array units. Infrared sensors can be used not only to reflect infrared information of the environment, but also to detect human targets. After knowing the emissivity of the target object to be measured in advance, the actual temperature of the target object can also be calculated. Similarly, in view of the instability of the infrared image signal, the temperature difference data between the calculated target object and the set target temperature and the appearance characteristics of the characteristic target object are combined for identification. Ultimately, the success rate of specific target recognition of the environmental sensor on the back of the rat robot is improved.

[0043] The onboard embedded microprocessor acquires the signal stream from the dual-channel stereo microphone, camera, and infrared sensor data, and uses the onboard wireless communication module to reliably connect to the remote control center, completing the real-time return of multimodal sensor data information.

[0044] The present invention also provides a multimodal perception rat robot autonomous navigation method, based on the multimodal perception rat robot autonomous navigation system, comprising the following steps:

[0045] Step S0: This step is the functional self-checking stage;

[0046] First, test the communication function between the wearable electronic control backpack and the remote PC host computer; second, test the signal acquisition and processing of the video module, audio module, and infrared sensor module; then, fasten the wearable electronic control backpack to the back of the rat robot with Velcro, complete the connection between the electrical stimulator module and the electrode interface implanted in the rat's head, and test the rat robot's response to reward stimulation and left and right steering stimulation; after completing the test, jump to step S1;

[0047] Step S1: This step is the beginning stage of exploration;

[0048] The remote PC sends a task start command to activate the rat's biological body excitement and increase its willingness to exercise. The PC will continue to send the task start command until the inertial sensor recognizes that the rat robot has entered the active state, and then jump to step S2.

[0049] Step S2: This step is the target site exploration stage;

[0050] First, the remote PC sends a search motion command of duration t. If no target object is found within this time, the remote PC sends a small-angle turn motion command and continues with the next search motion command of duration t. After the turn motion command, the inertial sensor detects the direction of the rat's movement. If the rat shows a tendency to stick to the wall, the turn motion command is sent again to correct the rat robot's movement direction.

[0051] Repeat the above steps of exploring the target site to ensure that the rat robot traverses the path in the exploration environment in a spiral manner until the video module or the infrared sensor module finds the target object, and then jump to step S4;

[0052] During the exploration process, if the audio sensor captures the sound characteristics of the target object, the relative position of the rat robot and the target object is identified using a two-channel audio signal. The remote PC sends a steering command to guide the rat robot toward the target until the audio sensor can no longer clearly distinguish between the left and right channels. The control process of the search motion command described above is then continued to guide the rat robot to complete the exploration task of the target site.

[0053] During the exploration process, if the dead zone discrimination model detects that the rat robot falls into the motion dead zone, the process jumps to step S3;

[0054] Step S3: This step is the phase of leaving the dead zone;

[0055] When the rat is detected to have fallen into the dead zone, the remote PC host computer stops sending the current stimulation command; the host computer sends a turning command with the opposite direction of the rat robot's head before it fell into the dead zone as the target direction; after turning to a specific direction, the rat robot executes the exploration movement command for a duration of t; if it is determined that the rat robot has not left the dead zone, the host computer continues to send small-angle turning commands and then executes the search movement command for a duration of t;

[0056] Repeat the above steps of escaping the dead zone until the rat robot escapes the dead zone, and then jump to step S2;

[0057] Step S4: This step is the directional guidance stage;

[0058] After the video module or infrared sensor module finds the target object to be searched, the remote PC host computer sends a forward movement instruction to guide the rat robot to move in a straight line towards the target object;

[0059] Repeat the above-mentioned directional guidance steps until the video module or the infrared sensor module determines that the rat robot is close enough to the target object, and then jump to step S5 of the process;

[0060] Step S5: This step is the end stage;

[0061] In this step, the rat has completed the target search task and all stimulation instructions will be stopped.

[0062] Furthermore, in step S4, if the image sensor or infrared sensor does not detect the target object for more than 30 seconds, it is considered that the rat has completely deviated from the target direction, and the corresponding forward movement instruction is stopped; at this time, jump to step S2 and re-traverse the site search.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] 1. This invention takes brain-computer interface technology as its core, biological rats as its research object, and a wearable electronic control backpack as its control core. It relies on stimulation electrodes implanted in the brain's nerve centers to build a connection channel from the host computer's machine brain to the biological rat brain.

[0065] 2. The present invention achieves a better compliant motion control effect for biological rats by adjusting the stimulation sites of the rat brain nerve center. With the auxiliary detection of inertial sensors, a set of serialized stimulation schemes for autonomous navigation is constructed and implemented, which has a closed-loop feedback control effect. The sensory enhancement technology of the rat robot is realized through multimodal environmental sensor technology, including but not limited to video modules, audio modules, and infrared sensor modules. With the assistance of brain-computer interface technology, a set of stimulation mapping models from the external environment that is imperceptible to biological rats to multiple sites of the rat brain nerve center are successfully constructed.

[0066] 3. Building on the two aforementioned technologies, this invention incorporates the actual movement patterns of biological rats in their environment and leverages the rodent's characteristic wall-clinging locomotion to develop a specific target search and autonomous navigation scheme for rat robots in unknown environments. This proposed control scheme leverages the biological characteristics of rats, such as their strong environmental adaptability and stealthy movements, enabling rat robots to autonomously navigate and search for specific targets in environments inaccessible to traditional mechanical robots, such as dark underground tunnels and narrow ventilation ducts. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a schematic diagram of the circuit structure of a multimodal perception rat robot autonomous navigation system of the present invention;

[0068] Figure 2 This is a schematic diagram of the PCB structure of the rat's head inertial sensor and head implant electrodes;

[0069] Figure 3 This is a schematic diagram of the structure of a wearable electronic control backpack;

[0070] Figure 4 This is a schematic diagram of wireless communication between the rat robot and the remote PC host computer;

[0071] Figure 5 The data interaction flow chart of the wearable electronic control backpack and the remote control platform PC. DETAILED DESCRIPTION

[0072] The present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be noted that the following examples are intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0073] like Figure 1-4 As shown, a multimodal perception rat robot autonomous navigation system mainly includes a biological rat body 26, a wearable electronic control backpack 27 and a remote PC host computer 29. The remote PC host computer is mainly a host computer independently written in the Qt framework.

[0074] In the embodiment of the present invention, the moving body of the rat robot is a biological rat body 26 , which is formed into a controlled animal robot after stimulating electrodes are implanted at the marked points in the rat brain.

[0075] Specifically, the stimulation electrodes are two nickel-chromium alloy electrode wires wound in a twisted pair, and about 0.5 mm of insulation layer of the electrode wires is stripped off to ensure sufficient electrical stimulation range after implantation.

[0076] The specific implantation sites in the rat brain are: the reward stimulation implantation sites of the rat are the left and right sides of the medial forebrain bundle of the cranial nerve center, and the steering stimulation sites of the rat are the left and right sides of the ventral posteromedial nucleus of the cranial nerve center.

[0077] Specifically, the training process of the rat robot adopts a positive feedback training model, using reinforcement learning ideas to establish a mapping relationship between stimulus instructions and specific actions.

[0078] Electrical stimulation of the medial forebrain bundle of the aforementioned rat brain nerve center can increase the rat's biological body's excitability and pleasure, and enhance the rat's biological body's willingness to move. Therefore, the following content refers to stimulation of the medial forebrain bundle of the rat brain nerve center as reward stimulation. During training, reward stimulation is used to induce the rat's biological body to move forward. After the rat provides feedback on the forward movement, reward stimulation is continued, forming a positive feedback training model of reward stimulation and forward movement of the rat.

[0079] Electrical stimulation of the ventroposterior medial nucleus of the above-mentioned rat brain nerve center can induce virtual tactile sensation in the rat's biological body. Therefore, the following content refers to the stimulation of the ventroposterior medial nucleus of the rat's brain nerve center as steering stimulation. Compared with the whisker area of ​​the primary somatosensory cortex, the stimulation of this brain area can more accurately control the turning movement of the rat. During the training process, steering stimulation is used to induce the rat's biological body to turn. After the rat shows the correct intention to turn in the direction, a reward stimulus is applied to form a positive feedback training model of preset action-steering stimulation-correct turning-reward stimulation. The above training process will be completed in an eight-arm maze.

[0080] In an embodiment of the present invention, in order to realize the motion control and external environmental signal perception of the rat robot, a wearable electronic control backpack 27 is designed, which is mainly composed of a stimulation wire 1, an electrical stimulation output interface 2, a power socket 3, an infrared sensor module 4, a video module 5, a front cover structure 6, a back cover structure 7, a stud nut 8, a video module cable 9, a power module 10, a Wi-Fi antenna 11, a Wi-Fi communication module 12, an audio sensor 13, an embedded microprocessor module 14, a dual-channel microphone 15, a nylon screw nut 16, an electrical stimulator module 17, an upper clamp 18, a wire-to-board connector 19, an electrode PCB board 20, a fixing screw 21, a lower clamp 22, a vertical clamp 23, a nylon screw nut 24, an inertial sensor 25 and a wearable structure 28.

[0081] At the same time, in the present invention, in order to realize the orderly processing of sensor signals, the inertial sensor 25 designed for extracting the motion signal of the rat robot and the audio module (composed of an audio sensor 13 and a dual-channel microphone 15) for extracting the environmental signal during the movement of the rat robot, the video module 5 and the infrared sensor module 4 all propose a set of signal processing processes to complete the extraction of the motion state of the rat robot and the characteristic signals of the target to be searched.

[0082] In the present invention, the power module primarily consists of a power socket 3 connected to a flat 3.7V lithium battery and a power module 10. The system power tree is divided into input 3.7V, analog 3.3V, digital 3.3V, and ±15V. The analog 3.3V and ±15V are primarily supplied to the electrical stimulation module 17, while the digital power supply primarily supplies the multimodal sensor module. Both the analog 3.3V and digital 3.3V are implemented using an LDO step-down chip for the input 3.7V power supply, while the ±15V stimulation power supply is implemented using a DC-DC boost circuit via the analog 3.3V.

[0083] The electrical stimulator module 17 primarily utilizes an onboard DAC chip for programmable stimulation voltage regulation, and a multi-channel op-amp structure for generating stimulation pulse waveforms from the DAC output. The op-amp structure implements the subtraction circuit and the forward and reverse amplification circuits. The bidirectional stimulation output of the op-amp and the multi-channel selector form the onboard multi-channel electrical stimulation module. A custom-made wire 1 connects the electrical stimulation output interface 2 to the rat head electrode PCB 20.

[0084] The embedded microprocessor module 14 is provided by the HiSilicon chip Hi3516, which uses GPIO, SPI, IIC, I2S, MIPI, and USB bus to realize the electrical connection of the electric stimulation module, audio and video sensor, infrared sensor, and Wi-Fi module.

[0085] The inertial sensor 25 uses a BMX160, and the embedded microprocessor module 14 uses the IIC bus to configure the inertial sensor 25 and read its raw data. The inertial sensor on the rat robot's head collects kinematic data from the rat's biological body. Using the Bosch-provided ground attitude angle calculation library, the actual deflection of the rat's head is determined after Kalman filtering. The deflection direction between the fixed position of the inertial sensor and the position of the rat's head is multiplied by the rotation matrix to obtain the actual deflection of the rat's head. Furthermore, a support vector machine (SVM) approach is used to generate a rat motion speed discrimination model and a dead zone detection model, divided into three speed levels: low, medium, and high.

[0086] The audio module is implemented using the WM8978, which includes a dual-channel microphone 15 and an audio sensor 13. An embedded microprocessor controls the WM8978 via the I2S and IIC buses, sampling the ambient sound at 8kHz / 44.1kHz in dual-channel high-definition resolution and capturing the recorded data in PCM encoding mode. The uncompressed raw audio data is processed by the Hi3516, extracting the left and right channel signal envelopes to determine the relative position between the rat robot and the target object. If the search target is a human, the human voice signal is extracted and transmitted back to the computer in real time. The remote control center on the PC then decodes the voice content of the searched object using the MFCC audio features.

[0087] Video module 5 uses the GC2053 module, and the embedded microprocessor uses the MIPI bus to acquire image digital signals. Among them, the environmental image information collected by the GC2053 image sensor can obtain video stream data of up to 720p and 30fps. The entire image data is converted into an H.264-encoded video stream signal through the Hi3516 internal video input VI, video processing VPSS, and video encoding VENC. The upper computer uses the FFmpeg library to complete the decoding of the video stream signal according to the H.264-encoded video IPS key frame. For the restored image information, face detection and object detection modules can be installed. In the face detection module, open source OpenVINO is used for implementation, and object detection completes feature extraction by inputting the corner features of the object and the color information of the picture. Both of the above solutions can find the target object to be searched in the image.

[0088] Infrared sensor module 4 uses the MLX90640 module, and the embedded microprocessor acquires sensor data features via the SPI bus. The MLX90640 maps the external environment to sensor sensing array elements in a 3x24 matrix array. To expand the matrix signal, polynomial interpolation is used to subdivide the signal elements. Furthermore, the lower the temperature of the infrared object and the farther it is from the sensor, the weaker the captured target infrared signal. Determining the infrared signal strength of the target object can be used to infer the distance between the rat robot and the target object. Furthermore, the temperature of the target object can be calculated based on the emissivity of the target object and environmental information, which can also serve as an auxiliary judgment method.

[0089] Wi-Fi communication module 12 is implemented using the RTL8189 module. An embedded microprocessor controls the Wi-Fi module via the SDIO bus, and antenna 11 uses the SMA-1 interface for signal transmission. To establish a reliable connection between the wearable electronic control backpack 27 and the remote PC host computer 29, TCP is used to achieve reliable communication between the two devices.

[0090] In the present invention, in order to achieve the stable fixation of the inertial sensor on the rat's head, and the stable fixation of the infrared sensor and camera on the upper PCB of the wearable electronic control backpack, the scheme adopts a self-designed mechanical fixture, such as Figure 2 and Figure 3 shown.

[0091] Specifically, the inertial sensor 25 is fixed to the stimulation wire 1 on the wire-to-board connector 19 welded on the electrode PCB board 20 on the head of the rat robot through the upper clamp 18 and the lower clamp 22 in a clamping form. The upper clamp 18 and the lower clamp 22 are designed to meet the size requirements of the small PCB carrying the inertial sensor, and four fixing screws 21 are provided around the back plate of the clamp. After the clamps are clamped, a total of four fixing screws are used around to fix the clamp and the wire. The vertical clamp 23 is at a 90-degree angle to the upper clamp 18, and the inertial sensor is embedded in the vertical clamp 23. The inertial sensor and the vertical clamp are fixed with nylon screw nuts 24, and finally a stable connection is achieved between the inertial sensor 25 and the electrode PCB board 20 on the rat head. At the same time, the small PCB also uses the stimulation wire 1 to achieve electrical connection with the electrical stimulation output interface 2 on the wearable electronic control backpack.

[0092] Specifically, the wearable electronic control backpack consists of two PCBs, one on top and one on the bottom. Fixing holes are punched in the corners between the two PCBs, and 8mm nylon studs and nylon nuts 16 are used to securely connect the two PCBs. The fixing holes in the upper PCB ensure a rigid connection between the sensor mounting bracket and the upper PCB.

[0093] Specifically, the sensor mounting bracket can be divided into a camera-mounted front cover structure 6 and a rear cover structure 7. The camera mounting bracket is based on a longitudinal mounting bracket mounted in a fixed hole. This longitudinal mounting bracket has a groove structure that adapts to the structure of the video module 5. Combined with the matching front cover structure 6 and rear cover structure 7, small studs and nuts 8 provide a rigid connection between the rear cover and the groove, ensuring a stable press fit for the camera. The infrared sensor is securely fixed to the camera by the camera-mounted front cover structure 6, based on the mounting structure of the vertical pin headers connecting it to the upper PCB.

[0094] In the embodiment of the present invention, in order to realize the motion control of the rat robot, a rat robot serialized control instruction set consisting of three stimulation modes and five combined control instructions is proposed. By combining stimulation, stimulation type, and interval time, a diversified rat motion control instruction sequence is constructed. In order to avoid permanent damage to the rat brain nerve center due to nerve electrical stimulation, the shortest time interval between different stimulations is recorded as T min The maximum excitement time of a single reward instruction is 400ms. Since the responses of different rats to reward stimuli vary greatly, the longest excitement time of a single reward instruction is recorded as T max .

[0095] Specifically, the task start instruction consists only of the reward instruction. According to the response of different rats to the reward instruction, the three stimulation time intervals are set as T min The reward instructions are a set of instructions that are intended to quickly drive the rats from a static state to a moving state.

[0096] Specifically, forward motion instructions consist solely of reward commands. The remote PC host computer sets the rat robot's head heading angle θ during forward motion, and a left / right deflection threshold θ is set based on the robot's motion. The inertial sensor on the rat's head detects the current deflection of the robot's head, and reward stimuli are applied only within the range of θ ± θ. If a significant deflection of the robot's head is detected, a small reverse steering command is applied. Once the rat's head's intention to turn is detected, a reward command is given, correcting the robot's forward motion direction.

[0097] Specifically, the turning movement instruction consists of a reward instruction and a left or right turning instruction. The specific turning direction of the rat is set via the remote PC host computer, and the corresponding turning stimulus is used according to the set direction. The inertial sensor detects the deflection of the rat's head movement. If the rat does not show the intention to turn in the corresponding direction, then the rat is given a turn at T. min Then continue to give turning stimulation until the rat shows the desire to turn in the corresponding direction, and give multiple reward stimulations to guide the rat to complete the turning action.

[0098] Specifically, the exploratory movement instruction consists only of reward instructions. According to the feedback results of the rat movement speed discrimination model, when the rat is in a low-speed movement state, multiple time intervals of T are given. min When the rats were in a high-speed movement state, the frequency of reward stimulation was reduced, and only T max The reward stimulation with the longest interval was used when a downward trend in the rat's movement speed was detected, which maintained the rat robot with the lowest movement activity.

[0099] Specifically, the task stop instruction has no stimulation instruction and is only a motion control instruction described in the rat robot serialized control instruction set to stop the rat robot from further exploring the target site. It has no task stimulation and is intended to stop the rat robot as soon as possible.

[0100] In an embodiment of the present invention, a universal search framework based on the motion characteristics of biological entities is proposed to enable a rat robot to search for a target and conduct autonomous navigation in a complex environment with unknown conditions. This framework primarily leverages the characteristics of rodents' habit of clinging to walls to design a solution for autonomous navigation and target search for rat robots in semi-enclosed scenarios. The designed exploration process details the sequenced stimulation instruction scheme for the rat robot in the target environment, from the full-field traversal exploration process when the target is not found to the directional navigation process after the target is found. Specific solutions are also provided for major abnormal conditions that may occur during the entire exploration process, such as when the rat's movement falls into a dead zone.

[0101] Specifically, process step S0 is the functional self-verification stage. This step is the functional self-verification stage. The wearable electronic control backpack is placed on the back of the rat with Velcro, and the multi-channel electrical stimulator is connected to the stimulation electrodes implanted in the rat's head with wires. During the power-on verification, the two-way reliable wireless signal transmission channel established between the onboard wireless communication module on the wearable electronic control backpack and the remote control platform PC is first tested. After confirming that the signal transmission channel is normal, the data acquisition, processing and return functions of the video, audio and infrared sensors on the backpack are tested and verified respectively. Finally, the effectiveness of the electrical stimulation is tested on the rat body to verify the response to the three stimulation commands of reward, left and right turn. After the test is completed, the rat robot is placed in the exploration field by the experimenter to start the target search experiment.

[0102] Step S1 of the process begins the exploration phase. The experimenter places the rat robot in a static position in the exploration area. A remote PC sends a task start command, using reward instructions to activate the rat's brain nerve center, achieving rapid excitation of the rat's biological body. The PC continues sending task start commands until the inertial sensor detects that the rat robot has entered an active state, at which point the process jumps to step S2.

[0103] Process step S2, this step is the target site exploration stage. In this stage, the rat robot will complete the traversal search in the entire site as much as possible in a manner that conforms to the wall-sticking movement habits of rodents under the control process of the serialized stimulation instructions. According to the size of the site to be detected, the time t in the exploration process is adjusted. In the specific process, the host computer first sends a search motion instruction with a duration of t. If no target object is found within this time, the host computer sends a small-angle turning motion instruction, and continues with the next search motion instruction with a duration of t. At this time, since the rat robot usually shows a movement trend away from the inner wall of the environment after the turning motion instruction, the inertial sensor detects the movement direction of the rat. Once the rat shows a trend of sticking to the wall, the turning motion instruction is sent again to correct the movement direction of the rat robot.

[0104] Repeat the above steps to ensure that the rat robot traverses the path in the exploration environment in a spiral manner until the image sensor or infrared sensor finds the target object, and then jumps to process step S4.

[0105] During the exploration process, if the audio sensor detects the target object's sound signature, the robot uses a dual-channel audio signal to identify its relative position to the target object. The host computer then sends a steering command, guiding the robot toward the target until the audio sensor can no longer clearly distinguish the left and right audio channels. The search motion control process continues, guiding the robot to complete its exploration of the target area.

[0106] During the exploration process, if the dead zone discrimination model detects that the rat robot falls into a motion dead zone, the process jumps to step S3.

[0107] Process step S3 is the dead zone exit stage. Because the actual relative position of the rat robot and the obstacle cannot be determined, the rat robot's main motion capability is leveraged, using the reverse direction as a directional indicator. Specifically, the host computer records the rat robot's head movement direction before entering the dead zone as θ1, uses θ1's reverse direction, θ2, as the target direction, and executes the rat robot's exploration motion command for a duration of t. If the rat robot is determined to have not escaped the dead zone, the host computer continues to send a small-angle steering command, followed by a search motion command for a duration of t. These steps are repeated until the rat robot escapes the dead zone, at which point the process jumps to process step S2.

[0108] Step S4 is the directional guidance phase. After the target object is detected by the image sensor or infrared sensor's target recognition model, the host computer sends a forward motion command, guiding the rat robot to move linearly toward the target object. Since it's difficult for the rat to maintain a straight line during forward motion, the forward motion command stimulus is only given when the image sensor or infrared sensor detects the target object, guiding the rat to complete the forward motion.

[0109] Furthermore, if the image sensor or infrared sensor fails to detect the target object for more than 30 seconds, forward movement commands cease. This indicates that the rat robot may have strayed from the correct path due to a lack of reward stimulation. The process then skips to step S2 and resumes the site search.

[0110] Step S5 is the final stage. When the image sensor and infrared sensor detect that the rat robot is close enough to the target object, the operator manually completes the final judgment and sends a task end instruction to the rat robot through the remote control platform PC. This concludes the rat robot's automated target search process.

[0111] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimodal perception rat robot autonomous navigation system, characterized in that: The system comprises a biological rat, a wearable electronic control backpack, and a remote PC host computer. Four pairs of stimulation electrodes are implanted on the skull of the biological rat. The reward stimulation is implanted on the left and right sides of the medial forebrain bundle of the rat's brain nerve center, and the steering stimulation is implanted on the left and right sides of the ventral posteromedial nucleus. The wearable electronic control backpack is fastened to the back of the biological rat and connected to the interfaces of the four pairs of stimulation electrodes via an electrical stimulator module. The remote PC host computer and the wearable electronic control backpack are connected by wireless communication; the wearable electronic control backpack is used to obtain the movement state information of the biological rat body and the environmental information of the movement process and send it to the remote PC host computer; An autonomous navigation control model is built in the remote PC host computer. After receiving data on motion state information and environmental information, the autonomous navigation control model generates serialized control instructions and sends them to the wearable electronic control backpack. The electrical stimulator module converts the control instructions into electrical stimulation instructions and applies electrical stimulation to the corresponding electrodes, thereby guiding the biological rat body to perform actions corresponding to the control instructions. The serialized control instructions include five types, namely task start instruction, forward movement instruction, turn movement instruction, search movement instruction and task end instruction; The search movement instructions consist only of reward instructions. Relying on the sniffing characteristics of the rat body in a low-speed movement state, the rat robot realizes autonomous target search in the field. The support vector machine is used to classify the rat's movement speed into fast, medium and slow speeds. The time interval of the reward instructions is adjusted according to different speed gears. The corresponding electrical stimulation is specifically: when the rat is in a low-speed movement state, multiple time intervals of T are given. min When the rats were in a high-speed movement state, the frequency of reward stimulation was reduced and only T max The longest interval of reward stimulation maintains the lowest motor activity of the rat robot.

2. The multimodal perception rat robot autonomous navigation system according to claim 1, characterized in that: The wearable electronic control backpack includes an electrical stimulator module, an embedded microprocessor module, an inertial sensor, a video module, an audio module, an infrared sensor module, and a Wi-Fi communication module; Among them, the inertial sensor is used to extract the rat's movement status information; the video module, audio module, and infrared sensor module are used to extract environmental information of the rat's movement process; the Wi-Fi communication module is used to realize wireless communication between the remote PC host and the wearable electronic control backpack; the embedded microprocessor is used to obtain information from the inertial sensor, video module, audio module, and infrared sensor module, and transmit it to the remote PC host through the Wi-Fi communication module. At the same time, it receives control instructions from the remote PC host, converts the control instructions into pre-set electrical stimulation instructions through the electrical stimulation module, and transmits them to the corresponding stimulation electrodes via wires, thereby building a two-way connection channel from the remote PC host to the rat's central nervous system.

3. The multimodal perception rat robot autonomous navigation system according to claim 1, characterized in that: The autonomous navigation control model uses environmental information as control input and motion state information as feedback detection to complete closed-loop control; after processing the characteristic data of environmental information and motion state information, the autonomous navigation control model generates serialized control instructions.

4. The multimodal perception rat robot autonomous navigation system according to claim 1, characterized in that: The task start instruction is realized by combining multiple reward instructions. The corresponding electrical stimulation is: applying multiple consecutive T min The reward stimulus at the interval between the stimulation instructions caused the rats to be quickly excited and out of the resting state; The forward motion instruction is based on the data from the inertial sensor and guides the rat to achieve linear motion through a combination of reward instructions. The corresponding electrical stimulation is specifically: continuous application of reward stimulation, monitoring the target angle of the rat's movement with the inertial sensor on the rat's head, and applying reward stimulation under the set deflection threshold condition to ensure that the rat moves forward within the range; The steering motion instruction is based on data from the inertial sensor and consists of a reward instruction and a steering instruction. The corresponding electrical stimulation is as follows: the inertial sensor calculates the heading angle using a Kalman filter, and corrects the actual movement direction of the rat robot's head based on the deviation between the actual installation orientation and the rat's head. Based on the rat robot's head movement intention, a single or multiple steering stimulation combinations are applied. If the rat's head shows an intention to deflect in the set direction, a reward stimulation is given. The task end instruction does not contain any stimulation instruction and only represents the end of the exploration process.

5. The multimodal perception rat robot autonomous navigation system according to claim 4, characterized in that: The reward stimulation refers to stimulating the left and right sides of the medial forebrain bundle of the rat's brain nerve center simultaneously, reducing the movement bias caused by the imbalance of charge in the brain area; the reward stimulation is used as a reward for the rat robot to complete the movement preset by the experimenter, and the functional electrical stimulation of the rat's brain nerves is solidified into the corresponding movement behavior of the biological body; During the training process, reward stimulation is used to induce the rat's biological body to move forward. After the rat makes feedback of forward movement, reward stimulation is continued to be applied, forming a positive feedback training model of reward stimulation-rat forward movement.

6. The multimodal perception rat robot autonomous navigation system according to claim 4, characterized in that: The steering stimulation refers to stimulating the left and right sides of the ventroposterior medial nucleus of the rat brain nerve center; During the training process, turning stimulation is used to induce the rat's biological body to make a turning movement. After the rat shows the correct intention to turn in the direction, a reward stimulus is applied, forming a positive feedback training model of preset action-turning stimulation-correct turning-reward stimulation; after the rat shows the wrong turning direction, the reward stimulus in that direction is immediately stopped.

7. A multimodal perception rat robot autonomous navigation method, characterized in that: The multimodal perception rat robot autonomous navigation system according to any one of claims 1 to 6 comprises the following steps: Step S0: This step is the functional self-checking stage; First, test the communication function between the wearable electronic control backpack and the remote PC host computer; second, test the signal acquisition and processing of the video module, audio module, and infrared sensor module; then, fasten the wearable electronic control backpack to the back of the rat robot with Velcro, complete the connection between the electrical stimulator module and the electrode interface implanted in the rat's head, and test the rat robot's response to reward stimulation and left and right steering stimulation; after completing the test, jump to step S1; Step S1: This step is the beginning stage of exploration; The remote PC sends a task start command to activate the rat's biological body excitement and increase its willingness to exercise. The PC will continue to send the task start command until the inertial sensor recognizes that the rat robot has entered the active state, and then jump to step S2. Step S2: This step is the target site exploration stage; First, the remote PC sends a search motion command of duration t. If no target object is found within this time, the remote PC sends a small-angle turn motion command and continues with the next search motion command of duration t. After the turn motion command, the inertial sensor detects the direction of the rat's movement. If the rat shows a tendency to stick to the wall, the turn motion command is sent again to correct the rat robot's movement direction. Repeat the above steps of exploring the target site to ensure that the rat robot traverses the path in the exploration environment in a spiral manner until the video module or the infrared sensor module finds the target object, and then jump to step S4; During the exploration process, if the audio sensor captures the sound characteristics of the target object, the relative position of the rat robot and the target object is identified using a two-channel audio signal. The remote PC sends a steering command to guide the rat robot toward the target until the audio sensor can no longer clearly distinguish between the left and right channels. The control process of the search motion command described above is then continued to guide the rat robot to complete the exploration task of the target site. During the exploration process, if the dead zone discrimination model detects that the rat robot falls into a motion dead zone, the process jumps to step S3; wherein the dead zone discrimination model is based on SVM; Step S3: This step is the phase of leaving the dead zone; When the rat is detected to have fallen into the dead zone, the remote PC host computer stops sending the current stimulation command; the host computer sends a turning command with the opposite direction of the rat robot's head before it fell into the dead zone as the target direction; after turning to a specific direction, the rat robot executes the exploration movement command for a duration of t; if it is determined that the rat robot has not left the dead zone, the host computer continues to send small-angle turning commands and then executes the search movement command for a duration of t; Repeat the above steps of escaping the dead zone until the rat robot escapes the dead zone, and then jump to step S2; Step S4: This step is the directional guidance stage; After the video module or infrared sensor module finds the target object to be searched, the remote PC host computer sends a forward movement instruction to guide the rat robot to move in a straight line towards the target object; Repeat the above-mentioned directional guidance steps until the video module or the infrared sensor module determines that the rat robot is close enough to the target object, and then jump to step S5 of the process; Step S5: This step is the end stage; In this step, the rat has completed the target search task and all stimulation instructions will be stopped.

8. The multimodal perception rat robot autonomous navigation method according to claim 7, characterized in that: In step S4, if the image sensor or infrared sensor does not detect the target object for more than 30 seconds, the forward movement instruction is stopped; at this time, the process jumps to step S2 and the site traversal search is performed again.

Images