Brain inspiration complex scene intelligent method and system

By building a decision support model for perception, goal and inference networks, using the combination of artificial neural networks and time-pulse neural networks, the problems of high computing resource consumption, opaque decision-making and insufficient adaptability in complex environments are solved, and high efficiency is achieved , explainable and reliable decision-making process.

CN119088218APending Publication Date: 2024-12-06TSINGHUA UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411152494.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing deep learning models are highly consuming in complex environments, opaque decision-making processes and inadequate adaptability, especially in the face of unknown environments or emergencies.

Method used

A well-known intelligent method for brain-inspiring complex scenarios is proposed. By constructing a decision-making support network model including perception network, target network and inference network, using artificial neural network and time pulse neural network combination, converting visual signals into pulse signals and symbolizing them Interaction, generate action decisions, and improve decision transparency and adaptability.

Benefits of technology

It realizes efficient, interpretable and reliable decision-making processes in complex and dynamic environments, significantly reduces computing resource consumption and improves robot response speed and accuracy in emergencies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088218A_ABST
    Figure CN119088218A_ABST
Patent Text Reader

Abstract

The invention discloses a brain enlightenment complex scene intelligent method and system. The method comprises the following steps: constructing a decision support network model in a brain enlightenment complex scene; wherein the decision support network model comprises a sensing network, a target network and an inference network; inputting a visual signal acquired by a robot into a decision support network model, and converting the visual signal into a pulse signal by using a sensing network so as to convert the pulse signal into different neuron groups; encoding the navigation information of the robot navigation system into a target-oriented neuron group through the target network; a reasoning network is utilized to simulate neuron groups from a sensing network and a target network, and complex symbolic reasoning is executed through symbolic interaction among the neuron groups to finally generate an action decision. According to the method, the problems of high computing resource consumption, opaque decision and insufficient adaptability in the prior art are solved, so that an efficient, explainable and reliable decision process in a dynamic and complex environment is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and robotics, and in particular to a brain-inspired complex scenario embodied intelligence method and system. Background Art

[0002] The application of intelligent robots in complex environments is becoming increasingly important, especially in the fields of health care, manufacturing, disaster relief, etc. At present, the decision-making framework of robots mainly relies on deep learning models to process visual and motion data to perform tasks and make responses. Existing deep learning solutions have been widely used in big data processing and have certain generalization capabilities and flexibility. These technologies mainly focus on parsing and processing input data through neural network architectures such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Although these models perform well on specific tasks, they usually require a lot of computing resources, and the decision-making process of the models is not transparent enough, making them difficult to explain and verify, especially in critical tasks and real-time decision-making. In addition, these models often need to be improved in terms of adaptability and reliability when facing unknown environments or new situations.

[0003] Although deep learning technology has made significant progress in the field of intelligent robots, existing technologies still have several key flaws in practical applications. First, most deep learning models, especially those based on large-scale data training, have very high requirements for computing resources, which becomes a major obstacle in mobile or resource-limited environments. For example, current LLM and other deep neural network models require a lot of GPU resources and power support when performing complex tasks.

[0004] Second, the decision-making process of existing models is often opaque and lacks explainability. This is particularly problematic in applications that require robots to make fast and accurate decisions, such as in emergency rescue or complex operating environments. The unexplainability of robotic systems increases operational risks and limits their reliability and trust in critical applications.

[0005] In addition, existing technologies are not adaptable enough when dealing with unknown or dynamically changing environments. Although some models can handle tasks in standard scenarios, their performance is often greatly reduced when faced with non-standard environments or scenarios that require advanced reasoning and decision-making capabilities. This shortcoming is mainly due to data limitations and insufficient generalization capabilities during model training. Summary of the invention

[0006] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0007] To this end, a brain-inspired complex scene embodied intelligence method is proposed, which can not only effectively reduce the consumption of computing resources, but also improve the transparency and explainability of the decision-making process, while enhancing the adaptability and reliability of robots in complex and dynamic environments.

[0008] Another object of the present invention is to propose a brain-inspired complex scenario embodied intelligent system.

[0009] To achieve the above objectives, the present invention proposes a brain-inspired complex scene embodied intelligence method, comprising:

[0010] Constructing a brain-inspired decision support network model in complex scenarios; wherein the decision support network model includes a perception network, a target network and a reasoning network;

[0011] Inputting the visual signal acquired by the robot into the decision support network model, and converting the visual signal into a pulse signal using a perception network, so as to convert the pulse signal into different neuron groups;

[0012] The navigation information of the robot navigation system is encoded into a target-directed neuronal population through the target network;

[0013] The inference network is used to simulate the neuron populations from the perception network and the target network, and complex symbolic reasoning is performed through symbolic interaction between the neuron populations to ultimately generate an action decision.

[0014] The brain-inspired complex scene embodied intelligence method of the embodiment of the present invention may also have the following additional technical features:

[0015] In one embodiment of the present invention, the perception network includes a combination network of an artificial neural network and a time pulse neural network.

[0016] In one embodiment of the present invention, converting the visual signal into a pulse signal using a perceptual network includes:

[0017] Acquire image data received by the sensor of the robot through a perception network;

[0018] The image data is converted into a feature vector.

[0019] In one embodiment of the present invention, converting the image data into a feature vector comprises:

[0020] The CLIP-based perception module is used to extract the category label, bounding box and semantic features of each object in the image data.

[0021] In one embodiment of the present invention, the neuron population includes subgroups of positive excitation neurons, negative excitation neurons and inhibitory neurons; the inhibitory neurons are closely connected to each pair of positive and negative excitation neurons to ensure that the values ​​of the positive and negative excitation neurons always remain opposite.

[0022] In one embodiment of the present invention, the inference network is used to integrate conflict detection neurons, wherein the conflict detection neurons include local conflict neurons and global conflict neurons.

[0023] In one embodiment of the present invention, the local conflict neurons reflect whether there is a contradiction or conflict in each local neural population; the global conflict neurons receive projections from all local conflict neurons and collect and process global conflict information.

[0024] To achieve the above-mentioned purpose, the second aspect of the present invention provides a brain-inspired complex scene embodied intelligent system, including:

[0025] The quadruped robot uses laser radar and IMU to determine position information, and the core control unit of the quadruped robot runs on an internal computing module;

[0026] The images captured by the cameras located on the head and gripper of the quadruped robot are transmitted to the external computing module via a wireless connection, and the artificial neural network module and the time pulse neural network module are run on the external computing module;

[0027] The artificial neural network module performs object detection and feature extraction from the images captured by the camera. The time pulse neural network module determines the action plan based on the information received from the artificial neural network module and generates corresponding instructions to move the quadruped robot to the fire source and operate the robotic arm for detection or manipulation.

[0028] The brain-inspired complex scenario embodied intelligence method and system of the embodiments of the present invention solve the problems of high computing resource consumption, opaque decision-making and insufficient adaptability in the prior art, so as to realize an efficient, explainable and reliable decision-making process in dynamic and complex environments.

[0029] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0031] Figure 1 is a flowchart of a brain-inspired complex scene embodied intelligence method according to an embodiment of the present invention;

[0032] Figure 2 is an architecture diagram of a brain-inspired complex scenario embodied intelligence method according to an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of a brain-inspired reasoning network structure according to an embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of the structure of a brain-inspired complex scenario embodied intelligent system according to an embodiment of the present invention;

[0035] Figure 5 It is a schematic diagram showing the decision-making process of a complex scenario embodied intelligent system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] The following describes a brain-inspired complex scenario embodied intelligence method and system according to an embodiment of the present invention with reference to the accompanying drawings.

[0039] Figure 1 is a flowchart of a brain-inspired complex scene embodied intelligence method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0040] S1, constructing a decision support network model in a brain-inspired complex scenario; wherein the decision support network model includes a perception network, a target network and a reasoning network;

[0041] S2, inputting the visual signal acquired by the robot into the decision support network model, and converting the visual signal into a pulse signal using a perception network, so as to convert the pulse signal into different neuron groups;

[0042] S3, encoding the navigation information of the robot navigation system into a target-directed neuronal population through the target network;

[0043] S4, using the inference network to simulate the neuron populations from the perception network and the target network, and performing complex symbolic reasoning through symbolic interactions between the neuron populations to ultimately generate an action decision.

[0044] It is understandable that the present invention uses time-spiking neural networks for embodied decision-making. The present invention sparses complex high-dimensional visual neural states into neuronal populations with symbolic meanings, and uses time-dependent plasticity to seamlessly integrate these symbols to form reliable inferences. This framework enables quadruped robots to identify, adapt and perform tasks in complex environments. To achieve this goal, the present invention uses abstract concepts established within and across neuronal populations to make decisions. This framework receives two types of input data: visual information from the robot's sensors and the current goal it aims to achieve. The output of the present invention is either a sub-goal of a subsequent task or a related action API to control the robot.

[0045] Specifically, the present invention realizes an advanced decision support network model through the close cooperation of the perception network, the target network and the reasoning network. Figure 2 As shown in the figure, the perception network converts visual signals into pulse signals through a combination of artificial neural networks and SNNs. These pulse signals are then transmitted to each neuron population, effectively deconstructing complex high-dimensional visual data into low-dimensional subspaces with symbolic meaning. Similarly, the target network converts input targets into corresponding neuron populations. In the reasoning stage, the robot identifies the functions or physical properties of all observed objects in the environment through the perception network and evaluates their availability to perform the corresponding tasks through the reasoning network. When an object suitable for performing a task is identified, the target network generates a sub-goal and adjusts its subsequent task goals accordingly.

[0046] In one embodiment of the present invention, the perception network uses a combination of artificial neural networks and time-spiking neural networks to process input data, converting it into different neuron populations that make decisions through their temporal combinations, using reservoir computing structures to enhance the robustness of decisions. The image data received by the perception network from the robot's sensors is converted into feature vectors, a process that uses a CLIP-based perception module to extract the category label, bounding box, and semantic features of each object. The resulting feature vectors are stored in a continuously updated scene memory dictionary for subsequent use.

[0047] In one embodiment of the present invention, the target network processes abstract information from the navigation system or mission planning module, such as mission goals or locations, and encodes this information into goal-oriented neural activities. These signals are transmitted to the reasoning network together with the signals from the perception network, where they are used to form complex decision paths. The design of the target network is inspired by the mechanism of symbolic representation and calculation in the biological brain, and symbolic reasoning is achieved through neural discharges and synaptic connections. In addition, inhibitory neurons and conflict neurons are introduced to control the temporal pulse behavior to ensure the logical correctness of the reasoning process.

[0048] In one embodiment of the present invention, the inference network receives signals from the perception network and the target network, simulates these signals using a time pulse neural network structure, and performs complex symbolic reasoning through symbolic interactions between neural populations to ultimately generate action decisions. The inference network contains carefully designed synaptic connections to ensure the accuracy of logical operations in the decision-making process. Time pulse interactions between neural populations are used to simulate the decision-making process. These neural populations are carefully organized into subgroups of positive excitation neurons, negative excitation neurons, and inhibitory neurons. The design of these subgroups enables the inference network to have functions similar to logic circuits, thereby ensuring the logical accuracy of its reasoning process. The inference network also integrates conflict detection neurons, including local conflict neurons and global conflict neurons, to resolve possible logical conflicts between different neural populations.

[0049] In one embodiment of the present invention, the reasoning network receives signals from the perception network and the target network, simulates these signals using a time-spiking neural network (SNN) structure, and performs complex symbolic reasoning through the interaction between symbolic neural populations to ultimately generate action decisions. The reasoning network design of the present invention includes carefully designed synaptic connections to ensure the accuracy of logical operations in the decision-making process.

[0050] Exemplarily, the logic operation in the present invention simulates the decision-making process through the time pulse interaction between neural groups. To ensure the accuracy of the logic operation in the decision, the present invention carefully designs the synaptic connection of the network, such as Figure 3 This design covers three key aspects: 1. The input of high-dimensional neural states is precisely decomposed and sparsely distributed into neural populations in the reasoning network; 2. Each neural population in the reasoning network represents a specific concept based on common sense knowledge; 3. To achieve reliable reasoning, logical conflicts generated by interactions between different neural populations need to be resolved.

[0051] To achieve these goals, each neural population is carefully organized into subpopulations of positive excitatory neurons, negative excitatory neurons, and inhibitory neurons. Specifically, Figure 3 The PEN and NEN neurons on the left are connected to the perception network, while those on the right are connected to the target network. At the same time, the inhibitory neurons are closely connected to each pair of positive and negative excitation neurons, ensuring that the values ​​of the positive and negative excitation neurons always remain opposite. This structural mechanism enables the reasoning network to have functions similar to logic circuits, thereby ensuring the logical accuracy of its reasoning process.

[0052] Furthermore, the reasoning network considers a wide range of factors to make a comprehensive judgment, which requires resolving conflicts that may arise within or between different neural populations. Therefore, it integrates conflict detection neurons, including local conflict neurons (LCNs) and global conflict neurons (GCNs). Local conflict neurons reflect whether there is a contradiction or conflict in each local neural population, while global conflict neurons receive projections from all local conflict neurons, collect and process global conflict information. This approach is crucial for understanding and processing the relationship between different populations. In particular, when a firing signal appears in a local conflict neuron, it indicates that there is a local logic mismatch in the current reasoning step (such as Figure 3 At this time, the signal emitted by the local conflict neuron will also be activated (as shown in Figure 3 As soon as the global conflict neuron is activated, the reasoning step ends with a negative outcome, and the framework continues to look for other available objects to complete the goal. Only when the global conflict neuron is activated with a positive outcome can the output subgoal or motor command be executed.

[0053] In summary, the present invention adopts a three-layer network structure, including a perception network (SenseNet), a target network (TargetNet) and a reasoning network (ReasonNet), to achieve efficient execution of complex decisions. The perception network converts the received visual signals into pulse signals by combining artificial neural networks (ANN) and time pulse neural networks (SNN), and then deconstructs high-dimensional visual data into low-dimensional subspaces with symbolic meanings. The target network is responsible for processing the abstract information received from the navigation system or task planning module and converting it into goal-oriented neural activities. The reasoning network receives signals from the perception network and the target network, simulates these signals using the SNN structure, and performs complex symbolic reasoning through symbolic interactions between neural groups to generate specific action decisions.

[0054] The brain-inspired complex scenario embodied intelligence method according to an embodiment of the present invention solves the problems of high computing resource consumption, opaque decision-making and insufficient adaptability in the prior art, so as to achieve an efficient, explainable and reliable decision-making process in dynamic and complex environments.

[0055] In order to implement the above embodiment, Figure 4 As shown, this embodiment also provides a brain-inspired complex scene embodied intelligent system, which is applied to a quadruped robot, including:

[0056] The quadruped robot uses a laser radar and an IMU to determine position information, and the core control unit of the quadruped robot runs on an internal computing module;

[0057] The images captured by the cameras located on the head and gripper of the quadruped robot are transmitted to the external computing module via a wireless connection, and the artificial neural network module and the time pulse neural network module are run on the external computing module;

[0058] The artificial neural network module performs object detection and feature extraction from the images captured by the camera. The time pulse neural network module determines the action plan based on the information received from the artificial neural network module and generates corresponding instructions to move the quadruped robot to the fire source and operate the robotic arm for detection or manipulation.

[0059] It is understandable that the present invention deploys a brain-inspired neural-symbolic binary embodied intelligence system in a practical scenario, specifically applied to a quadruped robot named Aliengo, which is used to respond to emergency fire situations in a kitchen environment. The Aliengo robot has 12 degrees of freedom and is equipped with a Z1 robotic arm with 6 degrees of freedom, enabling it to perform precise manipulation tasks. The core control unit uses an Intel NUC computer to coordinate the work of various sensors and actuators. The RGBD camera installed on the robotic arm provides depth perception and visual signals for object detection, grasp recognition, and distance measurement. A second RGBD camera installed on the head of Aliengo captures environmental data from another angle. The single-line laser radar (SLAMETECRPLIDAR A2) enhances perception and navigation capabilities, enabling real-time two-dimensional mapping of the robot's surroundings. An IMU (WHEELTEC N100) sensor is installed inside. Aliengo communicates via Wi-Fi with an external computer that runs the NEBULA framework and is equipped with an NVIDIA RTX 4090 graphics card and an Intel Core i9-13900K CPU with 24 cores and 128GB of RAM. Control commands are transmitted back to the quadruped robot via Wi-Fi.

[0060] Figure 4The overall picture of the system is shown. The quadruped robot determines its position using LiDAR and IMU. The simultaneous localization and mapping (SLAM) unit for movement runs on the Intel NUC. The images captured by the cameras located on the robot's head and gripper are transmitted to another computer via a Wi-Fi connection. On this second computer, the Artificial Neural Network (ANN) module and the Temporal Spiking Neural Network (SNN) module are run. The ANN module performs object detection and feature extraction from the input image. It uses the CLIP46 model at a frequency of 2Hz for object recognition and feature extraction, and transmits the extracted features to the SNN module. The SNN module runs at a higher frequency of 10Hz, integrating scene memory and target recognition, determines the appropriate action plan based on the information received, and generates the corresponding instructions to move the quadruped robot to the fire source and operate its robotic arm for detection or manipulation. These commands are transmitted wirelessly back to the robot, where they are executed by the quadruped robot's control system.

[0061] The brain-inspired complex scene embodied intelligent system according to an embodiment of the present invention integrates a physical robot Aliengo, which has multiple degrees of freedom and is equipped with advanced sensors and actuators, such as RGBD cameras and lidars, which work together to enable the robot to perform effective spatial navigation and task execution in complex environments. In addition, the system also includes an external computer equipped with a high-performance processor and graphics card, which is responsible for running and processing data collected from the robot's sensors, and achieving real-time communication with the robot via Wi-Fi to ensure the robot's reaction speed and decision-making accuracy when performing tasks such as fire fighting and other emergency situations. This overall architectural design enables the present invention to demonstrate excellent performance and high adaptability in practical applications.

[0062] The beneficial effects of the present invention are:

[0063] The brain-inspired embodied intelligence system of the present invention significantly surpasses traditional deep learning models in various key performance indicators, specifically in terms of computational efficiency and network scale in processing complex scenarios. According to the data in Table 1, the present invention is compared with traditional models such as Transformer, recurrent neural network (RNN), long short-term memory network (LSTM) and spiking neural network (SNN) in terms of network complexity (floating point operations, GFLOPs), model size (quantified by the number of parameters), relative network complexity (i.e., the number of floating point operations performed on each parameter), the percentage of activated synapses, and the accuracy of reasoning. The present invention demonstrates its extraordinary balance between size and performance, achieving a best accuracy of 96.3% while requiring only 0.00081 GFLOPs of computation—much lower than other models. In addition, the present invention is particularly outstanding in relative network complexity, which reflects the average number of floating point operations performed on each parameter in a single forward propagation. The value of the present invention is three orders of magnitude lower than that of Transformer, showing that only a small number of neurons need to be activated for reasoning. This huge difference is mainly due to its extremely sparse structure requiring only a small number of synaptic activations (only 1.25% compared to other models).

[0064] Table 1

[0065]

[0066] Figure 5 The task description and visualization of the internal neural states during the execution phase demonstrate the system's advanced capabilities in complex task execution. A series of schematic diagrams show how NEBULA generates specific action sequences through the dynamic firing states of the neural population during specific task execution, including different response strategies in fire scenarios.

[0067] exist Figure 5 In the scenario shown, when an oil pan catches fire, the robot needs to perform a series of actions to extinguish the flames, including grabbing a cup, filling it with water from a dispenser, pouring the water on the gloves, and finally placing the wet gloves on the oil pan. Figure 5 In another scenario shown, since there is a pot lid on the table, the robot can directly put the lid on the pot to extinguish the flame. Figure 5 b and Figure 5 The graphs in d show the firing state of the neural population of NEBULA during the execution phase. These graphs represent the activity inside the spiking neural network (SNN), showing the firing patterns of different neural populations associated with specific actions. The x-axis represents time, while the y-axis quantifies the firing rate of each neural population within a short time window. The present invention can effectively generate long-term executable atomic action sequences for controlling the Aliengo robot, demonstrating its robustness and accuracy in generating extended sequences.

[0068] The realization of these functions highlights the many beneficial effects of the present invention, especially in the performance of tasks with high dynamics and complexity. By precisely regulating the firing state of each neural population, the system not only optimizes the generation of action sequences, but also improves the speed and accuracy of response to emergency situations. In addition, by monitoring and adjusting the neural state in real time, the present invention can adapt to sudden changes, such as quickly switching firefighting strategies in fire scenes. This flexibility and adaptability are critical for applications in real-world conditions, especially in emergency response tasks that require fast and accurate decision-making.

[0069] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0070] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. A brain-inspired approach to embodied intelligence in complex scenarios, It is characterized in that include: Constructing a decision support network model in a brain-inspired complex scenario; wherein the decision support network model includes a perception network, a target network and a reasoning network; Inputting the visual signal acquired by the robot into the decision support network model, and converting the visual signal into a pulse signal using the perception network, so as to convert the pulse signal into different neuron groups; The navigation information of the robot navigation system is encoded into a target-directed neuronal population through the target network; The inference network is used to simulate the neuron populations from the perception network and the target network, and complex symbolic reasoning is performed through symbolic interaction between the neuron populations to ultimately generate an action decision.

2. The method according to claim 1, It is characterized in that The perception network includes a combination network of an artificial neural network and a time pulse neural network.

3. The method according to claim 1, It is characterized in that The visual signal is converted into a pulse signal using a perception network, including: Acquire image data received by the sensor of the robot through a perception network; The image data is converted into a feature vector.

4. The method according to claim 3, It is characterized in that Converting the image data into a feature vector includes: The CLIP-based perception module is used to extract the category label, bounding box and semantic features of each object in the image data.

5. The method according to claim 4, It is characterized in that The neuron population includes subgroups of positive excitation neurons, negative excitation neurons and inhibitory neurons; the inhibitory neurons are closely connected to each pair of positive and negative excitation neurons to ensure that the values ​​of the positive and negative excitation neurons always remain opposite.

6. The method according to claim 5, It is characterized in that The inference network is used to integrate conflict detection neurons, wherein the conflict detection neurons include local conflict neurons and global conflict neurons.

7. The method according to claim 6, It is characterized in that The local conflict neurons reflect whether there is a contradiction or conflict in each local neural population; the global conflict neurons receive projections from all local conflict neurons and collect and process global conflict information.

8. A brain-inspired complex scene embodied intelligence system, It is characterized in that Applications in quadruped robots include: The quadruped robot uses laser radar and IMU to determine position information, and the core control unit of the quadruped robot runs on an internal computing module; The images captured by the cameras located on the head and gripper of the quadruped robot are transmitted to the external computing module via a wireless connection, and the artificial neural network module and the time pulse neural network module are run on the external computing module; The artificial neural network module performs object detection and feature extraction from the images captured by the camera. The time pulse neural network module determines the action plan based on the information received from the artificial neural network module and generates corresponding instructions to move the quadruped robot to the fire source and operate the robotic arm for detection or manipulation.

9. The system according to claim 8, It is characterized in that The artificial neural network module uses the CLIP46 model to perform object recognition and feature extraction at a frequency of 2 Hz, and transmits the extracted features to the artificial neural network module.

10. The system according to claim 8, It is characterized in that The quadruped robot has 12 degrees of freedom and is equipped with a Z1 robotic arm with 6 degrees of freedom. The internal computing module uses an Intel NUC computer. The RGBD camera installed on the robotic arm of the quadruped robot provides depth perception and visual signals for object detection, grasping recognition and distance measurement. The RGBD camera installed on the head of the quadruped robot captures environmental data from another angle. The quadruped robot is equipped with an IMU sensor inside. The quadruped robot communicates with the external computing module via Wi-Fi, and the control commands are transmitted back to the quadruped robot via Wi-Fi.

Citation Information

Cited By

  • Intelligent agent simulation decision-making method and device based on large language model

    CN120069585A