705 results about "Perception" patented technology

Approaches presented herein provide for the use of language models to generate tokenized descriptions of physical environments. In at least one embodiment, sensor and / or observational data can be obtained for an environment and used to generate a set of perception data. The perception data can be analyzed, along with approximate positional data within the environment, to identify a set of aligned map data. The aligned map data and perception data can be provided as input to a trained language model, which can be trained to correlate and / or fuse the information to generate a single, consistent representation of the environment. The language model can output a tokenized description of the environment, which can be in a domain-specific language, that is a compact but robust textual description of the environment.

The invention discloses a scene interactive AI rehabilitation evaluation training and health monitoring system, and relates to the technical field of rehabilitation medical treatment and artificial intelligence, a semantic perception module is used for collecting and recognizing voice input, facial expressions, action tracks and eye movement paths of a user in a training process, and extracting context parameters; the knowledge-driven training generation module is used for calling a rehabilitation knowledge graph constructed by a graph neural network based on context parameters and individual training history, and generating a multi-path training scheme; training a feedback regulation engine, collecting posture offset, physiological stress and emotion feedback, and dynamically adjusting task difficulty, rhythm and prompt mode based on a dual-channel reinforcement learning model; the prediction module fuses training and monitoring data, and predicts a network identification function degradation risk through degradation driving; the cloud edge fusion platform is used for realizing task quick response and graph strategy iterative updating; according to the invention, the individuation, self-adaption and intelligent prediction capabilities of rehabilitation training are improved, and the rehabilitation effect and the system practicability are obviously optimized.

The invention relates to a remote education data processing system which comprises the following steps: under a remote teaching task, pre-defining a task intention and a data expectation point; a semantic timestamp and a task binding label are printed on each data fragment; mapping the collected confusion data including silence, eye movement drift and prediction into a unified learning semantic vector; a micro-expression + interactive behavior + time sequence decision path ternary modeling mode is introduced, and a potential cognitive intention corresponding to the feature combination is recognized; teaching context information is fused; constructing a cognitive state mapping model; reasoning a current cognitive state label from multi-modal sensing data; searching intervention track VS effect feedback data in a historical database; generating a predicted intervention behavior sequence by using a sequence modeling algorithm; a dynamic combination suggestion chain including light prompt, content reconstruction, personalized practice and tutoring invitation is adopted; and superposing the cognitive state sequences of all students into a group cognitive trajectory map.

The invention relates to the technical field of AI interaction data processing, and discloses an AI interaction data processing system based on multi-modal perception and dynamic decision, comprising the following modules: a multi-modal perception module used for collecting environment data through a multi-source sensor; the data fusion module is used for generating fused feature data; the causal decision-making module is used for generating a decision-making action to cope with the change of the environment; the decision security module is used for identifying potential safety hazards in the high-risk scene and generating alternative decision or early warning information; the sensing calibration module is used for optimizing a sensing strategy in a changing environment; and the adaptive optimization module continuously optimizes the perception and decision strategy. According to the invention, the multi-modal sensing module is combined with a cross-modal consistency learning mechanism and a noise robustness enhancement technology, and the data fusion module introduces a context sensing attention mechanism and a multi-level feature alignment network, so that the sensing ability of the system to complex environment information and the comprehensiveness and accuracy of feature representation are effectively improved.

The invention relates to the technical field of slope protection, in particular to a slope protection intelligent detection system based on deep learning. According to the technical scheme, the system comprises a multi-source heterogeneous data sensing module, a data fusion and feature extraction module, a slope state intelligent diagnosis and early warning module, an edge-cloud collaborative computing architecture and a system optimization module. Registration and feature complementation of multi-source heterogeneous data are realized through a multi-modal detection network, an overfitting phenomenon is effectively inhibited through a physical information neural network architecture, risk quantitative evaluation is realized through construction of a dynamic risk evaluation model, early warning response time is shortened in cooperation with a four-level early warning strategy, the false alarm rate is reduced, and the early warning efficiency is improved. Besides, the detection precision of the system in an extreme scene is improved through a physical constraint adversarial training method, so that the environmental adaptability of the system is improved, continuous updating and evolution of the model are realized through an online incremental learning module, and the problem of performance degradation of a traditional system caused by change of geological conditions is solved.

The invention provides an unmanned cluster brain-like navigation map fusion construction and cooperative positioning method. The method comprises the following steps: acquiring multi-modal perception data, respectively acquiring vision, sound wave and pose information through a binocular camera, a radar and an inertial sensor, simulating a human brain nerve coding mechanism, and generating a pulse sequence and a feature vector in combination with spatio-temporal information; constructing a multi-modal coding unit into a hypergraph node, and based on a dynamic hyperedge connection topological relation, aggregating spatial-temporal characteristics through a heterogeneous hypergraph convolutional network to generate a high-order brain-like semantic map of a single agent; sharing a local brain-like map by multiple agents through distributed communication, detecting geometric and semantic conflicts of an overlapped region, performing space-time alignment based on an anchor point reference, eliminating feature contradictions by utilizing probability distribution matching, and generating a global consistent high-confidence brain-like map; and outputting the optimal position estimation. Through bionic neural coding, heterogeneous hypergraph modeling and multi-agent collaborative optimization, establishment and positioning of a high-order brain-like map in a dynamic unknown environment are realized.

The invention discloses a brain tumor multi-mode large model construction method, device and equipment and a storage medium, and is applied to the technical field of brain tumor imagines.The method comprises the steps that pixel-concept level alignment is conducted on a multi-mode MRI image and a pathological text; constructing a multi-modal feature fusion network for fusing image features and text features by adopting an attention mechanism of pathology perception and combining medical semantic information; training the multi-modal feature fusion network to generate an analysis report and a segmentation result; according to the technical scheme of multi-task cooperation, cross-modal pathological semantic accurate alignment, pathological knowledge graph injection and lightweight and continuous optimization parallelization, full-process coverage of brain tumor accurate segmentation, analysis report generation and prognosis prediction is achieved, the problems that a traditional model lacks pathological semantic support and is insufficient in clinical adaptability are solved, and the clinical adaptability of the traditional model is improved. And the deployment feasibility and the dynamic optimization capability are also considered.

The present disclosure relates to a multi-modal brain network computation method associated with structural function, apparatus, device, and medium. The method is applied to train a brain disease prediction model, and the brain disease prediction model includes an association perception dual-channel generation module, a disease feature regression module, a topological structure discriminator, and a time-space joint discriminator. In a model training process, by performing a multi-level interactive fusion learning on a high-order topological feature of brain functional magnetic resonance data and magnetic resonance diffusion tensor imaging data, a multi-modal time series activity signal of each brain region is obtained.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as medical health fund, fusion science and technology and mental health recuperation, and discloses a guidance method, device, equipment and medium based on multi-modal perception.The guidance method comprises the steps that physiological index data and motion trail data are collected, and a multi-modal body data set is generated; inputting a pre-trained mind and body association model to generate a mind and body state mapping relation; combining the three-dimensional scene model library and the dynamic attention parameters to generate an immersive interaction scene, and analyzing a scene interaction instruction based on a domain knowledge graph; and fusing the user portrait and the real-time state vector, generating a comprehensive decision parameter, and screening personalized guidance content from the guidance content library according to the feature similarity between the comprehensive decision parameter and the scene interaction instruction and outputting the personalized guidance content. According to the method, through multi-modal data acquisition and body and mind state modeling, accurate perception of the physiological state and behavior characteristics of the user is realized, and the accuracy of personalized guidance is improved in combination with immersive scene interaction and knowledge graph analysis.

The invention relates to the technical field of child cognitive training, and discloses an ADHD child cognitive training system. The system comprises a multi-modal sensing module, an attention evaluation module, a dynamic task generation module, a behavior feedback module and a self-adaptive adjustment module. The multi-modal perception module collects multi-dimensional behavior data, the attention evaluation module extracts attention fluctuation features, the dynamic task generation module generates a cognitive training task sequence, the behavior feedback module performs emotion state classification processing to generate feedback signals, and the self-adaptive adjustment module generates personalized training difficulty steps. According to the system and the method, through comprehensive data acquisition, precise attention assessment, dynamic task generation, real-time emotion feedback and difficulty adaptive adjustment, personalized and precise cognitive training of ADHD children is realized, the training effect is effectively improved, and the problems of lack of personalization, insufficient state monitoring, unscientific difficulty adjustment and the like of an existing training method are solved.

A computer system for upsampling decompressed audio data after lossy compression using specialized neural network techniques. The system processes compressed audio channels through an audio pre-processor that extracts spectral information, detects speech activity, segments audio, and normalizes input levels. A trained deep learning algorithm with multi-channel transformers using channel-wise and self-attention mechanisms recovers information lost during compression. The system further enhances audio quality through a time-frequency domain transformer applying Fourier transforms and Mel-scale frequency processing, while a perceptual quality assessor employing psychoacoustic models evaluates the output. This specialized audio processing approach significantly improves reconstructed audio quality by leveraging correlations between audio channels, addressing both spectral and temporal features, and optimizing for human perception characteristics, resulting in higher fidelity audio reproduction from compressed formats.

The present invention relates to the technical field of expression recognition, and in particular, to an expression recognition method and system based on multi-scale features and spatial attention. The method includes: performing feature extraction on acquired facial image data by using an HNFER neural network model to obtain an original input feature map; performing pooling and concatenation on extracted features based on a CoordAtt attention mechanism to obtain a feature map; performing deep convolution processing on the feature map to obtain an attention map, and then performing element-by-element multiplication to obtain a final feature map; and performing feature transformation and normalization on the final feature map to obtain an expression category probability and output the expression category probability. In the present invention, by integrating scale perception and spatial attention technologies, the model can recognize and classify different emotional states more accurately and maintain high performance even under complex environmental conditions.

The invention belongs to the technical field of artificial intelligence and health monitoring, and particularly relates to an emotion recognition and intervention system based on facial micro-expression and physiological signal fusion. The emotion recognition precision is improved through space-time alignment analysis of facial micro-expressions and physiological signals, adaptive feedback is achieved in combination with cognitive load correlation modeling and a wearable multi-channel regulation and control terminal, cross-period emotion evolution prediction and group situation awareness are supported, a'awareness-decision-intervention 'complete closed loop is constructed, and the emotion recognition efficiency is improved. And the accuracy and initiative of emotion management in a complex environment are enhanced.

The invention belongs to the technical field of intelligent control of robots, and discloses a self-adaptive adjustment system for behavior modes of an intelligent robot with a body, which comprises a neuromorphic sensing module for acquiring and fusing multi-mode environment sensing information, performing space-time compression on the multi-mode environment sensing information by referring to a brain visual cortex sparse coding principle, and generating a neural network; generating an environment state representation vector, and constructing a neuromorphic perception path; the cognitive constraint decision module is used for generating an action strategy by adopting a neural symbol hybrid architecture based on the environment state representation vector; through a cognitive momentum optimization mechanism, a strategy inertia item is introduced to improve an updating process of an action strategy parameter, and a behavior decision vector is obtained; the variable impedance execution module is used for mapping the behavior decision vector into a plurality of virtual muscle cooperation elements through a muscle cooperation mapping rule, and solving an expected movement track of each execution joint of the robot; and the intelligent robot with the body has higher adaptability, stability and execution capability.

The invention discloses a health monitoring system and method based on multi-sensor fusion. The system comprises a health index acquisition system used for acquiring various health index data through various sensors; the multi-modal data analyzing and processing system is used for analyzing and processing the obtained multi-modal health index data to generate sensing information and analyzing and processing different data to obtain corresponding health indexes; and the health index evaluation report system is used for generating a health index evaluation report according to the perception information. According to the system, multiple data sources including images, sign signals, voice interaction and the like are fused through multi-modal data, and comprehensive health assessment is achieved. Different from a traditional health monitoring system which only depends on a single signal, the system fuses various data and can provide a more comprehensive and detailed health analysis result. Four diagnosis methods of watching, smelling, asking and cutting in traditional Chinese medicine are creatively combined with a modern intelligent technology, and modern application of health monitoring is achieved.

Provided is a method for token-based representation and learning of robotic perception data based on a graph neural network, comprising: obtaining a plurality of types of perception data of a robot; performing token-based representation according to types of the plurality of types of perception data; constructing an initial feature graph based on the plurality of types of perception data after the token-based representation; learning a compact representation of the initial feature graph based on an autoencoder and reconstructing a graph structure; after the autoencoder completes learning of the graph structure, fixing the graph structure; and converting the plurality of types of perception data into node feature vectors, constructing a feature graph based on the graph structure, and performing numerical encoding on each of the node feature vectors by utilizing the graph neural network to obtain a representation of high-dimensional feature vectors of the plurality of types of perception data.

The invention provides a self-adaptive robot sensing, guiding and controlling integrated system for a strange environment, and belongs to the field of intelligent unmanned systems. Each robot in the system comprises a sensing device, a sensing module, a navigation module and a control module. The sensing module is a VLM-based semantic mapping device, performs ground feature segmentation and semantic recognition on the environment information, and generates a global semantic map and a ground image embedded with a ground feature image semantic descriptor; the navigation module is a semantic planner based on LLM, adopts LLM to perform task understanding on natural language task description, and reasones a task sequence of a current environment; the control module is a terrain adaptation controller based on a VFM, the VFM is adopted to recognize terrain features from ground images, the terrain features and the real-time state of the robot are mapped into a driving matrix, and robot motion control is achieved. According to the invention, deep cooperation of active perception, autonomous exploration and efficient control of the robot is realized, and task planning and real-time adjustment capabilities of the robot in an unfamiliar environment are enhanced.

The invention belongs to the related technical field of brain image processing, and provides an Alzheimer's disease classification method and system based on topology perception and a group hypergraph in order to solve the problem of inaccurate classification of the Alzheimer's disease in the prior art. Constructing a dynamic function connection network sequence through a sliding window strategy; a local topology perception encoder and a global topology perception encoder are respectively used for extracting local topology features and global topology features of each time window, deep interaction and fusion are carried out, and comprehensive feature representation of a tested level is generated; according to the method, each subject is used as a hypergraph node, hyperedges are constructed on the basis of comprehensive feature representation of a subject level and by combining feature similarity calculated by diffusion tensor imaging features and clinical embedded features of the subject, then a group hypergraph is constructed, a classification result is obtained by using a hypergraph neural network, and the early classification diagnosis accuracy of the Alzheimer's disease is effectively improved.

The invention discloses an intelligent agent role switching method and system based on multi-modal perception and related components. The method comprises the steps that multi-modal perception information of a user is collected through a multi-modal perception module; determining candidate roles through an intelligent role switching decision-making mechanism based on the multi-modal perception information; when the number of the candidate role is one, setting the candidate role as a target role; when the number of the candidate roles is multiple, selecting one of the candidate roles as a target role through a multi-role conflict resolution mechanism, or synthesizing the candidate roles into the target role; and switching the current role of the intelligent agent to the target role based on a preset role model library. The role switching effect of the intelligent agent can be improved, and the use experience of the intelligent agent is enhanced.

The invention discloses a psychotherapy and healing robot based on high human emotion fitting degree simulation analysis, and the robot comprises a multi-mode perception layer which is used for collecting the interaction data of physiology, movement and environment; the multi-modal sensing layer comprises a heterogeneous data acquisition module, a spatial-temporal feature extraction network and an attention fusion mechanism module; the dynamic decision-making layer is used for generating an intervention strategy based on the interaction data; the dynamic decision-making layer comprises a reinforcement learning strategy engine and a hierarchical intervention selection tree; the generative interaction layer is used for generating a co-estrus response conforming to ethical specifications based on the intervention strategy; the generative interaction layer comprises an ethical constraint system and an emotional response generator; the brain science verification layer is used for monitoring neural feedback in real time through EEG and adjusting an intervention strategy; and the brain science verification layer comprises a neural feedback regulation module and a multi-mode feedback design module. Therefore, a precise and personalized psychological intervention decision closed loop is provided, and the defects of an existing AI psychological product in the aspects of emotion recognition, intervention strategies and effect quantification are overcome.

The invention relates to the technical field of cultural tourism guide, in particular to a marine culture intelligent guide system based on artificial intelligence and the Internet of Things, which integrates multi-source data acquisition, context semantic understanding, user portrait construction, personalized content recommendation and adaptive content presentation, and the system acquires user voice, position, facial image and environment information, so as to realize the intelligent navigation of the marine culture. Accurate data acquisition is realized; the system combines semantic understanding and user portraits, dynamically generates interest models, recommends personalized marine culture contents to users, ensures effective information transmission through a self-adaptive presentation module, optimizes resource allocation through a distributed processing module, guarantees stable operation of the system, constructs a user-environment-cultural relic three-dimensional perception network, improves the acquisition precision by 65%, and improves the real-time performance of the system. The data dimension is obviously expanded, a comprehensive data basis is provided for intelligent navigation, and efficient and personalized cultural tourism experience is achieved.

The invention provides a port facility management and maintenance large model report review agent construction method and system, and the method comprises the steps: collecting a cross-modal original data set, constructing a multi-modal feature fusion perception layer, and generating facility damage feature alignment data; constructing a cognitive neural network four-level architecture, and generating an inference decision tree; constructing a root cause-path-result causal chain, and generating a fault attribution analysis report; constructing a prediction-intervention-verification active defense closed loop, and generating a Pareto optimal maintenance strategy set; and executing an intervention strategy and feeding back a verification result by using the digital twin verification platform and the block chain evidence storage system. According to the method, cross-modal data deep semantic alignment is realized through the multi-modal feature fusion perception layer, a data island is broken, and the damage feature extraction accuracy is improved; an interpretable causal chain is constructed based on related architecture and modules, the decision black box problem is solved, and a maintenance strategy has causal logic support; and real-time verification and credible tracing of a strategy effect are realized through an active defense closed loop.

Disclosed in the present invention is a general-purpose intelligent agent. The general-purpose intelligent agent comprises: an input module, which is configured to acquire a preprocessed input signal; a consciousness module, which is configured to perform perception and cognitive operations in a transparent and interpretable abstract thinking form; a self-awareness module, which is configured for perception and cognition of an intelligent agent body; a subconsciousness module, which is configured to use a data-driven model to implement perception and cognitive functions at a subconscious level; an information exchange module, which is configured to perform mutual exchange between perception, cognition and other information generated by the consciousness module, the self-awareness module and the subconsciousness module, thereby enabling integration and utilization; and an output module, which is configured to output a processing result on demand. In a machine intelligent agent of the present invention, a machine uses a general-purpose language as an underlying language for thinking and interaction, a thinking process and result are completely transparent, and thus autonomous decision-making, autonomous learning and continuous evolution are realized under fully controllable conditions. Further disclosed is a method for constructing a general-purpose intelligent agent and a general-purpose language. Compared with the prior art in which a natural language is used, the machine intelligent agent based on the general-purpose language has wide versatility in application.

The invention relates to the technical field of artificial intelligence, in particular to a remote medical inquiry system based on AI recognition, which comprises a voice structure analysis module, a semantic association mapping module, a knowledge graph reasoning module, an inquiry path adjustment module and a disease traceability analysis module. According to the method, the logic relation of a description chain is accurately obtained by performing specific value mapping on the time delay and the semantic frequency of the voice information, the structural perception ability of the semantic information is enhanced, the symptom frequency and the health index trend are combined, accurate measurement is performed on abnormal fluctuation dependency, inquiry priority grading is optimized, and the key symptom recognition sensitivity is improved; through induction of a periodic fluctuation trend, formation of a dynamic evolution sequence, enhancement of the monitoring capability of health risk evolution, improvement of decision precision and optimization of a diagnosis chain, overall processing is performed through time sequence and semantic association analysis, the recognition and intervention capability of a complex health state is enhanced, and the health assessment adaptability and accuracy in multiple rounds of interaction are improved.

The invention discloses a self-adaptive adjustment nerve electrophysiology monitoring device for an orthopedic tumor operation, and belongs to the technical field of medical equipment. The system comprises a signal acquisition module, a signal feature extraction module, a self-adaptive adjustment control module and an intraoperative feedback early warning module, the signal feature extraction module carries out multi-angle feature extraction on neural signals and accurately identifies changes of neural states in a complex signal environment, and the self-adaptive adjustment control module dynamically adjusts monitoring and stimulation strategies according to changes of real-time neural signals in an operation, compares pre-operation baseline signals with the real-time signals in the operation, and determines whether the neural states are changed or not according to the pre-operation baseline signals and the real-time signals in the operation. The change trend of the nerve state is accurately judged, whether the nerve signal is in the normal physiological fluctuation range or not is evaluated, the intra-operation feedback early warning module converts the nerve signal into visual and auditory feedback, the perception ability of an operator to the nerve state is improved, the nerve injury risk is dynamically evaluated, and the operation safety is improved. And according to the risk level, different levels of early warning mechanisms are triggered, and an operator is suggested to adjust an operation strategy.

The invention provides an artificial intelligence and cognitive behavior combined depression treatment intervention and early warning system, which comprises the steps of collecting patient interaction data, physiological perception data and third-party data through a dynamic on-demand authorization mechanism, and extracting four-dimensional feature indexes of cognition, emotion, behavior and physiology; constructing a first psychological matrix of cognition, emotion, behavior and physiological dimensions, dynamically adjusting dimension weights by adopting a Bayesian network to generate a second psychological matrix, and fusing weighted feature scores, change rates and historical mean values to output psychological health state vectors; generating a basic scheme group based on the state vector matching treatment strategy library; and calculating the unit time fluctuation intensity of the mental health state vector in real time, and triggering three-level early warning in combination with a self-adaptive Z-score safety threshold. According to the method, multi-modal data fusion analysis, dynamic personalized intervention and active risk prevention and control are realized, and the accuracy and safety of depression management are improved.

The invention discloses a force sense feedback control method for an intelligent mechanical arm, which comprises the following steps of: 1, acquiring data through a multi-modal sensor and fusing the data to obtain a multi-dimensional perception vector; 3, calculating a force sense tracking error and a change rate and triggering an event-driven control decision mechanism; 4, designing a nonlinear compensation control rule and outputting a control torque instruction, wherein a control system comprises a multi-mode sensing module, a dynamic prediction module, an event-driven control module and a cooperative calculation module; according to the method, multi-mode sensing information is fused with the lifting force sense representation capacity, advanced adjustment is achieved in combination with a dynamic force sense prediction mechanism, event-driven control is used for reducing calculation redundancy, robustness to complex interference is enhanced through a nonlinear compensation strategy, and finally high-precision and low-delay force sense control of the mechanical arm in a dynamic interaction scene is achieved.

The embodiment of the invention provides a user state adjusting system based on multi-sensory focusing and neural remodeling, and the system comprises a physiological data interaction module which is configured to collect a multi-modal physiological signal of a user; the intelligent decision-making module is internally provided with a pre-trained neural state evaluation model so as to evaluate the current neural physiological activation state of the user according to the multi-modal physiological signal and execute feedback regulation logic based on the evaluated current neural physiological activation state of the user so as to generate a multi-modal feedback control instruction; the multi-mode feedback output module is configured to generate at least one sensory regulation stimulus based on the multi-mode feedback control instruction; and the nerve focusing guide module is configured to position a nerve physiological reflex area strongly related to the user state based on the subjective state description of the user in combination with the multi-modal physiological signal of the user, so as to generate man-machine interaction guidance to focus the attention of the user on a target reflex area and form a closed-loop regulation link of'perception feedback and re-perception '.

The invention relates to the technical field of medical image classification, in particular to a myopic macular traction lesion grading method and system. The method comprises the following steps: taking a convolutional neural network, a direction perception attention module and a classifier which are connected in sequence as an MTM classification model; a direction perception attention module extracts weight information of a space position through a direction perception space attention module, and a channel attention module is used for extracting weight information of a channel; an MTM classification model is used as a backbone network, direction perception attention modules and auxiliary branches which are connected in sequence are arranged after first M-1 feature coding stages of a convolutional neural network, and a self-distillation model is constructed; by combining a structural knowledge distillation strategy based on multi-stage feature fusion, a historical knowledge distillation strategy based on a linear growth mechanism and a category perception comparison learning strategy, multi-angle feature information interaction is fully utilized, multi-angle information collaborative optimization is realized, and the classification precision of the MTM classification model is effectively improved.

The invention provides an intelligent brain-like decision-making method, system and device with a body, and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: receiving a task instruction; analyzing the received task instruction based on a pre-trained large language model, outputting a task description semantic vector, and then carrying out multi-modal data acquisition on the environment to obtain multi-modal perception data; task description semantic vectors are supplemented based on the multi-modal perception data, and task description parameters are obtained; converting the task description parameter into a control signal based on a pre-constructed pulse neural network model, wherein the control signal comprises a plurality of pulse frequencies arranged according to the sequence of the action execution time; and executing corresponding actions according to the sequence of the plurality of pulse frequencies. In this way, the multi-modal perception data and the semantic vectors are combined in the large language model to generate the task description parameters to improve the perception precision of the environment, and the task description parameters of the large language model and the pulse sequence of the pulse neural network are combined to improve the decision response efficiency.