334 results about "Spiking neural network" patented technology

The invention provides a multi-modal heterogeneous model retrieval enhancement method and system, and the method comprises the steps: building a knowledge and application example double-corpus based on user multi-modal query, and designing a joint retrieval mechanism to obtain a result set; mapping and scheduling to obtain feature representation through special processing channels for texts, images and audios and a Spiking neural network with a segmented trapezoidal topological structure; constructing a three-stage cascade architecture of a basic model, an advanced model and human experts, and obtaining a decision path and answer candidate set in combination with a recursive and discarding decision mechanism; a Hamiltonian graph network is used for representing a multi-modal relation, and a gradient-free descent method is used for rapidly training and optimizing model parameters; an enhanced retrieval result is obtained through cross-modal semantic alignment and dynamic retrieval window adjustment; and high-quality response is obtained through context-aware sorting and retrieval enhanced reasoning. According to the method, the multi-modal information retrieval processing efficiency and the heterogeneous model reasoning response quality are improved.

A method is provided for processing data in a neural network system. The method includes receiving input data; processing the input data through a first set of neural network layers configured to perform data processing using MatMul-free techniques to produce intermediate data; further processing the intermediate data through a second set of neural network layers configured to simulate spiking neural network (SNN) functionalities using MatMul-free techniques; and outputting a result based on the processed data from the second set of neural network layers.

The invention is applied to the field of mechanical fault diagnosis signal processing, and particularly provides a pulse neural network mechanical fault diagnosis method based on STFT dimension transformation, and the method comprises the steps: collecting a one-dimensional mechanical vibration signal, carrying out the wavelet decomposition, carrying out the wavelet reconstruction of a low-frequency component and a denoised high-frequency component, and carrying out the wavelet reconstruction of the low-frequency component and the denoised high-frequency component; obtaining a denoised one-dimensional vibration signal; performing short-time Fourier transform, and converting the time-frequency two-dimensional matrix into a time-frequency two-dimensional matrix; inputting the time-frequency two-dimensional matrix into an improved HH threshold neuron model, carrying out Poisson sparse coding on the time-frequency two-dimensional matrix, and only carrying out pulse response on signal significant features; constructing a suprathreshold coding convolutional network with residual connection, inputting a sparse coding matrix, training by adopting an unsupervised learning rule based on STDP, and adaptively adjusting a network synaptic weight; and inputting to a trained above-threshold coding convolutional network, and obtaining pulse emission activity of neurons of an output layer through network forward propagation to determine a fault diagnosis result.

A hybrid neural network architecture is disclosed that integrates matrix multiplication-free (MatMul-free) transformation layers with spiking neural network (SNN) layers for efficient, low-power computation. The system includes an interface module configured to convert intermediate continuous-valued data from MatMul-free layers into a spike-compatible format using encoding techniques such as rate coding, phase coding, or threshold-based conversion. The SNN layers process the spike-encoded data in an event-driven manner, enabling sparse, temporal inference. Training is supported by a hybrid optimization strategy combining backpropagation in MatMul-free components with surrogate gradient descent or spike-timing-dependent plasticity (STDP) in SNN layers. The architecture reduces computational complexity, supports real-time adaptability, and enables deployment in energy-constrained environments such as edge devices and neuromorphic platforms. The system may be implemented in hardware, software, or a co-designed pipeline optimized for dynamic sensor data, control signals, or continuous inference tasks.

The invention relates to an edge end spiking neural network compression and deployment method and system, and belongs to the technical field of machine learning, the edge end spiking neural network compression and deployment method performs forward propagation on an initial spiking neural network model based on acquired multi-modal data, obtaining a memory use state in a forward propagation process based on hardware sensing initialization, and performing dynamic sparsification on the initial pulse neural network model based on the memory use state to construct a sparsified pulse neural network; based on the acquired activation statistical information of the sparse spiking neural network, performing dynamic pruning on the sparse spiking neural network by adopting a pruning strategy based on neuron activeness so as to realize compression of a spiking neural network model; the compressed spiking neural network model is optimized according to the edge end configuration, and the optimized spiking neural network model is deployed to the edge end to execute the reasoning task, so that the storage requirement and the calculation complexity are reduced.

The invention relates to the technical field of robot control, and discloses a spiking neural network-based robot control method, which comprises the following steps of: preprocessing collected multi-modal data to obtain an emotion pulse signal; inputting the emotion pulse signal into a pre-constructed emotion pulse neural network, and outputting a comprehensive emotion pulse; inputting the comprehensive emotion pulse into a central pattern generator, and outputting a behavior rhythm; acquiring an environment feedback signal generated by executing the behavior rhythm, and adjusting a connection weight according to the environment feedback signal; optimizing the comprehensive emotion pulse according to the connection weight, and converting the optimized comprehensive emotion pulse into emotion interaction voice information; and adjusting the emotion intensity according to the optimized comprehensive emotion pulse and the multi-modal data. According to the method, data are collected through the multi-mode sensor to generate emotion pulses, the emotion pulses are input into the central mode generator to generate behavior rhythms after SNN processing, weight optimization, emotional speech generation and emotional steady-state control setting are combined with the STDP algorithm, and the efficiency of a robot service scene is improved.

The invention relates to the technical field of machine vision and food quality detection, in particular to a food defect real-time detection method and system based on image processing. The method specifically comprises the following steps: integrating multi-modal data, and adaptively adjusting a weight coefficient to improve data quality; multi-stage noise reduction is carried out, key frames are extracted, and a key frame verification mechanism is enhanced through a 2FA control variable; constructing a YOLO-BioNet model, and optimizing the detection efficiency and precision in a complex scene through dynamic convolution kernel allocation and a spiking neural network; integrating Kalman filtering, LSTM (Long Short Term Memory) time sequence modeling and a deep reinforcement learning model, analyzing a food surface change track and predicting a defect state; real-time data and historical information are fused, and a dynamic threshold value is calculated through a Bayesian network; performing linkage response between an automatic sorting instruction and a production line; a distributed storage and graph neural network optimization model is adopted, and defect tracing is supported. The method has remarkable advantages in the aspects of real-time performance, detection accuracy and self-adaptive capability.

The invention relates to the technical field of artificial intelligence, and discloses an artificial limb driving method based on a spiking neural network, which comprises the following steps: acquiring a multi-modal signal, and preprocessing the multi-modal signal; converting the preprocessed multi-mode signal into a pulse signal; inputting the pulse signal into a pulse neural network for sensing fusion processing, and outputting a control strategy; dynamically adjusting a control strategy based on pulse reinforcement learning according to the behavior feedback information through a reward function, and generating a learning result; and generating a bionic motion instruction according to the adjusted control strategy and the learning result, and driving an artificial limb execution mechanism to act according to the bionic motion instruction. According to the method, the electromyographic signals, the tactile pressure signals and the inertial measurement data are uniformly coded into the pulse sequence, the pulse neural network is utilized to realize multi-modal sensing fusion, the pulse reinforcement learning algorithm is introduced, the artificial limb control strategy is optimized in real time according to user behavior feedback, and the accurate control of the artificial limb joint is improved.

The invention relates to a method for deploying a spiking neural network to a hardware device. The method includes providing the spiking neural network, training the spiking neural network to obtain a trained spiking neural network, mapping neurons and synapses in the trained spiking neural network to corresponding components of the hardware device, simulating deployment of the trained spiking neural network on a hardware device using the obtained mapping, and deploying the trained spiking neural network to the hardware device using the mapping and the simulation. The training, mapping, simulation and deployment steps are executed by using hardware information of the hardware device; wherein the hardware information comprises at least one of hardware resource constraints, hardware connection constraints, dynamic range of hardware design parameters, characterization or statistics of neurons and synapses, reconfigurability, programmability, yield, computing resources, temporal characteristics, constraints on pre-processing, interfaces and peripheral devices, and available encoders and / or decoders.

The invention discloses a spiking neural network deployment method and a deployment tool chain based on intermediate representation, and the method comprises the steps: carrying out the model analysis and structure mapping operation of a trained ANN-ONNX model, generating an SNN-ONNX model with a pulse characteristic, carrying out the optimization of a weight parameter through the combination of transfer learning and a precision fine tuning strategy, and carrying out the optimization of the weight parameter. Therefore, the expression ability and adaptability of the converted model are improved, and the reasoning execution performance of the SNN model on a target platform is improved by adopting an optimization means; meanwhile, an SNN-oriented modular operator component library is constructed, and the portability, maintainability and expandability of operators among different hardware platforms are enhanced by adopting an abstract interface and a design mode of specifically realizing decoupling. According to the method, efficient deployment of the SNN model on a target hardware platform is achieved, performance fidelity of the model under the function equivalent condition is ensured, and the method has wide engineering application prospects in the fields of edge calculation, low-power-consumption intelligent terminals, brain inspiration type artificial intelligence and the like.

The invention discloses a lightweight target trajectory prediction method based on a twin pulse neural network, and belongs to the field of target trajectory prediction. The method comprises the following steps: constructing a sample pair comprising a template frame and a search frame; constructing a twin spiking neural network model comprising a template branch, a search branch, an adaptive mask module, a spiking nerve Transform module and an output prediction layer, wherein the template branch and the search branch share weights; the adaptive mask module is used for aligning output features of two branches and generating a mask and feature enhancement, and the pulse nerve Transform module is used for acquiring a global dependency relationship of the enhanced features and outputting a current position and a tracking path end point of a target through an output prediction layer so as to generate a tracking plan path of the target; and training the constructed model based on the training data constructed by the sample pair to obtain a model used for obtaining the tracking plan path of the target. According to the method, key pulses are dynamically screened, the network structure is optimized, and double improvement of precision and efficiency is achieved.

The present invention discloses a node scale-adaptive neuron spike sorting method based on neuromorphic computing, and relates to the field of electroencephalogram signal spike sorting and decoding, the present invention proposes a spiking neural network framework comprising a two-layer spiking neural network and an attention neuron node, by incorporating prior knowledge of spike waveforms, this method automatically guides the addition and removal of network nodes to optimize computational resource allocation according to specific requirements, thereby minimizing hardware resource wastage. This method is characterized by low hardware overhead, high computational speed, and high consistency of results across different datasets. This method enhances the speed of spike sorting processes and shows potential for providing fully automated neuronal classification technology support for wireless implantable brain signal acquisition devices.

The invention relates to the technical field of neural networks, in particular to a visual perception method, device and equipment based on a pulse neural network and a medium, and adopts a visual perception system based on the pulse neural network, the method comprises the following steps: acquiring multi-source image data by using an input module, and preprocessing the multi-source image data to obtain a pre-processed image; obtaining an image tensor containing a time dimension; performing feature extraction on the image tensor by using an SNN-ResNet visual backbone network to obtain a target time sequence feature; and based on a preset sparse attention mechanism, a Transform perception output head is utilized to perform visual perception according to the target time sequence features, and a target detection and tracking result is obtained. Therefore, the problems that a traditional visual perception system is high in power consumption, insufficient in time sequence modeling capacity, low in reasoning speed and the like are solved, and efficient perception with low power consumption, high time dynamic modeling capacity and quick response is achieved.

The invention relates to the technical field of intelligent medical dispatching, in particular to a multi-terminal collaborative nursing worker resource intelligent allocation method, which comprises the following steps of: firstly, acquiring positioning, road, nursing worker physiology and old people demand data and generating multi-modal standardized data; constructing a three-dimensional digital twin potential field, predicting a corrected potential field by using photons, constructing a matching model by using the corrected potential field, a nursing worker capability vector and an old man demand vector, and performing annealing optimization to obtain initial matching; constructing a nursing worker-old person-time period tripartite graph based on a matching result, and obtaining optimal matching by adopting tension diffusion and gradient projection iteration; double digital signatures are executed on each piece of matching, and a non-homogeneous commitment is cast in the block chain, so that credible performance is realized; the wearing end spiking neural network continuously outputs fatigue probabilities, the fatigue probabilities are mapped into potential energy increments and written back to the potential field, and sub-potential field resolution and zero-knowledge post replacement are triggered and closed-loop updating is carried out. According to the method, the scheduling real-time performance and fairness are improved, the fatigue risk is reduced, and the whole service process is traceable.

The invention provides a cable testing method and a small handheld cable tester. The method comprises the following steps: carrying out wavelet transform noise reduction, Z-score standardization and time sequence dynamic regularization on original cable data; then constructing a geometric deep learning network by using a dispersed self-organizing structure of the rotating cube set, and extracting multi-modal depth features; fusing different modal features through a spiking neural network gating mechanism and a cross-modal attention mechanism; and finally, fault classification is carried out based on a differential evolution optimized neural network, and fault location is realized by combining a time sequence generative adversarial network and a dynamic probability neighborhood growth clustering algorithm. The system also evaluates the data contribution degree of each modal through information entropy and mutual information analysis, and optimizes the model performance by using adaptive weight distribution and lightweight neural network technologies. According to the invention, various fault types such as cable breakage, short circuit, insulation aging, poor contact and the like can be identified and positioned with high precision, and the efficiency and the accuracy of cable maintenance are remarkably improved.

The invention discloses a low-energy-efficiency crack detection method and device based on a spiking neural network, and belongs to the technical field of infrastructure intelligent detection. According to the method, details of an input image are enhanced through a super-resolution convolutional neural network, sparse pulse coding is carried out by using pulse neurons so as to greatly reduce calculation energy consumption, and a gated attention mechanism is fused with multi-scale features so as to improve model robustness. And finally, an accurate crack detection result is output through the decoder. According to the method, the problems of high energy consumption, poor environmental adaptability and limited computing resources when a traditional method is deployed on a mobile or embedded platform are effectively solved, and high-precision and strong-robustness real-time identification of cracks of scenes such as roads and tunnels on the premise of low power consumption is realized.

The invention provides a multi-mode man-machine interaction chip based on an adaptive threshold pulse neural network. The chip comprises a sensor module used for collecting input data of three modes of a visual signal, a pressure signal and a surface electromyography sEMG signal in real time; the acquisition module is used for performing analog-to-digital conversion and preprocessing on the pressure signal and the surface electromyography sEMG signal to generate a feature tensor; the recognition module comprises three adaptive thresholds SNN converted by a convolutional neural network CNN, respectively processes feature tensors of three modals, performs cross-modal feature fusion, and outputs a motion intention of the robot; the self-adaptive threshold SNN reduces the power consumption while maintaining the calculation precision by dynamically adjusting the spiking neuron membrane potential threshold. According to the method, feature fusion is carried out through input data of three modes in combination with the self-adaptive threshold SNN algorithm, the motion intention of an operator and the environment interaction state can be comprehensively captured, the accuracy and adaptability of robot motion generation are remarkably improved, and the naturalness and reliability of man-machine cooperation are enhanced.

The invention discloses a neuromorphic data classification method based on a gating space-time self-attention mechanism and a spiking neural network, and the method comprises the following steps: 1, selecting a public neuromorphic data set, carrying out the data preprocessing, and dividing the data into a training set, a verification set and a test set; 2, constructing a pulse neural network model to perform neuromorphic data classification, and introducing a gating space-time self-attention mechanism into the model to adaptively capture a global dependency relationship of space and time dimensions and enhance the space-time information representation capability of the model; and 3, training the constructed spiking neural network model by using the training set and the verification set, testing the trained model by using the samples in the preprocessed test set after training is completed, and outputting a prediction classification result of the samples by the model. According to the method, the representation capability of the spiking neural network in spatial-temporal feature modeling is remarkably enhanced through a gating spatial-temporal self-attention mechanism, and the precision of a neuromorphic data classification task is improved.

Koopman operator theory is a widely used method to analyze, control, and predict the behavior of the states of a non-linear dynamical system using measurement functions in Hilbert space. Real time approximation of the Koopman operator is crucial in order to adapt and understand behavior of underlying non-linear dynamical system. Traditional approaches leverage matrix-based methods or artificial neural networks to approximate Koopman operator. However, such methods necessitate significant power and computational resources, therefore may not be suitable for applications that require real-time on-board processing. The problems of the conventional approaches are resolved based on a recent development of brain inspired spiking neural networks and neuromorphic computing platforms, as these offer extremely low-energy computation and real-time responses. Embodiments of the present disclosure provide implementation of a Spiking Neural Network (SNN) based architecture that efficiently approximate Koopman operator with minimal length of data and demonstrates significant computational savings.

The invention relates to a high-efficiency three-dimensional point cloud classification method and device based on a pulse neural network, and the method obtains an event-driven point cloud expression form through sparse pulse voxel coding, and achieves the efficient extraction of local features through the combination of a pulse-driven sparse convolution structure. And a pulse self-attention module driven by logical operation is utilized to construct global correlation characteristics, and finally, category judgment is completed through a pulse time integration mechanism, so that the calculation amount and energy consumption are remarkably reduced while the classification precision is kept. According to the method, sparse activation and event driving characteristics of the pulse neural network are fully utilized, efficient processing of the three-dimensional point cloud is achieved, and the method is suitable for automatic driving, robots and embedded scenes sensitive to power consumption.

The invention discloses an efficient deployment method and device of an artificial neural network facing a brain-like chip. The efficient deployment method comprises the steps of pulse transmission between layers of the quantized artificial neural network, sparse coding of pulses and compatibility design based on the storage function of the brain-like chip. Generally, a network running on a brain-like chip is a spiking neural network, and the method is a brand-new method. The method comprises the following steps of: pulsing a quantized artificial neural network activation value so that the converted network can be subjected to high-precision and low-delay reasoning; according to the method, a neuron behavior equivalent to quantitative reasoning of an artificial neural network is provided, the network is successfully deployed on a brain-like chip, and discreteness of the quantitative network and event-driven characteristics of the brain-like chip are combined; in addition, limited by the precision of different hardware, the invention provides a method for scaling an operation intermediate result to avoid an overflow risk.

The invention relates to the technical field of crop disease and insect pest monitoring, and discloses a crop disease and insect pest monitoring system and method.The crop disease and insect pest monitoring method comprises the steps that event streams in candidate areas are coded into space-time pulse sequences; identifying a spatial-temporal dynamic mode representing early-stage micro-scab formation and expansion through a layered spiking neural network; according to the output of the spiking neural network, extracting growth trend parameters of micro disease spots; and generating an extremely early warning signal based on the growth trend parameter and a historical disease development model. By adopting a mode of combining event-driven asynchronous sensing and spiking nerve calculation, the power consumption of the monitoring system is effectively reduced, long-term autonomous operation in a resource-constrained environment is realized, and meanwhile, extremely early weak visual signals of diseases, which are difficult to identify by a traditional method, can be captured and analyzed; and technical support is provided for ultra-early warning and precise prevention and control of crop diseases and insect pests.

The invention discloses an arrhythmia detection method based on a pulse neural network. The method comprises the following steps: performing incremental modulation pulse coding and pooling operation on a target electrocardiosignal to obtain a corresponding pulse sequence; and inputting the pulse sequence into a trained multi-level pulse neural network classification model to obtain an arrhythmia detection result. Wherein the multi-level pulse neural network classification model comprises a weight sharing layer and a plurality of cascaded classifiers, the weight sharing layer is used for extracting electrocardiosignal feature information from the pulse sequence, and the plurality of classifiers are used for obtaining an arrhythmia detection result based on the electrocardiosignal feature information. According to the method, on the premise that the accuracy is equivalent, the classification result is more refined, and the requirements for hardware storage resources and computing resources are reduced.

The embodiment of the invention relates to the technical field of artificial intelligence, and discloses an image analysis method and device based on a pulse neural network, computer equipment and a computer readable storage medium, and the method comprises the steps: obtaining a to-be-analyzed image; inputting the to-be-analyzed image into a pulse neural network based on suppression neurons; the neural network comprises an SNN module based on convolution, an SNN module based on Transform, a pulse self-feedback suppression module and a reasoning module. The suppression pulse self-feedback module comprises at least one suppression neuron and is used for dynamically issuing suppression pulses to adjust the membrane potential state of surrounding excitation neurons; neurons of the pulse neural network carry out signal transmission through integer pulses. The neurons comprise an SNN module based on convolution, and excitation neurons and inhibition neurons in the SNN module based on Transform; and outputting an analysis result. By means of the mode, the model prediction precision is improved, and computing resources are remarkably reduced.

The invention relates to the technical field of computer image processing, and particularly provides a space-frequency domain pedestrian gait feature extraction method based on spiking neural network driving, and the method comprises the steps: carrying out the gait feature extraction of a gait image through employing a space-frequency domain pedestrian gait feature extraction model; the space-frequency domain pedestrian gait feature extraction model comprises a backbone network, a multi-stage time domain global feature interaction enhancement module and a distinguishing feature aggregation module; the backbone network comprises N stages which are connected in sequence, and each stage comprises a space-frequency feature fusion module for fusing space-domain features and frequency-domain features; a multi-stage time domain global feature interaction enhancement module extracts and enhances spatial domain global features and time domain global features; and the distinguishing feature aggregation module aggregates the output of the backbone network and the output of the multi-stage time domain global feature interaction enhancement module. According to the method, the capturing capability of the model on gait micro-detail features can be improved, and the adaptability to complex scenes is enhanced.

The invention belongs to the technical field of computer vision and neuromorphic computing. The invention provides a multi-scale dynamic fusion target detection method and system based on a spiking neural network. According to the embodiment of the invention, the network adopts a trainable integer LIF neuron, the SABC module adopts a dynamic channel expansion mechanism, the problem of deep feature degradation caused by pulse signal time sequence sparsity is effectively solved, the SMGFDH realizes multi-scale feature fusion of pulse distribution rate self-adaption through a differential attention mechanism, and the accuracy of the feature fusion is improved. And redundant convolution operation before traditional splicing is innovatively canceled, and an efficient detection framework completely adaptive to pulse sparsity is constructed. The problems of insufficient deep feature expression and low multi-scale fusion efficiency of the existing SNN in a complex visual task are solved.

The invention provides a brain-like calculation compiling method, device and equipment for a multi-chip heterogeneous system, and the method comprises the steps: describing a spiking neural network model based on a pre-defined spiking neural network dialect, and obtaining the dialect description of the spiking neural network model; analyzing the dialect description of the spiking neural network model to obtain key information of the spiking neural network model, optimizing the spiking neural network model based on the key information to obtain a target spiking neural network model, and determining an intermediate representation of the target spiking neural network model; and converting the intermediate representation of the target pulse neural network model to obtain instruction data matched with the multi-chip heterogeneous system. According to the method, the spiking neural network model is converted into the unified intermediate representation, so that the workload of brain-like calculation compiling can be reduced, the compiling complexity is reduced, and the execution efficiency and performance of brain-like calculation on a multi-chip heterogeneous system are improved.

This invention discloses a real-time voice interaction method and system for robots based on a spiking neural network. The method involves acquiring a voice dataset and superimposing it with robot operating noise to generate augmented samples. Voice signal features are extracted from these samples and data augmentation is performed. A recurrent spiking neural network model with loop connections is constructed and trained using the augmented data for speech recognition. Real-time voice stream input is acquired from the robot, features are extracted, and input into the trained recurrent spiking neural network model for inference. Based on the pulse decoding results, the robot is controlled to perform interactive actions. This invention reduces recognition power consumption, mitigates the impact of robot operating noise on the accuracy of real-time voice interaction, avoids the problem of no-pulse spiking neurons, and allows the system to adjust the sensitivity of voice interaction commands during operation.

The invention discloses a satellite Internet of Things uplink signal detection method and a satellite Internet of Things uplink signal detection system, which integrate traditional signal processing and advanced spiking neural network technologies, realize high-precision frame synchronization by performing two-dimensional cross-correlation matching in a time-frequency domain, and perform low-power-consumption and high-robustness symbol judgment by using an event-driven spiking neural network. Compared with a traditional method based on FFT or matched filtering, the method has higher adaptability and detection performance in a satellite Internet of Things scene with large Doppler frequency shift, low signal-to-noise ratio and dense users.

This invention belongs to the field of unmanned aerial vehicle (UAV) control technology, and particularly relates to a feedforward control method for multi-rotor UAVs based on a dynamic cascaded spiking neural network (RCN). The method includes: S1: constructing a network model of the RCN based on pulse signals; S2: constructing a cost function of the RCN based on the network model and pulse errors; S3: solving for the weights of the RCN; S4: setting a preset similarity threshold and determining the learning rules for the dynamic cascaded structure based on the preset similarity threshold; S5: obtaining the output signal of the RCN according to the weights and the learning rules of the dynamic cascaded structure, and implementing feedforward control of the multi-rotor UAV based on the output signal. This invention improves the adaptability and robustness of multi-rotor UAVs in complex flight environments.