Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

546 results about "Neural processing" patented technology

Neural processing originally referred to the way the brain works, but the term is more typically used to describe a computer architecture that mimics that biological function. In computers, neural processing gives software the ability to adapt to changing situations and to improve its function as more information becomes available.

Graph neural network execution on neural processing unit

Workloads for executing a graph neural network (GNN) may be divided among various processing units, such as a central processing unit (CPU) and a neural processing unit (NPU). The NPU may include a data processing unit (DPU) and a digital signal processor (DSP). The CPU may perform precomputation, model optimization, hardware optimization, and compilation. For example, the CPU may precompute a parameter matrix and use the parameter matrix as internal parameters of a GNN. The CPU may also perform node padding, approximation computation, or transfer of DSP operations to DPU to optimize the GNN. The CPU may also perform sparsity data compute and storage, vertical fusion of DSP operations and DPU operations, or data quantization to optimize performance of the NPU. The compiled GNN may be provided to the NPU, and the DPU and DSP may perform the operations in the compiled GNN to produce a prediction of the GNN.
Owner:INTEL CORP

Neuro-Generative Adversarial System for real-time detection and combating of malware morphing in high-density edge networks

A system for real-time detection and mitigation of morphing malware in high-density edge networks, consisting of: a data acquisition unit configured to receive, normalize, and encode multimodal telemetry data streams originating from at least one of the following domains: network traffic, process behavior, system call sequences, binary instruction traces, and control flow graphs; the data acquisition unit is further configured to compute feature embeddings over sliding time windows and apply privacy-preserving redactions prior to storage; a generative neural processor that is operationally coupled to the data acquisition unit and configured to generate synthetic morphing malware variants by learning probabilistic transformations of previously observed malicious data representations, maintaining semantic functionality while varying structural and behavioral features; a discriminative neural processor trained adversarially with the generative neural processor, wherein the discriminative neural processor is configured to detect morphing malware by evaluating a probability distribution over multimodal telemetry embeddings and classifying anomalous process and flow behaviors in real time; a coordination processor that is communicatively connected to both the generative neural processor and the discriminative neural processor and is configured to orchestrate adversarial co-training, regulate detection thresholds, calculate reinforcement-based penalties for false negative results, and trigger countermeasures as soon as a detection confidence level exceeds a predefined adaptive threshold; a secure, system-integrated inference and enforcement unit configured to perform low-latency countermeasures at the network edge, including selective packet filtering, flow isolation, process interruption, or system microsegmentation, based on instructions from the coordinating processor; and a hardware-embedded security enclave that is embedded in the system and configured to store cryptographic keys, neural model parameters, and integrity affirmation data to ensure the confidentiality, authenticity, and immutability of model artifacts and policy configurations.
Owner:ANAJAVADIDHODDI RAMACHANDRA NAIK CHAYAPATHI BENGALURU +7

Multimodal large language model with audio trigger

Systems and methods to trigger LLM inference based on the presences of relevant audio, such as a keyword or sound event of interest. A detection head receives acoustic embeddings from an audio encoder and determines whether the audio stream includes relevant sounds (e.g., a selected audio trigger). When the audio stream does not include relevant sounds, multimodal LLM inference is bypassed, thereby saving power and protecting privacy. When relevant sounds are detected in the audio stream by the detector, the acoustic embeddings from the audio encoder are transmitted to the multimodal LLM, which proceeds to perform inference on the acoustic embeddings. The audio encoder and / or detection head can be offloaded in the hardware and implemented before the multimodal LLM in the hardware pipeline, while the multimodal LLM can be implemented in a neural processing unit.
Owner:INTEL CORP

Controlling signal strengths in analog, in-memory compute units having crossbar array structures

Precision of a neural processing apparatus comprising two in-memory compute (IMC) units is controlled, wherein the IMC units include a first IMC unit and a second IMC unit, each designed to perform vector-matrix multiplication (VMM) to produce analog output signals. An artificial neural network (ANN) model is trained to learn its parameters (including synaptic weight values) in accordance with a dual objective. The ANN model comprises two neural layers, these including a first neural layer and a second neural layer. The method further comprises storing the synaptic weight values of the parameters learned in the two IMC units to respectively map the first neural layer and the second neural layer onto the first IMC unit and the second IMC unit. The second IMC unit is designed to perform VMM operations based on analog input signals generated from activation values produced by the first neural layer.
Owner:INTERNATIONAL BUSINESS MACHINE CORPORATION

Discrete three-dimensional processor

A discrete three-dimensional (3-D) processor comprises stacked first and second dice. The first die comprises three-dimensional memory (3D-M) arrays, whereas the second die comprises at least a portion of a logic / processing circuit and an off-die peripheral-circuit component of the 3D-M array(s). The preferred 3-D processor can be used to compute non-arithmetic function / model. In other applications, the preferred 3-D processor may also be a 3-D configurable computing array, a 3-D pattern processor, or a 3-D neuro-processor.
Owner:HONG KONG HAICUN TECHNOLOGY CO LTD

Offload multi-dependent machine learning inferences from a central processing unit

An information handling system includes a central processing unit, a neural processing unit, and an offload module. The offload module receives an inference container including multiple inference models and metadata associated with the inference models. Based on the metadata, the offload module determines whether a quality of service for the inference models may be met by the neural processing unit. In response to the quality of service being met in the neural processing unit, the neural processing unit executes the inference models. In response to the quality of service not being met in the neural processing unit, the central processing unit executes the inference models.
Owner:DELL PROD LP

Systems and methods for heterogeneous large language model encoder and decoder processing

Systems and methods are disclosed for efficient memory allocation for processing large language model encoders and decoders based on an attention model. The system can utilize a plurality of two types of processors suitable for different types of LLM processing. These include Neural Processor Units and Graphic Processing Units. Each NPU processor has dedicated DDR memory coupled to each NPU. The DDR memory caches the neural network weights used in the generation of neural network activations. A plurality of GPUs provides KVQ token processing. LLM tokens processing can be performed in parallel batches or sub-batches to utilize idle NPU processors within the neural network. In some embodiments, the NPUs are structured in a matrix with a bus between adjacent processors. In another embodiment, NPUs provide both KVQ processing and neural network processing. The system can be integrated on a silicon substrate using chiplets in a 2.5 or 3-D architecture.
Owner:EXPEDERA INC

System integrated machine-learning co-processing

The technology described adds a ML inference to the output of an image signal processor (ISP) associated with a camera. The combined image and ML inference may be described herein as an augmented image. Once generated, the augmented image may be communicated to other components of a computing system associated with the camera and / or ISP. The initial inference may be generated by a neural processing unit (NPU) associated with the ISP. The ISP may communicate a generated image to the NPU prior to communicating the image to a computing system. In an aspect, the NPU inference is combined with the image using image steganography. Once communicated from the camera to the computing device, the augmented image may be separated into a base image and inference by a camera driver or other component associated with the image management.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Intelligent device with artificial intelligence for monitoring neuropathic pain

An intelligent AI-based device for monitoring neuropathic pain, consisting of: a flexible, wearable structure made of biocompatible materials, designed for continuous wear on the user's body, a sensor array integrated into the structure, configured to capture physiological data from the user's body, wherein the sensor array includes at least one electrodermal activity (EDA) sensor for measuring skin conductance, one electromyography (EMG) sensor for detecting muscle activity, one heart rate variability (HRV) monitor for evaluating autonomic nervous system responses, and one thermal sensor for measuring skin temperature; a neural processing unit (NPU) housed in the portable structure, the NPU being configured to analyze multidimensional sensor data to detect patterns indicative of the intensity and nature of neuropathic pain; a communication module within the portable structure that includes Bluetooth Low Energy (BLE) and Wi-Fi connectivity options to enable seamless transfer of processed data to an external device or cloud-based platform; an energy management system, including a rechargeable battery integrated into the structure and configured to power the sensor array, the NPU, and the communication module; and a neuromodulation actuator embedded in the portable structure, configured for transcutaneous electrical nerve stimulation (TENS), whereby the actuator enables on-demand therapeutic intervention based on the detected pain intensity and pattern.
Owner:AL JAOUNI SOAD KHALIL PROF DR +11

Bypass chiplets for memory-logic stack

An integrated circuit (IC) layer stack is disclosed that integrates bypass chiplets in the IC layer stack without requiring fabrication of through-silicon vias (TSV) in a digital device for signals and power to a memory stack located above the digital device. Not requiring adding TSVs to an existing design high-performance digital device significantly reduces and / or eliminates digital device modification costs and time to manufacture new products consisting of three-dimensional stacked memory and digital device organizations. The bypass chiplets may be passive with just through conductors in silicon, or active with logic circuits therein. Flexibility of placement of the bypass chiplets in the IC layer stack in combination with an active interposer provides for many possible configurations of IC layer stacks. The digital device may be a microcontroller, a microprocessor, a PLA, an ASIC, a DSP, a GPU, a FPGA, neural processing unit, tensor processing unit, or any combination thereof.
Owner:ADVANCED MICRO DEVICES INC

Neural processing unit including post-processing unit

According to one example of the present disclosure, the neural processing unit may comprise a processing element array configured to perform operations of a neural network model and a post-processing unit configured to process data output from the processing element array. The post-processing unit includes a first computation circuit that extracts a subset of classes for each bounding box by comparing class scores of classes and a second computation circuit configured to extract one or more bounding boxes by comparing a class confidence score of each bounding box with a threshold confidence score.
Owner:DEEPX CO LTD

Edge device with built-in compiler for neural network models

A system includes a substrate on which a first memory, a neural processing unit (NPU) including a plurality of processing elements (PEs) with multiplier-accumulator circuits, a controller, and a second memory, and a central processing unit (CPU) are disposed. The CPU may be configured to execute a universal compiler to perform a conversion for a particular neural network model into a machine code executable by the NPU and store the machine code in the first memory or the second memory. When the particular neural network model, generated by one among a plurality of machine learning frameworks that are incompatible with each other, is received and stored in the first memory, the universal compiler may perform the conversion based on mapping information indicating mapping between elements of machine learning frameworks and functions or operations executable by the CPU or NPU.
Owner:DEEPX CO LTD

Reconfigurable heterogeneous radar data calculation module and calculation device

The invention discloses a reconfigurable heterogeneous radar data computing module and computing device, and relates to the technical field of computing devices, the reconfigurable heterogeneous radar data computing module comprises a ZYNQ processor, a digital signal processor (DSP) and a neural processing unit (NPU), the ZYNQ processor is connected with the DSP and the NPU; the ZYNQ processor comprises an ARM processor and an FPGA (Field Programmable Gate Array) logic resource; the FPGA logic resource is used for carrying out parallel class algorithm processing on the radar data; the DSP is used for carrying out serial algorithm processing on the radar data; the NPU is used for performing artificial intelligence algorithm processing on the radar data; the ARM processor is used for scheduling FPGA logic resources according to the calculation tasks, and the DSP and the NPU execute corresponding algorithms. According to the reconfigurable heterogeneous radar data calculation module, the high-performance calculation requirement can be met.
Owner:BEIJING DONGYUAN RUNXING TECH CO LTD

Updating of parameters of neural network model for efficient execution on neural processing unit

Embodiments relate to converting functions or function call instructions of a first neural network (NN) model into graph module. The relationship between one or more inputs and one or more outputs of the graph modules are analyzed. A second neural network (NN) model in a form of a directed acyclic graph (DAG) including using the graph modules is generated by mapping inputs and outputs of the graph modules based on the relationship. Markers are added to the graph modules in the second NN model. First calibration data is generated by collecting input values and output values of each of the graph modules using the markers. An adjustment value for outlier alleviation for each of the graph modules is generated based on the first calibration data. For each graph module of the second NN model, an input parameter and a weight parameter are updated based on the adjustment value.
Owner:DEEPX CO LTD

Neural processing unit that reuses feature maps and its operation method

A method of reusing feature maps that is operable on a neural processing unit (NPU) includes calculating, for a first layer of at least one artificial neural network (ANN) model, a space cost based on sizes of an input feature map, an output feature map, and a weight of a subsequent layer of the first layer; calculating a caching value for an operation of the first layer; determining a caching profit for the operation of the first layer based on the space cost and the caching value; determining a caching entry having maximum caching profit among caching candidate entries; and storing the caching entry in a variable memory. A system including the NPU includes a main memory to store a portion of the ANN model; a variable memory to selectively store a caching entry of the portion of the ANN model; and a controller to perform the method.
Owner:DEEPX CO LTD

Server cable management method and device, equipment and storage medium

The invention discloses a server cable management method and device, equipment and a storage medium, and relates to the technical field of fault detection, and the method comprises the steps: constructing a cable connection relation graph of a target cable of a target server based on a preset graph neural network; the target cable is a cable embedded in the controller and the target sensor in advance, and a neural processing unit is integrated in the controller; generating a cable code of the target cable, and determining a cable state of the target cable through the target sensor by using the controller; when the cable state is abnormal, determining a fault path of the target server according to the cable code and the cable connection relation graph; and displaying the fault path by using a positioning display device of the target cable so as to manage the target server. The controller and the sensor are implanted into the cable body, the cable connection relation graph is constructed in combination with the neural network model, cable troubleshooting and life management are further achieved through cable coding, the troubleshooting cost is greatly reduced, and the intelligent management process of a server is improved.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Device and method for providing information using on-device ai recognition and server interworking

A mobile artificial intelligence (AI) device includes a camera configured to capture a video of a product, and a neural processing unit (NPU) configured to perform inference with an AI recognition model to recognize product information from the captured video on the mobile AI device. A battery supplies power to the NPU. A transceiver is configured to transmit only the recognized product information to a remote server without transmitting the captured video, and to receive additional commercial information corresponding to the recognized product information. A display is configured to present the additional commercial information.
Owner:DEEPX CO LTD

Dynamic assignment of bus bandwidth for sending tensors to neural processing units

According to one example of the present disclosure, a system may be provided. The system may comprise at least one processing core configured to process computations of the at least one neural network model comprising at least one tensor, at least one memory circuit configured to store the at least one tensor, a bus circuit, electrically coupled to the at least one processing core and the at least one memory circuit, configured to transmit the at least one tensor based on a memory access operation instruction, and a controller configured to control a priority of a memory access operation for each tensor of the at least one processing core.
Owner:DEEPX CO LTD

Accelerating artificial neural network using hardware-implemented lookup table

The invention is particularly directed to a hardware system (1) designed for implementing an artificial neural network (ANN). The hardware system substantially includes a neural processing device (15) (e.g., involving a crossbar array structure), one or more lookup table circuits (17), and one or more processing units (18). The neural processing device is configured to implement M artificial neurons, where M > = 1. The lookup table circuit is configured to implement a lookup table (LUT). The system also includes M'processing units, where M > = M '> = 1. Each processing unit is connected by at least one neuron so as to be able to access a first value output by each connected neuron. In addition, each processing unit is connected to the LUT circuitry so as to efficiently access parameter values of the parameter set from the LUT. Finally, each processing unit is configured to output a second value corresponding to a value of a mathematical function with the first value as an argument. The mathematical function is additionally determined by a set of parameters whose parameter values are accessed in operation from the LUT by each processing unit. That is, the mathematical function is defined (and thus determined) by a set of parameters, the values of which can be efficiently retrieved from a hardware-implemented LUT. This results in a significant acceleration of the calculation of the function output beyond the acceleration that may have been achieved within the neural processing devices and processing units themselves. Thus, neuron outputs can be processed more efficiently before passing to the next neuron layer. The invention is also directed to a method of operating such a hardware system.
Owner:AXELERA AI BV

Consciousness perception quantum gravitational nerve acceleration unified computing platform and method based on formalized verification

The invention discloses an awareness perception quantum gravitational nerve acceleration unified computing platform and method based on formalized verification, and belongs to the field of high-performance computing, medical AI and quantum information processing. A quantum gravitational coupling neural processing unit (QGCNPU); a consciousness information integral calculation core (phi ICC); formalizing a receipt chain verifier (FRCV); and the mirror reflection symmetry correction module (MRSCM) is used for eliminating the continuous projection mistake and the discrete Boolean mistake through topological equivalence verification of a local observer and a global GodCam visual angle. The system adopts an FPGA + ASIC + quantum processor three-layer heterogeneous architecture, and supports real-time pathological modeling (precision gt; gt) of neurodegenerative diseases such as Alzheimer's disease, Parkinson's disease and ALS; 99.7%), drug target discovery (acceleration by 50 times) and consciousness state monitoring (time resolution lt; and meanwhile, the mathematical proving performance and the clinical auditing performance of the calculation process are ensured, and the requirements of the global 152 billion dollar neural science and technology market and the 89 billion dollar quantum calculation industry are met.
Owner:GUANGZHOU KINGPIN IND CO LTD

System for processing heterogeneous generative artificial intelligence models

In accordance with the present disclosure, an apparatus is provided. The apparatus includes: a first memory having a first capacity configured to store a first generative neural network model including a first parameter; and a first neural processing unit configured to generate a response corresponding to the input query using the first generative neural network model stored in the first memory; and wherein the first neural processing unit may be configured to store first execution code of the first generative neural network model, the first execution code being compiled to process the speculative decoding.
Owner:DEEPX CO LTD

Cloud AI-based system for real-time language learning and language improvement

A cloud-integrated, AI-powered smart speaker for language learning, which includes the following: A microphone array for speech capture; ◯ an on-device AI module with a Neural Processing Unit (NPU) or Tensor Processing Unit (TPU) for real-time speech analysis; • Wi-Fi and 5G connectivity for secure data synchronization with cloud servers; ◯ a cloud-based AI engine that uses serverless computing services to trigger AI models for advanced language assessment and adaptive learning; ◯ a cloud storage system with relational databases for structured data, such as user profiles and progress metrics, and object storage for unstructured data, such as audio recordings, secured with AES-256 encryption and RBAC for data protection; ◯ a true-to-life speaker for AI-generated speech output; ◯ a screen (OLED / E-Ink) for text corrections and real-time lesson guidance; ◯ Touch and gesture sensors for user interaction; ◯ A battery module with USB-C charging function for portability.
Owner:RUPAREL HARDIK DHARMESH SAN JOSE

Accelerating artificial neural networks using hardware-implemented lookup tables

The invention is notably directed to a hardware system (1) designed to implement an artificial neural network (ANN). The hardware system basically includes a neural processing apparatus (15), e.g., involving as crossbar array structure, one or more lookup table circuits (17), and one or more processing units (18). The neural processing apparatus is configured to implement M artificial neurons, where M≥1. The lookup table circuits are configured to implement a lookup table (LUT). The system further includes M′ processing units, where M≥M′≥1. Each processing unit is connected by at least one neuron, in order to be able to access a first value outputted by each connected neuron. In addition, each processing unit is connected to a LUT circuit, in order to efficiently access parameter values of a set of parameters from the LUT. Finally, each processing unit is configured to output a second value, corresponding to a value of a mathematical function taking said first value as argument. The mathematical function is otherwise determined by the set of parameters, the parameter values of which are accessed by each processing unit from the LUT, in operation. I.e., the mathematical function is defined (and thus determined) by a set of parameters, the values of which are efficiently retrieved from the hardware-implemented LUT. This results in a substantial acceleration of the computations of the function outputs, beyond the acceleration that may already be achieved within the neural processing apparatus and the processing units themselves. As a result, the neuron outputs can be more efficiently processed, prior to being passed to a next neuron layer. The invention is further directed to a method of operating such a hardware system.
Owner:AXELERA AI BV

Nonlinear tensor compression and decompression for neural networks

Devices and techniques are generally described for nonlinear tensor compression for neural networks. In various examples, a first tensor associated with a first layer of a neural network may be determined. One or more neural processing units of accelerator hardware may generate a first compressed tensor by applying a nonlinear compression function to the first tensor. The first compressed tensor may be stored in a first memory of the one or more computer-readable media. A first operation associated with a second layer of the neural network may be determined, where the first operation uses output of the first layer. The first operation may be performed based on the first compressed tensor.
Owner:AMAZON TECH INC

Dock-based neural processing unit handoff to client computing device on disconnect of the client computing device

A docking station includes a processor and a memory coupled to the processor. The docking station may be configured to establish a sideband connection between an information handling system and the docking station subsequent to detecting the information handling system docking at the docking station. The docking station may also execute a workload of the information handling system, subsequent to the establishment of the sideband connection. In addition, the docking station may transmit a notification to resume the workload via the sideband connection subsequent to detecting that the information handling system is undocked from the docking station while performing the executing of the workload.
Owner:DELL PROD LP

Methods And Apparatus For Managing Weight Data Accesses For Neural Network Processors

Artificial intelligence is an increasingly important sector of the computer industry. However, artificial intelligence is extremely computationally intensive field such that it can be expensive, time consuming, and energy consuming. Fortunately, many of the calculations required for artificial intelligence can be performed in parallel such that specialized processors can great increase computational performance. Specifically, artificial intelligence generally requires large numbers of matrix operations to implement neural networks such that specialized matrix processor circuits can improve performance. To perform all these matrix operations, the neural processing circuits must be quickly and efficiently supplied with data to process or else the matrix processor circuits end up idle or spending large amounts of time loading weight matrix data. Thus, this document discloses apparatus and methods for efficiently operating external interfaces to storage on matrix processor circuits that minimize latency and maximize interface utilization when accessing weight matrix data.
Owner:EXPEDERA INC

Apparatus, method, and system for deploying neural network model

A method may comprise receiving a first neural network (NN) model including one or more functions; generating a second NN model in a form of directed acyclic graph (DAG) including one or more graph modules by converting the one or more functions; calculating one or more scale values by obtaining maximum and minimum values of parameters input to the one or more graph modules; updating the parameters based on the one or more scale values; and generating a third NN model, in a form of machine code executable on a particular neural processing unit, including the updated parameters.
Owner:DEEPX CO LTD

Processing heterogeneous generative artificial intelligence models

According to the present disclosure, a device is provided. The device includes a first memory of a first capacity configured to store a first generative neural network model comprising a first parameters, and a first neural processing unit configured to generate a response corresponding to an input query utilizing the first generative neural network model stored in the first memory, and wherein the first neural processing unit may be configured to store a first execution code of the first generative neural network model compiled to process speculative decoding.
Owner:DEEPX CO LTD

Peer Configuration of Compute Resources Among Data Storage Devices

Example storage systems, storage devices, and methods provide peer configuration of hardware compute resources among peer storage devices. Data storage devices may include hardware circuits, such as graphics processor units, media processor circuits, compute accelerator circuits, and neural processing units, configured for compute operations targeting host data associated with that storage device. One of the storage devices is configured to act as a master storage device for determining firmware configurations for the other storage devices and sending an indication of the firmware image to be used for some operating period. The storage device receiving the firmware image loads the firmware image for the hardware circuit and reboots the hardware circuit to configure it with the firmware image.
Owner:SANDISK TECHNOLOGIES LLC

Heterogeneous nerve processing system of generative AI model

A heterogeneous neural processing system includes a first processor configured to perform encoding and decoding operations of an autoencoder, and a second processor configured to perform neural network operations of a particular task, the operations having iterative processing. The processors perform compute tasks through synchronous data exchange to implement a generative AI model. The first processor processes a feature map divided into data rows, caches the data into active memory using row-based depth-first scheduling, and selects deeper operations in the network hierarchy when processing branch inputs, outputs, and residual connections. And H reuses boundary pixels between the cache storage space segments, so that convolution and element-level operation can be executed simultaneously. A neural network adjusts the device analysis model to identify layer dependencies, applies search space constraints, performs an iterative search to generate a fusion plan, and selects an optimal plan based on external memory access and execution delays.
Owner:MEDIATEK INC