382 results about "Neural processing" patented technology

Workloads for executing a graph neural network (GNN) may be divided among various processing units, such as a central processing unit (CPU) and a neural processing unit (NPU). The NPU may include a data processing unit (DPU) and a digital signal processor (DSP). The CPU may perform precomputation, model optimization, hardware optimization, and compilation. For example, the CPU may precompute a parameter matrix and use the parameter matrix as internal parameters of a GNN. The CPU may also perform node padding, approximation computation, or transfer of DSP operations to DPU to optimize the GNN. The CPU may also perform sparsity data compute and storage, vertical fusion of DSP operations and DPU operations, or data quantization to optimize performance of the NPU. The compiled GNN may be provided to the NPU, and the DPU and DSP may perform the operations in the compiled GNN to produce a prediction of the GNN.

A system for real-time detection and mitigation of morphing malware in high-density edge networks, consisting of: a data acquisition unit configured to receive, normalize, and encode multimodal telemetry data streams originating from at least one of the following domains: network traffic, process behavior, system call sequences, binary instruction traces, and control flow graphs; the data acquisition unit is further configured to compute feature embeddings over sliding time windows and apply privacy-preserving redactions prior to storage; a generative neural processor that is operationally coupled to the data acquisition unit and configured to generate synthetic morphing malware variants by learning probabilistic transformations of previously observed malicious data representations, maintaining semantic functionality while varying structural and behavioral features; a discriminative neural processor trained adversarially with the generative neural processor, wherein the discriminative neural processor is configured to detect morphing malware by evaluating a probability distribution over multimodal telemetry embeddings and classifying anomalous process and flow behaviors in real time; a coordination processor that is communicatively connected to both the generative neural processor and the discriminative neural processor and is configured to orchestrate adversarial co-training, regulate detection thresholds, calculate reinforcement-based penalties for false negative results, and trigger countermeasures as soon as a detection confidence level exceeds a predefined adaptive threshold; a secure, system-integrated inference and enforcement unit configured to perform low-latency countermeasures at the network edge, including selective packet filtering, flow isolation, process interruption, or system microsegmentation, based on instructions from the coordinating processor; and a hardware-embedded security enclave that is embedded in the system and configured to store cryptographic keys, neural model parameters, and integrity affirmation data to ensure the confidentiality, authenticity, and immutability of model artifacts and policy configurations.

A discrete three-dimensional (3-D) processor comprises stacked first and second dice. The first die comprises three-dimensional memory (3D-M) arrays, whereas the second die comprises at least a portion of a logic / processing circuit and an off-die peripheral-circuit component of the 3D-M array(s). The preferred 3-D processor can be used to compute non-arithmetic function / model. In other applications, the preferred 3-D processor may also be a 3-D configurable computing array, a 3-D pattern processor, or a 3-D neuro-processor.

An information handling system includes a central processing unit, a neural processing unit, and an offload module. The offload module receives an inference container including multiple inference models and metadata associated with the inference models. Based on the metadata, the offload module determines whether a quality of service for the inference models may be met by the neural processing unit. In response to the quality of service being met in the neural processing unit, the neural processing unit executes the inference models. In response to the quality of service not being met in the neural processing unit, the central processing unit executes the inference models.

Systems and methods are disclosed for efficient memory allocation for processing large language model encoders and decoders based on an attention model. The system can utilize a plurality of two types of processors suitable for different types of LLM processing. These include Neural Processor Units and Graphic Processing Units. Each NPU processor has dedicated DDR memory coupled to each NPU. The DDR memory caches the neural network weights used in the generation of neural network activations. A plurality of GPUs provides KVQ token processing. LLM tokens processing can be performed in parallel batches or sub-batches to utilize idle NPU processors within the neural network. In some embodiments, the NPUs are structured in a matrix with a bus between adjacent processors. In another embodiment, NPUs provide both KVQ processing and neural network processing. The system can be integrated on a silicon substrate using chiplets in a 2.5 or 3-D architecture.

The technology described adds a ML inference to the output of an image signal processor (ISP) associated with a camera. The combined image and ML inference may be described herein as an augmented image. Once generated, the augmented image may be communicated to other components of a computing system associated with the camera and / or ISP. The initial inference may be generated by a neural processing unit (NPU) associated with the ISP. The ISP may communicate a generated image to the NPU prior to communicating the image to a computing system. In an aspect, the NPU inference is combined with the image using image steganography. Once communicated from the camera to the computing device, the augmented image may be separated into a base image and inference by a camera driver or other component associated with the image management.

An integrated circuit (IC) layer stack is disclosed that integrates bypass chiplets in the IC layer stack without requiring fabrication of through-silicon vias (TSV) in a digital device for signals and power to a memory stack located above the digital device. Not requiring adding TSVs to an existing design high-performance digital device significantly reduces and / or eliminates digital device modification costs and time to manufacture new products consisting of three-dimensional stacked memory and digital device organizations. The bypass chiplets may be passive with just through conductors in silicon, or active with logic circuits therein. Flexibility of placement of the bypass chiplets in the IC layer stack in combination with an active interposer provides for many possible configurations of IC layer stacks. The digital device may be a microcontroller, a microprocessor, a PLA, an ASIC, a DSP, a GPU, a FPGA, neural processing unit, tensor processing unit, or any combination thereof.

According to one example of the present disclosure, the neural processing unit may comprise a processing element array configured to perform operations of a neural network model and a post-processing unit configured to process data output from the processing element array. The post-processing unit includes a first computation circuit that extracts a subset of classes for each bounding box by comparing class scores of classes and a second computation circuit configured to extract one or more bounding boxes by comparing a class confidence score of each bounding box with a threshold confidence score.

A system includes a substrate on which a first memory, a neural processing unit (NPU) including a plurality of processing elements (PEs) with multiplier-accumulator circuits, a controller, and a second memory, and a central processing unit (CPU) are disposed. The CPU may be configured to execute a universal compiler to perform a conversion for a particular neural network model into a machine code executable by the NPU and store the machine code in the first memory or the second memory. When the particular neural network model, generated by one among a plurality of machine learning frameworks that are incompatible with each other, is received and stored in the first memory, the universal compiler may perform the conversion based on mapping information indicating mapping between elements of machine learning frameworks and functions or operations executable by the CPU or NPU.

The invention discloses a reconfigurable heterogeneous radar data computing module and computing device, and relates to the technical field of computing devices, the reconfigurable heterogeneous radar data computing module comprises a ZYNQ processor, a digital signal processor (DSP) and a neural processing unit (NPU), the ZYNQ processor is connected with the DSP and the NPU; the ZYNQ processor comprises an ARM processor and an FPGA (Field Programmable Gate Array) logic resource; the FPGA logic resource is used for carrying out parallel class algorithm processing on the radar data; the DSP is used for carrying out serial algorithm processing on the radar data; the NPU is used for performing artificial intelligence algorithm processing on the radar data; the ARM processor is used for scheduling FPGA logic resources according to the calculation tasks, and the DSP and the NPU execute corresponding algorithms. According to the reconfigurable heterogeneous radar data calculation module, the high-performance calculation requirement can be met.

Embodiments relate to converting functions or function call instructions of a first neural network (NN) model into graph module. The relationship between one or more inputs and one or more outputs of the graph modules are analyzed. A second neural network (NN) model in a form of a directed acyclic graph (DAG) including using the graph modules is generated by mapping inputs and outputs of the graph modules based on the relationship. Markers are added to the graph modules in the second NN model. First calibration data is generated by collecting input values and output values of each of the graph modules using the markers. An adjustment value for outlier alleviation for each of the graph modules is generated based on the first calibration data. For each graph module of the second NN model, an input parameter and a weight parameter are updated based on the adjustment value.

A method of reusing feature maps that is operable on a neural processing unit (NPU) includes calculating, for a first layer of at least one artificial neural network (ANN) model, a space cost based on sizes of an input feature map, an output feature map, and a weight of a subsequent layer of the first layer; calculating a caching value for an operation of the first layer; determining a caching profit for the operation of the first layer based on the space cost and the caching value; determining a caching entry having maximum caching profit among caching candidate entries; and storing the caching entry in a variable memory. A system including the NPU includes a main memory to store a portion of the ANN model; a variable memory to selectively store a caching entry of the portion of the ANN model; and a controller to perform the method.

The invention discloses a server cable management method and device, equipment and a storage medium, and relates to the technical field of fault detection, and the method comprises the steps: constructing a cable connection relation graph of a target cable of a target server based on a preset graph neural network; the target cable is a cable embedded in the controller and the target sensor in advance, and a neural processing unit is integrated in the controller; generating a cable code of the target cable, and determining a cable state of the target cable through the target sensor by using the controller; when the cable state is abnormal, determining a fault path of the target server according to the cable code and the cable connection relation graph; and displaying the fault path by using a positioning display device of the target cable so as to manage the target server. The controller and the sensor are implanted into the cable body, the cable connection relation graph is constructed in combination with the neural network model, cable troubleshooting and life management are further achieved through cable coding, the troubleshooting cost is greatly reduced, and the intelligent management process of a server is improved.

A mobile artificial intelligence (AI) device includes a camera configured to capture a video of a product, and a neural processing unit (NPU) configured to perform inference with an AI recognition model to recognize product information from the captured video on the mobile AI device. A battery supplies power to the NPU. A transceiver is configured to transmit only the recognized product information to a remote server without transmitting the captured video, and to receive additional commercial information corresponding to the recognized product information. A display is configured to present the additional commercial information.

According to one example of the present disclosure, a system may be provided. The system may comprise at least one processing core configured to process computations of the at least one neural network model comprising at least one tensor, at least one memory circuit configured to store the at least one tensor, a bus circuit, electrically coupled to the at least one processing core and the at least one memory circuit, configured to transmit the at least one tensor based on a memory access operation instruction, and a controller configured to control a priority of a memory access operation for each tensor of the at least one processing core.

The invention is particularly directed to a hardware system (1) designed for implementing an artificial neural network (ANN). The hardware system substantially includes a neural processing device (15) (e.g., involving a crossbar array structure), one or more lookup table circuits (17), and one or more processing units (18). The neural processing device is configured to implement M artificial neurons, where M > = 1. The lookup table circuit is configured to implement a lookup table (LUT). The system also includes M'processing units, where M > = M '> = 1. Each processing unit is connected by at least one neuron so as to be able to access a first value output by each connected neuron. In addition, each processing unit is connected to the LUT circuitry so as to efficiently access parameter values of the parameter set from the LUT. Finally, each processing unit is configured to output a second value corresponding to a value of a mathematical function with the first value as an argument. The mathematical function is additionally determined by a set of parameters whose parameter values are accessed in operation from the LUT by each processing unit. That is, the mathematical function is defined (and thus determined) by a set of parameters, the values of which can be efficiently retrieved from a hardware-implemented LUT. This results in a significant acceleration of the calculation of the function output beyond the acceleration that may have been achieved within the neural processing devices and processing units themselves. Thus, neuron outputs can be processed more efficiently before passing to the next neuron layer. The invention is also directed to a method of operating such a hardware system.

The invention discloses an awareness perception quantum gravitational nerve acceleration unified computing platform and method based on formalized verification, and belongs to the field of high-performance computing, medical AI and quantum information processing. A quantum gravitational coupling neural processing unit (QGCNPU); a consciousness information integral calculation core (phi ICC); formalizing a receipt chain verifier (FRCV); and the mirror reflection symmetry correction module (MRSCM) is used for eliminating the continuous projection mistake and the discrete Boolean mistake through topological equivalence verification of a local observer and a global GodCam visual angle. The system adopts an FPGA + ASIC + quantum processor three-layer heterogeneous architecture, and supports real-time pathological modeling (precision gt; gt) of neurodegenerative diseases such as Alzheimer's disease, Parkinson's disease and ALS; 99.7%), drug target discovery (acceleration by 50 times) and consciousness state monitoring (time resolution lt; and meanwhile, the mathematical proving performance and the clinical auditing performance of the calculation process are ensured, and the requirements of the global 152 billion dollar neural science and technology market and the 89 billion dollar quantum calculation industry are met.

In accordance with the present disclosure, an apparatus is provided. The apparatus includes: a first memory having a first capacity configured to store a first generative neural network model including a first parameter; and a first neural processing unit configured to generate a response corresponding to the input query using the first generative neural network model stored in the first memory; and wherein the first neural processing unit may be configured to store first execution code of the first generative neural network model, the first execution code being compiled to process the speculative decoding.

A cloud-integrated, AI-powered smart speaker for language learning, which includes the following: A microphone array for speech capture; ◯ an on-device AI module with a Neural Processing Unit (NPU) or Tensor Processing Unit (TPU) for real-time speech analysis; • Wi-Fi and 5G connectivity for secure data synchronization with cloud servers; ◯ a cloud-based AI engine that uses serverless computing services to trigger AI models for advanced language assessment and adaptive learning; ◯ a cloud storage system with relational databases for structured data, such as user profiles and progress metrics, and object storage for unstructured data, such as audio recordings, secured with AES-256 encryption and RBAC for data protection; ◯ a true-to-life speaker for AI-generated speech output; ◯ a screen (OLED / E-Ink) for text corrections and real-time lesson guidance; ◯ Touch and gesture sensors for user interaction; ◯ A battery module with USB-C charging function for portability.

The invention is notably directed to a hardware system (1) designed to implement an artificial neural network (ANN). The hardware system basically includes a neural processing apparatus (15), e.g., involving as crossbar array structure, one or more lookup table circuits (17), and one or more processing units (18). The neural processing apparatus is configured to implement M artificial neurons, where M≥1. The lookup table circuits are configured to implement a lookup table (LUT). The system further includes M′ processing units, where M≥M′≥1. Each processing unit is connected by at least one neuron, in order to be able to access a first value outputted by each connected neuron. In addition, each processing unit is connected to a LUT circuit, in order to efficiently access parameter values of a set of parameters from the LUT. Finally, each processing unit is configured to output a second value, corresponding to a value of a mathematical function taking said first value as argument. The mathematical function is otherwise determined by the set of parameters, the parameter values of which are accessed by each processing unit from the LUT, in operation. I.e., the mathematical function is defined (and thus determined) by a set of parameters, the values of which are efficiently retrieved from the hardware-implemented LUT. This results in a substantial acceleration of the computations of the function outputs, beyond the acceleration that may already be achieved within the neural processing apparatus and the processing units themselves. As a result, the neuron outputs can be more efficiently processed, prior to being passed to a next neuron layer. The invention is further directed to a method of operating such a hardware system.

Devices and techniques are generally described for nonlinear tensor compression for neural networks. In various examples, a first tensor associated with a first layer of a neural network may be determined. One or more neural processing units of accelerator hardware may generate a first compressed tensor by applying a nonlinear compression function to the first tensor. The first compressed tensor may be stored in a first memory of the one or more computer-readable media. A first operation associated with a second layer of the neural network may be determined, where the first operation uses output of the first layer. The first operation may be performed based on the first compressed tensor.

A docking station includes a processor and a memory coupled to the processor. The docking station may be configured to establish a sideband connection between an information handling system and the docking station subsequent to detecting the information handling system docking at the docking station. The docking station may also execute a workload of the information handling system, subsequent to the establishment of the sideband connection. In addition, the docking station may transmit a notification to resume the workload via the sideband connection subsequent to detecting that the information handling system is undocked from the docking station while performing the executing of the workload.

A method may comprise receiving a first neural network (NN) model including one or more functions; generating a second NN model in a form of directed acyclic graph (DAG) including one or more graph modules by converting the one or more functions; calculating one or more scale values by obtaining maximum and minimum values of parameters input to the one or more graph modules; updating the parameters based on the one or more scale values; and generating a third NN model, in a form of machine code executable on a particular neural processing unit, including the updated parameters.

According to the present disclosure, a device is provided. The device includes a first memory of a first capacity configured to store a first generative neural network model comprising a first parameters, and a first neural processing unit configured to generate a response corresponding to an input query utilizing the first generative neural network model stored in the first memory, and wherein the first neural processing unit may be configured to store a first execution code of the first generative neural network model compiled to process speculative decoding.

Example storage systems, storage devices, and methods provide peer configuration of hardware compute resources among peer storage devices. Data storage devices may include hardware circuits, such as graphics processor units, media processor circuits, compute accelerator circuits, and neural processing units, configured for compute operations targeting host data associated with that storage device. One of the storage devices is configured to act as a master storage device for determining firmware configurations for the other storage devices and sending an indication of the firmware image to be used for some operating period. The storage device receiving the firmware image loads the firmware image for the hardware circuit and reboots the hardware circuit to configure it with the firmware image.

A heterogeneous neural processing system includes a first processor configured to perform encoding and decoding operations of an autoencoder, and a second processor configured to perform neural network operations of a particular task, the operations having iterative processing. The processors perform compute tasks through synchronous data exchange to implement a generative AI model. The first processor processes a feature map divided into data rows, caches the data into active memory using row-based depth-first scheduling, and selects deeper operations in the network hierarchy when processing branch inputs, outputs, and residual connections. And H reuses boundary pixels between the cache storage space segments, so that convolution and element-level operation can be executed simultaneously. A neural network adjusts the device analysis model to identify layer dependencies, applies search space constraints, performs an iterative search to generate a fusion plan, and selects an optimal plan based on external memory access and execution delays.

Disclosed in the embodiments of the present application are a processor, a computing node, and a computing cluster, which are used for simplifying the network architecture of the computing cluster. The processor comprises: a general-purpose protocol port, which is used for supporting a first communication protocol and a second communication protocol, wherein the second communication protocol is different from the first communication protocol; the first communication protocol is a communication protocol of a first network, the second communication protocol is a communication protocol of a second network, the first network is used for supporting communication between a plurality of processors in one computing node including the processor, the second network is used for supporting communication between the processor and other processors comprised in other computing nodes, and the processor is any one of a graphics processing unit (GPU), a tensor processing unit (TPU), a neural processing unit (NPU) and a general-purpose graphics processing unit (GPGPU).

A computing device includes at least one processor, one or more non-transitory computer-readable storage media, a system for fine-tuning a large language model under low-bit weight-activation quantization. The computing device further comprises a graphics processing unit (GPU), a neural processing unit (NPU), or a tensor processing unit (TPU). The hardware interface module of the system is configured to load a low-bit model representation from the memory module and transmit the model representation to the GPU, NPU, or TPU for inference execution.

An electronic device according to various embodiments may comprise: a memory and a processor operatively connected to the memory. The processor may include: a central processing unit including a cache memory; a neural processing unit; and an interconnector connected to the neural processing unit. The central processing unit may include an arbiter for connecting the cache memory to any one of a processing circuit of the central processing unit and the interconnector. The neural processing unit may exchange data with the cache memory through the arbiter.

A neural processing device and a method for using shared page table thereof are provided. The neural processing device including at least one neural processor, a shared memory shared by the at least one neural processor, and a global interconnection configured to exchange data between the at least one neural processor and the shared memory, comprises at least one processing unit each of which included in each of the at least one neural processor and configured to provide logical addresses, a memory management unit configured to receive and translate the logical addresses into physical addresses, and a physical memory accessible by the physical addresses, wherein the memory management unit comprises a shared page table that has translation information between the logical addresses and the physical addresses and is shared by at least one process with each other.