4542results about "Concurrent instruction execution" patented technology

The invention relates to the technical field of artificial intelligence chips, and provides an operator optimization method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining the block size of operator data in each dimension based on the batch size and mask mode of the operator data and the hardware parameters of computing equipment, each dimension comprises a batch dimension and a sequence length dimension; segmenting the operator data based on the block size of the operator data in each dimension to obtain a plurality of data blocks; and distributing the calculation tasks corresponding to the plurality of data blocks to a plurality of processing units on the calculation equipment, and performing parallel execution on the calculation tasks corresponding to the plurality of data blocks based on the plurality of processing units. According to the method and the device, the data are segmented in the batch dimension and the sequence length dimension at the same time, so that when the data batch is small, a plurality of processing units on the computing equipment can also participate in computing at the same time, hardware resources are prevented from being idle, the utilization rate of the hardware resources is improved, and the overall computing efficiency is improved.

The invention provides an optimization method for improving parallelism and performance of hardware executing RISC-V vector instructions, which comprises the following steps: S1, an instruction fetching and decoding stage step: the step comprises an instruction fetching link and a decoding link, and the instruction fetching link reads instructions from a memory according to a program sequence and stores the instructions into an instruction queue; the decoding link comprises the steps of analyzing an instruction, identifying whether the instruction is a vector mask instruction or a vector length control instruction, and if the instruction needs an old value of a destination register, marking that the instruction needs to carry an old value dependency identifier, belonging to the technical field of optimization methods. According to the method, the instruction is transmitted without waiting for the old value of the target vector register to be ready, the instruction can be transmitted in advance after a certain condition is met, and the correct old value is obtained, so that the aim of improving the parallelism and the performance of hardware for executing the RISC-V vector instruction is fulfilled.

Techniques for optimizing and deploying deep neural network (CNN) machine learning models for inference using static analysis are described. A method includes obtaining a deep neural network (DNN) machine learning (ML) model, generating an intermediate representation for the ML model, the intermediate representation including one or more nodes corresponding to one or more operators utilized by the ML model, identifying, for at least one node of the intermediate representation, an optimized schedule for at least one operator corresponding to the at least one node using a static analysis that is based on a hardware-specific cost model, generating an optimized intermediate representation using the optimized schedule that is optimized for execution on a hardware platform, and generating code corresponding to the ML model based at least in part on the optimized intermediate representation, wherein the code is specific to the hardware platform.

Apparatuses, systems, and techniques to perform a neural network. In at least one embodiment, an application programming interface schedules two or more graph nodes to be performed by two or more parallel processing pipelines based, at least in part, on an order of layers in the neural network.

An apparatus providing tensor core matrix multiplication and accumulation (MMA) with hardware-based statistics collection and outlier suppression is disclosed. The apparatus includes processor circuitry comprising at least one processor core comprising matrix multiplication circuitry to: execute a matrix multiplication operation on first input data from a first set of registers and on second input data from a second set of registers; collect, as part of executing the matrix multiplication operation via statistics collection hardware circuitry of the matrix multiplication circuitry, output statistics data corresponding to the matrix multiplication operation; and output the output statistics data along with a result of the matrix multiplication operation; and output statistics storage to store the output statistics data.

The invention belongs to the technical field of data processing, relates to an aircraft multi-mode instruction conflict resolution method and system, and aims to solve the problems of high conflict detection time delay and priority strategy staticization of a traditional method. The method comprises the steps that after a multi-mode instruction is received, semantic feature extraction and time sequence alignment are completed through a three-layer attention mechanism and a dynamic time warping algorithm, and a structured instruction set is generated; evaluating the priority of each instruction by combining a four-dimensional dynamic weight system with reinforcement learning; inputting the instruction set with the priority into a six-tuple model to identify a conflict type and an instruction set; and a three-level progressive arbitration method is adopted to resolve conflicts, an optimal execution sequence is generated, and the four-dimensional dynamic weight system is fed back and optimized according to an execution result. According to the method, the real-time performance and the dynamic adaptability of multi-mode instruction conflict resolution are remarkably improved, and reliable guarantee is provided for aviation instruction interaction safety.

The invention discloses an artificial intelligence large model training method in a heterogeneous multi-machine and multi-card environment, and belongs to the technical field of artificial intelligence large model training. Load balancing of heterogeneous equipment is realized by constructing a uniform interface, and the communication efficiency is optimized by adopting hierarchical pipeline aggregation and dynamic quantization compression; the node dynamic adjustment is realized in combination with the elastic topological structure, the problems of poor equipment compatibility, high communication delay and rigid topological structure in the prior art are effectively solved, and the method has the remarkable advantages of improving the utilization rate of heterogeneous computing resources, reducing the cross-node communication overhead and enhancing the fault-tolerant capability of the system.

The invention relates to the technical field of artificial intelligence, can be applied to service scenes of pension service, financial science and technology, medical health and the like, and discloses a multi-level heterogeneous integrated chip task processing method, device, equipment and medium, and the method comprises the steps: constructing a multi-level heterogeneous integrated chip composed of a perception processing layer, an intelligent decision-making layer and a driving control layer, the layers are connected through a vertical interconnection structure; receiving multi-modal task data and extracting features to generate feature vectors; inputting the feature vector into a neuromorphic processing unit to determine a task decision result; converting the decision result into a driving signal to control an execution device; adjusting synaptic weights based on the feedback signal; and monitoring chip operation state parameters and dynamically adjusting processing frequency and structure parameters. By integrating multi-modal sensing, neuromorphic decision and a dynamic feedback mechanism, data sensing, decision and execution processing are completed in a chip, and the real-time performance and the calculation efficiency are improved by combining operation state monitoring and adjusting frequency and structure.

One aspect of the instant disclosure may provide a system and method for processing scatter-gather direct memory access (S-G DMA) instructions. During operation, the system may receive an S-G DMA instruction associated with a message and gather instruction context for the S-G DMA instruction. An S-G DMA processor may process the S-G DMA instruction based on the gathered instruction context and determine whether there exists a pending S-G DMA instruction associated with the message. In response to the presence of the pending S-G DMA instruction, the system stores the instruction context in a hot context cache at an address corresponding to the pending S-G DMA instruction. In response to the absence of the pending S-G DMA instruction, the system stores the instruction context in a cold context cache.

The invention relates to the field of energy-saving computing, and discloses a low-power-consumption control method and system of a controller, which are used for realizing prediction-sleep-wake-up-response full-link hardware in the controller. Time sequence state data of a controlled object is collected in real time, and a predictive sleep window value is dynamically generated by utilizing a predictive cooperative computing unit, so that intelligent scheduling of sleep and awakening is realized, and the power consumption of a controller is effectively reduced. Meanwhile, in combination with the technologies of threshold comparison, DMA data transmission optimization and the like, the data processing efficiency is improved, and invalid data transmission is reduced. And on the awakening mechanism, an asynchronous awakening circuit and an interrupt request mechanism are adopted to ensure that power supply is quickly recovered and control response is executed when an effective instruction is received, so that the real-time performance of the system is guaranteed. According to the invention, a prediction algorithm, hardware acceleration and an intelligent scheduling technology are fused, the performance of the controller is ensured, the power consumption is obviously reduced, and the endurance time of equipment is prolonged.

The invention discloses an intelligent data stream processing system based on Flink and an implementation method thereof, and belongs to the technical field of distributed streaming data processing, and the implementation method comprises the following steps: a data source access layer uniformly accesses multi-source data; the entity monitoring layer captures service entity change in a non-intrusive manner based on JPA, generates a standardized message body and asynchronously delivers the standardized message body to double channels; the unified event model layer maps the data into a standardized event and expands the standardized event; the Flink stream processing engine layer is used for complex event processing, state management and window calculation, and supports declarative pipeline definition and dynamic construction; the anomaly detection layer performs multi-dimensional anomaly detection; the distributed task scheduling layer schedules tasks according to a DAG model, and realizes load balancing through hotspot splitting and state transition; and the result storage and visualization layer adopts a cold and hot separation strategy to store data, and provides a dynamic visualization interface. The real-time performance, the consistency and the intelligent level of data processing are remarkably improved, and the development and maintenance cost is reduced.

The invention belongs to the field of high-performance computing, and relates to an integer parallel computing method and device based on distributed storage and computer equipment, and the method comprises the steps of collecting real-time resource indexes, dynamically identifying fault nodes, triggering task migration, and performing data verification and hard disk fault detection. The weight value of each node is calculated, the nodes are arranged according to the descending order of the weight values, and the nodes with high load capacity are selected to distribute tasks; dynamically distributing a data generation task to a computing node, executing parallel computing, and performing distributed storage on a result; obtaining an operand, converting the operand into a first-order tensor form of a basic operand, serializing tensor data, and sending the serialized tensor data to a parallel computing layer; distributing a search task to a computing node, retrieving storage data in parallel, reading effective data from a storage layer, and combining search results into a partial sum; and summarizing and then outputting. The system has dynamic resource management and fault-tolerant capabilities, and can realize efficient task allocation and load balancing.

A disclosed method may include initializing a deep learning recommendation model (DLRM) comprising a plurality of embedding tables, each embedding table comprising a plurality of embeddings. The method may also include receiving input data associated with accessing embeddings from the plurality of embedding tables and applying a parallelization strategy to process the plurality of embedding tables, the parallelization strategy configured to improve performance by distributing computational workloads and optimizing memory access. The method may also include processing the embeddings based on the input data in accordance with the parallelization strategy, the processing comprising aggregating embeddings accessed from the plurality of embedding tables. The method may also include generating, for further processing, output data based on the processed embeddings. Various other methods, systems, and computer-readable media are also disclosed.

The invention belongs to the technical field of computers, particularly relates to a prefetch instruction management method, system and equipment, and aims to solve the problem of low instruction fetch efficiency of an instruction prefetch technology. The method comprises the following steps: receiving branch prediction information from a branch prediction unit, and performing label comparison on table entries in an instruction fetching target queue and the branch prediction information; under the condition that the table item is hit, querying an instruction fetching address corresponding to the branch prediction information in the hit table item; under the condition that the table item is not hit, a new storage table item is allocated to the branch prediction information, and an instruction fetching address carried by the branch prediction information is determined through the storage table item; in response to a received prefetching request sent by the prefetching unit, determining a target table item based on the prefetching request, and returning an instruction fetching address queried in the target table item to the prefetching unit; and storing the stable cache line in the target table item to a flow buffer area. According to the method, multiple prediction requests can be processed in parallel, the front-end throughput is improved, and the instruction fetching efficiency is improved.

The invention belongs to the technical field of artificial intelligence and high-performance computing, and particularly relates to a neural network large model efficient reasoning method based on multiple GPGPUs. The method aims to solve the problems of high communication overhead, non-uniform load, low resource utilization rate, high data transmission delay and the like among multiple processors. Dividing a calculation task into a plurality of sub-graphs through static analysis and mixed granularity partitioning of a model calculation graph; distributing the sub-graphs to the optimal GPGPU based on a weighted cost function in combination with heterogeneous resource perception and a dynamic mapping strategy; a global pipeline scheduling plan is constructed by using communication topology perception, and calculation and communication overlap are maximized; data are loaded in advance through a host side hierarchical caching and asynchronous prefetching mechanism, and transmission delay is hidden; multi-stream concurrent execution and event-based lightweight synchronization are adopted on each GPGPU, so that waiting overhead is reduced. According to the method, the reasoning delay can be remarkably reduced, the throughput and the hardware utilization rate are improved, and the method has good adaptivity and expandability.

Aspects of the disclosure provide for an accelerator capable of accelerating data dependent, irregular, and / or memory-bound operations. An accelerator as described herein includes a programmable engine for efficiently executing computations on-chip that are dynamic, irregular, and / or memory-bound, in conjunction with a co-processor configured to accelerate operations that are predictable in computational load and behavior on the co-processor during design and fabrication.

The invention provides a processor memory access optimization implementation method and system with multi-channel concurrent instructions, and relates to the technical field of processor integrated circuits, and the method comprises the steps: obtaining a Load instruction and a Store instruction to be executed; distributing a plurality of memory access instructions to each memory access channel which is independently processed in parallel, wherein each memory access channel comprises a Load memory access channel and a Store memory access channel; an independent Load queue and an independent RAW check queue are arranged in the Load memory access channel, and an independent Store queue, an independent STD queue and an independent Store Buffer queue are arranged in the Store memory access channel; the Load instruction is sequentially subjected to an address generation stage, a TLB parallel query stage, an L1Cache access stage, a RAW check and data forward push stage and a data write-back stage in the Load memory access channel; according to the method, address and data separation processing is carried out on a Store instruction, an address part enters a Store queue, a data part enters an STD queue, then merging is carried out, multiple Store operations on the same Cache Line are merged into one-time writing, the merged data are written into a Store Buffer queue, and then the Store Buffer queue submits and caches in a unified mode.

The present application discloses a compute-in-memory chip, an instruction scheduling method, and a related apparatus. The compute-in-memory chip comprises an instruction memory, an instruction scheduler, and at least one compute-in-memory memory; each compute-in-memory memory comprises at least one storage array; the instruction memory is used for acquiring a first tensor instruction to be executed; and the instruction scheduler is used for scheduling, on the basis of the association relationship between the first tensor instruction and a second tensor instruction and the state of a target storage array needing to be operated for executing the first tensor instruction, the first tensor instruction to the compute-in-memory memory to which the target storage array belongs so that the compute-in-memory memory executes the first tensor instruction. According to embodiments of the present application, diversified compute-in-memory computing can be supported, efficient out-of-order execution scheduling of a tensor instruction set is achieved on the basis of the compute-in-memory chip, and the requirements of compute-in-memory technology for high concurrency and high throughput rate are met.

The invention belongs to the field of edge computing acceleration, and relates to a fast Fourier transform and neural network reasoning collaborative computing acceleration method, which comprises the following steps of: deploying a computing method which is based on butterfly computing merging and a tensor mapping strategy and is mixed with DFT (Discrete Fourier Transform) and FFT (Fast Fourier Transform) in an operator deployment level; on the interface integration level, a bus interface of a register access path in a tensor accelerator control path is subjected to lightweight reconstruction, so that the bus interface is adaptive to an edge computing platform; in an operation level, a user-defined instruction is introduced into a tensor calculation unit, and an accelerator is enabled to independently complete a whole-process task of FFT signal processing and NN intelligent identification. According to the method, an operation instruction set is expanded on hardware, and delay overhead caused by returning a large amount of intermediate data to a processor is avoided, so that efficient execution of FFT and neural network tasks on unified hardware is guaranteed, and multiple requirements of high real-time performance, high precision and low power consumption are met.

The invention discloses a self-adaptive precision adjustment method in the NPU field. In the invention, through a mixed precision conflict resolution mechanism, global optimal balance of precision and performance in a deep learning reasoning task is realized. According to a traditional method, when the calculation precision is adjusted, independent decision making is usually carried out only based on local operator characteristics, and therefore the overall precision of an operator group dependent on series connection is collapsed due to error accumulation. According to the method, an error propagation chain is constructed through a quantitative sensing model, influence paths of different precision combinations on task output are dynamically predicted, and differential precision distribution is implemented on conflict operator groups by adopting a critical path priority algorithm. In a target detection task, a key path such as a classification head forcibly retains high-precision calculation, and a background filtering layer is reduced to a low-precision mode, so that the mechanism fundamentally solves the global sub-optimal problem caused by local optimization in a traditional method, and ensures the calculation reliability of a high-value operator.

A data caching method and apparatus, an electronic device, and a readable storage medium, relating to the technical field of computers. The method comprises: receiving a memory access request sent by a processor, wherein the memory access request carries a memory access address; when the memory access request meets a bypass prediction condition, performing bypass prediction on a target data block corresponding to the memory access address to obtain a prediction result; when the prediction result indicates that the target data block corresponding to the memory access address needs to be bypassed, writing the target data block into a stream buffer; and when the prediction result indicates that the target data block corresponding to the memory access address does not need to be bypassed, writing the target data block into a basic cache.

The invention discloses an action scheme sample space search method based on ordinal optimization, particularly relates to the field of simulated confrontation, is used for solving the problem of high-dimensional constraint verification in a complex dynamic confrontation simulation scene, and provides a global view angle of topological dependence and influence degree by constructing a constraint coupling map and an influence degree matrix; thirdly, on the basis of a topological sorting and forward propagation algorithm of the atlas, preferably verifying core constraints, quickly filtering candidate schemes violating key conditions, and reducing a search space; then, through calculating an elastic adjustment factor and setting a dynamic tolerance threshold value, flexibly adjusting non-key constraints, so that the scheme generates adaptive battlefield situation changes; afterwards, residual constraints are coded into a Boolean logic expression, a verification task tree without data dependence is generated by utilizing a symbolic execution engine, and a decomposition basis is provided for parallel processing; finally, efficient parallel verification is achieved by means of GPU stream multiprocessor and CUDAwarp-level instruction synchronization, and a scheme set meeting full constraints is rapidly output.

The invention provides a vector core module of an artificial intelligence chip and an operation method of the vector core module, which are used for improving the efficiency of the vector core module in an application situation that a vector core is a host and a tensor core module is a slave. The vector core module comprises a vector core instruction execution pipeline, a tensor core instruction processing pipeline and an instruction scheduling unit. An instruction scheduling unit performs instruction classification to distinguish vector core instructions and tensor core instructions from a thread bundle. In response to the thread bundle including a vector core instruction, the instruction scheduling unit sends the vector core instruction to a vector core instruction execution pipeline. The vector core instruction execution pipeline executes the vector core instruction and stores an execution result in the memory module. In response to the thread bundle including a tensor core instruction, the instruction scheduling unit sends the tensor core instruction to a tensor core instruction processing pipeline. The tensor core instruction processing pipeline processes the tensor core instruction and sends a processing result to the tensor core module.

The invention belongs to the related technical field of high-performance computing, and provides a memory architecture-oriented dual-precision general matrix multiplication optimization method and system in order to solve the problems of limited computing power and access efficiency and the like in the prior art. Decomposing the matrix into a plurality of sub-matrix blocks according to the slave core array topology; the slave core receives the sub-matrix blocks issued by the master core, divides the sub-matrix blocks into small sub-matrix blocks based on a uniform blocking rule, loads the small sub-matrix blocks to an independent buffer area of a local data memory based on a DMA double-buffer protocol, divides the small sub-matrix blocks in the buffer area into SIMD vectors according to the SIMD unit characteristics of the slave core, and sends the SIMD vectors to the slave core; vectorization calculation and caching operation are alternately switched according to an iteration period through different independent buffer areas; and after all the slave cores finish calculation, the master core collects results written back to the master memory by the slave cores to obtain a final operation result, and double breakthrough of calculation power and memory access efficiency is realized.

Data structures and methods are described to provide a unique extendable group identifier for efficient aggregation query processing using multiple grouping keys. A method comprises retrieving a database query comprising an aggregate function of a selected column from a database table, grouped by a plurality of columns. The method further comprises maintaining a plurality of hash tables that map column values to grouping keys and identify a maximum grouping key. The method further comprises updating a plurality of bitmasks that define per-column grouping key bit positions, capable of storing the maximum grouping key, within a combined index value. The method further comprises allocating a result array sized according to the plurality of bitmasks. The method further comprises using the plurality of bitmasks and the plurality of hash tables to determine the combined index value to apply the aggregate function on the selected column for each row in the database table.

A system that includes at least one multi-threaded processor forms a multitude of converged thread sub-groups from a main thread group, wherein each thread sub-group includes a common code block. A loop is configured to jump to a different target address of a branch instruction in an order determined by a priority configured for each different target address.

The invention discloses an instruction scheduling method, device and system, a product and a medium, and relates to the technical field of processor design. The invention provides an instruction scheduling method aiming at the problem that the dynamic change in the instruction transmitting time cannot be accurately scheduled by the existing out-of-order scheduling strategy. The historical delay information of each instruction is stored through the delay cache, and when the instruction execution delay corresponding to the instruction is dynamic delay, the dynamic delay is predicted through the historical delay information stored in the delay cache, so that instruction scheduling is performed. According to the method, the dynamic delay of which the delay time cannot be predetermined is included in the consideration range of instruction scheduling based on the dynamic delay prediction mechanism. On the basis of the method, for dynamic change scenes in instruction transmission time such as memory access delay and cache miss, the optimal instruction transmission opportunity can be accurately predicted, so that the effect of instruction scheduling on performance improvement of a processor is further improved.

The invention discloses a heterogeneous computing power cooperative scheduling system and method for mixed precision training, and belongs to the technical field of artificial intelligence computing. The system comprises a computational graph analysis and operator portrait module which is used for analyzing and dividing a model computational graph and extracting operator features; the heterogeneous hardware capability sensing and matching module is used for managing performance files and real-time states of heterogeneous hardware in the cluster and matching optimal execution hardware for each calculation partition; and the data flow coordination and pipeline parallel controller is used for generating a global execution plan, managing cross-device data dependence and communication and calculating overlapping optimization execution efficiency through communication. According to the method, the problem of low scheduling efficiency of mixed precision training in a heterogeneous environment is solved, automatic and accurate mapping from a calculation task to heterogeneous hardware is realized, the training speed is remarkably improved, the training cost is reduced, and the overall resource utilization rate of a cluster is improved.

The invention provides an SIMT-based neural network processor and a task execution method thereof, and the method comprises the steps that a general processor queries a state register of a coprocessor, and the state register stores the resource condition of the coprocessor; the universal processor completes splitting from a thread block to a thread bundle according to the resource condition, and the universal processor forwards a thread bundle instruction to a thread bundle distributor of the coprocessor; and the thread beam distributor decodes instructions in the thread beams in sequence and schedules the instructions to instruction queues of different calculation cores, and the thread beams sequentially perform thread beam scheduling, instruction emission and instruction execution according to the sequence, so that all calculations of a neural network task are completed in parallel, and a running result of the neural network task is obtained. According to the method, a general processor for task splitting is introduced in front of the thread beam scheduler or a specific compiler is directly used, so that dynamic scheduling of the threads can be realized, and a scheme for dynamically expanding the number of the threads according to data precision becomes a feasible architecture option.

The invention relates to the technical field of electric digital data processing, and discloses a multi-dimensional data logic processing method based on an artificial intelligence algorithm, which comprises the following steps: receiving a target discrete data packet, extracting metadata, and calculating to generate a dimension entropy feature vector representing data logic complexity; inputting the vector into a preset topological mapping model, and outputting an initial adjacent matrix; calling a feedback suppression mask matrix generated based on a historical operator utility state, and executing bitwise logic AND operation with the initial adjacent matrix to generate a corrected effective topological matrix; the matrix is analyzed, a logic operator function pointer is dynamically indexed in an instruction cache, and a directed acyclic execution linked list is constructed; according to the method, redundant logic nodes in AI prediction are definitely eliminated through a bit operation mask mechanism based on historical feedback, and deterministic convergence of processing delay and optimal matching of computing power resources are achieved.