43 results about "Overhead (computing)" patented technology

The invention belongs to the technical field of artificial intelligence and high-performance computing, and particularly relates to a neural network large model efficient reasoning method based on multiple GPGPUs. The method aims to solve the problems of high communication overhead, non-uniform load, low resource utilization rate, high data transmission delay and the like among multiple processors. Dividing a calculation task into a plurality of sub-graphs through static analysis and mixed granularity partitioning of a model calculation graph; distributing the sub-graphs to the optimal GPGPU based on a weighted cost function in combination with heterogeneous resource perception and a dynamic mapping strategy; a global pipeline scheduling plan is constructed by using communication topology perception, and calculation and communication overlap are maximized; data are loaded in advance through a host side hierarchical caching and asynchronous prefetching mechanism, and transmission delay is hidden; multi-stream concurrent execution and event-based lightweight synchronization are adopted on each GPGPU, so that waiting overhead is reduced. According to the method, the reasoning delay can be remarkably reduced, the throughput and the hardware utilization rate are improved, and the method has good adaptivity and expandability.

The invention relates to the field of large model fine tuning, in particular to a power grid field-oriented lightweight large language model fine tuning method. The method specifically comprises the following steps: acquiring local power grid data at an edge node, and reasoning through a lightweight large language model to obtain operation state data; and uploading the operation state data to a central node, selecting a matched fine tuning model version from a lightweight model version library, generating a fine tuning strategy, and ensuring efficient screening of training samples and reasonable allocation of computing power resources. Through the incremental training and LoRA technology, the fine tuning process is optimized, the training efficiency is greatly improved, and the calculation overhead is reduced. According to the method, the real-time performance of a power grid task can be guaranteed, meanwhile, limited computing resources of edge nodes are fully utilized, efficient fine adjustment of a lightweight large language model in the power grid field is achieved, and the performance of the model in tasks such as monitoring, fault diagnosis and operation optimization is improved.

The invention particularly relates to a task scheduling method and system based on a distributed scheduling framework, and relates to the technical field of distributed computing and task scheduling. The resource coordinator is connected with a conflict detection module; a conflict resolution arbitration module; and a final consistency state synchronization module. According to the invention, the lock-free design of the optimistic scheduling decision module and the local cache mechanism break through the dependence of the traditional scheduling on the global real-time consistent state, and realize the top-speed decision in the high-concurrency scene; the local cache with timeliness deviation adopts a hierarchical index structure and a dual-mode updating mechanism, so that a scheduler is supported to complete calculation only by depending on local data while the cache freshness and the updating overhead are balanced, cross-node real-time communication is not needed, and the decision delay is lower.

The embodiment of the invention relates to the technical field of artificial intelligence chips, provides a communication link establishment method and device, equipment, a medium and a program product, and is used for reducing cluster establishment cost and avoiding delay overhead caused by inter-node communication while expanding the single-node set communication scale. The method comprises the steps that attribute information of all computing devices deployed in a node is acquired, a plurality of computing devices are deployed in the node, and each computing device comprises a plurality of bare chips; for each computing device, respectively executing the following operations: based on the attribute information of the computing device, configuring the attribute information of each bare chip in the computing device, and obtaining the attribute information of a plurality of bare chips; and constructing a point-to-point communication link among the plurality of bare chips according to the attribute information of the plurality of bare chips.

The present application relates to the field of large model fine-tuning, and specifically relates to a lightweight large language model fine-tuning method for the power grid field. The specific method of the present application comprises: obtaining local power grid data at the edge node and performing inference through a lightweight large language model to obtain operating state data; uploading the operating state data to the center node and selecting a matched fine-tuning model version from the lightweight model version library to generate a fine-tuning strategy, ensuring efficient screening of training samples and reasonable allocation of computing resources. Through incremental training and LoRA technology, the fine-tuning process is optimized, significantly improving training efficiency and reducing computational overhead. This method can ensure the real-time performance of the power grid task while fully utilizing the limited computing resources of the edge node, achieving efficient fine-tuning of lightweight large language models in the power grid field and improving the performance of the model in tasks such as monitoring, fault diagnosis and operation optimization.

The invention relates to the technical field of heterogeneous computing, in particular to a prediction model dynamic scheduling system and method based on a heterogeneous computing platform. Running the prediction model in the edge host, and abstracting a calculation-intensive operator in the model into a candidate unloading task; and constructing a time cost model containing host execution time, FPGA execution time and transmission overhead, and dynamically updating in combination with execution statistics. And the scheduling module compares the predicted completion time of the local and unloading paths according to a time cost model, and selects to execute a prediction task on the host or the FPGA. A page locking DMA buffer area is arranged in a system memory, and the FPGA directly reads and writes historical data; and the FPGA side schedules tasks according to the priority and the deadline, and returns to a traditional closed-loop correction time cost model. Compared with the prior art, the method can dynamically adapt to illumination conditions and system load changes, reduces the average reasoning time delay and high quantile tail time delay of tasks, and improves the performance and stability of edge side photovoltaic power prediction.

The invention provides a computer-implemented reconstruction method and a corresponding system, a medium and a product. The method is used for reconstructing an original program based on an original variable to obtain a reconstructed program which retains functions of the original program and is based on a target variable different from the original variable. The method includes: receiving an original program, a first mapping specification for converting an original variable or a representation based on the original variable into a corresponding representation using a target variable, and a second mapping specification for converting a target variable or a representation based on the target variable into a corresponding representation using the original variable; the original program is traversed to identify statements in the original program involving the original variable, and a reconstruction operation is performed on the identified statements using the first and second mapping specifications to reconstruct them into corresponding reconstructed statements involving only the target variable. The scheme provided by the invention has the beneficial effects of reducing the processing and computing overhead required by reconstruction, improving the reconstruction efficiency, reducing the requirements on computer hardware and the like.

The invention discloses a sparse data compression method, and belongs to the field of big data and real-time calculation. The method comprises the following steps of: 1, optimizing the layout of an UnsafeRow data memory; 2, encoding the UnsafeRow data; step 3, carrying out UnsafeRow data decoding, and carrying out UnsafeRow data decoding; step 4, carrying out serialization and deserialization on the compressed UnsafeRow; step 5, ORC write-in optimization is carried out; wherein the ORC is an efficient and high-performance column type storage format; and step 6, ORC reading optimization is carried out. By optimizing the memory layout of UnsafeRow and compressing the memory use of sparse data, the memory use efficiency of the data is improved, the overhead of a network and a disk is reduced, and the data processing efficiency of Spark SQL (Structured Query Language) is improved; meanwhile, the storage space of data storage is reduced through encoding optimization of ORC.

The application provides a kind of mass network data distributed parallel processing method and system, it is related to data processing field, solve the problem of low efficiency of distributed parallel processing caused by the mismatch of data characteristics and computing power characteristics and the large overhead of global rearrangement in prior art.The method comprises: obtaining network data to be processed;Based on the first feature information of network data, the time consumption of prediction calculation is determined based on the time consumption of calculation, and a plurality of data fragments are obtained based on the slice boundary;Based on the mapping relationship between the first feature information of data fragment and the second feature information of computing node, the data fragment is distributed to the corresponding computing node;In the parallel processing process of data fragment, in response to the deviation between the actual progress and the estimated progress of computing node exceeds the preset threshold, the task of computing node that has not been completed is divided into subtask slices, and is migrated to other computing nodes for local redistribution.The application is used for mass network data processing.

The invention discloses a federated learning acceleration method and system based on pipeline in-batch real-time data sampling and training. According to the method, in-batch data sampling is provided on the algorithm level aiming at the problems that computing resources of edge equipment are limited and importance is outdated and gradient deviation is caused by existing static sampling: the importance of samples in a fixed mini-batch is evaluated in real time by utilizing a latest model in each round of iteration, and a dynamic micro-batch is constructed; and through a gradient correction coefficient based on a sampling probability reciprocal, distribution deviation is eliminated, and unbiased training is realized. On the system level, an assembly line parallel mechanism based on a CPU-GPU heterogeneous architecture is designed, overlapping execution of sampling and training is achieved through a double-thread-double-flow concurrent model, data competition is solved through an annular buffer area and an event synchronization mechanism, and overhead is reduced in cooperation with mixing precision reasoning. According to the method, the hardware utilization rate can be remarkably improved, the model convergence precision is improved while the training time is greatly shortened, and the method is suitable for various edge computing scenes.

The invention discloses a fair resource management method and system for multi-user shared large language model reasoning, and the method is characterized in that the method employs a cold data recognition elimination mechanism and a fair cache distribution mechanism, carries out the unified quantification of the resource consumption of a request in GPU calculation, decoding and cache transmission, and carries out the fair scheduling based on the accumulated CPI value of each user; the system comprises a user request management module, a CPI fair scheduling module, a model reasoning execution module, a resource monitoring module and a cache management module. Compared with the prior art, the method has the advantages that users can share computing and caching resources fairly, efficient operation performance of the system is guaranteed, users with high cache hit rate are prevented from excessively occupying GPU execution opportunities due to low apparent computing overhead, the overall reasoning efficiency and the cache hit rate of the system are remarkably improved, and the user experience is improved. Unified and fair distribution of computing resources and cache resources is achieved, the effect is prominent especially in a multi-tenant high-load scene, and the method has good application prospects and commercial development value.

The invention discloses an AI data handling optimization method based on a multi-core system. The data handling optimization method comprises the steps that S1, initialization is carried out; s2, asynchronous request; s3, asynchronous processing; and S4, parallel execution. The processor A for executing the AI operation is always in the user state and does not need to fall into the kernel state, so that the system calling overhead is avoided; in the process that the processor A executes the AI operation, the processor B schedules DMA hardware to complete transmission processing of next data required by the AI reasoning operation, the DMA hardware and the AI reasoning operation are completely parallel, and when DMA transmission time is shorter than AI reasoning operation time, time overhead of DMA data transmission is completely masked; based on the software and hardware collaboration framework, the utilization rate of the computing part of the processor A can be close to 100%.

The present application relates to the field of specific computing model and information security technology, aiming at the problems of high computing cost and high cost in existing outsourcing reasoning technology, a polynomial chain verification method and device for outsourcing neural network reasoning are proposed.The method comprises: data input;in the preprocessing stage, the client generates the mask matrix of the input data, the model end generates the mask and the verification auxiliary matrix of the model parameter, the input data, the mask matrix and the Beaver triple are divided into multiple shares and distributed to different computing servers;by constructing a verifiable polynomial chain, the verification amount and the mask are generated under additive secret sharing and returned to the client, and by designing a polynomial form of verification equation, the verification of the reasoning process is completed.The present application completes the reasoning verification by designing a polynomial equation, ensures the integrity of the whole process and the privacy, and realizes the low-overhead, low-cost step-level verification with constant-level online verification calculation time of parallel computing.

The invention relates to the technical field of communication, and discloses a parallel computing communication method, device and equipment based on a distributed many-core processor, high-precision time sequence synchronization can be realized through a central polling controller, a partition shared memory, dual-port extension and combination of a standardized process, the reliability of a computing result is improved, the software complexity is greatly reduced, and the communication efficiency is improved. The communication flexibility is ensured, the communication energy efficiency ratio is remarkably improved, data conflicts are also avoided, and power consumption and time sequence overhead caused by bus state switching are reduced. Data processing is continuous due to assembly line work, the overall throughput rate of the system is increased, the deployment process is greatly simplified, and the applicability and usability of the platform are improved.

The disclosure provides a heterogeneous computing scheduling method, system, device and storage medium, and relates to the technical field of artificial intelligence. In some embodiments of the disclosure, according to the task descriptor, the page table mapping relationship of the input-output memory management unit is configured without the participation of the system core, and address translation preprocessing is completed; the virtual address interval accessed subsequently by the neural network processing unit is predicted; before the neural network processing unit initiates a memory access request, according to the virtual address interval, the page table hardware traversal of the input-output memory management unit is triggered, the page table entry is obtained, and the page table entry is loaded into the input-output conversion buffer; a start signal is sent to the neural network processing unit, so that the neural network processing unit accesses the shared main memory through the input-output memory management unit based on the page table mapping relationship. Through the dual-core division of the system core and the control core, the system overhead caused by the participation of the operating system in the underlying scheduling is reduced, and the additional power consumption overhead is reduced.

The invention relates to the technical field of processors, in particular to a tile block instruction set architecture and a processing method, and the tile block instruction set architecture comprises a tile block parallel processing system control instruction, a tile block descriptor register configuration instruction, a tile block data handling instruction, a tile block data copying instruction and a tile block arithmetic instruction. According to the tile block instruction set architecture processing method, one or more tile block operation units, one or more tile block storage management units and one or more tile block calculation task scheduling and synchronization units are combined into a tile block parallel processing system, so that tile block instructions are allowed to be processed in parallel, and tile block-based data operation is completed. The architecture aims at solving the problems that when an existing processing system processes two-dimensional data blocks, instruction overhead is too large, data abstraction is insufficient, and the coupling degree of software and hardware is too high.

The application provides an AI accelerator processing optimization method, system, device and medium, and the method is applied to an AI accelerator in communication connection with a main memory. Through the collaborative architecture of the AI accelerator and the 3D DRAM, the application realizes reduction of transmission overhead and power consumption in the data carrying process, improvement of the computing throughput and the energy efficiency ratio, and simultaneously supports large-scale parallel computing tasks relying on the high bandwidth characteristics of the 3D DRAM.

The application discloses a quantum circuit cutting method and a related device, and belongs to the technical field of quantum computing. The method comprises the following steps: obtaining a quantum bit interaction graph of a quantum circuit to be cut, wherein a node in the quantum bit interaction graph represents a logical bit, and an edge represents a two-bit logical gate acting between two corresponding logical bits; based on the quantum bit interaction graph, a preset number of target subgraphs and corresponding cutting cost overheads are obtained by recursively constructing subgraphs, wherein the number of nodes in the target subgraphs is determined by a first number and is smaller than the first number, the first number is the number of nodes in the quantum bit interaction graph, the cutting cost overheads are determined by cutting overheads and routing overheads, the cutting overheads are determined by the number of cutting points, and the routing overheads are determined by the similarity between the corresponding subgraphs and the physical bit layout of the target quantum hardware; and the quantum circuit to be cut is cut based on the target subgraph with the minimum cutting cost overhead. According to the application, the fidelity of the execution result can be improved.

The invention provides a plastic architecture design and pruning method based on heterogeneous experts and a related device, and the method comprises the steps: constructing a multi-modal large model architecture with a multi-layer heterogeneous expert group, considering task difference and environment heterogeneity, enabling each group of experts to have different scales and calculation overhead, integrating the resource perception capability into a model training process, and obtaining a multi-modal large model architecture with a multi-layer heterogeneous expert group. The model actively perceives parameter resource limitation on the architecture level, on the basis, expert combinations are enumerated on a calibration set in the post-training pruning stage, redundant experts with low contribution degree are permanently discarded to reduce static parameter quantity, high-contribution experts are dynamically activated and low-contribution experts are skipped during reasoning according to input features and expert weights in the dynamic pruning stage, and the static parameter quantity is reduced. And reasoning efficiency is improved. According to the scheme, collaborative optimization is carried out from model architecture design to the reasoning process, the consumption of model parameters and computing resources is greatly reduced while the multi-modal task performance is guaranteed, and an innovative solution is provided for efficient deployment of a large multi-modal model in a resource-constrained environment.

The invention provides a processing optimization method, system and device of an AI accelerator and a medium. The method is applied to the AI accelerator in communication connection with a main memory. Through the collaborative architecture of the AI accelerator and the 3D DRAM, the transmission overhead and power consumption in the data carrying process are reduced, the computing throughput and the energy efficiency ratio are improved, and meanwhile large-scale parallel computing tasks are supported by means of the high-bandwidth characteristic of the 3D DRAM.

The application discloses a high-dimensional vector graph index construction method based on GPU acceleration, and comprises the following steps: obtaining a high-dimensional vector dataset, using the parallel computing capacity of GPU, and adopting a two-stage k-neighbor graph construction algorithm to quickly initialize a k-neighbor graph; accelerating various neighbor graph pruning strategies through a unified GPU acceleration framework, and designing special GPU kernel processing serial and unbalanced calculation to finally generate a neighbor graph index; for a large-scale dataset, proposing an asynchronous GPU-CPU-disk index framework and a cluster-aware cache mechanism, and adopting a multi-GPU parallel mode to construct a large-scale index and reduce I / O overhead in the index construction process. The application solves the problems of poor generality and unbalanced load of the existing GPU method, significantly reduces the high-dimensional vector graph index construction time and memory overhead, and can maintain high query accuracy and efficiency.

This invention relates to the field of cloud computing resource management technology and discloses an AI-based dynamic scheduling and operation and maintenance system for large-scale data center computing resources. The system includes a kernel status monitoring module, a scheduling priority quantification module, a resource configuration retrieval module, and a synchronous clock scheduling execution module. It collects the system call trajectory of the task to be scheduled in the kernel state and the synchronous blocking state of the network protocol stack, extracts runtime characteristic data representing the execution phase, and calculates the scheduling urgency index by combining the topological weights of the task's directed acyclic graph. Then, it matches the target control group quota and writes the target control group quota into the resource limit parameter file when the task enters the communication waiting window. This invention locks the resource quota update action within the task's logical rest period, effectively eliminating the kernel state context switching overhead caused by computing power scheduling, ensuring the logical clock consistency of large-scale parallel computing tasks, and improving the global turnover efficiency of the resource pool.

The invention discloses a self-adaptive hyperdimensional computing (HDC) accelerator supporting incremental learning, and belongs to the technical field of artificial intelligence hardware acceleration. The accelerator comprises a feature extraction and coding circuit, an incremental prototype-like storage resource pool, a parallel reasoning array and a confidence perception retraining circuit. Wherein the incremental prototype-like storage resource pool comprises a pre-allocated BRAM array and a write enable gating logic, and class expansion during operation is realized through logic-physical address decoupling; the confidence perception retraining circuit is integrated with a double-threshold comparator group and a barrel-shaped shift scaling circuit without a multiplier, and monitors the minimum Hamming distance in real time through a hardware comparator and drives a barrel-shaped shifter to dynamically generate an update weight. According to the method, low-overhead incremental learning is realized on the premise that hardware does not need to be synthesized again through resource isolation and adaptive weight adjustment of a pure hardware level, the disastrous forgetting phenomenon is effectively inhibited, and the retraining efficiency is remarkably improved.

The invention discloses a heterogeneous computing power pool task adaptive migration and dynamic scheduling method and system, and relates to the technical field of computer scheduling and artificial intelligence computing power. The heterogeneous computing power pool task adaptive migration and dynamic scheduling method comprises the following steps: S1, constructing a scheduling mapping basic data set; s2, analyzing an adaptive structure difference between a task resource request and a node resource state based on the scheduling mapping basic data set, and analyzing resource load fluctuation characteristics in a node operation process; s3, performing multi-dimensional matching optimal path judgment in combination with the scheduling stability level and the node scheduling resource state; and S4, executing a task container transfer and context recovery process according to the path channel information and the node resource binding structure. The problem that the actual reasoning efficiency is obviously lower than the theoretical calculation capability due to the fact that queuing time consumption and context switching overhead of the GPU instruction are not tracked during multi-round concurrent task calling is solved.

An operator fusion method and device, electronic equipment and storage medium. The operator fusion method comprises: writing a first graph layer representation corresponding to at least one first operator and a second graph layer representation corresponding to at least one second operator into a target kernel function, wherein the first operator and the second operator are called by a device function in the target kernel function; obtaining a first intermediate representation corresponding to the at least one first operator; obtaining a second intermediate representation corresponding to the at least one second operator; and based on the first intermediate representation and the second intermediate representation, fusing the at least one first operator and the at least one second operator to obtain a fused target kernel function. The operator fusion method can support operator fusion of any graph topology, different categories and different implementation modes (automatic generation or handwriting), thereby greatly reducing the overhead of kernel function startup, intermediate temporary buffer, data relocation and other computing resources, improving computing efficiency and enhancing hardware performance.

The invention provides a heterogeneous computing scheduling method, system and device and a storage medium, and relates to the technical field of artificial intelligence. In some embodiments of the present disclosure, according to a task descriptor, a page table mapping relationship of an input / output memory management unit is configured without depending on the participation of a system core, and address translation preprocessing is completed; predicting a virtual address interval subsequently accessed by the neural network processing unit; before the neural network processing unit initiates a memory access request, page table hardware traversal of the input / output memory management unit is triggered according to the virtual address interval to obtain page table entries, and the page table entries are loaded into an input / output conversion lookaside buffer; and sending a starting signal to the neural network processing unit, so that the neural network processing unit accesses the shared main memory through the input / output memory management unit based on the page table mapping relationship. Through dual-core division of labor of the system core and the control core, system overhead caused by participation of the operating system in underlying scheduling is reduced, and extra power consumption overhead is reduced.

This invention discloses a high-efficiency XMSS computation unit architecture design method supporting the SPHINCS+ protocol, relating to the field of hardware acceleration technology, and solving the problem of slow hardware computation speed of traditional SPHINCS+ XMSS signatures. Firstly, at the algorithm level, the Merkle tree construction logic is optimized by adjusting the key pair selected during signing and reducing the number of nodes to be computed during signing, thereby reducing the overall overhead of tree construction. Secondly, in terms of hardware architecture, a parallel processing approach using multiple hash chains is adopted, optimizing the overall pipeline layout. Furthermore, an innovative method of alternating dual-key pair input is used, effectively reducing the overall clock cycles required for key pair generation and signature computation, significantly improving hardware utilization. Experimental results show that this design achieves speedups of 3.86 and 3.87 for key generation and signature computation, respectively, on an FPGA platform. This invention has promising application prospects in the field of hardware acceleration of post-quantum cryptography algorithms supporting the SPHINCS+ protocol.

The application provides an OpenMP parallel computing method of a numerical model of underwater explosion LDG, and belongs to the technical field of computer data processing and computing. The application innovatively introduces the OpenMP parallel computing method into the LDG method to simulate underwater explosion shock cavitation load research. The OpenMP parallel technology is adopted to decompose the calculation-intensive task in the underwater explosion calculation problem to multiple cores for cooperative processing, and overcomes the defects of low calculation efficiency and limited calculation scale in the traditional serial computing method. The OpenMP parallel computing of the underwater explosion shock cavitation load is optimized, two independent auxiliary variable calculation tasks in each unit are combined into one OpenMP parallel domain for execution, the creation and destruction overhead of the parallel region is reduced, and the thread synchronization and management overhead is reduced. Meanwhile, the array structure of the storage numerical flux is improved, and the layout in the memory is optimized.

This invention provides a configurable hardware architecture and bit manipulation method based on Banyan networks, belonging to the fields of digital logic circuits and high-performance computing technology. The invention aims to solve the problems of high hardware overhead, high power consumption, and long latency caused by the need for separate hardware modules—a Banyan network and a barrel shifter—to achieve combined bit-gathering and cyclic shifting operations in existing technologies. The architecture of this invention includes a Banyan network switching module and a control signal generation module. The control signal generation module generates an offset prefix count by superimposing the cyclic shift amount *r* with the original prefix count based on a bit mask, and uses this count to generate control signals for each stage of the Banyan network via an LROTC circuit. Through innovative modifications to the Banyan network control logic, this invention enables a single network to simultaneously perform both advanced bit operations, bit-gathering and cyclic shifting. This invention offers significant advantages such as hardware resource savings, reduced power consumption, improved performance, flexible design, and easy expansion, and can be widely applied in processor design, cryptographic acceleration, digital signal processing, and other fields.

The embodiment of the invention provides a data processing method and device, a storage medium and a program product, and relates to the technical field of cloud computing. In the embodiment of the invention, aiming at segmentation of a matrix multiplication task, a method for heuristically determining block information is provided, corresponding block information of the matrix multiplication task in multiple matrix dimensions is generated on a first calculation unit according to matrix parameters and resource parameters of a second calculation unit, and the block information is determined according to the generated block information. According to the method and the device, the block information meeting the resource occupancy rate requirement can be determined under the condition that extra overhead is hardly introduced. According to the method and the device, the determined block information is provided for the second computing unit, so that the second computing unit can perform multi-dimensional segmentation on the matrix multiplication task according to the block information, and the segmented block tasks are executed in parallel, so that the resource occupancy rate of the second computing unit during execution of the matrix multiplication task is increased.