178 results about "Memory bandwidth" patented technology

The invention belongs to the technical field of artificial intelligence, and discloses an artificial intelligence model training resource adaptive distribution system. The method comprises the following steps: acquiring and calculating graph structure data and hardware resource state data in real time, and calculating a data reuse rate and generating a candidate operator fusion scheme by constructing an operator execution time sequence constraint matrix and identifying an operator cluster of data locality characteristics; a resource competition hotspot prediction mechanism is introduced, memory bandwidth occupation fluctuation characteristics are analyzed, a resource conflict probability is calculated for a fusion scheme, and a dynamic balance optimization model of fusion income and resource conflicts is constructed. An optimal operator fusion decision sequence and a resource allocation strategy are generated through iterative solution, and accurate dynamic adjustment of computing resources in the training process is achieved. The training efficiency and the resource utilization rate are improved, the energy consumption is reduced, and the system stability is enhanced.

The invention discloses a model performance test method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. The theoretical maximum lexical throughput of a target large language model is calculated based on the video memory bandwidth of a graphics processor, the model parameter quantity, the byte number corresponding to the quantization precision and the video memory bandwidth utilization rate; meanwhile, the benchmark performance throughput is obtained, a theoretical corresponding first concurrency number is calculated in combination with the theoretical maximum lexical unit throughput and the concurrency competition loss coefficient, then the model test is executed based on the first concurrency number to obtain the actual maximum lexical unit throughput and a corresponding second concurrency number, and a model performance test result is generated. The problems that in the prior art, due to the fact that manual testing is conducted depending on manual intervention, a continuous approaching attempt mode is adopted, a reasonable test starting point is not deduced in combination with hardware core bottlenecks and key parameters, evaluation is time-consuming and labor-consuming, the result is prone to being affected by artificial factors, and accuracy and consistency are poor can be solved.

The invention discloses a method for accelerating secure metadata access in a secure memory system, a memory controller and a system, and belongs to the field of secure memory systems, and the method comprises the following steps: when a page table item corresponding to a logic page where data accessed by a processor is located does not hit a TLB, obtaining the page table item from a memory page table, extracting a physical page address from the page table item, and storing the physical page address in a memory; a counter corresponding to a physical page where the data to be accessed is located and a father node of the counter in the integrity tree are prefetched through the physical page address; adding a replacement dirty block address in a miss request sent by the last level of cache, after receiving the miss request containing a field of the replacement dirty block address, executing conventional memory reading and decryption, positioning a counter corresponding to the replacement dirty block address, and performing prefetching by using an idle memory bandwidth; in addition to the secure metadata cache, the prefetching queue is maintained to temporarily store the prefetched metadata. The cache hit rate of the security metadata in the security memory system can be improved, and the performance overhead caused by the cache miss can be reduced.

The invention discloses a hierarchical self-adaptive full block pre-filling scheduling method and system for large language model reasoning, and the method comprises the steps: carrying out the hierarchical portrait analysis of a to-be-served model, and dividing the to-be-served model into partitions with different calculation characteristics according to the calculation intensity and memory access characteristics of each layer; then, making a layering and partitioning strategy based on a partitioning result, allocating a larger partitioning size to a calculation-intensive partition, allocating a smaller partitioning size to a memory bandwidth-intensive partition, and generating a layering and partitioning mapping table; and finally, when the online scheduling is executed, querying the mapping table according to the request processing progress to determine the block target size, and jointly forming a batch processing unit by the decoding task and the pre-filled block with the heterogeneous size under the constraint of the iteration time budget to be executed. According to the method, accurate matching of calculation and bandwidth resources is achieved, the system throughput can be effectively improved, tail delay and fluctuation thereof can be remarkably reduced, bubbles under pipeline parallelism are reduced, and the method is suitable for various attention mechanisms and distributed reasoning architectures.

The invention provides an attention mechanism optimization calculation method and device, and relates to the technical field of artificial intelligence, and the method comprises the steps: constructing a packaging mask tensor of a target batch based on the length information of a plurality of input sequences in the target batch; when attention weight calculation is executed on the target batch based on the calculation unit, the packaged mask tensor and the attention score tensor are calculated, and the attention score tensor after mask processing is obtained; and determining an attention calculation result of the target batch based on the attention score tensor after mask processing. According to the method, real-time dynamic judgment on the effectiveness of sequence elements in the attention calculation process is replaced by pre-constructing the packaged mask tensor, and complex conditional judgment logic is converted into simple tensor operation. According to the invention, the branch prediction overhead and thread differentiation in the calculation process are greatly reduced, the parallel processing efficiency of the calculation unit is improved, and the occupation of memory bandwidth is reduced, so that the calculation efficiency of the attention mechanism is improved.

The invention relates to the technical field of video card power management, and discloses a power management method, device and equipment for a video card and a storage medium. The method comprises the following steps: collecting time sequence data of SM unit frequency of a graphics card, GDDR video memory bandwidth and VRM output voltage in real time, and establishing a GPU power consumption topology model; based on the GPU power consumption topology model, extracting a time sequence feature vector of the load change of the rendering pipeline; constructing a sparse representation matrix based on the time sequence feature vector; performing power consumption gradient threshold judgment and delay type classification by using the sparse representation matrix, and identifying a delay fault type in the current graphics card power management system; and adjusting a switching parameter of the GPU, a VRM compensation current and a fan rotating speed according to the identified delay fault type, and updating a delay detection threshold value and a repair strategy strength parameter. According to the invention, three different delay fault types of frequency climbing, voltage regulation and temperature control current limiting can be accurately distinguished, and continuous optimization of power management performance is realized.

The present invention relates to a processor designed to optimize memory bandwidth utilization for tensor transpositions in machine learning. An example processor includes an input tensor shift buffer, a staging buffer, and an output tensor shift buffer. The input tensor shift buffer reads an input tensor from input memory and performs multiple cycles of input tensor shifting. The shifted tensor data is then written into the staging buffer. The output tensor shift buffer reads the shifted tensor data from the staging buffer and performs multiple cycles of output tensor shifting. Finally, the result is written to the output memory. This configuration facilitates efficient handling and transformation of tensor data, optimizing the computational processes required in machine learning tasks.

The invention provides a memory access system oriented to vector and matrix calculation, the system comprises a vector memory access unit, a matrix memory access unit, a vector register group and a matrix register group, the vector memory access unit is connected with a memory interface and the vector register group, reads elements of a one-dimensional data structure or a two-dimensional data structure from a memory, and stores the elements of the one-dimensional data structure or the two-dimensional data structure; vector data are generated through data reorganization operation, a matrix access unit is connected with a memory interface and a matrix register set, matrix block data of a two-dimensional data structure are read from a memory together with the matrix access unit, matrix data are generated after data reorganization, and the matrix data are broadcasted to one or more computing units according to rows or columns. And performing calculation on the matrix data and the vector data. In the memory access system, the vector memory access unit and the matrix memory access unit can load data in parallel, and the utilization rate is improved through a plurality of computing units, so that the problems of low memory access efficiency and low memory bandwidth utilization rate are solved.

The first token prediction of a large language model is bottlenecked by compute and second token predictions onwards are bottlenecked by memory bandwidth. Inferences can be made more efficient through activation sparsity. An activation tensor is pruned using an importance threshold value. The mode of the activation tensor is centered in a lossless manner using an estimated mode value to improve activation sparsity further. Pruning and mode-centering mechanisms can be inserted into a neural network strategically and post-training to implement sparsification. A two-stage greedy grid search algorithm is implemented to determine the calibrated importance threshold values of various pruners and the estimated mode values using a small dataset. A modified neural network with pruning and lossless mode-centering can be deployed onto hardware.

The invention belongs to the field of Transform model acceleration, and discloses a hybrid bonding 3D stacked accelerator and acceleration method oriented to Transform reasoning, the accelerator adopts a 3D stacked architecture, and comprises a DRAM storage bare chip and a logic bare chip, the DRAM storage bare chip comprises 16 DRAM storage blocks, the logic bare chip comprises 16 processing groups, each processing group corresponds to one DRAM storage block, and the processing groups correspond to the DRAM storage blocks. Each processing group is integrated with two attention processing units, and the DRAM storage blocks are connected with the processing groups through 32 copper-copper hybrid bonding vertical channels. According to the method, the problems of dense calculation and high memory bandwidth requirement in the reasoning process are solved, efficient model reasoning acceleration is realized, and the method is suitable for natural language processing, computer vision, video analysis and other application scenes depending on the Transform model.

Providing memory bandwidth compression using compressed memory controllers (CMCs) in a central processing unit (CPU)-based system is disclosed. In this regard, in some aspects, a CMC is configured to receive a memory read request to a physical address in a system memory, and read a compression indicator (CI) for the physical address from a master directory and / or from error correcting code (ECC) bits of the physical address. Based on the CI, the CMC determines a number of memory blocks to be read for the memory read request, and reads the determined number of memory blocks. In some aspects, a CMC is configured to receive a memory write request to a physical address in the system memory, and generate a CI for write data based on a compression pattern of the write data. The CMC updates the master directory and / or the ECC bits of the physical address with the generated CI.

The invention relates to the technical field of server testing, in particular to a server multi-core computing processor load optimization testing method, device and equipment and a medium, and the method comprises the following steps: automatically identifying hardware configuration information of a server; generating a multi-dimensional pressure test plan based on the hardware configuration information; executing the test plan, and monitoring the load rate, the memory bandwidth, the cache hit rate and the I / O performance index of each CPU core in real time; dynamically adjusting a task allocation strategy based on a set load difference threshold to realize task dynamic optimization, and if the difference between the maximum core load rate and the minimum core load rate is monitored to exceed the threshold, triggering a task migration operation; system performance data are collected again, performance indexes before and after optimization are compared and analyzed, and system performance bottlenecks are recognized; and automatically generating a performance test report. The utilization rate and the overall task throughput of the multi-core processor are improved, and the problem that a static task allocation strategy is difficult to adapt to dynamic load changes is solved.

The invention belongs to the technical field of model simulation, and particularly relates to an efficient dynamics simulation analysis method based on a Fourier neural operator. The present invention proposes FNO-Speed, and a series of comprehensive solutions for inefficient operations that the FNO solver does not fully utilize hardware. According to the method, two unique optimization methods are adopted, and comprise a multi-level parallel implicit image-to-column general matrix multiplication optimization strategy and a user-defined size high-frequency signal filtering algorithm. According to the method, efficient general matrix multiplication is achieved through an implicit image-to-column and data division strategy to replace pointwise convolution, and fragmentary calculation of frequency domain local linear transformation is eliminated through the latter. The FNO-Speed makes full use of the memory bandwidth, improves the calculation efficiency, and aims to solve the problems of low utilization rate of calculation resources and delay influence caused by large-scale data access calculation.

The invention relates to the technical field of artificial intelligence, and provides a data processing method and device, a medium and a program product.The method comprises the steps that a target calculation task is executed on input data, and a first precision result is generated; before the first precision result is written into a global memory, calculating local absolute value maximum values corresponding to all result blocks in the first precision result in parallel, and determining a global absolute value maximum value in the first precision result based on the local absolute value maximum values of all the result blocks; and based on the global absolute value maximum value, calculating to obtain a scaling factor, and performing quantization processing on the first precision result by using the scaling factor to obtain a second precision result. Before the first precision result is written into the global memory, the on-chip memory is used for calculating the maximum value of the local absolute value of the first precision result, and redundant memory read-write operation on the first precision result is effectively avoided, so that memory bandwidth occupation and processing delay are remarkably reduced, and the overall execution efficiency is improved.

The invention discloses a reasoning optimization method of a pre-training model and electronic equipment, and relates to the technical field of reasoning of the pre-training model.The priority of each subtask is determined on the basis of load data of multiple task stages of the pre-training model.The calculation units are allocated to the subtasks according to the priorities of the subtasks, and therefore the calculation efficiency of the subtasks is improved. By dynamically decoupling each task stage, efficient allocation of computing resources is realized, and the resource utilization rate is improved. And on the other hand, the first data of the two adjacent task stages are transmitted through a double-buffering mechanism, the problem that the inter-layer data transmission efficiency is low in the related technology is solved, and the hardware bandwidth utilization rate is increased. Therefore, the technical problems of low utilization rate of idle memory bandwidth resources and low interlayer transmission efficiency are solved, and the technical effect of improving the resource utilization rate and the bandwidth utilization rate is achieved.

The invention relates to the technical field of data storage, and discloses a multi-platform dirty disk performance consistency testing method for a solid state disk, which comprises the following steps: acquiring configuration information of a to-be-tested solid state disk and recording a SMART data baseline; the method comprises the following steps: initializing a solid state disk, performing steady-state preprocessing in an empty disk state of the initialized solid state disk, and recording performance data at the moment as an empty disk reference; filling the dirty disk proportion of the solid state disk to a target value in a sequential or random write-in mode, applying a corresponding dynamic load model under the dirty disk proportion, and continuously performing performance sampling, recording performance data and generating a performance attenuation curve; and starting a CPU pressure test, a memory bandwidth test and a GPU load test at the background while running the solid state disk performance benchmark test. According to the SSD performance compatibility testing method and system, cross testing is carried out on two mainstream platforms of AMD and Intel, and the performance compatibility of the SSD in different system environments can be systematically evaluated.

Devices, methods and systems for managing resources in a computing device. Information regarding resource usage is captured. A prediction is generated, based on the information, that resource usage by a processor will exceed a threshold during an upcoming time. An operating parameter of the processor is adjusted, based on the prediction. In some implementations, information regarding memory bandwidth is captured. A prediction is generated, based on the information, that a memory region stored in a first memory device will be addressed by a memory intensive instruction during an upcoming time period. Data stored in the memory region is moved to a second memory device, based on the prediction.

The embodiment of the invention relates to the field of artificial intelligence, and discloses a reasoning method and device based on a large language model, electronic equipment and a storage medium. The method comprises the following steps: inputting a last token of a first token set into a prediction module, and outputting a second token set; carrying out parallel calculation on the second token set in a decoder layer, and outputting the next reasoning token of each token in the second token set; and inputting all the reasoning tokens into a prediction result decision device, for each prediction branch sequence in the second token set, matching the next reasoning token of the tokens in the sequence with the next token stage by stage from the first stage, and outputting the longest token sequence obtained by matching as a third token set by the prediction result decision device. Through a prediction-parallelization-judgment process, the problem of video memory bandwidth bottleneck caused by the fact that a large language model calculates tokens one by one is solved.

The application discloses a file unshelling method and electronic equipment, and relates to the technical field of computer security, and comprises the following steps: inserting a probe at a target system call of a kernel, and injecting the probe into a target program; identifying a suspiciously shelled process through the target program; monitoring a memory permission change behavior performed by the suspiciously shelled process through the target program, and counting a permission change frequency of a memory region, an entropy value of the memory region and a call chain depth; when it is confirmed that the suspiciously shelled process is in an unshelling stage according to the permission change frequency, the entropy value and the call chain depth, writing data decrypted by the suspiciously shelled process into a target buffer through the target program; reading the data from the target buffer in a user mode, and recombining the data into a memory image according to a base address and a dirty page bitmap and writing the memory image back to a file. The method can avoid I / O bottlenecks caused by full memory dumping, reduce invalid data in the user mode, and reduce CPU occupation and memory bandwidth.

Motion compensation requires a significant amount of memory bandwidth, especially for smaller prediction unit sizes. The worst case bandwidth requirements can occur when bi-predicted 4×8 or 8×4 PUs are used. To reduce the memory bandwidth requirements for such smaller PUs, methods are provided for restricting inter-coded PUs of small block sizes to be coded only in a uni-predictive mode, i.e., forward prediction or backward prediction. More specifically, PUs of specified restricted sizes in bi-predicted slices (B slices) are forced to be uni-predicted.

The invention discloses a data processing system, a data processing method and related equipment, and relates to the technical field of data storage. The data processing system comprises a first node and a second node, wherein the first node comprises a local storage space and a shared storage space which are not overlapped. The first node is used for acquiring first data, and when the shared storage space of the first node is sufficient to store the first data, the first data is written into the shared storage space of the first node and the shared storage space of the second node, and the shared storage spaces of the two nodes are both used for persistently storing the data. Since the first node directly writes the first data into the shared storage space of the plurality of nodes, the computing power consumption can be effectively reduced, and the memory bandwidth occupation is reduced. Moreover, the first node does not need to execute the process of applying for the storage space for storing the data, so that the computing power consumption generated by executing the process of applying for the storage space for storing the data each time can be avoided, and the overall performance of the data processing system is improved.

The embodiment of the application discloses a kind of method and device for monitoring memory bandwidth, it is related to computer field.The method comprises: the memory bandwidth size that process group is accessed volatile memory is used in the preset period is obtained by processor core, and after the memory operation that process group executes in the preset period is counted, the memory bandwidth size that process group is accessed non-volatile memory is used is calculated according to memory bandwidth size and the memory operation of counting.It is realized to monitor the memory bandwidth of non-volatile memory with process group as granularity, and then the memory bandwidth size of non-volatile memory is used as the basis for limiting the memory bandwidth of non-volatile memory.

The application discloses a lightweight method of river surface flow velocity field depth reconstruction model, relates to the computer science and fluid mechanics cross technical field, collects river surface PTV and PIV data, and constructs a flow velocity field matrix of the target river surface; based on the matrix block calculation strategy and the flow velocity field matrix, the target river surface is divided into a plurality of sub-matrix regions; based on the pattern recognition technology and the flow velocity gradient characteristics, each sub-matrix region is distinguished. The application divides the global flow velocity field into a plurality of sub-matrix regions through the matrix block strategy, adaptively determines the optimal block size according to the GPU parallel thread number, memory bandwidth and other constraints, maximizes the hardware utilization rate, and in the calculation process, adopts a differentiated resource allocation mechanism, concentrates high-precision calculation in a high-error clustering area, and adopts a simplified model in a low-error clustering area, significantly reduces the overall calculation load, and realizes efficient balance between calculation efficiency improvement and resource dynamic scheduling.

The application provides a memory expansion card, which can be applied to the technical field of memory expansion. The memory expansion card comprises a bandwidth mode controller, a routing control module and a plurality of interconnection controllers. The bandwidth mode controller is used for determining a memory bandwidth configuration mode of a host according to a received host control signal, attribute information of a plurality of memory modules and attribute information of a plurality of storage controllers, and generating a memory switching instruction according to the memory bandwidth configuration mode and the attribute information of the plurality of interconnection controllers. The routing control module comprises a link switching submodule and the plurality of interconnection controllers. The link switching submodule is used for connecting the memory link among the host, the interconnection controllers and the storage controllers according to the memory switching instruction, so that the interconnection controllers generate interconnection control signals according to the host data signal and the memory switching instruction. The plurality of storage controllers are used for determining target memory modules according to the interconnection control signals, so that the target memory modules receive and store host data in the host data signal.

The application discloses a low-delay streaming voice wake-up method and device for embedded terminals, which is applied to the technical field of data processing and adopts GRU as a backbone architecture in view of the low-power-consumption and high real-time requirement of voice wake-up of embedded terminals. Through time-domain lightweight preprocessing, the original audio is split and reconstructed and mapped through a linear layer to eliminate the FFT overhead. A multi-layer GRU full-causal streaming coding network is built, which has no sliding window cache and occupies O(1) memory bandwidth. A Token probability distribution containing a Blank is output through a CTCLoss training module, and a target Token probability is extracted. A target index sequence is constructed based on a target wake-up word and a CTC protocol, three types of state transition rules are set to generate anti-interference decoding information. A Top-K pruning decoding mechanism is not used, and a decay factor and a threshold are combined to determine the judgment result, which has constant calculation complexity and low false wake-up and missing wake-up, and the subsequent interaction is triggered if the judgment result is successful.

The invention discloses a video frame rendering method, system and device, a storage medium and a program product, and belongs to the technical field of video processing. And a decoder in the CPU transmits the reference frame data to a reference frame cache region of the GPU. And a graph renderer in the GPU renders the reference frame according to the reference frame data. And a decoder in the CPU generates residual data according to the pixel value difference between the target frame and the reference frame, and transmits the residual data to a residual cache region of the GPU. And a graphic renderer in the GPU renders a local image area of the target frame according to the residual data, and superposes the local image area of the target frame with the reference frame to obtain a rendered image of the target frame. The technical effects of reducing GPU computing resources and memory bandwidth consumption, reducing data transmission delay and improving rendering efficiency and resource utilization rate in a high-resolution and high-frame-rate scene are achieved.

A method for machine learning includes inferencing a machine learning model using a hardware accelerator of an electronic device. The method includes monitoring bandwidth of memory of the electronic device during the inferencing, the memory being separate from the hardware accelerator. The method includes determining a current bandwidth of the memory corresponds to a threshold bandwidth based on the monitoring. The method includes adjusting a frequency for the hardware accelerator based on the determining.

The invention discloses a three-dimensional memory array which comprises a substrate and a laminated structure stacked on the substrate, and further comprises channel holes penetrating through the laminated structure, the channel holes are perpendicular to the substrate, the multiple channel holes are arranged on the upper surface of the laminated structure in a regular hexagonal honeycomb shape, and multiple bit lines are further vertically arranged on the upper surface of the laminated structure; the included angle between the connecting line between the uppermost angle and the lowermost angle of the regular hexagon formed by the plurality of channel holes and the bit line is theta, and the calculation formula is shown in the specification; in the formula, d is the pitch between two adjacent bit lines, and s is the pitch between two adjacent channel holes. In addition, the invention further discloses a memory and electronic equipment, the whole laminated structure is rotated by theta degrees on the substrate, so that the arrangement density of the bit lines is not limited by the process of channel holes, the densest bit line layout in the process allowable range is obtained, and higher memory bandwidth is realized.

The present invention provides a hybrid precision MAC (multiple-and-multiple-accumulation) tree structure for maximizing memory bandwidth usage to accelerate the operation of a generative large scale language model. The present invention provides a hybrid precision MAC (multiple-and-multiple-accumulation) tree structure for maximizing memory bandwidth usage to accelerate the operation of the generative large scale language model. A MAC tree-based arithmetic unit according to one embodiment may include: a plurality of floating point multipliers connected in parallel and processing multiplication of data transferred from an external memory; a plurality of first converters for converting the output of each of the plurality of floating point multipliers from a floating point to a fixed point; a fixed-point adder tree which is connected to the plurality of first converters and processes the addition of the multiplication results of the plurality of floating-point multipliers; a fixed-point accumulator that accumulates the output of the fixed-point adder tree; and a second converter that converts the output of the fixed-point accumulator from a fixed point to a floating point.

The invention provides a scheduling method and device based on a memory broadband. The method comprises the following steps: receiving memory broadband margin information of a target host machine sent by the target host machine and a memory broadband load occupied by an instance on the target host machine; the memory broadband margin information and the memory broadband load are obtained based on monitoring of the target host machine; and sending the memory broadband margin information and the memory broadband load to a scheduling system, so that the scheduling system schedules the to-be-scheduled instance according to the memory broadband margin information and the memory broadband load, thereby scheduling the to-be-scheduled instance according to the memory broadband margin information on the target host machine and the memory broadband load occupied by the instance on the target host machine. The overall dynamic optimal scheduling is performed on the instance, so that the dependence of the whole host machine cluster on the memory broadband is effectively reduced, and the instance scheduling cost and the consumption of system resources in the instance scheduling process are further reduced.