1026 results about "Video memory" patented technology

The invention discloses a big language model reasoning optimization method and system based on dynamic video memory compression and memory isomerism, and intelligent management of video memory resources is realized by integrating a dynamic compression strategy of KV Cache and a memory parallel architecture. The method comprises the following steps: 1) analyzing the spatial-temporal characteristics of the KV Cache in real time, adaptively selecting a quantization compression algorithm, a rarefaction algorithm or a low-rank decomposition algorithm, performing hierarchical storage based on attention head importance scores, keeping high precision of a core head, and implementing low-bit quantization on a secondary head; (2) the compressed inactive data are divided into a plurality of data blocks to be stored in a system memory, a parallel data channel group is established according to the number of physical channels, the compressed blocks are concurrently read through multiple channels during loading, and parallel decompression of a sparse matrix is accelerated through a GPU tensor core; and 3) constructing a KV Cache multiplexing mechanism and a parallel channel, and parallelizing a compression / decompression process and model calculation by adopting a hardware acceleration compression and asynchronous pipeline mechanism.

The invention provides a GPU computing power scheduling method based on a one-cloud multi-core heterogeneous computing power platform, and the method comprises the following steps: S1, carrying out heterogeneous resource registration and modeling, accessing hardware equipment containing multiple types of GPUs through a resource registration module, collecting the equipment model, the video memory capacity and performance index data, and carrying out heterogeneous resource modeling; constructing a resource feature database containing a topological relation, and supporting hybrid access of chips; s2, virtualized resource reconstruction: pooling a physical GPU into virtual GPU resources by adopting a hardware abstraction layer technology, realizing video memory isolation and calculation unit division through a containerization technology, and configuring each virtual GPU instance with an independent drive stack and a security sandbox; and S3, submitting a multi-modal task, receiving a CUDA / OpenCL calculation task submitted by a user, analyzing task demand parameters including a calculation core number, a video memory occupation amount and a data throughput threshold, and generating a task descriptor containing a priority label.

The invention discloses a multi-model collaborative operation method based on large-model efficient training, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly, monitoring model training resource indexes, such as activation tensor memory occupation and gradient calculation intensity, and carrying out the optimization through a three-stage optimization mechanism; constructing a heterogeneous resource space-time diagram to predict resource requirements, and combining multi-modal feature dynamic fusion and prediction error online compensation to improve prediction accuracy; then dynamically allocating resources, determining a migration priority by evaluating a training income gradient, an energy consumption efficiency factor and the like, and implementing video memory block management and pipeline scheduling; carrying out cross-model hierarchical knowledge distillation and self-adaptive cooperative training, and adjusting a cooperative mode according to model similarity; the problems of heterogeneous model resource competition and isolated knowledge precipitation are effectively solved, the resource utilization rate is improved, repeated calculation is reduced, the overall performance and robustness of multi-model collaborative operation are enhanced, the training cost is reduced, and the training efficiency is improved.

The invention relates to the technical field of GPU resource dynamic scheduling, in particular to a cloud environment GPU resource dynamic scheduling method, which comprises the following steps: monitoring operation indexes of GPU instances in a cloud environment in real time, collecting GPU video memory allocation records, GPU video memory fragmentation degree magnitude and page migration activity count, and predicting task resource demand total quantity through statistics. And generating a video memory state evaluation result. According to the method, the resource pool adjustment decision and the task matching degree list are jointly optimized, the instance models and the number are screened under the double constraints of the instance starting time parameter and the release cost parameter, and operation delay and extra cost caused by blind capacity expansion and contraction are reduced. Based on a strong binding mechanism started by a video memory pre-allocation instruction and a task process, in combination with continuous tracking of a resource consumption trajectory, a video memory peak deviation and an execution time deviation are fed back in real time, and a closed-loop scheduling optimization link is formed.

The invention discloses a method for automatically drawing an OpenGL (Open Graphics Library) program by using Vulkan. The method comprises the following steps of: creating a context used by the Vulkan, initializing each module, processing an OpenGL instruction related to texture and data buffering, and managing storage of texture and data buffering resources in a video memory; a shader program used by the OpenGL is preprocessed into a format acceptable to Vulkan, and an OpenGL shader program instruction is created and destroyed; processing an OpenGL (Open Graphics Library) instruction related to frame buffering to generate structural body information required by Vulkan dynamic rendering; an OpenGL instruction of the sampler is also created; processing an OpenGL (Open Graphics Library) instruction for creating a vertex input format and managing a vertex data buffer area, and maintaining vertex input information, a vertex buffer area and an index buffer area required by Vulkan; and finally, drawing or calculating, distributing and calling Vulkan on the basis of all the instructions.

The invention relates to a multi-modal large model dynamic compression and reasoning optimization method based on a MoE architecture. The method comprises the following steps: establishing an edge computing system conforming to medical equipment specifications, constructing a medical image analysis network based on an improved hybrid expert MoE architecture, and adopting a three-layer cascade structure of a feature coding layer, a dynamic routing layer and an expert execution layer; executing expert module dynamic loading and video memory optimization; executing knowledge graph compensation and domain knowledge injection; executing hardware instruction level optimization and calculation acceleration; executing multi-expert feature fusion and decision weighting; performing diagnosis result generation and confidence evaluation; performing real-time data return and model iterative optimization; executing multi-device cooperation and load balancing; executing system security monitoring and exception handling; and generating a structured diagnostic report. The problem that the precision loss of a multi-modal large model is difficult to meet actual requirements is solved, and medical feature adaptive dynamic compression, medical hardware collaborative energy efficiency optimization and cross-modal compensation of medical knowledge enhancement are realized.

The invention discloses a resource scheduling method for dynamic rendering and related equipment, and relates to the technical field of dynamic rendering, and the method comprises the following steps: obtaining time sequence data of a user touch event; based on the time sequence data, predicting a user behavior state through a behavior prediction model, and determining a current resource demand level; dynamically adjusting a resource allocation strategy and a video memory management strategy of the graphic processing unit according to the resource demand level; and executing a rendering task based on the adjusted resource allocation strategy, and performing picture output by adopting a double-buffer architecture so as to reduce delay caused by time sequence mismatch. According to the method and the device, by establishing closed-loop control of a user behavior-resource demand-supply strategy, accurate scheduling and efficient utilization of resources are realized, the problems of time sequence mismatch, resource waste and jamming in the traditional technology are solved, the rendering efficiency and the user experience are improved, and technical support is provided for high-concurrency and low-delay scenes.

The invention relates to the technical field of graphics card power consumption adjustment, and discloses a method, a device and equipment for dynamically adjusting the power consumption of a graphics card component. The method comprises the following steps: carrying out multi-channel data acquisition and working load characteristic analysis on operation parameters of a graphics card GPU core, a video memory and a power management unit to obtain a component temperature state evaluation matrix; executing component-level power consumption distribution based on multi-objective optimization to obtain a power consumption distribution scheme of each component; inputting the power consumption distribution scheme of each component into a power consumption-frequency-voltage mapping model, and calculating an optimal frequency voltage configuration parameter through a binary search algorithm to obtain a frequency voltage control instruction; performing multi-step prediction to obtain a module power consumption demand prediction value; according to the method, the frequency voltage combination with the highest performance efficiency can be found under the condition that the power consumption constraint is met, and accurate power consumption control is achieved.

The invention relates to the technical field of data analysis, and discloses a GPU computing power resource scheduling method and system, and the method comprises the steps: collecting node hardware parameters and dynamic load indexes of a GPU cluster to construct a multi-dimensional resource feature vector of the GPU cluster, and constructing a resource portrait of the GPU cluster; establishing a node health degree scoring model of the GPU cluster, and generating a health degree score of a cluster node corresponding to the GPU cluster; analyzing a video memory demand of the GPU task request, and calculating an intensive identifier and a communication dependency relationship; determining the SLA weight of the GPU task request, calculating the resource shortage sensitivity of the GPU task request based on the video memory demand, and calculating the target task priority of the GPU task request in combination with the SLA weight; and determining a resource scheduling node group requested by the GPU task in the resource portrait, generating resource scheduling parameters of the resource scheduling node group, and executing scheduling of computing power resources of the GPU cluster based on the resource scheduling parameters. According to the method, the scheduling efficiency of the GPU computing power resources can be improved.

The invention provides a game cartoon character display method based on a virtual reality technology. The method comprises the following steps: acquiring three-dimensional model data of a game cartoon character in a target display area through a user interaction terminal; the virtual reality content management platform determines a dynamic rendering precision level according to the model data, and generates a real-time rendering strategy including a model patch reduction coefficient, a texture compression rate and a skeleton animation updating frequency in combination with terminal performance parameters; the strategy is sent to a virtual reality supervision platform and a user interaction terminal, and a virtual reality rendering engine platform is instructed to execute real-time rendering; and the virtual reality supervision platform monitors the frame rate fluctuation data of the head-mounted display device, calculates the scene rendering stability, sends a rendering optimization instruction if the scene rendering stability is lower than a threshold value, and adjusts the model data acquisition frequency and the video memory cleaning period to optimize the performance. The rendering efficiency and the system stability can be improved, and the hardware load and the frame rate fluctuation are reduced.

The invention provides a GPU heterogeneous computing resource allocation method and system based on task scheduling, and the method comprises the steps: carrying out the feature analysis of a to-be-scheduled task, so as to extract the calculation intensity, video memory demands, inter-task data dependence and historical execution data of the to-be-scheduled task, and obtaining a task feature vector; constructing a performance model of heterogeneous GPU resources to predict execution efficiency and resource consumption of the GPU under the target task based on hardware performance parameters and running states of the GPU; and matching the to-be-scheduled task with the GPU resources according to the task feature vector and the performance model of the heterogeneous GPU resources. Through task feature analysis and GPU resource performance modeling, the adaptive relation between tasks and GPU resources can be deeply mined, to-be-scheduled tasks are accurately allocated to the most appropriate GPU, the resource utilization rate is greatly increased, the task execution sequence and resource allocation are dynamically planned, and the task execution time is effectively shortened.

The invention provides an image rendering method and device, electronic equipment and a storage medium, and the method comprises the steps: calculating a distance between a virtual image obtaining device and a to-be-rendered object through determining the to-be-rendered object in a to-be-rendered scene, and determining the distance as a first loading factor; determining a rendering picture pixel proportion of each to-be-rendered object based on preset view cone size information, determining the rendering picture pixel proportion as a second loading factor, generating a rendering loading priority sequence according to the first loading factor, and determining a model level resource file and a texture level resource file according to the second loading factor, loading the resources to a video memory for rendering according to the sequence; the priority is determined by calculating the distance between the camera and the to-be-rendered object, it is ensured that a high-quality model and texture of a close-range object are loaded preferentially, the instantaneity of the rendering loading process is improved, texture and model grade division is conducted through the pixel proportion of the to-be-rendered object in the picture, and the rendering efficiency is improved. The occupation of a video memory is reduced, and the overall rendering performance of the system is improved.

The invention provides a deep learning training and reasoning task dynamic cooperation system based on GPU space-time resource sharing, and the system comprises a GPU resource state perceptron which is used for monitoring a GPU kernel function call sequence and a video memory distribution state of a distributed training task in real time, dynamically capturing a calculation gap and video memory fragments generated by the training task, and transmitting the calculation gap and the video memory fragments to the GPU resource state perceptron; generating a two-dimensional resource spatial-temporal characteristic spectrum, and predicting a GPU calculation idle period caused by communication synchronization based on an LSTM model; the kernel function dynamic scheduling decision maker is used for performing priority division and dynamic resource quota allocation on an online reasoning task and an offline reasoning task by adopting a self-adaptive allocation strategy on the basis of a resource spatial-temporal characteristic spectrum so as to realize spatial-temporal resource decoupling of the training task and the reasoning task; and the kernel function execution arbiter is used for dynamically controlling submission and blockage of the reasoning task kernel function according to the scheduling decision through video memory space multiplexing and a calculation instruction arbitration mechanism. According to the method, the GPU resource utilization rate is remarkably improved, on the premise that stable training task performance is guaranteed, fragmented resources are effectively utilized to support parallel execution of multiple types of reasoning tasks, and efficient resource collaboration of a deep learning task cluster is achieved.

The invention provides an end-side multi-modal large model accelerated reasoning method and system, and the method comprises the steps: carrying out the two-stage screening and rearrangement of visual tokens based on the CLS attention and text-to-visual attention in a visual encoder and pre-filling stage, and constructing a sparse attention and sparse key value cache; in a decoding stage, an important neuron set is judged according to activation gating or historical statistics, only a corresponding feedforward network weight is pulled and calculated, missed weights are loaded on demand through asynchronous I / O, and hot neurons are maintained in a high-speed memory to utilize model sparsity, so that video memory / memory occupancy and calculation overhead are remarkably reduced on an end side; throughput and time delay performance are improved. According to the method, the internal memory and computing resources required by reasoning of the multi-modal large language model are reduced from two dimensions by utilizing the endogenous sparsity of the end-side large language model in input and the model, so that a higher reasoning speed is achieved by utilizing fewer resources on the premise of keeping the size of the model unchanged, and the performance of the whole system is improved.

The invention discloses a resource allocation method and device for a large model cluster, equipment, a storage medium and a program product. Comprises: determining configuration parameters based on a configuration task selected by a user; determining at least one of a computing power demand, a bandwidth demand and a video memory demand based on the configuration task and the configuration parameters; constructing a constraint condition and a target function according to at least one of the computing power demand, the bandwidth demand and the video memory demand; and carrying out iterative optimization on the target function by utilizing a preset planning algorithm based on the constraint condition so as to solve the unknown configuration parameter and obtain a target configuration parameter. According to the resource allocation method for the large model cluster, provided by the embodiment of the invention, the optimal large model learning parameter or hardware resource allocation can be determined, and the efficiency and reliability of resource allocation of the large model cluster are improved.

The invention discloses a two-way staggered assembly line parallel method for accelerating large model training, and compared with an existing method, the method introduces V-shaped staggered scheduling into a two-way assembly line to form a two-way staggered assembly line parallel mode, and improves the training efficiency of a large model and the use balance of a video memory. A V-shaped staggered assembly line and an inverted V-shaped staggered assembly line are fused to form a bidirectional staggered assembly line, the number of parallel devices is increased, meanwhile, the calculation time of a single micro-batch is shortened, the cavitation rate of the assembly line can be effectively reduced, and more balanced video memory occupation is achieved; in addition, a communication optimization strategy is introduced for overlapping or reducing part of communication overhead, and global gradient synchronization is performed by optimizing pipeline equipment mapping and utilizing a high-speed network in a server, so that the parallel training efficiency of the large model is further improved. The invention further provides a two-way interleaving assembly line parallel system and equipment which are used for achieving the two-way interleaving assembly line parallel method.

The invention relates to a resource-limited large model heterogeneous training method, computer equipment and a storage medium, and the method comprises the steps: determining a first activation amount according to a dependency relationship between calculated amounts corresponding to the activation amounts and / or the calculated amount and the storage amount of each activation amount in the activation amounts generated when an image processor carries out model forward propagation, the first activation amount is asynchronously transmitted to the central processing unit; loading a first activation amount to the image processor from the central processing unit in a model back propagation process; obtaining the gradient of the model parameter calculated by the image processor based on the first activation amount, and asynchronously transmitting the gradient to the central processing unit; and determining an optimizer obtained based on gradient updating in the central processing unit, obtaining model parameters obtained by adjustment of the updated optimizer, and asynchronously transmitting the model parameters to the image processor. By adopting the method, the problems that a large model with limited video memory resources is difficult to train and the computing efficiency of a video card is low can be solved.

The invention provides a cache management method, a cache management device, a computer storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps of receiving a reasoning task request and distributing the reasoning task request to a target storage page; key value cache information of the first round of reasoning task is stored in a hard disk cache, and when the second round of reasoning task is executed, key value cache information generated before the second round of reasoning task is preloaded layer by layer from the hard disk cache; when the last round of reasoning task is received, storing first target key value cache information correspondingly generated by the last round of reasoning task into the matched target physical block; and performing hybrid grouping compression on key cache information and value cache information in the first target key value cache information to obtain second target key value cache information after quantization compression. According to the invention, triple balance of video memory-calculation performance-precision can be realized.

The invention discloses a big language model reasoning service optimization method, system and device and a medium. The method comprises the steps of obtaining a current reasoning request queue; performing dynamic sorting on each reasoning request in the current reasoning request queue; and performing dynamic batch processing on each reasoning request in the reasoning request queue by using an SLO-BS algorithm to obtain a current batch of reasoning requests, and inputting the current batch of reasoning requests into a large language model, the method, the system, the equipment and the medium can solve the problem of video memory fragmentation occurring during multi-task concurrent reasoning, and reduce the SLO violation rate at the same time.

The invention provides a computing card group scheduling method and device based on task perception and generation length prediction, and the method comprises the steps: sending a plurality of reasoning requests with user prompt words to a computing card group, packaging each reasoning request as a request object, and enabling the computing card group to have a plurality of intelligent computing cards, the video memories of all the intelligent computing cards form a shared video memory pool; adding the request object into a user request pool; performing generation length prediction on the user prompt word of the request object in the user request pool by adopting a predictor module to obtain a predicted generation length; creating a corresponding reasoning task and metadata for each request object in the user request pool, applying for a video memory block as required from the shared video memory pool according to the predicted generation length, obtaining a video memory block handle with a video memory allocation address and size, and writing the metadata into the video memory block handle; generating a task scheduling sequence according to the metadata of the reasoning task and the running state of each computing card; and reasoning tasks are selected from the task scheduling sequence in sequence.

The invention provides an expert access prediction method and system suitable for a hybrid expert architecture large language model, and the method comprehensively considers the model structure and hardware feature information, constructs an expert access predictor based on a neural network for each layer, and carries out the training through the historical data. And experts needing to be activated in the model reasoning process can be predicted in advance. In the process, the predictor executes prediction on the CPU in parallel, and the prefetcher loads expert parameters from a host memory to a GPU video memory in advance according to a prediction result. By adjusting the prediction distance, the prediction accuracy and the prefetching advance are balanced, and the prefetching number can be dynamically adjusted according to the prediction accuracy. Meanwhile, parallel loading is adopted, system resources are utilized to the maximum extent, and the influence on GPU reasoning performance is reduced. And finally, a prediction error condition is processed through mechanisms such as high-priority re-prefetching, and normal model reasoning is ensured.

The invention discloses an industrial personal computer and multi-graphics card collaborative parallel computation acceleration system, which relates to the technical field of industrial resource allocation and parallel computation, and comprises a resource monitoring and predicting module, a resource management module and a prediction type resource preparation module, the task splitting and collaborative execution module comprises a task splitting module, a cross-node collaborative module and a collaborative operation engine; the intelligent scheduling and dynamic resource allocation module comprises an intelligent scheduler, a dynamic resource allocation module and a conflict avoidance module. According to the method, the GPU video memory utilization rate, the core utilization rate, the temperature, the video memory fragment rate, the available video memory total amount, the CPU core total utilization rate, the load condition and the idle core number index are collected in real time through a resource monitoring module, and the video memory capacity, the GPU core occupancy rate and the CPU load requirement are predicted in advance before a task is submitted in combination with a gradient boosting decision tree and a neural network prediction model; and resources are reserved, so that the scheduling delay is remarkably reduced, and the scheduling hit rate is improved.

The invention discloses a large language model progressive field fine tuning and knowledge fusion method for shield engineering. The method comprises the following steps: constructing a layered shield training course containing a wide-area academic theory and a proprietary enterprise construction method; parallelly training a plurality of physically isolated parameter efficient adapters based on the frozen base; performing singular value decomposition on the adapter, extracting a geometric feature subspace representing knowledge distribution, and calculating a conflict correlation degree; based on this, a uniform adaptation mechanism of resource awareness is constructed. The mechanism not only can generate a static fusion model for conflict removal, but also can dynamically activate a specific rank slice of an adapter through a routing network based on real-time hardware resource budget (video memory / FLOPs) and geometry-resource signature. According to the method, multi-source knowledge is reserved, and adaptive dynamic scheduling of edge hardware resources by model reasoning is realized.

The invention relates to the field of artificial intelligence, and discloses a reasoning acceleration method and device for a large language model, equipment and a medium, and the method comprises the steps: obtaining a to-be-processed text in a reasoning request in response to the reasoning request input by a user; performing video memory demand prediction according to the to-be-processed text to obtain a video memory prediction result; distributing a corresponding initial video memory space for the large language model according to the video memory prediction result so as to execute a corresponding reasoning task on the to-be-processed text; when the large language model executes the reasoning task, monitoring an occupation index of the initial video memory space; and when the occupation index meets a preset condition, triggering a dynamic allocation mechanism, and allocating a new video memory space for the large language model according to the dynamic allocation mechanism. The corresponding initial video memory space is allocated by predicting the video memory demand in advance, the use condition of the initial video memory space is monitored in real time, and a dynamic allocation mechanism is triggered when resources are insufficient, so that flexible and controllable dynamic video memory allocation is realized, the waste of video memory resources is reduced, and the overall reasoning speed of the model is improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as medical health and financial science and technology, and discloses a model quantitative reasoning acceleration method, device, equipment and medium. The method comprises the steps that an input text is divided into a plurality of processing blocks, importance scoring is conducted on the non-first processing block, and calculation precision formats are distributed according to scoring results; determining a unified quantization configuration of each processing block; dividing the network modules into configuration sharing groups, and sharing quantitative configurations of corresponding processing blocks in the groups; and executing block-level quantization inference according to the unified quantization configuration, and generating a model inference result. The quantitative configuration of each processing block is uniformly determined on the basis of the token importance score, and the configuration is multiplexed in the network module group, so that block-level precision distribution and parallel quantitative reasoning are realized, the video memory overhead and the configuration time overhead are greatly reduced while the reasoning precision is guaranteed, and the reasoning efficiency is improved. And the execution efficiency and the video memory utilization rate in the long text reasoning task are effectively improved.

The invention discloses a scene rendering method and system based on three-dimensional Gaussian splashing. The method comprises the following steps: organizing and constructing an original 3D Gaussian set to obtain a spatial hierarchical structure; generating corresponding level details for the primitives in the spatial hierarchical structure to obtain a spatial hierarchical structure associated with the level details; traversing the spatial hierarchical structure associated with level details, and executing hierarchical view cone cutting and shielding elimination to obtain a visible node list; traversing the visible node list, calculating a level detail selection standard, and generating a level detail activity Gaussian set; performing optimization sorting on the level detail activity Gaussian set to obtain an activity primitive list; and performing tile-based rasterization on the active primitive list, and performing adaptive processing according to level details of the primitives to obtain a final color value of each pixel. According to the method, the rendering performance can be greatly improved, the occupation of a memory and a video memory is remarkably reduced, the rendering quality is improved, visual flaws are reduced, and the expandability of the 3DGS rendering method is enhanced.

The invention provides a dynamic video memory allocation method and system based on data characteristics, and relates to the technical field of data storage, and the method comprises the following steps: pre-allocating a plurality of video memory blocks with different sizes in a GP U video memory to form a three-stage video memory pool; acquiring data features input by the model in real time, and establishing a multi-dimensional feature vector; constructing a time sequence prediction model based on an LSTM network, and predicting a video memory demand value by inputting a feature vector sequence subjected to continuous multiple reasoning; selecting a hierarchical distribution strategy according to the predicted video memory demand value and the scene feature matrix; constructing a video memory dependency graph, dynamically tracking the earliest release time of a video memory block, and optimizing the utilization rate of the video memory through a multiplexing strategy; monitoring a GPU video memory state index; triggering a video memory optimization strategy according to the video memory state index; through accurate prediction of video memory demands, reasonable hierarchical distribution, optimization of video memory utilization rate and coping with different scenes, the video memory management problem in deep learning model reasoning is solved, and the video memory use efficiency and the system stability are improved.

The invention provides a hybrid expert model reasoning optimization method based on speculative preloading, and belongs to the field of artificial intelligence safety. The method aims at improving the reasoning performance of a hybrid expert model (MoE for short) in a resource limited scene, and solves the technical problems of high video memory occupation and bottleneck in calculation and data transmission of the hybrid expert model in a reasoning stage. According to the method, through hot expert identification, speculative preloading and pipeline execution, a fine-grained expert parameter preloading strategy and an asynchronous loading mechanism based on a priority queue are designed, and effective overlapping of a calculation process and a data transmission process is realized, so that the reasoning efficiency is improved. The method comprises the steps that expert parameters are divided into hot experts and non-hot experts, the hot experts are loaded to a computing device memory in advance, parameters of the next layer are asynchronously preloaded according to prediction for activation of experts of the future layer, and meanwhile computing of the current layer is executed. According to the method, resource optimization reasoning is realized on hybrid expert models with different scales and structures on the premise of not changing the structure and precision of the model.

The invention discloses an asynchronous parallel reasoning method, system and equipment for a hybrid expert model and a medium, which are corresponding schemes: decoupling synchronization of calculation and communication between GPUs (Graphics Processing Unit) caused by all-to-all set communication in expert parallelism, allowing asynchronous parallelism of model calculation and lexical metadata communication, and solving the problem of asynchronous parallelism of the model calculation and lexical metadata communication. Data communication overhead caused by expert parallelization is fully masked, and synchronization waiting overhead is eliminated; aiming at the phenomenon of uneven cold and heat of experts in reasoning, the hot experts are preferentially placed in the GPU, the cold experts are laterally loaded in the CPU so as to release the video memory space of the GPU, and the calculation efficiency of the GPU can be improved by increasing the batch size during reasoning; efficient resource scheduling is realized by dynamically selecting a computing unit which is most suitable for execution and a cold expert which needs to be loaded; generally speaking, the communication overhead and the waiting overhead during parallel reasoning of experts can be remarkably reduced, meanwhile, the calculation efficiency of the GPU is improved, and the overall throughput performance in the reasoning process is optimized.

The invention discloses a method for improving long text processing efficiency and accuracy, and relates to the technical field of natural language processing and large language models.According to the method, text word segmentation embedding, sliding block preprocessing, YaRN position code injection, dynamic sparse attention calculation, multi-level attention fusion, graded KV cache management and output generation are sequentially executed; position drift is inhibited through logarithmic scaling, and key contexts are adaptively screened according to the attention activeness, so that the attention calculation complexity is close to linearity; in million-level Token reasoning, the video memory occupation of the method is reduced, the remote dependency recall rate is improved, and the method is suitable for scenes such as document analysis, code auditing and multi-mode streaming understanding.