366 results about "Request queue" patented technology

The invention provides a data all-flash storage optimization method and system based on cloud computing, and the method comprises the steps: receiving a storage operation request stream of a user terminal through collecting a multi-dimensional performance data set of each node of an all-flash storage cluster in real time, wherein the multi-dimensional performance data set comprises a storage space fragmentation rate, an input / output request queue depth and a network link bandwidth utilization rate; analyzing a to-be-stored data block identifier set and an operation mode label, traversing a historical access record library based on to-be-stored data block identifiers, extracting historical access time sequence characteristics and physical storage position proximity of associated data blocks, and combining the multi-dimensional performance data and the historical access time sequence characteristics to obtain a data block identifier set; according to the method, a dynamic access popularity prediction model in a preset time period is constructed, and finally, a cross-node fragmentation storage topological graph and a cache preloading strategy matrix are generated according to the proximity of the dynamic access popularity prediction model and a physical storage position, so that the optimization of all-flash storage is realized, and the data storage performance in a cloud computing environment is improved.

The invention relates to the technical field of visual rendering, and provides a three-dimensional data visual rendering method and a visual rendering engine system.The method comprises the steps that space segmentation processing is conducted on three-dimensional model data to be rendered through the LOD technology to generate block slice data with different geometric detail levels, organizing the block slice data through a tree structure with a spatial relationship, and outputting the tree structure as an index file; the method comprises the following steps: adaptively screening block and slice data of different levels of detail based on a user vision field range, storing the block and slice data into a rendering request queue, querying block and slice data to be rendered from the rendering request queue according to index codes of the block and slice data, and updating a layer browser database; according to the method, to-be-rendered block slice data are downloaded and analyzed in parallel in batches in a separated collaborative operation mode, and a rendering task is executed according to a data analysis result, so that the rendering scheduling frame rate is remarkably improved, and rapid visualization of urban mass three-dimensional slice data at a webpage end is realized.

The invention discloses a big language model reasoning service optimization method, system and device and a medium. The method comprises the steps of obtaining a current reasoning request queue; performing dynamic sorting on each reasoning request in the current reasoning request queue; and performing dynamic batch processing on each reasoning request in the reasoning request queue by using an SLO-BS algorithm to obtain a current batch of reasoning requests, and inputting the current batch of reasoning requests into a large language model, the method, the system, the equipment and the medium can solve the problem of video memory fragmentation occurring during multi-task concurrent reasoning, and reduce the SLO violation rate at the same time.

In some implementations, a rollout component may deploy, in a pilot mode, an updated version of a software application in a runtime environment executing a legacy version of the software application. The rollout component may process requests in a request queue using the legacy version to generate processing results that are dispatched to a client interface and may process, using the updated version, copies of the requests processed by the legacy version. The rollout component may indicate whether the updated version passes one or more tests based on a comparison between the validation data generated by the updated version and processing results generated by the legacy version. The rollout component may switch the updated version from the pilot mode to a live mode based on the updated version passing the one or more tests.

The embodiment of the invention provides a read-write request processing method, a protocol conversion module, electronic equipment and a storage medium, and relates to the technical field of chips. The method comprises the steps that a protocol conversion module is arranged in a processor cluster, an original read-write request is converted into a first read-write request conforming to a CHI protocol through a first conversion module in the protocol conversion module, and a corresponding transmission channel is distributed to the first read-write request; and the request queue module orderly sends the received first read-write request, and the second conversion module recombines the first read-write request information to a second read-write request conforming to the CHI protocol, and allocates a corresponding channel for the second read-write request. According to the embodiment of the invention, the internal data channel (AXI or similar AXI) is directly converted into the CHI interface in the processor cluster, the internal circuit resources of the processor cluster are reused, the circuit area is saved, and the access path time delay is reduced.

The embodiment of the invention provides a data read-write method, system and device, computing equipment and a storage medium, and the data read-write method comprises the following steps: receiving a data read-write request sent by a user mode through an asynchronous I / O layer, and storing the data read-write request into a request queue; under the condition that the asynchronous I / O layer determines that the data read-write request is of a straight-through type, the data read-write request is sent to a driving layer; and analyzing the data read-write request through the driving layer, and sending request information of the data read-write request to target processing end equipment for read-write processing. The asynchronous I / O layer receives the data read-write request sent by the user mode and sends the data read-write request to the driving layer after the data read-write request is determined to be of the straight-through type, context switching and data copying expenses are reduced, the processing efficiency of I / O operation is improved, the request is analyzed in the driving layer and sent to the processing end equipment for read-write processing, scheduling of I / O tasks is optimized, and the user experience is improved. The disk addressing time and I / O delay are reduced, and the overall performance and the system stability are improved.

The invention discloses a request management method, device and system for model reasoning. The method comprises the following steps: obtaining an observation index of a model reasoning service; index prediction is carried out according to the observation indexes, prediction indexes are obtained, and the prediction indexes comprise the target memory utilization rate of the central processing unit and the target video memory utilization rate of the graphics processing unit; determining a scheduling strategy according to the prediction index, determining a scheduling suggestion according to the scheduling strategy in combination with the request queue, and adding the scheduling suggestion into a suggestion buffer queue; when the model reasoning service executes request scheduling, the scheduling suggestion is read from the suggestion buffer queue, a target scheduling decision is determined according to the scheduling suggestion in combination with a scheduling decision of the model reasoning service, and the model reasoning service is controlled to schedule corresponding request execution model reasoning according to the target scheduling decision; the problem of unreasonable request scheduling in the model reasoning process is solved, and efficient utilization and reasonable allocation of resources are realized.

The invention relates to the technical field of network resource scheduling, and provides a heterogeneous network intelligent slice resource scheduling method and device, electronic equipment and a medium. The method comprises the following steps: performing three-dimensional joint modeling on heterogeneous network physical resources to obtain a dynamic resource state tensor, and performing a dynamic resource allocation decision by combining a deep reinforcement learning model with a service demand matrix to obtain an initial slice resource allocation matrix; and performing mixed integer linear programming through a preemption decision model in combination with the URLLC service request queue to obtain a resource allocation sequence, and performing occupancy rate updating on the dynamic resource state tensor according to the resource allocation sequence to obtain ground network resource data, and carrying out resource fusion in combination with the satellite beam capacity data to obtain a space-ground integrated resource pool tensor, thereby carrying out traffic prediction in combination with historical traffic data through a long-short memory network to obtain a bandwidth pre-adjustment strategy. Through organic integration of technologies such as mixed integer linear programming and traffic prediction, the overall network resource utilization rate is improved.

The invention relates to the technical field of project management, in particular to a large-model batch reasoning and data flow optimization system oriented to an MOE architecture. According to the method, a collaborative architecture of the request access module, the environment sensing module, the expert routing engine, the resource scheduling module and the dynamic optimization control module is set, the text length and the subject type are extracted by using the request access module, a basis is provided for accurate routing, and the GPU video memory, the I / O bandwidth and the request queue depth are acquired in real time through the environment sensing module, so that the real-time routing is realized. The system load is comprehensively monitored, meanwhile, an expert sub-network is activated through an expert routing engine according to request features, invalid calculation is avoided, weight loading and resource allocation are managed through a resource scheduling module, the I / O bottleneck is reduced, and finally an optimization strategy is intelligently triggered through a dynamic optimization control module based on routing conflict factors. The problems of large reasoning delay fluctuation and unbalanced resource utilization rate mentioned in the background technology are solved, and stable low-delay response and resource collaborative optimization in a high-concurrency scene is realized.

The invention discloses a video stream concurrent access front-end dynamic scheduling control system and method based on a digital twinborn scene, and relates to the technical field of computer software and network communication, the system comprises three core modules: a self-adaptive request queue module manages UE rendering resources based on a session token pool, distributes tokens through preemptive scheduling, and sends the tokens to a server; in combination with a GPU frame rate feedback dynamic adjustment strategy, queuing through a Promise asynchronous queue when the request fails; the front-end fusing and backoff retry module monitors the connection failure rate, when the connection failure rate reaches a threshold value, fusing is triggered, retry is delayed by adopting an exponential backoff strategy, and a semi-open state tends to recover; and the local cache collaboration module intercepts the request through Service Worker, caches the latest video clip, pushes the local content when the request is delayed, and ensures the freshness through ETag verification. The method and the device are used for efficiently managing loading and interaction of multiple users on real-time rendering pictures at a browser end.

The invention relates to the technical field of graphics processors, and provides a dynamic graphics processor sharing method, which comprises the following steps of: receiving submission of a plurality of deep learning tasks; generating a dynamic priority queue containing high-priority tasks and low-priority tasks; transmitting a resource state to the low-priority task through a broadcast mechanism, and triggering resource adjustment according to a comparison result of the real-time thread block usage amount of the high-priority task and a preset threshold value; according to the difference value between the real-time thread block usage amount of the high-priority task and a preset threshold value, the number of executable thread blocks of the low-priority task is dynamically adjusted; checking whether the number of thread blocks required by the kernel is smaller than or equal to the number of executable thread blocks of the current low-priority task, if yes, starting execution, and if not, reserving the kernel in a kernel request queue to wait for next cycle scheduling; and periodically updating the dynamic priority queue until all deep learning tasks are completed. According to the technical scheme, the overall efficiency of the system can be improved, and resource waste is reduced.

According to an embodiment of the present disclosure, a computer implementation method using a device for managing a priority in a memory disaggregation network, the computer implementation method comprising: classifying received read requests by priority and storing the read requests in a request queue of a memory module; classifying the received read requests by response path indicating an output port of the memory module and storing the read requests in a response queue of the memory queue; and performing scheduling in consideration of states of the request queue and response queues.

The invention provides a chip verification method and device, equipment, a medium and a program product, and the method comprises the steps: receiving a first operation instruction of a to-be-verified module and a second operation instruction of a reference model in parallel, the reference model being a function model used for simulating the expected behavior of the to-be-verified module; searching a first instruction record matched with the first operation instruction from a request queue of the reference model, and searching a second instruction record matched with the second operation instruction from a request queue of the to-be-verified module; and determining a verification result of the to-be-verified module according to a comparison result of the first operation instruction and the first instruction record and / or a comparison result of the second operation instruction and the second instruction record. According to the invention, a structured and reusable universal chip verification method is realized, so that a tedious customized scoreboard design is replaced, and the efficiency and accuracy of verification work are greatly improved.

The invention discloses a cache resource request method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps that a consumer determines a target cache resource needing to be requested; when the consumer determines that the priority request cache partition corresponding to the target cache resource is a private cache partition, comparing whether the first current available size of the private cache partition is greater than or equal to the size of the request cache resource; if yes, applying for a first target cache resource fragment of the target cache resource type from a first provider of the private cache partition; if not, judging whether a request record of the shared cache partition exists or not; if not, comparing whether a second current available space corresponding to the consumer in the shared cache partition is greater than or equal to the size of the requested cache resource; if yes, applying for a second target cache resource fragment of the target cache resource type from a second provider of the shared cache partition; and if not, generating a request record, and writing the shared cache resource request instruction into a shared cache resource request queue.

The invention relates to the technical field of file storage, and discloses a method and a system for storing multiple concurrent large files. According to the method and the system, the format and the size of a file are verified, a unique file name is generated and used for identifying metadata of a storage file and outputting the file, each small block of the file is divided into a fixed size and is divided into small blocks, a unique identifier is generated for each small block, a database is connected, and the metadata of the file is stored in the database. Then traversing each small block, selecting a storage node for each small block, uploading the small blocks by using multiple threads, setting a request queue, processing file uploading and downloading requests in sequence, avoiding concurrent write-in conflicts of the same file by using a lock mechanism for file uploading, and reading and controlling a read request through a read-write lock; and acquiring the information of the storage file from the database, acquiring each small block by utilizing a concurrent downloading mode, recombining the small blocks into an original file in sequence after downloading is completed, and performing integrity verification after file recombining.

The subject application relates to separate command address (SCA) protocol-based memory controllers. For instance, a method may include receiving, by a separate command address (SCA)-based memory controller, a plurality of commands, scheduling, by the SCA-based memory controller, the plurality of commands using two or more data path scheduler (DPS) request queues, converting the scheduled plurality of commands into SCA-based commands, selecting, from the scheduled SCA-based commands, one or more scheduled SCA-based commands for a sequence execution, and executing in sequence the selected one or more SCA-based commands using one or more logical units (LUNs) of a channel in a memory component.

The invention provides a large-scale concurrent network request management method based on a computing power host management platform, belongs to the technical field of computer networks, and aims to reduce the request pressure on a rear-end host through technologies such as merging same or similar requests, allocating priorities for the requests, maintaining request queues and controlling the request sending frequency. The utilization efficiency of resources is improved, so that the system performance and the user experience are improved.

There is provided a computer-implemented method for handling updates to a computing environment, the method comprising: receiving, from a user, a change request comprising a text description justifying a change in hardware and / or software within the computing environment; applying natural language processing to the text description to extract one or more text features from the text description; performing a validation check of the received change request by applying a trained classification model to the extracted one or more text features, the classification model having been trained on a dataset of historical change requests; and if the received change request fails the validation check, preventing submission of the change request into a queue of change requests and / or implementation of the change associated with the change request.

The invention discloses a large-model adaptive batch reasoning system based on time delay measurement, and the system comprises a request receiving and queue management module which is used for monitoring and receiving an LLM reasoning request sent by an external user or an application; the real-time delay sensing and batch monitoring module is used for measuring and analyzing performance indexes directly related to the current batch processing strategy in real time in the LLM reasoning process; the Token-by-Token batch processing and reasoning execution module is used for selecting a corresponding number of requests from the request queue according to the size of the current batch determined by the scheduler, organizing the requests into an effective calculation batch and submitting the effective calculation batch to the LLM reasoning core at the bottom layer to execute one or more decoding steps; and the double-stage batch adjustment decision module is used for obtaining the normalized time delay ratio. The invention also discloses a large-model adaptive batch reasoning method based on time delay measurement. According to the invention, substantive improvement and intelligent management of the overall performance of the LLM reasoning service are realized.

The invention relates to the technical field of data processing, and provides a deep learning inference engine tensor optimization method and system for GPU acceleration. Performing asymmetric quantization on the first weight tensor of the original model to generate a second weight tensor, and updating the original model based on the second weight tensor to obtain an updated model; blocking the data tensor according to a preset dimension to generate input sub-blocks, and loading the input sub-blocks and a convolution kernel in the update model to a shared memory; calling the input sub-blocks from the shared memory, and processing the input sub-blocks of different batches in parallel based on a single kernel function formed by convolution kernels to generate a processing result; and based on the GPU utilization rate and the request queue length corresponding to the to-be-processed data, performing dynamic output adjustment to output a processing result. The performance and efficiency of a deep learning inference engine are improved, memory occupation and calculation delay are reduced, and powerful support is provided for deep learning application of GPU acceleration.

Computer-implemented methods for managing instances of serverless functions in a cloud computing system are provided. Aspects include obtaining a service level objective for a serverless function, obtaining a command queue length for a graphical processing unit disposed on each of a plurality of compute nodes in the cloud computing system, and obtaining a request queue length of the serverless function. Aspects also include calculating a number of instances of the serverless function to deploy in the cloud computing system, wherein the number of instances is determined based on the service level objective and the request queue length of the serverless function, identifying compute nodes from the plurality of compute nodes to deploy each of the number of instances of the serverless function, and creating an instance of the serverless function on each of the identified compute nodes.

The invention discloses a load balancing scheduling method and device for model reasoning and electronic equipment. The method comprises the following steps: receiving a current request of a user; estimating the processing time of the current request according to the parameters of the historical request, the processing time of the historical request and the parameters of the current request; judging whether idle service instances exist or not according to the service instance state list; wherein the service instance state list comprises service instance numbers and states and priorities thereof; if no idle service instance exists, the current request and the corresponding parameters are placed in a request queue; and if the idle service instance exists, distributing the request selected from the request queue according to the first index or the current request to the service instance selected according to the second index, and updating the state of the service instance in the service instance state list. According to the method, load balancing is carried out on the request by recording and managing the processing state of the back-end service instance, and the method is more suitable for model reasoning service.

The invention provides a model reasoning scheduling method and system, an electronic device and a storage medium, the system comprises a plurality of pre-filling server nodes and a plurality of decoding server nodes, and the method comprises the following steps: responding to received user request information, and combining a pre-filling scheduling model based on deep reinforcement learning, obtaining a target pre-filling server node selected from the plurality of pre-filling server nodes and a first priority; sending a pre-filling request to a request queue corresponding to the first priority in the target pre-filling server node, and enabling the target pre-filling server node to process the pre-filling request; after the pre-filling request is processed, obtaining a target decoding server node selected from a plurality of decoding server nodes and a second priority by combining a decoding scheduling model based on deep reinforcement learning; sending a decoding request to a request queue corresponding to the second priority in the target decoding server node, and enabling the target decoding server node to process the decoding request; and reasoning efficiency is improved.

Systems and methods disclosed herein for improving data retrieval for dashboards using data caching. The system may implement a cache memory to store results for executed queries and / or anticipated queries, allowing for data to be retrieved asynchronously, and may implement a request queue to distribute the queries among processing nodes.

The invention provides an electronic book borrowing management system based on OPAC bidirectional hitching, and relates to the technical field of library informatization. Comprising the steps that a hardware acceleration layer constructs a dynamic resource distribution matrix through an FPGA coprocessor and calculates a matrix norm in real time, and request queue scanning and a redundant copy generation algorithm per 100 ms are achieved in combination with an ASIC anti-conflict controller; a system service layer deploys a dynamic priority scheduling engine, realizes intelligent resource distribution based on a cross-campus coordination coefficient and an edge cloud coordination strategy, records borrowing operation through a block chain evidence storage service and executes automatic copyright division auditing; the user interaction layer provides a bidirectional association retrieval interface and a priority borrowing channel based on credit scores. According to the system, the resource scheduling efficiency in a high-concurrency scene is improved through matrix norm calculation accelerated by hardware, an elastic copy allocation mechanism and an anti-conflict algorithm solidified in ASIC, and cross-campus resource collaboration and accurate copyright revenue allocation are realized in combination with geographic position scoring and block chain technologies.

The present invention discloses a request processing method and device based on a neural network model, which relates to the field of computer technology and can solve the problem in the prior art that hardware resources cannot change rapidly and dynamically with changes in traffic and have poor flexibility. The method of the present invention mainly includes: determining whether at least one target neural network model is stored in the video memory of a network processor, the target neural network model including a neural network model corresponding to a queue of pending requests; if the target neural network model is not stored in the video memory, pre-fetching the target neural network model from the internal memory and loading it into the video memory to cover at least one neural network model pre-stored in the video memory; processing the requests in the queue of pending requests based on the target neural network model stored in the video memory. The present invention is mainly applicable to scenarios in which different requests are processed based on multiple neural network models.

A method includes: receiving an initial request message in a request queue of the messaging system; receiving an aggregation reply message in a reply queue of the messaging system, wherein the aggregation reply message is received from an integration system that processes the initial request message, and wherein the aggregation reply message includes an aggregation identifier associated with the initial request message; in response to receiving the aggregation reply message, starting a timeout process; monitoring the timeout process; holding one or more messages that are in the reply queue and that include the aggregation identifier until the timeout process expires or an expected number of responses has been received; and making available the one or more messages that are in the reply queue and that include the aggregation identifier based on the timeout process expiring or the expected number of responses having been received.

The invention discloses an archive management robot control system and method, and belongs to the technical field of robot operation and control, and the method comprises the steps: obtaining RFID scanning data and visual grayscale image data of a surface area of an archive rack, and generating a gap position data set; updating a preset priority dynamic matrix according to the gap position data set and a preset file borrowing request queue; based on the priority dynamic matrix and a preset multi-layer potential field model, robot advancing track parameters are adjusted, and a composite path instruction set is generated; and outputting the composite path instruction set to a robot driving unit to execute path tracking. According to the method, multi-modal sensing data fusion, dynamic priority matrix updating and multi-layer potential field model path planning technologies are adopted, high-precision space sensing, self-adaptive task scheduling and intelligent path planning can be achieved, and the operation efficiency and system reliability of the archive management robot are remarkably improved.

The present application relates to a data replay method, a data processing unit, a network interface card, a device, and a storage medium. The method comprises: when a storage stack is restarted, acquiring a replay-pending request queue from a persistent memory, the replay-pending request queue being a queue obtained by mapping a transmission buffer area into the persistent memory, and the transmission buffer area being used for receiving a request from a host side; on the basis of the replay-pending request queue, acquiring a replay-pending request from the persistent memory; and resending the replay-pending request to a target acceleration component directed to a storage device, so as to instruct the storage device to store target data which resides in the persistent memory and corresponds to the replay-pending request. In the method, a request queue, a request, and data corresponding to the request are stored in the persistent memory in advance, and if the storage stack is restarted, the replay-pending request is recovered from the persistent memory, eliminating the need for DMA and encryption / decryption techniques, and shortening an interruption duration of a storage data path, thereby improving data storage efficiency on the host side.

The invention provides a method and a device for releasing prefetch requests. The method comprises the following steps: acquiring an earliest prefetch request from a prefetch request queue as a prefetch request to be processed; if it is judged that the to-be-processed prefetch request is hit in a virtual-real address translation cache and hit in a target cache, or the to-be-processed prefetch request is hit in the virtual-real address translation cache and hit in an address missing state tracking register; and if so, releasing the to-be-processed prefetch request and the related prefetch request from the prefetch request queue. The device is used for executing the method. According to the prefetching request releasing method and device provided by the embodiment of the invention, the extra power consumption of data prefetching is reduced.