2 results about "Queries per second" patented technology

Aspects of the disclosure are directed to enhancing RAG through a structured schema for RAG that captures a wider range of RAG techniques to serve as a foundation for improving performance and an optimization framework for efficiently serving the wider range of RAG techniques based on significant performance variability across RAG workloads. The optimization framework may achieve up to a 2× increase or more in query per second (QPS) per chip compared to alternative RAG frameworks built on LLM-system extensions.

This application provides a rate limiting method, apparatus, computer device, computer-readable storage medium, and computer program product, belonging to the field of data processing. The rate limiting method includes: receiving rate limiting parameters broadcast by a master node of a distributed system, the rate limiting parameters including at least a global start time; determining the current warming-up phase of the node based on the global start time and the current time, the warming-up phase corresponding to several phases with different request rates, where each subsequent phase has a higher request rate than the previous phase, the request rate being the rate at which a target service is called, and the node being any execution node of the distributed system; and dynamically adjusting the request rate of the node calling the target service based on the warming-up phase. The technical solution of this application embodiment can achieve a gradual warming-up of the QPS (Queries Per Second) of a distributed system calling a target service, improving the stability of the target service calls and the overall success rate of the task.