3 results about "Cloud testing" patented technology

This application discloses a data processing method and apparatus, relating to the field of computer technology, for improving the throughput of large language models. The method is applied to edge devices or cloud testing devices, and includes: prefetching key KV cache for at least one future generation step at a first time, and calculating the attention score for the (T+N)th generation step based on the key KV cache. The first time is any time of the Tth generation step, which is used to generate token T. The scheme provided in this application prefetches key KV cache for one or more future steps during the inference process of the generation step. Since the prefetching of key KV cache and model inference can be executed in parallel, the transmission latency of the key KV cache can be effectively reduced. In this way, the scheme of this application can significantly improve the throughput of large models, i.e., more tokens can be generated per second.

A data conformance service determines, based at least on an information model of a machine at a remote site of a client, cloud test code to be executed by the data conformance service to perform at least a first type of conformance test for data provided by the machine and edge test code to be executed by an edge gateway at the remote site to perform at least a second type of conformance test for the data from the machine. The service deploys the edge test code to the edge gateway. The service receives data from the machine and executes the cloud test code using the data to generate cloud test results. The service receives edge test results and generates, based on the cloud and edge results, an indication of whether the data from the machine conforms to a specification.