1results about How to "Avoid communication" patented technology

This disclosure relates to parallel inference methods, apparatuses, devices, and media for multiple LoRA large models, and pertains to the field of deep learning technology. By adding a dedicated process for the LoRA model, it achieves decoupling and separate management from the original large model process at the process level. At the system architecture level, it realizes isolation and parallel computing of the basic large model and multiple LoRA models in terms of computing and storage resources. Compared with traditional methods that deploy the large model and LoRA models on the same device, this disclosure can deploy more LoRA models and improve inference concurrency. Before each iteration of inference, the metadata data is synchronized within the LoRA process group, and the input data is controlled to be broadcast only within the large model process group, avoiding communication and storage redundancy caused by broadcasting input data to the LoRA process and optimizing preprocessing efficiency. By separating the LoRA process and the large model process for parallel operation, efficient separation and collaboration of inference are achieved.