1results about How to "Achieve privacy" patented technology

The application relates to an efficient and privacy-considered large model end-cloud collaborative inference method and system, which comprises the following steps: obtaining input and configuring KV cache control parameters through an end side; performing embedding calculation to generate a feature tensor and uploading the feature tensor to a cloud side; performing decoding layer inference on the cloud side according to authorization to call the KV cache and returning a hidden state; in a current generation step, performing speculative decoding prediction on subsequent tokens based on the hidden state on the end side, and calculating an end-side local inference result corresponding to a low-dimensional feature according to a dimension splitting rule; calculating a cloud-side local inference result corresponding to a high-dimensional feature on the cloud side, and returning the cloud-side local inference result after language self-adaptive cutting and encryption; fusing the two-end local inference results to obtain a complete distribution, sampling to generate an incremental token, uploading the incremental token to the cloud side for verification, and updating the KV cache.