3 results about "Autoload" patented technology

The application provides a large model inference optimization method, system, medium, terminal and program product for Ascend processors. The method comprises: comparing the automatically detected version with a built-in compatibility table; if the versions do not match, automatically loading a runtime patch or calling a backup library; if the versions match, processing a user request based on a preset hybrid scheduling mechanism; dynamically selecting a MindIE inference framework or a vLLM inference framework based on a runtime adaptation layer according to the user request parameters; converting large model weights into low-bit weights and compensating based on an error compensation matrix; identifying subgraphs and fusing operators for a model graph according to the computing characteristics of the Ascend processor, and rearranging intermediate tensors according to the 512B memory access alignment rule of the Ascend processor; performing pipelined overlap processing on the matrix multiplication calculation of the current token and the full aggregation operation based on an asynchronous execution flow; and performing inference and generating an inference result according to the user request. The application can improve the efficiency and reliability of large model inference.

The application provides a configuration-driven DL4NLP model automatic training and deployment method, which comprises the following steps: parsing a YAML configuration file and verifying parameters to generate a unified process configuration; automatically loading data and performing preprocessing according to the configuration to generate a model input tensor; instantiating model components through a component factory and assembling the model components into a complete network model; automatically configuring a training environment to train and evaluate the model to obtain an optimal model; exporting the model into a TorchScript format binary file; and automatically generating a standardized deployment package based on a template engine. The application abstracts the whole process of model development and deployment into a configuration-driven standardized pipeline by parsing the YAML configuration file, automatically performs data loading and preprocessing, model component instantiation and assembly, training optimization and deployment package generation, and solves the problems of low development efficiency, long cycle and technical gap between training and deployment caused by repeated code writing and manual conversion and deployment for different tasks in traditional model training and deployment.

This invention discloses a metadata-driven method and system for constructing a large language model intelligent agent through bootstrapping, relating to the field of artificial intelligence technology. The method includes the following steps: collecting natural language requirements generated from business scenarios and organizing these requirements into a unified expression format; generating readable metadata description content within this unified expression format; performing field splitting and role attribution around the metadata description content; constructing data type definition metadata within the unified expression format; and embedding prompt words into the data type definition metadata to generate content. This invention achieves progressive generation from natural language to data types, rules, and intelligent agent definitions through unified metadata expression, supporting dynamic construction and automatic loading at runtime, reducing manual intervention, and improving automation. Simultaneously, new metadata is generated during runtime and fed back into the generation chain, forming a continuous bootstrapping closed loop, enhancing structural consistency and version evolution controllability.