Multi-scenario and Multi-base Large Model Engine System

Abstract

The present invention belongs to the technical field of artificial intelligence model management. The present invention discloses a multi-scenario and multi-base large model engine system, which includes a front-end interaction layer, a business function layer, a core technology layer, and a bottom support layer. The front-end interaction layer is used to provide access methods, interface protocols, permission management, and log recording. The business function layer is used to meet the core business requirements of multiple scenarios in the financial industry. The core technology layer includes a multi-task instruction fine-tuning platform, a large model automated evaluation platform, and a large model management platform. The present invention can dynamically match the optimal model and efficiently execute tasks. Through the multi-task instruction fine-tuning platform and the automated evaluation platform, the model performance can be continuously optimized, the efficient utilization of model resources can be realized, and at the same time, the accuracy and response efficiency of business processing can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence model management, and more specifically, to a multi-scenario and multi-base large model engine system. Background Art

[0002] Currently, the financial industry, such as banking and insurance, has achieved remarkable results in the application of large model technology. However, with the continuous expansion of business scale and the increasing complexity of scenarios, the traditional single large model technology has exposed many deficiencies in meeting the diverse needs of the financial industry.

[0003] These deficiencies are mainly manifested in the following aspects:

[0004] First, it is difficult to handle different types of tasks, especially when quickly switching between different business scenarios;

[0005] Second, it lacks the ability of dynamic fine-tuning and evolution, making it difficult to meet the growing demand of the banking industry for intelligent and precise services and difficult to adapt to the rapidly changing market demand;

[0006] Third, it is difficult to efficiently evaluate and quickly find the most suitable model from multiple models, which is mainly reflected in two aspects: one is that the evaluation coverage is limited and cannot comprehensively reflect the actual performance of the model in complex tasks; the other is the lack of dynamic adaptation ability to different task characteristics, resulting in difficulty in accurately evaluating the comprehensive performance of multi-task models, restricting the optimization effect and deployment efficiency of the model in actual applications.

[0007] In view of this, the present invention proposes a multi-scenario and multi-base large model engine system to solve the above problems. Summary of the Invention

[0008] In order to overcome the above defects of the prior art and achieve the above object, the present invention provides the following technical solution: a multi-scenario and multi-base large model engine system, including a front-end interaction layer, a business function layer, a core technology layer, and a bottom support layer;

[0009] The front-end interaction layer is used to provide access methods, interface protocols, permission management, and log recording;

[0010] The business function layer is used to face the core business needs of multi-scenarios in the financial industry;

[0011] The core technology layer includes a multi-task instruction fine-tuning platform, a large model automatic evaluation platform, and a large model management platform;

[0012] The bottom support layer includes data and storage, security guarantee, computing and reasoning support, and DevOps and operation and maintenance.

[0013] Furthermore, the access method supports access from the Web side, mobile side, third-party application APIs, multi-dimensional terminal interactions, and business systems;

[0014] The interface protocol interacts through HTTP, WebSocket, REST, JSON format, and HTML protocol;

[0015] The permission management and logging are used to perform multi-level permission management on the accessed users and systems, and at the same time record access logs.

[0016] Furthermore, the business function layer includes an intelligent customer service module, an intelligent knowledge base Q&A module, an intelligent report generation module, an intelligent contract review module, a financial product investment education module, an intelligent data query module, and an intelligent marketing module.

[0017] Furthermore, the multi-task instruction fine-tuning platform is used to identify real-time task requirements and dynamically load the corresponding LoRA fine-tuning module:

[0018] Among them, the method for identifying real-time task requirements is as follows:

[0019] After the system receives the task input, it analyzes the input content through the front-end interface and extracts the input data;

[0020] Extracts task features from the input data through a natural language processing model and maps the extracted task features to high-dimensional feature vectors;

[0021] Based on the task routing rules, the task is assigned to a specific scenario label and the fine-grained characteristics of the task requirements are analyzed;

[0022] Analyzes the multi-modal data contained in the input to determine the data types involved in the task;

[0023] Combined with the task scenario database in the system, the task features are matched with the predefined scenario characteristics to obtain the real-time task requirements;

[0024] In the LoRA fine-tuning module, multiple LoRA fine-tuning modules are pre-trained and registered, and each LoRA fine-tuning module focuses on several scenario requirements;

[0025] Register task feature tags for each LoRA fine-tuning module, and the real-time task requirements are matched through feature extraction and the registered features of the module;

[0026] The dynamic loading includes: when the task matches a single LoRA fine-tuning module, then the LoRA fine-tuning module is dynamically loaded;

[0027] When the task features cover multiple LoRA fine-tuning modules, a module combination mechanism is adopted, and multiple LoRA fine-tuning modules are loaded to process the task.

[0028] Furthermore, when the multi-task instruction fine-tuning platform is trained, multiple task modules are integrated into a unified model base through the LoRA technology;

[0029] By adopting a multi-expert model framework, the corresponding LoRA fine-tuning modules are dynamically activated according to the input task features; only the LoRA fine-tuning modules required for the current task are loaded through a selection mechanism.

[0030] Furthermore, when the multi-task instruction fine-tuning platform is trained, it also includes an early warning mechanism, and the way the early warning mechanism gives an early warning is as follows:

[0031] Multiple key data related to resources are collected in real time, and the key data includes computing resources, memory and storage, network, and DNS health checks;

[0032] After the key data is collected, the resource bottleneck is identified through an anomaly detection algorithm, and the identified resource bottleneck is notified to the user in real time;

[0033] Among them, if an abnormal node appears during the training process, the abnormal node is automatically isolated and detected;

[0034] The nodes are identified through an anomaly detection algorithm, and the nodes with abnormal operation are recorded as abnormal nodes, and vice versa, as healthy nodes;

[0035] After the abnormal nodes are automatically isolated, monitoring continues, and measures are taken to recover the nodes;

[0036] The abnormal logs and isolation process are recorded in the database, and the historical data is used to optimize the anomaly detection algorithm and automatic isolation;

[0037] The automatic isolation includes dynamic task migration, communication isolation, partial task restart, and resource recovery.

[0038] Furthermore, the large model automated evaluation platform includes multi-dimensional performance evaluation, automated testing and quantization feedback, and model version comparison and error analysis;

[0039] Among them, the multi-dimensional performance evaluation is used to evaluate the performance of the model in multiple dimensions such as accuracy, response time, and resource consumption;

[0040] Automated testing and quantization feedback are used for scenario-based automated testing functions;

[0041] Model version comparison and error analysis are used for horizontal comparison and analysis of different model versions.

[0042] Furthermore, the large model management platform includes intelligent routing and model adaptation, an AI-Agent development and operation platform, and full life cycle management;

[0043] The intelligent routing and model adaptation automatically selects and integrates the optimal model by combining the results of the large model evaluation platform;

[0044] The AI-Agent development and operation platform is based on an enhanced retrieval algorithm for knowledge texts and is used to combine context information and a real-time updated knowledge base;

[0045] The life cycle management is used to cover the processes of model training, evaluation, selection, deployment, and iterative optimization, forming a closed-loop management.

[0046] Furthermore, in the large model automated evaluation platform, task weighting and dynamic scoring are also included. Among them, task weighting and dynamic scoring set a task weighting system according to scenario requirements, and the task weighting system includes static weight allocation and dynamic weight allocation.

[0047] Furthermore, in the large model automated evaluation platform, scenario-based and multi-task evaluation are also included. The scenario-based and multi-task evaluation provides feedback and verification on the model by simulating a real business environment and multi-task adaptability evaluation.

[0048] Technical effects and advantages of the multi-scenario and multi-base large model engine system of the present invention:

[0049] Through the collaborative work of the multi-task instruction fine-tuning platform, the large model automated evaluation platform, and the large model management platform, the present invention realizes the dynamic adaptation, efficient evaluation, and precise deployment of the model, and can meet the complex and diverse business needs of the financial industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic structural diagram of the multi-scenario and multi-base large model engine system of the present invention; [[ID=E29]]

[0051] Figure 2 is a schematic diagram of the architecture of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] Embodiment 1

[0054] Please refer toFigure 1 and Figure 2 As shown in Figure 2 , the multi-scenario and multi-base large model engine system in this embodiment includes a front-end interaction layer, a business function layer, a core technology layer, and a bottom support layer;

[0055] The front-end interaction layer is used to provide access methods, interface protocols, permission management, and log recording;

[0056] The business function layer is used to meet the core business requirements of multiple scenarios in the financial industry;

[0057] The core technology layer includes a multi-task instruction fine-tuning platform, a large model automated evaluation platform, and a large model management platform;

[0058] The bottom support layer includes data and storage, security assurance, computing and inference support, and DevOps and operation and maintenance.

[0059] Further, the access methods support Web-side, mobile-side, third-party application APIs, multi-dimensional terminal interactions, and business system access;

[0060] The interface protocols interact through HTTP, WebSocket, REST (such as the RESTful architecture), JSON format, and HTML protocol;

[0061] The permission management and log recording are used to perform multi-level permission management on the accessed users and systems, and record access logs at the same time;

[0062] Specifically, multiple access methods can meet the multi-scenario requirements of the financial industry; multiple interface protocols can ensure the high compatibility and low-latency performance of the system; permission management and log recording can guarantee the security and compliance of the system.

[0063] Further, the business function layer includes an intelligent customer service module, an intelligent knowledge base Q&A module, an intelligent report generation module, an intelligent contract review module, a wealth management product investment education module, an intelligent data query module, and an intelligent marketing module;

[0064] Specifically, the intelligent customer service module supports multi-round conversations and context understanding, and is applicable to scenarios such as account inquiries and loan applications, providing customers with continuous and smooth interaction experiences and improving customer service quality; the intelligent knowledge base Q&A module extracts information from documents such as financial reports and customer feedback, providing accurate knowledge support for bank employees, helping to quickly answer compliance questions, and improving the efficiency and accuracy of information queries; the intelligent report generation module supports the generation of due diligence reports, credit reports, and research reports, with a professional bank style and high accuracy, providing in-depth support for decision-making and meeting the bank's report requirements; the intelligent contract review module conducts intelligent reviews on contracts in banking operations, automatically identifying key terms and potential risks, ensuring the compliance and effectiveness of contracts, and improving review efficiency; the wealth management product education module provides education and guidance functions for wealth management products, helping customers understand complex wealth management product information, and enhancing investors' risk awareness and decision-making abilities; the intelligent data query module (such as ChatBI) supports data interaction and intelligent chart generation, provides trend prediction and data analysis, helps bank personnel quickly gain insights into business data, and supports business decision-making. The intelligent marketing module provides personalized marketing copy recommendations for banks through customer behavior data analysis, improves marketing effectiveness, and helps banks achieve precise customer reach.

[0065] It should be noted that this system can accurately match the diverse needs of financial business scenarios and improve response efficiency and adaptability through the collaborative work of multiple models. For example, the system not only supports efficient fine-tuning of specific scenarios but also can evaluate and select the optimal model in real time to ensure a high degree of fit between model performance and business requirements. The application of this system has significantly enhanced the competitiveness of the banking industry in intelligent and precise services, providing customized intelligent support for it to cope with the rapidly changing market environment.

[0066] Furthermore, the multi-task instruction fine-tuning platform is used to identify real-time task requirements and dynamically load the corresponding LoRA fine-tuning module:

[0067] Among them, the way to identify real-time task requirements is as follows:

[0068] After the system receives a task input (such as a natural language question, a structured query, a text analysis request, etc.), it analyzes the input content through the front-end interface and extracts the input data. For example, in the intelligent customer service scenario, the input may be the user's natural language question; in the financial calculation scenario, the input may be a digital data table or a calculation formula;

[0069] Extract task features from the input data through a model of natural language processing (NLP); the main features extracted include: task type (such as question and answer, analysis, generation); scenario category (such as financial report generation, risk control, intelligent customer service, etc.); complexity features (such as whether the task requires multi-round interaction or high-precision calculation); map the extracted task features to high-dimensional feature vectors. For example, an input statement can be converted into a vector representation through a pre-trained model (such as BERT, GPT), with additional task scenario labels and keywords attached;

[0070] Based on classification algorithms (such as logistic regression, multi-layer perceptron) or task routing rules, assign tasks to specific scenario labels (for example, "How about the loan application process?" → intelligent customer service scenario; "Calculate the risk metrics of the current investment portfolio," → financial analysis scenario);

[0071] Analyze the fine-grained characteristics of task requirements through a rule-based engine or a deep learning model (for example, whether real-time processing is required? Whether multi-round conversations are involved? Whether specific knowledge base retrieval support is required?)

[0072] Parse the possible multi-modal data (text, tables, images, etc.) contained in the input to determine the data types involved in the task;

[0073] Combine the task scenario database in the system to match the task features with predefined scenario characteristics to determine the most suitable task requirements;

[0074] The system generates dynamic feedback based on the feature matching results (for example, if a task is identified as requiring knowledge base support, the feedback is "Need to load the knowledge base Q&A module");

[0075] The LoRA fine-tuning module includes:

[0076] In the system development stage, for different business scenarios or task types, pre-train and register multiple LoRA fine-tuning modules, and each module focuses on one or more scenario requirements (for example, intelligent customer service module: for multi-round conversations and context understanding; financial analysis module: for high-precision calculation and table data processing; risk review module: for compliance and risk control scenarios);

[0077] Register task feature labels for each LoRA fine-tuning module (for example, intelligent customer service module: task type = question and answer, scenario category = customer service, multi-round conversation = True; financial analysis module: task type = analysis, scenario category = finance, high calculation accuracy = True);

[0078] The real-time task requirements are matched with the registered features of the modules through feature extraction. (The matching methods include: rule matching based on task tags and module tags. For example, when the task requirements contain keywords such as "risk" and "audit", the risk review module is preferentially matched; calculating the cosine similarity or other similarity metrics between the vectorized task features and the module feature vectors, and selecting the LoRA fine-tuning module with the highest similarity;)

[0079] Dynamic loading includes: when a task matches a single LoRA fine-tuning module, the system dynamically loads the module (for example, when the user asks "How to calculate the return on investment?", the system loads the financial analysis module);

[0080] When the task features cover multiple modules (such as semantic understanding of user input and extraction in combination with the knowledge base), the system adopts a module combination mechanism, loads multiple LoRA fine-tuning modules, and processes the task sequentially or in parallel (for example, the intelligent customer service module + the knowledge base Q&A module cooperate to process the task);

[0081] Moreover, after the task is executed, the system monitors relevant metrics such as the output quality and response time of the module, and records the adaptability between the task and the module;

[0082] According to historical data and model performance optimization rules, improve the matching accuracy between the task and the module (for example, if a certain task is associated with a specific module multiple times, the system can automatically enhance the priority of the module);

[0083] Among them, the combination mechanism includes:

[0084] Extract task features according to the task type (such as question and answer, generation, calculation) and scenario requirements (such as financial report generation, risk analysis);

[0085] Task features include: input data type: text, table, image; target output type: classification, summary, answer, etc.; task complexity: whether it involves multi-round conversations, high-precision calculations;

[0086] Decompose complex tasks into multiple subtasks (for example, for the financial report generation task, subtask 1: extract key data from the document (information extraction);

[0087] Classify according to the characteristics of open-source models (for example, question and answer tasks: ChatGLM, LLaMA; generation tasks: GPT-4, Bloom, etc.);

[0088] Select the most suitable model for each subtask (for example, for information extraction: T5, because it performs better in document processing and extraction tasks);

[0089] Dynamically select the most suitable model according to the task characteristics, and the selection method is a rule engine based on feature matching or a deep learning classifier;

[0090] Set up a task pipeline for multiple models to ensure that tasks are completed in a logical order (for example, information extraction → data analysis → text generation);

[0091] For independent subtasks, execute them in parallel;

[0092] Unify the processing and optimization of the outputs of multiple models (for example, fuse the answers of multiple models, such as weighted voting, confidence ranking);

[0093] Set evaluation metrics and adjust the model combination method according to the evaluation results (for example, replace the models with poor performance, redefine the task division);

[0094] Specifically, for the seven core application scenarios in the banking industry (i.e., intelligent customer service, intelligent knowledge base Q&A, intelligent report generation, intelligent contract review, intelligent financial product investment education, intelligent data query, intelligent marketing), the platform is efficiently fine-tuned through the LoRA fine-tuning module to achieve high-precision support for multiple tasks and multiple scenarios; furthermore, this modular design enhances the flexibility and scalability of the system in complex tasks, enabling it to quickly respond to changes in business requirements.

[0095] Furthermore, the large model automated evaluation platform includes multi-dimensional performance evaluation, automated testing and quantitative feedback, and model version comparison and error analysis;

[0096] Among them, multi-dimensional performance evaluation is used to evaluate the performance of the model in multiple dimensions such as accuracy, response time, and resource consumption;

[0097] Automated testing and quantitative feedback are used for scenario-based automated testing functions, quickly evaluating the performance of the model in different business scenarios, generating quantitative feedback for each fine-tuning module, and guiding model optimization;

[0098] Model version comparison and error analysis are used to conduct horizontal comparative analysis of different model versions, and through functions such as automatic scoring, error classification, and detailed reports, quickly identify the advantages and disadvantages of the model, providing a scientific basis for continuous iterative optimization;

[0099] The performance evaluation of the model in different business scenarios includes:

[0100] Extract a small part of representative datasets from banking business scenarios (usually through sampling techniques, such as random sampling or stratified sampling) (for example, for the intelligent customer service task, the samples include common user questions and frequently asked questions);

[0101] Load predefined test tasks and run the model for prediction, and calculate the model performance immediately using preset metrics (such as accuracy, response time);

[0102] For each task, the system only tests key scenarios and metrics (for example, for the intelligent customer service task, only the fluency of multi-round conversations and semantic consistency are tested, and for the report generation task, only the accuracy of key data extraction is tested);

[0103] Generate a summary report based on the evaluation results, pointing out the strengths and weaknesses of the model;

[0104] The comparison and error analysis of the model versions (horizontal comparison and analysis of different model versions) include:

[0105] Quantify the results for each dimension (accuracy, response time, etc.) to generate scores (for example, accuracy: 95%; response time: 450 milliseconds (better than the standard 500 milliseconds));

[0106] Set weights according to task priorities and calculate the total score of the model;

[0107] Subsequently, an evaluation report is automatically generated, which includes detailed data, the advantages and disadvantages of the model, and improvement suggestions;

[0108] The guidance for model optimization includes:

[0109] Regard the module corresponding to the lowest score in the evaluation report as the main bottleneck (for example, the response time of a certain module is higher than the standard, and the semantic consistency score in the multi-round conversation task is low);

[0110] Adopt model pruning or quantization to technically optimize the module (for example, for the high response time problem, through quantitative analysis, the root cause of the problem is found, such as excessive model parameters, and model pruning or quantization techniques can be used for optimization. For the multi-round conversation problem, increase the training data related to the task or improve the fine-tuning module);

[0111] Specifically, in the intelligent customer service scenario, the quantitative feedback shows that the "semantic consistency score is low" (80%); according to the feedback, the system recommends improving the context understanding ability: increasing the data fine-tuning for multi-round conversations and optimizing the weight allocation of the LoRA fine-tuning module;

[0112] The automatic scoring includes:

[0113] The system records the input and output results of the model in real time during task execution; calculates the scores according to the corresponding metric formulas (such as accuracy, response time); synthesizes the total score through the task weight mechanism, that is, obtains the result of automatic scoring;

[0114] The error classification includes:

[0115] The system classifies the model outputs, marking the correct and incorrect results; performs clustering analysis on the incorrect results and attributes them to specific problem types (such as semantic understanding errors, knowledge base call failures). For example, in the intelligent customer service task, for the "loan application process" question, if the system fails to answer correctly and is misclassified as "knowledge shortage";

[0116] The detailed report is used to automatically generate a visual report, which includes evaluation metric scores, error classification statistics, and optimization suggestions;

[0117] Specifically, the large model automated evaluation platform can comprehensively reflect the applicability and stability of the model in the banking scenario through multi-dimensional performance evaluation; automated testing and quantitative feedback can quickly evaluate the performance of the model in different business scenarios through scenario-based automated testing functions, generate quantitative feedback for each fine-tuning module, and guide model optimization; model version comparison and error analysis support horizontal comparative analysis of different model versions, and through functions such as automatic scoring, error classification, and detailed reports, quickly identify the advantages and disadvantages of the model, providing a scientific basis for continuous iterative optimization;

[0118] It should be noted that the quick evaluation model has the following advantages: through sample simplification and parallel computing, the evaluation time is usually a few minutes, with high time efficiency; no manual intervention is required, and the system automatically completes task allocation, execution, and result analysis; the scene coverage is limited, and the quick evaluation only covers some key scenarios, but can screen models or discover obvious problems.

[0119] Furthermore, the large model management platform includes intelligent routing and model adaptation, an AI-Agent development and operation platform, and full life cycle management;

[0120] The intelligent routing and model adaptation automatically selects and integrates the optimal model by combining the results of the large model evaluation platform;

[0121] The AI-Agent development and operation platform is based on a knowledge text-enhanced retrieval algorithm and is used to combine context information and a real-time updated knowledge base;

[0122] The life cycle management is used to cover the model training, evaluation, selection, deployment, and iterative optimization processes, forming a closed-loop management;

[0123] Among them, the AI-Agent development and operation platform introduces the context understanding ability of Transformer-based sequence modeling technologies (such as BERT, GPT);

[0124] Supports content updates of the dynamic knowledge base through incremental update technologies (such as real-time index updates or dynamic vector storage);

[0125] Convert multi-modal information such as text, tables, and images into a unified vector representation, and complete retrieval based on the unified semantic space (for example, in financial analysis, relevant content can be extracted from both text and charts simultaneously);

[0126] Combine user behavior data and task priorities, and use reinforcement learning (such as RLHF, Reinforcement Learning with Human Feedback) to optimize the retrieval result ranking;

[0127] Life cycle management includes: supporting the complete training process from zero-base training to fine-tuning, including technologies such as Prompt Tuning and LoRA fine-tuning;

[0128] Analyze the model performance through multi-dimensional evaluations (such as accuracy, response time, resource consumption, etc.);

[0129] Intelligently select the optimal model based on the evaluation results, support cloud, edge, and local deployments, meet the high requirements of the financial industry for data security, and continuously optimize the model according to the feedback during operation;

[0130] It should be noted that the multi-scenario multi-base large model engine system has achieved all-round technological innovation from dynamic task fine-tuning, efficient evaluation to intelligent management, which not only improves the adaptability and flexibility of the model in the financial industry, but also significantly enhances the business processing efficiency and the overall performance of the system.

[0131] Furthermore, in the underlying support layer, the data and storage are used to support database systems such as TDSQL and Oceanbase, be compatible with multiple unstructured data formats (such as PDF, DOCX, JPEG, etc.), and support distributed file storage (DFS, Minio, Elasticsearch);

[0132] The security guarantee system includes internal control security audit, system security, network security, and privacy protection to ensure the data compliance and security of the system in financial scenarios;

[0133] The computing and reasoning support high-performance computing devices such as a certain AI processor and GPU (such as A100, V100) to improve the model training and reasoning speed;

[0134] The DevOps and operation and maintenance provide an automated testing, CI / CD process, and BUG feedback platform to achieve efficient development and operation and maintenance integration.

[0135] Furthermore, when the multi-task instruction fine-tuning platform conducts training, multiple task modules are integrated into a unified model base through LoRA technology;

[0136] Adopt a multi-expert model (MoE) framework to dynamically activate corresponding LoRA fine-tuning modules according to input task characteristics. Through a weighting or selection mechanism, only load the modules required for the current task;

[0137] Support the dynamic adaptation ability for multi-task processing. By activating different LoRA fine-tuning modules to handle tasks such as question answering, sentiment analysis, and financial calculations, ensure that the system can quickly respond to the diverse needs of the banking industry.

[0138] Furthermore, when the multi-task instruction fine-tuning platform is trained, it also includes an early warning mechanism. The way the early warning mechanism issues warnings is as follows:

[0139] During system operation, collect multiple key data related to resources in real time. The key data includes computing resources, memory and storage, network, DNS health check (specifically, computing resources include GPU / NPU utilization rate, monitoring video memory (Memory) utilization rate and occupancy of computing cores (ComputeCoreUtilization); its key metrics: GPU core utilization rate (such as reaching more than 90%), video memory utilization rate (such as reaching more than 95% or video memory overflow), temperature (such as exceeding the recommended temperature, usually 70°C - 80°C), CPU utilization rate: monitor the utilization rate of the host CPU, especially during the data preprocessing stage. Key metrics: single-core / multi-core occupancy rate exceeding 85%, data I / O latency. Memory and storage include memory (RAM), detect whether there is memory leakage or memory overflow, key metrics: memory utilization rate exceeding 90%, frequently triggered memory paging (PageSwapping). Storage (Disk): insufficient disk capacity or read / write speed bottleneck, key metrics: available disk space below 10%, abnormal low IOPS (input / output operations per second). Network includes network bandwidth, detect the transmission bandwidth of training data from storage to nodes, key metrics: bandwidth utilization rate exceeding 90%, network jitter (Jitter) and high latency (Latency). DNS health check: detect DNS resolution latency and whether there is resolution failure, key metrics: DNS resolution time greater than 200ms, resolution failure rate exceeding 2%);

[0140] After collecting the key data, identify resource bottlenecks through an anomaly detection algorithm (the anomaly detection algorithm includes any one of threshold-triggered detection, dynamic baseline detection, time series analysis, multi-dimensional index correlation analysis, or pattern matching and AI detection. Among them, pattern matching and AI detection can detect unknown anomalies by identifying known problem patterns and through a deep learning model);

[0141] Notify the user of the identified resource bottlenecks in real time. The notification methods are as follows: display on the dashboard, logging, email, SMS, and send warning messages to the user through instant messaging tools (such as Slack and DingTalk). The content of the notification includes bottleneck descriptions, possible problems, and recommended solutions.

[0142] During the training process, if an abnormal node appears, the system will automatically isolate and detect the abnormal node.

[0143] Identify nodes through an anomaly detection algorithm. Mark nodes with abnormal operations as abnormal nodes, and vice versa, mark them as healthy nodes.

[0144] After automatically isolating the abnormal nodes, continue to monitor the system and take measures to recover the nodes or optimize the system performance.

[0145] The system records the abnormal logs and isolation process in the database.

[0146] And use historical data to optimize the anomaly detection algorithm and the method of automatic isolation.

[0147] The automatic isolation includes dynamic task migration, communication isolation, partial task restart, and resource recovery.

[0148] The dynamic task migration includes migrating the training tasks on the abnormal node to other healthy nodes. For example, use a distributed training framework (such as Horovod or Ray) to support task migration (for example, if the GPU of a certain node fails, pause the tasks on that node and allocate the task shards to standby nodes).

[0149] The communication isolation includes stopping the communication between the abnormal node and other nodes to prevent the spread of the abnormal state and updating the topology between nodes (such as the Ring or Tree structure of distributed training) (for example, if a certain node has network instability problems, the system will disconnect its communication with the cluster and reconfigure the communication topology).

[0150] The partial task restart includes partially restarting the tasks on the abnormal node without affecting the global training process (for example, if the data file read by a certain node is damaged, reload the data shard of that node and restart the task).

[0151] The resource recovery includes releasing the resources occupied by the abnormal node to prevent resource waste (terminating the out-of-control process due to a crash and releasing video memory and memory).

[0152] The method of recovering the node includes: detecting and recovering hardware anomalies caused by temporary problems (such as restarting the node after the temperature drops) and service restart (automatically restarting the crashed service process).

[0153] The optimization of system performance includes adjusting system configuration according to abnormal patterns, dynamically adjusting resource allocation strategies, and optimizing communication topology to improve fault tolerance.

[0154] Specifically, for example, when there is a GPU overheating problem, the phenomenon is that the GPU temperature of a certain node reaches 90°C; the system stops the tasks on this GPU, migrates the tasks to the standby node; reduces the GPU frequency, waits for the temperature to recover and then re-enables it, and updates the heat dissipation strategy to prevent similar problems from recurring; for example, when there is a network latency problem, the phenomenon is that the communication latency between nodes suddenly increases; detection and isolation; the system disconnects the abnormal node and reconfigures the communication topology; then checks the network devices and adjusts the data transmission strategy.

[0155] Furthermore, in the large model automated evaluation platform, task weighting and dynamic scoring are also included. Among them, task weighting and dynamic scoring set a task weighting system according to scenario requirements. For example, financial calculation tasks focus on calculation accuracy, while intelligent customer service tasks pay more attention to language fluency and adaptability, and dynamically generate a comprehensive score to ensure the pertinence and practicality of the evaluation results. The task weighting system includes static weight allocation and dynamic weight allocation;

[0156] For static weight allocation, based on the priority of the business scenario, the weights of each evaluation index are defined;

[0157]

[0158] Among them, Weighted Score represents the weighted score, n is the number of evaluation indexes, w i , i is the weight of the i-th evaluation index, and s i is the score corresponding to the i-th evaluation index. Among them, is equal to 1;

[0159] For dynamic weight allocation, its weights are dynamically adjusted through real-time task characteristics;

[0160] Through a rule engine (such as RuleEngine): Dynamically match the weight template according to task tags (such as "finance", "customer service");

[0161] Through reinforcement learning (ReinforcementLearning): Optimize the weight allocation strategy through historical task data;

[0162] Design multiple expert modules (Experts), and dynamically select the expert most suitable for the current task type corresponding to the current task through a gating network (GatingNetwork);

[0163] Among them, when the gating network selects an expert, it uses task characteristics as input and outputs task weights;

[0164] Use the output result of MoE as the basis for scoring, and generate a comprehensive score in combination with a weighting system;

[0165] The weighting formula becomes:

[0166]

[0167] where Gating(w i ) is the weight dynamically adjusted by the gating network, which dynamically adjusts the original weight w of each metric according to the input context information (such as the characteristics of the current task, historical data, environmental changes, etc.); i perform dynamic adjustment;

[0168] Extract features from the task input (such as text, speech, or structured data), and then obtain the task type (question answering, calculation, generation), data complexity (text length, semantic depth), and business scenario (intelligent customer service, financial calculation, risk review);

[0169] Select appropriate weights from the predefined weight templates according to the extracted features;

[0170] Evaluate the task and calculate the individual scores of each metric;

[0171] Specifically, for example, the task type: the user requests to generate a due diligence report; the task characteristics: the task label: report generation; the data complexity: high (need to process tables and multiple paragraphs of text). Weight assignment: accuracy = 0.6, response time = 0.3, resources = 0.1; Score calculation: accuracy score = 90%, response time score = 85%, resource score = 80%; Substitute the comprehensive score into the above formula (the changed weighting formula) to get 87.5;

[0172] It should be noted that by using the expert assignment and gating network mechanism of MoE, more refined dynamic weighting is achieved; MoE is suitable for scenarios with diverse task types and complex features; in the case of high task complexity, combining the MoE framework can further improve the dynamic adaptation ability and scoring accuracy, especially for the optimization scenarios of diverse task requirements in the financial field.

[0173] Furthermore, in the large model automated evaluation platform, it also includes scenario-based and multi-task evaluation, which provides refined feedback for the model by simulating real business environments (such as intelligent investment advisors, compliance reviews, etc.); the multi-task adaptability evaluation module verifies the stability and output quality of the model during task switching;

[0174] Among them, the method of simulating the real business environment is:

[0175] Collect and analyze common tasks and user requirements in the target business, and define possible inputs and outputs in the scenario (e.g., in the intelligent investment advisor scenario: the user inputs investment goals and risk preferences, and the output is the recommended investment portfolio; input contract or policy text, and the output is the compliance analysis result);

[0176] Define specific task types and their goals for each business scenario (e.g., information extraction task: extract key terms from a document, question answering task: answer user questions based on a knowledge base);

[0177] Collect historical data from real businesses and perform data preprocessing (such as customer questions, contract texts, market data, etc., and data preprocessing includes data cleaning and data annotation);

[0178] Use a generative model (such as GPT) to create simulated input data and simulate diverse interactions of users;

[0179] Based on the scenario task definition, randomly or according to rules generate test tasks (e.g., simulate a user question "How to configure a low-risk investment portfolio?"; input contract text "Are the terms stipulated by Party A in the contract compliant with the tax policy?");

[0180] Input the simulated tasks into the business-related subsystems to ensure that the tasks are consistent with the actual processes (e.g., in the intelligent investment advisor scenario, call real-time market data, and in the contract review scenario, use a dynamic knowledge base);

[0181] Call the target model to complete the tasks and record the output results;

[0182] Define evaluation metrics exclusive to each type of task (e.g., in the intelligent investment advisor scenario: the risk-return ratio and timeliness of the recommended portfolio; in the compliance review scenario: the accuracy and recall rate of clause detection);

[0183] Compare the task output with the expected results and calculate various metrics;

[0184] Annotate and classify the error types in the task output (such as logical errors, knowledge gaps);

[0185] The way of task switching is as follows:

[0186] The system analyzes the input content and extracts task features;

[0187] Based on the task features, classify the input into different task types;

[0188] The system encapsulates the processing logic of each task into an independent module (such as the LoRA fine-tuning module);

[0189] According to the task classification result, dynamically load the corresponding module;

[0190] Specifically, the task description is as follows: The user continuously asks different types of questions: (Question 1: "What is the current loan interest rate?" → Data query task; Question 2: "How to calculate the monthly loan payment?" → Financial calculation task; Question 3: "Are there any recommended low-risk investment products?" → Intelligent investment advisor task);

[0191] The task switching process is as follows: Input parsing: (Question 1: Identify the keyword "loan interest rate" and classify it as a data query task; Question 2: Identify the keyword "monthly payment" and classify it as a financial calculation task; Question 3: Identify the keyword "low-risk investment" and classify it as an intelligent investment advisor task);

[0192] Module loading: (Data query task: Load the knowledge base Q&A module and extract answers from real-time interest rate data; Financial calculation task: Load the financial calculation module and calculate the monthly payment using formulas; Intelligent investment advisor task: Load the recommendation module and call the investment analysis model to generate product recommendations);

[0193] The output results are as follows: (Answer to Question 1: "The current loan interest rate is 4.5%;" Answer to Question 2: "According to the input amount and term, the monthly payment is ￥5000;" Answer to Question 3: "Based on your risk preference, it is recommended to invest in fixed-income products).

[0194] In this embodiment, through the matching and loading of the LoRA fine-tuning module, the appropriate LoRA module can be accurately selected according to the characteristics of the task, improving the task processing efficiency. Moreover, if the task characteristics cover multiple modules, a module combination mechanism can be adopted to improve the adaptability and flexibility of the system; through a unified model base, the overhead of model switching is reduced, and at the same time, by activating the corresponding expert modules, the efficiency and accuracy of task processing are improved. Only the required LoRA modules are loaded according to the task requirements, avoiding the loading of irrelevant modules and optimizing the use of computing resources; through real-time resource monitoring and anomaly detection, problems can be discovered in advance and measures can be taken to prevent system crashes or performance degradation. By automatically isolating abnormal nodes and recycling resources, the need for manual intervention can be greatly reduced, enhancing the autonomy and reliability of the system; multi-dimensional performance evaluation can ensure that the model maintains high performance in different scenarios, avoiding overfitting or underfitting. Multi-task evaluation can ensure that the system performs well in multiple task scenarios, improving task adaptability; through intelligent routing and model adaptation, it is ensured that resources are efficiently configured, reducing ineffective calculations. Lifecycle management can ensure that the model is effectively managed at each stage of its lifecycle, enhancing the continuous value of the model.

[0195] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present invention can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0196] In several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only one way, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0197] As described above, this is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention.

[0198] Finally: The above description is only the preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should all be included in the protection scope of the present invention.

Claims

1. Multi-scene and multi-base large model engine system, characterized by: Including front-end interaction layer, business function layer, core technology layer and underlying support layer; The front-end interaction layer is used to provide access methods, interface protocols, rights management and log records; The business function layer is used to meet the core business needs of multiple scenarios in the financial industry; The core technology layer includes a multi-task instruction fine-tuning platform, a large model automated evaluation platform, and a large model management platform; The underlying support layer includes data and storage, security, computing and reasoning support, and DevOps and operations; The multi-task instruction fine-tuning platform is used to identify real-time task requirements and dynamically load the corresponding LoRA fine-tuning module: Among them, the method of identifying real-time task requirements is: After receiving the task input, the system parses the input content through the front-end interface and extracts the input data; Extract task features from input data through a natural language processing model and map the extracted task features into high-dimensional feature vectors; Assign tasks to specific scenario tags based on task routing rules and analyze the fine-grained characteristics of task requirements; Parse the multimodal data contained in the input to determine the type of data involved in the task; Combined with the task scenario database in the system, the task characteristics are matched with the predefined scenario characteristics to obtain the real-time task requirements; In the LoRA fine-tuning module, multiple LoRA fine-tuning modules are pre-trained and registered, each LoRA fine-tuning module focuses on several scenario requirements; Register task feature labels for each LoRA fine-tuning module, and match real-time task requirements with module registration features through feature extraction; The dynamic loading includes: when the task matches a single LoRA fine-tuning module, dynamically loading the LoRA fine-tuning module; When the task characteristics cover multiple LoRA fine-tuning modules, the module combination mechanism is adopted and multiple LoRA fine-tuning modules are loaded to process the task.

2. The multi-scene multi-base large model engine system according to claim 1 is characterized in that: The access method supports Web, mobile, third-party application API, multi-dimensional terminal interaction and business system access; The interface protocol interacts via HTTP, WebSocket, REST, JSON format and HTML protocol; The authority management and log recording are used to perform multi-level authority management on access users and systems, and record access logs.

3. The multi-scene multi-base large model engine system according to claim 1 is characterized in that: The business function layer includes an intelligent customer service module, an intelligent knowledge base question and answer module, an intelligent report generation module, an intelligent contract review module, a financial product investment education module, an intelligent question and answer module, and an intelligent marketing module.

4. The multi-scene multi-base large model engine system according to claim 1, characterized in that: When the multi-task instruction fine-tuning platform is training, multiple task modules are integrated into a unified model base through LoRA technology; By adopting a multi-expert model framework, the corresponding LoRA fine-tuning module is dynamically activated according to the input task characteristics; through a selection mechanism, only the LoRA fine-tuning module required for the current task is loaded.

5. The multi-scene multi-base large model engine system according to claim 1 is characterized in that: When the multi-task instruction fine-tuning platform is training, it also includes an early warning mechanism, and the early warning mechanism provides an early warning in the following manner: Real-time collection of multiple key resource-related data, including computing resources, memory and storage, network, and DNS health checks; After collecting key data, resource bottlenecks are identified through anomaly detection algorithms and the identified resource bottlenecks are notified to users in real time; Among them, if an abnormal node appears during the training process, the abnormal node will be automatically isolated and detected; Nodes are identified through anomaly detection algorithms, and nodes with abnormal operation are recorded as abnormal nodes, and vice versa, they are recorded as healthy nodes; After automatically isolating the abnormal node, continue monitoring and take measures to restore the node; Record anomaly logs and isolation processes in a database, and use historical data to optimize anomaly detection algorithms and automatic isolation; The automatic isolation includes dynamic task migration, communication isolation, partial task restart and resource recovery.

6. The multi-scene multi-base large model engine system according to claim 1, characterized in that: The large-scale model automated evaluation platform includes multi-dimensional performance evaluation, automated testing and quantitative feedback, as well as model version comparison and error analysis; Among them, multi-dimensional performance evaluation is used to evaluate the performance of the model in multiple dimensions such as accuracy, response time, and resource consumption; Automated testing and quantitative feedback are used for scenario-based automated testing functions; Model version comparison and error analysis are used to perform horizontal comparative analysis of different model versions.

7. The multi-scene multi-base large model engine system according to claim 1, characterized in that: The large model management platform includes intelligent routing and model adaptation, AI-Agent development and operation platform, and full life cycle management; The intelligent routing and model adaptation automatically selects and integrates the optimal model by combining the results of the large model evaluation platform; The AI-Agent development and operation platform is based on an enhanced retrieval algorithm for knowledge texts and is used to combine contextual information with a real-time updated knowledge base; The lifecycle management is used to cover the model training, evaluation, selection, deployment and iterative optimization process, forming a closed-loop management.

8. The multi-scene multi-base large model engine system according to claim 1, characterized in that: The large-model automated evaluation platform also includes task weighting and dynamic scoring, wherein task weighting and dynamic scoring set up a task weighting system according to scenario requirements, and the task weighting system includes static weight allocation and dynamic weight allocation.

9. The multi-scene multi-base large model engine system according to claim 1, characterized in that: The large-scale model automated evaluation platform also includes scenario-based and multi-task evaluations, which provide feedback and verification on the model by simulating real business environments and multi-task adaptability evaluations.

Images