Large model application computing power-model-application break-through method

Abstract

The invention discloses a large model application computing power-model-application break-through method. The break-through method comprises the steps that an application building platform, a knowledge base platform, a large model square, a tool platform and a computing platform are built; designing a loose coupling module; independent life cycle management; the computing power, the model and the application are communicated from bottom to top; isolating a self-built tenant system from a data surface; modularized supply of application functions is realized; and zero-shutdown knowledge updating. The problem that the whole package must be restarted during upgrading in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of introducing a micro-service discovery mechanism in a large model application system, and in particular to a large model application computing power-model-application connection method. BACKGROUND

[0002] The rise of large language models has given rise to a large number of industry-level applications, such as intelligent customer service, knowledge Q&A, and report generation.

[0003] Dify, Coze, LangChain-SaaS, and other "one-stop" large model application development platforms have appeared at home and abroad. After large model applications enter the production stage, enterprises generally require "knowledge cold update without service interruption, model hot upgrade without interruption, and tool plug-ins with gray release". The existing mainstream platforms (Dify, Coze, LangChain-SaaS, etc.) use a "monolithic + plug-in" architecture: Knowledge management, hint engineering, tool execution, and model reasoning are in the same process and interact through local function calls; There is no unified service discovery, and upgrading the knowledge base requires restarting the entire container, causing online Q&A services to be interrupted; GPU training / fine-tuning tasks and online reasoning are separated into two operation and maintenance channels, and application-side feedback cannot automatically flow back to the training stage, resulting in a broken "application-model" closed loop.

[0004] Chinese patent application for invention with publication number CN120276723A discloses a code generation method, device and medium based on model low-code application, to solve the problem that the existing technology sacrifices the scalability and adaptability of the system due to platform limitations, and traditional coding cannot achieve the rapid trial and error and continuous iteration efficiency required by modern business environment. The method includes: based on the low-code platform and through the menu, webpage, and form, an application model is constructed, and the application model is compressed and uploaded to a remote object storage service; the target model determined in the application model is subjected to version checking, and the target model after the checking is passed is analyzed to generate front-end and back-end codes corresponding to the target model; the front-end and back-end codes are pushed to a GitLab repository, and based on the pipeline configuration in the GitLab repository, an automatic process of the back-end code is realized to achieve automatic operation of code generation. Through the above scheme, the development efficiency is greatly improved, the risk of manual coding is reduced, and the user's demand is met at a low cost. SUMMARY

[0005] In view of the above problems, the present application is proposed to provide a large model application computing power-model-application connection method to overcome the above problems or at least partially solve the above problems.

[0006] According to one aspect of the present application, a large model application computing power-model-application connection method is provided, which comprises: Build application building platform, knowledge base platform, large model square, tool platform and computing platform; Loosely coupled module design; Independent life cycle management; Computing power-model-application from bottom to top; Self-built tenant system and data surface isolation; Application function modularization supply; Zero downtime knowledge update.

[0007] Optionally, the application building platform, knowledge base platform, large model square, tool platform and computing platform specifically include: Application building platform: responsible for prompt template, process canvas, permission, session management; Knowledge base platform: provides chunk analysis, vectorization, multi-level indexing, permission isolation; Large model square: provides model weight, inference image, elastic scaling, A / B test; Tool platform: provides plug-in registration, parameter verification, sandbox execution, OAuth authorization; Computing platform: provides GPU / CPU hybrid, serverless inference, cold start acceleration, monitoring and alarm.

[0008] Optionally, the loosely coupled module design specifically includes: Modules only expose "function semantics" rather than specific URIs: KBP mainly provides "knowledge recall" semantics = f(query, top_k, filter) → List <chunk>; MMP primarily provides "model inference" semantics = f(model_id, prompt, param)→ Iterator <token>; TP mainly provides "tool execution" semantics = f(tool_name, args) → JSON; CP mainly provides "computing power application" semantics = f(gpu_type, count, duration) → ResourceTicket Module internal can be freely replaced implementation (Milvus→PgVector, vLLM→TensorRT), upstream no sense; Mesh automatically switches according to version routing strategy.

[0009] Optionally, the independent life cycle management specifically includes: Each sub-platform Helm chart is independent, and each module has an independent Deployment; Upgrade strategy: rolling + canary release, OAP does not need to restart; Each module has a rollback window, and single-module rollback will not trigger restart of other modules; The configuration center is based on etcd and supports watching by platform prefix; Observation: Each sub-platform outputs OTel format trace, and the trace header is injected in OAP, and the cross-platform link is complete.

[0010] Optionally, the bottom-up power-model-application specifically includes: Online logs → Kafka → Flink real-time cleaning → training sample pool → CP automatically submits training / LoRA fine-tuning tasks → outputs new weights → pushes to MMP model repository → OAP through version routing to gray new model → application effect index back to Kafka; Sample automatic generation rules are written to OAP canvas and saved with business version; CP provides "training as a service" API with built-in unified GPU resource pool; After the completion of the training task, security scanning and model card generation are automatically triggered, and models that do not meet the scale cannot be registered to MMP.

[0011] Optionally, the self-built tenant system is isolated from the data plane, specifically including: The tenant ID runs through all modules, and Kubernetes Namespace-level isolation is implemented; The four-layer resources of knowledge, model, tool, and computing resource quota are bound to the tenant tag; The storage layer uses encrypted object buckets + SSE-KMS, and the keys are different when sharing physical disks across tenants; Audit logs are independently dumped to tenant-specific Loki instances, meeting the financial compliance requirement of 6 months of traceability.

[0012] Optionally, the application function modularization supply specifically includes: The OAP splits the traditional Agent into reusable "ability blocks": KnowledgeBlock, ModelBlock, ToolBlock, ComputingBlock four categories; Through semantic ID, the application logic is completed, and the specific module version is dynamically bound at runtime; The same ability block can be referenced by multiple applications, realizing enterprise-level ability sedimentation; Block-level monitoring is independent, facilitating business and IT focusing on SLA.

[0013] Optionally, the zero-downtime knowledge update specifically includes: The operation and maintenance personnel uploads a new PDF, and the KBP generates a V2 vector by analysis; The KBP registers the knowledge-v2 subset to the Mesh; The OAP routes internal staff traffic to v2 according to user tags; The monitoring shows that the hit rate is improved by 8%, there is no error, and full switching; The old version V1 is automatically downlinked after 30 minutes, and the whole application layer is not restarted.

[0014] The application provides a large model application computing-power-model-application connection method, the connection method includes: constructing an application building platform, a knowledge base platform, a large model square, a tool platform and a computing platform; loose coupling module design; independent life cycle management; computing-power-model-application from bottom to top; self-built tenant system and data face isolation; application function modularization supply; zero downtime knowledge update. The problem that the existing technology upgrade must be restarted as a whole is solved.

[0015] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, the content of the specification can be implemented, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0017] Figure 1 A flowchart of a large model application computing power-model-application through method provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0019] The terms "include" and "have" and any variations thereof in the specification, claims and drawings of the present disclosure are intended to cover the non-exclusive inclusion, for example, the inclusion of a series of steps or units.

[0020] The technical solutions of the present application will be described in further detail below in conjunction with the drawings and embodiments.

[0021] As shown in Figure 1 The system provided by the present application is composed of "one application building platform + four sub-platforms", and each platform only exposes services through unified IDL, with complete separation of data plane and control plane: Application building platform (OAP, Orchestration & Application Platform): responsible for prompting templates, process canvas, permissions, and session management; Knowledge base platform (KBP, Knowledge Base Platform): provides chunk parsing, vectorization, multi-level indexing, and permission isolation; Large model square (MMP, Model Market Platform): provides model weights, inference images, elastic scaling, and A / B testing; Tool platform (TP, Tools Platform): provides plug-in registration, parameter verification, sandbox execution, and OAuth authorization; Computing platform (CP, Computing Platform): provides GPU / CPU hybrid deployment, serverless inference, cold start acceleration, and monitoring and alarm.

[0022] Each module is registered in the same Service Mesh (Istio) in the form of an independent microservice, and shares the following infrastructure: Registry center: etcd + gRPC Health Protocol; Configuration center: Apollo is isolated by "module-tenant-environment" in three levels; Event bus: Kafka topics per domain (knowledge / model / tool / compute); Observable: OpenTelemetry + Prometheus, trace headers injected uniformly by OAP.

[0023] Loosely coupled module design: Modules expose only "functional semantics" and not concrete URIs, e.g.: KBP provides primarily "knowledge recall" semantics = f(query, top_k, filter) → List <chunk> MMP primarily provides "model inference" semantics = f(model_id, prompt, param)→ Iterator <token>; TP mainly provides "tool execution" semantics = f(tool_name, args) → JSON; CP mainly provides "computing power application" semantics = f(gpu_type, count, duration) → ResourceTicket; Module internal free replacement implementation (Milvus→PgVector, vLLM→TensorRT), upstream unaware; Mesh automatically switches according to version routing strategy.

[0024] Independent life cycle management: Each sub-platform Helm chart is independent, and each module has an independent Deployment; Upgrade strategy: rolling + canary release, OAP does not need to restart; Each module has a rollback window, and single-module rollback will not trigger restart of other modules The configuration center is based on etcd and supports watching by platform prefix; Observability: Each sub-platform outputs OTel format trace, and the trace header is injected in OAP, and the cross-platform link is complete.

[0025] Computing power-model-application from bottom to top: Process closed-loop design: Online log → Kafka → Flink real-time cleaning → training sample pool → CP automatically submits training / LoRA fine-tuning task → produces new weight → pushes to MMP model repository → OAP uses version routing to gradually update new model → application effect index writes back to Kafka.

[0026] Key steps: Sample auto-generation rules are written to OAP canvas and saved together with business version; CP provides "training as a service" API with built-in unified GPU resource pool; After the completion of the training task, security scanning and model certificate generation are automatically triggered, and models that do not meet the scale cannot be registered to MMP.

[0027] Self-built tenant system is isolated from the data plane Tenant ID runs through all modules, and Kubernetes Namespace level isolation is used; Knowledge, model, tool, and computing resource quotas are bound to tenant tags at four levels; The storage layer uses encrypted object buckets + SSE-KMS, and the keys are different when sharing physical disks across tenants; Audit logs are independently dumped to a tenant-specific Loki instance, meeting the financial compliance requirement of 6 months of traceability.

[0028] Application function modularization supply (instead of "low-code arrangement" expression): OAP splits traditional agents into reusable "capability blocks": KnowledgeBlock, ModelBlock, ToolBlock, and ComputingBlock. Each block is arranged and referenced according to application logic through "semantic ID", and is dynamically bound to specific module versions at runtime. The same capability block can be referenced by multiple applications, enabling enterprise-level capability sedimentation. Block-level monitoring is independent, allowing business and IT to focus on SLA.

[0029] Zero-downtime knowledge update: The operations personnel upload a new PDF, and KBP generates V2 vectors through analysis; KBP registers the knowledge-v2 subset to the Mesh, with only 5% data traffic; OAP routes internal employee traffic to v2 based on user tags; Monitoring shows an 8% increase in hit rate with no errors, and full switching; The old version V1 is automatically downlinked after 30 minutes, and there is no restart at the application layer throughout the process.

[0030] Advantages: Microservice "1+4" module architecture: knowledge, model, tool, and computing are split into independent microservices, and version routing, circuit breaking, and gray release are achieved through Service Mesh, solving the problem of whole package restart during technology upgrade.

[0031] Power-model-application vertical closed-loop mechanism: using event bus + computing platform training as a service, automatically flowing online logs back to training tasks to achieve full-automatic connection from application requirements to model iteration.

[0032] Tenant-level Namespace + encrypted storage + independent audit data plane isolation solution meets the multi-tenant compliance requirements of finance, government, and other industries.

[0033] Functionality-based module abstraction: module internal implementation is replaceable, and the upstream canvas is unaware, enabling enterprise-level capability sedimentation and reuse.

[0034] Introducing microservice discovery mechanism in large model application systems enables knowledge, model, tool, and computing to be independently started, stopped, gray released, and regulated. Connecting four-domain capabilities through standardized interfaces enables upper-layer applications to still be "low-code" arranged.

[0035] Let "online application feedback" automatically flow back to "model training / fine-tuning" demand, and open up the vertical closed loop of computing power→model→application; Meet the different requirements of different tenants for data isolation, compliance deployment, and cost billing within the same platform.

[0036] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.< / token> < / chunk> < / token> < / chunk>

Claims

1. A method for integrating computing power, model, and application in large-scale model applications, characterized in that: The method for achieving the connection includes: Build an application platform, a knowledge base platform, a large model marketplace, a tool platform, and a computing platform; Loosely coupled module design; Independent lifecycle management; Computing power, models, and applications are integrated from the bottom up. Self-built tenant system and data plane isolation; Modular supply of application functions; Zero-downtime knowledge update.

2. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The application building platform, knowledge base platform, large model square, tool platform, and computing platform specifically include: Application building platform: responsible for prompt templates, workflow canvas, permissions, and session management; Knowledge base platform: provides chunk parsing, vectorization, multi-level indexing, and permission isolation; Large Model Plaza: Provides model weights, inference mirroring, elastic scaling, and A / B testing; Tool platform: Provides plugin registration, parameter validation, sandbox execution, and OAuth authorization; Computing platform: Provides GPU / CPU co-deployment, serverless inference, cold start acceleration, and monitoring and alerting.

3. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The loosely coupled module design specifically includes: The module exposes only "functional semantics" rather than specific URIs to the outside world: KBP primarily provides the semantics of "knowledge retrieval" = f(query, top_k, filter) → List <chunk> ；< / chunk> MMP primarily provides the semantics of "model inference" = f(model_id, prompt, param) → Iterator <token> ；< / token> TP primarily provides "tool execution" semantics = f(tool_name, args) → JSON; CP primarily provides the semantics of "computing power request" = f(gpu_type, count, duration) → ResourceTicket The module can be freely replaced (Milvus→PgVector, vLLM→TensorRT) without being detected by the upstream; the Mesh automatically switches according to the version routing policy.

4. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The independent lifecycle management specifically includes: Each sub-platform has an independent Helm chart, and each module has an independent Deployment. Upgrade strategy: Rolling + canary release, OAP requires no restart; Each module is configured with a rollback window; rolling back a single module will not trigger the restart of other modules. The configuration center is based on etcd and supports watching by platform prefix; Observable: Each sub-platform outputs an OTel format trace, with the trace header uniformly injected in OAP, ensuring a complete cross-platform trace.

5. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The bottom-up integration of computing power, model, and application specifically includes: Online logs → Kafka → Flink real-time cleaning → training sample pool → CP automatically submits training / LoRA fine-tuning tasks → produces new weights → pushes to MMP model repository → OAP routes new models for canary release via version → application performance metrics are written back to Kafka; The rules for automatically generating samples are written into the OAP canvas and saved along with the business version; CP provides a "training-as-a-service" API with a built-in unified GPU resource pool; After the training task is completed, a security scan and model card generation will be automatically triggered. Non-compliant models cannot be registered with MMP.

6. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The isolation between the self-built tenant system and the data plane specifically includes: Tenant ID is used across all modules, with Kubernetes namespace-level isolation; All four layers of resources—knowledge, models, tools, and computing resource quotas—are bound to tenant tags; The storage layer uses encrypted object buckets + SSE-KMS, and the keys are different when sharing physical disks across tenants; Audit logs are independently transferred to the tenant's dedicated Loki instance to meet the 6-month traceability requirement for financial compliance.

7. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The modular supply of application functions specifically includes: OAP breaks down traditional agents into four reusable "capability blocks": KnowledgeBlock, ModelBlock, ToolBlock, and ComputingBlock. Semantic IDs are used to orchestrate and reference applications according to their logic, and are dynamically bound to specific module versions at runtime. The same capability block can be referenced by multiple applications, enabling enterprise-level capability accumulation; Independent block-level monitoring allows business and IT departments to focus on their respective SLAs.

8. The method for integrating computing power, model, and application in a large-scale model application according to claim 1, characterized in that, The zero-downtime knowledge update specifically includes: The operations and maintenance personnel upload a new PDF, and the KBP parses it to generate a V2 vector; KBP registers a subset of knowledge-v2 with Mesh; OAP routes internal employee traffic to v2 based on user tags; Monitoring showed an 8% increase in hit rate, with no errors, and a full switch was performed. The older version V1 automatically disconnects from the chain after 30 minutes, with no application layer restarts throughout the process.

Images