Large language model dynamic adaptation method and system based on Java

Abstract

The invention discloses a Java-based large language model dynamic adaptation method and system, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of overcoming the interface difference of multiple model interfaces, simplifying the development process and providing standardized and high-expansibility large model management. Comprising the following steps: providing a model registration and dynamic loading service, a protocol conversion and parameter standardization service and a load balancing and failover service; a streaming transmission protocol service, a function call dynamic injection service and a global error processing mechanism are provided; when a user provides a document analysis and partitioning service, a vectorization index construction service and an RAG enhanced generation service to ask questions, relevant document blocks in the Pinecone are retrieved through the RAG enhanced generation service to serve as contexts to be injected into cue words of the large language model, and answers generated by the large language model are returned.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a large language model dynamic adaptation method and system based on Java. BACKGROUND

[0002] Due to differences in training data, architecture and interface protocol, large language models (LLM) such as GPT-4, GLM, etc. require developers to write separate calling codes for each model during integration. For example, OpenAI's API uses JSON format REST requests, while ZhiPu may use Protobuf protocol, resulting in low development efficiency and high maintenance cost.

[0003] Technical challenges of streaming output and function calls Traditional interfaces usually only support returning complete results at a time, making it difficult to meet real-time interaction requirements (such as word-by-word generation). In addition, existing platforms lack streaming support for parameter passing of function calls (Tool Calls), and cannot dynamically adjust the execution logic of complex tasks.

[0004] Fragmentation of multi-modal and vectorization processing Multi-modal tasks (such as image recognition, speech-to-text) require reliance on multiple independent services (such as Whisper, CLIP), and document vectorization often requires additional integration of Elasticsearch or FAISS. Existing solutions lack a unified interface, resulting in code redundancy and performance bottlenecks.

[0005] How to overcome the interface differences of multiple models, simplify the development process, and provide standardized and highly expandable large model management is a technical problem that needs to be solved. SUMMARY

[0006] The technical task of the present application is to overcome the above shortcomings, provide a large language model dynamic adaptation method and system based on Java, to solve the technical problem of how to overcome the interface differences of multiple models, simplify the development process, and provide standardized and highly expandable large model management.

[0007] In a first aspect, the present application provides a large language model dynamic adaptation method based on Java, applied to a management platform for multiple types of large language models, comprising the following steps:

[0008] Multi-model access: providing model registration and dynamic loading services, protocol conversion and parameter standardization services, and load balancing and failover services. For newly added large models, the model registration and dynamic loading service is used to load the adapter. For input user requests, the protocol conversion and parameter standardization service is used to convert the user request into the format required by the target large model and deserialize and encapsulate the response result, and the load balancing and failover service is used to select available service nodes to send the request.

[0009] Streaming output and error handling: Provides streaming protocol services, function call dynamic injection services, and a global error handling mechanism. The streaming protocol service is used for the reactive programming model based on Spring WebFlux, providing two streaming interfaces: SSE and WebSocket. During the streaming response process, if the large model returns a function call request, the external API is triggered based on the function call dynamic injection service and the result is injected into the subsequent response stream. The global error handling mechanism is used to intercept all large language model call exceptions through AOP, classify and handle various exception scenarios, and return standardized error codes.

[0010] Integration with RAG and vector databases: Provides document parsing and chunking services, vectorized index building services, and RAG enhanced generation services. The document parsing and chunking service parses and chunks the user-input document, and the vectorized index building service generates vectors from the chunked documents and writes them to Pinecone. When a user asks a question, the RAG enhanced generation service retrieves relevant document chunks from Pinecone, injects them as context prompts into the large language model, and returns the answer generated by the large language model.

[0011] As a preferred approach, multi-model access includes the following operations:

[0012] Provides model registration and dynamic loading services. Based on the model registration and dynamic loading services, the following operations are supported: Using the factory pattern and Spring Bean dynamic registration mechanism, users can declare model parameters in the configuration file of the large language model. The model parameters include model type, API key, and request timeout. When adding a new large language model, add configuration items including model parameters to the configuration file of the large language model. Based on the configuration items, load the adapter class of the large model video. The adapter is automatically loaded through classpath scanning without restarting the service.

[0013] Provides protocol conversion and parameter standardization services. Based on the protocol conversion and parameter standardization services, the following operations are supported: Define a unified request / response DTO for the interface protocols of different large language models. The adapter internally completes the protocol conversion. For user input requests, the request object is converted into the format required by the target large language model, and the response result is deserialized and encapsulated.

[0014] Provides load balancing and failover services, and based on the load balancing and failover services, it provides the following operations: supports multi-instance model service deployment, distributes request traffic through a weighted round-robin algorithm, and when a service node is unavailable, the adapter automatically switches to the standby node and records error logs.

[0015] As a preferred embodiment, streaming output and error handling include the following operations:

[0016] Provide streaming protocol support services, which provide the following operations: Based on the Spring WebFlux reactive programming model, provide two streaming interfaces, SSE and WebSocket. For non-Spring environments, asynchronous chunked return is implemented through Java's native CompletableFuture. When a user initiates a streaming conversation request, the server returns a token sequentially and keeps the connection active.

[0017] Provides a function call dynamic injection service, which provides the following operations: During the streaming response process, if the model returns a function call request, it will trigger an external API in real time and inject the result into the subsequent response stream, realizing a coherent process of thinking-execution-feedback;

[0018] A global error handling mechanism is provided, which provides the following operations: intercepting all call exceptions of large language models through AOP, classifying and handling network exception scenarios including network timeout, authentication failure, and insufficient quota, and returning standardized error codes. At the same time, an automatic retry strategy is provided, which allows for a maximum of predetermined number of retries for recoverable errors, including temporary network jitter.

[0019] As a preferred option, integrating RAG with vector databases includes the following operations:

[0020] Provides document parsing and chunking services, offering the following operations based on document parsing and chunking: integrates Apache Tika to parse various types of documents, chunks the documents according to semantic boundaries and fixed lengths, where if the document is a technical document, it is split into chapters and metadata is generated for each chunk, including source and page number;

[0021] It provides vectorization and index building services, and offers the following operations based on these services: For chunked documents, it uses the Sentence-BERT model to generate document vectors, writes the vector data in batches through Pinecone's gRPC interface, and supports hybrid indexing strategies to improve retrieval speed and accuracy.

[0022] RAG-enhanced generation is provided, and the following operations are offered based on the RAG-enhanced generation service: when a user asks a question, the top-K relevant document blocks in Pinecone are retrieved, and the retrieved relevant document blocks are injected as context prompts into the large language model.

[0023] Secondly, the present invention provides a management platform for large language models of multiple types, including a large model access module, a streaming output and error handling module, and an integrated RAG and vector database module.

[0024] The multi-model access module is used to perform the following: provide model registration and dynamic loading services, protocol conversion and parameter standardization services, and load balancing and failover services. For newly added large models, it loads adapters based on the model registration and dynamic loading services. For input user requests, it converts user requests into the format required by the target large model based on the protocol conversion and parameter standardization services, deserializes and encapsulates the response results, and selects available service nodes to send requests based on the load balancing and failover services.

[0025] The streaming output and error handling module is used to perform the following: provide streaming protocol services, function call dynamic injection services, and a global error handling mechanism. The streaming protocol service is used for the reactive programming model based on Spring WebFlux, providing two streaming interfaces: SSE and WebSocket. During the streaming response process, if the large model returns a function call request, the external API is triggered based on the function call dynamic injection service and the result is injected into the subsequent response stream. The global error handling mechanism is used to intercept all large language model call exceptions through AOP, classify and handle various exception scenarios, and return standardized error codes.

[0026] The integrated RAG and vector database module performs the following functions: providing document parsing and chunking services, vectorized index building services, and RAG enhancement generation services. The document parsing and chunking service parses and chunks the user-input document, and the vectorized index building service generates vectors from the chunked documents and writes them to Pinecone. When a user asks a question, the RAG enhancement generation service retrieves relevant document chunks from Pinecone, injects them as context prompts into the large language model, and returns the answer generated by the large language model.

[0027] As a preferred embodiment, the multi-model access module is used to provide the following operations:

[0028] Provides model registration and dynamic loading services. Based on the model registration and dynamic loading services, the following operations are supported: Using the factory pattern and Spring Bean dynamic registration mechanism, users can declare model parameters in the configuration file of the large language model. The model parameters include model type, API key, and request timeout. When adding a new large language model, add configuration items including model parameters to the configuration file of the large language model. Based on the configuration items, load the adapter class of the large model video. The adapter is automatically loaded through classpath scanning without restarting the service.

[0029] Provides protocol conversion and parameter standardization services. Based on the protocol conversion and parameter standardization services, the following operations are supported: Define a unified request / response DTO for the interface protocols of different large language models. The adapter internally completes the protocol conversion. For user input requests, the request object is converted into the format required by the target large language model, and the response result is deserialized and encapsulated.

[0030] Provides load balancing and failover services, and based on the load balancing and failover services, it provides the following operations: supports multi-instance model service deployment, distributes request traffic through a weighted round-robin algorithm, and when a service node is unavailable, the adapter automatically switches to the standby node and records error logs.

[0031] Preferably, the streaming output and error handling module is used to provide the following operations:

[0032] Provide streaming protocol support services, which provide the following operations: Based on the Spring WebFlux reactive programming model, provide two streaming interfaces, SSE and WebSocket. For non-Spring environments, asynchronous chunked return is implemented through Java's native CompletableFuture. When a user initiates a streaming conversation request, the server returns a token sequentially and keeps the connection active.

[0033] Provides a function call dynamic injection service, which provides the following operations: During the streaming response process, if the model returns a function call request, it will trigger an external API in real time and inject the result into the subsequent response stream, realizing a coherent process of thinking-execution-feedback;

[0034] A global error handling mechanism is provided, which provides the following operations: intercepting all call exceptions of large language models through AOP, classifying and handling network exception scenarios including network timeout, authentication failure, and insufficient quota, and returning standardized error codes. At the same time, an automatic retry strategy is provided, which allows for a maximum of predetermined number of retries for recoverable errors, including temporary network jitter.

[0035] Preferably, the RAG and vector database modules are integrated to provide the following operations:

[0036] Provides document parsing and chunking services, offering the following operations based on document parsing and chunking: integrates Apache Tika to parse various types of documents, chunks the documents according to semantic boundaries and fixed lengths, where if the document is a technical document, it is split into chapters and metadata is generated for each chunk, including source and page number;

[0037] It provides vectorization and index building services, and offers the following operations based on these services: For chunked documents, it uses the Sentence-BERT model to generate document vectors, writes the vector data in batches through Pinecone's gRPC interface, and supports hybrid indexing strategies to improve retrieval speed and accuracy.

[0038] RAG-enhanced generation is provided, and the following operations are offered based on the RAG-enhanced generation service: when a user asks a question, the top-K relevant document blocks in Pinecone are retrieved, and the retrieved relevant document blocks are injected as context prompts into the large language model.

[0039] The Java-based dynamic language model adaptation method and system of the present invention have the following advantages:

[0040] 1. Significantly Reduced Development Costs: By abstracting common logic through a unified adaptation layer, developers only need to focus on business differences. For example, when adding a DeepSeek model, the amount of code is reduced from 300 lines to 50 lines (only the protocol conversion class needs to be implemented), and the development cycle is shortened from 2 days to 3 hours. Furthermore, standardized interfaces decouple business logic from the model, eliminating the need to modify core code when switching model providers, reducing maintenance costs by over 60%.

[0041] 2. Improve real-time interaction efficiency and user experience: Streaming output enables word-by-word return (latency <200ms), allowing users to instantly perceive the generation process. For example, in intelligent customer service scenarios, users immediately see the intermediate state "Querying order information..." after entering a question, avoiding user churn due to long waiting times. Simultaneously, the streaming function call mechanism allows dynamic calls to external APIs (such as real-time weather data) during answer generation, further reducing end-to-end latency.

[0042] 3. Flexible scalability and technological foresight: Modular design supports rapid integration of emerging models and technologies. For example, when high-speed inference engines such as Groq emerge, developers only need to implement their adapters and register them to access them without modifying the platform's core code. In addition, the built-in multimodal interface can be seamlessly extended to scenarios such as video analysis and 3D model processing. For example, when integrating the Whisper V3 audio model, only an audio preprocessing module needs to be added to reuse the existing pipeline.

[0043] 4. Resource Utilization and System Stability Optimization: Connection pooling (e.g., Apache HttpClient Pool) reuses LLM API connections, reducing TCP handshake overhead and increasing single-node QPS (queries per second) from 100 to 500. The asynchronous non-blocking architecture (WebFlux) reduces thread blocking rates, enabling the simultaneous handling of 5000+ concurrent streaming requests on an 8-core server. Furthermore, error retry and degradation strategies (e.g., automatic switching to a lower-priority model after rate limiting) ensure service SLA. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] The invention will be further described below with reference to the accompanying drawings.

[0046] Figure 1 This is a flowchart of a Java-based dynamic language model adaptation method, as shown in Example 1. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0048] This invention provides a Java-based method and system for dynamic adaptation of large language models, which addresses the technical challenges of overcoming differences in multi-model interfaces, simplifying development processes, and providing standardized, highly scalable management of large models.

[0049] Example 1:

[0050] This invention discloses a dynamic adaptation method for large language models based on Java, which is applied to a management platform for multiple types of large language models. The method includes three steps: multi-model access, pipelined output and error handling, and integration of RAG and vector database.

[0051] Step S100 Multi-model access: Provides model registration and dynamic loading services, protocol conversion and parameter standardization services, and load balancing and failover services. For newly added large models, the adapter is loaded based on the model registration and dynamic loading service. For input user requests, the user requests are converted into the format required by the target large model based on the protocol conversion and parameter standardization service, and the response results are deserialized and encapsulated. Based on the load balancing and failover service, an available service node is selected to send the request.

[0052] In this embodiment, multi-model access includes the following operations:

[0053] (1) Provide model registration and dynamic loading services. Based on the model registration and dynamic loading services, the following operations are supported: using the factory pattern and Spring Bean dynamic registration mechanism, allowing users to declare model parameters in the configuration file of the large language model. The model parameters include model type, API key, and request timeout. When adding a new large language model, add configuration items including model parameters to the configuration file of the large language model. Based on the configuration items, load the adapter class of the large model video. Automatically load the adapter through class path scanning without restarting the service.

[0054] (2) Provide protocol conversion and parameter standardization services. Based on the protocol conversion and parameter standardization services, the following operations are supported: Define a unified request / response DTO for the interface protocols of different large language models. The adapter internally completes the protocol conversion. For user input requests, the request object is converted into the format required by the target large language model, and the response result is deserialized and encapsulated.

[0055] (3) Provide load balancing and failover services. Based on the load balancing and failover services, the following operations are provided: support multi-instance model service deployment, allocate request traffic through weighted round-robin algorithm, and when a service node is unavailable, the adapter automatically switches to the standby node and records error logs.

[0056] Step S200 Streaming Output and Error Handling: Provides streaming protocol services, function call dynamic injection services, and a global error handling mechanism. The streaming protocol service is used for the reactive programming model based on Spring WebFlux, providing two streaming interfaces: SSE and WebSocket. During the streaming response process, if the large model returns a function call request, the external API is triggered based on the function call dynamic injection service, and the result is injected into the subsequent response stream. The global error handling mechanism is used to intercept all large language model call exceptions through AOP, classify and handle various exception scenarios, and return standardized error codes.

[0057] In this embodiment, streaming output and error handling include the following operations:

[0058] (1) Provide streaming protocol support service. The following operations are provided through the streaming protocol support service: Based on the Spring WebFlux reactive programming model, it provides two streaming interfaces, SSE and WebSocket. For non-Spring environments, it implements asynchronous block return through Java native CompletableFuture. When the user initiates a streaming conversation request, the server returns a Token one by one and keeps the connection active.

[0059] (2) Provide a function call dynamic injection service. The function call dynamic injection service provides the following operations: During the streaming response process, if the model returns a function call request, the external API is triggered in real time and the result is injected into the subsequent response stream to realize a coherent process of thinking-execution-feedback.

[0060] (3) Provide a global error handling mechanism. Based on the global error handling mechanism, the following operations are provided: intercept all call exceptions of large language models through AOP, classify and handle network exception scenarios including network timeout, authentication failure, and insufficient quota, and return standardized error codes. At the same time, provide an automatic retry strategy, and retry a maximum of a predetermined number of times for recoverable errors, including temporary network jitter.

[0061] Step S300 integrates RAG and vector database: It provides document parsing and chunking services, vectorized index building services, and RAG enhancement generation services. The document parsing and chunking service parses and chunks the user-input document, and the vectorized index building service generates vectors from the chunked documents and writes them into Pinecone. When the user asks a question, the RAG enhancement generation service retrieves relevant document chunks in Pinecone, injects them as context prompts into the large language model, and returns the answer generated by the large language model.

[0062] In this embodiment, integrating RAG with the vector database includes the following operations:

[0063] (1) Provide document parsing and chunking services. Based on document parsing and chunking, the following operations are provided: integrate ApacheTika to parse various types of documents, chunk the documents according to semantic boundaries and fixed length. If the document is a technical document, the technical document is split into chapters and metadata is generated for each chunk. The metadata includes the source and page number.

[0064] (2) Provide vectorization and index building services. Based on the vectorization and index building services, the following operations are provided: For the segmented documents, the Sentence-BERT model is used to generate document vectors, and the vector data is written in batches through the Pinecone gRPC interface. Hybrid indexing strategies are also supported to improve retrieval speed and accuracy.

[0065] (3) Provide RAG enhancement generation. Based on the RAG enhancement generation service, the following operations are provided: When a user asks a question, retrieve the Top-K related document blocks in Pinecone and inject the retrieved related document blocks as context prompts into the large language model.

[0066] This embodiment provides a unified adaptation for multiple models: a dynamic adapter is designed based on the factory and strategy patterns, abstracting the protocol differences of various LLMs (such as HTTP / WebSocket / GRPC). Developers declare model types and parameters through configuration files (YAML / JSON), and the platform automatically generates corresponding request templates. For example, to address the differences between OpenAI and Zhipu, the adapter converts the unified input into the format required by the target API and dynamically loads third-party SDKs through reflection.

[0067] It provides a multimodal processing engine and vectorization pipeline: a unified interface integrating services such as image recognition and speech-to-text conversion, encapsulating multimodal tasks through a composition pattern. For example, after a user uploads an image, the platform calls the CLIP model to generate vectors and stores them in Pinecone, while simultaneously triggering the RAG process to generate text descriptions. The document vectorization interface supports PDF / Word format parsing, automatically segmenting the document and generating embedded vectors.

[0068] Provide a collaborative mechanism between streaming output and function calls: Design an asynchronous processing pipeline based on Reactive Streams, allowing function call parameters to be dynamically generated in the streaming response. For example, during a dialogue, the model can return an intermediate state of "checking the weather" in real time, while simultaneously triggering an external API to retrieve data and inject it into the subsequent response stream.

[0069] Example 2:

[0070] This invention discloses a Java-based dynamic adaptation system for large language models, applicable to a management platform for multiple types of large language models. The system includes a large model access module, a streaming output and error handling module, and an integrated RAG and vector database module.

[0071] The multi-model access module performs the following functions: providing model registration and dynamic loading services, protocol conversion and parameter standardization services, and load balancing and failover services. For newly added large models, it loads adapters based on the model registration and dynamic loading services. For input user requests, it converts user requests into the format required by the target large model based on the protocol conversion and parameter standardization services, deserializes and encapsulates the response results, and selects available service nodes to send requests based on the load balancing and failover services.

[0072] In this embodiment, the multi-model access module provides the following operations:

[0073] (1) Provide model registration and dynamic loading services. Based on the model registration and dynamic loading services, the following operations are supported: using the factory pattern and Spring Bean dynamic registration mechanism, allowing users to declare model parameters in the configuration file of the large language model. The model parameters include model type, API key, and request timeout. When adding a new large language model, add configuration items including model parameters to the configuration file of the large language model. Based on the configuration items, load the adapter class of the large model video. Automatically load the adapter through class path scanning without restarting the service.

[0074] (2) Provide protocol conversion and parameter standardization services. Based on the protocol conversion and parameter standardization services, the following operations are supported: Define a unified request / response DTO for the interface protocols of different large language models. The adapter internally completes the protocol conversion. For user input requests, the request object is converted into the format required by the target large language model, and the response result is deserialized and encapsulated.

[0075] (3) Provide load balancing and failover services. Based on the load balancing and failover services, the following operations are provided: support multi-instance model service deployment, allocate request traffic through weighted round-robin algorithm, and when a service node is unavailable, the adapter automatically switches to the standby node and records error logs.

[0076] The streaming output and error handling module is used to perform the following: provide streaming protocol services, function call dynamic injection services, and a global error handling mechanism. The streaming protocol service is used for the reactive programming model based on Spring WebFlux, providing two streaming interfaces: SSE and WebSocket. During the streaming response process, if the large model returns a function call request, the external API is triggered based on the function call dynamic injection service and the result is injected into the subsequent response stream. The global error handling mechanism is used to intercept all large language model call exceptions through AOP, classify and handle various exception scenarios, and return standardized error codes.

[0077] In this embodiment, the streaming output and error handling module provides the following operations:

[0078] (1) Provide streaming protocol support service. The following operations are provided through the streaming protocol support service: Based on the Spring WebFlux reactive programming model, it provides two streaming interfaces, SSE and WebSocket. For non-Spring environments, it implements asynchronous block return through Java native CompletableFuture. When the user initiates a streaming conversation request, the server returns a Token one by one and keeps the connection active.

[0079] (2) Provide a function call dynamic injection service. The function call dynamic injection service provides the following operations: During the streaming response process, if the model returns a function call request, the external API is triggered in real time and the result is injected into the subsequent response stream to realize a coherent process of thinking-execution-feedback.

[0080] (3) Provide a global error handling mechanism. Based on the global error handling mechanism, the following operations are provided: intercept all call exceptions of large language models through AOP, classify and handle network exception scenarios including network timeout, authentication failure, and insufficient quota, and return standardized error codes. At the same time, provide an automatic retry strategy, and retry a maximum of a predetermined number of times for recoverable errors, including temporary network jitter.

[0081] The integrated RAG and vector database module performs the following functions: providing document parsing and chunking services, vectorized index building services, and RAG enhancement generation services. The document parsing and chunking service parses and chunks the user-input document, and the vectorized index building service generates vectors from the chunked documents and writes them to Pinecone. When a user asks a question, the RAG enhancement generation service retrieves relevant document chunks from Pinecone, injects them as context prompts into the large language model, and returns the answer generated by the large language model.

[0082] In this embodiment, the integrated RAG and vector database module is used to provide the following operations:

[0083] (1) Provide document parsing and chunking services. Based on document parsing and chunking, the following operations are provided: integrate ApacheTika to parse various types of documents, chunk the documents according to semantic boundaries and fixed length. If the document is a technical document, the technical document is split into chapters and metadata is generated for each chunk. The metadata includes the source and page number.

[0084] (2) Provide vectorization and index building services. Based on the vectorization and index building services, the following operations are provided: For the segmented documents, the Sentence-BERT model is used to generate document vectors, and the vector data is written in batches through the Pinecone gRPC interface. Hybrid indexing strategies are also supported to improve retrieval speed and accuracy.

[0085] (3) Provide RAG enhancement generation. Based on the RAG enhancement generation service, the following operations are provided: When a user asks a question, retrieve the Top-K related document blocks in Pinecone and inject the retrieved related document blocks as context prompts into the large language model.

[0086] The system in this embodiment can execute the method disclosed in Embodiment 1 to achieve dynamic adaptation of large language models.

[0087] The above provides a detailed description of the Java-based large language model dynamic adaptation method and system provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for dynamic adaptation of a large language model based on Java, characterized in that, The management platform applied to large, multi-type language models includes the following steps: Multi-model access: Provides model registration and dynamic loading services, protocol conversion and parameter standardization services, and load balancing and failover services. For newly added large models, the adapter is loaded based on the model registration and dynamic loading service. For input user requests, the user requests are converted into the format required by the target large model based on the protocol conversion and parameter standardization service, and the response results are deserialized and encapsulated. The available service nodes are selected and the request is sent based on the load balancing and failover service. Streaming output and error handling: Provides streaming protocol services, function call dynamic injection services, and a global error handling mechanism. The streaming protocol service is used for the reactive programming model based on Spring WebFlux, providing two streaming interfaces: SSE and WebSocket. During the streaming response process, if the large model returns a function call request, the external API is triggered based on the function call dynamic injection service and the result is injected into the subsequent response stream. The global error handling mechanism is used to intercept all large language model call exceptions through AOP, classify and handle various exception scenarios, and return standardized error codes. Integration with RAG and vector databases: Provides document parsing and chunking services, vectorized index building services, and RAG enhanced generation services. The document parsing and chunking service parses and chunks the user-input document, and the vectorized index building service generates vectors from the chunked documents and writes them to Pinecone. When a user asks a question, the RAG enhanced generation service retrieves relevant document chunks from Pinecone, injects them as context prompts into the large language model, and returns the answer generated by the large language model.

2. The method for dynamic adaptation of a large language model based on Java according to claim 1, characterized in that, Multi-model access includes the following operations: Provides model registration and dynamic loading services. Based on the model registration and dynamic loading services, the following operations are supported: Using the factory pattern and Spring Bean dynamic registration mechanism, users can declare model parameters in the configuration file of the large language model. The model parameters include model type, API key, and request timeout. When adding a new large language model, add configuration items including model parameters to the configuration file of the large language model. Based on the configuration items, load the adapter class of the large model video. The adapter is automatically loaded through classpath scanning without restarting the service. Provides protocol conversion and parameter standardization services. Based on the protocol conversion and parameter standardization services, the following operations are supported: Define a unified request / response DTO for the interface protocols of different large language models. The adapter internally completes the protocol conversion. For user input requests, the request object is converted into the format required by the target large language model, and the response result is deserialized and encapsulated. Provides load balancing and failover services, and based on the load balancing and failover services, it provides the following operations: supports multi-instance model service deployment, distributes request traffic through a weighted round-robin algorithm, and when a service node is unavailable, the adapter automatically switches to the standby node and records error logs.

3. The method for dynamic adaptation of a large language model based on Java according to claim 1, characterized in that, Streaming output and error handling include the following operations: Provide streaming protocol support services, which provide the following operations: Based on the Spring WebFlux reactive programming model, provide two streaming interfaces, SSE and WebSocket. For non-Spring environments, asynchronous chunked return is implemented through Java's native CompletableFuture. When a user initiates a streaming conversation request, the server returns a token sequentially and keeps the connection active. Provides a function call dynamic injection service, which provides the following operations: During the streaming response process, if the model returns a function call request, it will trigger an external API in real time and inject the result into the subsequent response stream, realizing a coherent process of thinking-execution-feedback; A global error handling mechanism is provided, which provides the following operations: intercepting all call exceptions of large language models through AOP, classifying and handling network exception scenarios including network timeout, authentication failure, and insufficient quota, and returning standardized error codes. At the same time, an automatic retry strategy is provided, which allows for a maximum of predetermined number of retries for recoverable errors, including temporary network jitter.

4. The method for dynamic adaptation of a large language model based on Java according to claim 1, characterized in that, Integrating RAG with vector databases includes the following operations: Provides document parsing and chunking services, offering the following operations based on document parsing and chunking: integrates Apache Tika to parse various types of documents, chunks the documents according to semantic boundaries and fixed lengths, where if the document is a technical document, it is split into chapters and metadata is generated for each chunk, including source and page number; It provides vectorization and index building services, and offers the following operations based on these services: For chunked documents, it uses the Sentence-BERT model to generate document vectors, writes the vector data in batches through Pinecone's gRPC interface, and supports hybrid indexing strategies to improve retrieval speed and accuracy. RAG-enhanced generation is provided, and the following operations are offered based on the RAG-enhanced generation service: when a user asks a question, the top-K relevant document blocks in Pinecone are retrieved, and the retrieved relevant document blocks are injected as context prompts into the large language model.

5. A Java-based dynamic language model adaptation system, characterized in that, A management platform for various types of large language models, including a large model access module, a streaming output and error handling module, and an integrated RAG and vector database module; The multi-model access module is used to perform the following: provide model registration and dynamic loading services, protocol conversion and parameter standardization services, and load balancing and failover services. For newly added large models, it loads adapters based on the model registration and dynamic loading services. For input user requests, it converts user requests into the format required by the target large model based on the protocol conversion and parameter standardization services, deserializes and encapsulates the response results, and selects available service nodes to send requests based on the load balancing and failover services. The streaming output and error handling module is used to perform the following: provide streaming protocol services, function call dynamic injection services, and a global error handling mechanism. The streaming protocol service is used for the reactive programming model based on Spring WebFlux, providing two streaming interfaces: SSE and WebSocket. During the streaming response process, if the large model returns a function call request, the external API is triggered based on the function call dynamic injection service and the result is injected into the subsequent response stream. The global error handling mechanism is used to intercept all large language model call exceptions through AOP, classify and handle various exception scenarios, and return standardized error codes. The integrated RAG and vector database module performs the following functions: providing document parsing and chunking services, vectorized index building services, and RAG enhancement generation services. The document parsing and chunking service parses and chunks the user-input document, and the vectorized index building service generates vectors from the chunked documents and writes them to Pinecone. When a user asks a question, the RAG enhancement generation service retrieves relevant document chunks from Pinecone, injects them as context prompts into the large language model, and returns the answer generated by the large language model.

6. The Java-based large language model dynamic adaptation system according to claim 5, characterized in that, The multi-model access module provides the following operations: Provides model registration and dynamic loading services. Based on the model registration and dynamic loading services, the following operations are supported: Using the factory pattern and Spring Bean dynamic registration mechanism, users can declare model parameters in the configuration file of the large language model. The model parameters include model type, API key, and request timeout. When adding a new large language model, add configuration items including model parameters to the configuration file of the large language model. Based on the configuration items, load the adapter class of the large model video. The adapter is automatically loaded through classpath scanning without restarting the service. Provides protocol conversion and parameter standardization services. Based on the protocol conversion and parameter standardization services, the following operations are supported: Define a unified request / response DTO for the interface protocols of different large language models. The adapter internally completes the protocol conversion. For user input requests, the request object is converted into the format required by the target large language model, and the response result is deserialized and encapsulated. Provides load balancing and failover services, and based on the load balancing and failover services, it provides the following operations: supports multi-instance model service deployment, distributes request traffic through a weighted round-robin algorithm, and when a service node is unavailable, the adapter automatically switches to the standby node and records error logs.

7. The Java-based large language model dynamic adaptation system according to claim 5, characterized in that, The streaming output and error handling module provides the following operations: Provide streaming protocol support services, which provide the following operations: Based on the Spring WebFlux reactive programming model, provide two streaming interfaces, SSE and WebSocket. For non-Spring environments, asynchronous chunked return is implemented through Java's native CompletableFuture. When a user initiates a streaming conversation request, the server returns a token sequentially and keeps the connection active. Provides a function call dynamic injection service, which provides the following operations: During the streaming response process, if the model returns a function call request, it will trigger an external API in real time and inject the result into the subsequent response stream, realizing a coherent process of thinking-execution-feedback; A global error handling mechanism is provided, which provides the following operations: intercepting all call exceptions of large language models through AOP, classifying and handling network exception scenarios including network timeout, authentication failure, and insufficient quota, and returning standardized error codes. At the same time, an automatic retry strategy is provided, which allows for a maximum of predetermined number of retries for recoverable errors, including temporary network jitter.

8. The Java-based large language model dynamic adaptation system according to claim 5, characterized in that, The integrated RAG and vector database module provides the following operations: Provides document parsing and chunking services, offering the following operations based on document parsing and chunking: integrates Apache Tika to parse various types of documents, chunks the documents according to semantic boundaries and fixed lengths, where if the document is a technical document, it is split into chapters and metadata is generated for each chunk, including source and page number; Provides vectorization and index building services, and offers the following operations based on these services: For the segmented documents, the Sentence-BERT model is used to generate document vectors, which are then written in batches through Pinecone's gRPC interface. Hybrid indexing strategies are also supported to improve retrieval speed and accuracy. RAG-enhanced generation is provided, and the following operations are offered based on the RAG-enhanced generation service: when a user asks a question, the top-K relevant document blocks in Pinecone are retrieved, and the retrieved relevant document blocks are injected as context prompts into the large language model.

Images