AI model lightweight deployment and preheating system for edge calculation

Abstract

The invention discloses an edge computing-oriented AI model lightweight deployment and preheating system, and relates to the technical field of language models. Comprising an overall framework, a base model and a training route, a data governance and term system, retrieval enhancement generation and context management, term standardization and language output control, compliance and safety governance, inference service and interface, deployment form and offline availability and test and acceptance. According to the method, in engineering implementation, 70B adopts a QLoRA or LoRA parameter efficient method to be combined with gradient check points and tensor parallelization, video memory occupation is remarkably reduced, and throughput is improved; the 7B performs full-amount fine tuning under the condition of resource permission and is used for intention judgment, query rewriting and bridging answer lightweight tasks, and model routing follows the labor division of'heavy model stabilization and light model acceleration ': structured rewriting and short instructions are preferentially routed to the 7B, and open questions and answers and context synthesis routing to the 70B, so that the overall user experience and the resource utilization rate are ensured.

Description

Technical Field

[0001] This invention relates to the field of language model technology, specifically to a lightweight deployment and warm-up system for AI models oriented towards edge computing. Background Technology

[0002] The shortcomings of existing language models are:

[0003] Existing technology CN118210897A discloses an AI intelligent question answering method based on LLM (Large Language Model) natural language processing technology. This technology uses LLM for AI intelligent question answering. Each time a question is answered, the LLM continuously accumulates information from the previously asked question, allowing the AI ​​intelligent question answering system to continuously correct its direction and narrow its scope based on the questions raised. Through the continuous accumulation of questions, the AI ​​intelligent question answering result can continuously approach the accurate result, thereby effectively improving the accuracy of the AI ​​intelligent question answering system and demonstrating good performance. This invention has the advantages of diverse question input methods and high accuracy in AI intelligent question answering.

[0004] The aforementioned technologies process tasks using a uniform scale model, but the resource requirements for processing tasks with different memory locations are not significantly different, resulting in a waste of processing resources. Therefore, a lightweight deployment and warm-up system for AI models oriented towards edge computing that can allocate processing resources according to the task volume and can be started and verified offline is needed to solve this problem. Summary of the Invention

[0005] One objective of this application is to provide a lightweight deployment and warm-up system for AI models for edge computing, which can solve the technical problems raised in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a lightweight deployment and warm-up system for AI models for edge computing, including overall architecture, base model and training path, data governance and terminology system, retrieval enhancement generation and context management, terminology standardization and language output control, compliance and security governance, inference service and interface, deployment form and offline availability, and testing and acceptance.

[0007] Preferably, the overall architecture includes Client requests, input compliance review or sensitive word detection, multi-turn dialogue management or context splicing, a 7B lightweight inference module - intent judgment or query rewriting or bridging answer, hybrid retrieval RAG-Aho dictionary plus FAISS vector, terminology standardization injection, local 70B inference service, output compliance review or rejection template, audit and indicator reporting, and streamable response.

[0008] Preferably, the base model is the Qwen2.5 series, and the training path is carried out in three steps. The first step is continuous pretraining or domain adaptive pretraining, which performs causal language modeling on the above-mentioned Tibetan monolingual and high-quality paired corpora, so that the base can form a robust language prior on Tibetan. The second step is instruction pairing, i.e., SFT, which uses Tibetan-based question-and-answer and task instruction samples for supervised fine-tuning, emphasizing factuality, structure and neutrality. The third step is rejection and compliance bias, i.e., Safety-tuned SFT or ORPO, which teaches the model to reject answers with a uniform template and provide legal alternative suggestions when the policy is triggered on the policy-labeled data, forming a stable and auditable output behavior.

[0009] Preferably, the data governance and terminology system is a terminology database that has undergone segmentation, language detection, profanity removal, deduplication and near-duplication ablation, and then uses a quality scoring model to screen, train and retrieve candidates.

[0010] Preferably, the retrieval enhancement generation and context management, namely the RAG and context management system, adopt a hybrid retrieval of "dictionary matching + vector search" to cover the user's core intent. The 7B lightweight module generates a short "bridging answer" locally. This answer is not directly returned to the user, but only serves as auxiliary text for retrieval and supplementary evidence for dictionary matching. The actual retrieval text consists of "recent multi-turn dialogue summary + bridging answer", taking into account context memory and the current question.

[0011] Preferably, in the terminology standardization and language output control, terminology standardization is performed after retrieval, injecting the highest-scoring term pairs into the context template in a structured manner and taking effect during the generation stage. Two layers of constraints are adopted: first, continuing pre-training and SFT to make the model's language preference naturally tilt towards Tibetan; second, applying language character set masking and post-processing filtering during the decoding stage to suppress the generation of non-Tibetan characters and rewrite and repair occasional cross-language paragraphs to ensure stable output of Tibetan by the system.

[0012] Preferably, the compliance and security governance covers four stages: input, retrieval, generation, and output. In the input stage, the system first completes sensitive word detection, link and document filtering, and identification and desensitization of personal sensitive information. In the retrieval stage, external knowledge access channels are whitelisted and purified. In the generation stage, a decoding period security classifier and rejection template are introduced. In the output stage, secondary interception and audit archiving are performed.

[0013] Preferably, the inference service and the interface are provided as local private services, and the interface style is compatible with the mainstream ecosystem.

[0014] Preferably, the deployment mode and offline availability system provide two modes: containerized deployment and offline delivery. Containerization is based on Docker or Compose and supports running on local servers with multiple GPUs. It combines distributed tensor parallelism and weight quantization to achieve a balance between acceptable latency and throughput. Offline delivery packages images, indexes and data through the offline_bundle and manual_bundle directories, and can complete startup and verification without relying on the public network.

[0015] Preferably, the testing and acceptance process includes corpus sampling, degree evaluation, and security challenge testing before going live. Degree and readability are evaluated using a combination of manual and automatic methods, with random sampling covering multiple topics and formats. Security challenge testing includes sensitive word avoidance, unauthorized requests, and injection bypass scenarios to ensure that the system maintains robust rejection and legitimate alternative suggestions under multiple strategies.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] 1. In engineering implementation, this invention employs the QLoRA or LoRA parameter-efficient method combined with gradient checkpointing and tensor parallelism to significantly reduce memory usage and improve throughput. 7B undergoes full fine-tuning under resource-allowed conditions for lightweight tasks such as intent judgment, query rewriting, and bridging responses. Model routing follows a division of labor of "emphasizing model stability and prioritizing model speed": structured rewriting and short instructions are routed to 7B first, while open question answering and context synthesis are routed to 70B, ensuring overall user experience and resource utilization.

[0018] 2. The parallel and memory optimization of large-scale training in this invention adopts a combination of "data parallelism + tensor parallelism + pipeline parallelism". Tensor parallelism divides the single-layer weights into multiple cards by column or row, pipeline parallelism divides the layer stack into multiple stages, and data parallelism is responsible for cross-node gradient aggregation and synchronization. In actual engineering, it is combined with gradient accumulation and group attention to balance the throughput and stability of the context.

[0019] 3. The system of this invention provides two forms: containerized deployment and offline delivery. Containerization is based on Docker or Compose and supports running on local servers with multiple GPUs. It combines distributed tensor parallelism and weight quantization to achieve a balance between acceptable latency and throughput. Offline delivery packages images, indexes and data through the offline_bundle and manual_bundle directories. It can complete startup and verification without relying on the public network. When starting for the first time, the service will preheat and retrieve resources to avoid blocking the first request. Attached Figure Description

[0020] Figure 1 This is a diagram illustrating the overall architecture of the present invention;

[0021] Figure 2 This is a flowchart of the data governance process of the present invention;

[0022] Figure 3 This is a training cluster diagram of the present invention;

[0023] Figure 4 This is a diagram illustrating the combination of security pair and rejection bias of the present invention;

[0024] Figure 5 and Figure 6 This is a code diagram illustrating the normalization and fusion logic of the hybrid retrieval method of the present invention;

[0025] Figure 7 This is a code diagram illustrating the combined output format of compliant rejection and legitimate alternative suggestions of the present invention;

[0026] Figure 8 and Figure 9 This is a diagram of the overall structure code for the streaming return of this invention;

[0027] Figure 10 This is a code diagram illustrating the key points of the non-streaming data structure of the present invention;

[0028] Figure 11 This is a diagram of the preheating search resource code for the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 The present invention provides an embodiment of a lightweight deployment and warm-up system for AI models for edge computing;

[0031] This includes the overall architecture, base model and training path, data governance and terminology system, retrieval enhancement generation and context management (also known as RAG and context management), terminology standardization and language output control, compliance and security governance, inference services and interfaces, deployment form and offline availability, and testing and acceptance.

[0032] The overall architecture includes client requests, input compliance review or sensitive word detection, multi-turn dialogue management or context concatenation, a 7B lightweight inference module for intent judgment, query rewriting, or bridging responses, hybrid retrieval using the RAG-Aho dictionary with FAISS vectors, terminology standardization injection, local 70B inference service, output compliance review or rejection templates, auditing and metric reporting, and streamable responses. The overall architecture diagram is shown below. Figure 11 As shown, DeepZang adopts a layered system of "local large model inference + retrieval enhancement generation + terminology standardization + compliance and security control". After the request enters, the compliance module completes input review and sensitive word detection, and then enters the retrieval and terminology injection stage. Subsequently, the local 70B inference service completes the generation, and the output is reviewed and audited again. The core process does not initiate external network access. Optional online information retrieval is provided by a controlled intranet service. The results are denoised for security before participating in the answer.

[0033] The first implementation of the overall architecture is as follows: Client request -- input compliance review or sensitive word detection -- multi-turn dialogue management or context splicing -- 7B lightweight inference module -- intent judgment or query rewriting or bridging answer -- hybrid retrieval RAG-Aho dictionary plus FAISS vector -- terminology standardization injection -- local 70B inference service -- output compliance review or rejection template -- audit and indicator reporting -- streamable response.

[0034] The second implementation of the overall architecture is as follows: Client request -- input compliance review or sensitive word detection -- multi-turn dialogue management or context splicing -- hybrid retrieval RAG-Aho dictionary plus FAISS vector -- terminology standardization injection -- local 70B inference service -- output compliance review or rejection template -- audit and indicator reporting -- streamable response.

[0035] Please see Figure 2 , Figure 3 and Figure 4 The present invention provides an embodiment of a lightweight deployment and warm-up system for AI models for edge computing;

[0036] The base model is the Qwen2.5 series, with continuous pre-training and instruction alignment completed locally. The 70B model is responsible for the main answer, while the 7B model handles lightweight tasks such as intent judgment, query rewriting, and fast bridging answers. To generate Tibetan content more stably, we incrementally expanded the word segmenter by introducing Tibetan morphemes and common compound words to reduce the confusion caused by cross-language segmentation. Simultaneously, during the continued pre-training phase, language modeling was primarily performed in Tibetan, allowing the model's language priors to transfer to Tibetan.

[0037] The training data covers 65 million Tibetan-Chinese sentence pairs, a considerable amount of Tibetan monolingual corpus, and more than 120,000 Tibetan-Chinese terminology pairs that have been reviewed by experts. The sentence pairs are used to stabilize the cross-language correspondence and syntactic patterns of question and answer, the monolingual corpus is used to enhance the fluency and stylistic diversity of pure Tibetan, and the terminology data is used to unify the translation of proper nouns and professional expressions.

[0038] Data governance follows a pipeline of "cleaning-deduplication-language recognition-pairing-segmentation-acceptance," with all processes completed within a local isolated environment to ensure that raw data and outputs do not leave the domain. The pairing stage employs a heuristic + pairing-machine combination strategy, prioritizing the retention of high-confidence pairs. The segmentation stage performs sentence-based and paragraph-based processing of the text to facilitate subsequent batch processing and training scheduling. The process is as follows: Figure 2 As shown.

[0039] The training process consists of three steps. The first step is continuous pretraining, or Domain Adaptive Pretraining, which performs causal language modeling on the aforementioned Tibetan monolingual and high-quality paired corpora to establish a robust language prior on Tibetan. The second step is instruction pairs, or SFT, which uses Tibetan-based question-and-answer and task instruction samples for supervised fine-tuning, emphasizing factuality, structure, and neutrality. The third step is rejection and compliance bias, or Safety-tuned SFT and ORPO, which uses policy-labeled data to teach the model to reject responses with a uniform template and provide legitimate alternative suggestions when the policy is triggered, thus forming stable and auditable output behavior.

[0040] In terms of engineering implementation, 70B employs efficient methods such as QLoRA or LoRA parameters combined with gradient checkpointing and tensor parallelism, significantly reducing memory usage and improving throughput. 7B undergoes full fine-tuning within resource constraints, serving lightweight tasks such as intent judgment, query rewriting, and bridging responses. Model routing follows a division of labor: prioritizing model stability and speeding up the model: structured rewriting and short instructions are routed to 7B first, while open question answering and context synthesis are routed to 70B, ensuring overall user experience and resource utilization.

[0041] To ensure the stability of Tibetan text and the consistency of terminology during the generation stage, we added Tibetan morphemes and high-frequency compound words to the original vocabulary and further adapted the expanded vocabulary through pre-training. During the reasoning stage, we linked with the terminology database and injected standardized translations into the context to further constrain the output.

[0042] Typical hyperparameter settings follow the principles of "contextuality, stable convergence, and reproducibility." For 70BQLoRA, the settings include bf16 computation, 8-bit weight loading, LoRArank 64, learning rate 1e-4, context degree 8192, and effective batch size 256. For 7B full-scale fine-tuning, the settings include a learning rate 2e-5, context degree 8192, and effective batch size 512. Below is a simplified training configuration snippet demonstrating key engineering constraints and auditable parameters:

[0043] Parallelism and memory optimization for large-scale training employ a combination of "data parallelism + tensor parallelism + pipelining parallelism", such as... Figure 3As shown, tensor parallelism splits the weights of a single layer into multiple GPUs by column or row, pipeline parallelism divides the layer stack into multiple stages, and data parallelism is responsible for gradient aggregation and synchronization across nodes. In actual engineering, it works in conjunction with gradient accumulation and group attention to balance the throughput and stability of the context.

[0044] Safety pairs and rejection bias are achieved through a combination of "rule labeling + classifier distillation + templated pairs", such as... Figure 4 As shown, at the annotation level, we perform rule-based annotation on sensitive topics and inappropriate requests; at the model level, we distill the rule signals into a learnable safe classifier, which, together with the rejection template, participates in SFT, enabling the model to stably output a unified expression and legitimate alternative suggestions when the policy is triggered.

[0045] One embodiment of the present invention: a lightweight deployment and warm-up system for AI models for edge computing;

[0046] The data governance and terminology system involves segmenting the terminology database, performing language detection, removing profanity, deduplication, and near-duplication ablation, and then using a quality scoring model to screen, train, and retrieve candidates. The data comes from the self-developed Jue Luo Digital "Tibetan-Chinese Parallel Corpus Dataset," legally authorized public Tibetan corpora, and professional terminology databases. After segmentation, language detection, profanity removal, deduplication, and near-duplication ablation, a quality scoring model is used to screen, train, and retrieve candidates. The terminology database is extracted based on academic standards and publications and reviewed by experts. To ensure the consistent use of standardized terminology in responses, we inject the retrieved terminology pairs into the model's context template during the inference stage, making the generated translations tend towards standardized translations. This terminology injection and suggestion management is implemented locally and does not rely on external services.

[0047] Please see Figure 5 and Figure 6 The present invention provides an embodiment of a lightweight deployment and warm-up system for AI models for edge computing;

[0048] For enhanced retrieval generation and context management, the system employs a hybrid retrieval approach combining dictionary matching and vector search. On the dictionary side, two Aho-Corasick automata sets, one for Chinese and one for Tibetan, are used for high-speed matching, corresponding to the data or aho_zh.dat and data or aho_bo.dat files in the repository. When the automata is unavailable or misses a match, the system reverts to matching substrings acceptable for the shorter text. On the vector side, a local FAISS index is used for similarity retrieval, employing L2 metric by default and obtaining a [0,1] dimension through min-max normalization. The scores from the two channels are weighted and fused. The current online configuration uses α=0.7 to favor dictionary hits and β=0.3 to supplement semantic relevance, ensuring the retrieval covers the user's core intent. The 7B lightweight module generates a short "bridging answer" locally. This answer is not directly returned to the user but serves only as supplementary evidence for the retrieval and dictionary matching. The actual retrieval text consists of "recent multi-turn dialogue summaries + bridging answers," taking into account both contextual memory and the current question. The following... Figure 5 and Figure 6 The fragment demonstrates the normalization and fusion logic of hybrid retrieval, derived from the implementation in the repository, and reflects two normalization approaches, "max-hit" and "min-max", as well as α or β weighted fusion.

[0049] One embodiment of the present invention: a lightweight deployment and warm-up system for AI models for edge computing;

[0050] In the terminology standardization and language output control, terminology standardization is performed after retrieval. The top-scoring term pairs are injected into the context template in a structured manner and take effect during the generation stage. Two constraints are adopted: first, the model's language preference is naturally tilted towards Tibetan by continuing pre-training and SFT; second, language character set masking and post-processing filtering are applied during the decoding stage to suppress the generation of non-Tibetan characters and rewrite and repair occasional cross-language paragraphs to ensure that the system outputs Tibetan stably. This combined scheme does not require external dependencies and has the advantages of being controllable, interpretable and stable.

[0051] Please see Figure 7 The present invention provides an embodiment of a lightweight deployment and warm-up system for AI models for edge computing;

[0052] Compliance and security governance covers four stages: input, retrieval, generation, and output. In the input stage, the system first completes sensitive word detection, link and document filtering, and identification and de-identification of personal sensitive information. In the retrieval stage, external knowledge access channels are whitelisted and cleaned up. In the generation stage, a decoding-period security classifier and rejection template are introduced. In the output stage, secondary interception and audit archiving are performed.

[0053] Sensitive words and compliance strategies. Implemented using a "rule base + automaton + strategy engine" approach. The rule base includes both Chinese and Tibetan word lists, and categorizes synonyms, variants, and common misspellings. The automaton uses Aho-Corasick to generate multi-modal matching structures, ensuring extremely low latency. The strategy engine provides a hot-updateable DSL, supporting the rapid enabling / disabling of policies and thresholds in different business scenarios. Example rules: Figure 7 As shown, this illustrates the combined output format of compliant refusal to respond and legitimate alternative suggestions.

[0054] The auditing system structures and stores inputs, decisions, candidate retrieval, generated summaries, and output results on disk, including timestamps, hash chains, policy hit information, and handling information. High-risk requests are automatically tagged and support manual review for a closed-loop process.

[0055] Please see Figure 8 , Figure 9 and Figure 10 The present invention provides an embodiment of a lightweight deployment and warm-up system for AI models for edge computing;

[0056] The inference service is provided as a local private service with an interface style compatible with the mainstream ecosystem to facilitate integration with existing SDKs and toolchains. The service supports both non-streaming and streaming return methods and uses server-side event SSE for incremental push, which significantly reduces the latency of the first packet. The service returns debugging and auditing metadata for each request, including search candidates, delays, search or processing time, query and bridging text, which facilitates the location of quality issues and iteration parameters.

[0057] Figure 8 and Figure 9 The code shown illustrates the overall structure of streaming return. The core is to split the inference stream into several data blocks and push them to the client in an event-stream manner. It employs exception fallback to ensure that the failure of a single block serialization will not interrupt the entire stream, and marks completion with [DONE].

[0058] In non-streaming mode, the service will include a structured metadata segment containing search candidates, latency statistics, search patterns, and key parameters. Figure 10 The code shown illustrates the key data structure points of this metadata, facilitating observation and evaluation by the business side.

[0059] Please see Figure 11 The present invention provides an embodiment of a lightweight deployment and warm-up system for AI models for edge computing;

[0060] The system offers two deployment models: containerized deployment and offline delivery. Containerization, based on Docker or Compose, supports running on local servers with multiple GPUs. It achieves a balance between acceptable latency and throughput by combining distributed tensor parallelism and weight quantization. Offline delivery packages images, indexes, and data through the `offline_bundle` and `manual_bundle` directories, enabling startup and verification without relying on the public internet. Upon initial startup, such as... Figure 11 As shown, the service preheats the retrieved resources, including the glossary, FAISS index, and automata, to avoid blocking the first request. The corresponding implementation is located in the service startup hook.

[0061] One embodiment of the present invention: a lightweight deployment and warm-up system for AI models for edge computing;

[0062] Before going live, the system underwent corpus sampling, alignment evaluation, and security challenge testing during testing and acceptance. Alignment and readability were evaluated using a combination of manual and automated methods, with random sampling covering multiple topics and formats. Security challenge testing included scenarios such as sensitive word avoidance, unauthorized requests, and injection bypass, ensuring the system remained robust in rejecting responses and providing legitimate alternative suggestions under multiple strategies. A gray-scale optimization mechanism was provided for key parameters α or β, retrieval thresholds, and compliance thresholds to ensure continuous convergence under real traffic.

[0063] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the same elements of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the rights involved.

Claims

1. A lightweight deployment and warm-up system for AI models for edge computing, characterized by: This includes the overall architecture, base model and training path, data governance and terminology system, retrieval enhancement generation and context management, terminology standardization and language output control, compliance and security governance, inference service and interface, deployment form and offline availability, and testing and acceptance.

2. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The overall architecture includes Client requests, input compliance review or sensitive word detection, multi-turn dialogue management or context splicing, a 7B lightweight inference module for intent judgment or query rewriting or bridging answers, hybrid retrieval RAG-Aho dictionary plus FAISS vector, terminology standardization injection, local 70B inference service, output compliance review or rejection templates, auditing and indicator reporting, and streamable responses.

3. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The base model is the Qwen2.5 series, and the training path is divided into three steps. The first step is continuous pre-training, which performs causal language modeling on the aforementioned Tibetan monolingual and high-quality paired corpora to enable the base to form a robust language prior in Tibetan. The second step is instruction pairs, which uses question-and-answer and task instruction samples, mainly in Tibetan, for supervised fine-tuning, emphasizing factuality, structure, and neutrality. The third step is rejection and compliance bias, which teaches the model to reject answers with a uniform template and provide legitimate alternative suggestions when the policy is triggered, forming a stable and auditable output behavior.

4. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The data governance and terminology system involves segmenting the terminology database, performing language detection, removing obscene words, deduplication, and near-duplication ablation, and then using a quality scoring model to screen, train, and retrieve candidates.

5. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The system employs a hybrid retrieval approach of "dictionary matching + vector search" to enhance retrieval and manage context. This approach covers the user's core intent. The 7B lightweight module generates a short "bridging answer" locally. This answer is not directly returned to the user but serves only as supplementary text for retrieval and dictionary matching. The actual retrieval text consists of "summary of recent multi-turn conversations + bridging answer," taking into account both contextual memory and the current question.

6. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: In the terminology standardization and language output control, terminology standardization is performed after retrieval, injecting the highest-scoring term pairs into the context template in a structured manner and taking effect during the generation stage. Two constraints are adopted: first, pre-training and SFT are continued to make the model's language preference naturally tilt towards Tibetan; second, language character set masking and post-processing filtering are applied during the decoding stage to suppress the generation of non-Tibetan characters and rewrite and repair occasional cross-language paragraphs to ensure that the system outputs Tibetan stably.

7. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The compliance and security governance covers four stages: input, retrieval, generation, and output. In the input stage, the system first completes sensitive word detection, link and document filtering, and identification and desensitization of personal sensitive information. In the retrieval stage, external knowledge access channels are whitelisted and cleaned up; in the generation stage, a decoding-period security classifier and a rejection template are introduced. In the output stage, secondary interception and audit archiving are performed.

8. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The inference service and interface are provided as a local private service with an interface style compatible with the mainstream ecosystem.

9. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: The deployment model and offline availability system provide two forms: containerized deployment and offline delivery. Containerization is based on Docker or Compose and supports running on local servers with multiple GPUs. It combines distributed tensor parallelism and weight quantization to achieve a balance between acceptable latency and throughput. Offline delivery packages images, indexes and data through the offline_bundle and manual_bundle directories, and can complete startup and verification without relying on the public network.

10. The lightweight deployment and warm-up system for AI models oriented towards edge computing according to claim 1, characterized in that: Before going live, the testing and acceptance process included corpus sampling, correlation evaluation, and security challenge testing. Correlation and readability were evaluated using a mix of manual and automated methods, with random sampling covering multiple topics and formats. Security challenge testing included sensitive word avoidance, unauthorized requests, and injection bypass scenarios to ensure the system maintained robust rejection and legitimate alternative suggestions under multiple strategies.

Images