A Method for Deploying a Private Large Model and Building a RAG Knowledge Base

Abstract

This invention discloses a method for deploying a private large-scale model and building a RAG knowledge base, relating to the field of database technology. It includes a layered deployment of the private large-scale model: based on the target terminal hardware resource threshold, a three-level deployment architecture of "core model layer - lightweight adaptation layer - interface adaptation layer" is constructed; the pre-trained large-scale model is structurally pruned to retain the core neuron cluster for model inference; an incremental RAG knowledge base is built; and collaborative inference between the large-scale model and the RAG knowledge base completes the construction. This invention, through innovation across the entire process of deployment architecture, knowledge base construction, collaborative inference, and security protection, effectively solves the pain points of traditional private large-scale model deployment, such as high resource consumption, slow RAG knowledge base retrieval, unreliable inference results, and insufficient system security, providing enterprises with an efficient, accurate, secure, and scalable solution.

Description

Technical Field

[0001] This invention relates to the field of database construction technology, specifically a method for deploying a private large model and building a RAG knowledge base. Background Technology

[0002] Existing technology (invention patent publication number CN120455066A) discloses a method and application for constructing a large-scale security model for industrial internet security protection. This method includes the following steps: loading the base large-scale model GLM4 and constructing a knowledge retrieval module, accessing the MITRE ATT&CK knowledge base; embedding and vectorizing the security knowledge within the MITRE ATT&CK knowledge base using knowledge features; building a vector database storage and retrieval module; activating the retrieval enhancement generation mechanism and loading the test dataset mitrettpmapping; constructing a local model and a comparative model respectively, and conducting model comparison experiments; evaluating the performance of the constructed large-scale security model in network security situation analysis tasks, evaluating the improvement effect of the model after applying the retrieval enhancement generation (RAG) mechanism, and outputting a RAG-based large-scale security model solution. This application is applicable to security situation assessment in multiple scenarios such as enterprises, industry, and mobile, and can improve the accuracy and interpretability of security content generation. Existing technology, however, cannot simultaneously improve the adaptability and resource utilization of private deployments, cannot guarantee the dynamism and data quality of the RAG knowledge base, and cannot enhance system security and operational stability. Summary of the Invention

[0003] This invention aims to solve at least one of the technical problems existing in the prior art. To this end, this invention proposes a method for deploying a private large model and building a RAG knowledge base. Through innovation in the entire process of deployment architecture, knowledge base construction, collaborative reasoning and security protection, this invention effectively solves the pain points of traditional private large model deployment, such as high resource consumption, slow RAG knowledge base retrieval, unreliable reasoning results and insufficient system security, providing enterprises with an efficient, accurate, secure and scalable solution.

[0004] To achieve the above objectives, a first aspect of the present invention provides a method for deploying a private large model and building a RAG knowledge base, comprising: S1. Layered Deployment of the Private Large Model: Based on the target terminal hardware resource threshold, a three-tier deployment architecture is constructed: "Core Model Layer - Lightweight Adaptation Layer - Interface Adaptation Layer," where: S11. Deploy the core model layer: Perform structured pruning on the pre-trained large model, retain the core neuron cluster of model inference, convert the parameter precision from FP16 to INT8 / INT4 through model weight quantization, and at the same time use sparse matrix storage optimization to reduce the storage occupation of the core model. S12. Construct the lightweight adaptation layer: Based on the number of GPU / CPU cores, memory capacity, and video memory bandwidth, dynamically generate model inference acceleration plugins. The plugins include an instruction set optimization module, a memory scheduling module, and a parallel computing module. The instruction set optimization module adaptively selects the x86 / ARM instruction set according to the hardware architecture. The memory scheduling module adopts a "hot data priority caching + cold data page storage" strategy. The parallel computing module splits the inference task into a sub-task queue that matches the number of hardware cores. S13. Deploy the interface adaptation layer: Provide RESTfulAPI and gRPC dual interface modes, support both synchronous and asynchronous inference calling methods, and have a built-in request rate limiting module and permission verification module. The permission verification module adopts a three-level authentication mechanism of "device fingerprint + API key + role permission". S2. Incremental construction of the RAG knowledge base, specifically including: S21. Multi-source data acquisition and standardization: Heterogeneous data is acquired through three methods: web crawler interface, local file parsing, and database synchronization. The heterogeneous data includes text files, tabular data, image OCR recognition results, and audio transcribed text. The acquired data is standardized and uniformly converted into UTF-8 encoded structured documents. Tabular data is converted into key-value pair arrays, and image OCR text is divided into paragraphs by region. S22. Knowledge Unit Decomposition and Semantic Encoding: The standardized document is divided into independent knowledge units by adopting the strategy of "sentence-level decomposition + semantic association clustering". Each knowledge unit contains text content, source identifier and timestamp. The knowledge unit is semantically encoded by the pre-trained BERT model to generate a 768-dimensional vector embedding. At the same time, the keyword weight of each knowledge unit is calculated. S23. Dynamic Knowledge Base Construction: A main knowledge base is built based on a vector database, and a keyword index library and a source index library are established at the same time. The main knowledge base stores the vector embedding and association information of knowledge units. The keyword index library establishes the mapping relationship between keywords and knowledge units. The source index library stores knowledge units according to data source. S24. Incremental Update and Optimization of Knowledge Base: Set up a scheduled update task and a triggered update interface. The scheduled update task collects new data at a preset period and repeats steps S21-S23. The triggered update interface supports manual data upload and real-time updates. During the update process, a "similarity deduplication algorithm" is used to calculate the semantic similarity between new knowledge units and existing knowledge units. When the similarity is higher than a preset threshold, duplicate storage is rejected. At the same time, the knowledge base is periodically defragmented, invalid knowledge units are deleted, and the vector index is optimized. S3. Collaborative reasoning between the large model and the RAG knowledge base, completed and including: S31. User query processing: After receiving a user query request, the interface adaptation layer performs permission verification and request parsing to extract query keywords and semantic intent; S32. Knowledge Base Retrieval: The parsed query request is converted into a vector embedding, and multi-dimensional retrieval is performed based on keywords, including vector similarity retrieval of the main knowledge base, exact matching retrieval of the keyword index, and range retrieval of the source index. The candidate knowledge set is generated by combining the three types of retrieval results. S33. Reasoning Result Generation: Input the candidate knowledge set and the user query request into the private big model. The big model combines its own parameter knowledge and the candidate knowledge set to perform reasoning, generate the response content, and mark the source of the knowledge unit referenced in the response. S34. Inference Effect Feedback and Optimization: Collect user satisfaction ratings for the response content. When the rating is unsatisfactory, record the query request, response content, and user feedback. Adjust the inference parameters of the large model and the retrieval weights of the knowledge base through reinforcement learning algorithms. At the same time, supplement the knowledge base with effective feedback data as incremental data to complete the construction.

[0005] Preferably, the structured pruning adopts a combination of "intra-layer neuron pruning + inter-layer connection pruning". Intra-layer neuron pruning retains neurons in each layer whose activation value is greater than a preset threshold, and inter-layer connection pruning deletes connection weights whose absolute weight value is less than the pruning threshold. The pruning threshold is dynamically adjusted according to the hardware memory capacity. The parallel computing module adopts a pipeline mechanism of "task splitting-result merging". The inference task is split into preprocessing subtasks, attention calculation subtasks and output decoding subtasks according to the token generation order. Each subtask is assigned to an independent hardware core for parallel computing, and data interaction between subtasks is realized through shared memory.

[0006] Preferably, the keyword weight calculation adopts a combination of TF-IDF algorithm and TextRank algorithm. The TF-IDF algorithm calculates the word frequency weight of the keyword in the current knowledge unit, and the TextRank algorithm calculates the importance weight of the keyword in the global document. The two are weighted and summed to obtain the final keyword weight. The vector database uses Milvus or FAISS and supports IVF_FLAT or HNSW indexing algorithms. The vector index of the main knowledge base is maintained in a way that combines daily incremental construction with weekly full optimization to improve retrieval efficiency. The similarity deduplication algorithm adopts a fusion algorithm of cosine similarity and Jaccard similarity to calculate the semantic cosine similarity and text intersection similarity between the new knowledge unit and the existing knowledge unit. When the weighted average of the two is higher than 85%, it is determined to be a duplicate knowledge unit.

[0007] Preferably, the specific method for integrating the three types of search results is as follows: sort the vector similarity search results by similarity score and select the top 20 results; sort the keyword exact match search results by the number of matched keywords and select the top 15 results; sort the source index range search results by timestamp and select the top 10 results; after removing duplicates from the three types of results, calculate the comprehensive score using the weighted formula "vector similarity score × 0.6 + keyword matching weight × 0.3 + time freshness × 0.1", and select the top 15 results with the comprehensive score to form a candidate knowledge set; The reinforcement learning algorithm uses the Proximal Policy Optimization (PPO) algorithm, which uses user satisfaction ratings as reward signals to adjust the allocation coefficients of the attention weight parameters and knowledge base retrieval weights of the large model, and updates the parameters once every 100 valid feedback data points.

[0008] Preferably, the security protection for the layered deployment of the private large model specifically includes: S41. Data transmission encryption: The SSL / TLS protocol is used to encrypt user query requests, inference results and data transmission process, and the key is automatically rotated periodically; S42. Storage Encryption: The core model parameters and knowledge base data are stored using the AES-256 encryption algorithm, and the encryption key is stored using a hardware security module (HSM). S43. Access Control: Set up multi-level access permissions based on the RBAC (Role-Based Access Control) model, so that users with different roles can only access knowledge base data and model functions within their corresponding permission scope; S44. Operation Audit: Record all users' access logs, query logs, and knowledge base operation logs. Log information includes the operator, operation time, operation content, and device information. Log data is retained for at least 90 days.

[0009] Preferably, step S1 further includes deployment resource monitoring and dynamic adjustment, specifically including: S14. Resource Monitoring: Real-time monitoring of hardware CPU usage, memory usage, video memory usage, and network bandwidth; setting resource usage thresholds. S15. Dynamic Adjustment: When CPU utilization or memory usage exceeds the threshold for 5 consecutive minutes, the lightweight adaptation layer automatically starts the load balancing mechanism to allocate inference tasks to idle hardware resources; when GPU memory usage exceeds the threshold, the core model layer automatically enables model sharding inference to split model parameters into multiple GPU memories for parallel computation; when resource usage is below the threshold of 30%, some redundant parallel computing modules are shut down to reduce resource consumption.

[0010] Compared with the prior art, the beneficial effects of the present invention are: 1. Improved Adaptability and Resource Utilization in Private Deployment. This invention innovatively designs a three-tier deployment architecture: "Core Model Layer - Lightweight Adaptation Layer - Interface Adaptation Layer." Through structured pruning, weight quantization, and sparse matrix storage optimization, it significantly reduces the storage footprint and inference overhead of large models, solving the dependence of traditional private deployments on high-end hardware. This allows the model to adapt to x86 / ARM multi-architecture hardware and terminal devices with different configurations. The dynamic acceleration plugin in the lightweight adaptation layer can adjust the instruction set, memory scheduling, and parallel computing strategies in real time based on the number of hardware cores and memory capacity, achieving precise matching between inference tasks and hardware resources. It can still guarantee inference efficiency on low-configuration hardware, and improve task throughput through a parallel pipeline mechanism on high-configuration hardware, improving resource utilization by more than 30% compared to traditional fixed deployment methods. The dual-interface mode and three-level authentication mechanism of the interface adaptation layer not only support multi-scenario calling needs but also build a solid first line of defense for access security, balancing practicality and security.

[0011] 2. RAG Knowledge Base: Dual Guarantee of Dynamics and Data Quality. This invention adopts an incremental construction approach, achieving full compatibility with heterogeneous data such as text, tables, image OCR, and audio transcription through multi-source data collection and standardized processing, solving the pain point of traditional knowledge bases relying on a single data source. The "sentence-level segmentation + semantic association clustering" strategy, combined with keyword weight calculation integrating TF-IDF and TextRank, ensures the rationality of knowledge unit segmentation and the accuracy of semantic encoding, laying the foundation for efficient retrieval. The collaborative architecture of the main knowledge base and dual index databases, coupled with an index maintenance mechanism of "daily incremental construction + weekly full optimization," reduces retrieval response time to milliseconds. Similarity deduplication algorithms and fragmentation defragmentation mechanisms effectively avoid data redundancy, delete invalid knowledge units, ensure the purity of the knowledge base and the accuracy of retrieval, and reduce duplicate data storage rate to below 5%. An incremental mechanism combining scheduled and triggered updates enables real-time dynamic iteration of the knowledge base, ensuring knowledge timeliness and meeting the needs of enterprises for rapid updates of business data.

[0012] 3. Breakthroughs in both accuracy and self-optimization capabilities of collaborative reasoning. This invention utilizes multi-dimensional collaborative reasoning between a large model and the RAG knowledge base. Through a fusion strategy of vector similarity retrieval, precise keyword matching, and source range retrieval, it constructs a comprehensive candidate knowledge set, avoiding the limitations of single retrieval methods. A weighted comprehensive scoring mechanism further filters highly relevant knowledge, providing precise support for large model reasoning and improving the knowledge matching accuracy of the response content by over 40%. Simultaneously, the response content is annotated with its knowledge source, enhancing the traceability of results, solving the "illusion" problem of large models, and increasing user trust. The feedback optimization mechanism based on the PPO algorithm transforms user satisfaction into adjustment signals for model reasoning parameters and retrieval weights, achieving a closed-loop iteration of "reasoning-feedback-optimization." With increased usage time, the response accuracy continuously improves; effective feedback data supplements the knowledge base, further strengthening knowledge reserves and forming a virtuous cycle.

[0013] 4. Enhanced System Security and Operational Stability. The proprietary deployment of this invention provides end-to-end security protection through SSL / TLS transmission encryption, AES-256 storage encryption, and HSM key management, constructing a dual security barrier for data transmission and storage, meeting Level 3 or higher data security requirements. RBAC multi-level access control and 90-day operation audit logs enable precise control and full traceability of access behavior, preventing data leakage and unauthorized operations. A resource monitoring and dynamic adjustment mechanism is deployed to monitor the status of resources such as CPU, memory, and video memory in real time. Through load balancing, model sharding, and other dynamic adjustment strategies, system lag or crashes caused by resource overload are avoided. Simultaneously, redundant modules are shut down to reduce consumption during resource redundancy, improving system stability to over 99.9%, balancing high performance and low power consumption.

[0014] In summary, this solution effectively addresses the pain points of traditional private large-scale model deployment, such as high resource consumption, slow RAG knowledge base retrieval, unreliable inference results, and insufficient system security, through innovation across the entire process of deployment architecture, knowledge base construction, collaborative reasoning, and security protection. It provides enterprises with an efficient, accurate, secure, and scalable solution, suitable for scenarios with high requirements for data privacy and business adaptability, such as finance, healthcare, and industrial manufacturing, and has significant technical value and commercial application prospects. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 The first aspect of this application provides a method for deploying a private large model and building a RAG knowledge base, including: S1. Layered Deployment of the Private Large Model: Based on the target terminal hardware resource threshold, a three-tier deployment architecture is constructed: "Core Model Layer - Lightweight Adaptation Layer - Interface Adaptation Layer," where: S11. Deploy the core model layer: Perform structured pruning on the pre-trained large model, retain the core neuron cluster of model inference, convert the parameter precision from FP16 to INT8 / INT4 through model weight quantization, and at the same time use sparse matrix storage optimization to reduce the storage occupation of the core model. Furthermore, the structured pruning adopts a combination of "intra-layer neuron pruning + inter-layer connection pruning". Intra-layer neuron pruning retains neurons in each layer whose activation value is greater than a preset threshold, and inter-layer connection pruning deletes connection weights whose absolute weight value is less than the pruning threshold. The pruning threshold is dynamically adjusted according to the hardware memory capacity. S12. Construct the lightweight adaptation layer: Based on the number of GPU / CPU cores, memory capacity, and video memory bandwidth, dynamically generate model inference acceleration plugins. The plugins include an instruction set optimization module, a memory scheduling module, and a parallel computing module. The instruction set optimization module adaptively selects the x86 / ARM instruction set according to the hardware architecture. The memory scheduling module adopts a "hot data priority caching + cold data page storage" strategy. The parallel computing module splits the inference task into a sub-task queue that matches the number of hardware cores. Furthermore, the parallel computing module adopts a pipeline mechanism of "task splitting-result merging", which splits the inference task into preprocessing subtasks, attention calculation subtasks and output decoding subtasks according to the token generation order. Each subtask is assigned to an independent hardware core for parallel computing, and data interaction between subtasks is realized through shared memory.

[0019] S13. Deploy the interface adaptation layer: Provide RESTfulAPI and gRPC dual interface modes, support both synchronous and asynchronous inference calling methods, and have a built-in request rate limiting module and permission verification module. The permission verification module adopts a three-level authentication mechanism of "device fingerprint + API key + role permission". Specifically, step S1 also includes deployment resource monitoring and dynamic adjustment, specifically including: S14. Resource Monitoring: Real-time monitoring of hardware CPU usage, memory usage, video memory usage, and network bandwidth; setting resource usage thresholds. S15. Dynamic Adjustment: When CPU utilization or memory usage exceeds the threshold for 5 consecutive minutes, the lightweight adaptation layer automatically initiates a load balancing mechanism to allocate inference tasks to idle hardware resources; when GPU memory usage exceeds the threshold, the core model layer automatically enables model sharding inference, splitting model parameters across multiple GPU memory locations for parallel computation; when resource usage falls below 30% of the threshold, some redundant parallel computing modules are shut down to reduce resource consumption. S2. Incremental construction of the RAG knowledge base, specifically including: S21. Multi-source data acquisition and standardization: Heterogeneous data is acquired through three methods: web crawler interface, local file parsing, and database synchronization. The heterogeneous data includes text files, tabular data, image OCR recognition results, and audio transcribed text. The acquired data is standardized and uniformly converted into UTF-8 encoded structured documents. Tabular data is converted into key-value pair arrays, and image OCR text is divided into paragraphs by region. S22. Knowledge Unit Decomposition and Semantic Encoding: The standardized document is divided into independent knowledge units by adopting the strategy of "sentence-level decomposition + semantic association clustering". Each knowledge unit contains text content, source identifier and timestamp. The knowledge unit is semantically encoded by the pre-trained BERT model to generate a 768-dimensional vector embedding. At the same time, the keyword weight of each knowledge unit is calculated. Furthermore, the keyword weight calculation adopts a combination of the TF-IDF algorithm and the TextRank algorithm. The TF-IDF algorithm calculates the word frequency weight of the keyword in the current knowledge unit, and the TextRank algorithm calculates the importance weight of the keyword in the global document. The two are weighted and summed to obtain the final keyword weight. S23. Dynamic Knowledge Base Construction: A main knowledge base is built based on a vector database, and a keyword index library and a source index library are established at the same time. The main knowledge base stores the vector embedding and association information of knowledge units. The keyword index library establishes the mapping relationship between keywords and knowledge units. The source index library stores knowledge units according to data source. Furthermore, the vector database uses Milvus or FAISS and supports IVF_FLAT or HNSW indexing algorithms. The vector index of the main knowledge base is maintained in a "daily incremental construction + weekly full optimization" manner to improve retrieval efficiency. S24. Incremental Update and Optimization of Knowledge Base: Set up a scheduled update task and a triggered update interface. The scheduled update task collects new data at a preset period and repeats steps S21-S23. The triggered update interface supports manual data upload and real-time updates. During the update process, a "similarity deduplication algorithm" is used to calculate the semantic similarity between new knowledge units and existing knowledge units. When the similarity is higher than a preset threshold, duplicate storage is rejected. At the same time, the knowledge base is periodically defragmented, invalid knowledge units are deleted, and the vector index is optimized. Furthermore, the similarity deduplication algorithm employs a fusion algorithm combining cosine similarity and Jaccard similarity to calculate the semantic cosine similarity and text intersection similarity between the newly added knowledge unit and existing knowledge units. When the weighted average of the two is higher than 85%, it is determined to be a duplicate knowledge unit. S3. Collaborative reasoning between the large model and the RAG knowledge base, completed and including: S31. User query processing: After receiving a user query request, the interface adaptation layer performs permission verification and request parsing to extract query keywords and semantic intent; S32. Knowledge Base Retrieval: The parsed query request is converted into a vector embedding, and multi-dimensional retrieval is performed based on keywords, including vector similarity retrieval of the main knowledge base, exact matching retrieval of the keyword index, and range retrieval of the source index. The candidate knowledge set is generated by combining the three types of retrieval results. Furthermore, the specific method for integrating the three types of search results is as follows: the vector similarity search results are sorted by similarity score, and the top 20 results are selected; the keyword exact match search results are sorted by the number of matched keywords, and the top 15 results are selected; the source index range search results are sorted by timestamp, and the top 10 results are selected; after removing duplicates from the three types of results, a comprehensive score is calculated using the weighted formula "vector similarity score × 0.6 + keyword matching weight × 0.3 + time freshness × 0.1", and the top 15 results with the comprehensive score are selected to form a candidate knowledge set; S33. Reasoning Result Generation: Input the candidate knowledge set and the user query request into the private big model. The big model combines its own parameter knowledge and the candidate knowledge set to perform reasoning, generate the response content, and mark the source of the knowledge unit referenced in the response. S34. Inference Effect Feedback and Optimization: Collect user satisfaction ratings for the response content. When the rating is unsatisfactory, record the query request, response content, and user feedback. Adjust the inference parameters of the large model and the retrieval weights of the knowledge base through reinforcement learning algorithms. At the same time, supplement the knowledge base with effective feedback data as incremental data to complete the construction.

[0020] The reinforcement learning algorithm uses the Proximal Policy Optimization (PPO) algorithm, which uses user satisfaction ratings as reward signals to adjust the allocation coefficients of the attention weight parameters and knowledge base retrieval weights of the large model, and updates the parameters once every 100 valid feedback data points.

[0021] Furthermore, the security protection for the layered deployment of the privatized large model specifically includes: S41. Data transmission encryption: The SSL / TLS protocol is used to encrypt user query requests, inference results and data transmission process, and the key is automatically rotated periodically; S42. Storage Encryption: The core model parameters and knowledge base data are stored using the AES-256 encryption algorithm, and the encryption key is stored using a hardware security module (HSM). S43. Access Control: Set up multi-level access permissions based on the RBAC (Role-Based Access Control) model, so that users with different roles can only access knowledge base data and model functions within their corresponding permission scope; S44. Operation Audit: Record all users' access logs, query logs, and knowledge base operation logs. Log information includes the operator, operation time, operation content, and device information. Log data is retained for at least 90 days.

[0022] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for deploying a private large-scale model and building a RAG knowledge base, characterized in that, include: S1. Layered Deployment of the Private Large Model: Based on the target terminal hardware resource threshold, a three-tier deployment architecture is constructed: "Core Model Layer - Lightweight Adaptation Layer - Interface Adaptation Layer," where: S11. Deploy the core model layer: Perform structured pruning on the pre-trained large model, retain the core neuron cluster of model inference, convert the parameter precision from FP16 to INT8 / INT4 through model weight quantization, and at the same time use sparse matrix storage optimization to reduce the storage occupation of the core model. S12. Construct the lightweight adaptation layer: Based on the number of GPU / CPU cores, memory capacity, and video memory bandwidth, dynamically generate model inference acceleration plugins. The plugins include an instruction set optimization module, a memory scheduling module, and a parallel computing module. The instruction set optimization module adaptively selects the x86 / ARM instruction set according to the hardware architecture. The memory scheduling module adopts a "hot data priority caching + cold data page storage" strategy. The parallel computing module splits the inference task into a sub-task queue that matches the number of hardware cores. S13. Deploy the interface adaptation layer: Provide RESTfulAPI and gRPC dual interface modes, support synchronous inference and asynchronous inference calling methods, and also have a built-in request rate limiting module and permission verification module. The permission verification module adopts a three-level authentication mechanism of "device fingerprint + API key + role permission". S2. Incremental construction of the RAG knowledge base, specifically including: S21. Multi-source data acquisition and standardization: Heterogeneous data is acquired through three methods: web crawler interface, local file parsing, and database synchronization. The heterogeneous data includes text files, tabular data, image OCR recognition results, and audio transcribed text. The acquired data is standardized and uniformly converted into UTF-8 encoded structured documents. Tabular data is converted into key-value pair arrays, and image OCR text is divided into paragraphs by region. S22. Knowledge Unit Segmentation and Semantic Encoding: The standardized document is divided into independent knowledge units by adopting the strategy of "sentence-level segmentation + semantic association clustering". Each knowledge unit contains text content, source identifier and timestamp. The knowledge unit is semantically encoded by the pre-trained BERT model to generate a 768-dimensional vector embedding. At the same time, the keyword weight of each knowledge unit is calculated. S23. Dynamic Knowledge Base Construction: A main knowledge base is built based on a vector database, and a keyword index library and a source index library are established at the same time. The main knowledge base stores the vector embedding and association information of knowledge units. The keyword index library establishes the mapping relationship between keywords and knowledge units. The source index library stores knowledge units according to data source. S24. Incremental Update and Optimization of Knowledge Base: Set up scheduled update tasks and triggered update interfaces. The scheduled update tasks collect new data at a preset period and repeat steps S21-S23. The triggered update interface supports manual data upload and real-time updates. During the update process, a "similarity deduplication algorithm" is used to calculate the semantic similarity between new knowledge units and existing knowledge units. When the similarity is higher than a preset threshold, duplicate storage is rejected. At the same time, the knowledge base is periodically defragmented, invalid knowledge units are deleted, and the vector index is optimized. S3. Collaborative reasoning between the large model and the RAG knowledge base, completed and including: S31. User query processing: After receiving a user query request, the interface adaptation layer performs permission verification and request parsing to extract query keywords and semantic intent; S32. Knowledge Base Retrieval: The parsed query request is converted into a vector embedding, and multi-dimensional retrieval is performed based on keywords, including vector similarity retrieval of the main knowledge base, exact matching retrieval of the keyword index, and range retrieval of the source index. The candidate knowledge set is generated by combining the three types of retrieval results. S33. Reasoning Result Generation: Input the candidate knowledge set and the user query request into the private big model. The big model combines its own parameter knowledge and the candidate knowledge set to perform reasoning, generate the response content, and mark the source of the knowledge unit referenced in the response. S34. Inference Effect Feedback and Optimization: Collect user satisfaction ratings for the response content. When the rating is unsatisfactory, record the query request, response content, and user feedback. Adjust the inference parameters of the large model and the retrieval weights of the knowledge base through reinforcement learning algorithms. At the same time, supplement the knowledge base with effective feedback data as incremental data to complete the construction.

2. The method for deploying a private large-scale model and building a RAG knowledge base according to claim 1, characterized in that, The structured pruning adopts a combination of "intra-layer neuron pruning + inter-layer connection pruning". Intra-layer neuron pruning retains neurons in each layer whose activation value is greater than a preset threshold. Inter-layer connection pruning deletes connection weights whose absolute weight value is less than the pruning threshold. The pruning threshold is dynamically adjusted according to the hardware memory capacity. The parallel computing module adopts a pipeline mechanism of "task splitting-result merging", which splits the inference task into preprocessing subtasks, attention calculation subtasks and output decoding subtasks according to the token generation order. Each subtask is assigned to an independent hardware core for parallel computing, and data interaction between subtasks is realized through shared memory.

3. The method for deploying a private large-scale model and building a RAG knowledge base according to claim 1, characterized in that, The keyword weight calculation adopts the TF-IDF algorithm combined with the TextRank algorithm. The TF-IDF algorithm calculates the word frequency weight of the keyword in the current knowledge unit, and the TextRank algorithm calculates the importance weight of the keyword in the global document. The two are weighted and summed to obtain the final keyword weight. The vector database uses Milvus or FAISS and supports IVF_FLAT or HNSW indexing algorithms. The vector index of the main knowledge base is maintained in a way that combines daily incremental construction with weekly full optimization to improve retrieval efficiency. The similarity deduplication algorithm adopts a fusion algorithm of cosine similarity and Jaccard similarity to calculate the semantic cosine similarity and text intersection similarity between the new knowledge unit and the existing knowledge unit. When the weighted average of the two is higher than 85%, it is determined to be a duplicate knowledge unit.

4. The method for deploying a private large-scale model and building a RAG knowledge base according to claim 1, characterized in that, The specific method for integrating the three types of search results is as follows: the vector similarity search results are sorted by similarity score, and the top 20 results are selected; the keyword exact match search results are sorted by the number of matched keywords, and the top 15 results are selected; the source index range search results are sorted by timestamp, and the top 10 results are selected; after removing duplicates from the three types of results, a comprehensive score is calculated using the weighted formula "vector similarity score × 0.6 + keyword matching weight × 0.3 + time freshness × 0.1", and the top 15 results with the comprehensive scores are selected to form a candidate knowledge set; The reinforcement learning algorithm uses the Proximal Policy Optimization (PPO) algorithm, which uses user satisfaction ratings as reward signals to adjust the allocation coefficients of the attention weight parameters and knowledge base retrieval weights of the large model, and updates the parameters once every 100 valid feedback data points.

5. The method for deploying a private large-scale model and building a RAG knowledge base according to claim 1, characterized in that, The security protection for the layered deployment of the private large model specifically includes: S41. Data transmission encryption: The SSL / TLS protocol is used to encrypt user query requests, inference results and data transmission process, and the key is automatically rotated periodically; S42. Storage Encryption: The core model parameters and knowledge base data are stored using the AES-256 encryption algorithm, and the encryption key is stored using a hardware security module (HSM). S43. Access Control: Set up multi-level access permissions based on the RBAC (Role-Based Access Control) model, so that users with different roles can only access knowledge base data and model functions within their corresponding permission scope; S44. Operation Audit: Record all users' access logs, query logs, and knowledge base operation logs. Log information includes the operator, operation time, operation content, and device information. Log data is retained for at least 90 days.

6. The method for deploying a private large-scale model and building a RAG knowledge base according to claim 1, characterized in that, Step S1 also includes deployment resource monitoring and dynamic adjustment, specifically including: S14. Resource Monitoring: Real-time monitoring of hardware CPU usage, memory usage, video memory usage, and network bandwidth; setting resource usage thresholds. S15. Dynamic Adjustment: When CPU utilization or memory usage exceeds the threshold for 5 consecutive minutes, the lightweight adaptation layer automatically starts the load balancing mechanism to allocate inference tasks to idle hardware resources; when GPU memory usage exceeds the threshold, the core model layer automatically enables model sharding inference to split model parameters into multiple GPU memories for parallel computation; when resource usage is below the threshold of 30%, some redundant parallel computing modules are shut down to reduce resource consumption.

Images