Customer service intelligent response method and system based on semantic understanding

Abstract

The invention discloses a customer service intelligent response method and system based on semantic comprehension. The method comprises the following steps: processing an original large language model by adopting a mixed precision quantification strategy; pruning the quantized model based on a structured pruning algorithm to obtain a lightweight model, and performing fine tuning training by using the labeled field data set on the basis of the lightweight model to construct a precision compensation model; constructing a three-level cache architecture, defining a cache data classification standard, configuring a cache storage rule, and establishing a cache refreshing mechanism; processing a user request and extracting key information, and inputting a lightweight model for reasoning; filling a response template according to the cached data, and feeding back to the user after compliance verification; and carrying out dynamic iteration on the model and the caching strategy. While the reasoning speed is increased, the semantic understanding accuracy is recovered to a high level close to that of an original model, and the requirement of financial customer service for business interpretation accuracy is met.

Description

Technical Field

[0001] This invention relates to the field of intelligent customer service, specifically to a customer service intelligent response method and system based on semantic understanding. Background Technology

[0002] In the process of digital transformation in the financial sector, the intelligent upgrading of customer service has become a core direction for improving service quality and efficiency. Semantic understanding technology, as the core support of intelligent response systems, directly determines the service experience. Currently, large language models are widely used in financial customer service scenarios due to their powerful semantic processing capabilities. However, the large number of parameters in these models makes it difficult to meet the real-time response needs of customer service. To improve response efficiency, the industry often uses quantization and pruning lightweight techniques to compress the models, but this generally faces the dilemma of a significant decrease in semantic understanding accuracy, failing to accurately match the high requirements of financial customer service for accurate business interpretation.

[0003] Patent document CN119474280A discloses an intelligent customer service system based on an AI large-scale model, belonging to the technical field of intelligent customer service systems. This invention enhances the system's flexibility and user experience through multimodal interaction technology. It intelligently identifies data types and processes them accordingly, automatically adjusting the language style and emotional tone of the response content based on the user's tone and intent. The knowledge base integrates static and dynamic interactive data to ensure real-time updates and comprehensiveness of knowledge content. It enables rapid retrieval based on the knowledge base and provides accurate responses using a large language model. Through rule-based reasoning and generative models, it generates high-quality answers and solutions. A dialogue state tracker records and manages the state information of each round of user interaction, achieving accurate tracking and updating of the context. For complex tasks, the system breaks down user requests into multiple sub-tasks and executes them sequentially according to priority and dependencies, ensuring efficient and coherent task processing.

[0004] Patent document CN120450843A discloses a financial business processing method, apparatus, and system based on a large language model. The method includes: when a customer initiates a financial business consultation or processing request, collecting various data generated during the consultation or processing process; preprocessing the collected data and extracting key information; inputting the preprocessed data and the extracted key information into a pre-trained large language model, identifying the customer's intent through the large language model, extracting key business elements, conducting risk assessment, and performing sentiment analysis on the customer; and performing business processing and decision-making based on the customer intent, key business elements, risk assessment results, and sentiment analysis results, combined with the pre-set business rules and risk strategies of the financial institution.

[0005] Meanwhile, significant issues exist in the data loading process for financial customer service: customer inquiries often involve business processes and historical data, and existing systems rely heavily on direct database queries for information, leading to frequent duplicate accesses. This not only increases database load but also causes data loading delays, further slowing down the overall response speed. How to achieve lightweight models to improve inference efficiency while ensuring semantic understanding accuracy, and balancing real-time data loading with caching optimization, has become a critical issue that urgently needs to be addressed in the practical application of intelligent response systems for financial customer service. Summary of the Invention

[0006] The purpose of this invention is to provide a customer service intelligent response method based on semantic understanding, and at the same time, to provide a customer service intelligent response system based on semantic understanding, so as to solve the problems mentioned in the background art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] The first aspect is a semantic understanding-based intelligent customer service response method, which includes the following steps:

[0009] S1. Filter high-frequency customer service dialogue data in the database, and perform deduplication, anonymization, and intent labeling;

[0010] S2. A mixed-precision quantization strategy is used to process the original large language model;

[0011] S3. The quantized model is pruned based on the structured pruning algorithm to obtain a lightweight model. During the pruning process, key parameters are locked by calculating the gradient contribution.

[0012] S4. Based on the lightweight model, fine-tune the training using the labeled domain dataset to build a precision compensation model.

[0013] S5. Construct a three-level caching architecture, including memory cache L1, distributed cache L2, and database cache L3, define cache data classification criteria, configure cache storage rules, and establish a cache refresh mechanism;

[0014] S6. Process user requests and extract key information, then input them into the lightweight model for inference.

[0015] S7. Populate the response template based on the cached data, and then provide feedback to the user after compliance verification;

[0016] S8. Dynamically iterate on the model and caching strategy.

[0017] Furthermore, the high-frequency customer service dialogue data in the database is deduplicated, anonymized, and labeled with intent. Specifically, high-frequency dialogue data is selected from the financial customer service database. Based on the core semantics and business information of the dialogue, redundant data with duplicates and highly similar core content is eliminated by generating data feature fingerprints for comparison. Sensitive customer information is anonymized by focusing on removing privacy content such as names and ID numbers. At the same time, the key business consultation points and interaction logic in the dialogue are fully preserved. The data is labeled according to a preset intent classification system, and finally, a domain-labeled dataset that meets the needs of model training is formed.

[0018] Furthermore, the original large language model is processed using a hybrid precision quantization strategy. Specifically, a large language model adapted to financial customer service scenarios and pre-trained on financial business corpus is selected as the quantization object. The quantization process adopts a hybrid precision strategy of FP16 and INT8. The core semantic understanding layer, which involves customer consultation semantic parsing, financial business terminology understanding, and customer core intent recognition, retains FP16 precision. The non-core computing layer, which is responsible for general text cleaning, sentence standardization, and redundant information filtering, uses INT8 precision for quantization compression.

[0019] Furthermore, the structured pruning algorithm is used to prune the quantized model to obtain a lightweight model. Specifically, based on the mixed-precision quantized model, pre-inference is performed using a labeled financial customer service domain dataset. The gradient contribution of each parameter is calculated layer by layer through the gradient backpropagation mechanism. Based on the cumulative effect of the parameters in multi-round dialogue sample inference, a parameter importance ranking table containing information on parameter position, network layer, and contribution is generated. Parameters directly related to the extraction of financial business semantic features in the core semantic understanding layer and key parameters supporting the basic semantic processing logic in the non-core computing layer are locked. The structured pruning operation retains the locked parameters and removes parameters with low contribution and high redundancy.

[0020] Furthermore, based on the lightweight model, a precision compensation model is constructed by fine-tuning the training using labeled domain datasets. Specifically, based on the lightweight model, fine-tuning training is performed using high-frequency customer service dialogue data selected in the early stage. An optimizer and loss function are used for training, and training quality is ensured through validation set monitoring and early stopping mechanisms. After fine-tuning, the model maintains the lightweight inference speed while restoring the semantic understanding accuracy to a high level of the original model, thus forming a precision compensation model.

[0021] Furthermore, the three-level caching architecture includes a memory cache L1, a distributed cache L2, and a database cache L3. Specifically, based on the real-time requirements of data access frequency and storage cost differentiation, L1 is deployed in the core server memory to achieve high-speed access to store high-frequency data. L2 uses a Redis cluster to ensure high availability and handle requests that are missed by L1. L3 works in conjunction with the business database to preload low-frequency data to reduce direct queries. The three-level cache achieves hierarchical flow through a data synchronization protocol to ensure consistency, and each layer is configured with an independent fault isolation mechanism.

[0022] Furthermore, the defined cached data classification standard specifically divides customer service-related data into three categories: high-frequency fixed data, including basic business process descriptions, standard answers to common questions, and standardized service scripts; medium-frequency dynamic data, including historical consultation records, business processing progress, and personalized service preferences; and low-frequency real-time data, including real-time financial data, temporary business announcements, and real-time data related to special business consultations.

[0023] Furthermore, the configured cache storage rules are as follows: L1 cache stores high-frequency fixed data, which automatically expires upon reaching its expiration date and triggers a synchronous update with L2 cache. L2 cache, as the core relay layer, stores all high-frequency fixed data and all medium-frequency dynamic data. It improves concurrent access capabilities by balancing node load through distributed storage. L3 cache undertakes the functions of low-frequency real-time data storage and full data backup. It synchronizes the latest data from the business database according to the rules and only activates backup data to ensure access continuity when the upper-level cache fails or misses.

[0024] Furthermore, the establishment of the cache refresh mechanism specifically involves the system receiving service requests initiated by users from multiple channels in real time, standardizing the format of the request content, calling a lightweight natural language processing tool, and performing word segmentation, part-of-speech tagging, and entity recognition operations on the standardized request text based on a semantic dictionary for the financial customer service domain; extracting intent keywords and auxiliary information, and constructing a structured information matrix containing user intent categories, core entity content, and key request elements after extraction, associating core entities with customer identity identifiers, and re-executing the information extraction operation after clarifying the user's core needs.

[0025] Secondly, the system for intelligent customer service response methods includes:

[0026] The data preprocessing module is used to filter high-frequency customer service dialogue data in the database, and complete deduplication, desensitization and intent labeling.

[0027] The model lightweight module is used to process the original large language model using a mixed-precision quantization strategy, prune the quantized model based on the structured pruning algorithm, and lock key parameters by calculating gradient contribution.

[0028] The accuracy compensation module is used to build an accuracy compensation model based on the lightweight model and fine-tuned training using an annotated domain dataset.

[0029] The cache management module is used to build a three-level cache architecture, define cache data classification standards, configure cache storage rules, and establish a cache refresh mechanism.

[0030] The request inference module is used to process user requests and extract key information, and input the information into the lightweight model to complete the inference.

[0031] The response generation module is used to populate the response template based on cached data and, after compliance verification, to return the response content to the user.

[0032] The iterative optimization module is used to dynamically iteratively optimize the model and caching strategy.

[0033] Compared with existing technologies, this invention has the following beneficial effects: Addressing the issues of slow inference in large language models and decreased accuracy after lightweighting, the solution achieves model lightweighting through hybrid precision quantization and structured pruning. Simultaneously, it leverages high-frequency domain data to fine-tune and construct a precision compensation model. This improves inference speed while restoring semantic understanding accuracy to a level close to the original model, meeting the accuracy requirements of financial customer service for business interpretation. Regarding the issues of frequent repeated accesses and high latency during data loading, a three-level caching architecture combines data classification standards with differentiated storage and refresh rules. This enables precise matching of data to cache levels based on access characteristics, prioritizing data retrieval from the high-speed cache, significantly reducing the frequency and load of direct database access, lowering data loading latency, and improving cache hit rate and real-time data loading. The synergistic effect of the model and cache significantly improves overall response speed, meeting the real-time needs of customer service. The dynamic iteration mechanism continuously optimizes the model and caching strategies through user feedback and performance data, ensuring that both adapt to business changes and maintain efficient and accurate service performance in the long term. Attached Figure Description

[0034] Figure 1 This is a flowchart of the semantic understanding-based intelligent customer service response method of the present invention. Detailed Implementation

[0035] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] This application discloses a customer service intelligent response method based on semantic understanding, such as... Figure 1The steps include:

[0037] A semantic understanding-based intelligent customer service response method includes the following steps:

[0038] S1. Filter high-frequency customer service dialogue data in the database, and perform deduplication, anonymization, and intent labeling;

[0039] S2. A mixed-precision quantization strategy is used to process the original large language model;

[0040] S3. The quantized model is pruned based on the structured pruning algorithm to obtain a lightweight model. During the pruning process, key parameters are locked by calculating the gradient contribution.

[0041] S4. Based on the lightweight model, fine-tune the training using the labeled domain dataset to build a precision compensation model;

[0042] S5. Construct a three-level caching architecture, including memory cache L1, distributed cache L2, and database cache L3, define cache data classification criteria, configure cache storage rules, and establish a cache refresh mechanism;

[0043] S6. Process user requests and extract key information, then input them into the lightweight model for inference;

[0044] S7. Populate the response template based on the cached data, and then provide feedback to the user after compliance verification;

[0045] S8. Dynamically iterate on the model and caching strategy.

[0046] S1 involves filtering high-frequency customer service dialogue data from the database, performing deduplication, anonymization, and intent labeling. Specifically, this involves filtering high-frequency dialogue data from a financial customer service database, covering typical customer service scenarios such as inquiries, complaints, and processing, with a total data volume of no less than 500,000 entries. Deduplication is based on the core semantics and business information of the dialogue, using data feature fingerprints for comparison to eliminate completely duplicated or highly similar redundant data. Anonymization focuses on sensitive customer information, precisely removing customer names, ID numbers, bank card numbers, contact numbers, and other personal privacy content, while fully preserving the key business consultation points and interaction logic of the dialogue. Intent labeling is performed according to a pre-defined customer service intent classification system. After labeling, quality is controlled through two-person cross-validation to ensure an accuracy rate of no less than 99%, ultimately forming a customer service domain labeled dataset that meets the needs of model training.

[0047] In generating data feature fingerprints, each dialogue data is first structured and parsed to extract core semantic vectors, keyword frequency distribution, business elements (such as consultation service type, core requests, and names of involved products / services), and sentence structure features. The semantic vectors are generated by encoding the dialogue text using Word2Vec or BERT models, with dimensions set to 256-512. The keyword frequency distribution is based on the frequency of core words in a semantic dictionary for financial customer service (including financial business terms, common request words, etc.). The sentence structure features are converted into standardized feature vectors by parsing the subject-verb-object grammatical structure. After normalizing the extracted multi-dimensional features, the SHA-256 hash algorithm is used to encrypt the fused feature vectors to generate a fixed-length 256-bit data feature fingerprint. During comparison, the Hamming distance between the feature fingerprints of two dialogue data is calculated. A Hamming distance threshold of ≤3 is set as the criterion for judging the high similarity of core content. When the Hamming distance between two data is ≤3, it is judged as redundant data and is removed to ensure the repeatability and accuracy of the deduplication operation.

[0048] In step S2, a hybrid precision quantization strategy is employed to process the original large language model. Specifically, large language models adapted to financial customer service scenarios and pre-trained on basic financial business corpora are prioritized for quantization, ensuring that the model possesses preliminary semantic processing capabilities in the financial domain. The quantization process uses a hybrid precision strategy of FP16 and INT8 to precisely differentiate the model's functional modules for differentiated processing. The core semantic understanding layer, which involves customer consultation semantic parsing, financial business terminology understanding, and customer core intent recognition, retains FP16 precision to maintain high-precision semantic feature extraction capabilities and ensure the accuracy of business interpretation. Meanwhile, non-core computational layers responsible for general text cleaning, sentence standardization, and redundant information filtering are quantized and compressed using INT8 precision, reducing the number of model parameters and computational overhead while avoiding damage to the core semantic processing performance. Before quantization, the labeled financial customer service dataset was used as the calibration basis. The quantization parameters of each layer were determined by dynamic range statistics to reduce quantization error. After quantization, inference tests were immediately conducted using high-frequency customer service dialogue samples to ensure that the semantic understanding error of the quantized model in typical financial customer service scenarios was controlled within an acceptable range, laying the foundation for subsequent pruning operations and accuracy compensation.

[0049] Among them, the large language model adapted for financial customer service scenarios has a financial business corpus covering core financial business areas, specifically including text data of core business modules such as personal savings (e.g., deposit interest rate inquiry, account loss reporting), credit business (e.g., loan application conditions, repayment plan consultation), credit card services (e.g., bill inquiry, installment processing), wealth management products (e.g., risk level explanation, return calculation), and payment and settlement (e.g., transfer limits, arrival time). It also includes regulatory policy clauses (e.g., anti-money laundering regulations, consumer rights protection regulations), internal financial service standards (e.g., customer service response script standards, business processing process guidelines), and historical customer service interaction corpus (high-frequency dialogue records from the past 3 years, and a frequently asked questions database). The total amount of pre-training corpus is no less than 100GB, and the data has been manually screened to ensure accuracy and compliance. The pre-training stage focuses on completing The model addresses three core tasks: accurate understanding of financial terminology, classification of financial business intent, and generation of compliant expressions. Its basic architecture utilizes a Chinese-optimized version of Transformer (such as RoBERTa-Chinese or MacBERT), with 12-24 encoder layers, 768-1024 hidden layer dimensions, and 12-16 attention heads. This architecture ensures the model's semantic understanding of Chinese financial text. Pre-training employs batch sizes of 32-64, an initial learning rate of 5e-5, and at least 10 training epochs. Finally, the model is validated using a financial domain semantic understanding test set (containing 100,000 labeled samples covering 20 core financial customer service intents). The model's intent recognition accuracy is no less than 92%, ensuring it possesses the basic semantic processing capabilities suitable for financial customer service scenarios.

[0050] In S3, a structured pruning algorithm is used to prune the quantized model. Specifically, the model with mixed-precision quantization is used as the processing object, focusing on the core needs of the financial customer service semantic understanding scenario. The pruning targets are the fully connected layers and convolutional layers with high parameter redundancy, avoiding damage to key semantic parsing modules. Before pruning, the quantized model is pre-inferenced using a labeled customer service domain dataset. Gradient backpropagation is used to calculate the gradient contribution of each layer's parameters in the core tasks of financial business intent recognition and business terminology understanding, generating a parameter importance ranking table to identify the key parameters that determine the accuracy of semantic understanding. A structured pruning algorithm is used for layer-by-layer pruning, with the pruning rate strictly controlled between 30% and 40%. A low pruning rate (around 30%) is used for the core semantic understanding layer, while a relatively high pruning rate (around 40%) is used for non-core computational layers to achieve differentiated compression. During the pruning process, the semantic matching accuracy of the model output is monitored in real time. If the intent recognition accuracy of the model in typical customer service scenario samples drops by more than 2% after a certain round of pruning, the previous round of pruning results is reverted and the pruning ratio of that layer is adjusted. After pruning, the remaining parameters of the model are reconnected and standardized to eliminate the parameter discontinuity caused by pruning, ensuring the integrity of the model structure and the coherence of the computational logic, and providing a stable model foundation for subsequent accuracy compensation training.

[0051] In the S3 pruning process, key parameters are locked through gradient contribution calculation. Specifically, based on a model with mixed-precision quantization, the model is first warmed up using a labeled high-frequency dialogue dataset from the financial customer service domain. During inference, the focus is on core financial customer service tasks: identifying financial business intent, accurately understanding business terminology, and analyzing core customer needs. The gradient contribution of each parameter in these tasks is calculated layer by layer through a gradient backpropagation mechanism. The gradient contribution calculation uses the weight of the parameter's influence on the model's output semantic matching results as the core indicator. Based on the cumulative effect of the parameter in multi-round dialogue sample inference, a parameter importance ranking table is generated, including parameter position, network layer, and contribution value. For parameters in the core semantic understanding layer directly related to the extraction of financial business semantic features, and for key parameters in non-core computation layers supporting basic semantic processing logic, their gradient contribution values ​​are locked when they exceed a preset threshold. Locked parameters are retained in subsequent structured pruning operations, ensuring that the pruning process only removes parameters with low contribution and high redundancy, avoiding damage to core semantic processing capabilities.

[0052] In S4, a precision compensation model is constructed based on a lightweight model and fine-tuned using an annotated domain dataset. Specifically, the lightweight model, after undergoing hybrid precision quantization and structured pruning, serves as the basis for fine-tuning. This model retains core semantic understanding parameters through gradient contribution calculation, reducing the number of parameters by 30%-40% compared to the original model, while maintaining a complete core semantic processing module structure. The annotated domain dataset consists of over 500,000 high-frequency dialogue data points selected from a financial customer service database. After deduplication and anonymization, the data was jointly annotated by business experts and algorithm engineers, achieving an accuracy rate of no less than 99%. It covers 10 typical customer service scenarios, including inquiries, complaints, and processing, and includes complete customer intent tags and business information.

[0053] Before fine-tuning the training, the dataset is divided into training, validation, and test sets in an 8:1:1 ratio. The training set is used for updating model parameters, the validation set is used to monitor training performance in real time, and the test set is used for final performance evaluation. The training process employs a mini-batch gradient descent strategy with a batch size of 32. The AdamW optimizer is used to improve training stability, and the cross-entropy loss function is used to calculate the model's loss value on the customer intent recognition task. The training epochs are set to 50, with an initial learning rate of 0.001. The learning rate decays by 50% every 10 epochs to prevent later training oscillations. After each epoch, the validation set is used to test the model's semantic understanding accuracy in typical customer service scenarios. If the accuracy improvement on the validation set is less than 0.1% for three consecutive epochs, an early stopping mechanism is triggered to prevent overfitting.

[0054] After fine-tuning, a comprehensive performance evaluation was conducted using a test set to ensure that the model's semantic understanding accuracy in the financial customer service domain was restored to more than 98% of the original large language model, while maintaining the inference speed advantage brought by lightweight design, ultimately forming a precision compensation model that combines high accuracy and high efficiency.

[0055] This application further customizes the design to address the characteristics of "strong correlation of financial semantics and high requirements for the accuracy of business intent" in financial customer service scenarios, thereby improving the pertinence of accuracy compensation. In specific implementation, instead of using a fixed ratio for division before fine-tuning training, dynamic stratification is achieved based on financial semantic clustering.

[0056] First, each dialogue text is encoded into a semantic vector using a pre-trained financial domain word vector model (such as a financial version of BERT). d is the vector dimension, i is the number of rows in the dataset, i = 1, 2, ..., N, and N is the total number of rows in the dataset;

[0057] Subsequently, the density peak clustering algorithm was used to cluster all semantic vectors, and the local density ρ of each vector was calculated. i With distance δ iSpecifically:

[0058] Local density ρ i =∑ j≠i χ(d ij -d c ), where d ij For vector x i With x j The Euclidean distance, d ij =||x i -x j ||2,d c To truncate the distance, χ(x) is an indicator function (χ(x) = 1 when x < 0, otherwise χ(x) = 0); distance (If ρ i If δ is the maximum value, then i =max j d ij ).

[0059] According to ρ i With δ i Product selection of cluster centers, calculation of γ i =ρ i ·δ i (γ i For selecting cluster centers, the index is dimensionless, calculated by γ. i The samples are sorted in descending order, and the top 20 are taken as cluster centers (K=20, which strictly corresponds to the number of intent subclasses, ensuring that each cluster center corresponds to the core semantics of an intent subclass). Data is randomly extracted from each cluster in a ratio of 7:1.5:1.5 to form training, validation and test sets, ensuring that the business intent distribution and financial entity coverage of each set are completely consistent, avoiding the semantic distribution offset problem caused by traditional fixed division.

[0060] The training process employs a "domain semantic alignment gradient adaptive optimizer," which is an extension of the Adam optimizer. The core feature is the introduction of a domain semantic weight factor ω. t Dynamically adjust the update step size of core financial parameters:

[0061] For the parameters θ in the t-th training iteration of the model, calculate the gradient g. t , in, For loss function, This represents the training batch data for round t.

[0062] Calculate the first moment estimate m t =β1m t-1 +(1-β1)g t Second-order moment estimation (β1 = 0.9 is the first-order moment attenuation coefficient, β2 = 0.999 is the second-order moment attenuation coefficient;

[0063] semantic weighting factor in computing domain Where M represents the size of the core financial terminology database (containing 1000 core terms such as "installment fee rate" and "loan approval"). Let s be the set of financial terms contained in the t-th batch of data. k The domain importance score for term k (annotated by business experts, with a value range of [1, 5], and a score ≥ 3.5 for core business terms). For indicator functions;

[0064] The parameter update formula is as follows in The moment estimate is the bias-corrected value, η = 0.0015 is the initial learning rate, and ∈ = 10. -8 To prevent the minimum value where the denominator is 0.

[0065] The optimizer uses ω t To achieve "larger update steps for parameters related to core financial terminology and smaller update steps for general text processing parameters," ensuring that accuracy compensation is more focused on the core semantic understanding needs of financial customer service scenarios, a "cross-entropy-financial semantic distance hybrid loss function" is adopted. This function simultaneously constrains intent classification accuracy and domain semantic adaptability. The formula is as follows:

[0066]

[0067] Where α = 0.7 is the weighting coefficient (balancing the contributions of the two types of losses), and the specific calculation is as follows:

[0068] Cross-entropy loss B is the batch size (set to 32), c = 20 is the number of intent categories, y b,c Let p be the true label of the b-th sample (one-hot encoded, correct class is 1, all others are 0). b,c Predict the probability that the b-th sample belongs to class c for the model;

[0069] Financial semantic distance loss Where h b Let z be the semantic feature vector output by the last layer of the model for the b-th sample. c Let h be the semantic center vector of category c in the financial domain (composed of h values ​​of all samples of category c in the training set). b (Average obtained), ||·||2 is the Euclidean distance, ∈=10 -6 This is a smoothing term.

[0070] By constraining the distance between the sample semantic features and the corresponding category domain center, we ensure that the semantic features output by the model are more in line with the domain characteristics of the financial customer service scenario, and avoid general semantic shift.

[0071] The training rounds are set to 60, and the learning rate adopts "semantic clustering adaptive decay": after every 15 rounds of training, the financial semantic distance loss of the validation set is calculated. like If the decrease in learning rate is less than 0.005, the learning rate is reduced by 40% (initial learning rate η = 0.0015); otherwise, the current learning rate is maintained to avoid premature stagnation caused by traditional fixed decay.

[0072] After each round of training, two metrics are calculated using the validation set: 1. Semantic understanding accuracy (intent classification accuracy); 2. Financial business entity recognition accuracy (such as the accuracy of product name and amount recognition). If the improvement in semantic understanding accuracy on the validation set is less than 0.08% for four consecutive rounds, and the improvement in financial business entity recognition accuracy is less than 0.1%, an early stopping mechanism is triggered to prevent overfitting.

[0073] After fine-tuning, a comprehensive performance evaluation was conducted using a test set, while maintaining the inference speed advantage brought by lightweight design (inference time per request ≤100ms), ultimately forming a precision compensation model that combines high accuracy, high domain adaptability, and high efficiency.

[0074] The S5 architecture employs a three-tiered caching system, comprising a memory cache (L1), a distributed cache (L2), and a database cache (L3). In practice, based on the data access characteristics and real-time response requirements of financial customer service, a hierarchical and complementary three-tiered caching architecture is constructed. Each cache tier is designed differently based on data access frequency, real-time requirements, and storage costs, forming an efficient data access chain. The memory cache (L1) is deployed in the local memory of the core application server of the intelligent customer service response system. It is implemented using a hash table-based in-memory database, possessing microsecond-level data read / write capabilities, and is specifically used to store frequently accessed core business data, ensuring extremely fast access speeds. The distributed cache (L2) adopts a Redis cluster deployment mode, ensuring high availability and data consistency through master-slave replication and a sentinel mechanism. Distributed across multiple server nodes, it overcomes the memory limitations of a single node, achieving distributed data storage and load balancing. It handles data access requests that miss in the L1 cache. The L3 database cache serves as the underlying support of the caching system, linking with the core financial business database. It establishes dedicated cache partitions based on the database table structure, preloading and indexing low-frequency access but long-term retention business data to reduce the frequency of direct access to the business database. The three-level caching architecture achieves data flow between layers through a data synchronization protocol, ensuring data consistency across all cache layers. At the same time, each layer is configured with an independent fault isolation mechanism to prevent a single cache layer failure from affecting the stability of the overall data access service. It works in synergy with subsequent cache data classification standards, storage rules, and refresh mechanisms to jointly improve the loading efficiency of financial customer service data.

[0075] The S5 defines a cache data classification standard. In practice, this means establishing a multi-dimensional cache data classification standard based on the data access characteristics, real-time requirements, and business attributes in the financial customer service scenario, to ensure that the classification results are compatible with the storage requirements and service response efficiency goals of the subsequent three-level cache architecture. Based on data access frequency, update frequency, and business relevance, customer service-related data is clearly divided into three categories: Category 1 is high-frequency fixed data, which is relatively stable in content, with a low update frequency but a high access frequency. This category mainly includes descriptions of various basic financial business processes, such as the personal online banking activation process, credit card application requirements, standard answers to common questions (such as password reset methods and transfer limit regulations), and standardized service scripts. Category 2 is medium-frequency dynamic data, which is strongly correlated with specific customers, with a medium update frequency and a low access frequency. This category mainly covers customers' historical consultation records within the past three months, the progress of currently processed business (such as loan approval milestones and wealth management product redemption status), and customers' personalized service preference settings. Category 3 is low-frequency real-time data, which has extremely high real-time requirements, is frequently updated but has a low access frequency. This category mainly includes real-time financial market data (such as real-time exchange rates and fund net asset values), temporary financial announcements (such as system upgrade notices and temporary branch closure information), and real-time data related to customers' sudden special business inquiries. With clear classification criteria, we can provide a precise basis for the formulation of subsequent cache storage rules and the allocation of cache data at each level, ensuring that the corresponding cache level can be quickly matched when data is loaded, thereby improving the cache hit rate.

[0076] The S5 configuration includes cache storage rules. Specifically, based on cache data classification standards and the functional positioning of the three-level cache architecture, differentiated cache storage rules are formulated to achieve precise matching of data storage and access efficiency. High-frequency fixed data is stored in the L1 cache using business type + data identifier as the key-value pair. The storage threshold is set to an access frequency exceeding 100 times within the past hour, and the cache validity period is set to 2 hours. After the validity period expires, the cache automatically expires and triggers a synchronous update with the L2 cache. The L2 cache, as the core relay layer, stores all high-frequency fixed data and all medium-frequency dynamic data. High-frequency fixed data uses a unified data code as the key-value pair, and the cache validity period is extended to 24 hours. Medium-frequency dynamic data is associated and stored using customer ID + business scenario as a composite key-value pair, ensuring that the personalized data of different customers is isolated from each other. The data validity period is set to 4 hours. At the same time, with the help of the data sharding strategy of the Redis cluster, the data is distributed and stored on different nodes according to the credit, wealth management and payment business types, so as to balance the node load and improve the concurrent access capability. The L3 cache is responsible for the storage of low-frequency real-time data and the backup of full data. The low-frequency real-time data uses the data source + timestamp as the key value and the cache validity period is set to 5 minutes. After each cache cycle, the latest data is automatically synchronized from the business database. The backup data of high-frequency fixed data and medium-frequency dynamic data maintains the same key value as L1 and L2 caches and is only activated when the upper-level cache fails or misses, so as to ensure the continuity of data access. All cache data at all levels are stored in a lightweight compression encoding format, which reduces storage overhead while retaining the complete data structure, ensuring that no additional data parsing operations are required during the subsequent response template filling process.

[0077] The S5 implementation establishes a cache refresh mechanism. Specifically, based on the classification characteristics of cached data and the hierarchical positioning of the three-level cache architecture, a differentiated and efficient cache refresh mechanism is established to ensure the real-time and accuracy of various types of data in the cache, while reducing the additional performance overhead of refresh operations. For high-frequency fixed data, which has a low update frequency but high core importance, a scheduled full refresh strategy is adopted. Relying on the system's scheduled task scheduling mechanism, a refresh is triggered daily at 3:00 AM during off-peak business hours. The latest business process descriptions and frequently asked questions answers are extracted from the financial customer service business database and synchronously overlaid on the corresponding content in L1, L2, and L3 caches. After the refresh is completed, a data verification mechanism confirms that the old and new data have been correctly replaced, ensuring data consistency during service periods. For mid-frequency dynamic data, an event-driven real-time incremental refresh mode is adopted. When a customer's business status changes, such as loan approval node updates, addition of historical consultation records, or modification of personalized service preferences, the system instantly captures the relevant business events and generates refresh instructions. Within one second, the corresponding customer ID-related data in the L2 cache is updated, and simultaneously pushed to the L1 cache to ensure that the latest dynamic information can be obtained when customers consult later. Incremental refresh only operates on changed data blocks, avoiding the resource waste caused by processing all data. For low-frequency real-time data, a dual refresh strategy of "access trigger + timed fallback" is adopted. Whenever a user request triggers access to this type of data, the system first queries the business database to obtain the latest data, such as real-time exchange rates and temporary business announcements, and synchronously overwrites the old data in the three-level cache. At the same time, a 5-minute timed refresh task is set to actively pull updates from the database even without access requests, ensuring that the data timeliness meets service requirements. If database access delays or anomalies occur during the refresh process, the system temporarily uses existing data in the cache and marks it as "data updating" to avoid service interruption. To ensure the stable operation of the refresh mechanism, a cache refresh log recording system is established to record in detail the refresh time, data type, and operation result information of each cache level. At the same time, a refresh conflict handling mechanism is set up. When the same data faces multiple refresh requests, a distributed lock is used to queue the operations, prevent data overwriting errors, and ensure that the cached data maintains consistency and integrity while being updated in real time. This works in conjunction with the cache storage rules and data loading mechanism to further improve data access efficiency.

[0078] The S6 process handles user requests and extracts key information. Specifically, in implementation, the system receives service requests from users through multiple channels, including online customer service, telephone voice, and mobile financial apps, in real time. First, the request content is standardized. If it's a voice request, it's converted to text using speech-to-text technology, with an accuracy rate of at least 99.5%. If it's a text request, special symbols, meaningless interjections, and redundant information are filtered out. Then, a lightweight natural language processing tool is used to perform word segmentation, part-of-speech tagging, and entity recognition on the standardized request text. The word segmentation process optimizes word boundary division based on a semantic dictionary for the financial customer service domain, ensuring the integrity of professional terms such as "personal online banking" and "credit card installment." The process involves segmentation and entity recognition, focusing on core entities such as customer ID, business type, transaction amount, and processing time limit. It also extracts keywords related to intent ("inquiry," "complaint," "processing") and auxiliary information such as time and location. After extraction, a structured information matrix is ​​built, containing three dimensions: user intent category, core entity content, and key request elements. Core entities must be associated with customer identity identifiers to match subsequent cached data. The entire process is optimized through parallel computing, ensuring the total time from receiving the request to completing information extraction is controlled within 50ms. If a vague request is encountered, the system automatically adds a prompt such as "Are you inquiring about the processing procedure for XX business?" to further clarify the user's core needs before re-executing the information extraction operation.

[0079] The S6 input is a lightweight model for inference. Specifically, in implementation, a pre-processed structured information matrix containing user intent keywords and core entity information, along with associated data loaded from a three-level cache system (including customer historical consultation records, corresponding business type process specifications, and frequently asked questions), are input into a lightweight semantic model that has undergone mixed precision quantization, structured pruning, and precision compensation fine-tuning for inference. During inference, the model prioritizes calling the built-in financial customer service domain semantic dictionary to enhance the parsing of financial terms such as "personal online banking activation" and "credit card installment rates," deepening the understanding of potential user needs based on historical customer interaction data. The inference process first processes the input information through a semantic feature extraction module, generating feature vectors containing business attributes and intent tendencies. Then, an intent classifier accurately categorizes user needs, outputting the response template identifier with the highest semantic matching degree and the inference confidence score. To ensure real-time response, the inference process employs parallel model computation optimization, allocating feature extraction and intent classification tasks to different computing units for synchronous processing, ensuring that the inference time for a single request is controlled within 100ms. If the inference confidence level is lower than a preset threshold, the system will automatically trigger a second inference, loading more relevant historical dialogue data to assist in the judgment. If the confidence level still does not meet the standard after the second inference, it will be marked as an ambiguous intent and a lightweight manual intervention prompt will be triggered, while the current inference result will be retained as a reference. After the inference is completed, the model outputs the user intent category, inference confidence level, and the matched response template identifier, providing a core basis for the generation of subsequent response content.

[0080] In S7, the response template is populated based on cached data and fed back to the user after compliance verification. Specifically, after the model outputs the user intent category, inference confidence, and matching response template identifier, the system first retrieves the corresponding basic response template from a dedicated financial customer service response template library based on the template identifier. This library stores templates categorized into 10 customer service scenarios: consultation, complaints, and processing. It includes various template types such as business description, question answering, and process guidance, and the template content has been reviewed and filed by the financial compliance department. Subsequently, based on the extracted core user entity information, such as customer ID and business type, the system accurately matches and loads related data from the three-level cache system. Specifically, it extracts historical customer consultation records and current business processing progress data from the L1 or L2 cache, and high-frequency fixed data such as business process descriptions and answers to frequently asked questions from the L2 or L3 cache. If real-time exchange rates or temporary announcements are involved, low-frequency real-time data is simultaneously obtained from both the cache and the business database after dual verification. After the data is loaded, the system automatically fills in the corresponding content according to the preset field rules of the template, generating an initial response containing personalized customer information and precise business content. For example, for a request to "check loan approval progress," the system automatically fills in the customer's name, application number, current approval stage, and estimated completion time. Next, a compliance verification process is initiated. The first verification checks the accuracy of the business descriptions in the response against the official financial business terminology manual, avoiding errors in process descriptions and discrepancies in fee explanations. The second verification uses a sensitive information filtering algorithm to scan for and block any residual customer ID numbers or bank card numbers, ensuring compliance with data security regulations. The third verification, based on the user request scenario and historical interaction records, determines whether the response tone is appropriate; for example, an empathetic and reassuring tone is used for complaint requests, while a clear and professional tone is used for inquiry requests. If any issues are detected during verification, the system immediately triggers a correction mechanism. If business terminology is incorrect, standard expressions are retrieved and replaced; if sensitive information remains, secondary desensitization is performed; and if the tone is inappropriate, the contextual tone adjustment module is invoked for optimization. After verification, the final response content is sent to the user in real time through the requested channel, such as an online customer service pop-up, telephone voice broadcast, or APP message push. The entire population and verification process takes less than 20ms. At the same time, the response content, verification results, and feedback time data are recorded to provide a basis for subsequent model and caching strategy iterations.

[0081] The S8 implementation dynamically iterates the model and caching strategy. Specifically, this involves establishing a linked iterative mechanism between the model and caching strategy, using real-time collected service operation data and user feedback as the core basis for dynamic optimization. Data collection dimensions cover two aspects: first, user feedback data, including satisfaction ratings (1-5 points) for response results, trigger scenarios for manual intervention, and the accurate response content after manual correction; second, system performance data, including the single inference time of the lightweight model, data loading time for different cache levels, cache hit rate for various data types, and the number of repeated accesses and total access frequency of the business database. All data is recorded in real-time, and data cleaning and summary analysis are performed daily at 2 AM to generate a structured iterative analysis report. Iterative optimization is implemented in modules based on model and caching strategies. For model iteration, at 4 AM daily, based on aggregated data, the semantic understanding accuracy of the model in 10 typical customer service scenarios (consultation, complaints, and processing) is monitored. If the accuracy in a certain scenario falls below 95%, new dialogue data for that scenario is immediately extracted, deduplicated, anonymized, and intent-labeled (with an accuracy of at least 99%), before being added to the domain dataset. AdamW is used as the optimizer, and cross-entropy as the loss function to incrementally fine-tune the accuracy compensation model. The fine-tuning rounds are set to 20 rounds, with an initial learning rate of 0.0005, decaying by 50% every 5 rounds. The fine-tuning process is used to validate... The system integrates real-time monitoring of accuracy changes to avoid overfitting. Regarding cache strategy iteration, for data types with an L1 cache hit rate below 60%, the cache level is adjusted based on their access frequency. For example, the validity period of the L1 cache for high-frequency fixed data is extended from 2 hours to 3 hours, or some medium-frequency dynamic data is promoted from L2 cache to L1 cache storage. If the number of repeated accesses to the business database exceeds 1,000 times per day, the data access log is used to locate the category to which the data that did not hit the cache belongs, and its cache storage rules are optimized. For example, the L3 cache preloading range for low-frequency real-time data is expanded, or the cache association key value for medium-frequency dynamic data is adjusted to improve matching efficiency. The iterative model and caching strategy undergo dual verification. The model is verified through a typical customer service scenario test set to ensure that the semantic understanding accuracy is restored to over 98% and the inference speed does not decrease significantly. The caching strategy is verified through simulated high-concurrency access tests to ensure that the overall cache hit rate is maintained above 90%. After verification, it is deployed in a canary release mode, initially covering 10% of user service traffic. Response time, user satisfaction, and system stability indicators are continuously monitored for 24 hours. If there are no abnormal fluctuations, the coverage ratio is gradually expanded to 100%. At the same time, an iteration version rollback mechanism is established. If a problem occurs with the new solution, it can be rolled back to the previous stable version within 5 minutes to ensure service continuity. This forms a complete iterative closed loop of "data collection - analysis and optimization - verification and release - monitoring and rollback" to ensure that the model and caching strategy always adapt to the dynamic changes in the financial customer service business.

[0082] This application also protects a customer service intelligent response system based on semantic understanding, including: a data preprocessing module for filtering high-frequency customer service dialogue data in the database to complete deduplication, desensitization and intent labeling;

[0083] The model lightweight module is used to process the original large language model using a mixed-precision quantization strategy, prune the quantized model based on the structured pruning algorithm, and lock key parameters by calculating gradient contribution.

[0084] The accuracy compensation module is used to build an accuracy compensation model based on the lightweight model and fine-tuned training using an annotated domain dataset.

[0085] The cache management module is used to build a three-level cache architecture, define cache data classification standards, configure cache storage rules, and establish a cache refresh mechanism.

[0086] The request inference module is used to process user requests and extract key information, and input the information into the lightweight model to complete the inference.

[0087] The response generation module is used to populate the response template based on cached data and, after compliance verification, to return the response content to the user.

[0088] The iterative optimization module is used to dynamically iteratively optimize the model and caching strategy.

[0089] This application also protects a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the semantic understanding-based intelligent customer service response method provided in the embodiments of the present invention.

[0090] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device.

[0091] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, system, or device.

[0092] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0093] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A customer service intelligent response method based on semantic understanding, characterized in that, Including the following steps: S1. Filter high-frequency customer service dialogue data in the database, and perform deduplication, anonymization, and intent labeling; S2. A mixed-precision quantization strategy is used to process the original large language model; S3. The quantized model is pruned based on the structured pruning algorithm to obtain a lightweight model. During the pruning process, key parameters are locked by calculating the gradient contribution. S4. Based on the lightweight model, fine-tune the training using the labeled domain dataset to build a precision compensation model. S5. Construct a three-level caching architecture, including memory cache L1, distributed cache L2, and database cache L3, define cache data classification criteria, configure cache storage rules, and establish a cache refresh mechanism; S6. Process user requests and extract key information, then input them into the lightweight model for inference. S7. Populate the response template based on the cached data, and then provide feedback to the user after compliance verification; S8. Dynamically iterate on the model and caching strategy.

2. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The process involves filtering high-frequency customer service dialogue data from the database, performing deduplication, anonymization, and intent labeling. Specifically, high-frequency dialogue data is filtered from the financial customer service database. Based on the core semantics and business information of the dialogue, redundant data with highly similar core content is eliminated through data feature fingerprint comparison. Sensitive customer information is anonymized by focusing on removing privacy content such as names and ID numbers, while retaining the key business consultation points and interaction logic in the dialogue. The data is then labeled according to a preset intent classification system, ultimately forming a domain-labeled dataset that meets the needs of model training.

3. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The process employs a hybrid precision quantization strategy to process the original large language model. Specifically, a large language model adapted to financial customer service scenarios and pre-trained on a financial business corpus is selected as the quantization object. The quantization process uses a hybrid precision strategy of FP16 and INT8. The core semantic understanding layer, which involves customer consultation semantic parsing, financial business terminology understanding, and customer core intent recognition, retains FP16 precision. The non-core computational layer, which is responsible for general text cleaning, sentence standardization, and redundant information filtering, uses INT8 precision for quantization compression.

4. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The structured pruning algorithm is used to prune the quantized model to obtain a lightweight model. Specifically, based on the mixed-precision quantized model, pre-inference is performed using an annotated financial customer service domain dataset. The gradient contribution of each parameter is calculated layer by layer through the gradient backpropagation mechanism. Based on the cumulative effect of the parameters in multi-round dialogue sample inference, a parameter importance ranking table containing information on parameter position, network layer, and contribution is generated. Parameters directly related to the extraction of financial business semantic features in the core semantic understanding layer and key parameters supporting the basic semantic processing logic in the non-core computing layer are locked. The structured pruning operation retains the locked parameters and removes parameters with low contribution and high redundancy.

5. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The process involves building a precision compensation model based on a lightweight model and fine-tuning it using labeled domain datasets. Specifically, the model is trained using high-frequency customer service dialogue data selected in the early stages, employing an optimizer and loss function. Training quality is ensured through validation set monitoring and early stopping mechanisms. The fine-tuned model maintains lightweight inference speed while restoring semantic understanding accuracy to a high level compared to the original model, thus forming a precision compensation model.

6. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The three-level caching architecture includes an in-memory cache (L1), a distributed cache (L2), and a database cache (L3). Specifically, it is designed based on the real-time requirements of data access frequency and storage cost differentiation. L1 is deployed in the core server's memory to achieve high-speed access to store high-frequency data. L2 uses a Redis cluster to ensure high availability and handle requests that are not hit by L1. L3 works in conjunction with the business database to preload low-frequency data to reduce direct queries. The three-level cache achieves hierarchical flow through a data synchronization protocol to ensure consistency, and each layer is configured with an independent fault isolation mechanism.

7. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The defined cached data classification standard specifically divides customer service related data into three categories: high-frequency fixed data, including basic business process descriptions, standard answers to common questions, and standardized service scripts; Mid-frequency dynamic data includes historical consultation records, business processing progress, and personalized service preferences; Low-frequency real-time data includes real-time financial data, temporary business announcements, and real-time data related to special business inquiries.

8. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The configured cache storage rules are as follows: L1 cache stores high-frequency fixed data, which automatically expires upon reaching its validity period and triggers a synchronous update with L2 cache. L2 cache, as the core relay layer, stores all high-frequency fixed data and all medium-frequency dynamic data. It improves concurrent access capabilities by balancing node load through distributed storage. L3 cache undertakes the functions of low-frequency real-time data storage and full data backup. It synchronizes the latest data from the business database according to the rules and only activates backup data to ensure access continuity when the upper-level cache fails or misses.

9. The customer service intelligent response method based on semantic understanding according to claim 1, characterized in that, The aforementioned cache refresh mechanism involves the system receiving service requests initiated by users from multiple channels in real time, standardizing the format of the request content, calling a lightweight natural language processing tool, and performing word segmentation, part-of-speech tagging, and entity recognition operations on the standardized request text based on a semantic dictionary for the financial customer service domain. Intent keywords and auxiliary information are extracted, and a structured information matrix containing user intent categories, core entity content, and key request elements is constructed. Core entities are associated with customer identity identifiers, and the information extraction operation is re-executed after clarifying the user's core needs.

10. A system utilizing the semantic understanding-based intelligent customer service response method according to any one of claims 1-9, characterized in that, include: The data preprocessing module is used to filter high-frequency customer service dialogue data in the database, and complete deduplication, desensitization and intent labeling. The model lightweight module is used to process the original large language model using a mixed-precision quantization strategy, prune the quantized model based on the structured pruning algorithm, and lock key parameters by calculating gradient contribution. The accuracy compensation module is used to build an accuracy compensation model based on the lightweight model and fine-tuned training using an annotated domain dataset. The cache management module is used to build a three-level cache architecture, define cache data classification standards, configure cache storage rules, and establish a cache refresh mechanism. The request inference module is used to process user requests and extract key information, and input the information into the lightweight model to complete the inference. The response generation module is used to populate the response template based on cached data and, after compliance verification, to return the response content to the user. The iterative optimization module is used to dynamically iteratively optimize the model and caching strategy.

Images