Insurance intelligent customer service method and device based on large language model

Abstract

The invention provides an insurance intelligent customer service method and device based on a large language model. The device comprises an insurance knowledge base module, a data access module, a data processing module, an insurance knowledge graph module, a multi-modal fusion module, a large language model module and a response generation module. By introducing an optimized GPT-NEOX large language model and fusing an InsuranceQA professional corpus and an InsurKG insurance knowledge graph, diversified insurance problems proposed by users can be dynamically understood, and natural, accurate and interpretable answers are generated. Compared with a traditional static answer extraction mode based on BERT and other models, the method has the remarkable advantages in the aspects of context understanding, open domain question and answer adaptability, deployment efficiency and professional domain knowledge coverage, and the response quality and user experience of insurance intelligent customer service are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer science and artificial intelligence, and in particular to an insurance intelligent customer service method and device based on a large language model. BACKGROUND

[0002] As a core supporting technology for the digital transformation of the insurance industry, the insurance intelligent customer service system is widely used in key business scenarios such as customer consultation, clause interpretation, and claim settlement services. With the development of natural language processing technology, related technologies have built a complete technical system covering data collection, semantic understanding, and decision generation through the collaborative work of the Transformer architecture, knowledge graph, and large language model. Specifically, this technology covers the entire process from user intent recognition, multi-source data fusion, to dynamic response generation, including rule-based question and answer engines, pre-training language models (such as BERT) for context encoding, and structured reasoning combined with knowledge graphs. Among them, traditional question and answer systems mostly use static answer extraction mechanisms, which encode text in both directions through pre-training models, and then generate answers by matching knowledge bases. Modern technologies gradually introduce autoregressive generation models to improve semantic processing capabilities in complex scenarios through multi-modal feature fusion.

[0003] However, in existing insurance intelligent customer service technical solutions, static answer extraction methods are directly used without fully considering the unique complexity of insurance clauses and dynamic generation requirements, which may lead to problems such as fragmented semantic understanding and insufficient answer generation flexibility. Specifically, although BERT-based question and answer systems achieve context encoding through Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) tasks, their bidirectional encoding characteristics limit the coherence of the generation process, and answer extraction relies on pre-set text fragments, which cannot handle scenarios that require dynamic interpretation such as "premium calculation logic explanation." At the same time, existing technologies usually use a single model architecture to handle multi-dimensional tasks, lacking deep integration of insurance knowledge graphs, which makes it difficult for the system to provide traceable decision-making in cases involving multi-line superimposed clauses, exceptions to liability conditions, and other long-distance dependency problems. This technical limitation not only affects the processing accuracy of complex insurance scenarios, but also reduces user trust in intelligent customer service due to the lack of transparency, thereby restricting the technology's landing in financial compliance scenarios. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art.

[0005] To this end, a first object of the present application is to propose an insurance intelligent customer service method based on a large language model.

[0006] Another object of the present application is to propose an insurance intelligent customer service device based on a large language model.

[0007] A third object of the present application is to propose a computer device.

[0008] A fourth object of the present application is to propose a non-transitory computer-readable storage medium.

[0009] To achieve the above object, the first aspect of the present application proposes an insurance intelligent customer service method based on a large language model, comprising: S1, obtaining insurance knowledge question data of a user, and performing denoising, standardization and feature extraction on the question data through a data processing module to generate a text vector that can be input into a large language model; S2, extracting a structured knowledge feature vector related to the text vector based on an insurance knowledge graph module, and using a multi-modal fusion module to dynamically fuse the text vector and the knowledge feature vector to generate a composite vector containing context semantics and knowledge association; S3, inputting the composite vector into an optimized GPT-NEOX large language model, and performing context-aware semantic reasoning through a self-recurrent decoder architecture to generate a dynamically scalable insurance problem solution vector; S4, constructing a natural language response in the response generation module according to the solution vector, and automatically labeling the cited insurance clauses or case sources in the response through a knowledge traceability mechanism.

[0010] In one embodiment of the present application, the S1 comprises: S11, using a BERT-based entity embedding method to encode key entities in insurance clauses to generate entity feature vectors; S12, enhancing the position information of the text vector through a rotation position embedding technology to improve the long-distance dependency modeling capability.

[0011] In one embodiment of the present application, the S2 comprises: S21, using a feature vector splicing technology to align the dimensions of the text vector and the knowledge graph feature vector; S22, performing weighted fusion on the spliced vector through a multi-head attention mechanism, and the calculation formula is:

[0012] wherein, , , query vector, key vector and value vector, respectively, is the dimension of the key vector.

[0013] In one embodiment of the present application, the S3 comprises: S31 employs a parallel computing optimization scheme to pipeline the computation processes of the attention layer and the feedforward layer in parallel. S32, through an improved parameter initialization scheme, uses an orthogonal initialization method to initialize the model weight matrix, the formula is:

[0014] in, This is the weight matrix. For input dimensions.

[0015] In one embodiment of the present invention, S4 includes: S41, Use the path backtracking algorithm to trace the entity relationships in the knowledge graph and generate a tracing tag containing the clause number and case ID; S42 uses dynamic template matching technology to associate source markers with natural language responses, ensuring that each answer can be traced back to a specific clause or case.

[0016] To achieve the above objectives, a second aspect of the present invention provides an intelligent insurance customer service device based on a large language model, comprising: The text preprocessing module is used to acquire users' insurance knowledge questions and generate text vectors that can be input into large language models through denoising, standardization, and feature extraction. The knowledge graph and multimodal fusion module is used to extract structured knowledge feature vectors related to the text vector, and dynamically fuse the text vector and knowledge feature vectors through multimodal fusion technology to generate a composite vector containing contextual semantics and knowledge associations. The model inference and vector generation module is used to input the composite vector into the optimized GPT-NEOX large language model, and perform context-aware semantic inference through an autoregressive decoder architecture to generate a dynamically scalable insurance question answer vector. The response construction and knowledge tracing module is used to construct a natural language response based on the answer vector, and automatically mark the source of the insurance clauses or cases cited in the response through a knowledge tracing mechanism.

[0017] The present invention discloses an intelligent insurance customer service method and device based on a large language model, which can dynamically generate personalized insurance questions and answers, improve contextual understanding and interpretability, and effectively improve the accuracy of the system's answers and user satisfaction in complex insurance scenarios.

[0018] To achieve the above objectives, a third aspect of this application provides a computer device comprising a processor and a memory; wherein the processor runs a program corresponding to the executable program code stored in the memory, for implementing an insurance intelligent customer service method based on a large language model as described in the first aspect embodiment.

[0019] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an insurance intelligent customer service method based on a large language model as described in the first aspect embodiment.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an intelligent insurance customer service method based on a large language model according to an embodiment of the present invention; Figure 2 This is a distributed measurement system architecture diagram according to an embodiment of the present invention; Figure 3 This is a structural diagram of an insurance intelligent customer service device based on a large language model according to an embodiment of the present invention; Figure 4 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] The following description, with reference to the accompanying drawings, describes an intelligent insurance customer service method and apparatus based on a large language model, according to an embodiment of the present invention.

[0025] Example 1 Figure 1 is a flowchart of an insurance intelligent customer service method based on a large language model according to an embodiment of the present invention, as Figure 1 shown, including: S1, obtaining the insurance knowledge question data of the user, and denoising, normalizing, and feature extracting the question data through a data processing module to generate a text vector that can be input into the large language model.

[0026] Specifically, for the step of "obtaining the insurance knowledge question data of the user, and denoising, normalizing, and feature extracting the question data through a data processing module to generate a text vector that can be input into the large language model" of the present invention, this step uses multi-stage natural language processing (NLP) technology to convert the original text input by the user into a structured vector representation suitable for processing by the large language model (LLM), thereby improving the understanding accuracy and generalization ability of the model for insurance domain problems.

[0027] Furthermore, the data processing module first docks with the data access module to receive user question data from multiple channels such as web pages and Apps. Subsequently, denoising processing is performed on the original question data, including removing non-semantic information such as HTML tags, special characters, duplicate spaces, and invalid punctuation marks to ensure the semantic integrity of the input text. Then, normalization processing is carried out, unifying the case, date format (such as `YYYY-MM-DD`), currency unit (such as "yuan", "ten thousand yuan", etc.), and using a Chinese word segmentation tool (such as jieba, HanLP) to segment the text and remove stop words (such as "de", "le", "shi", etc.) to reduce noise interference. Further, through stemming or lemmatization technology, words in different forms are normalized into a unified root form to improve semantic consistency.

[0028] Furthermore, an encoder based on the Transformer architecture (such as BERT, RoBERTa) is used to perform an embedding representation on the normalized text. The encoder maps each word to a high-dimensional vector (such as 768 dimensions or higher) and captures the context semantic relationship through the self-attention mechanism. The final output text vector form is where is the input sequence length, is the embedding dimension. This vector can be used as the input of the large language model for subsequent semantic reasoning and answer generation.

[0029] Furthermore, through structured processing and semantic vectorization, the understanding ability of the large language model for insurance domain problems is significantly improved, the sensitivity of the model to the input format is reduced, and the overall response quality and user satisfaction of the system are improved.

[0030] Furthermore, S1 includes: S11 uses a BERT-based entity embedding method to encode key entities in the insurance terms and generate entity feature vectors.

[0031] Specifically, in the technical solution of this invention, the key entities in the insurance terms are encoded using a BERT-based entity embedding method to generate entity feature vectors. This step uses a deep learning model to perform semantic modeling on entities in unstructured text, thereby providing high-quality semantic representation for subsequent multimodal fusion and large language model reasoning.

[0032] Furthermore, this step first preprocesses the insurance terms text using the bidirectional Transformer encoder of the BERT model. The BERT model is pre-trained through masked language modeling and next-sentence prediction tasks, enabling it to possess context-sensitive semantic representation capabilities. Specifically, after word segmentation, the input text is converted into a sequence of BERT tokens, and special markers (such as [CLS], [SEP]) are inserted into the sequence to distinguish different entities or sentences. The model output is a hidden state vector for each token, where the vectors of key entities are obtained by averaging or max-pooling their tokens to obtain entity-level feature vector representations.

[0033] Furthermore, BERT models typically use 768-dimensional hidden layer outputs as the dimension of entity feature vectors. In this invention, the length of the BERT input sequence is limited to 512 tokens to ensure the model's stability when processing long texts. Additionally, an entity boundary detection mechanism can be introduced during entity recognition, filtering high-confidence entity embeddings by setting confidence thresholds (e.g., 0.85) for the start and end positions of entities. This process can be combined with entity type labels (e.g., "insurance product," "claim conditions") for classification encoding, further improving the semantic discriminative power of the feature vectors.

[0034] Furthermore, this step is primarily used for entity modeling and retrieval within the insurance knowledge graph module. By encoding key entities in insurance terms into vectors, the system can semantically align unstructured text with entity nodes in the structured knowledge graph, thereby improving the retrieval efficiency and matching accuracy of the knowledge graph. For example, when a user asks, "What are the conditions for claim settlement under third-party liability insurance in car insurance?", the system can identify "third-party liability insurance" and "conditions for claim settlement" as key entities, and perform similarity matching in the knowledge graph using BERT embedding vectors to obtain structured information about the relevant entities.

[0035] Furthermore, by leveraging BERT's context-aware capabilities, entity feature vectors with semantic consistency and domain relevance can be generated, thereby enhancing the model's understanding of insurance terms and its reasoning ability. Simultaneously, this method provides a semantic bridge between structured and unstructured information for subsequent multimodal fusion, enhancing the system's interpretability and knowledge fusion capabilities. It is a crucial foundation for achieving a high-precision, highly robust intelligent insurance customer service system.

[0036] S12 enhances the positional information of text vectors through rotational position embedding technology, thereby improving the ability to model long-distance dependencies.

[0037] Specifically, this invention enhances the positional information of text vectors by introducing Rotation Position Embedding (RoPE), thereby improving the long-distance dependency modeling capability of large language models when handling intelligent customer service tasks in the insurance field. RoPE is a dynamic positional encoding method that embeds positional information into word vectors in a rotational manner, rather than adding positional information as an independent vector as in traditional positional encoding methods (such as absolute or relative positional encoding). By rotating positional information in the word embedding space, RoPE enables the model to more effectively capture semantic relationships between different positions when processing long sequences.

[0038] Specifically, RoPE is implemented based on block rotation of word vectors. Let the word vector dimension be... The model divides word vectors into several two-dimensional sub-blocks, each sub-block being represented as... ,in and For adjacent dimension indexes.

[0039] Furthermore, RoPE is integrated into the attention mechanism of the GPT-NEOX model to enhance the model's context-awareness when processing user queries and knowledge graph information. Particularly in the insurance field, user questions often involve combined reasoning of multiple terms, products, and processes. RoPE helps the model accurately capture the positional relationships of key information within long texts, improving the accuracy and interpretability of question answers.

[0040] Furthermore, the introduction of RoPE eliminates the need for additional model parameters; dynamic encoding of positional information can be achieved solely through a position-related rotation matrix. This improves model performance while reducing computational complexity and memory usage. This technology allows text vectors and knowledge graph feature vectors to maintain positional sensitivity during fusion, thereby enhancing the overall system's semantic understanding and reasoning capabilities.

[0041] S2, based on the insurance knowledge graph module, extract the structured knowledge feature vector related to the text vector, and use the multimodal fusion module to dynamically fuse the text vector and the knowledge feature vector to generate a composite vector containing contextual semantics and knowledge association.

[0042] Specifically, in some implementations, the multimodal fusion module of this invention dynamically fuses the user-input text vector with the structured knowledge feature vector extracted by the insurance knowledge graph module to generate a composite vector containing contextual semantics and knowledge associations, thereby improving the semantic understanding and reasoning capabilities of the large language model in insurance-related question-answering tasks. This step is technically implemented based on semantic alignment and attention mechanisms in the vector space to achieve deep fusion of text and knowledge.

[0043] Furthermore, text vectors are typically generated by an encoder in the data processing module, and their dimension is... This represents the dense representation of the input text in the semantic space. Meanwhile, the insurance knowledge graph module, based on the open-source knowledge graph InsurKG, extracts entity and relation features semantically related to the input text, forming a knowledge feature vector. Its dimensions are consistent with the text vectors to ensure the feasibility of subsequent fusion operations. The extraction process of knowledge feature vectors includes entity recognition, relation extraction, and graph embedding. Graph embedding can use knowledge graph representation learning methods such as TransE and RotatE to map entities and relations to a unified vector space.

[0044] Furthermore, this step is widely used in intelligent insurance customer service systems, especially when dealing with complex, multi-hop, or reasoning-based questions, such as explaining claims processes, comparing insurance terms, and providing risk assessment recommendations. By integrating text and knowledge graph information, the system can provide more accurate and explainable responses, improving user satisfaction and system credibility.

[0045] Furthermore, by introducing structured knowledge feature vectors, the shortcomings of large language models in terms of the depth and interpretability of private domain knowledge are compensated, and the model's ability to understand insurance domain professional terms and complex semantics is enhanced, thereby significantly improving the accuracy and response quality of the question answering system.

[0046] Furthermore, S2 includes: S21 uses feature vector concatenation technology to align the dimensions of the text vector with the feature vector of the knowledge graph.

[0047] Specifically, in the technical solution of the present invention, the step "using feature vector concatenation technology to dimensionally align text vectors with knowledge graph feature vectors" aims to effectively integrate semantic representations from different modalities, thereby improving the semantic understanding ability and interpretability of large language models in insurance question-answering tasks.

[0048] Further, this step first encodes the user-input text using a natural language processing module to generate a text vector. This text vector is typically a hidden layer representation of the model output based on the Transformer architecture, with a dimension of 6144, corresponding to the hidden layer size of the model. Simultaneously, the insurance knowledge graph module encodes entities and relationships in the knowledge graph using graph neural networks or graph embedding methods to generate a knowledge graph feature vector. The dimension of this feature vector is usually inconsistent with the text vector, therefore dimension alignment is required.

[0049] Furthermore, a feature vector concatenation technique is employed to concatenate the text vector and the knowledge graph feature vector along the feature dimension. In some implementations, if the dimension of the knowledge graph feature vector is... The vector can then be mapped to the same dimension 6144 as the text vector through a linear transformation (such as a fully connected layer), thus achieving dimension alignment. The concatenated vector is then used as the output of the multimodal fusion module and input into the large language model module for subsequent semantic reasoning and answer generation.

[0050] Furthermore, the concatenation operation is typically performed after the hidden layers of the model to ensure that the fused vector retains the original semantic information. Additionally, to improve the fusion effect, normalization layers or residual connections can be optionally introduced to stabilize the model training process. In practical applications, this step is widely used in insurance knowledge-based question-answering scenarios, such as when a user asks about "car insurance claims process" or "the coverage of health insurance." The system achieves more accurate semantic matching and answer generation by fusing the semantic vector of the user's question with the structured information in the knowledge graph.

[0051] Furthermore, by fusing textual semantics with structured knowledge graph information, the model's semantic understanding and reasoning accuracy in the insurance field are significantly improved. Simultaneously, the introduction of the knowledge graph enhances the system's interpretability, enabling the model's output to be traced back to specific entities and relationships within the knowledge graph, thereby increasing user trust and satisfaction with the system.

[0052] S22 uses a multi-head attention mechanism to weighted fuse the concatenated vectors, and the calculation formula is as follows:

[0053] in, , , These are the query vector, key vector, and value vector, respectively. The dimension of the key vector.

[0054] Specifically, in the implementation of this invention, the weighted fusion of the concatenated vectors through a multi-head attention mechanism is a key step in realizing multimodal information integration and context modeling. This step is technically implemented based on the self-attention mechanism in the Transformer architecture, specifically employing multi-head attention to fuse feature vectors from user input text and the insurance knowledge graph, thereby enhancing the model's ability to understand complex semantic relationships.

[0055] Furthermore, the multi-head attention mechanism captures semantic relationships across different dimensions by mapping the input vector into multiple different subspaces and calculating attention weights for each. In practice, the input vector is first linearly transformed into three matrices: query, key, and value, with each head performing attention calculations independently.

[0056] Furthermore, this step primarily involves fusing the semantic vector of the user's question with the structured information in the knowledge graph, thereby improving the accuracy and interpretability of the large language model in insurance-related question-answering tasks. Through a multi-head attention mechanism, the model can dynamically focus on the entities and relationships in the knowledge graph most relevant to the question, achieving context-aware semantic fusion.

[0057] Furthermore, by weighted fusion of feature vectors from different modalities, the model can more comprehensively understand the semantics of user questions and generate more logical and accurate answers by combining structured information from the knowledge graph. In addition, the introduction of a multi-head attention mechanism significantly improves the model's generalization ability, enabling it to maintain high response quality even when facing complex and ever-changing insurance-related problems.

[0058] S3. Input the composite vector into the optimized GPT-NEOX large language model, and perform context-aware semantic reasoning through an autoregressive decoder architecture to generate a dynamically scalable insurance question answer vector.

[0059] Specifically, inputting the composite vector into the optimized GPT-NEOX large language model and performing context-aware semantic reasoning through an autoregressive decoder architecture to generate a dynamically scalable insurance question answer vector is one of the core steps in achieving intelligent response generation in this invention. In some implementations, this step, based on the feature vector output by the multimodal fusion module, fuses the text vector of the user's question with the structured information vector in the knowledge graph to form a semantically rich composite vector, which serves as the input to the GPT-NEOX model.

[0060] Furthermore, the GPT-NEOX model employs an autoregressive decoder architecture, generating answers word-by-word based on the Transformer's self-attention mechanism and a feed-forward network. During training, the model maximizes conditional probabilities... To learn the distribution of generated answers, where This represents the composite vector of the input. Indicates the first One generated word, This represents the maximum length of the answer. During the decoding process, the model predicts the next word step by step based on the generated word sequence, achieving context-aware semantic reasoning.

[0061] The GPT-NEOX model used in this invention has 20 billion parameters, includes 44 Transformer decoder layers, each equipped with 72 attention heads, and a hidden layer dimension of 6144. In actual deployment, the model is trained on the InsuranceQA database through parameter fine-tuning to adapt to the professional semantics and question-answering patterns of the insurance domain. The composite vector dimension of the model input is... It is consistent with the hidden layer dimension of the model, ensuring seamless information transmission and efficient processing.

[0062] Furthermore, running on cloud servers or edge computing nodes, it supports high-concurrency user requests. By dynamically generating answers, the system can flexibly respond to changes in user question formats, such as grammatical differences and semantic ambiguities, thereby significantly improving answer accuracy and user satisfaction. In addition, by combining the structured information from the InsurKG knowledge graph, the model can provide interpretable reasoning paths while generating answers, enhancing the system's transparency and credibility.

[0063] Furthermore, S3 includes: S31 employs a parallel computing optimization scheme, which pipelines the computation processes of the attention layer and the feedforward layer in parallel.

[0064] Specifically, in some implementations, this invention employs a parallel computing optimization scheme, pipelined in parallel the computation processes of the attention layer and the feedforward layer to improve the inference efficiency and resource utilization of large language models in insurance intelligent customer service systems. This optimization scheme is based on the Transformer decoder architecture and is designed specifically for the structural characteristics of the GPT-NEOX model. Its core lies in dividing and scheduling the model's computation process across multiple computing devices (such as GPUs or TPUs), thereby achieving parallel execution of computational tasks.

[0065] Furthermore, this parallel computing scheme deploys the model's attention layer and feedforward layer on different computing nodes. Within each node, the attention layer calculates the attention weights between positions in the input sequence, capturing contextual information through a multi-head attention mechanism; the feedforward layer performs nonlinear transformations on the vector at each position, enhancing the model's expressive power. In pipelined parallelism, the input data is divided into multiple micro-batches, each of which sequentially passes through the computing units of the attention layer and the feedforward layer, achieving pipelined computation. This approach effectively reduces computational latency and improves throughput.

[0066] Furthermore, in this scheme, the GPT-NEOX model's attention layer contains 72 attention heads, the hidden layer dimension is 6144, and the total number of layers is 20. In parallel computing, each computing node is responsible for the computation of several layers, and the specific partitioning method can be adjusted according to hardware resources and load balancing strategies. For example, the attention layer and feedforward layer can be deployed alternately on different devices to achieve optimal scheduling of the computation pipeline. In addition, the model's parameter initialization and optimizer settings (such as AdamW) also need to be adapted to the parallel computing environment to ensure the stability and convergence of the training process.

[0067] Furthermore, this parallel computing optimization scheme is particularly suitable for scenarios requiring real-time responses to large-scale user inquiries in intelligent insurance customer service systems. In actual deployment, the system acquires user input through a data access module, converts it into a model-recognizable vector form through a data processing module, and then fuses it with knowledge graph feature vectors through a multimodal fusion module. The fused vector is then input into a pipelined parallel GPT-NEOX model for efficient reasoning and answer generation. This optimization method can significantly reduce response time and improve the overall system performance when handling high-concurrency requests.

[0068] Furthermore, by pipelined parallel processing of the attention layer and feedforward layer, this invention effectively alleviates the computational bottleneck of large language models when handling complex insurance question-and-answer tasks, improving the model's inference efficiency and resource utilization. This optimization scheme not only reduces the load on individual computing nodes but also improves the system's throughput through a parallel scheduling mechanism, thereby enabling faster and more stable intelligent customer service responses in practical applications.

[0069] S32, through an improved parameter initialization scheme, uses an orthogonal initialization method to initialize the model weight matrix, the formula is:

[0070] in, This is the weight matrix. For input dimensions.

[0071] Specifically, in the technical solution of the present invention, the step "initializing the model weight matrix using an orthogonal initialization method through an improved parameter initialization scheme" in some implementations adopts the orthogonal initialization method. By constructing an orthogonal matrix as the initial weight, the activation value of each layer maintains a stable variance during propagation, thereby alleviating the problem of gradient vanishing or exploding and improving the stability and convergence speed of the model in the early stage of training.

[0072] Furthermore, the mathematical principle of orthogonal initialization is based on the property of orthogonal matrices, that is, for any orthogonal matrix... It satisfies W^TW=I, where It is the identity matrix. In deep learning, the initialization of the weight matrix has a significant impact on the model training process. Traditional random initialization methods (such as Xavier or He initialization) can control the variance of activation values ​​to some extent, but they may still cause gradient propagation instability when dealing with high-dimensional data or deep network structures. Orthogonal initialization constructs a weight matrix that satisfies the orthogonality condition, so that the activation values ​​of each layer maintain an approximately constant distribution during propagation, thereby improving the training efficiency of the model.

[0073] Furthermore, orthogonal initialization is applied to the weight matrix initialization of the GPT-NEOX model, especially in the autoregressive Transformer decoder structure. This method effectively maintains the activation stability of the attention mechanism and the feedforward network. Specifically, during the model initialization phase, the weight matrix of each layer is orthogonalized, typically using QR decomposition or random orthogonalization methods to generate initial weights. For example, for a weight matrix of shape... weight matrix A random matrix can be generated first. Then, orthogonal matrices are obtained through QR decomposition. and take it as The initial value. In practice, a scaling factor may be optionally introduced. To control the magnitude of the initial weights, for example ,in Typically set to or This is to further optimize the distribution of activation values.

[0074] Furthermore, since the GPT-NEOX model needs to process a large amount of unstructured text and multimodal features fused from knowledge graphs during the initial training phase, orthogonal initialization can effectively improve the model's robustness to complex inputs and accelerate the convergence speed during the fine-tuning phase, thereby shortening the training cycle and improving model performance. In addition, this method can also enhance the model's adaptability to insurance domain terminology and complex semantic structures, providing a more stable semantic representation foundation for subsequent multimodal reasoning and response generation.

[0075] S4. Construct a natural language response in the response generation module based on the solution vector, and automatically mark the source of the insurance clauses or cases cited in the response through a knowledge tracing mechanism.

[0076] Specifically, in the response generation module, the system constructs a natural language response based on the answer vector output by the large language model module, and automatically marks the source of the insurance clauses or cases cited in the response through a knowledge tracing mechanism.

[0077] Furthermore, the response generation module first receives a high-dimensional semantic vector from the large language model module. This vector integrates the textual features of the user's question with structured information from the insurance knowledge graph through a multimodal fusion module. In some implementations, this vector dimension is [missing information]. The system generates natural language responses using an autoregressive approach. During generation, the model predicts the output sequence word by word based on a decoder structure, while also incorporating contextual information and entity relationships from the knowledge graph to ensure the semantic coherence and accuracy of the generated content. Furthermore, the system introduces a knowledge tracing mechanism, using entity recognition and vector matching techniques to retrieve insurance clauses or cases related to the current response content from the knowledge graph, and returns the matching results to the response generation module in a structured manner.

[0078] Furthermore, the knowledge tracing mechanism employs similarity-based retrieval algorithms, such as cosine similarity. ,in For the response semantic vector, This refers to the embedding vector of candidate terms or cases in the knowledge graph. The system sets a similarity threshold. This ensures high relevance of the cited content. Furthermore, the source tracing results are embedded in the response text as hyperlinks or citation numbers, conforming to the readability standards of natural language generation.

[0079] Furthermore, this step is widely used in insurance customer service systems, such as for user inquiries about claims processes, interpretation of insurance terms, or policy changes. When generating a response, the system automatically marks the source of the reference, such as "according to Article 3.2 of the Auto Insurance Claims Terms" or "Reference Case Number: 2023-INS-00456," thereby enhancing the authority and credibility of the response.

[0080] Furthermore, this not only improves the quality of the natural language responses but also enhances the system's interpretability and compliance. Through a knowledge traceability mechanism, users can track the basis of their responses, increasing their trust in the system's output. It also facilitates regulatory review and knowledge updates and maintenance, demonstrating significant practical value and industry-wide potential.

[0081] Furthermore, S4 includes: S41, use the path backtracking algorithm to trace the entity relationships in the knowledge graph and generate a tracing tag containing the clause number and case ID.

[0082] Specifically, in the technical solution of the present invention, a path backtracking algorithm is used to trace the entity relationships in the knowledge graph. The core of this step is to accurately match the user's question with the entities, relationships and corresponding knowledge sources (such as clause number and case ID) in the knowledge graph through path tracing in the graph structure, thereby generating tag information with traceability capability.

[0083] Furthermore, the path backtracking algorithm is based on a graph traversal mechanism, typically employing breadth-first search (BFS) or depth-first search (DFS) strategies, combined with a semantic matching model to reason about entity relationships in the knowledge graph. Specifically, the system first parses the user-input question into a semantic vector representation using a natural language processing (NLP) module, and then performs entity recognition and relationship extraction in the knowledge graph module. The identified entities serve as the starting point for graph traversal, and the system expands the path through relation edges to find the knowledge nodes most semantically relevant to the question.

[0084] Furthermore, path backtracking algorithms typically set a maximum search depth (e.g., ) and maximum number of paths (e.g.) This helps control computational complexity and ensure response efficiency. Furthermore, the system can incorporate entity embedding models (such as TransE and RotatE) to vectorize entity relationships, improving the accuracy of path matching.

[0085] Furthermore, it is particularly suitable for scenarios in the insurance field involving complex clause references and case reasoning. For example, when a user asks "What materials are needed for car insurance claims?", the system can trace back to the knowledge graph and identify paths related to entities such as "car insurance," "claims process," and "materials list," marking the corresponding clause number and case ID, thereby enhancing the authority and traceability of the answer.

[0086] Furthermore, the path backtracking algorithm not only improves the system's efficiency in utilizing the knowledge graph but also significantly enhances the interpretability of the model's output. By binding answers to specific terms and cases, users can trace the source of knowledge, increasing their trust in the system's output and facilitating subsequent auditing and compliance verification. This step serves as a crucial bridge connecting the large language model and the structured knowledge base in this invention, and is an important guarantee for achieving accurate and transparent service in the intelligent customer service system.

[0087] S42 uses dynamic template matching technology to associate source markers with natural language responses, ensuring that each answer can be traced back to a specific clause or case.

[0088] Specifically, the step in this invention of "associating the source marker with the natural language response through dynamic template matching technology to ensure that each answer can be traced back to a specific clause or case" is technically implemented based on a multimodal feature fusion and semantic alignment mechanism, combined with the reasoning capabilities of insurance knowledge graphs and large language models to achieve source tracing of the generated answer.

[0089] Furthermore, this step first standardizes the natural language question input by the user through a data processing module, including word segmentation, stop word removal, part-of-speech tagging, and syntactic analysis, in order to extract the semantic vector representation of the question. ,in This is for embedding dimensions. Simultaneously, clause or case information in the insurance knowledge graph module is encoded into structured feature vectors. ,in Indicates the first Each knowledge unit (such as insurance clause number, case number, etc.).

[0090] Furthermore, dynamic template matching technology, based on this, employs semantic similarity calculation methods (such as cosine similarity, dot product, etc.) to analyze the problem vector. With knowledge vectors Perform matching and calculate matching scores. The top-scoring knowledge units will be used as the basis for answer generation, and key information in the answer will be linked to specific clauses or cases through a source-tracing and tagging mechanism (such as entity linking and relation extraction).

[0091] Furthermore, this step introduces a source tag embedding mechanism in the response generation module. When generating the answer, the matched clause or case number is embedded into the answer in a structured manner, such as inserting `[Clause Number: K00123]` or `[Case Number: C20230501]` into the answer, thereby achieving a direct association between the answer and the knowledge source. This mechanism can be implemented based on template generation or post-processing to ensure the traceability of the output answer.

[0092] Furthermore, this step is widely applicable to scenarios such as policy consultation, claims basis inquiry, and clause interpretation in insurance customer service systems. For example, when a user asks, "What materials are needed for car insurance claims?", the system not only generates a natural language response but also automatically links to Article 12 of the "Motor Vehicle Insurance Terms and Conditions" and embeds the clause number in the answer, making it easy for the user to find the original basis.

[0093] The technical benefits of this step lie in significantly improving the explainability and compliance of the intelligent customer service system. By linking answers to specific clauses or cases, the system can provide legally grounded responses, enhancing user trust and facilitating regulatory review and system auditing. Furthermore, this mechanism can be used for continuous model optimization, guiding knowledge base updates and model retraining by analyzing frequently cited clauses.

[0094] The insurance intelligent customer service dynamic response method of this invention can dynamically generate natural and fluent insurance customer service responses, improve the ability to understand and answer complex and open domain questions, and enhance the interpretability and user satisfaction of the system.

[0095] Example 2 This invention also proposes an intelligent insurance customer service system based on a large language model, such as... Figure 2 As shown, it includes: an insurance knowledge base module, a data access module, a data processing module, an insurance knowledge graph module, a multimodal fusion module, a large language model module, and a response generation module.

[0096] Specifically, in this embodiment, the InsuranceQA database is used. It is the first open-source QA corpus in the insurance field, developed in 2016 by the Department of Computer Science at Carnegie Mellon University and the Insurance Services Office, Inc. It covers a wide range of topics, involving various types of insurance products and services, which gives the generated model higher generalization ability. It contains over 16,000 questions and answer pairs, covering all aspects of insurance, such as auto insurance, health insurance, and life insurance.

[0097] The data access module acquires users' insurance knowledge question data and transmits it to the data processing module and the insurance knowledge graph module. The data processing module receives and processes the user's insurance knowledge question data, obtaining question text vectors that can be recognized by the language model. These question text vectors are then transmitted to the multimodal fusion module to obtain question-and-answer results based on a large language model. The insurance knowledge graph module uses InsurKG, an open-source knowledge graph proposed in 2020 by Fudan University and Ant Group. InsurKG contains over 100,000 entities and over 1 million relations, making it one of the largest knowledge graphs in the insurance field. Data sources include insurance company product brochures, insurance terms, insurance contracts, and insurance-related documents and news reports from the internet, ensuring high data quality. The multimodal fusion module fuses the question text vectors from the data processing module and the knowledge graph feature vectors from the insurance knowledge graph module, transmitting the fused vectors to the large language model module to obtain question-and-answer results based on the large language model.

[0098] Furthermore, the large language model employs an optimized GPT-NEOX model. GPT-NeoX is an autoregressive Transformer decoder model whose architecture largely follows that of GPT-3, but with some significant deviations. This model has 20 billion parameters, 44 layers, 64 attention heads, and a hidden layer dimension of 6144. The main differences from GPT-3 are the change in the tokenizer, the addition of rotational position embeddings, the parallel computation of attention layers and feedforward layers, and different initialization schemes and hyperparameters. Based on this, the large language model reduces the hidden layers to 20 and increases the number of attention heads to 72, resulting in the modified model.

[0099] Furthermore, the pre-trained large language model is trained using the InsuranceQA database. After parameter fine-tuning, an initial large language model is obtained. The fused feature vectors are fed into the large language model for multimodal inference, and the inference results are passed to the response generation module.

[0100] Furthermore, the response generation module converts the reasoning results of the large language model into answers in natural language form and presents them to the user.

[0101] This invention utilizes a large language model combined with a knowledge graph to provide intelligent insurance customer service. The large language model, represented by the GPT-NEOX model, can understand human questions and provide answers in a human-like manner. It can accurately understand users' insurance knowledge questions even when the user changes the question's syntax and provide corresponding answers. However, large language models suffer from insufficient depth and interpretability regarding private knowledge such as insurance information. Therefore, this embodiment loads the insurance-related database InsuranceQA and the knowledge graph InsurKG onto the large language model. InsurKG can retrieve knowledge data based on question-answer vectors, significantly improving the retrieval rate of knowledge data. This data is then matched with the large language model to respond to user questions, demonstrating a high level of understanding and answering ability. The response process is more intelligent, with better human-like effects, and it performs well even with varied question formats, greatly improving user satisfaction and user experience. Furthermore, by combining the large language model with the InsurKG knowledge graph, the large language model can provide more interpretable output, helping users better understand the model's decision-making process and enhancing user trust.

[0102] Example 3 To achieve the above embodiments, such as Figure 3 As shown, this embodiment also provides an insurance intelligent customer service device 10 based on a large language model, including: The text preprocessing module 100 is used to acquire users' insurance knowledge question data and generate text vectors that can be input into a large language model through denoising, standardization and feature extraction. The knowledge graph and multimodal fusion module 200 is used to extract structured knowledge feature vectors related to the text vector, and dynamically fuse the text vector and knowledge feature vectors through multimodal fusion technology to generate a composite vector containing contextual semantics and knowledge associations. The model inference and vector generation module 300 is used to input the composite vector into the optimized GPT-NEOX large language model, and perform context-aware semantic inference through an autoregressive decoder architecture to generate a dynamically scalable insurance question answer vector. The response construction and knowledge tracing module 400 is used to construct a natural language response based on the answer vector and automatically mark the source of the insurance clauses or cases cited in the response through a knowledge tracing mechanism.

[0103] Furthermore, the text preprocessing module 100 is also used for: The key entities in the insurance terms are encoded using a BERT-based entity embedding method to generate entity feature vectors; By using rotational position embedding technology to enhance the positional information of text vectors, the ability to model long-distance dependencies is improved.

[0104] Furthermore, the knowledge graph and multimodal fusion module 200 is also used for: The text vectors and knowledge graph feature vectors are aligned in dimensions using feature vector concatenation techniques. The concatenated vectors are weighted and fused using a multi-head attention mechanism, calculated as follows:

[0105] in, , , These are the query vector, key vector, and value vector, respectively. The dimension of the key vector.

[0106] An intelligent insurance customer service device based on a large language model, according to an embodiment of the present invention, can dynamically generate personalized insurance Q&A, improve contextual understanding and interpretability, and effectively improve the accuracy of the system's answers and user satisfaction in complex insurance scenarios.

[0107] Example 4 To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 4 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads the executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the insurance intelligent customer service method based on a large language model described above.

[0108] Example 5 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an insurance intelligent customer service method based on a large language model as described in the foregoing embodiments.

[0109] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0110] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0111] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for intelligent insurance customer service based on a large language model, characterized in that, include: S1, acquire users' insurance knowledge question data, and perform noise reduction, standardization and feature extraction on the question data through the data processing module to generate text vectors that can be input into large language models; S2, based on the insurance knowledge graph module, extract the structured knowledge feature vector related to the text vector, and use the multimodal fusion module to dynamically fuse the text vector and the knowledge feature vector to generate a composite vector containing contextual semantics and knowledge association; S3, input the composite vector into the optimized GPT-NEOX large language model, and perform context-aware semantic reasoning through an autoregressive decoder architecture to generate a dynamically scalable insurance question answer vector; S4. Construct a natural language response in the response generation module based on the solution vector, and automatically mark the source of the insurance clauses or cases cited in the response through a knowledge tracing mechanism.

2. The method as described in claim 1, characterized in that, The process of acquiring users' insurance knowledge question data and performing noise reduction, standardization, and feature extraction on the question data through a data processing module to generate text vectors that can be input into a large language model further includes: S11, The BERT-based entity embedding method is used to encode the key entities in the insurance terms and generate entity feature vectors; S12 enhances the positional information of text vectors through rotational position embedding technology, thereby improving the ability to model long-distance dependencies.

3. The method as described in claim 1, characterized in that, The step of extracting structured knowledge feature vectors related to the text vector based on the insurance knowledge graph module, and dynamically fusing the text vector and knowledge feature vectors using a multimodal fusion module to generate a composite vector containing contextual semantics and knowledge associations, further includes: S21, use feature vector concatenation technology to align the dimensions of text vectors with knowledge graph feature vectors; S22 uses a multi-head attention mechanism to weighted fuse the concatenated vectors, and the calculation formula is as follows: in, , , These are the query vector, key vector, and value vector, respectively. The dimension of the key vector.

4. The method as described in claim 1, characterized in that, The process of inputting the composite vector into the optimized GPT-NEOX large language model and performing context-aware semantic reasoning through an autoregressive decoder architecture to generate dynamically scalable insurance question answer vectors further includes: S31 employs a parallel computing optimization scheme to pipeline the computation processes of the attention layer and the feedforward layer in parallel. S32, through an improved parameter initialization scheme, uses an orthogonal initialization method to initialize the model weight matrix, the formula is: in, This is the weight matrix. For input dimensions.

5. The method as described in claim 1, characterized in that, The step of constructing a natural language response in the response generation module based on the solution vector, and automatically annotating the source of the cited insurance clauses or cases in the response through a knowledge tracing mechanism, also includes: S41, Use the path backtracking algorithm to trace the entity relationships in the knowledge graph and generate a tracing tag containing the clause number and case ID; S42 uses dynamic template matching technology to associate source markers with natural language responses, ensuring that each answer can be traced back to a specific clause or case.

6. An intelligent insurance customer service device based on a large language model, characterized in that, include: The text preprocessing module is used to acquire users' insurance knowledge questions and generate text vectors that can be input into large language models through denoising, standardization, and feature extraction. The knowledge graph and multimodal fusion module is used to extract structured knowledge feature vectors related to the text vector, and dynamically fuse the text vector and knowledge feature vectors through multimodal fusion technology to generate a composite vector containing contextual semantics and knowledge associations. The model inference and vector generation module is used to input the composite vector into the optimized GPT-NEOX large language model, and perform context-aware semantic inference through an autoregressive decoder architecture to generate a dynamically scalable insurance question answer vector. The response construction and knowledge tracing module is used to construct a natural language response based on the answer vector, and automatically mark the source of the insurance clauses or cases cited in the response through a knowledge tracing mechanism.

7. The apparatus as claimed in claim 6, characterized in that, The text preprocessing module is also used for: The key entities in the insurance terms are encoded using a BERT-based entity embedding method to generate entity feature vectors; By using rotational position embedding technology to enhance the positional information of text vectors, the ability to model long-distance dependencies is improved.

8. The apparatus as claimed in claim 6, characterized in that, The knowledge graph and multimodal fusion module is also used for: The text vector is dimensionally aligned with the knowledge graph feature vector using feature vector concatenation techniques. The concatenated vectors are weighted and fused using a multi-head attention mechanism, calculated as follows: in, , , These are the query vector, key vector, and value vector, respectively. The dimension of the key vector.

9. A computer device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement an insurance intelligent customer service method based on a large language model as described in any one of claims 1-5.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements an insurance intelligent customer service method based on a large language model as described in any one of claims 1-5.

Images