A construction method and device for an intelligent question-answering system based on a lightweight large model

Abstract

The present application provides a method and apparatus for constructing an intelligent question-answering system based on a lightweight large model. Among them, the received natural language question is parsed to obtain the user's query intention; the pre-constructed knowledge graph is located to obtain a set of knowledge nodes related to the natural language question; the internal parameter configuration of the lightweight large model is dynamically adjusted to generate an optimized lightweight large model; answer generation processing is performed to obtain an answer text corresponding to the natural language question; for the answer text that does not meet the preset quality standard, a secondary query process is triggered, or the user is prompted to modify the question description. The technical solution provided by the present application can improve the response speed, accuracy and flexibility of the question-answering system to adapt to different knowledge fields.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of lightweight large models, and in particular, to a method and device for constructing an intelligent question-answering system based on a lightweight large model. Background Art

[0002] With the rapid development of information technology, intelligent question-answering systems have been widely used in multiple fields such as customer service, medical consultation, and online education. Users expect to interact with the system through natural language and obtain immediate and accurate answers. Especially in professional fields, accurately understanding the user's query intention and the knowledge field to which the question belongs is crucial for providing high-quality services.

[0003] Most current intelligent question-answering systems rely on large pre-trained language models. Although these models perform well in a wide range of natural language processing tasks, they have limitations in specific domain applications. On the one hand, the deployment of large models requires high computing resources, which restricts their use in edge devices or resource-constrained environments; on the other hand, traditional question-answering systems usually use fixed rule sets or simple keyword matching to find answers, and this method is difficult to capture complex semantic information. Especially when facing polysemous words or complex sentence patterns, it is easy to misinterpret the user's actual query intention.

[0004] Existing intelligent question-answering solutions mainly face the following challenges: First, large pre-trained models are huge in size and high in deployment cost, which is not conducive to use in mobile devices or other scenarios with limited computing resources; second, methods based on rules or keyword matching have insufficient context understanding ability and cannot fully meet the user's needs for complex queries; finally, there is a lack of an effective feedback mechanism. When the generated answer does not meet expectations, the system cannot effectively self-adjust or optimize, affecting the user experience and service quality. In addition, there is still room for improvement in ensuring the quality and relevance of answers in existing systems, especially in the case where a satisfactory answer cannot be generated initially, there is a lack of an effective secondary query or user guidance mechanism. Summary of the Invention

[0005] The embodiments of the present application provide a method and device for constructing an intelligent question-answering system based on a lightweight large model, so as to solve the problems of slow response speed, low accuracy, and low flexibility in adapting to different knowledge fields in the prior art.

[0006] Receive real-time data streams from different sources, where the real-time data streams include structured data and unstructured data;

[0007] In a first aspect, the embodiments of the present application provide a method for constructing an intelligent question-answering system based on a lightweight large model, including:

[0008] Parse and process the received natural language question to obtain the user's query intention and the knowledge domain to which the natural language question belongs;

[0009] According to the query intention, use the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning to perform positioning processing on the pre-constructed knowledge graph, and obtain a set of knowledge nodes highly relevant to the natural language question;

[0010] Based on the set of knowledge nodes, activate the adaptive context understanding mechanism in the lightweight large model, and adopt a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to dynamically adjust the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model;

[0011] Use the optimized lightweight large model, and combine it with the method of providing diverse expression suggestions by the adversarial generation network, and cooperate with the sequence-to-sequence architecture and the beam search decoding strategy to perform answer generation processing to obtain the answer text corresponding to the natural language question;

[0012] Perform quality inspection processing on the answer text. For the answer text that does not meet the preset quality standard, trigger a secondary query process or prompt the user to modify the question description to ensure that the answer text meets the preset quality standard.

[0013] Optionally, the method of using the optimized lightweight large model, combining the diverse expression suggestions provided by the adversarial generation network, and cooperating with the sequence-to-sequence architecture and the beam search decoding strategy to perform answer generation processing to obtain the answer text corresponding to the natural language question includes:

[0014] Use the optimized lightweight large model, and combine it with the method of providing diverse expression suggestions by the adversarial generation network to perform semantic understanding and expression enrichment processing on the natural language question to obtain candidate answer texts;

[0015] Based on the sequence-to-sequence architecture, cooperate with the optimized lightweight large model to perform encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework;

[0016] Apply the beam search decoding strategy to optimize the preliminary answer text framework, and explore the most likely answer sequence by maintaining multiple possible answer paths to generate the answer text corresponding to the natural language question. Optionally, the method of based on the sequence-to-sequence architecture, cooperating with the optimized lightweight large model, and performing encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework includes:

[0017] Encode the candidate answer text according to the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture, and convert the candidate answer text into a context vector with a fixed length, where the context vector is used to capture the core semantic information of the candidate answer;

[0018] Based on the context vector, through the decoder part in the sequence-to-sequence architecture, gradually construct the answer sequence, predict the probability distribution of the next word at each time step, and use the probability distribution to select appropriate words according to the set strategy and add them to the answer sequence to obtain an updated answer sequence;

[0019] Based on the updated answer sequence, update the context vector and continue to select target words according to the set strategy and add them to the answer sequence until the preset maximum sentence length is reached or the end symbol is encountered to generate a preliminary answer text framework.

[0020] Among them, the process of selecting target words according to the set strategy includes selecting the word with the highest probability distribution as the target word at each time step, or selecting the top k highest probability distributions from the predicted multiple probability distributions and randomly selecting any one of them as the target word.

[0021] Optionally, apply the beam search decoding strategy to optimize the preliminary answer text framework, and explore the most likely answer sequence by maintaining multiple possible answer paths to generate the answer text corresponding to the natural language question, including:

[0022] Use the set fixed beam width to initialize the preliminary answer text framework to obtain an initial set of candidate answer paths;

[0023] Based on the initial set of candidate answer paths, expand and maintain multiple possible answer paths at each time step to obtain a gradually expanding set of answer paths;

[0024] Introduce a length normalization and diversity penalty mechanism to adjust the gradually expanding set of answer paths to obtain an optimized answer path, where the adjustment process is used to prevent shorter sentences from being preferentially selected due to a higher cumulative probability distribution and to generate more diverse answers;

[0025] Use the optimized answer path to explore the most likely answer sequence, select the answer sequence with the highest cumulative probability distribution, and generate the answer text corresponding to the natural language question.

[0026] Optionally, it is characterized in that the initial candidate answer path set is obtained by initializing the preliminary answer text framework using a set fixed beam width, including:

[0027] Set the fixed beam width to k, and the fixed beam width is used to determine the number of candidate answer paths to be retained;

[0028] Select the top k highest-probability words from the probability distribution of the predicted words in the decoder part of the sequence-to-sequence architecture as the starting words to form k initial candidate answer paths, and construct an initial candidate answer path set;

[0029] Based on the initial candidate answer path set, expand and maintain multiple possible answer paths at each time step to obtain a gradually expanding answer path set, including:

[0030] For each time step, according to all candidate answer paths generated in the previous time step, for each candidate answer path, use the current context vector to predict the probability distribution of the next word, and obtain the probability distribution of possible subsequent words for each candidate answer path;

[0031] Based on the subsequent word probability distribution, select the top k highest-probability words from the newly generated words expanded from each candidate answer path, and combine them with the corresponding candidate answer path to form an optimized candidate answer path;

[0032] From all the optimized candidate answer paths, select the top k optimized candidate answer paths with the highest cumulative probability to continue expanding, ensuring that only the most likely candidate answer paths are retained each time until the preset maximum sentence length is reached or the end symbol is encountered, to obtain a gradually expanding answer path set.

[0033] Optionally, based on the knowledge node set, activate the adaptive context understanding mechanism in the lightweight large model, and adopt a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to dynamically adjust the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model, including:

[0034] Perform a conversion process on the relevance score of the knowledge node set to obtain a knowledge node vector representation that can capture the knowledge node set;

[0035] According to the knowledge node vector representation, adopt a progressive parameter fine-tuning scheme to perform the dynamic adjustment process on the parameters in the lightweight large model to obtain a preliminary optimized model;

[0036] The attention mechanism is introduced to enhance the preliminary optimization model, and based on the memory network, the preliminary optimization model is further optimized to ensure that the preliminary optimization model can retrieve and utilize the knowledge node set when generating the answer text, thereby obtaining the optimized lightweight large model.

[0037] Optionally, the dynamically adjusting the parameters in the lightweight large model according to the knowledge node vector representation by adopting a progressive parameter fine-tuning scheme to obtain a preliminary optimization model includes:

[0038] Using the knowledge node vector representation as input, through a phased evaluation mechanism, performing an initial state analysis on the parameters of the lightweight large model to obtain a state snapshot of the parameters of the lightweight large model;

[0039] According to the state snapshot, determining the parameters in the lightweight large model that need to be adjusted, and setting a fine-tuning step size and direction for each parameter in the lightweight large model that needs to be adjusted to generate a progressive fine-tuning plan;

[0040] Based on the progressive fine-tuning plan, performing small-amplitude and multi-round iterative update processing on the parameters in the lightweight large model that need to be adjusted, and evaluating the performance change of the lightweight large model after each round of iterative update processing to obtain an optimized intermediate model;

[0041] Introducing an adaptive learning rate adjustment mechanism, automatically adjusting the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model to ensure the balance between the speed and accuracy of the parameter adjustment of the optimized intermediate model, and obtaining an adjustment result;

[0042] By accumulating the adjustment results, performing a screening process on all the optimized intermediate models, and finally selecting the optimized intermediate model with the most stable performance as the preliminary optimization model.

[0043] In a second aspect, an embodiment of the present application provides an intelligent question-answering system based on a lightweight large model, including:

[0044] A receiving module, configured to parse the received natural language question to obtain the user's query intention and the knowledge domain to which the natural language question belongs;

[0045] A positioning module, configured to, according to the query intention, use the adaptive algorithm corresponding to the knowledge domain and a selection mechanism based on deep reinforcement learning to perform a positioning process on a pre-constructed knowledge graph to obtain a knowledge node set highly relevant to the natural language question;

[0046] An adjustment module, configured to activate the adaptive context understanding mechanism in the lightweight large model based on the knowledge node set, and perform dynamic adjustment processing on the internal parameter configuration of the lightweight large model by adopting a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network, so as to generate an optimized lightweight large model;

[0047] A generation module, configured to use the optimized lightweight large model, and in combination with a method of providing diversified expression suggestions by an adversarial generation network, cooperate with a sequence-to-sequence architecture and a beam search decoding strategy to perform answer generation processing, so as to obtain an answer text corresponding to the natural language question;

[0048] An inspection module, configured to perform quality inspection processing on the answer text, and for an answer text that does not meet the preset quality standard, trigger a secondary query process, or prompt the user to modify the question description, so as to ensure that the answer text meets the preset quality standard.

[0049] In a third aspect, an embodiment of the present application provides a computing device, which includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a method for constructing an intelligent question-answering system based on a lightweight large model as described in any item of the first aspect.

[0050] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program, and when the computer program is executed by a computer, it implements a method for constructing an intelligent question-answering system based on a lightweight large model as described in any item of the first aspect.

[0051] In an embodiment of the present application, the received natural language question is parsed to obtain the user's query intention and the knowledge field to which the natural language question belongs; according to the query intention, the corresponding adaptive algorithm and a selection mechanism based on deep reinforcement learning in the knowledge field are used to perform positioning processing on a pre-constructed knowledge graph to obtain a knowledge node set highly relevant to the natural language question; based on the knowledge node set, the adaptive context understanding mechanism in the lightweight large model is activated, and a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network are adopted to perform dynamic adjustment processing on the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model; the optimized lightweight large model is used, and in combination with a method of providing diversified expression suggestions by an adversarial generation network, cooperate with a sequence-to-sequence architecture and a beam search decoding strategy to perform answer generation processing to obtain an answer text corresponding to the natural language question; quality inspection processing is performed on the answer text, and for an answer text that does not meet the preset quality standard, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standard.

[0052] The technical solution of this application has the following beneficial effects:

[0053] Parse and process the received natural language question to obtain the user's query intention and the knowledge domain to which the natural language question belongs; according to the query intention, use the knowledge domain adaptation algorithm and the selection mechanism based on deep reinforcement learning to perform positioning processing on the pre-constructed knowledge graph to obtain a set of knowledge nodes highly relevant to the natural language; based on the set of knowledge nodes, activate the adaptive context understanding mechanism in the lightweight large model, and use the progressive parameter fine-tuning scheme, attention mechanism, and memory network to perform dynamic adjustment processing on the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model; use the optimized lightweight large model, and combine the method of providing diversified expression suggestions by the generative adversarial network, cooperate with the sequence-to-sequence architecture and beam search decoding strategy to perform answer generation processing to obtain the answer text corresponding to the natural language question; perform quality inspection processing on the answer text, and for the answer text that does not meet the preset quality standard, trigger a secondary query process, or prompt the user to modify the question description to ensure that the answer text meets the preset quality standard.

[0054] Furthermore, by combining the optimized lightweight large model and the diversified expression suggestions provided by the generative adversarial network, this method can perform in-depth semantic understanding and expression enrichment processing on natural language questions, thereby generating more accurate and diversified candidate answer texts. Using the combination of the sequence-to-sequence architecture and the beam search decoding strategy not only improves the flexibility and accuracy of answer generation but also ensures the quality and relevance of the answers. This method can provide more natural and readable answers while maintaining high precision.

[0055] Furthermore, by converting the candidate answer text into a fixed-length context vector, the core semantic information of the candidate answer can be effectively captured, providing a solid foundation for subsequent answer generation. Based on the context vector, gradually predict and select appropriate words to add to the answer sequence. This process allows the system to make optimal choices at each time step, ensuring that the generated answer is both grammatically correct and logically coherent. As the answer sequence is continuously updated, the context vector is also adjusted accordingly, ensuring the timeliness and accuracy of information during the answer generation process. This dynamic mechanism enables the system to flexibly adjust the subsequent answer direction based on the generated part of the content, enhancing the system's adaptability.

[0056] Furthermore, the beam search decoding strategy is used to maintain multiple possible answer paths, enabling the system to explore the most likely answer sequence among numerous possibilities, greatly increasing the probability of finding the best answer. By introducing length normalization and diversity penalty mechanisms, it prevents shorter sentences from being preferentially selected due to higher cumulative probabilities and promotes the generation of more diverse responses, ensuring that the final answer is not only the most probable but also the most in line with user needs. Selecting the answer sequence with the highest cumulative probability distribution from the optimized answer paths ensures that the generated answer text is not only optimal in a statistical sense but also provides the best user experience in practical applications.

[0057] Furthermore, a fixed beam width k is set to determine the number of candidate answer paths to be retained. By selecting the top k most probable words as the starting words, it ensures that in the initial stage, the system can focus on the most promising answer paths, improving efficiency. At each time step, multiple possible answer paths are expanded and maintained. By continuing to expand by selecting the top k optimized candidate answer paths with the highest cumulative probabilities, it guarantees that each iteration can focus on the most likely answer paths, reducing unnecessary computational overhead. This process continues until the preset maximum sentence length is reached or the end symbol is encountered. This setting defines the boundary conditions for answer generation, avoiding infinite loops or overly long responses while ensuring the integrity and rationality of the answers.

[0058] These methods or other aspects of the present application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] To more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0060] Figure 1 It is a flowchart of a method for constructing an intelligent question-answering system based on a lightweight large model provided by an embodiment of the present application;

[0061] Figure 2 It is a schematic structural diagram of a big data processing system for implementing hybrid data analysis provided by an embodiment of the present application;

[0062] Figure 3 It is a schematic structural diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] To enable those skilled in the art to better understand the solution of this application, the technical solution in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application.

[0064] In some processes described in the specification, claims and above-mentioned accompanying drawings of this application, a plurality of operations that appear in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish each different operation, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0065] The technical solution in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0066] Figure 1 The flowchart of a method for constructing an intelligent question-answering system based on a lightweight large model is provided for the embodiments of this application. As Figure 1 shown, the method includes:

[0067] 101. Parse and process the received natural language question to obtain the user's query intention and the knowledge domain to which the natural language question belongs;

[0068] In this step, natural language processing (NLP) technology: refers to a series of technologies that enable a computer to understand and generate human natural language. This includes, but is not limited to, word segmentation, part-of-speech tagging, named entity recognition (NER), syntactic analysis, and semantic analysis, etc.

[0069] Query intention: The specific purpose or requirement implied in the question or request raised by the user.

[0070] Knowledge domain: The professional field or subject range to which the question belongs.

[0071] This step utilizes natural language processing (NLP) technologies, such as word segmentation, part-of-speech tagging, named entity recognition, etc., as well as domain classification algorithms, to understand and classify the user's question. Through these technologies, the system can more accurately understand the actual needs of the user and locate the question to a specific knowledge domain, providing guidance for subsequent steps.

[0072] In the embodiments of the present application, in the field of intelligent customer service AI Q&A, when receiving a question input by a user, the system will first use a pre-trained language model to parse the text and extract key information. For example, for a question about "how to set the mobile phone alarm", the system will identify that this is a query related to "mobile device operation" and further refine it to the specific "alarm setting" task. Next, the system will select the most appropriate answer strategy or knowledge base according to the parsing result.

[0073] For example, assume the user asks "how to set the alarm on an iPhone?" The system will first parse this question and identify keywords such as "iPhone" and "set the alarm". Then, it will determine that this is a query about "mobile device operation" and specifically about the alarm function of the iOS system. This information is used to guide subsequent knowledge graph search and answer generation.

[0074] 102. According to the query intention, use the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning to perform positioning processing on the pre-constructed knowledge graph, and obtain a set of knowledge nodes highly relevant to the natural language question;

[0075] In this step, the adaptive algorithm: a technology that adjusts algorithm parameters or structures according to the requirements of specific tasks to improve performance or efficiency.

[0076] The selection mechanism of deep reinforcement learning: a machine learning method that guides the model to make optimal choices through reward feedback.

[0077] Knowledge Graph: A graph database where nodes represent entities or concepts, and edges express the relationships between them.

[0078] In step 102, the system, according to the query intention obtained in the first step, applies the adaptive algorithm and the selection mechanism based on deep reinforcement learning to find a set of highly relevant knowledge nodes from the pre-constructed knowledge graph. Here, the adaptive algorithm refers to an algorithm that can adjust its behavior according to different query types, while the deep reinforcement learning mechanism helps the system make optimal decisions among a large number of possible choices, ensuring that the information found is both accurate and comprehensive.

[0079] In the embodiments of the present application, in the intelligent customer service AI Q&A scenario, once the query intention of the user and the field to which the question belongs are clarified, the system will use the deep reinforcement learning algorithm to explore the nodes in the knowledge graph and find the parts that best match the user's question. In this process, the system not only considers directly relevant information but also mines potentially related content to provide users with richer and more detailed answers.

[0080] For example, continuing with the previous example, the system already knows that what the user wants to know is "how to set an alarm on an iPhone". So, it starts searching for all the nodes in the knowledge graph related to the iOS system, the alarm application, and its setting options. With the help of the deep reinforcement learning algorithm, the system is able to pick out the best path to reach the target information, that is, the specific steps and tips for setting the alarm.

[0081] 103. Based on the set of the knowledge nodes, activate the adaptive context understanding mechanism in the lightweight large model, and adopt a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to perform dynamic adjustment processing on the internal parameter configuration of the lightweight large model, and generate an optimized lightweight large model;

[0082] In this step, the lightweight large model: a language model that has been pre-trained and is relatively small but still maintains high performance.

[0083] Adaptive context understanding mechanism: allows the model to dynamically adjust its internal configuration according to the history and current environment of the conversation, so as to better understand the context information.

[0084] Progressive parameter fine-tuning scheme: a method of continuously updating the model parameters, aiming to gradually improve the performance of the model as new data arrives.

[0085] Attention mechanism: enables the model to focus on certain parts of the input sequence while ignoring other unimportant content.

[0086] Memory network: enhances the memory function of the model, enabling it to remember the information of previous interactions.

[0087] In step 103, the lightweight large model is activated to perform adaptive context understanding. This involves dynamically adjusting the internal parameters of the model to better adapt to the current conversation context. Progressive parameter fine-tuning allows the model to gradually optimize its performance; the attention mechanism helps the model focus on the most important part of the data; the memory network enables the model to remember the history of previous interactions, thereby enhancing the understanding ability of consecutive conversations.

[0088] In the embodiments of the present application, for an intelligent customer service, activating the adaptive context understanding means enabling the model to have stronger context awareness ability. For example, when discussing complex issues with a user, the model needs to understand the historical background of the conversation while paying attention to the latest question content. In this way, the model can give more coherent and targeted answers, improving the user experience.

[0089] For example, continuing with the previous example, during the process of answering "How to set an alarm on an iPhone?", if the user then asks "Then how do I turn it off?", the system will activate the adaptive context understanding mechanism, remember the previously mentioned alarm setting topic, and directly provide the method to turn off the alarm without the user having to describe it in detail again.

[0090] 104. Use the optimized lightweight large model, combined with the diverse expression suggestions provided by the generative adversarial network, cooperate with the sequence-to-sequence architecture and the beam search decoding strategy to perform answer generation processing to obtain the answer text corresponding to the natural language question;

[0091] In this step, the generative adversarial network (GAN): helps create more diverse and natural answers.

[0092] The sequence-to-sequence architecture (Seq2Seq): is used to convert one sequence into another sequence and is widely used in fields such as machine translation and dialogue systems.

[0093] The beam search decoding strategy: retains multiple possible answer paths during the generation process and finally selects the answer with the highest quality.

[0094] In step 104, the optimized lightweight large model combines the diverse expression suggestions provided by the generative adversarial network, and through the sequence-to-sequence architecture and the beam search decoding strategy, generates the final answer text. The generative adversarial network helps create diverse expressions, making the answer more natural and fluent; while the sequence-to-sequence architecture and beam search ensure that the answer structure is reasonable and logically clear.

[0095] In the embodiments of the present application, when the intelligent customer service AI prepares an answer, it will use the above technical combination to construct an accurate and easy-to-read answer. For example, for the question "How to set an alarm on an iPhone?", the system may generate multiple expressions and select the most suitable one to present to the user to ensure the effectiveness of information transmission. For example, continuing with the previous example, when the system is ready to answer the question "How to set an alarm on an iPhone?", it will comprehensively consider all available resources, including but not limited to the relevant information collected previously, and then tell the user in the most appropriate way: "To set an alarm on your iPhone, open the 'Clock' app, click on the 'Alarm' tab at the bottom, and then click the '+' sign in the upper right corner to add a new alarm."

[0096] 105. Perform quality inspection processing on the answer text. For the answer text that does not meet the preset quality standard, trigger a secondary query process or prompt the user to modify the question description to ensure that the answer text meets the preset quality standard;

[0097] In this step, quality inspection: Review the generated answers to ensure their accuracy, completeness, and applicability.

[0098] Secondary query process: When it is found that the answer does not meet the preset standards, trigger the process of re-querying.

[0099] User prompt to modify the question: Guide the user to provide more information or a more specific problem description in order to obtain a more satisfactory answer.

[0100] Step 105 involves quality inspection of the generated answer text to ensure that they meet the predetermined standards. Answers that do not meet the standards will trigger the secondary query process or suggest that the user re-describe the problem, thereby ensuring that the information provided is always reliable and useful. This link is the key to maintaining service quality and user satisfaction.

[0101] In the embodiments of the present application, in the intelligent customer service AI question and answer, quality inspection is an essential step. It not only guarantees the accuracy of the information but also improves the user experience. If the initially generated answer is not ideal, the system will try to improve it or guide the user to provide more information until a satisfactory result is obtained.

[0102] Suppose the answer initially provided by the system fails to fully solve the user's question, such as only explaining the method of setting the alarm but ignoring how to set the repeat days. At this time, the system will automatically start the secondary query process to supplement the missing information or remind the user to describe their needs more specifically, for example, by asking if they also need to know other setting options. In this way, ensure that the final answer provided is complete and meets the user's expectations.

[0103] In summary, five key steps from natural language question parsing to answer generation and quality inspection are analyzed in detail. These steps together constitute a complete question and answer processing flow, aiming to ensure that the entire process from receiving user input to finally outputting a high-quality answer is efficient and accurate.

[0104] Optionally, the method described in step 104 of using the optimized lightweight large model and combining the diverse expression suggestions provided by the generative adversarial network, in cooperation with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain the answer text corresponding to the natural language question, includes:

[0105] Using the optimized lightweight large model and combining with the diverse expression suggestions provided by the generative adversarial network, perform semantic understanding and expression enrichment processing on the natural language question to obtain candidate answer texts; based on the sequence-to-sequence architecture and in cooperation with the optimized lightweight large model, perform encoding-decoding conversion processing on the candidate answer texts to generate a preliminary answer text framework; apply the beam search decoding strategy to optimize the preliminary answer text framework, and explore the most likely answer sequence by maintaining multiple possible answer paths to generate the answer text corresponding to the natural language question.

[0106] The core of step 104 is to use the optimized lightweight large model, combine with the diverse expression suggestions provided by the generative adversarial network (GAN), and generate the answer to the natural language question through the sequence-to-sequence architecture and the beam search decoding strategy. Here are several key concepts:

[0107] Lightweight large model: It refers to a deep learning model optimized by techniques such as compression or pruning, aiming to reduce computational resource consumption while maintaining high performance. It is used to understand the semantics of the input question and generate a preliminary answer text based on this.

[0108] Generative adversarial network (GAN): It consists of a generator and a discriminator. The generator attempts to create realistic data samples, while the discriminator evaluates the authenticity of these samples. In this context, GAN is used to provide diverse expression suggestions to make the generated answers more natural and more variable.

[0109] Sequence-to-sequence architecture (Seq2Seq): A deep learning framework for processing sequence data, commonly used in tasks such as translation and question answering. This architecture includes two parts: an encoder and a decoder. The former converts the input sequence into a fixed-length vector representation, and the latter generates the output sequence based on this vector.

[0110] Beam search decoding strategy: A method that maintains multiple candidate answer paths during the decoding process and finally selects the most likely answer sequence. Compared with the greedy algorithm, this method can explore more possibilities and improve the quality of the generated text.

[0111] In the embodiment of this application, in the field of intelligent customer service AI question answering, when it is necessary to generate an answer to the user's question, the system first uses the optimized lightweight large model to perform in-depth semantic analysis on the input question. Then, in order to make the generated answer more natural and diverse, the system introduces the expression suggestions provided by the generative adversarial network. These suggestions are used to enrich the content of the candidate answer text, making it not only accurate but also having different expression ways.

[0112] Subsequently, based on the sequence-to-sequence architecture, the system encodes the candidate answer text into an intermediate representation and then decodes it into a preliminary answer text framework. This stage is completed by an optimized lightweight large model, ensuring that the answer is both grammatically correct and logically coherent.

[0113] Finally, the beam search decoding strategy is applied to further optimize the preliminary answer text framework. This strategy improves the likelihood of finding the optimal answer by simultaneously tracking multiple potential answer paths. Beam search allows the system to explore multiple possible answer combinations and select the best one to present to the user as the final answer.

[0114] The following is a scenario example:

[0115] The user asks, "How to cancel the alarm on iPhone?" First, the system uses the lightweight large model to parse this question and determines that its intention is about canceling the set alarm. Then, the adversarial generation network provides several different but correct ways of answering, such as: "You can cancel the alarm on your iPhone by following these steps" or "To cancel the alarm on iPhone, please follow these instructions."

[0116] Next, the system adopts the sequence-to-sequence architecture to convert the above candidate answers into a structured text framework. For example, it may construct a framework like this: "[Start] To cancel the alarm on iPhone, please [specific steps][End]".

[0117] Finally, the beam search decoding strategy comes into play and selects the most suitable answer from multiple possible answer paths. For example, "To cancel the alarm on iPhone, open the 'Clock' app, select the 'Alarm' tab, find the alarm you want to cancel, and then swipe to delete." This multi-step processing method ensures that the answer is both detailed and easy to understand, while also demonstrating the flexibility and adaptability of the system.

[0118] In this way, the intelligent customer service can not only quickly respond to the user's needs but also provide high-quality and diverse answers, greatly enhancing the user experience.

[0119] Among them, the sequence-to-sequence architecture, in cooperation with the optimized lightweight large model, performs encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework, including:

[0120] Encode the candidate answer text according to the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture, convert the candidate answer text into a context vector of a fixed length, and the context vector is used to capture the core semantic information of the candidate answer; based on the context vector, through the decoder part in the sequence-to-sequence architecture, gradually construct an answer sequence, predict the probability distribution of the next word at each time step, and use the probability distribution to select appropriate words to add to the answer sequence according to the set strategy to obtain an updated answer sequence; based on the updated answer sequence, update the context vector and continue to select target words to add to the answer sequence according to the set strategy until the preset maximum sentence length is reached or an end symbol is encountered to generate a preliminary answer text framework. Among them, the process of selecting target words according to the set strategy includes selecting the word with the highest probability distribution as the target word at each time step, or selecting the top k highest probability distributions from the predicted multiple probability distributions and randomly selecting any one of them as the target word.

[0121] In an intelligent question-answering system, the encoding-decoding conversion process based on the sequence-to-sequence architecture (Seq2Seq) is a process of converting the input text into a context vector of a fixed length and using this vector to construct an answer. The encoder part is responsible for receiving the candidate answer text and compressing its core semantic information into a context vector, which serves as a bridge from the input to the output. The decoder part then uses the context vector to gradually construct an answer sequence, predicting the probability distribution of the next word each time and selecting appropriate words to add to the answer sequence according to the set strategy.

[0122] Among them, applying the beam search decoding strategy to optimize the preliminary answer text framework, exploring the most likely answer sequence by maintaining multiple possible answer paths to generate the answer text corresponding to the natural language question, includes:

[0123] Use the set fixed beam width to initialize the preliminary answer text framework to obtain an initial set of candidate answer paths; based on the initial set of candidate answer paths, expand and maintain multiple possible answer paths at each time step to obtain a gradually expanding set of answer paths; introduce a length normalization and diversity penalty mechanism to adjust the gradually expanding set of answer paths to obtain an optimized answer path, where the adjustment process is used to prevent shorter sentences from being preferentially selected due to a higher cumulative probability distribution and to generate more diverse answers; use the optimized answer path to explore the most likely answer sequence, select the answer sequence with the highest cumulative probability distribution, and generate the answer text corresponding to the natural language question.

[0124] Beam Search is a method for optimizing the decoding process. Instead of simply choosing the word with the highest probability at each step, it maintains multiple possible answer paths (i.e., "beams") to explore a wider possibility space and finally selects the most likely answer sequence. This method can significantly improve the quality and diversity of the generated answers. By introducing length normalization and diversity penalty mechanisms, it can prevent shorter sentences from being preferentially selected due to high cumulative probabilities and encourage the generation of more diverse responses.

[0125] Optionally, initialize the preliminary answer text framework using a set fixed beam width to obtain an initial set of candidate answer paths, including: setting a fixed beam width k, which is used to determine the number of candidate answer paths to be retained; selecting the top k highest-probability words from the probability distribution of the predicted vocabulary in the decoder part of the sequence-to-sequence architecture as starting words to form k initial candidate answer paths and construct an initial set of candidate answer paths; based on the initial set of candidate answer paths, expand and maintain multiple possible answer paths at each time step to obtain a gradually expanding set of answer paths, including: for each time step, according to all candidate answer paths generated in the previous time step, for each candidate answer path, use the current context vector to predict the probability distribution of the next word to obtain the probability distribution of possible subsequent words for each candidate answer path; based on the subsequent word probability distribution, select the top k highest-probability words from the newly expanded words of each candidate answer path and combine them with the corresponding candidate answer path to form an optimized candidate answer path; from all optimized candidate answer paths, select the top k optimized candidate answer paths with the highest cumulative probability to continue expanding, ensuring that only the most likely candidate answer paths are retained each time until the preset maximum sentence length is reached or the end symbol is encountered to obtain a gradually expanding set of answer paths.

[0126] In this step, use an optimized lightweight large model and combine with diverse expression suggestions provided by the Generative Adversarial Network (GAN) to perform semantic understanding and expression enrichment processing on natural language questions to obtain candidate answer texts. Then, based on the sequence-to-sequence architecture and in cooperation with the optimized lightweight large model, perform encoding-decoding conversion processing on the candidate answer texts to generate a preliminary answer text framework. Finally, apply the beam search decoding strategy to explore the most likely answer sequence by maintaining multiple possible answer paths to generate the answer text corresponding to the final natural language question. The technologies involved in this process include but are not limited to semantic understanding, context vector construction, vocabulary probability distribution prediction, length normalization, and diversity penalty mechanisms, etc., which work together to ensure that the generated answers are both accurate and have good expression diversity.

[0127] In the embodiments of the present application, the system first encodes the candidate answer text using an optimized lightweight large model and the encoder part of the sequence-to-sequence architecture. This includes converting the candidate answer text into a fixed-length context vector that can capture its core semantic information. This context vector serves as the basis for subsequent decoding to ensure that the generated answer faithfully reflects the intention of the original question.

[0128] Secondly, based on the generated context vector, the system gradually constructs the answer sequence through the decoder part. At each time step, the system predicts the probability distribution of the next word and selects the target word to add to the answer sequence according to the set strategy. This process may repeatedly update the context vector to adapt to the new answer content until the preset maximum sentence length is reached or the end symbol is encountered.

[0129] Furthermore, to optimize the preliminary answer text framework, the system adopts a beam search decoding strategy, maintaining multiple possible answer paths to explore the most likely answer sequence. First, a fixed beam width (e.g., k) is set, and an initial set of candidate answer paths is obtained after initialization processing. Then, these paths are expanded and maintained at each time step, and at the same time, a length normalization and diversity penalty mechanism is applied to adjust the path set to ensure that the finally generated answer is both reasonable and diverse.

[0130] Finally, among all the candidate answer paths, the system will select the answer sequence with the highest cumulative probability distribution as the final answer. This process not only considers the probability of word selection but also comprehensively considers the sentence length and diversity to ensure that the generated answer is both accurate and natural.

[0131] The following is a scenario example:

[0132] Suppose the user asks: "How to set an alarm on an iPhone?" After parsing and knowledge graph positioning in the previous steps, the system enters the answer generation stage:

[0133] First, the system uses an optimized lightweight large model and the encoder of the sequence-to-sequence architecture to encode the relevant information about "iPhone alarm setting" into a context vector, which centrally reflects the core semantics of this question - that is, how to operate the alarm function on the iPhone device.

[0134] Next, the decoder starts to work and gradually constructs the answer sequence based on the context vector. For example, the first word may be "Open" because this is the starting point of most operation guides. The system will continue to predict subsequent words, such as "Clock app", "Tap", "+", etc., to gradually form a complete answer.

[0135] Furthermore, to ensure the quality and diversity of answers, the system adopts a beam search decoding strategy to maintain multiple possible answer paths. For example, for the selection after "open", the system may consider both the "Clock app" and the "Alarm app" simultaneously. As the conversation progresses, the system continuously evaluates and retains the most likely path until a complete answer is generated.

[0136] Finally, when the preset maximum sentence length is reached or an end symbol is encountered, the system selects the path with the highest cumulative probability as the final answer, such as: "To set an alarm on your iPhone, open the 'Clock' app, tap the 'Alarm' tab at the bottom, and then tap the '+' sign in the upper right corner to add a new alarm."

[0137] In this way, the intelligent customer service AI not only provides direct and effective answers, but also ensures the diversity and naturalness of the answers due to the adoption of the beam search strategy, enhancing the user experience.

[0138] Optionally, in step 103, based on the set of knowledge nodes, the adaptive context understanding mechanism in the lightweight large model is activated, and a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network are used to dynamically adjust the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model, including:

[0139] Perform a conversion process on the relevance scores of the set of knowledge nodes to obtain a knowledge node vector representation that can capture the set of knowledge nodes; according to the knowledge node vector representation, use a progressive parameter fine-tuning scheme to perform the dynamic adjustment process on the parameters in the lightweight large model to obtain a preliminary optimized model; introduce the attention mechanism to enhance the preliminary optimized model, and further optimize the preliminary optimized model based on the memory network to ensure that the preliminary optimized model can retrieve and utilize the set of knowledge nodes when generating answer texts, obtaining the optimized lightweight large model.

[0140] In this alternative solution, the system first performs a relevance scoring on the set of knowledge nodes located from the knowledge graph. These scores reflect the degree of association of each node with the user's question and are data calculated based on multiple factors (such as semantic similarity, context relevance, etc.). Next, the system converts these scores into a vector representation that can capture the relationships between knowledge nodes, namely the knowledge node vector representation. This vectorization process enables the machine learning model to more effectively understand and utilize this information.

[0141] Optionally, the step of using a progressive parameter fine-tuning scheme to perform the dynamic adjustment process on the parameters in the lightweight large model according to the knowledge node vector representation to obtain a preliminary optimized model includes:

[0142] Using the representation of the knowledge node vectors as input, through a phased evaluation mechanism, perform an initial state analysis on the parameters of the lightweight large model to obtain a state snapshot of the parameters of the lightweight large model; according to the state snapshot, determine the parameters in the lightweight large model that need to be adjusted, and set a fine-tuning step size and direction for each parameter in the lightweight large model that needs to be adjusted to generate a progressive fine-tuning plan; based on the progressive fine-tuning plan, perform small-amplitude and multi-round iterative update processing on the parameters in the lightweight large model that need to be adjusted, and evaluate the performance change of the lightweight large model after each round of iterative update processing to obtain an optimized intermediate model; introduce an adaptive learning rate adjustment mechanism, and automatically adjust the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model to ensure the balance between the speed and accuracy of the parameter adjustment of the optimized intermediate model to obtain an adjustment result; by accumulating the adjustment results, perform a screening process on all the optimized intermediate models, and finally select the optimized intermediate model with the most stable performance as the preliminary optimized model.

[0143] This process involves gradually optimizing the parameters in the lightweight large model to better adapt to a specific task or dataset. Through a phased evaluation mechanism, the system can analyze the current state of the model and adjust specific parameters as needed. The fine-tuning step size and direction determine the specific way of each adjustment, while the adaptive learning rate adjustment mechanism ensures the balance between the speed and accuracy of parameter adjustment, thus achieving an efficient and stable optimization process.

[0144] In the embodiments of this application, in the field of intelligent customer service AI question answering, in order to enable the lightweight large model to generate answer texts more accurately, we introduce the above optional solution. Specifically:

[0145] First, by converting the correlation scores of the knowledge node set, we obtain a vector representation that can reflect the relationships between the nodes. This step is crucial for subsequent model optimization because it provides important clues about the knowledge structure.

[0146] Next, using the generated representation of the knowledge node vectors as input, the system performs an initial state analysis and records a state snapshot of the model parameters. Based on this snapshot, determine which parameters need to be adjusted and set a fine-tuning plan for each parameter. This plan includes specific fine-tuning step sizes and directions, guiding subsequent small-amplitude and multi-round iterative updates.

[0147] Finally, based on the preliminary optimization, the system further introduces the attention mechanism to enhance the performance of the model. Meanwhile, combined with the function of the memory network, the model can retrieve and utilize the previous set of knowledge nodes when generating answers. Finally, after a series of performance evaluations and screenings, the most stable and best-performing model is selected as the preliminary optimization model.

[0148] The following is an example scenario:

[0149] Suppose in an intelligent customer service scenario, the user asks, "How to set a repeating alarm on an iPhone?" The system will first recognize that this is a query related to the "iOS alarm function" and find the corresponding set of knowledge nodes through the deep reinforcement learning algorithm. Then, the system will score the relevance of these nodes and convert the scoring results into vector representations.

[0150] Next, the system starts progressive parameter fine-tuning of the lightweight large model. It first analyzes the state of the existing model, determines which parameters are most critical for answering this question, and formulates a detailed fine-tuning plan for these parameters. For example, if it is found that certain parameters are closely related to the logic of "repeating alarm setting", then these parameters may be given priority for adjustment.

[0151] In the following multi-round iterative update process, the system continuously evaluates the changes in model performance to ensure that each adjustment brings positive effects. In addition, the adaptive learning rate adjustment mechanism ensures that the speed of parameter adjustment is neither too fast nor too slow, but just right to promote the improvement of the model.

[0152] As the model is gradually optimized, the system also adds the attention mechanism to help the model focus more on the parts directly related to "repeating alarm setting"; and with the support of the memory network, the model can review the previous interaction history when generating answers to provide a more coherent response. Finally, after a series of optimization steps, the system successfully generates an optimized lightweight large model that can accurately answer the user's question: "To set a repeating alarm on your iPhone, open the 'Clock' app, tap the 'Alarm' tab at the bottom, select the alarm you want to edit, then turn on the 'Repeat' option and choose the days you want the alarm to sound."

[0153] This embodiment not only demonstrates how the system improves its performance through knowledge node vector representation and progressive parameter fine-tuning, but also reflects the flexibility and powerful capabilities of the intelligent customer service AI answering system in practical applications.

[0154] In the optimization process of deep learning models, especially for lightweight large models in specific domains, how to efficiently adjust model parameters to adapt to new tasks or datasets is a key issue. The traditional batch gradient descent method may not fully consider the complex relationships and changing trends among model parameters. Therefore, a progressive parameter fine-tuning scheme is introduced, which combines advanced mechanisms such as state snapshots, Hessian matrices, and adaptive learning rates, aiming to achieve more refined parameter adjustment and thus improve model performance.

[0155] Optionally, the dynamic adjustment process of the parameters in the lightweight large model is performed by adopting a progressive parameter fine-tuning scheme according to the knowledge node vector representation, and a preliminary optimized model is obtained, including:

[0156] Using the knowledge node vector representation as input, through a phased evaluation mechanism, an initial state analysis process is performed on the lightweight large model parameters to obtain a state snapshot of the lightweight large model parameters; wherein, the state snapshot of the lightweight large model parameters is calculated by the following formula:

[0157] ;

[0158] wherein, is the state snapshot of the lightweight large model parameters, are the lightweight large model parameters, is the knowledge node vector representation, is the loss function the gradient of the parameter with respect to, is the Hessian matrix, which is used to capture second-order derivative information;

[0159] According to the state snapshot, determine the set of parameters in the lightweight large model that need to be adjusted with emphasis and set a fine-tuning step size and direction for each parameter in the lightweight large model that needs to be adjusted with emphasis, and generate a progressive fine-tuning plan; wherein, the set of parameters, fine-tuning step size, and direction in the lightweight large model are calculated by the following formula:

[0160] ;

[0161] ;

[0162] ;

[0163] wherein, is the set of parameters in the lightweight large model, is the fine-tuning step size of the th parameter, is the direction, is the loss function, Represents a single parameter in the lightweight large model, is the initial learning rate, is the historical gradient weight, is the historical gradient weighting coefficient for the th iteration, is the influence factor of the Hessian matrix, used to enhance the gradient direction, is the loss function with respect to the parameter partial derivative, measuring the impact of the parameter on the loss, is the reciprocal of the diagonal element of the Hessian matrix, used to adjust the importance of the gradient, is the direction of the dot product of the Hessian matrix and the gradient, used to enhance the gradient direction, The loss function with respect to the th parameter cumulative historical gradient information from the 1st iteration to the th iteration;

[0164] Based on the progressive fine-tuning plan, small-scale and multi-round iterative update processing is performed on the parameters in the lightweight large model that need to be adjusted with emphasis. After each round of update, the performance change of the lightweight large model is evaluated to obtain an optimized intermediate model; among them, the optimized intermediate model is calculated by the following formula:

[0165] ;

[0166] where, represents the optimized intermediate model, represents the model parameters of the previous iteration;

[0167] In the iterative update process, an adaptive learning rate adjustment mechanism is introduced. According to the performance feedback of the optimized intermediate model, the learning rate of the progressive fine-tuning plan is automatically adjusted to ensure the balance between the speed and accuracy of the parameter adjustment of the optimized intermediate model, and a fine adjustment result is obtained; among them, the learning rate of the progressive fine-tuning plan is calculated by the following formula:

[0168] ;

[0169] where, is the learning rate of the progressive fine-tuning plan, is the initial learning rate, is the learning rate adjustment factor based on the performance of the validation set, is the time decay coefficient, is the gradient variance influence factor, which is used to dynamically adjust the learning rate according to the gradient variance; is the gradient variance, which measures the change of the gradient;

[0170] By accumulating the adjustment results, screening processing is performed on all the optimized intermediate models, and finally the preliminary optimized model is obtained; wherein, the preliminary optimized model is calculated by the following formula:

[0171] ;

[0172] wherein, is the preliminary optimized model, is the Kullback-Leibler divergence, which is used to measure the difference between the parameter distributions of the new and old models, is the regularization coefficient, which controls the model complexity.

[0173] The purpose of the overall formula is to guide the dynamic adjustment process of the parameters of the lightweight large model through a series of mathematical operations, specifically including: State snapshot: Capture the state information of the current model parameters (including the first-order derivative and the second-order derivative) to better understand the change trend of the parameters. Key parameter selection: Identify the set of parameters that have a greater impact on the model output, and set reasonable fine-tuning step sizes and directions to ensure the effectiveness of the adjustment. Multiple-round iterative update: Based on the selected key parameters, perform small-scale and multi-round iterative updates, and at the same time monitor the change of the model performance to ensure the stability and accuracy of the adjustment process. Adaptive learning rate adjustment: Automatically adjust the learning rate according to the model performance feedback to balance the adjustment speed and accuracy. Screen the optimal intermediate model: Evaluate all the optimized intermediate models to finally determine the preliminary optimized model with the best performance and stability.

[0174] The following briefly explains the design reasons for each item in the formula:

[0175] : represents the parameters of the lightweight large model. This is the core part of the model, which determines the behavior and output of the model;

[0176] : loss function The gradient of the parameter , which reflects the influence degree of the parameter on the loss. By calculating the gradient, it can be known how to adjust the parameters to reduce the loss and thus improve the model performance;

[0177] : Hessian matrix, used to capture second derivative information. It provides additional information about the trend of parameter changes, helps understand the interrelationships between parameters, and can be used in some cases to accelerate convergence or avoid local minima;

[0178] Set of key parameters : Select the set of parameters that most need to be adjusted, determined based on the maximum value of the product of the partial derivative and the inverse diagonal element of the Hessian matrix. This method ensures that resources are concentrated on the parameters that have the greatest impact on the model performance;

[0179] Fine-tuning step size : The fine-tuning step size formula comprehensively considers the current gradient, historical gradient, and the influence factor of the Hessian matrix. The initial learning rate Controls the step size of each update; is the historical gradient weight, allowing the model to remember past information; is the influence factor of the Hessian matrix, enhancing the gradient direction to ensure the correct adjustment direction;

[0180] Direction : The direction is determined by the sign of the partial derivative, indicating whether the parameter should be increased or decreased to reduce the loss. This ensures that each adjustment is made in the direction of reducing the loss;

[0181] Intermediate model : Each update is based on the result of the previous round plus the newly calculated step size and direction, gradually approaching the optimal solution;

[0182] Adaptive learning rate : The learning rate gradually decays over time, but is also affected by the gradient variance. The coefficient controls the adjustment strength of the gradient variance on the learning rate, ensuring that the learning process is both fast and stable, is the learning rate adjustment factor based on the performance of the validation set, dynamically adjusting the learning rate according to the model performance to prevent overfitting or underfitting.

[0183] Preliminary optimized model : Select the optimal model by minimizing the weighted sum of the loss variance and the Kullback-Leibler divergence. The KL divergence is used to measure the difference between the parameter distributions of the new and old models, preventing overfitting, is the regularization coefficient, controlling the model complexity to ensure that the model not only performs well on the training data but also generalizes to unseen data..

[0184] The acquisition methods of the parameters in the following formulas in the intelligent Q&A spare:

[0185] : The parameters of the lightweight large model can be obtained by loading the pre-trained model and then fine-tuned for the specific task of intelligent question answering;

[0186] : Loss function For the parameters The gradient is usually calculated using the cross-entropy loss function and the backpropagation algorithm in intelligent question answering;

[0187] : Hessian matrix, which is often approximated by its diagonal elements in practice, or other efficient methods such as the finite difference method are used;

[0188] Set of key parameters : By analyzing the product of the partial derivative and the inverse diagonal element of the Hessian matrix, select the set of parameters that have the greatest impact on the model output;

[0189] Fine-tuning step size : Initial learning rate It can be set according to empirical values, while , Coefficients such as etc. need to be adjusted according to experimental results;

[0190] Direction : The direction is determined by the sign of the partial derivative, indicating whether the parameter should be increased or decreased to reduce the loss;

[0191] Intermediate model : Each update is based on the result of the previous round plus the newly calculated step size and direction;

[0192] Adaptive learning rate : The learning rate gradually decays over time, but is also affected by the gradient variance. The coefficient controls the adjustment strength of the gradient variance on the learning rate to ensure that the learning process is both fast and stable;

[0193] Preliminary optimized model : Select the optimal model by minimizing the weighted sum of the loss variance and the Kullback-Leibler divergence. The KL divergence is used to measure the difference between the parameter distributions of the old and new models to prevent overfitting;

[0194] Parameter setting and substituting values:

[0195] Suppose there is a lightweight large model that has been pre-trained on general text data and now wants to be applied to the field of intelligent question answering. It will be optimized using the above formula.

[0196] Calculation of state snapshot: First, obtain a vector representation of the knowledge nodes regarding a certain question , and calculate the model parameters of the status snapshot . Here, the loss function can be the cross-entropy loss, which is applicable to classification tasks such as question classification.

[0197] Key parameter selection: Next, determine the set of parameters that need to be adjusted with emphasis according to the status snapshot , and set the fine-tuning step size and direction for each parameter. For example, if a certain parameter corresponds to the key features of the question, it may be selected into .

[0198] Multi-round iterative update: Then, perform small-scale, multi-round iterative updates on the selected parameters according to a predetermined progressive fine-tuning plan. Assume that after several rounds of iteration, a set of optimized intermediate models is obtained.

[0199] Adaptive learning rate adjustment: During this process, the system will dynamically adjust the learning rate to ensure the stability and efficiency of the adjustment process.

[0200] Select the optimal intermediate model: Finally, select the most stable and condition-satisfying one from all the optimized intermediate models as the final preliminary optimization model.

[0201] To simplify the explanation, assume that only one parameter is considered, and the following values are known:

[0202] Initial learning rate ;

[0203] Time decay coefficient ;

[0204] Gradient variance influence factor ;

[0205] Regularization coefficient ;

[0206] Assume that the gradient variance at the round of iteration;

[0207] Kullback-Leibler divergence ;

[0208] Calculate the adaptive learning rate according to the formula ;

[0209] Assume (i.e., without additional learning rate adjustment), and :

[0210] ;

[0211] This calculation shows that as the number of iterations increases, the learning rate gradually decreases but still remains within a reasonable range, which helps the model converge stably.

[0212] For the selection of the preliminary optimized model, assume there are three intermediate models Their loss variances are respectively , and the corresponding KL divergences are . Then according to the formula:

[0213] ;

[0214] The scores of each model can be calculated:

[0215] For ;

[0216] For ;

[0217] For ;

[0218] Obviously, has the lowest score, so will be selected as the preliminary optimized model because it not only has the smallest loss variance, but also the change in the distribution of model parameters is relatively small, meaning it is relatively more stable and will not deviate too far from the original pre-training state.

[0219] Figure 2 FIG. provides a schematic structural diagram of an intelligent question-answering system based on a lightweight large model according to an embodiment of the present application, as Figure 2 shown, the device includes:

[0220] A receiving module 21, configured to parse and process the received natural language question to obtain the user's query intention and the knowledge domain to which the natural language question belongs;

[0221] A positioning module 22, configured to perform positioning processing on the pre-constructed knowledge graph according to the query intention, using the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning, to obtain a set of knowledge nodes highly relevant to the natural language question;

[0222] An adjustment module 23, configured to activate an adaptive context understanding mechanism in the lightweight large model based on the set of knowledge nodes, and perform dynamic adjustment processing on the internal parameter configuration of the lightweight large model by adopting a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network, so as to generate an optimized lightweight large model;

[0223] A generation module 24, configured to use the optimized lightweight large model, and in combination with a method of providing diversified expression suggestions by an adversarial generation network, cooperate with a sequence-to-sequence architecture and a beam search decoding strategy to perform answer generation processing to obtain an answer text corresponding to the natural language question;

[0224] An inspection module 25, configured to perform quality inspection processing on the answer text, and for an answer text that does not meet the preset quality standard, trigger a secondary query process or prompt the user to modify the question description to ensure that the answer text meets the preset quality standard.

[0225] Figure 2 The described intelligent question-answering system based on a lightweight large model can execute Figure 1 The method of the intelligent question-answering system based on a lightweight large model in the illustrated embodiment, and its implementation principle and technical effects will not be elaborated further. For the intelligent question-answering system based on a lightweight large model in the above embodiment, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0226] In a possible design, Figure 2 The intelligent question-answering system based on a lightweight large model in the illustrated embodiment can be implemented as a computing device, such as Figure 3 shown, and the computing device can include a storage component 31 and a processing component 32;

[0227] The storage component 31 stores one or more computer instructions, and the one or more computer instructions are called and executed by the processing component 32.

[0228] The processing component 32 is used for: A method for constructing an intelligent question-answering system based on a lightweight large model, characterized by including: parsing and processing the received natural language question to obtain the user's query intention and the knowledge domain to which the natural language question belongs; according to the query intention, using the corresponding adaptation algorithm and the selection mechanism based on deep reinforcement learning in the knowledge domain to perform positioning processing on the pre-constructed knowledge graph to obtain a set of knowledge nodes highly relevant to the natural language question; based on the set of knowledge nodes, activating the adaptive context understanding mechanism in the lightweight large model, and using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to perform dynamic adjustment processing on the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model; using the optimized lightweight large model, and combining with the method of providing diverse expression suggestions by an adversarial generation network, and cooperating with a sequence-to-sequence architecture and a beam search decoding strategy to perform answer generation processing to obtain the answer text corresponding to the natural language question; performing quality inspection processing on the answer text, and for the answer text that does not meet the preset quality standard, triggering a secondary query process or prompting the user to modify the question description to ensure that the answer text meets the preset quality standard.

[0229] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for executing the above method.

[0230] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0231] Of course, the computing device may necessarily also include other components, such as input / output interfaces, display components, communication components, etc.

[0232] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0233] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0234] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server. The above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from a cloud computing platform.

[0235] The embodiment of the present application also provides a computer storage medium storing a computer program, which when executed by a computer can implement the above-mentioned Figure 1 construction method of an intelligent question-answering system based on a lightweight large model shown in the embodiment.

[0236] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0237] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0238] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0239] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A method for constructing an intelligent question answering system based on a lightweight large model, characterized in that: include: Parsing the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; According to the query intent, the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning are used to locate the pre-constructed knowledge graph to obtain a set of knowledge nodes that are highly relevant to the natural language question; Based on the knowledge node set, the adaptive context understanding mechanism in the lightweight large model is activated, and the internal parameter configuration of the lightweight large model is dynamically adjusted by using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to generate an optimized lightweight large model; Using the optimized lightweight large model, combined with the method of diversified expression suggestions provided by the adversarial generative network, with the sequence-to-sequence architecture and beam search decoding strategy, answer generation processing is performed to obtain the answer text corresponding to the natural language question; Performing a quality check on the answer text, and for answer texts that do not meet the preset quality standards, triggering a secondary query process, or prompting the user to modify the question description to ensure that the answer text meets the preset quality standards; The process of generating the optimized lightweight large model includes: converting the relevance scores of the knowledge node set to obtain a knowledge node vector representation that can capture the knowledge node set; according to the knowledge node vector representation, using a progressive parameter fine-tuning scheme to dynamically adjust the parameters in the lightweight large model to obtain a preliminary optimized model; introducing the attention mechanism to enhance the preliminary optimized model, and further optimizing the preliminary optimized model based on a memory network to ensure that the preliminary optimized model can retrieve and use the knowledge node set when generating the answer text, thereby obtaining the optimized lightweight large model; The method of dynamically adjusting the parameters in the lightweight large model by using a progressive parameter fine-tuning scheme according to the knowledge node vector representation to obtain a preliminary optimization model includes: Using the knowledge node vector representation as input, the lightweight large model parameters are subjected to initial state analysis and processing through a phased evaluation mechanism to obtain a state snapshot of the lightweight large model parameters; wherein the state snapshot of the lightweight large model parameters is calculated by the following formula: ； in, is a state snapshot of the parameters of the lightweight large model, is a lightweight large model parameter, is the knowledge node vector representation, is the loss function Parameters The gradient of is the Hessian matrix, which is used to capture the second-order derivative information; According to the state snapshot, a set of parameters in the lightweight large model that need to be adjusted is determined, and a fine-tuning step and direction are set for each parameter in the lightweight large model that needs to be adjusted, and a progressive fine-tuning plan is generated; wherein the set of parameters, the fine-tuning step and the direction in the lightweight large model are calculated by the following formula: ； ； ； in, is the set of parameters in the lightweight large model, It is The fine-tuning step size of the parameters, It's the direction. is the loss function, represents a single parameter in a lightweight large model, is the initial learning rate, is the historical gradient weight, It is The historical gradient weight coefficient of the iteration, is the influencing factor of the Hessian matrix, used to enhance the gradient direction, is the loss function Parameters The partial derivative of The impact on losses, is the inverse of the diagonal elements of the Hessian matrix, which is used to adjust the importance of the gradient. is the direction of the dot product of the Hessian matrix and the gradient, which is used to enhance the gradient direction. Loss Function For Parameters The partial derivative of From the first iteration to the Accumulation of historical gradient information of iterations; Based on the progressive fine-tuning plan, the parameters in the lightweight large model that need to be adjusted are updated in small increments and multiple rounds of iterative updates, and the performance changes of the lightweight large model are evaluated after each round of iterative update to obtain an optimized intermediate model; Introducing an adaptive learning rate adjustment mechanism to automatically adjust the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model, thereby ensuring a balance between the speed and accuracy of parameter adjustment of the optimized intermediate model and obtaining an adjustment result; By accumulating the adjustment results, all the optimized intermediate models are screened, and finally the optimized intermediate model with the most stable performance is selected as the preliminary optimization model.

2. The method according to claim 1, characterized in that The optimized lightweight large model is combined with the method of diversified expression suggestions provided by the adversarial generative network, and the sequence-to-sequence architecture and beam search decoding strategy are used to perform answer generation processing to obtain the answer text corresponding to the natural language question, including: Using the optimized lightweight large model and combining it with the method of diversified expression suggestions provided by the adversarial generative network, the natural language question is semantically understood and the expression is enriched to obtain candidate answer texts; Based on the sequence-to-sequence architecture, in conjunction with the optimized lightweight large model, the candidate answer text is encoded-decoded to generate a preliminary answer text framework; The preliminary answer text framework is optimized by applying a beam search decoding strategy, and the most likely answer sequence is explored by maintaining multiple possible answer paths to generate an answer text corresponding to the natural language question.

3. The method according to claim 2, characterized in that The sequence-to-sequence architecture is used in conjunction with the optimized lightweight large model to perform encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework, including: According to the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture, the candidate answer text is encoded to convert the candidate answer text into a context vector of a fixed length, wherein the context vector is used to capture the core semantic information of the candidate answer; Based on the context vector, the answer sequence is gradually constructed through the decoder part of the sequence-to-sequence architecture, the probability distribution of the next word is predicted at each time step, and the probability distribution is used to select appropriate words to be added to the answer sequence according to the set strategy to obtain an updated answer sequence; Based on the updated answer sequence, the context vector is updated, and target words are continuously selected according to a set strategy to be added to the answer sequence until a preset maximum sentence length is reached or an end symbol is encountered, so as to generate a preliminary answer text framework; The process of selecting the target vocabulary according to the set strategy includes selecting the vocabulary with the highest probability distribution as the target vocabulary at each time step, or selecting the top k words with the highest probability distribution from the predicted multiple probability distributions and randomly selecting any word from them as the target vocabulary.

4. The method according to claim 2, characterized in that: The application of the beam search decoding strategy to optimize the preliminary answer text framework and explore the most likely answer sequence by maintaining multiple possible answer paths to generate an answer text corresponding to the natural language question includes: Initializing the preliminary answer text frame using a set fixed beam width to obtain an initial candidate answer path set; Based on the initial candidate answer path set, multiple possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set; Introducing a length normalization and diversity penalty mechanism to adjust the gradually expanded answer path set to obtain an optimized answer path, wherein the adjustment process is used to prevent shorter sentences from being preferentially selected due to higher cumulative probability distribution and to generate more diverse answers; The optimized answer path is used to explore the most likely answer sequence, select the answer sequence with the highest cumulative probability distribution, and generate the answer text corresponding to the natural language question.

5. The method according to claim 4, characterized in that The initial answer text framework is initialized using the set fixed beam width to obtain an initial candidate answer path set, including: Set a fixed beam width to k, where the fixed beam width is used to determine the number of candidate answer paths to be retained; Selecting the first k highest probability words from the probability distribution of the words predicted by the decoder part of the sequence-to-sequence architecture as starting words to form k initial candidate answer paths and construct an initial set of candidate answer paths; Based on the initial candidate answer path set, a plurality of possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set, including: For each time step, based on all candidate answer paths generated in the previous time step, for each candidate answer path, use the current context vector to predict the probability distribution of the next word, and obtain the possible subsequent word probability distribution of each candidate answer path; Based on the subsequent word probability distribution, select the top k words with the highest probability from the new words expanded from each candidate answer path, and combine them with the corresponding candidate answer path to form an optimized candidate answer path; From all the optimized candidate answer paths, select the top k optimized candidate answer paths with the highest cumulative probability to continue expanding, ensuring that only the most likely candidate answer paths are retained each time, until the preset maximum sentence length is reached or the end symbol is encountered, and a gradually expanded answer path set is obtained.

6. An intelligent question answering system based on a lightweight large model, characterized in that: include: A receiving module is used to parse and process the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; A positioning module is used to locate the pre-built knowledge graph according to the query intent, using the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning, to obtain a set of knowledge nodes that are highly relevant to the natural language question; An adjustment module is used to activate the adaptive context understanding mechanism in the lightweight large model based on the knowledge node set, and dynamically adjust the internal parameter configuration of the lightweight large model by using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to generate an optimized lightweight large model; A generation module is used to use the optimized lightweight large model, in combination with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain the answer text corresponding to the natural language question; An inspection module is used to perform quality inspection on the answer text, and for answer texts that do not meet the preset quality standards, trigger a secondary query process, or prompt the user to modify the question description to ensure that the answer text meets the preset quality standards; The process of generating the optimized lightweight large model includes: converting the relevance scores of the knowledge node set to obtain a knowledge node vector representation that can capture the knowledge node set; according to the knowledge node vector representation, using a progressive parameter fine-tuning scheme to dynamically adjust the parameters in the lightweight large model to obtain a preliminary optimized model; introducing the attention mechanism to enhance the preliminary optimized model, and further optimizing the preliminary optimized model based on a memory network to ensure that the preliminary optimized model can retrieve and use the knowledge node set when generating the answer text, thereby obtaining the optimized lightweight large model; The method of dynamically adjusting the parameters in the lightweight large model by using a progressive parameter fine-tuning scheme according to the knowledge node vector representation to obtain a preliminary optimization model includes: Using the knowledge node vector representation as input, the lightweight large model parameters are subjected to initial state analysis and processing through a phased evaluation mechanism to obtain a state snapshot of the lightweight large model parameters; wherein the state snapshot of the lightweight large model parameters is calculated by the following formula: ； in, is a state snapshot of the parameters of the lightweight large model, is a lightweight large model parameter, is the knowledge node vector representation, is the loss function Parameters The gradient of is the Hessian matrix, which is used to capture the second-order derivative information; According to the state snapshot, a set of parameters in the lightweight large model that need to be adjusted is determined, and a fine-tuning step and direction are set for each parameter in the lightweight large model that needs to be adjusted, and a progressive fine-tuning plan is generated; wherein the set of parameters, the fine-tuning step and the direction in the lightweight large model are calculated by the following formula: ； ； ； in, is the set of parameters in the lightweight large model, It is The fine-tuning step size of the parameters, It's the direction. is the loss function, represents a single parameter in a lightweight large model, is the initial learning rate, is the historical gradient weight, It is The historical gradient weight coefficient of the iteration, is the influencing factor of the Hessian matrix, used to enhance the gradient direction, is the loss function Parameters The partial derivative of The impact on losses, is the inverse of the diagonal elements of the Hessian matrix, which is used to adjust the importance of the gradient. is the direction of the dot product of the Hessian matrix and the gradient, which is used to enhance the gradient direction. Loss Function For Parameters The partial derivative of From the first iteration to the Accumulation of historical gradient information of iterations; Based on the progressive fine-tuning plan, the parameters in the lightweight large model that need to be adjusted are updated in small increments and multiple rounds of iterative updates, and the performance changes of the lightweight large model are evaluated after each round of iterative update to obtain an optimized intermediate model; Introducing an adaptive learning rate adjustment mechanism to automatically adjust the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model, thereby ensuring a balance between the speed and accuracy of parameter adjustment of the optimized intermediate model and obtaining an adjustment result; By accumulating the adjustment results, all the optimized intermediate models are screened, and finally the optimized intermediate model with the most stable performance is selected as the preliminary optimization model.

7. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a method for constructing an intelligent question-answering system based on a lightweight large model as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a method for constructing an intelligent question-answering system based on a lightweight large model as described in any one of claims 1 to 5 is implemented.

Images