Construction method and device of intelligent question answering system based on lightweight large model

Abstract

The invention provides an intelligent question answering system construction method and device based on a lightweight large model, and the method comprises the steps: carrying out the analysis of a received natural language question, and obtaining a query intention of a user; performing positioning processing on a pre-constructed knowledge graph to obtain a knowledge node set related to the natural language problem; performing dynamic adjustment processing on internal parameter configuration of the lightweight large model to generate an optimized lightweight large model; performing answer generation processing to obtain an answer text corresponding to the natural language question; and for the answer text which does not reach the preset quality standard, triggering a secondary query process, or prompting a user to modify question description. According to the technical scheme, the response speed and accuracy of the question answering system and the flexibility of adapting to different knowledge fields can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of lightweight large models, and in particular to a method and device for constructing an intelligent question-answering system based on a lightweight large model. Background Art

[0002] With the rapid development of information technology, intelligent question-answering systems have been widely used in many fields such as customer service, medical consultation, and online education. Users expect to interact with the system through natural language and get instant and accurate answers. Especially in professional fields, accurately understanding the user's query intent and the knowledge domain to which the question belongs is crucial to providing high-quality services.

[0003] Most current intelligent question-answering systems rely on large pre-trained language models, which, while performing well on a wide range of natural language processing tasks, have limitations in specific domain applications. On the one hand, the deployment of large models requires high computing resources, limiting their use in edge devices or resource-constrained environments; on the other hand, traditional question-answering systems typically use fixed rule sets or simple keyword matching to find answers, which is difficult to capture complex semantic information, especially when faced with polysemous words or complex sentences, and can easily lead to misunderstandings of the user's actual query intent.

[0004] Existing intelligent question-answering solutions mainly face the following challenges: First, large pre-trained models are bulky and expensive to deploy, which is not conducive to use on mobile devices or other scenarios with limited computing resources; second, rule-based or keyword matching methods have insufficient understanding of context and cannot fully meet users' needs for complex queries; finally, there is a lack of effective feedback mechanism. When the generated answers do not meet expectations, the system cannot effectively adjust or optimize itself, affecting user experience and service quality. In addition, the existing system still has room for improvement in ensuring the quality and relevance of answers, especially when a satisfactory answer is not generated for the first time, there is a lack of effective secondary query or user guidance mechanism. Summary of the invention

[0005] The embodiments of the present application provide a method and device for constructing an intelligent question-answering system based on a lightweight large model, so as to solve the problems of slow response speed, low accuracy and low flexibility in adapting to different knowledge fields in the prior art.

[0006] Receiving real-time data streams from different sources, the real-time data streams containing structured data and unstructured data; In a first aspect, an embodiment of the present application provides a method for constructing an intelligent question-answering system based on a lightweight large model, comprising: Parsing the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; According to the query intent, the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning are used to locate the pre-constructed knowledge graph to obtain a set of knowledge nodes that are highly relevant to the natural language question; Based on the knowledge node set, the adaptive context understanding mechanism in the lightweight large model is activated, and the internal parameter configuration of the lightweight large model is dynamically adjusted by using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to generate an optimized lightweight large model; Using the optimized lightweight large model, combined with the method of diversified expression suggestions provided by the adversarial generative network, with the sequence-to-sequence architecture and beam search decoding strategy, answer generation processing is performed to obtain the answer text corresponding to the natural language question; The answer text is quality checked, and for answer text that does not meet the preset quality standard, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standard.

[0007] Optionally, the optimized lightweight large model is used in combination with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain the answer text corresponding to the natural language question, including: Using the optimized lightweight large model and combining it with the method of diversified expression suggestions provided by the adversarial generative network, the natural language question is semantically understood and the expression is enriched to obtain candidate answer texts; Based on the sequence-to-sequence architecture, in conjunction with the optimized lightweight large model, the candidate answer text is encoded-decoded to generate a preliminary answer text framework; Applying a beam search decoding strategy to optimize the preliminary answer text framework, exploring the most likely answer sequence by maintaining multiple possible answer paths, so as to generate an answer text corresponding to the natural language question. Optionally, the sequence-to-sequence architecture is used in conjunction with the optimized lightweight large model to perform encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework, including: According to the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture, the candidate answer text is encoded to convert the candidate answer text into a context vector of a fixed length, wherein the context vector is used to capture the core semantic information of the candidate answer; Based on the context vector, the answer sequence is gradually constructed through the decoder part of the sequence-to-sequence architecture, the probability distribution of the next word is predicted at each time step, and the probability distribution is used to select appropriate words to be added to the answer sequence according to the set strategy to obtain an updated answer sequence; Based on the updated answer sequence, the context vector is updated, and target vocabulary is continuously selected according to a set strategy to be added to the answer sequence until a preset maximum sentence length is reached or an end symbol is encountered, so as to generate a preliminary answer text framework.

[0008] The process of selecting the target vocabulary according to the set strategy includes selecting the vocabulary with the highest probability distribution as the target vocabulary at each time step, or selecting the top k words with the highest probability distribution from the predicted multiple probability distributions and randomly selecting any word from them as the target vocabulary.

[0009] Optionally, the application of a beam search decoding strategy to optimize the preliminary answer text framework and explore the most likely answer sequence by maintaining multiple possible answer paths to generate an answer text corresponding to the natural language question includes: Initializing the preliminary answer text frame using a set fixed beam width to obtain an initial candidate answer path set; Based on the initial candidate answer path set, multiple possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set; Introducing a length normalization and diversity penalty mechanism to adjust the gradually expanded answer path set to obtain an optimized answer path, wherein the adjustment process is used to prevent shorter sentences from being preferentially selected due to higher cumulative probability distribution and to generate more diverse answers; The optimized answer path is used to explore the most likely answer sequence, select the answer sequence with the highest cumulative probability distribution, and generate the answer text corresponding to the natural language question.

[0010] Optionally, it is characterized in that the initialization processing of the preliminary answer text framework using the set fixed beam width to obtain an initial candidate answer path set includes: Set a fixed beam width to k, where the fixed beam width is used to determine the number of candidate answer paths to be retained; Selecting the first k highest probability words from the probability distribution of the words predicted by the decoder part of the sequence-to-sequence architecture as starting words to form k initial candidate answer paths and construct an initial set of candidate answer paths; Based on the initial candidate answer path set, a plurality of possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set, including: For each time step, based on all candidate answer paths generated in the previous time step, for each candidate answer path, use the current context vector to predict the probability distribution of the next word, and obtain the possible subsequent word probability distribution of each candidate answer path; Based on the subsequent word probability distribution, select the top k words with the highest probability from the new words expanded from each candidate answer path, and combine them with the corresponding candidate answer path to form an optimized candidate answer path; From all the optimized candidate answer paths, select the top k optimized candidate answer paths with the highest cumulative probability to continue expanding, ensuring that only the most likely candidate answer paths are retained each time, until the preset maximum sentence length is reached or the end symbol is encountered, and a gradually expanded answer path set is obtained.

[0011] Optionally, based on the knowledge node set, activating the adaptive context understanding mechanism in the lightweight large model, using a progressive parameter fine-tuning scheme, an attention mechanism and a memory network, dynamically adjusting the internal parameter configuration of the lightweight large model, and generating an optimized lightweight large model, including: Converting the relevance scores of the knowledge node set to obtain a knowledge node vector representation that can capture the knowledge node set; According to the knowledge node vector representation, a progressive parameter fine-tuning scheme is adopted to perform the dynamic adjustment process on the parameters in the lightweight large model to obtain a preliminary optimized model; The attention mechanism is introduced to enhance the preliminary optimization model, and the preliminary optimization model is further optimized based on the memory network to ensure that the preliminary optimization model can retrieve and utilize the knowledge node set when generating the answer text to obtain the optimized lightweight large model.

[0012] Optionally, the step of dynamically adjusting the parameters in the lightweight large model using a progressive parameter fine-tuning scheme according to the knowledge node vector representation to obtain a preliminary optimization model includes: Using the knowledge node vector representation as input, the lightweight large model parameters are subjected to initial state analysis processing through a phased evaluation mechanism to obtain a state snapshot of the lightweight large model parameters; Determine the parameters in the lightweight large model that need to be adjusted according to the state snapshot, set a fine-tuning step and direction for each parameter in the lightweight large model that needs to be adjusted, and generate a progressive fine-tuning plan; Based on the progressive fine-tuning plan, the parameters in the lightweight large model that need to be adjusted are updated in small increments and multiple rounds of iterative updates, and the performance changes of the lightweight large model are evaluated after each round of iterative update to obtain an optimized intermediate model; Introducing an adaptive learning rate adjustment mechanism to automatically adjust the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model, thereby ensuring a balance between the speed and accuracy of parameter adjustment of the optimized intermediate model and obtaining an adjustment result; By accumulating the adjustment results, all the optimized intermediate models are screened, and finally the optimized intermediate model with the most stable performance is selected as the preliminary optimization model.

[0013] In a second aspect, the embodiment of the present application provides an intelligent question-answering system based on a lightweight large model, including: A receiving module is used to parse and process the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; A positioning module is used to locate the pre-built knowledge graph according to the query intent, using the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning, to obtain a set of knowledge nodes that are highly relevant to the natural language question; An adjustment module is used to activate the adaptive context understanding mechanism in the lightweight large model based on the knowledge node set, and dynamically adjust the internal parameter configuration of the lightweight large model by using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to generate an optimized lightweight large model; A generation module is used to use the optimized lightweight large model, in combination with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain the answer text corresponding to the natural language question; The checking module is used to perform quality checking on the answer text. For answer texts that do not meet the preset quality standards, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standards.

[0014] In a third aspect, an embodiment of the present application provides a computing device, characterized in that it includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a method for constructing an intelligent question and answer system based on a lightweight large model as described in any one of the first aspects.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium, characterized in that a computer program is stored therein, and when the computer program is executed by a computer, a method for constructing an intelligent question and answer system based on a lightweight large model as described in any one of the first aspects is implemented.

[0016] In an embodiment of the present application, a received natural language question is parsed to obtain the user's query intent and the knowledge field to which the natural language question belongs; based on the query intent, an adaptive algorithm corresponding to the knowledge field and a selection mechanism based on deep reinforcement learning are used to locate the pre-constructed knowledge graph to obtain a set of knowledge nodes that are highly relevant to the natural language question; based on the set of knowledge nodes, an adaptive context understanding mechanism in the lightweight large model is activated, and a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network are used to dynamically adjust the internal parameter configuration of the lightweight large model to generate an optimized lightweight large model; using the optimized lightweight large model, in combination with a method for diversified expression suggestions provided by a generative adversarial network, a sequence-to-sequence architecture, and a beam search decoding strategy, an answer generation process is performed to obtain an answer text corresponding to the natural language question; a quality check is performed on the answer text, and for answer texts that do not meet the preset quality standards, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standards.

[0017] The technical solution of this application has the following beneficial effects: The received natural language question is parsed to obtain the user's query intent and the knowledge domain to which the natural language question belongs; according to the query intent, the knowledge domain adaptability algorithm and the selection mechanism based on deep reinforcement learning are used to locate the pre-constructed knowledge graph to obtain a set of knowledge nodes that are highly relevant to the natural language; based on the knowledge node set, the adaptive context understanding mechanism in the lightweight large model is activated, and the internal parameter configuration of the lightweight large model is dynamically adjusted using a progressive parameter fine-tuning scheme, an attention mechanism and a memory network to generate an optimized lightweight large model; the optimized lightweight large model is used in combination with a method of diversified expression suggestions provided by a generative adversarial network, a sequence-to-sequence architecture and a beam search decoding strategy to generate an answer and obtain an answer text corresponding to the natural language question; the answer text is quality checked, and for answer texts that do not meet the preset quality standards, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standards.

[0018] Furthermore, by combining the optimized lightweight large model with the diverse expression suggestions provided by the adversarial generative network, this method can perform deep semantic understanding and expression enrichment processing on natural language questions, thereby generating more accurate and diverse candidate answer texts. The use of a sequence-to-sequence architecture combined with a beam search decoding strategy not only improves the flexibility and accuracy of answer generation, but also ensures the quality and relevance of the answers. This method can provide more natural and readable answers while maintaining high accuracy.

[0019] Furthermore, by converting the candidate answer text into a fixed-length context vector, the core semantic information of the candidate answer can be effectively captured, providing a solid foundation for subsequent answer generation. Based on the context vector, the system gradually predicts and selects appropriate words to add to the answer sequence. This process allows the system to make the best choice at each time step, ensuring that the generated answer is both grammatical and logically coherent. As the answer sequence is continuously updated, the context vector is also adjusted accordingly, ensuring the real-time and accuracy of information during the answer generation process. This dynamic mechanism enables the system to flexibly adjust the direction of the next answer based on the part of the content that has been generated, enhancing the system's adaptability.

[0020] Furthermore, the beam search decoding strategy is used to maintain multiple possible answer paths, allowing the system to explore the most likely answer sequence among many possibilities, greatly increasing the probability of finding the best answer. By introducing length normalization and diversity penalty mechanisms, shorter sentences are prevented from being prioritized due to their higher cumulative probability, and more diverse answers are generated, ensuring that the final answer is not only the most probable, but also the most in line with user needs. Selecting the answer sequence with the highest cumulative probability distribution from the optimized answer path ensures that the generated answer text is not only statistically optimal, but also provides the best user experience in practical applications.

[0021] Furthermore, a fixed beam width k is set to determine the number of candidate answer paths to be retained. By selecting the top k words with the highest probability as the starting words, it is ensured that the most potential answer path can be focused on in the initial stage, thus improving efficiency. Multiple possible answer paths are expanded and maintained at each time step, and the top k optimized candidate answer paths with the highest cumulative probability are selected to continue expanding, ensuring that each iteration can focus on the most likely answer path, reducing unnecessary computational overhead. This setting clarifies the boundary conditions for answer generation, avoids infinite loops or overly long answers, and ensures the integrity and rationality of the answers until the preset maximum sentence length is reached or the end symbol is encountered.

[0022] These methods or other aspects of the present application will be more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A flowchart of a method for constructing an intelligent question-answering system based on a lightweight large model provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a big data processing system for implementing hybrid data analysis provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0026] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] Figure 1 A flowchart of a method for constructing an intelligent question-answering system based on a lightweight large model is provided for an embodiment of the present application. Figure 1 As shown, the method includes: 101. Analyze the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; In this step, natural language processing (NLP) technology refers to a series of technologies that allow computers to understand and generate human natural language. This includes but is not limited to word segmentation, part-of-speech tagging, named entity recognition (NER), syntactic analysis, and semantic analysis.

[0029] Query intent: The specific purpose or need behind a user’s question or request.

[0030] Knowledge domain: The area of ​​expertise or subject matter to which the problem belongs.

[0031] This step uses natural language processing (NLP) technologies, such as word segmentation, part-of-speech tagging, named entity recognition, and domain classification algorithms, to understand and categorize user questions. Through these technologies, the system can more accurately understand the user's actual needs and locate the problem in a specific knowledge field, providing guidance for subsequent steps.

[0032] In the embodiment of the present application, in the field of intelligent customer service AI question and answer, when receiving a question input by a user, the system will first use a pre-trained language model to parse the text and extract key information. For example, for a question about "how to set a mobile phone alarm", the system will recognize that this is a query related to "mobile device operation" and further refine it to the specific "alarm setting" task. Next, the system will select the most appropriate answer strategy or knowledge base based on the analysis results.

[0033] For example, suppose a user asks "How do I set an alarm on my iPhone?" The system will first parse the question and identify keywords such as "iPhone" and "set an alarm." It will then determine that this is a query about "mobile device operations" and specifically about the alarm function of the iOS system. This information is used to guide subsequent knowledge graph search and answer generation.

[0034] 102. According to the query intent, use the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning to locate the pre-constructed knowledge graph and obtain a set of knowledge nodes that are highly relevant to the natural language question; In this step, adaptive algorithms: techniques that adjust algorithm parameters or structures according to the needs of specific tasks to improve performance or efficiency.

[0035] Selection mechanism for deep reinforcement learning: a machine learning method that uses reward feedback to guide the model to make optimal choices.

[0036] Knowledge Graph: A graph database in which nodes represent entities or concepts and edges express the relationships between them.

[0037] In step 102, the system applies adaptive algorithms and deep reinforcement learning-based selection mechanisms based on the query intent obtained in the first step to find a set of highly relevant knowledge nodes from the pre-built knowledge graph. The adaptive algorithm here refers to an algorithm that can adjust its behavior according to different query types, while the deep reinforcement learning mechanism helps the system make the best decision among a large number of possible choices, ensuring that the information found is both accurate and comprehensive.

[0038] In the embodiment of this application, in the intelligent customer service AI question-answering scenario, once the user's query intention and the field to which the question belongs are clarified, the system will use a deep reinforcement learning algorithm to explore the nodes in the knowledge graph and find those parts that best match the user's question. In this process, the system not only considers directly related information, but also mines potentially related content in order to provide users with richer and more detailed answers.

[0039] For example, continuing with the previous example, the system already knows that what the user wants to know is "how to set an alarm on an iPhone". So, it starts searching the knowledge graph for all nodes related to the iOS system, the alarm application, and its setting options. With the help of deep reinforcement learning algorithms, the system is able to select the best path to reach the target information, which is the specific steps and tips for setting an alarm.

[0040] 103. Based on the knowledge node set, activate the adaptive context understanding mechanism in the lightweight large model, adopt a progressive parameter fine-tuning scheme, an attention mechanism and a memory network, dynamically adjust the internal parameter configuration of the lightweight large model, and generate an optimized lightweight large model; In this step, a lightweight large model is used: a language model that is pre-trained and relatively small but still maintains efficient performance.

[0041] Adaptive context understanding mechanism: allows the model to dynamically adjust its internal configuration based on the history of the conversation and the current environment to better understand contextual information.

[0042] Progressive parameter tuning scheme: A method that continuously updates model parameters, aiming to gradually improve the model's performance as new data arrives.

[0043] Attention mechanism: enables the model to focus on certain parts of the input sequence and ignore other unimportant content.

[0044] Memory Network: Enhances the memory function of the model so that it can remember information from previous interactions.

[0045] In step 103, the lightweight large model is activated to perform adaptive context understanding. This involves dynamically adjusting the internal parameters of the model to better adapt it to the current conversation context. Progressive parameter fine-tuning allows the model to gradually optimize its performance; the attention mechanism helps the model focus on the most important part of the data; and the memory network enables the model to remember the history of previous interactions, thereby improving the understanding of continuous conversations.

[0046] In the embodiment of the present application, for intelligent customer service, activating adaptive context understanding means giving the model a stronger context perception ability. For example, when discussing complex issues with users, the model needs to understand the historical background of the conversation while paying attention to the latest question content. In this way, the model can give more coherent and targeted answers to improve the user experience.

[0047] For example, continuing with the previous example, while answering the question “How do I set an alarm on my iPhone?”, if the user then asks “How do I turn it off?”, the system will activate the adaptive context understanding mechanism, remember the previously mentioned alarm setting topic, and directly provide a method to turn off the alarm without the user having to describe it in detail again.

[0048] 104. Using the optimized lightweight large model, combined with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, perform answer generation processing to obtain the answer text corresponding to the natural language question; In this step, Generative Adversarial Networks (GANs): help create more diverse and natural responses.

[0049] Sequence-to-sequence architecture (Seq2Seq): used to convert one sequence into another sequence, widely used in fields such as machine translation and dialogue systems.

[0050] Beam search decoding strategy: retain multiple possible answer paths during the generation process and finally select the answer with the highest quality.

[0051] In step 104, the optimized lightweight large model combines the diverse expression suggestions provided by the adversarial generative network, and generates the final answer text through the sequence-to-sequence architecture and beam search decoding strategy. The adversarial generative network helps to create diverse expressions and make the answers more natural and fluent; while the sequence-to-sequence architecture and beam search ensure that the answer structure is reasonable and logically clear.

[0052] In the embodiment of the present application, when preparing answers, the intelligent customer service AI will use the above-mentioned combination of technologies to construct answers that are both accurate and easy to read. For example, in response to the question "How do I set an alarm on my iPhone?", the system may generate multiple expressions and select the most suitable one to present to the user to ensure the effectiveness of information transmission. For example, continuing with the previous example, when the system is ready to answer the question "How do I set an alarm on my iPhone?", it will comprehensively consider all available resources, including but not limited to the relevant information collected previously, and then tell the user in the most appropriate way: "To set an alarm on your iPhone, open the 'Clock' app, tap the 'Alarm' tab at the bottom, and then tap the '+' sign in the upper right corner to add a new alarm." 105. Performing a quality check on the answer text, and for answer texts that do not meet the preset quality standard, triggering a secondary query process, or prompting the user to modify the question description to ensure that the answer text meets the preset quality standard; In this step, quality check: the generated answers are reviewed to ensure their accuracy, completeness, and applicability.

[0053] Secondary query process: When it is found that the answer does not meet the preset criteria, a re-query process is triggered.

[0054] User prompts to modify questions: guide users to provide more information or clearer question descriptions in order to obtain more satisfactory answers.

[0055] Step 105 involves quality checking of the generated answer texts to ensure that they meet the predetermined standards. Answers that do not meet the standards will trigger a secondary query process or suggest that the user rephrase the question to ensure that the information provided is always reliable and useful. This step is key to maintaining service quality and user satisfaction.

[0056] In the embodiment of the present application, quality inspection is an indispensable step in the intelligent customer service AI question and answer. It not only ensures the accuracy of information, but also improves the user experience. If the answer generated for the first time is not ideal, the system will try to improve it or guide the user to provide more information until a satisfactory result is obtained.

[0057] If the system initially provides an answer that does not fully address the user's question, such as only explaining how to set an alarm but ignoring how to set the number of recurring days, the system will automatically initiate a secondary query process to supplement the missing information or remind the user to be more specific about their needs, such as asking if there are other setting options that need to be understood. In this way, the final answer is ensured to be complete and meet the user's expectations.

[0058] In summary, the five key steps from natural language question parsing to answer generation and quality checking are analyzed in detail. These steps together constitute a complete question-answering processing flow, which aims to ensure that the entire process from receiving user input to the final output of high-quality answers is efficient and accurate.

[0059] Optionally, the optimized lightweight large model in step 104 is used in combination with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain the answer text corresponding to the natural language question, including: The optimized lightweight large model is used in combination with the method of diversified expression suggestions provided by the adversarial generative network to perform semantic understanding and expression enrichment processing on the natural language question to obtain candidate answer texts; based on the sequence-to-sequence architecture, in conjunction with the optimized lightweight large model, the candidate answer texts are encoded-decoded to generate a preliminary answer text framework; the preliminary answer text framework is optimized using a beam search decoding strategy, and the most likely answer sequence is explored by maintaining multiple possible answer paths to generate an answer text corresponding to the natural language question.

[0060] The core of step 104 is to use the optimized lightweight large model, combined with the diverse expression suggestions provided by the Generative Adversarial Network (GAN), to generate answers to natural language questions through a sequence-to-sequence architecture and beam search decoding strategy. Here are a few key concepts: Lightweight large model: refers to a deep learning model that has been optimized through compression or pruning techniques, aiming to maintain high performance while reducing computing resource consumption. It is used to understand the semantics of the input question and generate preliminary answer text based on it.

[0061] Generative Adversarial Networks (GANs): Consists of a generator that tries to create realistic data samples and a discriminator that evaluates the authenticity of these samples. In this context, GANs are used to provide diverse expression suggestions, making the generated answers more natural and varied.

[0062] Sequence-to-sequence architecture (Seq2Seq): A deep learning framework for processing sequence data, commonly used in tasks such as translation and question answering. The architecture consists of two parts: an encoder and a decoder. The former converts the input sequence into a fixed-length vector representation, and the latter generates an output sequence based on this vector.

[0063] Beam Search decoding strategy: A method that maintains multiple candidate answer paths during the decoding process and ultimately selects the most likely answer sequence. Compared with the greedy algorithm, this method can explore more possibilities and improve the quality of the generated text.

[0064] In the embodiment of the present application, in the field of intelligent customer service AI question and answer, when it is necessary to generate answers to user questions, the system first uses the optimized lightweight large model to perform in-depth semantic analysis on the input questions. Then, in order to make the generated answers more natural and diverse, the system introduces expression suggestions provided by the adversarial generative network. These suggestions are used to enrich the content of the candidate answer text, making it not only accurate but also with different expressions.

[0065] Subsequently, based on the sequence-to-sequence architecture, the system encodes the candidate answer text into an intermediate representation and then decodes it into a preliminary answer text framework. This stage is completed by an optimized lightweight large model, ensuring that the answer is both grammatical and logically coherent.

[0066] Finally, a beam search decoding strategy is applied to further optimize the preliminary answer text framework. This strategy increases the possibility of finding the best answer by simultaneously tracking multiple potential answer paths. Beam search allows the system to explore multiple possible answer combinations and select the best one as the final answer to present to the user.

[0067] Here is an example scenario: The user asks "How do I cancel an alarm on my iPhone?" The system first uses a lightweight large model to parse this question and determine that its intention is to cancel an alarm that has been set. Then, the adversarial generative network provides several different but correct answers, such as: "You can cancel the alarm on your iPhone by following the steps below", or "To cancel the alarm on your iPhone, follow these instructions."

[0068] Next, the system uses a sequence-to-sequence architecture to convert the candidate answers into a structured text framework. For example, it might construct a framework like this: "[Start] To cancel the alarm on your iPhone, please [specific steps][end]".

[0069] Finally, the beam search decoding strategy comes into play, selecting the most appropriate answer from multiple possible answer paths. For example, "To cancel an alarm on your iPhone, open the Clock app, select the Alarms tab, find the alarm you want to cancel, and swipe to delete." This multi-step approach ensures that the answer is both detailed and easy to understand, while also reflecting the system's flexibility and adaptability.

[0070] In this way, intelligent customer service can not only respond to user needs quickly, but also provide high-quality and diverse answers, greatly improving the user experience.

[0071] The sequence-to-sequence architecture is used in conjunction with the optimized lightweight large model to perform encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework, including: According to the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture, the candidate answer text is encoded and converted into a fixed-length context vector, which is used to capture the core semantic information of the candidate answer; based on the context vector, the answer sequence is gradually constructed through the decoder part in the sequence-to-sequence architecture, the probability distribution of the next word is predicted at each time step, and the probability distribution is used to select appropriate words to add to the answer sequence according to the set strategy to obtain an updated answer sequence; based on the updated answer sequence, the context vector is updated, and the target word is continued to be added to the answer sequence according to the set strategy until the preset maximum sentence length is reached or the end symbol is encountered, so as to generate a preliminary answer text framework. Wherein, the process of selecting the target word according to the set strategy includes selecting the word with the highest probability distribution as the target word at each time step, or selecting the top k words with the highest probability distribution from the predicted multiple probability distributions, and randomly selecting any word from them as the target word.

[0072] In intelligent question-answering systems, the encoding-decoding conversion process based on the sequence-to-sequence architecture (Seq2Seq) is the process of converting the input text into a fixed-length context vector and using the vector to construct the answer. The encoder part is responsible for receiving the candidate answer text and compressing its core semantic information into a context vector, which acts as a bridge from input to output. The decoder part uses the context vector to gradually build the answer sequence, predicting the probability distribution of the next word each time, and selecting the appropriate word to add to the answer sequence according to the set strategy.

[0073] The application of the beam search decoding strategy to optimize the preliminary answer text framework and explore the most likely answer sequence by maintaining multiple possible answer paths to generate the answer text corresponding to the natural language question includes: The preliminary answer text framework is initialized using a set fixed beam width to obtain an initial set of candidate answer paths; based on the initial set of candidate answer paths, multiple possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set; length normalization and diversity penalty mechanisms are introduced to adjust the gradually expanded answer path set to obtain an optimized answer path, wherein the adjustment is used to prevent shorter sentences from being preferentially selected due to their higher cumulative probability distribution and to generate more diverse answers; using the optimized answer path, the most likely answer sequence is explored, the answer sequence with the highest cumulative probability distribution is selected, and the answer text corresponding to the natural language question is generated.

[0074] Beam Search is a method to optimize the decoding process. Instead of simply selecting the word with the highest probability at each step, it maintains multiple possible answer paths (i.e., "beams") to explore a wider range of possibility spaces and ultimately select the most likely answer sequence. This method can significantly improve the quality and diversity of generated answers. By introducing length normalization and diversity penalty mechanisms, shorter sentences can be prevented from being preferentially selected due to their high cumulative probability, and the generation of more diverse answers can be encouraged.

[0075] Optionally, the method uses a set fixed beam width to initialize the preliminary answer text framework to obtain an initial candidate answer path set, including: setting a fixed beam width k, the fixed beam width is used to determine the number of candidate answer paths to be retained; selecting the first k highest probability words from the probability distribution of the words predicted by the decoder part in the sequence-to-sequence architecture as the starting words to form k initial candidate answer paths and construct an initial candidate answer path set; based on the initial candidate answer path set, expanding and maintaining multiple possible answer paths at each time step to obtain a gradually expanded answer path set, including: for each time step, according to the probability distribution of the words generated at the previous time step, For all candidate answer paths, for each candidate answer path, use the current context vector to predict the probability distribution of the next word to obtain the possible subsequent word probability distribution of each candidate answer path; based on the subsequent word probability distribution, select the top k words with the highest probability from the new words expanded from each candidate answer path, and combine them with the corresponding candidate answer path to form an optimized candidate answer path; from all the optimized candidate answer paths, select the top k optimized candidate answer paths with the highest cumulative probability to continue expanding, ensuring that only the most likely candidate answer path is retained each time, until the preset maximum sentence length is reached or the end symbol is encountered, to obtain a gradually expanded answer path set.

[0076] In this step, the optimized lightweight large model is used, combined with the diverse expression suggestions provided by the Generative Adversarial Network (GAN), to perform semantic understanding and expression enrichment processing on natural language questions to obtain candidate answer texts. Next, based on the sequence-to-sequence architecture, the optimized lightweight large model is used to perform encoding-decoding conversion processing on the candidate answer texts to generate a preliminary answer text framework. Finally, the beam search decoding strategy is applied to explore the most likely answer sequence by maintaining multiple possible answer paths to generate the final answer text corresponding to the natural language question. The technologies involved in this process include but are not limited to semantic understanding, context vector construction, vocabulary probability distribution prediction, length normalization, and diversity penalty mechanisms, which work together to ensure that the generated answers are both accurate and have good expression diversity.

[0077] In the embodiment of the present application, the system first uses the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture to encode the candidate answer text. This includes converting the candidate answer text into a fixed-length context vector that can capture its core semantic information. This context vector serves as the basis for subsequent decoding to ensure that the generated answer can faithfully reflect the intention of the original question.

[0078] Secondly, based on the generated context vector, the system gradually builds the answer sequence through the decoder part. At each time step, the system predicts the probability distribution of the next word and selects the target word to be added to the answer sequence according to the set strategy. This process may repeatedly update the context vector to adapt to the new answer content until the preset maximum sentence length is reached or the end symbol is encountered.

[0079] Furthermore, in order to optimize the preliminary answer text framework, the system adopts a beam search decoding strategy to maintain multiple possible answer paths to explore the most likely answer sequence. First, a fixed beam width (such as k) is set, and the initial set of candidate answer paths is obtained after initialization. Then, these paths are expanded and maintained at each time step, and length normalization and diversity penalty mechanisms are applied to adjust the path set to ensure that the final generated answers are both reasonable and diverse.

[0080] Finally, among all candidate answer paths, the system selects the answer sequence with the highest cumulative probability distribution as the final answer. This process not only takes into account the probability of vocabulary selection, but also takes into account sentence length and diversity to ensure that the generated answers are both accurate and natural.

[0081] Here is an example scenario: Suppose a user asks: "How do I set an alarm on my iPhone?" After the previous steps of parsing and knowledge graph positioning, the system enters the answer generation stage: First, the system uses an optimized lightweight large model and a sequence-to-sequence architecture encoder to encode relevant information about "iPhone alarm setting" into a context vector that focuses on reflecting the core semantics of this question - that is, how to operate the alarm function on an iPhone device.

[0082] Next, the decoder starts working, gradually building a sequence of answers based on the context vector. For example, the first word might be "open" because this is the starting point of most operation guides. The system will continue to predict subsequent words, such as "clock app", "click", "+ sign", etc., gradually forming a complete answer.

[0083] Furthermore, in order to ensure the quality and diversity of answers, the system uses a beam search decoding strategy to maintain multiple possible answer paths. For example, for the options after "open", the system may consider "clock app" and "alarm app" at the same time. As the conversation deepens, the system continuously evaluates and retains the most likely path until a complete answer is generated.

[0084] Finally, when the preset maximum sentence length is reached or the end symbol is encountered, the system selects the path with the highest cumulative probability as the final answer, such as: "To set an alarm on your iPhone, open the 'Clock' app, tap the 'Alarm' tab at the bottom, and then tap the '+' sign in the upper right corner to add a new alarm." In this way, the intelligent customer service AI not only provides direct and effective responses, but also ensures the diversity and naturalness of the answers due to the use of beam search strategies, thereby improving the user experience.

[0085] Optionally, in step 103, based on the knowledge node set, activating the adaptive context understanding mechanism in the lightweight large model, using a progressive parameter fine-tuning scheme, an attention mechanism and a memory network, dynamically adjusting the internal parameter configuration of the lightweight large model, and generating an optimized lightweight large model, including: The relevance scores of the knowledge node set are converted to obtain a knowledge node vector representation that can capture the knowledge node set; based on the knowledge node vector representation, a progressive parameter fine-tuning scheme is used to dynamically adjust the parameters in the lightweight large model to obtain a preliminary optimized model; the attention mechanism is introduced to enhance the preliminary optimized model, and based on a memory network, the preliminary optimized model is further optimized to ensure that the preliminary optimized model can retrieve and utilize the knowledge node set when generating an answer text, thereby obtaining the optimized lightweight large model.

[0086] In this alternative, the system first scores the relevance of the set of knowledge nodes located from the knowledge graph. These scores reflect the degree of relevance of each node to the user's question and are calculated based on multiple factors (such as semantic similarity, contextual relevance, etc.). Next, the system converts these scores into vector representations that can capture the relationship between knowledge nodes, namely knowledge node vector representations. This vectorization process enables machine learning models to understand and utilize this information more effectively.

[0087] Optionally, the step of dynamically adjusting the parameters in the lightweight large model using a progressive parameter fine-tuning scheme according to the knowledge node vector representation to obtain a preliminary optimization model includes: Using the knowledge node vector representation as input, the parameters of the lightweight large model are analyzed and processed for the initial state through a phased evaluation mechanism to obtain a state snapshot of the parameters of the lightweight large model; according to the state snapshot, the parameters in the lightweight large model that need to be adjusted are determined, and a fine-tuning step and direction are set for each parameter in the lightweight large model that needs to be adjusted to generate a progressive fine-tuning plan; based on the progressive fine-tuning plan, the parameters in the lightweight large model that need to be adjusted are updated in small amplitudes and multiple rounds, and the performance changes of the lightweight large model are evaluated after each round of iterative update to obtain an optimized intermediate model; an adaptive learning rate adjustment mechanism is introduced to automatically adjust the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model to ensure a balance between the speed and accuracy of the adjustment of the parameters of the optimized intermediate model to obtain an adjustment result; by accumulating the adjustment results, all the optimized intermediate models are screened, and finally the optimized intermediate model with the most stable performance is selected as the preliminary optimization model.

[0088] This process involves gradually optimizing the parameters in a lightweight large model to better suit a specific task or dataset. Through a phased evaluation mechanism, the system can analyze the state of the current model and adjust specific parameters as needed. The fine-tuning step size and direction determine the specific method of each adjustment, while the adaptive learning rate adjustment mechanism ensures a balance between the speed and accuracy of parameter adjustment, thereby achieving an efficient and stable optimization process.

[0089] In the embodiment of this application, in the field of intelligent customer service AI question and answer, in order to enable the lightweight large model to generate answer text more accurately, we introduced the above optional solution. Specifically: First, by transforming the relevance scores of the knowledge node set, we obtain a vector representation that reflects the relationship between the nodes. This step is crucial for subsequent model optimization because it provides important clues about the knowledge structure.

[0090] Next, using the generated knowledge node vector representation as input, the system performs an initial state analysis and records a state snapshot of the model parameters. Based on this snapshot, it determines which parameters need to be adjusted and sets a fine-tuning plan for each parameter. This plan includes specific fine-tuning steps and directions, guiding subsequent small-scale, multi-round iterative updates.

[0091] Finally, based on the initial optimization, the system further introduces the attention mechanism to enhance the performance of the model, and combines the function of the memory network to enable the model to retrieve and utilize the previous knowledge node set when generating answers. Finally, after a series of performance evaluations and screening, the most stable and best performing model is selected as the initial optimization model.

[0092] Here is an example scenario: Suppose in an intelligent customer service scenario, a user asks "How do I set a repeating alarm on my iPhone?" The system first identifies this as a query related to the "iOS alarm function" and finds the corresponding knowledge node set through a deep reinforcement learning algorithm. The system then scores the relevance of these nodes and converts the scoring results into a vector representation.

[0093] Next, the system begins to incrementally fine-tune the parameters of the lightweight large model. It first analyzes the state of the existing model, determines which parameters are most critical to answering this question, and develops a detailed fine-tuning plan for these parameters. For example, if it is found that certain parameters are closely linked to the logic of "repeated alarm settings", then these parameters may be given priority for adjustment.

[0094] In the subsequent multiple rounds of iterative updates, the system continuously evaluates changes in model performance to ensure that each adjustment can bring positive results. In addition, the adaptive learning rate adjustment mechanism ensures that the speed of parameter adjustment is not too fast or too slow, but just right to promote model improvement.

[0095] As the model was gradually optimized, the system also added an attention mechanism to help the model focus more on the parts directly related to "repeating alarm settings"; and with the support of the memory network, the model was able to review the previous interaction history when generating answers and provide more coherent answers. Finally, after a series of optimization steps, the system successfully generated an optimized lightweight large model that can accurately answer users' questions: "To set a repeating alarm on your iPhone, open the 'Clock' app, tap the 'Alarm' tab at the bottom, select the alarm you want to edit, then turn on the 'Repeat' option and select the days you want the alarm to go off." This example not only demonstrates how the system can improve its performance through knowledge node vector representation and progressive parameter fine-tuning, but also demonstrates the flexibility and powerful capabilities of the intelligent customer service AI question-and-answer system in practical applications.

[0096] In the process of optimizing deep learning models, especially lightweight large models for specific fields, how to efficiently adjust model parameters to adapt to new tasks or data sets is a key issue. The traditional batch gradient descent method may not fully consider the complex relationship and change trend between model parameters. Therefore, a progressive parameter fine-tuning scheme is introduced, which combines advanced mechanisms such as state snapshots, Hessian matrices, and adaptive learning rates to achieve more refined parameter adjustments and thus improve model performance.

[0097] Optionally, the step of dynamically adjusting the parameters in the lightweight large model using a progressive parameter fine-tuning scheme according to the knowledge node vector representation to obtain a preliminary optimization model includes: Using the knowledge node vector representation as input, the lightweight large model parameters are subjected to initial state analysis and processing through a phased evaluation mechanism to obtain a state snapshot of the lightweight large model parameters; wherein the state snapshot of the lightweight large model parameters is calculated by the following formula: ; in, is a state snapshot of the parameters of the lightweight large model, is a lightweight large model parameter, is the knowledge node vector representation, is the loss function Parameters The gradient of is the Hessian matrix, which is used to capture the second-order derivative information; According to the state snapshot, a set of parameters in the lightweight large model that need to be adjusted is determined, and a fine-tuning step and direction are set for each parameter in the lightweight large model that needs to be adjusted, and a progressive fine-tuning plan is generated; wherein the set of parameters, the fine-tuning step and the direction in the lightweight large model are calculated by the following formula: ; ; ; in, is the set of parameters in the lightweight large model, It is The fine-tuning step size of the parameters, It's the direction. is the loss function, represents a single parameter in a lightweight large model, is the initial learning rate, is the historical gradient weight, It is The historical gradient weight coefficient of the iteration, is the influencing factor of the Hessian matrix, used to enhance the gradient direction, is the loss function Parameters The partial derivative of The impact on losses, is the inverse of the diagonal elements of the Hessian matrix, which is used to adjust the importance of the gradient. is the direction of the dot product of the Hessian matrix and the gradient, which is used to enhance the gradient direction. Loss Function For Parameters The partial derivative of From the first iteration to the Accumulation of historical gradient information of iterations; Based on the progressive fine-tuning plan, the parameters in the lightweight large model that need to be adjusted are updated in small increments and multiple rounds of iterations. After each round of update, the performance change of the lightweight large model is evaluated to obtain an optimized intermediate model; wherein the optimized intermediate model is calculated by the following formula: ; in, represents the optimized intermediate model, Represents the model parameters of the previous iteration; An adaptive learning rate adjustment mechanism is introduced into the iterative update process, and the learning rate of the progressive fine-tuning plan is automatically adjusted according to the performance feedback of the optimized intermediate model to ensure a balance between the speed and accuracy of the parameter adjustment of the optimized intermediate model and obtain a fine adjustment result; wherein the learning rate of the progressive fine-tuning plan is calculated by the following formula: ; in, is the learning rate of the progressive fine-tuning schedule, is the initial learning rate, is the learning rate adjustment factor based on the validation set performance, is the time decay coefficient, It is the gradient variance influencing factor, which is used to dynamically adjust the learning rate according to the gradient variance; is the gradient variance, which measures the change in the gradient; By accumulating the adjustment results, all the optimized intermediate models are screened and processed to finally obtain the preliminary optimization model; wherein the preliminary optimization model is calculated by the following formula: ; in, is the preliminary optimization model, is the Kullback-Leibler divergence, which measures the difference between the distribution of parameters of the new and old models. is the regularization coefficient, which controls the model complexity.

[0098] The purpose of the overall formula is to guide the dynamic adjustment process of lightweight large model parameters through a series of mathematical operations, including: State snapshot: Capture the state information of the current model parameters (including first-order derivatives and second-order derivatives) to better understand the trend of parameter changes. Key parameter selection: Identify the parameter set that has a greater impact on the model output, and set a reasonable fine-tuning step size and direction to ensure the effectiveness of the adjustment. Multi-round iterative update: Perform small-scale, multi-round iterative updates based on the selected key parameters, while monitoring model performance changes to ensure the stability and accuracy of the adjustment process. Adaptive learning rate adjustment: Automatically adjust the learning rate based on model performance feedback to balance adjustment speed and accuracy. Screening the optimal intermediate model: By evaluating all optimized intermediate models, the optimal and stable preliminary optimization model is finally determined.

[0099] The following is a brief explanation of the design reasons for each item in the formula: : Represents the parameters of a lightweight large model. This is the core part of the model and determines the model's behavior and output; : Loss function Parameters The gradient of the parameter The degree of impact on the loss. By calculating the gradient, we can know how to adjust the parameters to reduce the loss and thus improve the model performance; : Hessian matrix, used to capture second-order derivative information. It provides additional information about the trend of parameter changes, helps understand the relationship between parameters, and can be used to accelerate convergence or avoid local minima in some cases; Key parameter set : Select the set of parameters that need to be adjusted most, based on the maximum value of the product of the partial derivative and the inverse diagonal elements of the Hessian matrix. This method ensures that resources are concentrated on those parameters that have the greatest impact on model performance; Fine-tuning step size : The fine-tuning step size formula comprehensively considers the influencing factors of the current gradient, historical gradient, and Hessian matrix. Initial learning rate Control the pace of each update; is the historical gradient weight, allowing the model to remember past information; It is the influencing factor of the Hessian matrix, enhancing the gradient direction and ensuring the correct adjustment direction; direction : The direction is determined by the sign of the partial derivative, indicating whether the parameter should be increased or decreased to reduce the loss. This ensures that each adjustment is made in the direction of reducing the loss; Intermediate Model :Each update is based on the results of the previous round plus the newly calculated step size and direction, gradually approaching the optimal solution; Adaptive learning rate : The learning rate decays gradually over time, but is also affected by the gradient variance. The coefficient controls the adjustment of the gradient variance to the learning rate, ensuring that the learning process is both fast and stable. It is a learning rate adjustment factor based on the performance of the validation set. It dynamically adjusts the learning rate according to the model performance to prevent overfitting or underfitting.

[0100] Preliminary optimization model :The optimal model is selected by minimizing the weighted sum of the loss variance and the Kullback-Leibler divergence. The KL divergence is used to measure the difference between the distribution of parameters of the new and old models to prevent overfitting. is the regularization coefficient, which controls the model complexity and ensures that the model not only performs well on the training data, but also generalizes to unseen data.

[0101] The following formula parameters are obtained in the smart question and answer space: : The parameters of a lightweight large model can be obtained by loading a pre-trained model, and then fine-tuned for the specific task of intelligent question answering; : Loss function Parameters The gradient of , in intelligent question answering, the cross entropy loss function is usually used, and the gradient is calculated through the back propagation algorithm; : Hessian matrix, in practice, its diagonal elements are often used to approximate it, or other efficient methods such as finite difference method are used; Key parameter set : By analyzing the product of partial derivatives and the inverse diagonal elements of the Hessian matrix, select the parameter set that has the greatest impact on the model output; Fine-tuning step size : Initial learning rate Can be set according to experience, , The coefficients need to be adjusted according to the experimental results; direction : The direction is determined by the sign of the partial derivative, indicating whether the parameter should be increased or decreased to reduce the loss; Intermediate Model : Each update is based on the results of the previous round plus the newly calculated step size and direction; Adaptive learning rate : The learning rate decays gradually over time, but is also affected by the gradient variance. The coefficient controls the adjustment of the gradient variance to the learning rate, ensuring that the learning process is both fast and stable; Preliminary optimization model :The optimal model is selected by minimizing the weighted sum of the loss variance and the Kullback-Leibler divergence. KL divergence is used to measure the difference between the distribution of parameters of the new and old models to prevent overfitting; Parameter setting and input value: Suppose there is a lightweight large model that has been pre-trained on general text data, and now you want to apply it to the field of intelligent question answering. You will use the above formula to optimize it.

[0102] State snapshot calculation: First, obtain a knowledge node vector representation of a certain problem , and calculate the model parameters Status snapshot Here, the loss function It can be a cross-extraction loss, which is suitable for classification tasks such as question classification.

[0103] Key parameter selection: Next, determine the set of parameters that need to be adjusted based on the status snapshot , and set the fine-tuning step size for each parameter and direction For example, if a parameter Corresponding to the key features of the problem, it may be selected .

[0104] Multiple rounds of iterative updates: Then, the selected parameters are iterated in small increments and multiple rounds according to the predetermined progressive fine-tuning plan. Assume that after several rounds of iterations, a set of optimized intermediate models are obtained. .

[0105] Adaptive learning rate adjustment: During this process, the system dynamically adjusts the learning rate based on model performance feedback , ensuring the stability and efficiency of the adjustment process.

[0106] Filter the best intermediate model: Finally, select the one with the most stable performance and meets the conditions from all optimized intermediate models. As the final preliminary optimization model.

[0107] To simplify the explanation, assume that only one parameter is considered. , and the following values ​​are known: Initial learning rate ; Time decay coefficient ; Gradient variance influence factor ; Regularization coefficient ; Assume Gradient variance during round selection ; Kullback-Leibler divergence ; Calculate the adaptive learning rate according to the formula ; assumed (i.e. no additional learning rate adjustment), and : ; This calculation shows that as the number of iterations increases, the learning rate gradually decreases, but still remains within a reasonable range, which helps the model to converge stably.

[0108] For the selection of the preliminary optimization model, it is assumed that there are three intermediate models Their loss variances are , the corresponding KL divergence is Then according to the formula: ; The score for each model can be calculated: for ; for ; for ; Obviously, has the lowest score, so it will be selected As the initial optimization model, it not only has the smallest loss variance, but also has a smaller change in the distribution of model parameters, which means that it is relatively more stable and will not deviate too far from the original pre-trained state.

[0109] Figure 2A structural diagram of an intelligent question-answering system based on a lightweight large model is provided for an embodiment of the present application, such as Figure 2 As shown, the device includes: The receiving module 21 is used to parse the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; A positioning module 22 is used to locate the pre-constructed knowledge graph according to the query intent, using the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning, to obtain a set of knowledge nodes that are highly relevant to the natural language question; An adjustment module 23 is used to activate the adaptive context understanding mechanism in the lightweight large model based on the knowledge node set, and dynamically adjust the internal parameter configuration of the lightweight large model by using a progressive parameter fine-tuning scheme, an attention mechanism and a memory network to generate an optimized lightweight large model; A generation module 24 is used to use the optimized lightweight large model, in combination with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain an answer text corresponding to the natural language question; The inspection module 25 is used to perform quality inspection on the answer text. For answer text that does not meet the preset quality standard, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standard.

[0110] Figure 2 The intelligent question answering system based on a lightweight large model can be executed Figure 1 The implementation principle and technical effect of the method of an intelligent question-answering system based on a lightweight large model described in the illustrated embodiment will not be described in detail. The specific manner in which each module and unit performs operations in the intelligent question-answering system based on a lightweight large model in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated here.

[0111] In one possible design, Figure 2 An intelligent question-answering system based on a lightweight large model in the illustrated embodiment can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0112] The processing component 32 is used for: a method for constructing an intelligent question-answering system based on a lightweight large model, characterized in that it includes: parsing the received natural language question to obtain the user's query intent and the knowledge field to which the natural language question belongs; based on the query intent, using the adaptive algorithm corresponding to the knowledge field and the selection mechanism based on deep reinforcement learning, positioning the pre-constructed knowledge graph to obtain a set of knowledge nodes highly related to the natural language question; based on the knowledge node set, activating the adaptive context understanding mechanism in the lightweight large model, using a progressive parameter fine-tuning scheme, an attention mechanism and a memory network, dynamically adjusting the internal parameter configuration of the lightweight large model, and generating an optimized lightweight large model; using the optimized lightweight large model, and combining the method of diversified expression suggestions provided by the adversarial generative network, cooperating with the sequence-to-sequence architecture and the beam search decoding strategy, performing answer generation processing to obtain the answer text corresponding to the natural language question; performing quality inspection processing on the answer text, and for the answer text that does not meet the preset quality standard, triggering a secondary query process, or prompting the user to modify the question description to ensure that the answer text meets the preset quality standard.

[0113] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0114] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0115] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0116] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0117] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0118] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0119] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a method for constructing an intelligent question-answering system based on a lightweight large model.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing an intelligent question answering system based on a lightweight large model, characterized in that: include: Parsing the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; According to the query intent, the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning are used to locate the pre-constructed knowledge graph to obtain a set of knowledge nodes that are highly relevant to the natural language question; Based on the knowledge node set, the adaptive context understanding mechanism in the lightweight large model is activated, and the internal parameter configuration of the lightweight large model is dynamically adjusted by using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to generate an optimized lightweight large model; Using the optimized lightweight large model, combined with the method of diversified expression suggestions provided by the adversarial generative network, with the sequence-to-sequence architecture and beam search decoding strategy, answer generation processing is performed to obtain the answer text corresponding to the natural language question; The answer text is quality checked, and for answer text that does not meet the preset quality standard, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standard.

2. The method according to claim 1, characterized in that: The optimized lightweight large model is combined with the method of diversified expression suggestions provided by the adversarial generative network, and the sequence-to-sequence architecture and beam search decoding strategy are used to perform answer generation processing to obtain the answer text corresponding to the natural language question, including: Using the optimized lightweight large model and combining it with the method of diversified expression suggestions provided by the adversarial generative network, the natural language question is semantically understood and the expression is enriched to obtain candidate answer texts; Based on the sequence-to-sequence architecture, in conjunction with the optimized lightweight large model, the candidate answer text is encoded-decoded to generate a preliminary answer text framework; The preliminary answer text framework is optimized by applying a beam search decoding strategy, and the most likely answer sequence is explored by maintaining multiple possible answer paths to generate an answer text corresponding to the natural language question.

3. The method according to claim 2, characterized in that The sequence-to-sequence architecture is used in conjunction with the optimized lightweight large model to perform encoding-decoding conversion processing on the candidate answer text to generate a preliminary answer text framework, including: According to the optimized lightweight large model and the encoder part in the sequence-to-sequence architecture, the candidate answer text is encoded to convert the candidate answer text into a context vector of a fixed length, wherein the context vector is used to capture the core semantic information of the candidate answer; Based on the context vector, the answer sequence is gradually constructed through the decoder part of the sequence-to-sequence architecture, the probability distribution of the next word is predicted at each time step, and the probability distribution is used to select appropriate words to be added to the answer sequence according to the set strategy to obtain an updated answer sequence; Based on the updated answer sequence, the context vector is updated, and target words are continuously selected according to a set strategy to be added to the answer sequence until a preset maximum sentence length is reached or an end symbol is encountered, so as to generate a preliminary answer text framework; The process of selecting the target vocabulary according to the set strategy includes selecting the vocabulary with the highest probability distribution as the target vocabulary at each time step, or selecting the top k words with the highest probability distribution from the predicted multiple probability distributions and randomly selecting any word from them as the target vocabulary.

4. The method according to claim 2, characterized in that: The application of the beam search decoding strategy to optimize the preliminary answer text framework and explore the most likely answer sequence by maintaining multiple possible answer paths to generate an answer text corresponding to the natural language question includes: Initializing the preliminary answer text frame using a set fixed beam width to obtain an initial candidate answer path set; Based on the initial candidate answer path set, multiple possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set; Introducing a length normalization and diversity penalty mechanism to adjust the gradually expanded answer path set to obtain an optimized answer path, wherein the adjustment process is used to prevent shorter sentences from being preferentially selected due to higher cumulative probability distribution and to generate more diverse answers; The optimized answer path is used to explore the most likely answer sequence, select the answer sequence with the highest cumulative probability distribution, and generate the answer text corresponding to the natural language question.

5. The method according to claim 4, characterized in that The initial answer text framework is initialized using the set fixed beam width to obtain an initial candidate answer path set, including: Set a fixed beam width to k, where the fixed beam width is used to determine the number of candidate answer paths to be retained; Selecting the first k highest probability words from the probability distribution of the words predicted by the decoder part of the sequence-to-sequence architecture as starting words to form k initial candidate answer paths and construct an initial set of candidate answer paths; Based on the initial candidate answer path set, a plurality of possible answer paths are expanded and maintained at each time step to obtain a gradually expanded answer path set, including: For each time step, based on all candidate answer paths generated in the previous time step, for each candidate answer path, use the current context vector to predict the probability distribution of the next word, and obtain the possible subsequent word probability distribution of each candidate answer path; Based on the subsequent word probability distribution, select the top k words with the highest probability from the new words expanded from each candidate answer path, and combine them with the corresponding candidate answer path to form an optimized candidate answer path; From all the optimized candidate answer paths, select the top k optimized candidate answer paths with the highest cumulative probability to continue expanding, ensuring that only the most likely candidate answer paths are retained each time, until the preset maximum sentence length is reached or the end symbol is encountered, and a gradually expanded answer path set is obtained.

6. The method according to claim 1, characterized in that Based on the knowledge node set, the adaptive context understanding mechanism in the lightweight large model is activated, and the internal parameter configuration of the lightweight large model is dynamically adjusted by using a progressive parameter fine-tuning scheme, an attention mechanism and a memory network to generate an optimized lightweight large model, including: Converting the relevance scores of the knowledge node set to obtain a knowledge node vector representation that can capture the knowledge node set; According to the knowledge node vector representation, a progressive parameter fine-tuning scheme is adopted to perform the dynamic adjustment process on the parameters in the lightweight large model to obtain a preliminary optimized model; The attention mechanism is introduced to enhance the preliminary optimization model, and the preliminary optimization model is further optimized based on the memory network to ensure that the preliminary optimization model can retrieve and utilize the knowledge node set when generating the answer text to obtain the optimized lightweight large model.

7. The method according to claim 6, characterized in that The method of dynamically adjusting the parameters in the lightweight large model by using a progressive parameter fine-tuning scheme according to the knowledge node vector representation to obtain a preliminary optimization model includes: Using the knowledge node vector representation as input, the lightweight large model parameters are subjected to initial state analysis processing through a phased evaluation mechanism to obtain a state snapshot of the lightweight large model parameters; Determine the parameters in the lightweight large model that need to be adjusted according to the state snapshot, set a fine-tuning step and direction for each parameter in the lightweight large model that needs to be adjusted, and generate a progressive fine-tuning plan; Based on the progressive fine-tuning plan, the parameters in the lightweight large model that need to be adjusted are updated in small increments and multiple rounds of iterative updates, and the performance changes of the lightweight large model are evaluated after each round of iterative update to obtain an optimized intermediate model; Introducing an adaptive learning rate adjustment mechanism to automatically adjust the learning rate of the progressive fine-tuning plan according to the performance feedback of the optimized intermediate model, thereby ensuring a balance between the speed and accuracy of parameter adjustment of the optimized intermediate model and obtaining an adjustment result; By accumulating the adjustment results, all the optimized intermediate models are screened, and finally the optimized intermediate model with the most stable performance is selected as the preliminary optimization model.

8. An intelligent question answering system based on a lightweight large model, characterized in that: include: A receiving module is used to parse and process the received natural language question to obtain the user's query intention and the knowledge field to which the natural language question belongs; A positioning module is used to locate the pre-built knowledge graph according to the query intent, using the adaptive algorithm corresponding to the knowledge domain and the selection mechanism based on deep reinforcement learning, to obtain a set of knowledge nodes that are highly relevant to the natural language question; An adjustment module is used to activate the adaptive context understanding mechanism in the lightweight large model based on the knowledge node set, and dynamically adjust the internal parameter configuration of the lightweight large model by using a progressive parameter fine-tuning scheme, an attention mechanism, and a memory network to generate an optimized lightweight large model; A generation module is used to use the optimized lightweight large model, in combination with the method of diversified expression suggestions provided by the adversarial generative network, in conjunction with the sequence-to-sequence architecture and the beam search decoding strategy, to perform answer generation processing to obtain the answer text corresponding to the natural language question; The checking module is used to perform quality checking on the answer text. For answer texts that do not meet the preset quality standards, a secondary query process is triggered, or the user is prompted to modify the question description to ensure that the answer text meets the preset quality standards.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a method for constructing an intelligent question-answering system based on a lightweight large model as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a method for constructing an intelligent question-answering system based on a lightweight large model as described in any one of claims 1 to 7 is implemented.

Images