Learning enhanced intelligent question answering method, device and equipment and medium

Abstract

The invention relates to the field of artificial intelligence, and discloses a learning-enhanced intelligent question answering method and device, equipment and a medium, and the method comprises the steps: receiving a question text inputted by a user; the question text is input into a trained learning enhancement question and answer model, the learning enhancement question and answer model comprises a professional vocabulary embedding layer and a large language model, and the learning enhancement question and answer model carries out professional corpus pre-training through the professional vocabulary embedding layer; word embedding processing is carried out on the question text through the professional word embedding layer to obtain professional embedding vectors; word embedding processing is carried out on the question text through the large language model to obtain a universal embedded vector, and after the professional embedded vector and the universal embedded vector are fused, retrieval is carried out to generate an answer text matched with the question text. The question text is processed by the learning enhancement question and answer model which performs professional corpus pre-training by using the professional vocabulary embedding layer, so that the professional vocabulary can be subjected to better semantic processing, and the accuracy and reliability of question and answer under professional contents are improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and financial technology, and in particular to a learning-enhanced intelligent question-answering method, device, equipment and medium. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent question-answering systems have been widely used in various industries. Intelligent question-answering systems in the financial field play an important role in improving user experience and service efficiency. However, traditional question-answering systems often rely on rules and templates, which are difficult to cope with complex and changing user needs, and have high development costs and difficult maintenance.

[0003] In recent years, the emergence of large-scale pre-trained language models (LLMs) has brought new solutions to intelligent question-answering systems. These models learn rich semantic and grammatical knowledge through unsupervised pre-training on large-scale corpora, so that they can handle natural language tasks such as question-answering and dialogue.

[0004] However, existing question-answering systems based on large language models are mainly based on a single language or limited multilingual capabilities, and are unable to cope with diverse professional vocabulary language environments. When answering questions in a variety of professional content environments, it is difficult to accurately analyze and output user questions, reducing the accuracy and reliability of intelligent question-answering. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a learning-enhanced intelligent question-answering method, device, equipment and medium that can be applied to financial technology or other related fields. Its main purpose is to improve the accuracy and reliability of intelligent question-answering under multi-specialized content to meet the intelligent question-answering needs in different specialized contexts.

[0006] The technical solution of the present invention is as follows:

[0007] A first aspect of the present invention provides a learning-enhanced intelligent question-answering method, comprising:

[0008] Receive question text entered by the user;

[0009] Inputting the question text into a trained learning enhanced question answering model, wherein the learning enhanced question answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question answering model is pre-trained with professional corpus through the professional vocabulary embedding layer;

[0010] Performing word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector;

[0011] The question text is word-embedded using the large language model to obtain a universal embedding vector, and the professional embedding vector is fused with the universal embedding vector to generate a search result to generate an answer text that matches the question text.

[0012] A second aspect of the present invention provides a learning-enhanced intelligent question-answering device, comprising:

[0013] Receiving module, semantically receives the question text input by the user;

[0014] An input module, used for inputting the question text into a trained learning enhanced question answering model, wherein the learning enhanced question answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question answering model is pre-trained with professional corpus through the professional vocabulary embedding layer;

[0015] An embedding conversion module, used for performing word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector;

[0016] The answer generation module is used to perform word embedding processing on the question text through the large language model to obtain a universal embedding vector, and to retrieve and generate an answer text matching the question text after fusing the professional embedding vector with the universal embedding vector.

[0017] A third aspect of the present invention provides a computer device, comprising at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned learning-enhanced intelligent question-answering method.

[0020] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned learning-enhanced intelligent question-answering method.

[0021] Beneficial effects: The present invention discloses a learning-enhanced intelligent question-answering method, device, equipment and medium. Compared with the prior art, the embodiment of the present invention receives a question text input by a user; inputs the question text into a trained learning-enhanced question-answering model, the learning-enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning-enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; word embedding processing is performed on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector; word embedding processing is performed on the question text through the large language model to obtain a universal embedding vector, and the professional embedding vector and the universal embedding vector are fused and retrieved to generate an answer text matching the question text. The learning-enhanced question-answering model pre-trained with professional corpus using the professional vocabulary embedding layer processes the question text, so that the professional field problems can be better understood through the professional vocabulary embedding layer, and the stability and accuracy in processing universal language are also ensured through the large language model. The fused professional and universal embedding vectors help to generate more accurate and relevant answers, improve the accuracy and reliability of questions and answers under professional content, and provide a wider range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the scheme in the present invention, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A schematic diagram of an application environment of the learning-enhanced intelligent question-answering method provided in an embodiment of the present invention;

[0024] Figure 2 A flow chart of a learning-enhanced intelligent question-answering method provided by an embodiment of the present invention;

[0025] Figure 3 Another flow chart of the learning-enhanced intelligent question-answering method provided by an embodiment of the present invention;

[0026] Figure 4 A flow chart of a professional corpus pre-training process in a learning-enhanced intelligent question-answering method provided in an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the functional modules of the learning-enhanced intelligent question-answering device provided in an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0030] The learning-enhanced intelligent question-answering method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment of FIG. 1 , it includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0031] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0032] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0033] The server 105 may be a server that provides various services, such as a background server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The background server may analyze and process the received data such as the user request, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0034] It should be noted that the learning-enhanced intelligent question-answering method provided in the embodiment of the present application can generally be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the learning-enhanced intelligent question-answering device provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103. Alternatively, the learning-enhanced intelligent question-answering method provided in the embodiment of the present invention can also generally be executed by the server 105. Accordingly, the learning-enhanced intelligent question-answering device provided in the embodiment of the present invention can generally be set in the server 105.

[0035] It should be understood that the numbers of the above terminal devices, networks and servers are only illustrative. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0036] like Figure 2 As shown, the learning-enhanced intelligent question-answering method provided by the embodiment of the present invention specifically includes the following steps:

[0037] S201, receiving a question text input by a user.

[0038] In this embodiment, when the user interacts with the intelligent question and answer system, the user can input the corresponding questions on the front-end interactive interface. The input form can be text, voice, or picture, etc. The system receives the corresponding question text based on the user's input on the front-end interactive interface. It can be understood that if the user asks a question in voice or picture form, the corresponding question text is obtained through voice-to-text conversion or picture-to-text recognition, so that the system can receive and understand the questions asked by the user in natural language, and provide input data for subsequent processing.

[0039] The question text entered by the user may involve content from one or more professional fields. For example, when the intelligent question-and-answer system is applied to an insurance scenario, the user asks the intelligent question-and-answer system through the front-end interactive interface: "How do I calculate my insurance compensation amount?" This question text involves a single financial insurance field. Based on the diversity of insurance types, users may also ask complex questions involving multiple professional fields. For example, a user asks the intelligent question-and-answer system through the front-end interactive interface: "I was in a car accident. How should the insurance company compensate me for my medical expenses and vehicle losses?" This question text involves compensation processing in the financial insurance field as well as the transportation field, thereby flexibly responding to a variety of professional contexts and providing a more accurate and personalized user experience.

[0040] S202. Input the question text into a trained learning enhanced question-answering model, wherein the learning enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer.

[0041] In this embodiment, the question text input by the user is input into the learning enhanced question-answering model that integrates the professional vocabulary embedding layer and the large language model. The learning enhanced question-answering model has been pre-trained with professional corpus through the professional vocabulary embedding layer. That is, different from the conventional fine-tuning model after the general pre-training large language model, this embodiment also adds a link of integrating professional corpus pre-training after the large language model has completed the general pre-training. And because if the general LLM model framework is directly used for professional corpus pre-training, it may cause the professional corpus to affect the parameters that have been trained in the original general LLM, and because of the small amount of data, the model will be overfitted, resulting in hallucinations in the model during prediction, that is, all answers will be linked to the professional corpus; and the LLM model parameters are too large. If professional corpus pre-training is performed on its basis, the LLM model parameters will be updated, which will also lead to a long training time. Therefore, different from the general pre-training method that directly uses the general LLM model framework for pre-training, this embodiment adds a professional vocabulary embedding layer based on the large language model, and implements the pre-training process of the newly added professional corpus through the professional vocabulary embedding layer. The professional vocabulary embedding layer can better perform separate embedding processing on professional vocabulary, so that the learning enhanced question-answering model can better capture the professional concepts in the question text and provide support for generating accurate answers.

[0042] Specifically, the learning enhanced question-answering model can perform pre-training of professional corpora in one or more professional fields through a professional vocabulary embedding layer to achieve enhanced learning of specialized vocabulary. For example, the model can be pre-trained on professional corpora in the field of finance and insurance alone, or on a combination of professional corpora in the field of finance and insurance and in the field of transportation. The settings can be flexibly selected according to the specific needs of the intelligent question-answering model to balance the question-answering performance and model deployment efficiency.

[0043] S203, performing word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector.

[0044] In this embodiment, for the input question text, the professional vocabulary embedding layer in the learning enhanced question answering model performs word embedding processing on the question text to obtain a professional embedding vector. Specifically, word embedding processing refers to converting the vocabulary in the text into a vector representation in numerical form. In the word embedding process, each word or phrase is mapped to a point in a high-dimensional space, called an embedding vector. These vectors usually have hundreds to thousands of dimensions and are obtained by learning vocabulary patterns in a text corpus. The main purpose of word embedding is to bring semantically similar words close to each other in the vector space, so that the model can capture the similarities and relationships between words.

[0045] In the learning enhanced question-answering model, the question text is embedded in words through the newly added professional vocabulary embedding layer. Since the model is pre-trained with professional corpus through the professional vocabulary embedding layer, the professional terms in the question text can be better processed through the professional vocabulary embedding layer to obtain professional embedding vectors, thereby better understanding problems in professional fields.

[0046] For example, for the question text "I was in a car accident. How should the insurance company compensate me for my medical expenses and vehicle losses?", the learning-enhanced question-answering model is pre-trained on a professional corpus that combines the financial insurance field and the transportation field through a professional vocabulary embedding layer. The professional vocabulary embedding layer can well capture professional vocabulary such as "car accident", "insurance company", "medical expenses" and "vehicle loss" in the question text and generate accurate professional embedding vectors, so that the professional embedding vectors contain professional knowledge in the insurance and transportation fields, which helps the model understand the problem context more accurately.

[0047] S204, performing word embedding processing on the question text through the large language model to obtain a universal embedding vector, and fusing the professional embedding vector with the universal embedding vector to retrieve and generate an answer text matching the question text.

[0048] In this embodiment, on the basis of obtaining a professional embedding vector by performing word embedding processing through a professional vocabulary embedding layer, the entire question text is also processed through a large language model, that is, the question text is subjected to word embedding processing through a large language model to obtain a universal embedding vector, so that the model can understand the context and semantics of the question, and maintain an understanding of the universal language even in the professional background of one or more professional fields.

[0049] In addition, the large language model also fuses professional embedding vectors and general embedding vectors to form a comprehensive semantic representation, for example, through simple concatenation or attention mechanism. The fused embedding vector combines professional knowledge with the understanding of general language, which improves the accuracy of the model's understanding of the problem. Knowledge retrieval is performed based on the fused embedding vector, for example, searching for the most relevant information to the fused embedding vector from a pre-built knowledge base or database. The knowledge base may contain a large number of facts, rules, previous questions and answers, etc., and one or more possible answers are generated according to the context of the question and the retrieved information. When multiple possible answers are generated or multiple relevant information is retrieved, these answers can be further sorted according to relevance, and the answer that best matches the question intent is selected as a response to the question text, that is, an answer text matching the question text is generated, and the answer text is output and displayed on the front-end interactive interface, realizing an accurate question-answering interactive process under professional knowledge content.

[0050] For example, for the question text "I was in a car accident. How should the insurance company compensate me for my medical expenses and vehicle losses?", the large language model obtains a general embedding vector after processing, and then fuses the professional embedding vector with the general embedding vector to search in the relevant legal database, insurance clause database and other knowledge bases to generate the answer text "According to the Civil Code and relevant insurance laws and regulations, car accident compensation usually includes vehicle losses and medical expenses. You should first contact the insurance company to report the accident and provide details of the accident. The insurance company will compensate based on the accident liability determination and your insurance terms. For vehicle losses, damage assessment is required, and repairs or compensation will be made based on the extent of the loss. Medical expenses will be reimbursed based on the actual expenses incurred and the insurance terms."

[0051] In the above embodiment, the present invention discloses a learning-enhanced intelligent question-answering method, which receives a question text input by a user; inputs the question text into a trained learning-enhanced question-answering model, the learning-enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning-enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; the question text is word-embedded through the professional vocabulary embedding layer to obtain a professional embedding vector; the question text is word-embedded through the large language model to obtain a universal embedding vector, and the professional embedding vector and the universal embedding vector are fused and retrieved to generate an answer text matching the question text. The question text is processed by the learning-enhanced question-answering model pre-trained with professional corpus using the professional vocabulary embedding layer, so that the professional field problems can be better understood through the professional vocabulary embedding layer, and the stability and accuracy in processing universal language are also ensured through the large language model. The fused professional and universal embedding vectors help to generate more accurate and relevant answers, improve the accuracy and reliability of questions and answers under professional content, and provide a wider range of application scenarios.

[0052] In one embodiment, Figure 3 As shown, after step S201, the method further includes:

[0053] S301, performing word segmentation processing on the question text to obtain a plurality of words;

[0054] S302, matching the plurality of words with a pre-built professional corpus, wherein the professional corpus contains corpus in at least one professional field;

[0055] S303: Mark each word in the question text as a professional word or a general word according to the matching result.

[0056] In this embodiment, after receiving the question text, the question text is further segmented to obtain several words. For example, for the question text "I had a car accident, how should the insurance company compensate my medical expenses and vehicle losses?", after word segmentation, multiple words such as "I", "had", "car accident", "insurance company", "should", "how", "compensation", "my", "medical", "expenses", "and", "vehicle", and "loss" can be obtained. The word segmentation processing can better identify the keywords in the question and lay the foundation for subsequent matching and processing.

[0057] Match the segmented vocabulary with the corpus entries in the professional corpus, which is a pre-built database containing at least one professional field corpus. For example, the professional corpus built during pre-training can be used directly, and of course, a professional corpus for segmentation matching can also be built separately. For example, the words "car accident", "insurance company", "compensation", "medical expenses" and "vehicle loss" have corresponding professional expressions and related compensation processing rules in the corpus in the insurance field; at the same time, the words "car accident" and "liability" also have related responsibility determination and processing procedures in the corpus in the transportation field. Through vocabulary matching, it is possible to identify the professional knowledge fields involved in the question text, providing support for generating accurate answers.

[0058] Based on the matching results, each word in the question text is marked as a professional word or a general word. For example, "car accident", "compensation", "medical expenses", "vehicle loss", etc. are marked as professional words because they have specific meanings and usages in the fields of insurance and transportation; while "I", "and", "of", etc. are marked as general words. Classification and marking help the model to distinguish which words need to be interpreted and processed based on professional knowledge and which can be processed based on general language when they are subsequently input into the learning enhanced question-answering model. This enables the model to accurately understand and process complex problems involving multiple professional fields and provide users with accurate and professional answers.

[0059] In one embodiment, the word embedding process of the question text through the professional vocabulary embedding layer to obtain a professional embedding vector specifically includes:

[0060] Performing word embedding processing on the professional vocabulary in the question text through the professional vocabulary embedding layer to obtain a professional embedding vector;

[0061] The word embedding process of the question text by the large language model to obtain a universal embedding vector specifically includes:

[0062] The common words in the question text are processed by word embedding through the original embedding layer in the large language model to obtain a common embedding vector.

[0063] In this embodiment, based on the word segmentation and vocabulary matching of the question text, the type of each word in the question text can be obtained, that is, whether each word is a professional word or the same word. Therefore, after the question text is input into the learning enhanced question-answering model, the professional vocabulary embedding layer and the large language model can be used to perform word embedding processing on different types of words respectively to improve the accuracy and efficiency of word embedding processing.

[0064] For example, words such as "car accident", "insurance company", "compensation", "medical expenses" and "vehicle loss" in the question text are matched with the pre-built professional corpus. These words have specific professional meanings in the fields of insurance and transportation. Therefore, these professional words are processed through the professional vocabulary embedding layer to obtain professional embedding vectors. These vectors can capture the semantic information of words in specific fields and provide support for generating field-related answers. The common words in the question text, such as "I", "out", "should", "how", etc., are processed through the original embedding layer in the large language model to obtain universal embedding vectors. These vectors represent the universal language features of the words, which helps the model understand the overall structure and meaning of the question while retaining the universality of the language, so that the answers are not only limited to professional fields, but also conform to common language habits.

[0065] In this embodiment, the learning enhanced question-answering model can effectively embed the professional vocabulary and general vocabulary in the question text, generate embedding vectors for their respective fields, and provide rich semantic information and contextual understanding for subsequent question-answering generation, which can not only improve the accuracy and professionalism of the answers, but also ensure that the answers are natural and fluent, in line with the language habits of ordinary users.

[0066] In one embodiment, Figure 4 As shown, the process of performing professional corpus pre-training on the learning enhanced question answering model through the professional vocabulary embedding layer includes:

[0067] S401, obtaining professional corpus for pre-training from a pre-built professional corpus, wherein the professional corpus contains corpus in at least one professional field;

[0068] S402, inputting the acquired professional corpus into the professional vocabulary embedding layer for embedding conversion, and semantically processing the output of the professional vocabulary embedding layer by completing the general pre-trained large language model;

[0069] S403, fixing the parameters of the large language model, and updating the parameters of the professional vocabulary embedding layer according to the semantic processing result and a preset parameter updating strategy, until the professional corpus pre-training is completed.

[0070] In this embodiment, the learning enhanced question-answering model is pre-trained with professional corpus through the newly added professional vocabulary embedding layer. During pre-training, the professional corpus used for pre-training is first obtained from a pre-built professional corpus. The professional corpus contains corpus in at least one professional field, so that during the pre-training of professional corpus, it can be carried out for a professional field corpus, so that the model specializes in questions and answers in a certain professional field, and it can also be carried out for multiple professional field corpus at the same time, so that the model can give reliable answers when facing multi-field questions. For example, when pre-training is carried out on professional corpus combining the financial insurance field and the transportation field, the corpus in the insurance field and the transportation field can be obtained from the pre-built professional corpus. The corpus in the insurance field may include insurance contract terms, claims process instructions, laws and regulations, etc.; the corpus in the transportation field may include traffic regulations, accident liability determination standards, vehicle maintenance knowledge, etc., to ensure that the model is exposed to professional terms and concepts related to insurance and transportation during the pre-training stage, and to provide rich context information for subsequent word embedding processing.

[0071] The acquired professional corpus is input into the professional vocabulary embedding layer for embedding conversion. For example, words such as "car accident", "compensation", "medical expenses", and "vehicle loss" are converted into vectors that can express their professional meanings in the embedding layer, so that the semantic information of professional vocabulary can be encoded into the vector, providing semantic input in the professional field for the large language model. The output of the professional vocabulary embedding layer will serve as the input of the large language model. The model performs semantic processing on these embedded vectors to understand how the professional vocabulary in the question text is combined with the general language, and uses its pre-trained knowledge on the general language, combined with the semantic information of professional vocabulary, to form a comprehensive understanding of the question text.

[0072] When adjusting parameters during pre-training, the parameters of the large language model are fixed, that is, the parameters of all other modules except the newly added professional vocabulary embedding layer are fixed, and the parameters of the professional vocabulary embedding layer are only updated according to the semantic processing results and the loss function. By fixing the parameters of the large language model, interference with existing general language knowledge can be avoided. At the same time, the model can be optimized specifically for terminology in professional fields, which improves the applicability and accuracy of the model in specific fields.

[0073] In one embodiment, the step of inputting the acquired professional corpus into the professional vocabulary embedding layer for embedding conversion includes:

[0074] Dividing the acquired professional corpus according to preset indicators to obtain a number of professional corpus sets; the preset indicators include field indicators, importance indicators, difficulty indicators and / or time sequence indicators;

[0075] The plurality of professional corpus sets are sequentially input into the professional vocabulary embedding layer for embedding conversion.

[0076] In this embodiment, when the acquired professional corpus is input into the model for pre-training, different pre-training requirements can be met by dividing the professional corpus into data and inputting it in sequence. Specifically, the acquired professional corpus can be divided into data according to preset indicators, which include field indicators, importance indicators, difficulty indicators and / or time sequence indicators, etc., so as to obtain several professional corpus sets. During pre-training, each professional corpus set is sequentially input into the professional vocabulary embedding layer in a certain order for embedding conversion and subsequent semantic analysis and parameter adjustment and other pre-processing, so as to meet different pre-training requirements and improve the effect of pre-training by integrating the model with professional corpus.

[0077] For example, taking the acquisition of professional corpus in the insurance field and the transportation field as an example, when dividing the data, the professional corpus can be divided into corpus in the insurance field and corpus in the transportation field according to the field indicators; first add the corpus in the insurance field to the model for pre-training, and then add the corpus in the transportation field for pre-training.

[0078] You can also divide professional corpora by importance. For example, in the insurance field, the latest regulations may be more important than old regulations. Therefore, after dividing the data by importance, you can first add the most important professional corpus set (such as the latest regulations), and then gradually add the second most important professional corpus set, and so on, to ensure that the model learns the most critical information first.

[0079] You can also divide professional corpora by difficulty. For example, in the insurance field, a simple claims process is simpler than a complex claims process. Therefore, after dividing the data by difficulty, you can first add easier professional corpus sets (such as simple claims processes) to allow the model to master the basics first, and then gradually add more difficult data professional corpus sets (such as complex claims processes).

[0080] You can also divide professional corpora in chronological order. For example, in the insurance field, relevant news reports are very timely. Therefore, after dividing the data in chronological order, you can first add older professional corpus sets (such as older news data) for pre-training, and then gradually add more recent professional corpus sets (such as the latest news data) to ensure that the model can capture the latest trends and changes.

[0081] In one embodiment, the updating of the parameters of the professional vocabulary embedding layer according to the semantic processing result and the preset parameter updating strategy includes:

[0082] Calculate the parameter update amount of the professional vocabulary embedding layer for the current input vocabulary according to the semantic processing result and the loss function;

[0083] If the current input vocabulary only exists in the professional corpus, then the parameters of the professional vocabulary embedding layer are updated according to the complete parameter update amount;

[0084] If the current input vocabulary exists in both the professional corpus and the general corpus, the parameters of the professional vocabulary embedding layer are updated according to a preset ratio and parameter update amount.

[0085] In this embodiment, based on the semantic processing results and loss functions, such as cross entropy loss or mean square error loss, the parameter update amount of the professional vocabulary embedding layer for the current input vocabulary is calculated, and the corresponding parameter update is flexibly performed based on the corpus position of the current input vocabulary. If vocabulary such as "insurance clause" or "traffic accident liability" only appears in the professional corpus, the model will update the embedding vectors of these vocabulary according to the complete parameter update amount, ensuring that the model can fully learn and adapt to the unique concepts and terms in the professional field, and improve the professionalism and accuracy of the model in these fields. If vocabulary such as "expense" or "loss" appears in both professional corpus and general corpus, the model will be updated according to the preset ratio and the calculated parameter update amount. By updating in proportion, the model can adapt to the specific usage of the professional field while maintaining the understanding of the general language, achieving a balance between professional field knowledge and general language knowledge.

[0086] In one embodiment, if the current input vocabulary exists in both the professional corpus and the general corpus, the parameters of the professional vocabulary embedding layer are updated according to a preset ratio and parameter update amount, including:

[0087] If the current input vocabulary exists in both the professional corpus and the general corpus, determining the number n of data sets in the professional corpus;

[0088] The parameters of the professional vocabulary embedding layer are updated according to 1 / n of the parameter update amount.

[0089] In this embodiment, when processing the current input vocabulary, if the current input vocabulary exists in both the professional corpus and the general corpus, the occurrence of the vocabulary in the professional corpus is first confirmed, and the number of data sets related to the vocabulary is counted. For example, if "car accident" appears in three different data sets (insurance field data, traffic field data, and medical field data), then n=3. By determining the number of data sets, the model can better understand the diversity of the vocabulary in the professional field. According to the determined n value, the calculated parameter update amount is divided by n, and 1 / n of the total parameter update amount is applied to the parameter update of the professional vocabulary embedding layer. Assume that in one training iteration, the model calculates that the amount by which the embedding vector of "car accident" should be updated is Δ according to the loss function. Under normal circumstances, the model will apply the entire Δ to the embedding vector, but because "car accident" has appeared in all three data sets, the model will only apply 1 / 3 of Δ (i.e., Δ / 3) to ensure that the update is not overly biased towards any one data set. By updating in proportion, the model can gradually adapt to the specific usage in professional fields while retaining general language knowledge. Controlling the update weights between general and professional corpora can prevent the model from overfitting to a specific corpus and ensure its flexibility and accuracy in dealing with a variety of professional terms.

[0090] It should be noted that there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0091] Further references Figure 5 , as a response to the above Figure 2 The present invention provides an embodiment of a learning-enhanced intelligent question-answering device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0092] like Figure 5 As shown, the learning-enhanced intelligent question-answering device 50 described in this embodiment includes:

[0093] Receiving module 501, semantically receiving question text input by the user;

[0094] An input module 502 is used to input the question text into a trained learning enhanced question answering model, wherein the learning enhanced question answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question answering model is pre-trained with professional corpus through the professional vocabulary embedding layer;

[0095] An embedding conversion module 503, configured to perform word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector;

[0096] The answer generation module 504 is used to perform word embedding processing on the question text through the large language model to obtain a universal embedding vector, and to retrieve and generate an answer text matching the question text after fusing the professional embedding vector with the universal embedding vector.

[0097] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions, which are more suitable for describing the execution process of learning-enhanced intelligent question-answering than programs. For the specific implementation methods of each module, please refer to the corresponding method embodiments above, which will not be repeated here.

[0098] In one embodiment, the device 50 further includes:

[0099] A word segmentation module is used to perform word segmentation processing on the question text to obtain a plurality of words;

[0100] A vocabulary matching module, used for matching the plurality of vocabulary with a pre-built professional corpus, wherein the professional corpus contains corpus in at least one professional field;

[0101] The marking module is used to mark each word in the question text as a professional word or a general word according to the matching result.

[0102] In one embodiment, the embedding conversion module 503 includes:

[0103] A professional conversion unit, used for performing word embedding processing on the professional vocabulary in the question text through the professional vocabulary embedding layer to obtain a professional embedding vector;

[0104] The answer generation module 504 includes:

[0105] A universal conversion unit is used to perform word embedding processing on the universal vocabulary in the question text through the original embedding layer in the large language model to obtain a universal embedding vector.

[0106] In one embodiment, the device 50 further includes:

[0107] A corpus acquisition module, used to acquire professional corpus for pre-training from a pre-built professional corpus, wherein the professional corpus contains corpus in at least one professional field;

[0108] A professional corpus pre-training module is used to input the acquired professional corpus into the professional vocabulary embedding layer for embedding conversion, and to perform semantic processing on the output of the professional vocabulary embedding layer by completing the general pre-trained large language model;

[0109] The parameter updating module is used to fix the parameters of the large language model and update the parameters of the professional vocabulary embedding layer according to the semantic processing results and a preset parameter updating strategy until the professional corpus pre-training is completed.

[0110] In one embodiment, the professional corpus pre-training module includes:

[0111] A data division unit is used to divide the acquired professional corpus according to preset indicators to obtain a plurality of professional corpus sets; the preset indicators include field indicators, importance indicators, difficulty indicators and / or time sequence indicators;

[0112] The sequential input unit is used to sequentially input the plurality of professional corpus sets into the professional vocabulary embedding layer for embedding conversion.

[0113] In one embodiment, the parameter updating module includes:

[0114] A calculation unit, used for calculating the parameter update amount of the professional vocabulary embedding layer for the current input vocabulary according to the semantic processing result and the loss function;

[0115] A first updating unit, configured to update the parameters of the professional vocabulary embedding layer according to the complete parameter update amount if the current input vocabulary only exists in the professional corpus;

[0116] The second updating unit is used to update the parameters of the professional vocabulary embedding layer according to a preset ratio and parameter update amount if the current input vocabulary exists in both the professional corpus and the general corpus.

[0117] In one embodiment, the second updating unit includes:

[0118] a data set determining unit, configured to determine the number n of data sets in the professional corpus if the current input vocabulary exists in both the professional corpus and the general corpus;

[0119] An updating unit is used to update the parameters of the professional vocabulary embedding layer according to 1 / n of the parameter update amount.

[0120] In the above embodiment, the present invention discloses a learning-enhanced intelligent question-answering device, which receives a question text input by a user; inputs the question text into a trained learning-enhanced question-answering model, the learning-enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning-enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; the question text is word-embedded through the professional vocabulary embedding layer to obtain a professional embedding vector; the question text is word-embedded through the large language model to obtain a universal embedding vector, and the professional embedding vector and the universal embedding vector are fused and retrieved to generate an answer text that matches the question text. The question text is processed by the learning-enhanced question-answering model pre-trained with professional corpus using the professional vocabulary embedding layer, so that the professional field problems can be better understood through the professional vocabulary embedding layer, and the stability and accuracy in processing universal language are also ensured through the large language model. The fused professional and universal embedding vectors help to generate more accurate and relevant answers, improve the accuracy and reliability of questions and answers under professional content, and provide a wider range of application scenarios.

[0121] Another embodiment of the present invention provides a computer device, such as Figure 6 As shown, the computer device 60 includes:

[0122] One or more processors 601 and memory 602, Figure 6 A processor 601 is used as an example for the description. The processor 601 and the memory 602 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0123] Processor 601 is used to complete various control logics of computer device 60, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component or any combination of these components. In addition, processor 601 can also be any traditional processor, microprocessor or state machine. Processor 601 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0124] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the learning-enhanced intelligent question-answering method in the embodiment of the present invention. The processor 601 executes various functional applications and data processing of the computer device 60 by running the non-volatile software programs, instructions and units stored in the memory 602, that is, implementing the learning-enhanced intelligent question-answering method in the above method embodiment.

[0125] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device 60, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 601, and these remote memories may be connected to the computer device 60 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 602, and when executed by one or more processors 601, the steps of learning the enhanced intelligent question-answering method in any of the above-mentioned method embodiments are executed.

[0126] In the above embodiment, the present invention discloses a computer device, which receives a question text input by a user; inputs the question text into a trained learning enhanced question-answering model, the learning enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; the question text is word-embedded through the professional vocabulary embedding layer to obtain a professional embedding vector; the question text is word-embedded through the large language model to obtain a universal embedding vector, and the professional embedding vector and the universal embedding vector are fused and retrieved to generate an answer text matching the question text. The learning enhanced question-answering model pre-trained with professional corpus using the professional vocabulary embedding layer processes the question text, so that the professional field problems can be better understood through the professional vocabulary embedding layer, and the stability and accuracy in processing universal language are also ensured through the large language model. The fused professional and universal embedding vectors help to generate more accurate and relevant answers, improve the accuracy and reliability of questions and answers under professional content, and provide a wider range of application scenarios.

[0127] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of learning the enhanced intelligent question-answering method in any of the above method embodiments are executed.

[0128] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which receives a question text input by a user; inputs the question text into a trained learning enhanced question-answering model, the learning enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; the question text is word-embedded through the professional vocabulary embedding layer to obtain a professional embedding vector; the question text is word-embedded through the large language model to obtain a universal embedding vector, and the professional embedding vector and the universal embedding vector are fused and retrieved to generate an answer text that matches the question text. The learning enhanced question-answering model that has been pre-trained with professional corpus using the professional vocabulary embedding layer processes the question text, so that the professional field problems can be better understood through the professional vocabulary embedding layer, and the stability and accuracy in processing universal language are also ensured through the large language model. The fused professional and universal embedding vectors help to generate more accurate and relevant answers, improve the accuracy and reliability of questions and answers under professional content, and provide a wider range of application scenarios.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0130] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0131] In summary, the present invention discloses a learning-enhanced intelligent question-answering method, device, equipment and medium, the method includes: receiving a question text input by a user; inputting the question text into a trained learning-enhanced question-answering model, the learning-enhanced question-answering model includes a professional vocabulary embedding layer and a large language model, and the learning-enhanced question-answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; word embedding processing is performed on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector; word embedding processing is performed on the question text through the large language model to obtain a universal embedding vector, and the professional embedding vector and the universal embedding vector are fused and retrieved to generate an answer text matching the question text. The learning-enhanced question-answering model pre-trained with professional corpus using the professional vocabulary embedding layer is used to process the question text, so that the professional field problems can be better understood through the professional vocabulary embedding layer, and the stability and accuracy in processing universal language are also ensured through the large language model. The fused professional and universal embedding vectors help to generate more accurate and relevant answers, improve the accuracy and reliability of questions and answers under professional content, and provide a wider range of application scenarios.

[0132] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0133] It should be noted that if any software tools or components other than those of the Company appear in the embodiments of the present application, they are merely used for illustration and do not represent actual use. It should be understood that the application of the present invention is not limited to the above examples, and that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes shall fall within the scope of protection of the claims attached to the present invention.

Claims

1. A learning-enhanced intelligent question-answering method, characterized in that: include: Receive question text entered by the user; Inputting the question text into a trained learning enhanced question answering model, wherein the learning enhanced question answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; Performing word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector; The question text is word-embedded using the large language model to obtain a universal embedding vector, and the professional embedding vector is fused with the universal embedding vector to generate a search result to generate an answer text that matches the question text.

2. The learning-enhanced intelligent question-answering method according to claim 1, characterized in that: After receiving the question text input by the user, the method further includes: Performing word segmentation processing on the question text to obtain a number of words; Matching the plurality of words with a pre-built professional corpus, wherein the professional corpus contains corpus in at least one professional field; Each word in the question text is marked as a professional word or a general word according to the matching result.

3. The learning-enhanced intelligent question-answering method according to claim 2, characterized in that: The step of performing word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector specifically includes: The professional vocabulary in the question text is processed by word embedding through the professional vocabulary embedding layer to obtain a professional embedding vector; The word embedding process of the question text by the large language model to obtain a universal embedding vector specifically includes: The common words in the question text are processed by word embedding through the original embedding layer in the large language model to obtain a common embedding vector.

4. The learning-enhanced intelligent question-answering method according to claim 1, characterized in that: The process of the learning enhanced question answering model performing professional corpus pre-training through the professional vocabulary embedding layer includes: Acquire professional corpus for pre-training from a pre-built professional corpus, wherein the professional corpus contains corpus in at least one professional field; Input the acquired professional corpus into the professional vocabulary embedding layer for embedding conversion, and perform semantic processing on the output of the professional vocabulary embedding layer by completing the general pre-trained large language model; The parameters of the large language model are fixed, and the parameters of the professional vocabulary embedding layer are updated according to the semantic processing results and a preset parameter update strategy until the professional corpus pre-training is completed.

5. The learning-enhanced intelligent question-answering method according to claim 4, characterized in that: The step of inputting the acquired professional corpus into the professional vocabulary embedding layer for embedding conversion includes: Dividing the acquired professional corpus according to preset indicators to obtain a number of professional corpus sets; the preset indicators include field indicators, importance indicators, difficulty indicators and / or time sequence indicators; The plurality of professional corpus sets are sequentially input into the professional vocabulary embedding layer for embedding conversion.

6. The learning-enhanced intelligent question-answering method according to claim 4, characterized in that: The updating of the parameters of the professional vocabulary embedding layer according to the semantic processing result and the preset parameter updating strategy includes: Calculate the parameter update amount of the professional vocabulary embedding layer for the current input vocabulary according to the semantic processing result and the loss function; If the current input vocabulary only exists in the professional corpus, then the parameters of the professional vocabulary embedding layer are updated according to the complete parameter update amount; If the current input vocabulary exists in both the professional corpus and the general corpus, the parameters of the professional vocabulary embedding layer are updated according to a preset ratio and parameter update amount.

7. The learning-enhanced intelligent question-answering method according to claim 6, characterized in that: If the current input vocabulary exists in both the professional corpus and the general corpus, the parameters of the professional vocabulary embedding layer are updated according to a preset ratio and parameter update amount, including: If the current input vocabulary exists in both the professional corpus and the general corpus, determining the number n of data sets in the professional corpus; The parameters of the professional vocabulary embedding layer are updated according to 1 / n of the parameter update amount.

8. A learning-enhanced intelligent question-answering device, characterized in that: include: Receiving module, semantically receives the question text input by the user; An input module, used for inputting the question text into a trained learning enhanced question answering model, wherein the learning enhanced question answering model includes a professional vocabulary embedding layer and a large language model, and the learning enhanced question answering model is pre-trained with professional corpus through the professional vocabulary embedding layer; An embedding conversion module, used for performing word embedding processing on the question text through the professional vocabulary embedding layer to obtain a professional embedding vector; The answer generation module is used to perform word embedding processing on the question text through the large language model to obtain a universal embedding vector, and to retrieve and generate an answer text matching the question text after fusing the professional embedding vector with the universal embedding vector.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the learning-enhanced intelligent question-answering method described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the learning-enhanced intelligent question-answering method described in any one of claims 1-7.

Images