Domain Retrieval Methods Based on Meta-Learning and Knowledge Enhancement

Abstract

This invention provides a domain retrieval method based on meta-learning and knowledge enhancement, comprising: S1, acquiring corpus data of a specific knowledge domain, and constructing a text matching triplet dataset and a keyword extraction dataset for that knowledge domain; S2, constructing and training a keyword extraction model and a text vector model using a pre-trained model; S3, parsing the document to be analyzed to generate structured question-and-answer text pairs in three dimensions, and storing them in a database; S4, after inputting a user question, accurately matching the user question with the question-and-answer text pairs in the database using the trained keyword extraction model and text vector model to obtain the question-and-answer retrieval results. This invention can be applied to different knowledge domains, achieving efficient and low-resource text vector model training and accurate user question-and-answer matching, providing a scientific and effective technical solution for various practical applications such as specific domain knowledge organization, knowledge question-and-answer database establishment, and solution provision.

Description

Technical Field

[0001] This invention relates to the field of knowledge question answering, and in particular to a domain retrieval method based on meta-learning and knowledge enhancement. Background Technology

[0002] In recent years, with the rapid development of the Internet, the amount of information has exploded. In order to retrieve and utilize this massive amount of information more efficiently, retrieval methods have become an important background technology.

[0003] Traditional search methods primarily rely on keyword matching, where users input keywords and the system returns related documents. However, this approach has limitations. First, it struggles to accurately understand user intent, as users often use different keywords to describe the same concept. Second, keyword matching cannot process semantic information within documents, potentially leading to inaccurate results. Furthermore, in certain specific domains, relying solely on keyword matching often falls short of expectations. Summary of the Invention

[0004] To address the problems existing in current technologies, this paper presents a domain retrieval method based on meta-learning and knowledge enhancement. This method converts text into high-dimensional vector representations and determines semantic relationships between texts by calculating the similarity between vectors. By converting user queries and documents into vector representations, more accurate semantic matching can be achieved, improving the accuracy and efficiency of retrieval. This leads to a better understanding of the user's query intent and provides more precise search results. It offers a scientific and effective technical solution for various practical applications, including domain-specific knowledge organization, knowledge question-answering database construction, and solution provision.

[0005] The technical solution adopted in this invention is as follows: a domain retrieval method based on meta-learning and knowledge enhancement, comprising:

[0006] Step S1: Obtain corpus data for a certain knowledge domain, and label the text matching information and keyword information of the documents to be analyzed, and construct a text matching triplet dataset and a keyword extraction dataset for that knowledge domain.

[0007] Step S2: Construct and train the keyword extraction model and text vector model using the pre-trained model;

[0008] Step S3: Parse the table of contents of the document to be analyzed, generate structured question-and-answer text pairs in three dimensions: sentence level, heading level, and point level, and store them in the database;

[0009] Step S4: After the user inputs the question, the trained keyword extraction model and text vector model are used to accurately match the user's question with the question-and-answer text pairs in the database, and the question-and-answer retrieval results are obtained.

[0010] Furthermore, in step S1, the specific sub-steps for constructing the text matching triplet dataset and keyword extraction dataset under a certain knowledge domain include:

[0011] Step S1.1: Integrate documents from a specific knowledge domain, parse the document information, segment the text in the documents by periods, and construct a sentence-level corpus;

[0012] Step S1.2: Calculate the vector value of each text in the sentence-level corpus, as well as the cosine similarity between the text and other texts. Select one text from the top n texts in the cosine similarity score as the positive sample of the text, and select one text from the bottom n texts in the cosine similarity score as the negative sample of the text. The text itself, the positive sample and the negative sample of the text constitute the text matching triplet data. Summarize all the text matching triplet data to form the text matching triplet dataset.

[0013] Step S1.3: Assign a label to each character in each text. The label includes the keyword start position, middle / end position and non-keyword position, forming keyword extraction data of the text composed of characters and corresponding labels. Summarize the keyword extraction data of all texts to form a keyword extraction dataset.

[0014] Furthermore, in step S2, the keyword extraction model construction and training process includes:

[0015] Step S2.1.1: Construct a keyword extraction model, which includes a text embedding layer, a text encoding layer, a feature extraction layer and a keyword extraction layer connected in sequence.

[0016] Step S2.1.2: Obtain the keyword extraction data from the keyword extraction dataset, concatenate the characters in the keyword extraction data into a text sequence, and then convert it into a character sequence;

[0017] Step S2.1.3: Input the character sequence into the text embedding layer to obtain the embedded representation of the character sequence;

[0018] Step S2.1.4: Input the embedding representation into the text encoding layer and model the embedding representation to obtain its hidden layer representation;

[0019] Step S2.1.5: In the feature extraction layer, BiLSTM is used to extract features from the hidden layer representation to obtain the context encoding of the text sequence;

[0020] Step S2.1.6: Obtain the keyword extraction data from the keyword extraction dataset, concatenate the labels in the keyword extraction data into a label sequence, and obtain the emission probability and transition matrix based on the context encoding of the corresponding text sequence. Construct a scoring function to calculate the corresponding score of the text sequence label, and then use the softmax function to calculate the probability of the output predicted label sequence.

[0021] Step S2.1.7: Based on the probability obtained in step S2.1.6, use its negative log-likelihood function as the loss function to train the keyword extraction model.

[0022] Furthermore, the specific steps of step S2.1.5 include:

[0023] Step S2.1.5.1: Obtain the i-th hidden vector in the hidden vector representation and the hidden vectors of the previous and next positions. Define the first weight matrix, the second weight matrix and the first bias. Calculate the feature information of the character at the current position i on the left and right sides of the semantic space using the Sigmoid function.

[0024] Step S2.1.5.2: Define the third weight matrix and the second bias. Use the Tanh function to perform a nonlinear transformation on the feature information on the left and right sides of the obtained semantic space to obtain the hidden features of the character at the current position i on the left and right sides of the semantic space and concatenate them into the final feature vector, which is the context encoding of the i-th character in the text sequence. Repeat this process for all hidden vectors to obtain the context encoding of the text sequence.

[0025] Furthermore, in step S2, the text vector model is constructed based on the Roberta model, which is used as a shared encoder, and the text vector model is trained using two sub-tasks: triple difference judgment and text consistency judgment.

[0026] Furthermore, in step S2, the training process for the triplet difference judgment task includes:

[0027] Step S2.2.1: Based on the trained keyword extraction model, extract the text keywords from the triplet data, and then concatenate the sentence-level and word-level text information to obtain the original text and its similar and dissimilar texts.

[0028] Step S2.2.2: Obtain the original text and the text vectors corresponding to its similar and dissimilar texts, and establish a triplet loss. Optimize and train the text vector model parameters using the triplet loss.

[0029] Furthermore, in step S2, the training process for the text consistency judgment task includes:

[0030] Step S2.3.1: Based on the existing natural language inference dataset, obtain the text pair for each data point, generate the text vector of the text pair, and then calculate the cosine similarity between the text vectors of the text pair.

[0031] Step S2.3.2: Establish mean squared error loss based on cosine similarity, and optimize the text vector model parameters using mean squared error loss.

[0032] Furthermore, the specific sub-steps of step S3 include:

[0033] Step S3.1: Parse the text format in the document to be analyzed, obtain the font size, color, bold / italic information of each text segment, so as to distinguish the body text and headings of the document, and parse out the hierarchical structure of the headings to generate the document table of contents;

[0034] Step S3.2: Generate hierarchical heading-paragraph data based on the document's table of contents and content;

[0035] Step S3.3: Based on the title-paragraph data, the document content is divided into text sentences according to the period and assigned an ID. Combined with the information of the title, the sentence-level structured data is generated.

[0036] Step S3.4: Generate structured data at the heading level based on the heading-paragraph data;

[0037] Step S3.5: Based on the title-paragraph data, extract the text segments containing point-by-point explanations in the document content, summarize and categorize the content of the text segments to obtain the corresponding segment themes, and generate structured data at the point level.

[0038] Step S3.6: Standardize the format of the structured data at the sentence level, the heading level, and the point level;

[0039] Step S3.7: Divide the structured data after standardization into two forms: question and answer, to form structured question-and-answer text pairs; wherein, the structured data in question form includes question ID and question, and the structured data in answer form includes question ID, text content, the parent title ID of the current text, and sentence level / subtitle ID;

[0040] Step S3.8: Use the trained retrieval model to generate corresponding text vectors for the structured data in question form, and store them in the vector database along with the ID information; store the structured data in answer form in the relational database.

[0041] Furthermore, the specific sub-steps of step S4 include:

[0042] Step S4.1: Obtain keywords from user questions through keyword extraction model, concatenate them with user questions, generate text vectors using text vector model, and calculate the inner product with the text vectors stored in the vector database and sort them from largest to smallest.

[0043] Step S4.2: Calculate the cosine similarity between the text vector of the user's question and the top n text vectors with the calculated inner product, then re-sort them and select the top n structured data in question form with the highest cosine similarity as candidate questions.

[0044] Step S4.3: Based on the ID corresponding to the candidate question, obtain its corresponding structured data in the form of the answer in the relational database. If the parent title ID of the current text of the candidate question is an empty list, its question matching score with the user question is the cosine similarity score between the two. If the parent title ID of the current text of the candidate question is not an empty list, query the relational database for the parent question whose candidate question ID is equal to the parent title ID of the current text, and calculate the cosine similarity score between the parent question and the user question. Cosine similarity score Cosine similarity score A weighted sum is used as the question matching score between candidate questions and user questions;

[0045] Step S4.4: Obtain the question matching scores of all candidate questions and user questions, sort them from largest to smallest, select the candidate question corresponding to the first ranked question as the real matching question, and retrieve the matching answer from the relational database according to the corresponding ID; based on the real matching question, if the corresponding sentence level / subheading ID is not an empty list, the final answer is the answer obtained from the relational database; if the corresponding sentence level / subheading ID is an empty list, the position of the real matching question's corresponding ID in the list of all text sentence IDs is used to query and retrieve the question text at the ID-1 position, the question text at the ID position, and the question text at the ID+1 position from the relational database, and concatenate them sequentially to obtain the final answer;

[0046] Step S4.1: Provide the final answer as the matching answer to the user's question.

[0047] Furthermore, in step S2.2.2, the triplet loss is specifically as follows:

[0048]

[0049]

[0050]

[0051] in, For the text vector of the original text, For text vectors of similar texts, For dissimilar text, the text vectors are... This represents the distance between the text vector of the original text and the text vectors of similar texts. This represents the distance between the text vector of the original text and the text vectors of dissimilar texts, where n represents the length of the text sequence. This represents the boundary distance.

[0052] Compared with existing technologies, the beneficial effects of adopting the above technical solution are as follows: the present invention can be applied to different knowledge domains, realize efficient and low-resource text vector model training and accurate user question-answer matching, and provide scientific and effective technical solutions for multiple practical applications such as knowledge organization in specific domains, knowledge question-answer base establishment, and solution provision. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the domain retrieval method based on meta-learning and knowledge enhancement in an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the keyword extraction model structure in an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the text vector generation process for knowledge enhancement using a keyword extraction model and a text vector model in an embodiment of the present invention. Detailed Implementation

[0056] The embodiments of this application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar modules or modules having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. Rather, the embodiments of this application include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0057] To address the issue that traditional methods relying solely on keyword matching often fail to achieve the desired results in certain domains, a domain retrieval method based on meta-learning and knowledge enhancement is proposed. This method converts text into high-dimensional vector representations and determines semantic relationships between texts by calculating the similarity between vectors. By converting user queries and documents into vector representations, more accurate semantic matching can be achieved, improving retrieval accuracy and efficiency, thereby better understanding the user's query intent and providing more precise search results. This provides a scientific and effective technical solution for various practical applications, such as domain-specific knowledge organization, knowledge question-answering base construction, and solution provision. The specific solution is as follows:

[0058] Please refer to Figures 1-3 This embodiment proposes a domain retrieval method based on meta-learning and knowledge enhancement, including the following steps:

[0059] Step S1: Collect corpus data for a specific knowledge domain, annotate the text matching information and keyword information of the documents to be analyzed, and construct a text matching triplet dataset and a keyword extraction dataset for that knowledge domain.

[0060] Step S2: Construct and train the keyword extraction model and text vector model using the pre-trained model;

[0061] Step S3: Parse the document to be analyzed to generate structured question-and-answer text pairs in three dimensions: sentence level, heading level, and point level, and store them in a vector database and a relational database;

[0062] Step S4: Utilize the trained keyword extraction model and text vector model under the knowledge domain to achieve accurate matching between user questions and question-answer text pairs in the constructed database, and obtain question-answer retrieval results.

[0063] Specifically, in step S1, constructing a text matching triplet dataset and a keyword extraction dataset under a certain knowledge domain includes the following steps:

[0064] Step S1.1: Integrate journal articles, news reports and other documents in a certain knowledge domain, parse the document information, and segment the text in the document according to the period to construct a sentence-level corpus;

[0065] Step S1.2: Calculate the value of each text in the sentence-level corpus constructed in step S1.1 using a general text vector generation model. The vector value is then used; subsequent internal retrieval involves iterating through the texts, calculating their cosine similarity, and randomly selecting one text from the top-n ranked texts by cosine similarity score and the next n ranked texts as vectors. Positive samples and negative samples Secondly, if and If keywords in the text have synonyms, they are randomly replaced to generate text matching triplet data. :

[0066]

[0067] Summarize and construct a text matching triplet dataset ;

[0068] Step S1.3: Based on the text content of the corpus, use sequence labeling tools to process the text. Every word Assigning labels , It includes three types: B, I, and O, representing the start position of the keyword, the middle / end position, and the non-keyword position, respectively. The keyword information in the text, where n is the length of the text sequence, is used to generate keyword extraction data. :

[0069]

[0070] Summarize and construct the keyword extraction dataset .

[0071] In one embodiment, such as Figure 2 As shown, the keyword extraction model consists of the following layers in sequence: text embedding layer, text encoding layer, feature extraction layer, and keyword extraction layer.

[0072] In a preferred embodiment, the text embedding layer and text encoding layer in the keyword extraction model are based on the Roberta model.

[0073] This embodiment provides a training method for the keyword extraction model, as follows:

[0074] A1. Obtain the keyword extraction dataset constructed in step S1.3. In ,Will And concatenate them into a text sequence The character sequence is obtained through the tokenizer. , where n is the length of the text sequence;

[0075] A2. The character sequence generated in A1 After the text embedding layer, obtain Embedded representation ;

[0076] A3. The embedded representation generated in A2 After the text encoding layer, the embedded representation Modeling is performed to obtain the hidden layer representation of the sequence. , The unfolded form is ,in, Represents a text sequence The hidden layer vector at the i-th position;

[0077] A4. The feature extraction layer uses BiLSTM to represent the hidden layer of the text generated in A3. Feature extraction is performed.

[0078] In this embodiment, BiLSTM consists of two LSTM layers:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084]

[0085]

[0086] The specific extraction process includes: first obtaining... and the hidden layer vectors of its preceding and following positions. , Define the weight matrix , With bias They are then used to calculate the semantic features of the character at position i in the semantic space using the Sigmoid function. , The weight matrix is ​​defined subsequently. With bias Use the Tanh function to , Perform a nonlinear transformation to obtain the hidden features of both. , And concatenate them into the final feature vector. , This represents the context encoding of the i-th character in the text sequence, and the overall context encoding of the final text sequence. for ;

[0087] A5. In the keyword extraction layer, obtain the keyword extraction dataset from step S1.3. In In And concatenate them into a label sequence. Subsequent context encoding based on A4 Obtain the emission probability P and the transition matrix. And define a scoring function S to calculate the corresponding score of the text sequence label, where For the predicted label sequence , represent Transferred to The score, Represents a text sequence The i-th character Predict as label The probability of:

[0088]

[0089] Then, the Softmax function is used to calculate the output label sequence. The probability of, where For the set of all possible label sequences:

[0090]

[0091] A6. Use the negative log-likelihood function as the loss function to train the model:

[0092]

[0093] In one embodiment, the text vector model is based on the Roberta model, which is used as a shared encoder to obtain text-specific information within a specific knowledge domain.

[0094] In this embodiment, the text vector model is trained using two sub-tasks: triplet difference judgment and text consistency judgment, based on the meta-learning concept.

[0095] Specifically, the training process for the triplet difference judgment task includes the following steps:

[0096] B1. Based on the trained keyword extraction model, extract text keywords from the triplet data, and concatenate sentence-level and word-level text information. , , This is to enhance the semantic information of the text, among which , , These represent the original text, its similar text, and its dissimilar text, respectively. , , These represent keyword groups extracted from the above text, where... This indicates that the given text information will be concatenated into a single text in sequence. For a special separator:

[0097]

[0098]

[0099]

[0100] For example, concatenate('you', [sep],'I') returns 'you[sep]I'.

[0101] B2, Obtain , , text vector , , In order to narrow down and Push away at the same distance and The distance, i.e. Using triplet loss The model parameters are optimized, where margin is the boundary distance. In this context, vector distance is calculated using the L2 norm, i.e., the function... :

[0102]

[0103]

[0104]

[0105] Correspondingly, the training process for the text consistency discrimination task includes the following steps:

[0106] C1. Based on existing natural language inference datasets, obtain the text pairs in each data entry. Generate text vectors , ; through functions calculate , Cosine similarity between And normalize it to

[0107]

[0108] )

[0109]

[0110] C2, given tags =0 / 1 and Mean square error loss Optimize the model parameters:

[0111]

[0112] In step S3, the document data to be analyzed is parsed to generate structured question-and-answer text pairs in three dimensions, which are then stored in the vector database. This includes the following steps:

[0113] Step S3.1: Parse the text format in the document to be analyzed, obtain the font size, color, bold / italic information of each text segment, distinguish the body text and headings of the document based on the above information, parse out the hierarchical structure of the headings, and automatically generate the document table of contents.

[0114] Step S3.2: Based on the document directory obtained in step S3.1, generate hierarchical heading-paragraph data M by combining the document content. Each heading and its corresponding content are represented as follows: ,in, Indicates the document title. This indicates the text content of the paragraph under this heading. To give ID, Representing the present The hierarchy ensures that heading-paragraph pairs possess both hierarchical order and document arrangement order.

[0115]

[0116]

[0117] Step S3.3: Based on the title-paragraph data generated in step S3.2, The text is segmented into sentences based on periods and assigned an ID, while also combining... Belonging to Information generates sentence-level structured data S, where each sentence is represented as: Due to the special nature of sentence-level data, among which... and It is the same text, and in form it is the same as , same. Represents the current text to which it belongs , for A list of all text sentence IDs, arranged in the order in which the text sentences appear in the paragraph. ].

[0118] Step S3.4: Based on the title-paragraph data obtained in step S3.2, generate structured data at the title level. ].in and A list of parent and sibling heading IDs representing the current heading; if the current heading... If it is the lowest level heading, then for The list of sentence-level IDs segmented by periods has a structure similar to that in S3.3. If the current If a subheading exists, then This represents the list of its subtitle IDs.

[0119] Step S3.5: Based on the title-paragraph data from step S3.2, extract... It contains text fragments that are explained in points. Use and Summarize and categorize the content to obtain the corresponding segment themes. And assign an ID to generate structured data at the point level. .in and This represents a list of parent title IDs and sibling title IDs for the current topic. for The list of sentence-level IDs segmented by periods has a structure similar to that in S3.3. ,and Contained in middle.

[0120] Step S3.6: Summarize the three levels of structured data constructed in steps S3.3, S3.4, and S3.5, and unify their format. Considering that sentence-level structured data does not exist... Therefore, it is set to an empty list.

[0121] Step S3.7: Divide the structured data from step S3.6 into problems. ,Answer Two forms, namely and The former uses a pre-trained retrieval model to... Generate text vectors and Both are stored in a vector database; the latter is stored in a relational database.

[0122] In step S4, a pre-trained keyword extraction model and text vector model for a specific knowledge domain are used to accurately match user questions with question-and-answer text pairs in the constructed database, obtaining question-and-answer retrieval results. This includes the following steps:

[0123] Step S4.1: Based on the questions raised by the user The keyword extraction model is used to obtain the keywords present and compare them with... The text is concatenated, and then a text vector model is used to generate text vectors. , with vector database Iterate through the loop, calculate the inner product, and sort the results from largest to smallest; then calculate again. Top-n in inner product ranking Re-rank the cosine similarity between the pairs; select the top-n pairs based on their cosine similarity ranking. As a candidate question;

[0124] Step S4.2: Based on the candidate problems obtained in step S4.1 Corresponding To retrieve its corresponding value in the relational database , and .when of When it is an empty list, its relation to The question matching score is the cosine similarity score between the two. ;when of When the list is not empty, query the relational database. for The problem of time and calculate and cosine similarity ,Will and Perform weighted summation as and Question matching score;

[0125] Step S4.3: Obtain the question matching score based on step S4.2. Sort them from largest to smallest, and select the score corresponding to the first ranked score. As a real matching problem, and based on its corresponding Retrieve matching data from a relational database. Based on the final choice If the corresponding When the list is not empty, Matching in relational databases If the corresponding If the list is empty, then retrieve... exist The actual position i in the relational database is queried and concatenated. equal{ }of Text Formation Ultimately, it will Provided to users as matched answers to their questions.

[0126] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the various functions defined in this application.

[0127] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0130] In another aspect, this application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the domain retrieval method based on meta-learning and knowledge enhancement described in the above embodiments.

[0131] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the domain retrieval method based on meta-learning and knowledge enhancement described in the above embodiments.

[0132] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0133] For those skilled in the art, the specific meanings of the above terms in this invention can be understood according to the specific circumstances; the accompanying drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of this invention. Obviously, the described embodiments are some embodiments of this invention, but not all embodiments. Generally, the components of the embodiments of this invention described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0134] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A domain retrieval method based on meta-learning and knowledge enhancement, characterized in that, include: Step S1: Obtain corpus data for a certain knowledge domain, and label the text matching information and keyword information of the documents to be analyzed, and construct a text matching triplet dataset and a keyword extraction dataset for that knowledge domain. Step S2: Construct and train the keyword extraction model and text vector model using the pre-trained model; Step S3: Parse the table of contents of the document to be analyzed to generate structured question-and-answer text pairs in three dimensions: sentence level, heading level, and point level, and store them in the database; Step S4: After the user inputs the question, the trained keyword extraction model and text vector model are used to achieve accurate matching between the user's question and the question-and-answer text pairs in the database, and the question-and-answer retrieval results are obtained. In step S2, the text vector model is built based on the Roberta model, which is used as a shared encoder, and the text vector model is trained using two sub-tasks: triplet difference judgment and text consistency judgment. In step S2, the training process for the triplet difference judgment task includes: Step S2.2.1: Based on the trained keyword extraction model, extract the text keywords from the triplet data, and then concatenate the sentence-level and word-level text information to obtain the original text and its similar and dissimilar texts. Step S2.2.2: Obtain the original text and the text vectors corresponding to its similar and dissimilar texts, and establish a triplet loss. Optimize and train the text vector model parameters using the triplet loss. In step S2, the training process for the text consistency judgment task includes: Step S2.3.1: Based on the existing natural language inference dataset, obtain the text pair for each data point, generate the text vector of the text pair, and then calculate the cosine similarity between the text vectors of the text pair. Step S2.3.2: Establish mean squared error loss based on cosine similarity, and optimize the text vector model parameters using mean squared error loss.

2. The domain retrieval method based on meta-learning and knowledge enhancement according to claim 1, characterized in that, In step S1, the specific sub-steps for constructing the text matching triplet dataset and keyword extraction dataset under a certain knowledge domain include: Step S1.1: Integrate documents from a specific knowledge domain, parse the document information, segment the text in the documents by periods, and construct a sentence-level corpus; Step S1.2: Calculate the vector value of each text in the sentence-level corpus, as well as the cosine similarity between the text and other texts. Select one text from the top n texts in the cosine similarity score as the positive sample of the text, and select one text from the bottom n texts in the cosine similarity score as the negative sample of the text. The text itself, the positive sample and the negative sample of the text constitute the text matching triplet data. Summarize all the text matching triplet data to form the text matching triplet dataset. Step S1.3: Assign a label to each character in each text. The label includes the keyword start position, middle / end position and non-keyword position, forming keyword extraction data of the text composed of characters and corresponding labels. Summarize the keyword extraction data of all texts to form a keyword extraction dataset.

3. The domain retrieval method based on meta-learning and knowledge enhancement according to claim 2, characterized in that, In step S2, the keyword extraction model construction and training process includes: Step S2.1.1: Construct a keyword extraction model, which includes a text embedding layer, a text encoding layer, a feature extraction layer and a keyword extraction layer connected in sequence. Step S2.1.2: Obtain the keyword extraction data from the keyword extraction dataset, concatenate the characters in the keyword extraction data into a text sequence, and then convert it into a character sequence; Step S2.1.3: Input the character sequence into the text embedding layer to obtain the embedded representation of the character sequence; Step S2.1.4: Input the embedding representation into the text encoding layer and model the embedding representation to obtain its hidden layer representation; Step S2.1.5: In the feature extraction layer, BiLSTM is used to extract features from the hidden layer representation to obtain the context encoding of the text sequence; Step S2.1.6: Obtain the keyword extraction data from the keyword extraction dataset, concatenate the labels in the keyword extraction data into a label sequence, and obtain the emission probability and transition matrix based on the context encoding of the corresponding text sequence. Construct a scoring function to calculate the corresponding score of the text sequence label, and then use the softmax function to calculate the probability of the output predicted label sequence. Step S2.1.7: Based on the probability obtained in step S2.1.6, use its negative log-likelihood function as the loss function to train the keyword extraction model.

4. The domain retrieval method based on meta-learning and knowledge enhancement according to claim 3, characterized in that, The specific steps of step S2.1.5 include: Step S2.1.5.1: Obtain the i-th hidden vector in the hidden vector representation and the hidden vectors of the previous and next positions. Define the first weight matrix, the second weight matrix and the first bias. Calculate the feature information of the character at the current position i on the left and right sides of the semantic space using the Sigmoid function. Step S2.1.5.2: Define the third weight matrix and the second bias. Use the Tanh function to perform a nonlinear transformation on the feature information on the left and right sides of the obtained semantic space to obtain the hidden features of the character at the current position i on the left and right sides of the semantic space and concatenate them into the final feature vector, which is the context encoding of the i-th character in the text sequence. Repeat this process for all hidden vectors to obtain the context encoding of the text sequence.

5. The domain retrieval method based on meta-learning and knowledge enhancement according to claim 2, characterized in that, The specific sub-steps of step S3 include: Step S3.1: Parse the text format in the document to be analyzed, obtain the font size, color, bold / italic information of each text segment, so as to distinguish the body text and headings of the document, and parse out the hierarchical structure of the headings to generate the document table of contents; Step S3.2: Generate hierarchical heading-paragraph data based on the document's table of contents and content; Step S3.3: Based on the title-paragraph data, the document content is divided into text sentences according to the period and assigned an ID. Combined with the information of the title, the sentence-level structured data is generated. Step S3.4: Generate structured data at the heading level based on the heading-paragraph data; Step S3.5: Based on the title-paragraph data, extract the text segments containing point-by-point explanations in the document content, summarize and categorize the content of the text segments to obtain the corresponding segment themes, and generate structured data at the point level. Step S3.6: Standardize the format of the structured data at the sentence level, the heading level, and the point level; Step S3.7: Divide the structured data after standardization into two forms: question and answer, to form structured question-and-answer text pairs; wherein, the structured data in question form includes question ID and question, and the structured data in answer form includes question ID, text content, the parent title ID of the current text, and sentence level / subtitle ID; Step S3.8: Use the trained retrieval model to generate corresponding text vectors for the structured data in question form, and store them in the vector database along with the ID information; store the structured data in answer form in the relational database.

6. The domain retrieval method based on meta-learning and knowledge enhancement according to claim 5, characterized in that, The specific sub-steps of step S4 include: Step S4.1: Obtain keywords from user questions through keyword extraction model, concatenate them with user questions, generate text vectors using text vector model, and iterate through and calculate the inner product with the text vectors stored in the vector database and sort them from largest to smallest. Step S4.2: Calculate the cosine similarity between the text vector of the user's question and the top n text vectors with the calculated inner product, then re-sort them and select the top n structured data in question form with the highest cosine similarity as candidate questions. Step S4.3: Based on the ID corresponding to the candidate question, obtain its corresponding structured data in the form of the answer in the relational database. If the parent title ID of the current text of the candidate question is an empty list, its question matching score with the user question is the cosine similarity score between the two. If the parent title ID of the current text of the candidate question is not an empty list, query the relational database for the parent question whose candidate question ID is equal to the parent title ID of the current text, and calculate the cosine similarity score between the parent question and the user question. Cosine similarity score Cosine similarity score A weighted sum is used as the question matching score between candidate questions and user questions; Step S4.4: Obtain the question matching scores of all candidate questions and user questions, sort them from largest to smallest, select the candidate question corresponding to the first ranked question as the real matching question, and retrieve the matching answer from the relational database according to the corresponding ID; based on the real matching question, if the corresponding sentence level / subheading ID is not an empty list, the final answer is the answer obtained from the relational database; if the corresponding sentence level / subheading ID is an empty list, the position of the real matching question's corresponding ID in the list of all text sentence IDs is used to query and retrieve the question text at the ID-1 position, the question text at the ID position, and the question text at the ID+1 position from the relational database, and concatenate them sequentially to obtain the final answer; Step S4.1: Provide the final answer as the matching answer to the user's question.

7. The domain retrieval method based on meta-learning and knowledge enhancement according to claim 1, characterized in that, In step S2.2.2, the triplet loss is specifically as follows: in, For the text vector of the original text, For text vectors of similar texts, For dissimilar text, the text vectors are... This represents the distance between the text vector of the original text and the text vectors of similar texts. This represents the distance between the text vector of the original text and the text vectors of dissimilar texts, where n represents the length of the text sequence. This represents the boundary distance.

Images