Corpus localization method, intention recognition method and model training method and device thereof

Abstract

The present invention discloses a corpus positioning method, an intention recognition method, and a model training method, device, equipment, and medium thereof. The training method of the corpus positioning method includes: obtaining multiple groups of training samples, inputting the query text in the training samples and the query text into a pre-training model for feature extraction, and obtaining feature extraction results; performing secondary feature extraction on the overall features of the training samples, the character features of each character contained in the query text, and the overall features of the query text to obtain word features; inputting the feature extraction results and the word features into a machine reading comprehension model; calculating the first calculation error of the machine reading comprehension model based on the annotated positioning information and predicted positioning information of the query text in the corpus, and adjusting the parameters of the machine reading comprehension model based on the first calculation error until the convergence condition is met, and obtaining a trained corpus positioning model. Inputting the feature extraction results and the word features into the machine reading comprehension model, the output is more accurate.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a corpus positioning method, an intent recognition method, and a model training method, device, equipment, and medium thereof. Background Art

[0002] Current intent recognition methods are mainly based on dictionaries and rules. However, when the query text entered by the user is complex, machine learning and deep learning are needed to flexibly process the query text. However, the above technologies are relatively weak in locating important information in the corpus. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defect of the relatively weak ability to locate important information in the corpus in the prior art, and to provide a corpus positioning method, an intent recognition method and its model training method, device, equipment and medium.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The present invention provides a model training method for corpus localization, comprising the following steps:

[0006] Acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text;

[0007] For each group of training samples, the query text and the query text are input into a pre-trained pre-trained model for feature extraction to obtain a feature extraction result; wherein the feature extraction result includes the overall features of the training samples, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0008] Performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features;

[0009] Inputting the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is used to output predicted location information of the query text in the corpus;

[0010] According to the annotated positioning information of the query article in the corpus and the predicted positioning information, a first calculation error of the machine reading comprehension model is calculated, and the parameters of the machine reading comprehension model are adjusted according to the first calculation error until the convergence condition is met, thereby obtaining a trained corpus positioning model.

[0011] Preferably, the step of the machine reading comprehension model being used to output the predicted location information of the query article in the corpus specifically includes:

[0012] The output result of the machine reading comprehension model is classified into a first probability corresponding to each word in the query text through Softmax modeling;

[0013] Each dimension of the logic vector corresponds to a feature of each word in the pre-calculated result; the predicted positioning information is the position of the word with the highest probability in the first probability.

[0014] Specifically, the step of the machine reading comprehension model being used to output the predicted location information of the query article in the corpus specifically includes:

[0015] The output result of the machine reading comprehension model is classified into a first probability corresponding to each word in the query text through Softmax modeling;

[0016] Each dimension of the logic vector corresponds to a feature of each word in the pre-calculated result; the predicted positioning information is the position of the word with the highest probability in the first probability.

[0017] Preferably, the step of obtaining the query text and the corpus corresponding to the query text specifically comprises the following steps:

[0018] Determining similar words to the query text;

[0019] The corpus containing the query text and / or the similar words is determined as the corpus corresponding to the query text.

[0020] Preferably, the step of performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features specifically includes the following steps:

[0021] The character features are input into the neural network and are weighted and summed to obtain word features.

[0022] As a second aspect of the present invention, the present invention provides a corpus localization method, comprising the following steps:

[0023] Get the query text;

[0024] The query text is input into a corpus positioning model, and the positioning information of the query text in the corpus is determined according to the corpus positioning model; wherein the corpus positioning model is trained using the above-mentioned model training method.

[0025] As a third aspect of the present invention, the present invention provides a training method for an intent recognition model, the training method comprising the following steps:

[0026] Acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text;

[0027] For each group of training samples, the query text and the query text are input into a pre-trained pre-trained model for feature extraction to obtain a feature extraction result; wherein the feature extraction result includes the overall features of the training samples, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0028] Performing secondary feature extraction on the feature extraction result to obtain word features;

[0029] Inputting the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is used to output predicted location information of the query text in the corpus;

[0030] The output result of the machine reading comprehension model is reduced to a first weight corresponding to each word in the query text, and the first weight is weighted and summed with the feature extraction result to obtain a first calculation result;

[0031] Each dimension of the output result of the machine reading comprehension model corresponds to each word in the query text and the corpus;

[0032] According to the predicted positioning information, the corresponding dimension in the output result of the machine reading comprehension model is intercepted to obtain dimension interception data;

[0033] and mapping the probability of each word in the corpus being the query text into a second weight through dimension mapping;

[0034] Perform a weighted summation of the dimension intercepted data and the second weight to obtain a second calculation result;

[0035] sequentially inputting the word feature, the first calculation result, and the second calculation result into an intention recognition model to obtain the intention of the query text;

[0036] Calculating a first calculation error of the machine reading comprehension model according to the annotated location information of the query article in the corpus and the predicted location information;

[0037] Obtaining a true intent label of the query text, and calculating a second calculation error of the intent recognition model;

[0038] Adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model;

[0039] Adjusting the parameters of the intent recognition model according to the second calculated error until a convergence condition is met to obtain a trained intent recognition model;

[0040] Alternatively, the first calculation error and the second calculation error are summed according to a preset ratio to obtain a third calculation error, and the parameters of the machine reading comprehension model and the intent recognition model are adjusted according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

[0041] As a fourth aspect of the present invention, the present invention provides an intention recognition method, comprising the following steps:

[0042] Get the query text;

[0043] The query text is input into a corpus positioning model and an intent recognition model, and the intent of the query text is determined according to the corpus positioning model and the intent recognition model; wherein the corpus positioning model and the intent recognition model are trained using the model training method of the third aspect of the present invention.

[0044] As a fifth aspect of the present invention, a training device for a corpus localization model is provided, comprising a data acquisition module, a pre-training module, a word feature extraction module, a machine reading comprehension recognition module, and a model training module;

[0045] The data acquisition module is used to acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text;

[0046] The pre-training module is used for inputting the query text and the query text into a pre-trained pre-training model for feature extraction for each group of training samples to obtain feature extraction results; wherein the feature extraction results include the overall features of the training samples, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0047] The word feature extraction module is used to perform secondary feature extraction on the overall features of the training sample, the word features of each word contained in the query text, and the overall features of the query text to obtain word features;

[0048] The machine reading comprehension module is used to input the feature extraction result and the word feature into a machine reading comprehension model, and the machine reading comprehension model is used to output the predicted location information of the query text in the corpus;

[0049] The model training module is used to calculate the first calculation error of the machine reading comprehension model based on the annotation positioning information of the query article in the corpus and the predicted positioning information, and adjust the parameters of the machine reading comprehension model according to the first calculation error until the convergence condition is met, thereby obtaining a trained corpus positioning model.

[0050] As a sixth aspect of the present invention, there is provided a training device for an intention recognition model, the device comprising a data acquisition module, a pre-training module, a word feature extraction module, a machine reading comprehension recognition module, an intention recognition module, and a model training module;

[0051] The data acquisition module is used to acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text;

[0052] The pre-training module is used for inputting the query text and the query text into a pre-trained pre-training model for feature extraction for each group of training samples to obtain feature extraction results; wherein the feature extraction results include the overall features of the training samples, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0053] The word feature extraction module is used to perform secondary feature extraction on the feature extraction result to obtain word features;

[0054] The machine reading comprehension recognition module is used to input the feature extraction result and the word feature into a machine reading comprehension model, and the machine reading comprehension model is used to output the predicted location information of the query text in the corpus;

[0055] The intention recognition module is used to reduce the output result of the machine reading comprehension model to a first weight corresponding to each word in the query text, and perform a weighted summation of the first weight and the feature extraction result to obtain a first calculation result;

[0056] Each dimension of the output result of the reading comprehension model corresponds to each word in the query text and the corpus;

[0057] According to the predicted positioning information, the corresponding dimension in the output result of the machine reading comprehension model is intercepted to obtain dimension interception data;

[0058] and mapping the probability of each word in the corpus being the query text into a second weight through dimension mapping;

[0059] Perform a weighted summation of the dimension intercepted data and the second weight to obtain a second calculation result;

[0060] sequentially inputting the word feature, the first calculation result, and the second calculation result into an intention recognition model to obtain the intention of the query text;

[0061] The training module is used to calculate a first calculation error of the machine reading comprehension model according to the annotation location information of the query article in the corpus and the predicted location information;

[0062] Obtaining a true intent label of the query text, and calculating a second calculation error of the intent recognition model;

[0063] Adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model;

[0064] and adjusting the parameters of the intent recognition model according to the second calculated error until a convergence condition is met to obtain a trained intent recognition model;

[0065] Alternatively, the first calculation error and the second calculation error are summed according to a preset ratio to obtain a third calculation error, and the parameters of the machine reading comprehension model and the intent recognition model are adjusted according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

[0066] As the seventh aspect of the present invention, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first to fourth aspects of the present invention is implemented.

[0067] As an eighth aspect of the present invention, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the first to fourth aspects of the present invention is implemented.

[0068] The positive improvement effect of the present invention is that by inputting the feature extraction results of the character dimension and the word feature vector of the word dimension into the machine reading comprehension model, there are two dimensions, so that the machine reading comprehension model can better determine the location of the query text in the corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a schematic diagram of a first flow chart of a model training method for the corpus localization method in Embodiment 1 of the present invention.

[0070] Figure 2 This is a second flow chart of the model training method of the corpus localization method in Embodiment 1 of the present invention.

[0071] Figure 3 The figure is a flow chart of the corpus localization method in Embodiment 2 of the present invention.

[0072] Figure 4 This is a first flow chart of the model training method of the intent recognition method in Example 3 of the present invention.

[0073] Figure 5 This is a second flow chart of the model training method of the intent recognition method in Example 3 of the present invention.

[0074] Figure 6 A flowchart of the intention recognition method in Embodiment 4 of the present invention.

[0075] Figure 7 A schematic diagram of the structure of a training device for a corpus localization model in Embodiment 5 of the present invention.

[0076] Figure 8 A structural diagram of a training device for an intent recognition model in Example 6 of the present invention.

[0077] Fig. 9 This is a schematic diagram of the structure of an electronic device in Example 7 of the present invention. DETAILED DESCRIPTION

[0078] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0079] Example 1

[0080] See also Figure 1 This embodiment provides a model training method for a corpus localization method, comprising the following steps:

[0081] S1. Obtain multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus includes at least one corpus corresponding to the query text;

[0082] S2. For each set of training samples, input the query text and the query text into a pre-trained pre-trained model for feature extraction to obtain feature extraction results;

[0083] The feature extraction results include the overall features of the training sample, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0084] S3, performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features;

[0085] S4, inputting the feature extraction result and the word feature into a machine reading comprehension model, and the machine reading comprehension model is used to output the predicted location information of the query text in the corpus;

[0086] S5. Calculate a first calculation error of the machine reading comprehension model based on the real location information of the query text in the corpus, and adjust the parameters of the machine reading comprehension model based on the first calculation error until a convergence condition is met, thereby obtaining a trained machine reading comprehension model.

[0087] See also Figure 2 , the query text is the “Query” in the figure, and the corpus is the “Corpus set” in the figure;

[0088] In this embodiment, when inputting the training sample into the pre-training model, [CLS] and [SEP] need to be input. The [CLS] (classification) flag is placed at the beginning of the first sentence. After the pre-training model (here, it can be a BERT model), the overall features of the training sample are obtained (the features can be represented by a vector), which can be used for subsequent classification tasks. The [SEP] flag is used to separate two input sentences. For example, when inputting sentences A and B, the [SEP] flag should be added after sentences A and B.

[0089] After the pre-training model, the feature extraction results are obtained, which include the overall features of the training samples (i.e., H CLS ), the word features of each word contained in the query text (i.e., H Query ), the overall characteristics of the training samples (i.e., H SEP ), the character features of each character in the corpus (i.e., H P ), and the overall characteristics of the corpus (i.e., H SEP_ )’s pre-training results;

[0090] Among them, H CLS Indicates H Query , H SEP , H P , and H SEP_ The overall features of the query text, the character features of each character contained in the query text, the character features corresponding to the features of each character in the input query text; the first flag feature vector is used to represent the overall features of the character vector in the query text character feature vector,

[0091] Secondary feature extraction is performed on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features (i.e. Figure 2 The word vector in H CLS , H Query, and H SEP The word vector is extracted based on the query text level.

[0092] The feature extraction result and the word feature are input into a machine reading comprehension model (also referred to as an MRC model) to obtain the predicted location information of the query text in the corpus. It should be noted that the predicted location information refers to the position of the query text in the corpus. For example, if the input is "Shanghai Disney" and the corpus is a description of Shanghai Disney, then the predicted location information is the position of "Shanghai Disney" in the description.

[0093] In this embodiment, word features at the query text level are obtained through word feature extraction, and the pre-training results and word features are spliced ​​and input into the machine reading comprehension model to obtain the predicted location information of the query text in the corpus;

[0094] In this embodiment, since the pre-training results and word features are input into the machine reading comprehension model, and since the pre-training results are based on word features extracted from characters, which are at the character feature level, and the word features are at the word level, the position of the query text in the corpus can be located more accurately.

[0095] Specifically, the step of the machine reading comprehension model being used to output the predicted location information of the query article in the corpus specifically includes:

[0096] The output result of the machine reading comprehension model is classified into a first probability corresponding to each word in the query text through Softmax modeling;

[0097] Each dimension of the output result of the machine reading comprehension model corresponds to the feature of each word in the pre-calculated result; the predicted positioning information is the position of the word with the highest probability in the first probability.

[0098] In this embodiment, the output result of the machine reading comprehension model refers to the intermediate output logits layer of the machine reading comprehension model (in the context of deep learning, the logits layer refers to the layer that feeds into normalization), and the first probability after Softmax. Since the sum of the first probabilities corresponding to each word output is 1, the calculation result has good convergence.

[0099] Specifically, the step of obtaining a query text and a corpus corresponding to the query text specifically includes the following steps:

[0100] Determining similar words to the query text;

[0101] The corpus containing the query text and / or the similar words is determined as the corpus corresponding to the query text.

[0102] In this embodiment, by searching for similar words corresponding to the query text in the knowledge base, for example, the query text is "Shanghai Disney", and similar texts include "Universal Studios". That is, through the fragments obtained by searching in the knowledge base, information is extracted from the fragments, and the required fragments are the corpus. In this case, the corpus found contains the fragment of "Shanghai Disney" and the fragment of "Universal Studios". This makes the results obtained by the machine reading comprehension model more accurate.

[0103] Example 2

[0104] See also Figure 3 This embodiment provides a corpus positioning method, comprising the following steps:

[0105] Obtaining the query text; in this embodiment, obtaining the query text refers to:

[0106] S201, obtaining corpus location data, where the corpus location data includes a query text and a corpus set, where the corpus set includes at least one corpus corresponding to the query text;

[0107] S202, for the corpus location data, input the query text and the query text into a pre-trained model for feature extraction to obtain a feature extraction result;

[0108] The feature extraction results include the overall features of the training sample, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0109] S203, performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features;

[0110] S204: Input the feature extraction result and the word feature into a machine reading comprehension model, and the machine reading comprehension model is used to output the predicted location information of the query text in the corpus.

[0111] Wherein, the corpus localization model is trained by the model training method in Example 1.

[0112] Using the corpus positioning method in this embodiment, since the input data of the machine reading comprehension model are feature extraction results and word features, and since the input data are two different dimensions of data, the machine reading comprehension model can more accurately locate the position of the query text in the corpus.

[0113] Example 3

[0114] See also Figure 4This embodiment provides a model training method for an intent recognition method, the method comprising the following steps:

[0115] S301, acquiring intent recognition data, where the intent recognition data includes a query text and a corpus, where the corpus contains at least one corpus corresponding to the query text;

[0116] S302, for the intent recognition data, input the query text and the query text into a pre-trained model for feature extraction to obtain a feature extraction result;

[0117] The feature extraction result includes the overall feature of the training sample, the overall feature of the query text, the character feature of each character contained in the query text, the overall feature of the corpus, and the character feature of each character contained in the corpus;

[0118] S303, performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features;

[0119] S304, inputting the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is used to output predicted location information of the query text in the corpus;

[0120] S305, reducing the output result of the machine reading comprehension model to a first weight corresponding to each word in the query text, and weighted summing the first weight and the feature extraction result to obtain a first calculation result;

[0121] Each dimension of the output result of the reading comprehension model corresponds to each word in the query text and the corpus;

[0122] S306, intercepting the corresponding dimension in the output result of the machine reading comprehension model according to the predicted positioning information to obtain dimension interception data;

[0123] S307, mapping the probability of each word in the corpus being the query text into a second weight through dimension mapping;

[0124] S308, performing weighted summation of the dimension intercepted data and the second weight to obtain a second calculation result;

[0125] S309, sequentially inputting the word feature, the first calculation result, and the second calculation result into an intention recognition model to obtain the intention of the query text;

[0126] S310, calculating a first calculation error of the machine reading comprehension model according to the annotation location information of the query text in the corpus and the predicted location information;

[0127] S311, obtaining a true intent label of the query text, and calculating a second calculation error of the intent recognition model;

[0128] S312, adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model;

[0129] Adjusting the parameters of the intent recognition model according to the second calculated error until a convergence condition is met to obtain a trained intent recognition model;

[0130] Alternatively, in this embodiment, the training method of the intent recognition model may also include another implementation method, see Figure 5 , this implementation is the same as the previous implementation, steps S301-S311 are all the same, only step S312 is replaced by:

[0131] S313. Sum the first calculation error and the second calculation error according to a preset ratio to obtain a third calculation error, and adjust the parameters of the machine reading comprehension model and the intent recognition model according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

[0132] See also Figure 2 (The model training method of a corpus positioning method in Example 1 can be considered as a part of the model training method of an intention recognition method in this embodiment). In this embodiment, the corresponding part of the output result of the machine reading comprehension model is intercepted according to the predicted positioning information to obtain dimension interception data (this process is Figure 2 For example, the prediction position information is from the fifth to the tenth position in the corpus. Since the dimension of the output result of the machine reading comprehension model corresponds to the input feature, for example, the corpus input is 15 characters, and the character features of the corpus are (A1, A2, …, A 15 ), then the output of the corresponding machine reading comprehension model corresponds to the part of the corpus: (a1, a2, …, a 15 ), then intercept (a5,a6,a7,a8,a9,a 10 In the output result of the machine reading comprehension model, the features corresponding to each word are multi-dimensional, and the multi-dimensionality is mapped to one dimension, converted into a second weight, and the elements in the intercepted vector are weighted and summed according to the second weight (that is, Figure 2The valid word interval in the output) is output to obtain the second calculation result.

[0133] It should be noted that in this embodiment, the intent recognition model can use a classification layer to input word features, the first calculation result, and the second calculation result into the intent recognition model. After the intermediate output of the intent recognition model is input into Softmax, the predicted intent label and the corresponding probability of the query text can be obtained. The predicted intent label with the highest probability is used as the predicted intent of the query text.

[0134] Since the input data of the machine reading comprehension model are feature extraction results and word features, the first calculation result outputted by it represents all the data including the query text and the corpus, and the second result is based on the predicted positioning information, which only represents the dimension of the word vector; and the word feature vector is the dimension of the word. Therefore, the word feature vector, the first calculation result and the second calculation result are input into the intent recognition model, and the intent of the query text is determined more accurately.

[0135] In addition, in this embodiment, the first calculation error and the second calculation error can be summed according to a preset ratio to obtain a third calculation error, and the parameters of the machine reading comprehension model and the intent recognition model can be adjusted according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

[0136] In the actual process of training the model, since the first calculation error and the second calculation error will affect each other, the first calculation error and the second calculation error are summed according to a predetermined ratio as a whole calculation error, so that the training model can reach the predetermined convergence condition faster.

[0137] Example 4

[0138] This embodiment provides an intention recognition method, comprising the following steps:

[0139] Get the query text;

[0140] Inputting the query text into a corpus location model and an intention recognition model, and determining the intention of the query text according to the corpus location model and the intention recognition model;

[0141] Among them, the corpus localization model and the intention recognition model are trained using the model training method in Example 3.

[0142] See also Figure 6 In this embodiment, “obtaining the query text” specifically refers to:

[0143] S401, obtaining multiple groups of training samples, each group of training samples including a query text and a corpus, wherein the corpus includes at least one corpus corresponding to the query text;

[0144] “Inputting the query text into a corpus positioning model and an intention recognition model, and determining the intention of the query text according to the corpus positioning model and the intention recognition model; wherein the corpus positioning model and the intention recognition model are trained using the model training method in Example 3.” means:

[0145] S402, for each group of training samples, input the query text and the query text into a pre-trained pre-trained model to perform feature extraction, and obtain feature extraction results; wherein the feature extraction results include overall features of the training samples, overall features of the query text, character features of each character included in the query text, overall features of the corpus, and character features of each character included in the corpus;

[0146] S403, performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features;

[0147] S404, inputting the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is used to output predicted location information of the query text in the corpus;

[0148] S405, reducing the output result of the machine reading comprehension model to a first weight corresponding to each word in the query text, and weighted summing the first weight and the feature extraction result to obtain a first calculation result;

[0149] Each dimension of the output result of the reading comprehension model corresponds to each word in the query text and the corpus;

[0150] S406, intercepting the corresponding dimension in the output result of the machine reading comprehension model according to the predicted positioning information to obtain dimension interception data;

[0151] S407, mapping the probability of each word in the corpus being the query text into a second weight through dimension mapping;

[0152] S408, performing weighted summation of the dimension intercepted data and the second weight to obtain a second calculation result;

[0153] S409: input the word feature, the first calculation result, and the second calculation result into an intention recognition model in sequence to obtain the intention of the query text.

[0154] In this embodiment, since each model has been trained. The first calculation result is to input the feature extraction result and the word feature into the machine reading comprehension model, so the first calculation result represents the query text and the overall level of the corpus, and the second calculation result is intercepted in the corpus according to the predicted positioning information, so the features in the second calculation result correspond to each word, which is a word-level feature, and the word feature represents the word-level feature. Inputting the three into the intent recognition model can judge the user's intent more accurately and with higher accuracy for the query text entered by the user.

[0155] Example 5

[0156] See also Figure 7 , this embodiment provides a corpus locating device, including a first data acquisition module 101, a first pre-training module 102, a first word feature extraction module 103, a first machine reading comprehension module 104, and a first model training module 105;

[0157] The data acquisition module 101 is used to acquire multiple sets of training samples, each set of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text;

[0158] A first pre-training module 102 is used for, for each group of training samples, inputting the query text and the query text into a pre-trained pre-training model to perform feature extraction to obtain a feature extraction result; wherein the feature extraction result includes the overall features of the training samples, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0159] A first word feature extraction module 103 is used to perform secondary feature extraction on the overall feature of the training sample, the word features of each word contained in the query text, and the overall feature of the query text to obtain word features;

[0160] A first machine reading comprehension module 104, configured to input the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is configured to output predicted location information of the query text in the corpus;

[0161] A first model training module 105, configured to calculate a first calculation error of the machine reading comprehension model according to the annotation location information of the query text in the corpus and the predicted location information;

[0162] Adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model;

[0163] Example 6

[0164] See also Figure 8 , this embodiment provides an intention recognition device, including a second data acquisition module 201, a second pre-training module 202, a second word feature extraction module 203, a second machine reading comprehension recognition module 204, and a second model training module 205;

[0165] A second data acquisition module 201 is used to acquire multiple sets of training samples, each set of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text;

[0166] A second pre-training module 202 is used for, for each set of training samples, inputting the query text and the query text into a pre-trained pre-training model to perform feature extraction to obtain a feature extraction result; wherein the feature extraction result includes the overall features of the training samples, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus;

[0167] A second word feature extraction module 203 is used to perform secondary feature extraction on the overall feature of the training sample, the word features of each word contained in the query text, and the overall feature of the query text to obtain word features;

[0168] A second machine reading comprehension module 204, configured to input the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is configured to output predicted location information of the query text in the corpus;

[0169] A second model training module 205, configured to calculate a first calculation error of the machine reading comprehension model according to the annotation location information of the query text in the corpus and the predicted location information;

[0170] Obtaining a true intent label of the query text, and calculating a second calculation error of the intent recognition model;

[0171] Adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model;

[0172] Adjusting the parameters of the intent recognition model according to the second calculated error until a convergence condition is met to obtain a trained intent recognition model;

[0173] Alternatively, the first calculation error and the second calculation error are summed according to a preset ratio to obtain a third calculation error, and the parameters of the machine reading comprehension model and the intent recognition model are adjusted according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

[0174] Example 7

[0175] Fig. 9 The schematic diagram of the structure of an electronic device provided in this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, any one of the methods in Embodiments 1-4 is implemented. Fig. 9 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0176] like Fig. 9 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0177] The bus 33 includes a data bus, an address bus, and a control bus.

[0178] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0179] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0180] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as any one of the methods of embodiments 1-4 of the present invention.

[0181] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model-generated device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via a bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0182] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0183] Example 8

[0184] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods of Embodiments 1-4 is implemented.

[0185] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0186] In a possible implementation manner, the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute any one of the methods of implementing embodiments 1-4.

[0187] The program code for executing the present invention may be written in any combination of one or more programming languages, and the program code may be executed entirely on the user device, partially on the user device, as a separate software package, partially on the user device, partially on a remote device, or entirely on a remote device.

[0188] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that this is only an example, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications fall within the scope of protection of the present invention.

[0189] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A model training method for a corpus localization method, characterized in that: The corpus localization model includes a cascaded pre-training model and a machine reading comprehension model, and the training method includes: Acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text; For each group of training samples, the query text and the corpus are input into a pre-trained pre-training model for feature extraction to obtain a feature extraction result; wherein the feature extraction result includes the overall features of the training sample, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus; Performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features; Inputting the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is used to output predicted location information of the query text in the corpus; According to the annotated positioning information of the query article in the corpus and the predicted positioning information, a first calculation error of the machine reading comprehension model is calculated, and the parameters of the machine reading comprehension model are adjusted according to the first calculation error until the convergence condition is met, thereby obtaining a trained corpus positioning model.

2. The model training method for the corpus localization method according to claim 1, characterized in that: The step of the machine reading comprehension model being used to output the predicted location information of the query article in the corpus specifically includes: The output result of the machine reading comprehension model is classified into a first probability corresponding to each word in the query text through Softmax modeling; Each dimension of the output result of the machine reading comprehension model corresponds to the feature of each word in the pre-calculated result; the predicted positioning information is the position of the word with the highest probability in the first probability.

3. The model training method for the corpus localization method according to claim 1, characterized in that: The step of obtaining the query text and the corpus corresponding to the query text specifically comprises the following steps: Determining similar words to the query text; The corpus containing the query text and / or the similar words is determined as the corpus corresponding to the query text.

4. The model training method for the corpus localization method according to claim 1, characterized in that: The step of performing secondary feature extraction on the overall features of the training sample, the character features of each character contained in the query text, and the overall features of the query text to obtain word features specifically includes the following steps: The character features are input into the neural network and the word features are weighted and summed.

5. A corpus localization method, characterized in that: include: Get the query text; The query text is input into a corpus positioning model, and the positioning information of the query text in the corpus is determined according to the corpus positioning model; wherein the corpus positioning model is trained using the model training method described in any one of claims 1-4.

6. A model training method for an intent recognition method, characterized in that: The training method comprises: Acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text; For each group of training samples, the query text and the corpus are input into a pre-trained pre-training model for feature extraction to obtain a feature extraction result; wherein the feature extraction result includes the overall features of the training sample, the overall features of the query text, the character features of each character contained in the query text, the overall features of the corpus, and the character features of each character contained in the corpus; Performing secondary feature extraction on the feature extraction result to obtain word features; Inputting the feature extraction result and the word feature into a machine reading comprehension model, wherein the machine reading comprehension model is used to output predicted location information of the query text in the corpus; The output result of the machine reading comprehension model is reduced to a first weight corresponding to each word in the query text, and the first weight is weighted and summed with the feature extraction result to obtain a first calculation result; Each dimension of the output result of the machine reading comprehension model corresponds to each word in the query text and the corpus; According to the predicted positioning information, the corresponding dimension in the output result of the machine reading comprehension model is intercepted to obtain dimension interception data; Mapping the probability that each word in the corpus is the query text into a second weight through dimension; Perform a weighted summation of the dimension intercepted data and the second weight to obtain a second calculation result; sequentially inputting the word feature, the first calculation result, and the second calculation result into an intention recognition model to obtain the intention of the query text; Calculating a first calculation error of the machine reading comprehension model according to the annotated location information of the query article in the corpus and the predicted location information; Obtaining a true intent label of the query text, and calculating a second calculation error of the intent recognition model; Adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model; Adjusting the parameters of the intent recognition model according to the second calculated error until a convergence condition is met to obtain a trained intent recognition model; Alternatively, the first calculation error and the second calculation error are summed according to a preset ratio to obtain a third calculation error, and the parameters of the machine reading comprehension model and the intent recognition model are adjusted according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

7. A method for intention recognition, characterized in that: include: Get the query text; The query text is input into a corpus positioning model and an intent recognition model, and the intent of the query text is determined according to the corpus positioning model and the intent recognition model; wherein the corpus positioning model and the intent recognition model are trained using the model training method in claim 6.

8. A training device for a corpus localization model, characterized in that: It includes a first data acquisition module, a first pre-training module, a first word feature extraction module, a first machine reading comprehension recognition module, and a first model training module; The first data acquisition module is used to acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text; The first pre-training module is used for inputting the query text and the corpus into a pre-trained pre-training model for feature extraction for each group of training samples to obtain feature extraction results; wherein the feature extraction results include overall features of the training samples, overall features of the query text, character features of each character contained in the query text, overall features of the corpus, and character features of each character contained in the corpus; The first word feature extraction module is used to perform secondary feature extraction on the overall feature of the training sample, the word features of each word contained in the query text, and the overall feature of the query text to obtain word features; The first machine reading comprehension module is used to input the feature extraction result and the word feature into a machine reading comprehension model, and the machine reading comprehension model is used to output the predicted location information of the query text in the corpus; The first model training module is used to calculate the first calculation error of the machine reading comprehension model based on the annotation positioning information of the query article in the corpus and the predicted positioning information, and adjust the parameters of the machine reading comprehension model according to the first calculation error until the convergence condition is met, thereby obtaining a trained corpus positioning model.

9. A training device for an intention recognition model, characterized in that: The device includes a second data acquisition module, a second pre-training module, a second word feature extraction module, a second machine reading comprehension recognition module, a second intention recognition module, and a second model training module; The second data acquisition module is used to acquire multiple groups of training samples, each group of training samples includes a query text and a corpus, and the corpus contains at least one corpus corresponding to the query text; The second pre-training module is used for inputting the query text and the corpus into a pre-trained pre-training model for feature extraction for each group of training samples to obtain feature extraction results; wherein the feature extraction results include overall features of the training samples, overall features of the query text, character features of each character contained in the query text, overall features of the corpus, and character features of each character contained in the corpus; The second word feature extraction module is used to perform secondary feature extraction on the feature extraction result to obtain word features; The second machine reading comprehension recognition module is used to input the feature extraction result and the word feature into a machine reading comprehension model, and the machine reading comprehension model is used to output the predicted location information of the query text in the corpus; The second intention recognition module is used to reduce the output result of the machine reading comprehension model to a first weight corresponding to each word in the query text, and perform a weighted summation of the first weight and the feature extraction result to obtain a first calculation result; Each dimension of the output result of the machine reading comprehension model corresponds to each word in the query text and the corpus; According to the predicted positioning information, the corresponding dimension in the output result of the machine reading comprehension model is intercepted to obtain dimension interception data; and mapping the probability of each word in the corpus being the query text into a second weight through dimension mapping; Perform a weighted summation of the dimension intercepted data and the second weight to obtain a second calculation result; sequentially inputting the word feature, the first calculation result, and the second calculation result into an intention recognition model to obtain the intention of the query text; The second model training module is used to calculate a first calculation error of the machine reading comprehension model according to the annotation location information of the query article in the corpus and the predicted location information; Obtaining a true intent label of the query text, and calculating a second calculation error of the intent recognition model; Adjusting the parameters of the machine reading comprehension model according to the first calculated error until a convergence condition is met, thereby obtaining a trained corpus localization model; and adjusting the parameters of the intent recognition model according to the second calculated error until a convergence condition is met to obtain a trained intent recognition model; Alternatively, the first calculation error and the second calculation error are summed according to a preset ratio to obtain a third calculation error, and the parameters of the machine reading comprehension model and the intent recognition model are adjusted according to the third calculation error to obtain a trained corpus localization model and intent recognition model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images