A tourism scene slot filling method for boundary-sensitive data

By designing a consistent guided integrated slot filling architecture in tourism scenarios, combining multiple learners and arbitration algorithms, the problem of unclear slot boundaries is solved, the accuracy and evaluation quality of slot filling are improved, and the tourism scenarios are improved The extraction effect of the task-based dialogue system.

CN118690742BActive Publication Date: 2024-11-05SHANDONG UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411172448.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-05
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

The existing slot filling method fails to effectively solve the problem of unclear slot boundaries in tourism scenarios, resulting in poor actual draw performance and traditional evaluation indicators are not enough to reflect the final draw quality.

Method used

A tourist scene slot filling architecture CGSSF based on consistency guidance integration is designed, combining the main base learner, auxiliary base learner and meta learner, using pre-trained language model, lexical analysis model and conditional random field to pass consistency Arbitration algorithm and the similarity index of the difference coefficient DCSI improve the accuracy and evaluation quality of slot filling.

Benefits of technology

It significantly reduces the possibility of blurred slot boundaries, improves the accuracy and information integrity of slot filling, and reflects the extraction effect more realistically through DCSI indicators, improving the application effect of task-based dialogue systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118690742B_ABST
    Figure CN118690742B_ABST
Patent Text Reader

Abstract

The present invention discloses a tourism scene slot filling method for boundary-sensitive data, which belongs to the technical field of natural language processing. The present invention constructs a tourism scene slot filling framework based on consistency-guided integration, and the framework includes a main base learner, an auxiliary base learner and a meta-learner. The main base learner adopts a slot filling model of a pre-trained language model, the auxiliary base learner adopts a lexical analysis model and a lightweight slot filling model based on a conditional random field, and the meta-learner adopts a consistency arbitration algorithm to fill slots based on the tourism scene slot filling framework; at the same time, a difference coefficient similarity index is constructed to evaluate the effect of the tourism scene slot filling framework. The present invention improves the accuracy and information integrity of slot filling, and can also help to more accurately identify and process key information in tourism scenes, thereby improving the practical application effect of task-based dialogue systems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a tourism scene slot filling method for boundary-sensitive data. Background Art

[0002] Existing pipeline task-based dialogue systems can generally be divided into natural language understanding, dialogue state tracking, dialogue strategy, and natural language generation modules. Slot filling is one of the key tasks under the natural language understanding module, which aims to mark and identify the meaningful parts of the sentence and associate them with pre-defined labels. Through slot filling, text can be converted into structured data for easy processing in the next step.

[0003] Typically, slot filling is considered a sequence labeling task, that is, a multi-classification is performed on each word in the text. Existing methods use the sequence labeling paradigm to solve the slot filling problem, and use indicators such as precision and recall for evaluation, achieving high performance. However, existing models do not solve the problem of unclear slot boundaries very well. In tourism scenarios, some important information, such as time, phone number, place name, etc., are boundary-sensitive data. The existence of such data leads to surface indicators such as model accuracy being very high, but the actual extraction performance is often unsatisfactory. For example, suppose the correctly predicted slot value pair is (address, Shanghai, China), (telephone number, 12345678901), and the slot value pair predicted by the existing model is (address, China), (telephone number, 1234567890). Due to the fuzziness of the boundary, the extracted key information is biased, so the actual extraction performance is very poor. It can also be concluded that traditional evaluation indicators such as precision often cannot well reflect the actual extraction quality.

[0004] Most existing methods that attempt to solve the problem of slot boundary fuzziness focus on the model level, and try to solve this problem to a certain extent by combining information such as position encoding. However, these methods may be accompanied by problems such as complex models and low interpretability. At the same time, most methods use traditional deep learning indicators for evaluation, which only shows that the model has good performance in sequence labeling tasks, but does not mean that the final extraction quality is high, and it cannot guarantee that those boundary-sensitive data can be accurately extracted.

[0005] In summary, the existing methods have the following problems.

[0006] 1. In tourism scenarios, the existing slot filling methods do not really solve the problem of unclear slot boundaries, resulting in poor actual extraction performance and inaccurate slot value prediction results. It is also accompanied by problems such as model complexity and low interpretability.

[0007] 2. The evaluation indicators used by existing methods are not sufficient to reflect the final extraction quality: Existing methods use indicators such as accuracy and recall rate to focus only on the deep learning task itself, and do not pay attention to the quality of the final extraction results. They ignore the completeness and accuracy of the results, resulting in deviations in the key information extracted in tourism scenarios. Summary of the invention

[0008] In order to solve the above problems, the present invention proposes a tourism scene slot filling method for boundary-sensitive data, and designs a tourism scene slot filling architecture CGSSF (Consistency-Guided Stacking for Slot Filling) based on consistency-guided integration. This architecture ensures that the possibility of boundary fuzziness is greatly reduced while not making the model overly complex, significantly improving the actual performance of slot filling in tourism scene task-based dialogues and improving the final extraction quality. In addition, the present invention also proposes a difference-based similarity measurement index DCSI (Difference Coefficient Similarity Index), which can better test the quality of slot value pairs extracted by the model and more truly reflect the actual extraction effect.

[0009] The technical solution of the present invention is as follows:

[0010] A tourism scene slot filling method for boundary-sensitive data is proposed. A tourism scene slot filling framework based on consistency-guided integration is constructed. The framework includes a main base learner, an auxiliary base learner and a meta-learner. The main base learner adopts a slot filling model of a pre-trained language model, the auxiliary base learner adopts a lexical analysis model and a lightweight slot filling model based on a conditional random field, and the meta-learner adopts a consistency arbitration algorithm. The slot filling method includes the following steps:

[0011] Step 1: Obtain boundary-sensitive data to build a tourism scene dataset;

[0012] Step 2: Build and fine-tune the slot filling model based on the pre-trained language model, and obtain the predicted slot value pairs based on the main base learner;

[0013] Step 3: Build and train a lexical analysis model and a lightweight slot filling model based on conditional random fields, and obtain spare slot value pairs based on the auxiliary basis learner;

[0014] Step 4: Construct a meta-learner based on the consistency arbitration algorithm, obtain the final result based on the predicted slot value pair and the backup slot value pair, and then complete the slot filling;

[0015] Step 5: Construct a coefficient of difference similarity index, extract slot filling tasks based on a labeled tourism scene dataset, and evaluate the effectiveness of the tourism scene slot filling framework based on consistency-guided integration.

[0016] Furthermore, in step 1, the tourism scene dataset constructed is The dataset is in the form of a conversation, which includes time, phone number, and place name; Tourism scene dataset Included samples, each sample contains several dialogues, and the number of slot categories used for sequence annotation in the entire dataset is kind.

[0017] Furthermore, the specific process of step 2 is as follows:

[0018] Step 2.1: Construct the main base learner network structure. The working process of the main base learner is:

[0019] First, use the pre-trained BERT-like encoder to encode the tourism scene data:

[0020] (1);

[0021] Among them, the hidden layer vector length of the model is , the length of the word sequence is , Represents the representation vector of each word in the sentence, represents the global pooling representation vector, and Respectively represent the sentences corresponding to the current dialogue and the historical dialogue, Indicates the first word unit; For encoder;

[0022] Then, perform tensor transformation according to the following rules to achieve feature fusion:

[0023] (2);

[0024] in, For update operations; Specify the dimensions for repeated tensor operations; Expand tensor operations for dimensions; It is a tensor concatenation operation;

[0025] Finally, the sequence labeling prediction result is obtained through the feedforward neural network; the feedforward neural network includes a linear layer, a classification layer, and an activation function layer. The specific calculation formula is as follows:

[0026] (3);

[0027] in, It is the sequence labeling prediction result obtained by feedforward neural network; and Represent the weight and bias of the linear layer respectively; is the ReLU activation function; and Represent the weight and bias of the classification layer respectively;

[0028] The sequence annotation prediction results obtained by the feedforward neural network are converted into a dictionary form to obtain a predicted slot-value pair dictionary; the key of the dictionary represents the slot, and the value corresponding to each key represents a set of several slot values;

[0029] Step 2.2: Use the cost-sensitive learning strategy to fine-tune the slot filling model based on the pre-trained language model. The specific process is as follows:

[0030] First, count the number of occurrences of each slot category in the statistical data set, and define Represents a vector that records the number of occurrences of all slot categories; slot category The formula for calculating the number of occurrences is as follows:

[0031] (4);

[0032] in, Indicates the slot category number; Indicates the slot type The number of occurrences of Tourism scene dataset Samples in is the slot category occurrence count variable; Express satisfaction exist Appeared in The value is 1 when the condition is met, otherwise it is zero;

[0033] Then, the category weight of each slot category is calculated, and the definition Represents the category weight vector that records the category weights of all slots; slot category The calculation formula for category weight is as follows:

[0034] (5);

[0035] in, express The sum of all values ​​of a vector; Indicates the slot type The category weights of is a real number;

[0036] According to the category weight vector, the first loss function is weighted , and use the gradient descent method to optimize:

[0037] (6);

[0038] in, is the cross entropy loss function; Represents the labels in one-hot encoding form in the first loss function.

[0039] Furthermore, the specific process of step 3 is as follows:

[0040] Step 3.1: Build a lexical analysis model based on semi-supervised incremental learning, and use the lexical analysis model to complete word segmentation and part-of-speech tagging; the specific process is as follows:

[0041] Get a pre-trained lexical analysis model. First, use the tourism scene dataset The lexical analysis model is used to perform fine-grained Chinese word segmentation sentence by sentence to obtain several word segment sequences. Next, the non-empty slots in the word segment sequences are annotated to obtain several pieces of partially annotated data. Then, the pre-trained lexical analysis model is used for prediction to complete the part-of-speech tagging. The unannotated parts use the tagging results generated in this round, and the rest use the results of the first round of tagging. After two rounds of tagging, a fully annotated tourism scene dataset is obtained. Finally, the fully annotated tourism scene dataset is used to incrementally train the model.

[0042] Step 3.2: Build and train a lightweight slot filling model based on conditional random fields. The working process of the lightweight slot filling model based on conditional random fields is as follows:

[0043] Use the pre-trained static word vector table to obtain the word meaning vector of each word in the word segment sequence, and use the word2vec method to obtain the part-of-speech vector corresponding to each part of speech, and match it to each part of speech in the sequence; concatenate the two to obtain a vector representation that combines word meaning and part-of-speech features:

[0044] (7);

[0045] in, and They represent the word meaning tensor and part-of-speech tensor corresponding to the word segment sequence respectively. represents the length of word granularity sequence, represents the dimension of word sense vector, Represents the part-of-speech vector dimension; The vector representation result after concatenation is represented;

[0046] The concatenated vector representation result is processed using a bidirectional gated recurrent unit:

[0047] (8);

[0048] in, is the processed tensor, is the hidden layer dimension size; Represents the constructed bidirectional gated unit model;

[0049] Use conditional random fields to complete the final classification and get the result:

[0050] (9);

[0051] in, It is the sequence labeling prediction result obtained by conditional random field; Represents the constructed conditional random field model;

[0052] Convert the sequence annotation prediction results obtained through the conditional random field into a dictionary form to obtain a dictionary of alternate slot-value pairs;

[0053] Finally, the second loss function is calculated using cross entropy , optimize the lightweight slot filling model:

[0054] (10);

[0055] in, Represents the labels in one-hot encoding form in the second loss function.

[0056] Furthermore, in step 4, the similarity calculation formula is defined as follows:

[0057] (11);

[0058] in, Representation calculation and Similarity score of two slot value strings; Indicates finding the maximum value of all parameters; Express request and The longest common substring of two slot value strings; Indicates the length;

[0059] The specific process of arbitration by the consistency arbitration algorithm is as follows: for each slot that appears in the predicted slot value pair dictionary output by the main base learner, check whether there is a corresponding result in the backup slot value pair dictionary output by the auxiliary base learner; if there is a corresponding slot and the slot value results are consistent, then it can be added to the final arbitration result without additional processing; if there is a corresponding slot but the slot value results are inconsistent, similarity calculation is performed based on formula (11), and a similarity threshold is pre-defined. If the similarity is greater than the threshold, the slot value boundary is adjusted to correct the slot value result, and the corrected slot value is added to the final arbitration result; if there is no corresponding slot, the slot value corresponding to the current slot is taken out, and the slot value most similar to the slot value corresponding to the current slot is found and taken out from all the slot values ​​of the auxiliary base learner. The similarity between the taken out slot value and the slot value corresponding to the current slot is calculated based on formula (11). If the similarity is greater than the threshold, the slot value boundary is adjusted to correct the slot value result, and the corrected slot value is added to the final arbitration result; finally, the slot filling is completed according to the final arbitration result.

[0060] Furthermore, in step 5, a difference coefficient similarity index is designed, and the formula is:

[0061] (12);

[0062] in, is the coefficient of difference similarity index; is the set of predicted values; is a collection of tags; Express satisfaction The value is 1 when the condition is met, otherwise it is zero; When , the difference coefficient similarity index is simplified to:

[0063] (13);

[0064] In addition to the form of formula (13), there is another method to calculate the difference coefficient similarity index. The two methods are equivalent. When using, choose the one that is easiest to implement. Another method to calculate the difference coefficient similarity index is as follows:

[0065] (14).

[0066] Furthermore, in step 5, the slot-value pairs extracted from the slot filling task are represented as a "key-value set", that is, stored in a key-value dictionary data structure; the slot-value pair dictionaries of the prediction and label are respectively and , the sets of dictionary keys in predictions and labels are , ,make:

[0067] (15);

[0068] in, is the intersection of the sets of dictionary keys in predictions and labels;

[0069] Then we have:

[0070] (16);

[0071] in, for and The similarity score obtained by the difference coefficient similarity index; Slot Type A dictionary of predicted slot-value pairs; Slot Type A dictionary of label-slot-value pairs;

[0072] Formula (16) is used to evaluate the results of the slot filling task extraction. If the results meet expectations, the tourism scene slot filling framework based on consistency-guided integration is deployed and put into use. If the results do not meet expectations, steps 2 to 4 are repeated for retraining until expectations are met.

[0073] The beneficial technical effects brought about by the present invention are as follows.

[0074] 1. It helps to improve the accuracy and completeness of slot filling. This invention significantly reduces the possibility of slot boundary ambiguity by designing the consistency-guided integrated architecture CGSSF, so that the task-based dialogue system in the tourism scenario can extract key information more accurately. Compared with traditional methods, this method can more effectively identify and mark integrity-sensitive data, such as time, place names, telephone numbers, etc., thereby improving the accuracy of slot filling and information completeness.

[0075] 2. It helps to improve the quality of slot filling evaluation. The DCSI (Difference Coefficient Similarity Index) proposed in this invention provides a new measurement method for the actual extraction quality of the model. Compared with traditional indicators such as accuracy and recall, DCSI can more truly reflect the extraction effect of slot-value pairs, focusing on the completeness and accuracy of the results. This new evaluation method can not only provide more efficient feedback to guide model improvement, but also help to more accurately identify and process key information in tourism scenarios, and improve the actual application effect of task-based dialogue systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flow chart of the tourism scene slot filling method for boundary-sensitive data of the present invention.

[0077] Figure 2This is a visualization diagram of the indicators of the training set in the experiment of the present invention.

[0078] Figure 3 This is a visualization diagram of the indicators of the validation set in the experiment of the present invention.

[0079] Figure 4 This is a visualization diagram of the indicators of the test set in the experiment of the present invention. DETAILED DESCRIPTION

[0080] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0081] The present invention proposes a tourism scene slot filling method for boundary-sensitive data, and constructs a tourism scene slot filling framework CGSSF based on consistency-guided integration to fill tourism scene slots; the tourism scene slot filling framework CGSSF based on consistency-guided integration includes a main base learner, an auxiliary base learner and a meta-learner based on a consistency arbitration algorithm; the main base learner includes a slot filling model fine-tuned by a cost-sensitive learning strategy; the auxiliary base learner includes a lexical analysis model trained through semi-supervised incremental training and a lightweight slot filling model based on a conditional random field; the two base learners are responsible for completing slot filling and generating preliminary tourism scene prediction results; the meta-learner is responsible for completing consistency arbitration based on the results given by the main base learner and the auxiliary base learner, and giving the final tourism scene arbitration result.

[0082] Unlike the traditional Stacking method that integrates a large number of base learners, the method of the present invention only uses two base learners. One is a slot filling model based on a pre-trained language model, and the other is a slot filling model based on lexical analysis. There are two advantages to doing this: on the one hand, integrating a large number of base learners may lead to a significant increase in computational overhead; on the other hand, the model based on the pre-trained language model (PLM) has a strong semantic understanding ability, but it is difficult to completely solve the boundary fuzziness problem. In contrast, the model based on lexical analysis has good word segmentation and part-of-speech discrimination capabilities. Although it is insufficient in semantic understanding, it can usually ensure the accuracy of word boundaries. Therefore, the two models can complement each other's strengths.

[0083] Different from the traditional Stacking method that uses a meta-learner based on machine learning or deep learning methods, the method of the present invention uses a more interpretable consistency arbitration algorithm, which is transparent inside and does not require secondary training. The basic idea is to use the algorithm to perform consistency checks on the slot value pair results predicted by the main base learner and the auxiliary base learner, and try to correct them when inconsistencies occur, so that the two base learners can perform their respective duties and give full play to their respective advantages, and try to find the most accurate slot value pair results through the meta-learner.

[0084] like Figure 1 As shown, a tourism scene slot filling method for boundary-sensitive data specifically includes the following steps:

[0085] Step 1: Obtain boundary-sensitive data to build a tourism scene dataset The dataset is in the form of a conversation, which includes boundary-sensitive data such as time, phone number, and place name; Tourism scene dataset Included samples, each sample contains several dialogues, and the number of slot categories used for sequence annotation in the entire dataset is kind.

[0086] Step 2: Build the main base learner in the framework, build and fine-tune the slot filling model based on the pre-trained language model in the main base learner, and obtain the predicted slot value pairs based on the main base learner.

[0087] The main base learner includes a slot filling model based on a pre-trained language model. The model is fine-tuned based on a cost-sensitive learning strategy, and then can predict slot-value pairs. The basic idea is to first build a network structure, then use a cost-sensitive learning strategy to optimize through gradient descent, and finally predict the slot-value pairs through inference of the trained model.

[0088] The input of the slot filling model based on the pre-trained language model is the tourism scene dataset , the output is the slot-value pair prediction result.

[0089] Step 2.1: Construct the main base learner network structure. The working process of the main base learner is:

[0090] First, use the pre-trained BERT-like encoder to encode the tourism scene data. Select the appropriate BERT series model, and use the encoder instead, then:

[0091] (1);

[0092] Among them, the hidden layer vector length of the model is , the length of the word sequence is , Represents the representation vector of each word in the sentence, represents the global pooling representation vector, and Respectively represent the sentences corresponding to the current dialogue and the historical dialogue, Indicates the first word unit; For the encoder.

[0093] Then, perform tensor transformation according to the following rules to achieve feature fusion:

[0094] (2);

[0095] in, For update operation; Specify the dimensions for repeated tensor operations; Expand tensor operations for dimensions; It is a tensor concatenation operation; , and The first parameter indicates the tensor to be operated on, and the second parameter indicates the dimension of the transformation.

[0096] Finally, the sequence annotation prediction result is obtained through the feedforward neural network. The feedforward neural network (FFN) includes a linear layer, a classification layer, and an activation function layer; the calculation formula is as follows:

[0097] (3);

[0098] in, It is the sequence labeling prediction result obtained by feedforward neural network; and Represent the weight and bias of the linear layer respectively; is the ReLU activation function; and Represent the weight and bias of the classification layer respectively;

[0099] The sequence annotation prediction results obtained by the feedforward neural network are converted into a dictionary form. The key of the dictionary represents the slot, and the value corresponding to each key represents a set of several slot values, that is, the predicted slot value pair dictionary is obtained. ; It is the prediction result of the main base learner and also the input of the meta learner.

[0100] Step 2.2: Use a cost-sensitive learning strategy to fine-tune the slot filling model based on the pre-trained language model.

[0101] The slot filling task is essentially a multi-classification problem. However, due to the characteristics of the language itself, it is almost impossible to make all categories of slots evenly distributed. Most of the words correspond to empty slots, and meaningful slots are also unevenly distributed. Therefore, the slot filling task faces a serious long-tail distribution problem. The cost-sensitive learning strategy can alleviate the problem caused by the long-tail distribution to a certain extent.

[0102] First, count the number of occurrences of each slot category in the statistical data set, and define Represents a vector that records the number of occurrences of all slot categories; slot category The formula for calculating the number of occurrences is as follows:

[0103] (4);

[0104] in, Indicates the slot category number; Indicates the slot type The number of occurrences of Tourism scene dataset Samples in is the slot category occurrence count variable; Express satisfaction exist Appeared in The value is 1 if the condition is met, otherwise it is zero.

[0105] Then, the category weight of each slot category is calculated, and the definition Represents the category weight vector that records the category weights of all slots. The category weight vector is used for cost-sensitive learning; slot category The calculation formula for category weight is as follows:

[0106] (5);

[0107] in, express The sum of all values ​​of a vector; Indicates the slot type The category weights of is a real number.

[0108] According to the obtained category weight vector, the first loss function is weighted , and use the gradient descent method to optimize:

[0109] (6);

[0110] in, is the cross entropy loss function; Represents the label in the one-hot encoding form in the first loss function. The cross entropy loss function in formula (6) uses category weights, and different items need to be weighted when calculating the loss value.

[0111] Step 3: Build the auxiliary base learner in the framework, build and train the lexical analysis model and the lightweight slot filling model based on conditional random fields, and obtain the spare slot value pairs based on the auxiliary base learner;

[0112] The auxiliary base learner includes a lexical analysis model and a lightweight slot filling model based on conditional random fields. The basic idea is to obtain a pre-trained lexical analysis model, perform semi-supervised incremental training on relevant corpus to complete word segmentation and part-of-speech tagging, and focus on ensuring the correct prediction results at key slots; then build a lightweight slot filling model based on conditional random fields and perform supervised training with existing data.

[0113] Although the auxiliary base learner uses two models, both are relatively lightweight, and its slot filling performance is obviously inferior to that of the main base learner using a pre-trained language model. However, since the slot filling model in the auxiliary base learner introduces word granularity features, the probability of word boundary fuzziness will be significantly reduced, so the existence of the auxiliary base learner can effectively make up for the shortcomings of the main base learner.

[0114] The specific process of step 3 is:

[0115] Step 3.1: Build a lexical analysis model based on semi-supervised incremental learning, use the lexical analysis model to complete word segmentation and part-of-speech tagging, and obtain word segment sequences and corresponding part-of-speech sequences; the specific process is as follows:

[0116] First, obtain a pre-trained lexical analysis model that can perform word segmentation and part-of-speech tagging at the same time. It must have good general performance and meet the language and scenario requirements of the task.

[0117] Tourism scene dataset This is the form of a round-by-round dialogue in a tourism scenario, so for the tourism scenario dataset , use the lexical analysis model to perform fine-grained Chinese word segmentation sentence by sentence, and obtain several word segment sequences; then annotate the non-empty slots in the word segment sequence to ensure that the annotations of these key parts are as accurate as possible, and obtain several partially annotated data. This process can use a generative large language model to assist in annotation, reducing labor costs. Taking the PKU standard as an example, the annotation examples are shown in the following table:

[0118] Table 1 Annotation example table

[0119] .

[0120] Then, the pre-trained lexical analysis model is used for prediction and part-of-speech tagging is completed. The untagged parts use the tagging results generated in this round, and the rest use the results of the first round of tagging. After two rounds of tagging, a completely annotated tourism scene dataset is obtained.

[0121] Finally, we use the labeled tourism scene dataset to incrementally train the model to improve the performance of the model on specific scene corpora. If new style corpora are added, we can continue to perform incremental training.

[0122] Construct a dictionary of proper nouns, such as the CrossWOZ dataset used in the present invention, that is, a dictionary of scenic spot names, restaurant names, hotel names, and recommended dish names can be constructed, such as "Badaling Great Wall", "Beijing Roast Duck", etc. In the process of part-of-speech tagging, a table lookup method is used. If it is a proper noun that appears in the table, the label in the table can be used.

[0123] Step 3.2: Build and train a lightweight slot filling model based on conditional random fields. The working process of the lightweight slot filling model based on conditional random fields is as follows:

[0124] Use the pre-trained static word vector table to obtain the word meaning vector of each word in the word segment sequence, and use the word2vec method to obtain the part-of-speech vector corresponding to each part of speech and match it to each part of speech in the sequence. The two are concatenated to obtain a vector representation that combines word meaning and part-of-speech features.

[0125] (7);

[0126] in, and They represent the word meaning tensor and part-of-speech tensor corresponding to the word segment sequence respectively. represents the length of word granularity sequence, represents the dimension of word sense vector, Represents the part-of-speech vector dimension; The vector representation result after concatenation is represented;

[0127] The concatenated vector representation result is processed using a bidirectional gated recurrent unit:

[0128] (8);

[0129] in, is the processed tensor, is the hidden layer dimension size; Represents the constructed bidirectional gated unit model, the input size of which is .

[0130] Use conditional random fields to complete the final classification and get the result:

[0131] (9);

[0132] in, It is the sequence labeling prediction result obtained by conditional random field; Represents the constructed conditional random field model;

[0133] The sequence annotation prediction results obtained by the conditional random field are converted into a dictionary form. The key of the dictionary represents the slot, and the value corresponding to each key represents a set of several slot values, that is, the dictionary of spare slot value pairs is obtained. , That is the prediction result of the auxiliary base learner;

[0134] Finally, the second loss function is calculated using cross entropy , optimize the lightweight slot filling model:

[0135] (10);

[0136] in, Represents the label in the form of one-hot encoding in the second loss function. Due to the different word segmentation granularity, when calculating the cross entropy loss function, formula (10) is not exactly the same as formula (6). Formula (10) does not use category weights. When calculating the loss value, different items do not need to be weighted. That is, the labels can be converted by combining a simple string processing algorithm.

[0137] Step 4: Build a meta-learner based on the consistency arbitration algorithm in the framework to complete the consistency arbitration, obtain the final arbitration result based on the predicted slot value pair and the backup slot value pair, and then complete the slot filling;

[0138] The meta-learner needs to make a final arbitration on the results output by the two base learners and output the final arbitration result. For this purpose, the present invention proposes a consistency arbitration algorithm for slot value pairs. The basic idea of ​​the algorithm is to use the prediction results of the auxiliary base learner to check the prediction results of the main base learner, use the similarity algorithm based on the longest common substring to measure the consistency, and make arbitration based on whether the consistency is greater than the threshold, thereby realizing the verification and correction of the slot value of the main base learner.

[0139] The specific process of arbitration by the consistency arbitration algorithm is as follows:

[0140] For each slot that appears in the predicted slot value dictionary pair output by the main base learner, check whether there is a corresponding result in the backup slot value pair dictionary output by the auxiliary base learner. If there is a corresponding slot and the slot value results are consistent, it can be added to the final arbitration result without additional processing; if there is a corresponding slot but the slot value results are inconsistent, similarity calculation is performed based on the algorithm given in formula (11), and a similarity threshold is predefined. If the similarity is greater than the threshold, try to adjust the slot value boundary to correct the slot value result, and add it to the final arbitration result after correction; if there is no corresponding slot, take out the slot value corresponding to the current slot, try to find and take out the slot value most similar to the slot value corresponding to the current slot from all the slot values ​​of the auxiliary base learner, and calculate the similarity between the taken slot value and the slot value corresponding to the current slot based on the algorithm given in formula (11). If the similarity is greater than the threshold, try to adjust the slot value boundary to correct the slot value result, and add it to the final arbitration result after correction. The final arbitration result is in the form of a slot value pair, and slot filling is completed based on the slot value pair.

[0141] This consistency arbitration algorithm can ensure that some relatively ambiguous slot values ​​in tourism scenarios are accurately extracted and matched to the correct slots. For example, the slot values ​​corresponding to the two slots "tourist attraction hours" and "restaurant business hours" are both related to time, and it is easy to misidentify which category they belong to without combining the context; the two slot values ​​"Beijing" and "Beijing Haidian District" under the location slot are obviously incomplete. The consistency arbitration algorithm takes into account both types of problems because it makes full use of the context understanding ability of the main base learner and the boundary identification ability of the auxiliary base learner.

[0142] (11);

[0143] in, Representation calculation and Similarity score of two slot value strings; It means to find the maximum value of all parameters. Express request and The longest common substring of two slot value strings, Indicates the length.

[0144] Step 5: Construct a coefficient of difference similarity index, extract slot filling tasks based on the labeled tourism scene dataset, and evaluate the effect of the tourism scene slot filling framework based on consistency-guided integration. If the extraction effect meets expectations, it can be deployed and put into use.

[0145] DCSI is a consistency evaluation index based on set difference proposed by the present invention. Its full name is Difference Coefficient Similarity Index. It assumes that the predicted value set and the label set are represented by and Indicates that:

[0146] (12);

[0147] in, is the coefficient of difference similarity index; Express satisfaction The value is 1 when the condition is met, otherwise it is zero. , the indicator can be simplified as:

[0148] (13);

[0149] In addition to the form of formula (13), there is another calculation method for this indicator. The two methods are equivalent. When using, just choose the one that is easier to implement. Another calculation method is as follows:

[0150] (14);

[0151] The slot-value pairs extracted by the slot filling task can be represented as a "key-value set", that is, they can be stored in a key-value dictionary data structure. The slot-value pair dictionaries for prediction and label are and , the sets of dictionary keys in predictions and labels are , ,make:

[0152] (15);

[0153] in, is the intersection of the sets of dictionary keys in predictions and labels;

[0154] Then we have:

[0155] (16);

[0156] in, for and The similarity score obtained by the difference coefficient similarity index; Slot Type A dictionary of predicted slot-value pairs; Slot Type A dictionary of label-slot-value pairs;

[0157] Formula (16) is used to evaluate the extraction results of the slot filling task, which can more realistically reflect the extraction quality. If the expected result is met, the tourism scene slot filling framework based on consistency-guided integration built earlier can be deployed and put into use. If the expected result is not met, steps 2 to 4 are repeated for retraining until the expected result is met.

[0158] The advantages of the evaluation index designed in this invention are: (1) the value range is The larger the index, the better the extraction performance. The value is 1 only when the label and the predicted result are completely consistent, which is convenient for analysis and statistics. (2) The implemented algorithm is relatively clear and concise, and only two basic operations, set difference operation and number of set elements, are used. (3) Focusing on the final extraction result rather than the sequence labeling result can effectively avoid the problem of inflated indicators and truly reflect the actual extraction effect of the slot-value pair.

[0159] Next, we will provide a mathematical proof of advantage (1) by combining knowledge of set theory.

[0160] For any two different sets and , given the variable , according to the properties of the set, there is always:

[0161] (17);

[0162] therefore:

[0163] (18);

[0164] therefore:

[0165] (19);

[0166] The equality sign is obtained if and only if the difference of two sets is an empty set. When , the following formula always holds true:

[0167] (20);

[0168] exist In this case, formula (19) can be deduced as follows:

[0169] (twenty one);

[0170] exist and When the number of elements in is fixed, the more the number of intersection elements, The larger the value of , the more it can reflect the similarity of the two sets to a certain extent. The proof is complete.

[0171] In order to prove the feasibility and superiority of the present invention, the following comparative experiments are given. The present invention conducts experiments on a large Chinese task-oriented dialogue dataset CrossWOZ, considering three fields: hotels, restaurants, and attractions. Of course, the method proposed in the present invention is not limited to these three fields, and can be used in more task-oriented dialogue corpora in tourism scenarios.

[0172] Table 2 Sample set description table

[0173] .

[0174] Table 3 Experimental results

[0175] .

[0176] CrossWOZ consists of three parts: training set, validation set and test set. Only the data of the training set is used in the model training process. In order to more fully demonstrate the results of the method of the present invention in different scenarios, the present invention divides all sample sets in CrossWOZ into 7 categories as shown in Table 2, namely Ⅰ, Ⅱ, Ⅲ, Ⅳ, Ⅴ, Ⅵ, and Ⅶ. Experiments are carried out separately to show the performance differences of the method of the present invention under different circumstances. The experimental results are shown in Table 3. During the experiment, the model used by the baseline method is the model of the main base learner. No optimization strategy proposed by the present invention is used, and the training hyperparameters remain consistent. The evaluation index uses the DCSI index proposed by the present invention.

[0177] In order to make a more intuitive comparison, the present invention uses a radar chart to visualize the data. The visualization results of the training set, validation set, and test set are shown in the following figure. Figure 2-Figure 4 As shown. Figure 2-Figure 4 It can be seen that the method proposed in the present invention is better and more robust than the traditional method in terms of training set, validation set and test set. The traditional method does not specifically solve the problem of fuzzy boundaries, resulting in seemingly high indicators but poor actual extraction effects. The method proposed in the present invention, combined with ensemble learning, greatly improves the extraction performance without incurring syntactic computational costs, thereby improving the quality of the final extraction results.

[0178] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A tourism scene slot filling method for boundary-sensitive data, characterized in that: A tourism scene slot filling framework based on consistency-guided integration is constructed. The framework includes a main base learner, an auxiliary base learner and a meta-learner. The main base learner adopts a slot filling model of a pre-trained language model, the auxiliary base learner adopts a lexical analysis model and a lightweight slot filling model based on conditional random fields, and the meta-learner adopts a consistency arbitration algorithm. The slot filling method includes the following steps: Step 1: Obtain boundary-sensitive data to build a tourism scene dataset; Step 2: Build and fine-tune the slot filling model based on the pre-trained language model, and obtain the predicted slot value pairs based on the main base learner; Step 3: Build and train a lexical analysis model and a lightweight slot filling model based on conditional random fields, and obtain spare slot value pairs based on the auxiliary basis learner; Step 4: Construct a meta-learner based on the consistency arbitration algorithm, obtain the final result based on the predicted slot value pair and the backup slot value pair, and then complete the slot filling; Step 5: Construct a coefficient of difference similarity index, extract slot filling tasks based on a labeled tourism scene dataset, and evaluate the effectiveness of the tourism scene slot filling framework based on consistency-guided integration.

2. The tourism scene slot filling method for boundary-sensitive data according to claim 1 is characterized in that: In step 1, the tourism scene dataset constructed is The dataset is in the form of a conversation, which includes time, phone number, and place name; Tourism scene dataset Included samples, each sample contains several dialogues, and the number of slot categories used for sequence annotation in the entire dataset is kind.

3. The tourism scene slot filling method for boundary-sensitive data according to claim 2 is characterized in that: The specific process of step 2 is: Step 2.1: Construct the main base learner network structure. The working process of the main base learner is: First, use the pre-trained BERT-like encoder to encode the tourism scene data: (1); Among them, the hidden layer vector length of the model is , the length of the word sequence is , Represents the representation vector of each word in the sentence, represents the global pooling representation vector, and Respectively represent the sentences corresponding to the current dialogue and the historical dialogue, Indicates the first word unit; For encoder; Then, perform tensor transformation according to the following rules to achieve feature fusion: (2); in, For update operations; Specify the dimensions for repeated tensor operations; Expand tensor operations for dimensions; It is a tensor concatenation operation; Finally, the sequence labeling prediction result is obtained through the feedforward neural network; the feedforward neural network includes a linear layer, a classification layer, and an activation function layer. The specific calculation formula is as follows: (3); in, It is the sequence labeling prediction result obtained by feedforward neural network; and Represent the weight and bias of the linear layer respectively; is the ReLU activation function; and Represent the weight and bias of the classification layer respectively; The sequence annotation prediction results obtained by the feedforward neural network are converted into a dictionary form to obtain a predicted slot-value pair dictionary; the key of the dictionary represents the slot, and the value corresponding to each key represents a set of several slot values; Step 2.2: Use the cost-sensitive learning strategy to fine-tune the slot filling model based on the pre-trained language model. The specific process is as follows: First, count the number of occurrences of each slot category in the statistical data set, and define Represents a vector that records the number of occurrences of all slot categories; slot category The formula for calculating the number of occurrences is as follows: (4); in, Indicates the slot category number; Indicates the slot type The number of occurrences of Tourism scene dataset Samples in is the slot category occurrence count variable; Express satisfaction exist Appeared in The value is 1 when the condition is met, otherwise it is zero; Then, the category weight of each slot category is calculated, and the definition Represents the category weight vector that records the category weights of all slots; slot category The calculation formula for category weight is as follows: (5); in, express The sum of all values ​​of a vector; Indicates the slot type The category weights of is a real number; According to the category weight vector, the first loss function is weighted , and use the gradient descent method to optimize: (6); in, is the cross entropy loss function; Represents the labels in one-hot encoding form in the first loss function.

4. The tourism scene slot filling method for boundary-sensitive data according to claim 3 is characterized in that: The specific process of step 3 is as follows: Step 3.1: Build a lexical analysis model based on semi-supervised incremental learning, and use the lexical analysis model to complete word segmentation and part-of-speech tagging; the specific process is as follows: Get a pre-trained lexical analysis model. First, use the tourism scene dataset The lexical analysis model is used to perform fine-grained Chinese word segmentation sentence by sentence to obtain several word segment sequences. Next, the non-empty slots in the word segment sequences are annotated to obtain several pieces of partially annotated data. Then, the pre-trained lexical analysis model is used for prediction to complete the part-of-speech tagging. The unannotated parts use the tagging results generated in this round, and the rest use the results of the first round of tagging. After two rounds of tagging, a fully annotated tourism scene dataset is obtained. Finally, the fully annotated tourism scene dataset is used to incrementally train the model. Step 3.2: Build and train a lightweight slot filling model based on conditional random fields. The working process of the lightweight slot filling model based on conditional random fields is as follows: Use the pre-trained static word vector table to obtain the word meaning vector of each word in the word segment sequence, and use the word2vec method to obtain the part-of-speech vector corresponding to each part of speech, and match it to each part of speech in the sequence; concatenate the two to obtain a vector representation that combines word meaning and part-of-speech features: (7); in, and They represent the word meaning tensor and part-of-speech tensor corresponding to the word segment sequence respectively. represents the length of word granularity sequence, represents the dimension of word sense vector, Represents the part-of-speech vector dimension; The vector representation result after concatenation is represented; Use a bidirectional gated recurrent unit to process the concatenated vector representation result: (8); in, is the processed tensor, is the hidden layer dimension size; Represents the constructed bidirectional gated unit model; Use conditional random fields to complete the final classification and get the result: (9); in, It is the sequence labeling prediction result obtained by conditional random field; Represents the constructed conditional random field model; Convert the sequence annotation prediction results obtained through the conditional random field into a dictionary form to obtain a dictionary of alternate slot-value pairs; Finally, the second loss function is calculated using cross entropy , optimize the lightweight slot filling model: (10); in, Represents the labels in one-hot encoding form in the second loss function.

5. The tourism scene slot filling method for boundary-sensitive data according to claim 4 is characterized in that: In step 4, the similarity calculation formula is defined as follows: (11); in, Representation calculation and Similarity score of two slot value strings; Indicates finding the maximum value of all parameters; Express request and The longest common substring of two slot value strings; Indicates the length; The specific process of arbitration by the consistency arbitration algorithm is as follows: for each slot that appears in the predicted slot value pair dictionary output by the main base learner, check whether there is a corresponding result in the backup slot value pair dictionary output by the auxiliary base learner; if there is a corresponding slot and the slot value results are consistent, then it can be added to the final arbitration result without additional processing; if there is a corresponding slot but the slot value results are inconsistent, similarity calculation is performed based on formula (11), and a similarity threshold is pre-defined. If the similarity is greater than the threshold, the slot value boundary is adjusted to correct the slot value result, and the corrected slot value is added to the final arbitration result; if there is no corresponding slot, the slot value corresponding to the current slot is taken out, and the slot value most similar to the slot value corresponding to the current slot is found and taken out from all the slot values ​​of the auxiliary base learner. The similarity between the taken out slot value and the slot value corresponding to the current slot is calculated based on formula (11). If the similarity is greater than the threshold, the slot value boundary is adjusted to correct the slot value result, and the corrected slot value is added to the final arbitration result; finally, the slot filling is completed according to the final arbitration result.

6. The tourism scene slot filling method for boundary-sensitive data according to claim 5 is characterized in that: In step 5, the difference coefficient similarity index is designed, and the formula is: (12); in, is the coefficient of difference similarity index; is the set of predicted values; is a collection of tags; Express satisfaction The value is 1 when the condition is met, otherwise it is zero; When , the difference coefficient similarity index is simplified to: (13); In addition to the form of formula (13), there is another method to calculate the difference coefficient similarity index. The two methods are equivalent. When using, choose the one that is easiest to implement. Another method to calculate the difference coefficient similarity index is as follows: (14)。 7. The tourism scene slot filling method for boundary-sensitive data according to claim 6 is characterized in that: In step 5, the slot-value pairs extracted from the slot filling task are represented as "key-value sets", that is, stored in a key-value dictionary data structure; the slot-value pair dictionaries of the prediction and label are respectively and , the sets of dictionary keys in predictions and labels are , ,make: (15); in, is the intersection of the sets of dictionary keys in predictions and labels; Then we have: (16); in, for and The similarity score obtained by the difference coefficient similarity index; Slot Type A dictionary of predicted slot-value pairs; Slot Type A dictionary of label-slot-value pairs; Formula (16) is used to evaluate the results of the slot filling task extraction. If the results meet expectations, the tourism scene slot filling framework based on consistency-guided integration is deployed and put into use. If the results do not meet expectations, steps 2 to 4 are repeated for retraining until expectations are met.