Intelligent home user spoken language understanding system oriented to environmental information augmentation

Abstract

The application provides a smart home user spoken language understanding system facing environmental information augmentation, the system divides and stores source domain and target domain data files according to intention domain; training and testing tasks are constructed by sampling on the source domain and the target domain; text and environmental information are extracted and standardized organized, filled and spliced; intention and slot information are extracted and intention and slot dictionaries are constructed; a pre-training language model, a graph convolution network, a label attention layer and a mutual attention layer are used to realize text and environmental information coding; an attention mechanism and a similarity matrix of environmental information and text semantic information of intention and slot labels are used to realize the fusion of user speech text information and domain general and domain augmented environmental information; a support set and a query set are used for few-shot training and testing on the source domain and the target domain; vector coded intention and slot categories are converted into visual text output. The application provides a user spoken language understanding solution fusing environmental information.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of information technology, and relates to a technology for mining text and its related environmental information, in particular to an intelligent home user spoken language understanding system oriented to environmental information augmentation, which adopts text encoding, attention mechanism and other text information processing and fusion methods, uses a few-shot learning method such as a prototype network to perform intent recognition and key semantic component extraction on user spoken language text, and provides an intelligent home robot using the system with an understanding of user speech, so as to make a response in line with user needs on this basis. BACKGROUND

[0002] In an intelligent home system, a task-oriented dialogue system is widely used, and user spoken language understanding is an important component of the dialogue system. User spoken language understanding includes two sub-tasks of intent classification and slot filling, which aims to extract intent and key semantic components from user speech, help the intelligent home robot understand the intent of user speech and the key information therein, and make a reasonable response in dialogue management, which has important practical significance.

[0003] Early technologies related to the user spoken language understanding task usually use implicit joint models, which share the underlying word encoding between the two sub-tasks of intent classification and slot filling. Although this method can directly combine and share knowledge, it cannot explicitly model the interaction between sub-tasks, and the model performance and interpretability are poor. The explicit joint model after that builds a one-way or bidirectional information interaction flow between intent and slot, explicitly controls the knowledge transfer between tasks, captures shared knowledge across tasks, and improves the model performance.

[0004] The above user spoken language understanding model cannot solve the problem of ambiguous semantics of user speech in actual scenarios, so the existing technology combines additional environmental information such as entity knowledge graph information, user preference information, and user state information in the speech to handle speech ambiguity, and the implementation method is called user spoken language understanding based on environmental information. The user spoken language understanding model based on environmental information fuses user spoken language text information and vectorized environmental information through an attention mechanism, and performs well on intents with sufficient and uniform data, but its disadvantage is that it cannot adapt to new intent domains with unknown environmental characteristics and only a small number of labeled samples.

[0005] In addition, with the rapid updating of smart home products, new intents and environmental features appear rapidly. However, in the newly emerging unknown field, the amount of sample data is usually small, and the model training has the problem of overfitting. The existing few-shot spoken language understanding technology usually uses a metric-based method, an optimization-based method or a retrieval-based method to solve this problem. The metric-based method, such as the Prototypical Network, the Matching Network, the Induction Network and the like, models the semantic similarity between input data, learns the output coding space or the similarity metric function on the source domain with sufficient data, so that the input data with high similarity tend to be classified into the same category, thereby efficiently utilizing the small amount of labeled data on the target domain. The optimization-based method, such as MAML (Model-Agnostic Meta Learning), learns the initialization or fine-tuning process of the deep network, realizes domain migration based on a small amount of data and a small number of parameter updates. The retrieval-based method maintains an explicit retrieval index for the labeled samples, and by training a similarity model, retrieving the most similar sample to the given input, the problem of sensitivity to outliers of other few-shot methods is alleviated. The above few-shot methods can solve the problem of imbalance of data resources from different fields based on rich prior knowledge, but the method has the disadvantage that the input is only pure text without environmental information, and cannot handle user utterances with ambiguous semantics.

[0006] From the above background art, it is proposed that the smart home user spoken language understanding system for environmental information augmentation can solve the problem of ambiguous semantics of user utterances while adapting to the new intent domain with less sample data, meet the application requirements of the real world, and have high practical value. SUMMARY

[0007] The purpose of the present application is to solve the above problems, and a smart home user spoken language understanding system for environmental information augmentation is proposed, which uses an information coding fusion method based on an attention mechanism to construct user spoken language input coding that fuses text and environmental information, and then uses a few-shot learning method such as a Prototypical Network to obtain good intent recognition and slot filling performance on a target domain with less data. The technologies involved in the present application include using a pre-trained language model for text coding, using mutual attention for general environmental information fusion, using a Prototypical Network for few-shot training, and the method proposed by the present application of using a graph convolution network to interactively encode entity information triples in text obtained from an open source knowledge graph library, and using a label semantic attention matrix for augmented environmental information fusion.

[0008] The technical scheme of the present application is:

[0009] The application discloses an environment information augmentation-oriented smart home user spoken language understanding system, and relates to the technical field of smart home user spoken language understanding.

[0010] a data storage module configured to store a smart home user spoken language understanding dataset, wherein each sample contains user spoken language text, user environment information, and intent classification and slot filling annotation information corresponding to the user spoken language text; and a processing result of the domain division module;

[0011] a domain division module configured to divide an intent domain into a plurality of source domains and a plurality of target domains, and then divide samples in the smart home user spoken language understanding dataset according to the intent domain to which the samples belong, save sample information of the intent domain belonging to a source domain i to a source domain data file corresponding to the source domain i, and save sample information of the intent domain belonging to a target domain j to a target domain data file corresponding to the target domain j;

[0012] a task construction module configured to, in a few-sample training stage, sample a determined intent domain from the source domains, construct a training task according to a source domain data file corresponding to the intent domain and determine task data of the training task, wherein the task data of the training task includes a support set and a query set; and configured to, in a few-sample test stage, sample a determined intent domain from the target domains, construct a test task according to a target domain data file corresponding to the intent domain and determine task data of the test task, wherein the task data of the test task includes a support set and a query set;

[0013] a text and environment information extraction module configured to extract text and environment information of a sample from the task data and fill and splice the extracted information according to an input specification of an information coding module;

[0014] an intent and slot information extraction module configured to extract intent and slot information from the task data and construct an intent and slot dictionary table;

[0015] an information coding module configured to vectorize and encode the text and environment information of the sample;

[0016] an information fusion module configured to fuse environment information common among domains, environment information augmented among domains and user spoken language text information;

[0017] a few-sample training and test module configured to perform few-sample training of a user spoken language understanding model on the source domains and perform few-sample prediction of the user spoken language understanding model on the target domains.

[0018] Further, the text and environment information extraction module extracts the text and environment information of the sample from the task data of the current task, then obtains the knowledge graph environment information corresponding to the utterance text entity from the open source knowledge graph library, and then fills the text to the same length, and splices the various user preference environment information or user state environment information represented by vectors to provide standardized text and environment information vectors as inputs of the information coding module.

[0019] Further, the information coding module encodes the input information using a pre-trained language model to obtain pre-trained language model encodings of each entity description in the standardized user colloquial text information and the standardized knowledge graph environment information; then, based on the pre-trained language model encoding of the user colloquial text information, uses a label attention layer and a mutual attention layer to obtain user colloquial text information encoding based on intent-dependent slot representation and user colloquial text information encoding based on slot-dependent intent representation, and model the information interaction between the intent recognition and slot filling subtasks; then, taking each entity description in the knowledge graph environment information as a node, connecting the nodes corresponding to multiple entity descriptions of the same entity with edges, and connecting all entity description corresponding nodes with the node corresponding to the standardized user colloquial text information, the colloquial entity interaction graph is constructed; then, taking the pre-trained language model encoding of the entity description as the node encoding, using a graph convolution network to perform information transmission and information aggregation between adjacent nodes on the colloquial entity interaction graph, realizing the interaction between entities and the interaction between entities and user utterances, so that each word has different attention degrees for different entities, and the interaction and fusion between the user utterance and the knowledge graph environment information are encoded in a fine-grained manner; then, for each entity description, the top vector output of the graph convolution network on the node corresponding to the entity description is taken as the knowledge graph information encoding.

[0020] Further, the information fusion module uses mutual attention to fuse the domain-universal environment information, that is, splices the information encodings corresponding to the domain-universal environment information, and then uses the attention mechanism to obtain the environment information fusion text encoding based on the intent representation and the environment information fusion encoding based on the slot representation, respectively; for the domain-expanded environment information, the expanded environment information label description text semantic information is used to calculate the similarity matrix between the intent and slot label text semantic information, and the domain expansion probability distribution on the intent class and the slot class is obtained to realize the fusion of the domain-expanded environment information and the user colloquial text information.

[0021] Further, the domain-universal environment information includes knowledge graph environment information and user state environment information; and the domain-expanded environment information is user preference environment information.

[0022] Further, the few-sample training and testing module provides user spoken language understanding model training and testing services; for the training task, for all samples corresponding to each intent category on the support set of the training task, taking the mean of the environment information fusion text encoding corresponding to the user spoken language text sentence based on the intent representation as the intent category prototype, for all samples corresponding to each slot category, taking the mean of the environment information fusion text encoding corresponding to the user spoken language text word based on the slot representation as the slot category prototype, on the query set, calculating the distance between each sample and the intent category prototype and the slot category prototype to construct the category probability distribution, taking the domain expansion probability distribution in the information fusion module as the proportion coefficient with the learnable parameters to obtain the sum, taking the category corresponding to the maximum component in the probability distribution as the category prediction result, and performing gradient backpropagation to optimize the parameters in the user spoken language understanding model; for the test task, using the optimized user spoken language understanding model to take the mean of the environment information fusion text encoding corresponding to the user spoken language text sentence based on the intent representation as the intent category prototype for all samples corresponding to each intent category on the support set of the test task, and taking the mean of the environment information fusion text encoding corresponding to the user spoken language text word based on the slot representation as the slot category prototype for all samples corresponding to each slot category; then according to the distance between each test sample and the intent and slot category prototype on the query set of the test task, the intent and slot category prediction result of the corresponding test sample is obtained.

[0023] Further, the user environment information includes user preference environment information and user state environment information.

[0024] Further, the sample information is a unique number of the sample.

[0025] Further, the result output and index evaluation module is further included, which is used for encoding the few-sample test output to obtain a readable result and calculating an evaluation index value.

[0026] The advantages of the present application are as follows:

[0027] The present application introduces a few-shot spoken language understanding task based on environmental information, constructs an intelligent home user spoken language understanding system oriented to rapid expansion of environmental information, solves the problem of rapid iteration and upgrading of intelligent home products, and rapid emergence of intentions and environmental information from different fields. In the few-shot setting, as new intention groups appear, the model needs to consider more personal information such as user preferences. Therefore, more environmental information may be needed in the test stage than in the training stage, and the model parameters obtained in the training domain cannot be directly migrated to the target domain for fine-tuning. The present application uses semantic attention between intention labels and user preference labels to realize the fusion of expanded environmental information, and has the advantage of domain adaptability. For the fusion of knowledge graph information of text entities, the present application uses a graph convolution network to realize the interaction between entities and the interaction between entities and user utterances, so that each word tends to focus on different entities, and has the advantage of being able to extract fine-grained environmental information. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a structure schematic diagram of an intelligent home user spoken language understanding system oriented to environmental information expansion of an embodiment of the present application.

[0029] Figure 2 is an example diagram of an intelligent home intention domain classification tree of an embodiment of the present application.

[0030] Figure 3 is an example diagram of an intelligent home user spoken language understanding example of an embodiment of the present application. DETAILED DESCRIPTION

[0031] The present application will be further described in detail below with reference to the accompanying drawings, and the examples are only used to explain the present application and are not used to limit the scope of the present application.

[0032] Figure 1 is a structure schematic diagram of an intelligent home user spoken language understanding system oriented to environmental information expansion of an embodiment of the present application, and the functions of each module shown therein are described as follows:

[0033] Data storage module: used for storing intelligent home user spoken language understanding dataset, wherein each data sample contains user spoken language text, user preference environmental information, user state environmental information and other related environmental information, and intention recognition and slot filling label information corresponding to the user spoken language text. The source domain and target domain data files generated by the domain division module are also written back to the data storage module. The data storage module provides data storage services for the core system module of the patent, and can use data storage methods including but not limited to TSV format file, CSV format file, JSON format file, etc.

[0034] The domain division module is connected with the data storage module, and provides services of dividing the smart home user spoken language understanding data set according to the intent domain, determining the source domain and the target domain, and exporting the data file of each intent domain. Figure 2 is an example diagram of the smart home intent domain classification tree of the embodiment of the present application, containing four intent domains of curtain control, bathroom control, clothes airing control and air conditioning, wherein the bathroom control intent domain is taken as an example, and three control operation intents of drying mode, warm air mode and ventilation mode can be contained. The domain division module stores the data of each intent domain in the smart home user spoken language understanding data set and the list of unique data sample numbers corresponding to each intent in the intent domain into a file corresponding to the intent domain, and finally writes the corresponding files of all intent domains back to the data storage module, so as to facilitate data sampling in the intent domain according to the number list during subsequent task construction. At the same time, the domain division module divides all intent domains into the source domain and the target domain, such as Figure 2 , the curtain control intent domain, the bathroom control intent domain and the clothes airing control intent domain can be determined as the source domain, and the air conditioning intent domain is determined as the target domain, so as to facilitate the sampling and use of the task construction module.

[0035] The task construction module is connected with the domain division module, and provides an interface of constructing tasks by sampling data from the source domain and the target domain. Since the few-sample method is trained on the relatively rich source domain, and the target domain which is invisible and has only a few labeled samples is tested without fine-tuning, in order to ensure the consistency of the training and testing processes, the system effectively migrates the prior knowledge of user spoken language understanding based on a small number of samples learned from the source domain to the target domain, and the task construction module needs to construct few-sample training tasks and few-sample testing tasks by sampling data from the source domain and the target domain. The task construction module first reads the data files of each intent domain obtained by the domain division module in the data storage module. In the few-sample training stage, the task construction module determines a source intent domain by random uniform sampling from multiple source domains, and constructs a training task from the data file of the intent domain. In the few-sample testing stage, the task construction module constructs a testing task from the data file of the target intent domain. The module obtains the sample number list corresponding to each intent from the single intent domain data file, calculates the size of the support set and the query set corresponding to the intent domain through the number of samples corresponding to each intent, randomly and uniformly samples the sample numbers used by the support set and the query set from the sample number list corresponding to the intent, and obtains the data entries corresponding to the support set and the query set for the subsequent information extraction module.

[0036] Text and environment information extraction module: connected to the task construction module, provides the standardization organization service of text, environment information. This module extracts user spoken language text information and user preference environment information, user state environment information and other related environment information from the current task, obtains the knowledge graph environment information corresponding to the speech text entity from the open source knowledge graph library, and finally fills the text into the same length, splices the various user preference environment information or user state environment information represented by vectors, and provides standardized text and environment information vectors as the input of the information coding module. Figure 3 is an example of an intelligent home user spoken language understanding example of an embodiment of the present application, and the user spoken language text is "toilet bath heater stop blowing". Since the user speech corresponding to the data entries in the task is of different lengths, it is necessary to extract and fill it into the same length to become standardized user spoken language text information. The construction of related knowledge graph environment information needs to obtain the corresponding triple information of each entity from external open source knowledge graph library such as Xizhi for entities such as "bath heater" mentioned in the user spoken language text. The triple corresponding to the same entity is spliced, and the splicing result is an entity description sentence such as "entity: bath heater, type: device, description: bath heater is…". For the case of multiple entities in a sentence, since each entity description is of different length, it is also necessary to fill each entity description into the same length as the user spoken language text as the standardized knowledge graph environment information. The standardized user preference environment information can be a plurality of spliced user preference information vectors, each preference corresponding to a non-negative floating point array, and the sum of the array components is 1. For example, the user preference vector for "dry", "ventilation", "warm wind" can be "[0.5, 0.3, 0.2]", which means the user prefers to use the "dry" function rather than the "ventilation" and "warm wind" functions. The standardized user state environment information can include the user's motion state, geographic location, posture, etc., and the representation is similar to the user preference. For example, the motion state item corresponding to the vector "[0, 1, 0]" is "walking", "running", "still", indicating that the user's current motion state is the second component, i.e. running. Since the user preference and user state environment information are of the same length in the same domain, they do not need to be filled. It should be noted that the user preference environment information and the user state environment information can be continuously increased with the iteration and upgrading of the smart home products, and the specific performance in the system is that the target domain may appear user preference environment information and user state environment information that have not appeared in the source domain and the validation domain, so the encoding length of these two environment information in the target domain is likely to be greater than that in the source domain and the validation domain.

[0037] Intention and slot information extraction module: connected to the task construction module, provides intention and slot information data extraction organization service, and intention and slot dictionary table construction service. Extract the annotated intention and slot information from the data entries obtained from the task construction module, such as Figure 3, the user intent category is "stop drying", and the slot annotation sequence of the user utterance is in the B-I-O format annotated by humans. B represents the start position of the slot, I represents the middle position of the slot, and O represents a non-slot. In this example, "bath heater" is annotated as the slot "device", "stop" is annotated as the slot "operation", and "blow" is annotated as the slot mode. The user utterance intent category and slot annotation sequence are converted into vector form using one-hot encoding, thereby organizing the intent and slot information data as input to the result output and index evaluation module. At the same time, this module counts the intent and slot label categories contained in the task, constructs a correspondence dictionary table of intent labels, slot labels and one-hot encoding for subsequent use in system output evaluation.

[0038] The information coding module is connected with the text and environment information extraction module and provides coding services for user spoken language text and environment information. For the standardized user spoken language text information and the standardized knowledge graph environment information obtained by the text and environment information extraction module, pre-training language model coding of each entity description in the standardized user spoken language text information and the standardized knowledge graph environment information can be obtained by using ELECTRA, XLNet, BERT and other pre-training language models. Each entity description in the knowledge graph environment information is regarded as a node, and multiple entity descriptions of the same entity are connected by edges. At the same time, all entity description nodes are connected with the nodes corresponding to the standardized user spoken language text information, so as to complete the construction of the dialogue entity interaction graph. For the dialogue entity interaction graph, the information transmission and information aggregation between adjacent nodes are realized by using a graph convolution network, so as to automatically realize the interaction between entities and the interaction between entities and user dialogue, so that each word has different attention degrees for different entities, and the interaction and fusion between the knowledge graph environment information and the user dialogue are coded in a fine-grained manner. For each entity description, the top vector output of the graph convolution network on the node is used as the knowledge graph information coding. For the standardized user preference and user state environment information obtained by the text and environment information extraction module, since the vectorization has been completed, the user preference and user state environment information coding can be directly obtained. The two sub-tasks of intent recognition and slot filling are jointly trained in the embodiment of the application, that is, the cross-entropy loss functions corresponding to the intent recognition and slot filling are used as the joint training loss function with the sum of the hyperparameters as the proportion coefficient. Based on the pre-training language model coding of the user spoken language text information, the user spoken language text information coding based on the intent-dependent slot representation and the user spoken language text information coding based on the slot-dependent intent representation are obtained by using a label attention layer and a mutual attention layer, and the information interaction between the intent recognition and slot filling sub-tasks is modeled. The label attention layer uses the pre-training language model to code the description text of the intent label and the slot label respectively, and then uses the attention mechanism to code the user spoken language text and the pre-training language model coding of the description text of the intent label and the slot label, so as to obtain the user dialogue coding based on the intent representation and the user dialogue coding based on the slot representation. The mutual attention layer takes the user dialogue coding based on the intent representation and the user dialogue coding based on the slot representation as input, and uses the attention mechanism to further realize information interaction, so as to obtain the user spoken language text information coding based on the intent-dependent slot representation and the user spoken language text information coding based on the slot-dependent intent representation.

[0039] The information fusion module is connected with the information coding module, uses the knowledge graph information coding, the user preference information coding, the user state environment information coding, the user spoken language text information coding based on the intent dependent slot representation and the user spoken language text information coding based on the slot dependent intent representation obtained by the information coding module, provides the fusion service of the user spoken language text information and the environment information, so as to distinguish the difficult-to-judge intent in combination with the environment information. For example, Figure 3 From the speech “toilet bath heater stop blowing”, it is difficult to judge the intent of the speech as stopping drying, stopping ventilation or stopping warm air. But in combination with the user's use preference for the drying mode, it can be guessed that the user's speech intent is to stop drying. This module uses mutual attention to fuse the general environment information between domains, that is, for the environment information such as knowledge graph environment information and user state environment information, which does not expand with the expansion of the domain, the corresponding information coding is spliced and used with the attention mechanism based on the user spoken language text information coding based on the intent dependent slot representation and the user spoken language text information coding based on the slot dependent intent representation, to obtain the environment information fusion text coding based on the intent representation and the environment information fusion text coding based on the slot representation; for the environment information such as user preference environment information which expands with the expansion of the domain, the expanded environment information label is used to describe the text semantic information, such as the text semantic information of the user preference environment information label “drying”, “ventilation” and the like, the similarity matrix of the text semantic information of the intent and slot label is calculated, the domain expansion probability distribution on the intent class and the slot class is obtained, and the fusion of the environment information and the user spoken language text information between the domains is realized.

[0040] The few-sample training and testing module is connected to the information fusion module and provides user spoken language understanding model training and testing services. For training tasks sampled from the source domain, on the support set, for all data entries corresponding to each intent category, the mean of the environment information fusion text encoding based on the intent representation corresponding to the user spoken language text sentence is taken as the intent category prototype, and for all data entries corresponding to each slot category, the mean of the environment information fusion text encoding based on the slot representation corresponding to the user spoken language text word is taken as the slot category prototype. On the query set, the distance between each data entry and the intent and slot category prototypes is calculated to construct a category probability distribution, and the domain expansion probability distribution in the information fusion module is added with the probability distribution as a learnable parameter to obtain a sum, and the category corresponding to the maximum component in the probability distribution is taken as the category prediction result, and gradient backpropagation is performed to optimize the parameters in the above module. On the target domain, the target domain data file stored by the data storage module and constructed by the domain division module is used as input, and the environment information fusion text encoding based on the intent representation and the environment information fusion encoding based on the slot representation are obtained in the same way as in the source domain, and the optimized user spoken language understanding model parameters are used. For test tasks sampled from the target domain, on the support set, for all data entries corresponding to each intent category, the mean of the environment information fusion text encoding based on the intent representation corresponding to the user spoken language text sentence is taken as the intent category prototype, and for all samples corresponding to each slot category, the mean of the environment information fusion text encoding based on the slot representation corresponding to the user spoken language text word is taken as the slot category prototype. On the query set, the distance between each test data entry and the intent and slot category prototypes is calculated, and the probability distribution is obtained by the Softmax function, and the domain expansion probability distribution in the information fusion module is added with the proportion coefficient learned on the training domain to obtain a sum, and the category corresponding to the maximum component in the probability distribution is taken as the intent and slot category prediction result of the test data entry.

[0041] The result output and index evaluation module is connected to the few-sample training and testing module and the intent and slot information extraction module and provides output prediction result and index evaluation services. The module converts the vector-encoded intent category and slot category prediction results into visualized text output, and calculates the accuracy index Accuracy of intent recognition and the evaluation index F1-Score of slot filling. F1-Score is calculated from the accuracy Precision and recall Recall.

[0042] While specific embodiments of the application have been disclosed in order to illustrate the application and to assist those skilled in the art in practicing the application, it is to be understood that various substitutions, modifications and changes can be made by those skilled in the art without departing from the spirit of the application and the scope of the appended claims. Accordingly, it is intended that the application not be limited, except by the scope of the claims.

Claims

1. An intelligent home user spoken language understanding system oriented to environmental information augmentation, characterized in that, Comprising a data storage module for storing a smart home user spoken language understanding dataset, wherein each sample contains user spoken language text, user environment information, and intent classification and slot filling annotation information corresponding to the user spoken language text; and the processing result of the domain division module; a domain division module for dividing the intent domain into a plurality of source domains and a plurality of target domains, and then dividing the samples in the smart home user spoken language understanding dataset according to the belonging intent domain, saving the sample information of the intent domain belonging to the source domain i to the source domain data file corresponding to the source domain i, and saving the sample information of the intent domain belonging to the target domain j to the target domain data file corresponding to the target domain j; a task construction module for, in the few-shot training stage, sampling to determine an intent domain from the source domain, and then constructing a training task according to the source domain data file corresponding to the intent domain and determining the task data of the training task, the task data of the training task including a support set and a query set; and for, in the few-shot test stage, sampling to determine an intent domain from the target domain, and then constructing a test task according to the target domain data file corresponding to the intent domain and determining the task data of the test task, the task data of the test task including a support set and a query set; a text and environment information extraction module for extracting the text and environment information of the sample from the sample of the task data and filling and splicing the extracted information according to the input specification of the information coding module; an intent and slot information extraction module for extracting the intent and slot information from the task data and constructing an intent and slot dictionary table; an information coding module for vectorizing and coding the text and environment information of the sample; the information coding module uses a pre-trained language model to code the input information to obtain the pre-trained language model coding of each entity description in the standardized user spoken language text information and the standardized knowledge graph environment information; then, based on the pre-trained language model coding of the user spoken language text information, the label attention layer and the mutual attention layer are used to obtain the user spoken language text information coding based on the intent-dependent slot representation and the user spoken language text information coding based on the slot-dependent intent representation, to model the information interaction between the intent recognition and slot filling subtasks; then, each entity description in the knowledge graph environment information is taken as a node, the nodes corresponding to multiple entity descriptions of the same entity are connected by edges, and all nodes corresponding to the entity descriptions are connected to the node corresponding to the standardized user spoken language text information, to complete the construction of the spoken language entity interaction graph; then, the pre-trained language model coding of the entity description is taken as the node coding, the graph convolution network is used to perform information transmission and information aggregation between adjacent nodes on the spoken language entity interaction graph, to realize the interaction between entities and the interaction between entities and user utterances, so that each word has different attention degrees to different entities, and the interaction and fusion between the fine-grained coded knowledge graph environment information and the user utterance are realized; then, for each entity description, the top-level vector output of the graph convolution network on the node corresponding to the entity description is taken as the knowledge graph information coding. An information fusion module is configured to fuse the inter-domain common environment information, the inter-domain augmented environment information and the user spoken language text information. A few-sample training and testing module is configured to perform few-sample training of the user spoken language understanding model on a source domain and few-sample prediction of the user spoken language understanding model on a target domain. 2.The smart home user spoken language understanding system of claim 1, wherein, The text and environment information extraction module extracts text and environment information of a sample from task data of a current task, acquires knowledge graph environment information corresponding to a spoken language text entity from an open source knowledge graph library, fills the text to the same length, concatenates various user preference environment information or user state environment information represented by vectors, and provides normalized text and environment information vectors as inputs of the information encoding module. 3.The smart home user spoken language understanding system of claim 1, wherein, The information fusion module uses mutual attention to fuse the inter-domain common environment information, that is, concatenates information encoding of the inter-domain common environment information, and respectively uses attention mechanisms for user spoken language text information encoding based on intent dependency slot representation and user spoken language text information encoding based on slot dependency intent representation to obtain environment information fusion text encoding based on intent representation and environment information fusion encoding based on slot representation. 4.The smart home user spoken language understanding system according to claim 1 or 3, characterized in that, For the inter-domain augmented environment information, augmented environment information labels are used to describe text semantic information, a similarity matrix of the text semantic information and intent and slot label text semantic information is calculated, a domain augmentation probability distribution on the intent class and the slot class is obtained, and fusion of the inter-domain augmented environment information and the user spoken language text information is realized. The inter-domain common environment information includes knowledge graph environment information and user state environment information, and the inter-domain augmented environment information is user preference environment information. 5.The smart home user spoken language understanding system of claim 1, wherein, The few-sample training and testing module provides user spoken language understanding model training and testing services; for the training task, for all samples corresponding to each intent category on the support set of the training task, take the mean of the environment information fusion text encoding corresponding to the user spoken language text sentence based on the intent representation as the intent category prototype, for all samples corresponding to each slot category, take the mean of the environment information fusion text encoding corresponding to the user spoken language text word based on the slot representation as the slot category prototype, on the query set, calculate the distance between each sample and the intent category prototype and the slot category prototype to construct the category probability distribution, take the domain expansion probability distribution in the information fusion module as the proportion coefficient with the learnable parameters to get the sum, take the category corresponding to the maximum component in the probability distribution as the category prediction result, perform gradient backpropagation to optimize the parameters in the user spoken language understanding model; for the test task, use the optimized user spoken language understanding model to take the mean of the environment information fusion text encoding corresponding to the user spoken language text sentence based on the intent representation as the intent category prototype for all samples corresponding to each intent category on the support set of the test task, take the mean of the environment information fusion text encoding corresponding to the user spoken language text word based on the slot representation as the slot category prototype for all samples corresponding to each slot category; then according to the distance between each test sample and the intent and slot category prototype on the query set of the test task, obtain the intent and slot category prediction result of the corresponding test sample. 6.The smart home user spoken language understanding system of claim 1, wherein, The user environment information includes user preference environment information and user state environment information. 7.The smart home user spoken language understanding system of claim 1, wherein, The sample information is a unique number of the sample. 8.The smart home user spoken language understanding system of claim 1, wherein, Further comprising a result output and index evaluation module for encoding the few-sample test output to obtain a readable result and calculating an evaluation index value.

Images