Understanding query intent for medical artificial intelligence systems using semi-supervised deep learning

Abstract

A method and apparatus for receiving a query including a sequence of multiple words is provided. The sequence of multiple words is converted into a sequence of multiple feature vectors. A semantic meaning of the sequence of multiple words is generated using a bidirectional long short-term memory (LSTM). The LSTM is regularized based on the generated semantic meaning of the sequence of multiple feature vectors using a Gaussian mixture model (GMM).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 16 / 560,440, filed in the U.S. Patent and Trademark Office on September 4, 2019, the disclosure of which is incorporated herein by reference in its entirety. Background Art

[0003] Unlike other fields, and due to patient privacy and protection, medical data is more restricted and more difficult to obtain. In a medical chatbot system, it is very important to understand the customer's question or intention. It is impractical to directly train a deep learning model for accurate query classification with a limited number of labeled data samples. The present disclosure utilizes deep learning and semi-supervised learning to improve the generalization of supervised tasks in the context of limited labeled data.

[0004] There are many different semi-supervised learning methods. A common solution is to use a generative approach to model the joint distribution between data examples and labels. Due to the diversity of Chinese expressions, a deep semantic understanding of Chinese sentences is required. Deep learning models are good at acquiring semantically meaningful features. Therefore, the present disclosure uses a deep learning model and a Gaussian mixture model (GMM) as a regularizer to achieve medical query classification under limited labels. At the same time, considering the large amount of unlabeled data, the present disclosure helps to improve the generalization of supervised tasks. Summary of the invention

[0005] According to some possible implementations, a method includes: receiving, by a device, a query comprising a sequence of multiple words; converting, by a device, the sequence of multiple words into a sequence of multiple feature vectors; generating, by a device and using a bidirectional long short-term memory (LSTM), a semantic meaning of the sequence of multiple feature vectors; and regularizing, by a device and using a Gaussian mixture model (GMM), the LSTM based on the semantic meaning of the generated sequence of multiple feature vectors.

[0006] According to some possible implementations, a device includes: at least one memory configured to store program code; at least one processor configured to read the program code and operate according to the instructions of the program code, the program code including: a receiving code configured to cause at least one processor to receive a query including a sequence of multiple words; a conversion code configured to cause at least one processor to convert the sequence of multiple words into a sequence of multiple feature vectors; a generating code configured to cause at least one processor to use a bidirectional long short-term memory (LSTM) to generate the semantic meaning of the sequence of multiple feature vectors; and a regularization code configured to cause at least one processor to use a Gaussian mixture model (GMM) to regularize the LSTM based on the semantic meaning of the generated sequence of multiple feature vectors.

[0007] According to some possible implementations, a non-transitory computer-readable medium stores instructions, which include one or more instructions, which, when executed by one or more processors of a device, cause the one or more processors to: receive a query including a sequence of multiple words; convert the sequence of multiple words into a sequence of multiple feature vectors; use a bidirectional long short-term memory (LSTM) to generate semantic meanings of the sequence of multiple feature vectors; and use a Gaussian mixture model (GMM) to regularize the LSTM based on the semantic meanings of the generated sequence of multiple feature vectors. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A and Figure 1B is a schematic diagram of an example implementation described herein;

[0009] Figure 2 is a diagram of an example environment in which the systems and / or methods described herein may be implemented;

[0010] Figure 3 yes Figure 2 diagrams of example components of one or more devices; and

[0011] Figure 4 is a flowchart of an example process for training a bidirectional long short-term memory (LSTM) to recognize queries using a Gaussian mixture model (GMM). DETAILED DESCRIPTION

[0012] In medical chatbot systems, understanding query intent is a major step in continuous chat. Training an accurate query classifier requires domain knowledge and a time-consuming labeling process, especially for question-answering (QA) systems based on knowledge graphs. In addition, understanding the relationship between different parts of the query statement is important for obtaining intent. For example, the query "What are the symptoms of lung cancer?" asks about the characteristics of the disease, which associates symptoms with the disease through characteristic relationships. The query "What disease causes fever?" asks about the disease that causes symptoms, which associates symptoms with the disease through causal relationships. Both of the above queries involve symptoms and diseases, but the query intent is completely different. Determining the query intent is conducive to obtaining accurate answers from the knowledge graph. Therefore, it is important to obtain a deep representation of the query and accurately classify the query in a knowledge graph-based QA chatbot.

[0013] Methods for classifying queries related to medicine fall into two categories: In the rule-based category, the classifier only considers the presence of a specific set of keywords (or similar words) in the query. For example, "What are the symptoms of fever?" is classified as a disease-symptom query containing disease named entities, and symptom words are considered keywords. Sentence-level classifiers (e.g., RNN or LSTM) are used to model sentence features and convert sequences of multiple words into class labels.

[0014] For the above two types of methods, the disadvantage is that rule-based systems usually cannot handle keywords outside the vocabulary. Second, rule-based methods usually rely on only a set of keywords, which cannot be generalized and does not consider the semantic meaning of the entire sentence. Therefore, it often leads to poor performance. For sentence-level classifiers, using RNN / LSTM models can improve performance, but usually requires a large amount of training data. Even if plain text query data can be obtained through crowdsourcing, accurately labeling such data is expensive and time-consuming.

[0015] The present disclosure combines deep learning methods with semi-supervised learning methods that deal with the difficulties associated with the lack of labeled data. Therefore, some implementations of this document improve the performance of classifiers.

[0016] The present disclosure can be used to classify different types of queries in a QA chatbot system based on a knowledge graph. The present disclosure can also be used as a preprocessing step for separating different types of phrases or sentences for event extraction before building a knowledge graph.

[0017] The present disclosure provides a system that provides the above functionality by training with a limited labeled data set and a large unlabeled data set available from multiple resources. In order to obtain a deep representation of a query, the present disclosure provides a deep learning model using a bidirectional long short-term memory (LSTM) and a follow-up attention mechanism. In order to generalize a supervised model using limited labeled data and large unlabeled data, the present disclosure includes a Gaussian mixture model (GMM) for modeling data distribution.

[0018] Reference Figure 1A , the present disclosure minimizes -[log(p(y|x))+log(p(x))], the classifier p(y|x) is modeled by a bidirectional LSTM (Bi-LSTM), followed by an attention mechanism and a softmax layer. The data p(x) is modeled by a Gaussian mixture model.

[0019] In the preprocessing step, as shown in Figure 1, the query sentence is regarded as a sequence of words, and each word is converted into a vector by concatenating the word embedding and the position embedding.

[0020] Further Figure 1AAs shown, the word embedding and position embedding layers convert a word sequence into a sequence of multiple feature vectors, where each feature vector is a concatenation of a word feature and a position feature. The word feature is parameterized by a weight matrix, where the rows of the weight matrix are indexed by the word position in the dictionary. The position feature is the relative position of the word in the sequence.

[0021] Further Figure 1A As shown, the Bi-LSTM layer transforms the input feature sequence into an output feature sequence, where the output feature sequence encodes the semantic meaning of the sentence by obtaining the dependency relationship between the current word and its previous word or the following word.

[0022] Further Figure 1A As shown in Figure 1, the attention mechanism layer allows the system to focus on specific parts of the source sentence. Each attention node gets a different part of the sentence, and each part of the sentence is treated as a target named entity (node ​​in the knowledge graph).

[0023] Further Figure 1A As shown, the feature vectors of the sentences are aggregated by GMM. The labeled parts of the features are classified based on minimizing the softmax cross entropy loss between the predicted probabilities and the true labels.

[0024] Further Figure 1B As shown, in the training step, the query is first converted into sentence features using a deep learning model (BERT), and then these features are treated as input to a transductive support vector machine (SVM) model to train a classifier. In the testing step, the query is converted into features and the learned classifier is used to obtain its predicted label.

[0025] Model training is performed based on an alternative minimization mechanism, where the deep network parameters are first updated to produce sentence embeddings, and then the embedded sentence features are treated as inputs to the GMM to minimize the negative log-likelihood of the GMM. This process can be repeated again.

[0026] The present invention utilizes a large amount of web-based data and can improve the query type classification and summarization capabilities, thereby improving performance. In addition, the present disclosure reduces the burden of labeling and prepares large training text queries for the operation of deep learning models.

[0027] Figure 2 2 is a diagram of an example environment 200 in which the systems and / or methods described herein may be implemented. Figure 2 As shown, environment 200 may include user device 210, platform 220, and network 230. The devices of environment 200 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0028] The user device 210 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with the platform 220. For example, the user device 210 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smart phone, a wireless phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), or the like. In some implementations, the user device 210 may receive information from the platform 220 and / or send information to the platform 220.

[0029] The platform 220 includes one or more devices capable of using a Gaussian mixture model (GMM) to train a bidirectional long short-term memory (LSTM) to recognize queries, as described elsewhere herein. In some implementations, the platform 220 may include a cloud server or a group of cloud servers. In some implementations, the platform 220 may be designed as a modular platform so that certain software components can be swapped in or out according to specific needs. Therefore, the platform 220 can be easily and / or quickly reconfigured for different purposes.

[0030] In some implementations, as shown, the platform 220 can be hosted in a cloud computing environment 222. It should be noted that although the implementations described herein describe the platform 220 as being hosted in a cloud computing environment 222, in some implementations, the platform 220 is not cloud-based (i.e., can be implemented outside of a cloud computing environment) or can be partially cloud-based.

[0031] Cloud computing environment 222 includes an environment hosting platform 220. Cloud computing environment 222 can provide computing, software, data access, storage, and other services that do not require end users (e.g., user devices 210) to know the physical location and configuration of the systems and / or devices hosting platform 220. As shown, cloud computing environment 222 may include a set of computing resources 224 (this set of computing resources is collectively referred to as "computing resources 224", and a single computing resource is referred to as "computing resource 224").

[0032] The computing resources 224 include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some implementations, the computing resources 224 may control the platform 220. Cloud resources may include computing instances running in the computing resources 224, storage devices provided in the computing resources 224, data transmission devices provided by the computing resources 224, etc. In some implementations, the computing resources 224 may communicate with other computing resources 224 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0033] Further Figure 2As shown, the computing resources 224 include a set of cloud resources, such as one or more applications ("APP") 224-1, one or more virtual machines ("VM") 224-2, virtualized storage ("VS") 224-3, one or more hypervisors ("HYP") 224-4, etc.

[0034] The applications 224-1 include one or more software applications that can be provided to or accessed by the user device 210 and / or the sensor device 220. The applications 224-1 can eliminate the need to install and run software applications on the user device 210. For example, the applications 224-1 can include software associated with the platform 220 and / or any other software that can be provided through the cloud computing environment 222. In some implementations, one application 224-1 can send information to / receive information from one or more other applications 224-1 through the virtual machine 224-2.

[0035] Virtual machine 224-2 includes a software implementation of a machine (e.g., a computer) that runs a program, similar to a physical machine. Depending on the degree and purpose of the virtual machine 224-2 to any actual machine, virtual machine 224-2 can be a system virtual machine or a process virtual machine. A system virtual machine can provide a complete system platform that supports the operation of a complete operating system ("OS"). A process virtual machine can run a single program and can support a single process. In some implementations, virtual machine 224-2 can run on behalf of a user (e.g., user device 210) and can manage the infrastructure of cloud computing environment 222, such as data management, synchronization, or long-term data transfer.

[0036] Virtualized memory 224-3 includes one or more storage systems and / or one or more devices using virtualization technology within the storage system or device of computing resource 224. In some implementations, in the environment of the storage system, the type of virtualization may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage, so that the storage system can be accessed without considering physical storage or heterogeneous structures. Separation allows administrators of storage systems to have flexibility in how administrators manage storage for end users. File virtualization can eliminate the dependency between data accessed at the file level and the location where the file is physically stored. This can optimize storage usage, server consolidation and / or non-interference file migration performance.

[0037] Hypervisor 224-4 may provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer such as computing resource 224. Hypervisor 224-4 may present a virtual operating platform to the guest operating systems and may manage the operation of the guest operating systems. Multiple instances of each operating system may share virtualized hardware resources.

[0038] The network 230 includes one or more wired networks and / or wireless networks. For example, the network 230 may include a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber-based network, etc., and / or a combination of these or other types of networks.

[0039] Figure 2 The number and arrangement of devices and networks shown are provided as examples. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different Figure 2 The devices and / or networks shown are arranged differently. Figure 2 Two or more of the devices shown may be implemented in a single device, or Figure 2 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one set of devices (eg, one or more devices) of environment 200 may perform one or more functions described as being performed by another set of devices of environment 200.

[0040] Figure 3 300 is a diagram of example components of device 300. Device 300 may correspond to user device 210 and / or platform 220. Figure 3 As shown, device 300 may include a bus 310 , a processor 320 , a memory 330 , a storage component 340 , an input component 350 , an output component 360 , and a communication interface 370 .

[0041] The bus 310 includes components that allow communication between components of the device 300. The processor 320 is implemented in hardware, firmware, or a combination of hardware and software. The processor 320 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some implementations, the processor 320 includes one or more processors that can be programmed to perform functions. The memory 330 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 320.

[0042] Storage component 340 stores information and / or software related to the operation and use of device 300. For example, storage component 340 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of non-transitory computer-readable medium, and a corresponding drive.

[0043] Input components 350 include components that allow device 300 to receive information, such as input through user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone). In addition or alternatively, input components 350 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). Output components 360 include components that provide output information from device 300 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs)).

[0044] The communication interface 370 includes transceiver-like components (e.g., a transceiver and / or a separate receiver and transmitter) that enable the device 300 to communicate with other devices, such as through a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. The communication interface 370 may allow the device 300 to receive information from another device and / or provide information to another device. For example, the communication interface 370 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0045] Device 300 may perform one or more processes described herein. Device 300 may perform these processes in response to processor 320 executing software instructions stored by a non-transitory computer-readable medium such as memory 330 and / or storage component 340. In this document, a computer-readable medium is defined as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0046] The software instructions may be read into the memory 330 and / or storage component 340 from another computer-readable medium or from another device through the communication interface 370. When executed, the software instructions stored in the memory 330 and / or storage component 340 may cause the processor 320 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0047] Figure 3 The number and arrangement of components shown are provided as examples. In practice, device 300 may include additional components, fewer components, different components, or different components. Figure 3 Additionally or alternatively, one set of components (eg, one or more components) of device 300 may perform one or more functions described as being performed by another set of components of device 300.

[0048] Figure 4 is a flow chart of an example process 400 for training a bidirectional long short-term memory (LSTM) to recognize queries using a Gaussian mixture model (GMM). In some implementations, the process may be performed by the platform 220 Figure 4 In some implementations, the process may be performed by another device or group of devices (e.g., user device 210) that is separate from or includes platform 220. Figure 4 One or more process blocks.

[0049] like Figure 4 As shown, process 400 may include receiving a query including a sequence of multiple words (block 410). For example, platform 220 may receive a query including a sequence of multiple words to identify a semantic meaning of the query.

[0050] In some implementations, the query may correspond to a medical-based query. For example, the query may include "What are the symptoms of a fever?"

[0051] In some implementations, the platform 220 may receive an input data set including tagged queries. That is, the platform 220 may receive an input data set including queries mapped to semantic meanings. In other words, the tagged queries may be associated with predetermined meanings. In this way, the platform 220 may use the tagged queries to train the LSTM, as described elsewhere herein.

[0052] In some implementations, the platform 220 may receive an input data set including unlabeled queries. That is, the platform 200 may receive an input data set including queries that are not mapped to semantic meanings. The platform 220 may use the unlabeled queries for further model refinement and training.

[0053] In some implementations, the platform 220 can receive an input data set from another device based on performing web mining techniques, based on performing information retrieval techniques, and / or the like.

[0054] Further Figure 4 As shown, process 400 may include converting, by a device, a sequence of a plurality of words into a sequence of a plurality of feature vectors (block 420).

[0055] In some implementations, the platform 220 may convert a sequence of multiple words into a sequence of multiple feature vectors based on performing a word embedding technique and a position embedding technique.

[0056] For example, the platform 220 may perform a word embedding technique that generates a word feature based on the position of the word in the dictionary. In addition, the platform 220 may perform a position embedding technique that generates a position feature based on the position of the word in the query. In addition, the platform 220 may generate a feature vector based on the word feature and the position feature.

[0057] Further Figure 4 As shown, process 400 may include generating, by a device, a semantic meaning of a sequence of multiple feature vectors using a bidirectional long short-term memory (LSTM) (block 430). For example, platform 220 may generate a semantic meaning of a sequence of multiple feature vectors using LSTM.

[0058] In some implementations, the semantic meaning can correspond to the meaning, intent, etc. of the query. The semantic meaning can be specified by a value, a label, a name, etc.

[0059] In some implementations, the platform 220 may input a sequence of multiple feature vectors into an LSTM. The LSTM may receive a sequence of multiple feature vectors and may generate semantic meaning as output.

[0060] In some implementations, the LSTM may include a parameter set that allows the LSTM to generate semantic meaning based on a sequence of multiple feature vectors input. The platform 220 may enable the parameter set to be adjusted as described below.

[0061] Further Figure 4 As shown, process 400 may include regularizing, by the device and using a Gaussian mixture model (GMM), the LSTM based on the generated semantic meaning of the sequence of the plurality of feature vectors (block 440).

[0062] For example, platform 220 may regularize the LSTM by adjusting a set of parameters of the LSTM.

[0063] In some implementations, the platform 220 can regularize the LSTM by using an attention mechanism that focuses on specific parts of the source sentence. Each attention node obtains a different part of the sentence, and each part of the sentence is regarded as a target named entity (node ​​in the knowledge graph).

[0064] Additionally or alternatively, the platform 220 can regularize the LSTM by using loss minimization for regularization techniques. For example, the feature vectors of all sentences are aggregated by a GMM. The labeled portion of the feature is classified based on minimizing the softmax cross entropy loss between the predicted probability and the true label.

[0065] The platform 220 may regularize the LSTM based on an alternative minimization mechanism, where the deep network parameters of the LSTM are first updated to produce sentence embeddings, and then all embedded sentence features are considered as inputs to the GMM to minimize the negative log-likelihood value of the GMM.

[0066] Further Figure 4 As shown, process 400 may include determining whether a training condition is met (block 450). For example, the training condition may correspond to a threshold number of iterations to regularize the LSTM, a threshold number of iterations to update parameters of the LSTM, a threshold amount of time, reaching a threshold model accuracy, etc.

[0067] If the training condition is not met, process 400 may include performing step 400. If the training condition is met, process 400 may end.

[0068] Although Figure 4 Example blocks of process 400 are shown, but in some implementations, process 400 may include additional blocks, fewer blocks, different blocks, or different blocks. Figure 4 Depicted blocks are blocks that are arranged differently. Additionally or alternatively, two or more blocks of process 400 may be performed in parallel.

[0069] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementations to the precise forms disclosed. Modifications and variations may be made in light of the foregoing disclosure or may be acquired from practice of the implementations.

[0070] As used herein, the term "component" is intended to be broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0071] Obviously, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code - it should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0072] Even though particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may only be directly dependent on one claim, the disclosure of possible implementations includes the combination of each dependent claim with every other claim in the claim set.

[0073] Unless explicitly described per se, the elements, actions or instructions used herein should not be understood as being essential or necessary. In addition, as used herein, the articles "one" and "an" are intended to include one or more items, and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related items and unrelated items, etc.), and can be used interchangeably with "one or more". In the case where the intention is only one item, the term "one" or similar language is used. In addition, as used herein, the term "having", "having", "containing" or similar terms are intended to be open terms. In addition, the phrase "based on" means "based on at least in part", unless otherwise explicitly stated.

Claims

1. A method for identifying a query, comprising: receiving, by a device, a query comprising a sequence of a plurality of words; converting the sequence of the plurality of words into a sequence of a plurality of feature vectors by the device; Generating semantic meaning of the sequence of the plurality of feature vectors by the device and using a bidirectional long short-term memory (LSTM), followed by an attention mechanism and a softmax layer; as well as The LSTM is regularized based on the semantic meaning of the sequence of the generated multiple feature vectors by the device and using a Gaussian mixture model GMM, wherein the regularized LSTM includes: the GMM classifies the labeled part of the feature vector based on minimizing the softmax cross entropy loss between the predicted probability and the true label, and minimizing the negative log-likelihood of the GMM.

2. The method according to claim 1, wherein: The conversion includes at least one of a cascade technique, a word embedding technique, and a position embedding technique.

3. The method according to claim 1 or 2, wherein: Each feature vector in the sequence of the plurality of feature vectors is a concatenation of a word feature and a position feature.

4. The method according to claim 3, wherein: The word feature corresponds to a position of the word in a dictionary, and wherein the position feature corresponds to a position of the word in a sequence of the plurality of words.

5. The method according to claim 1 or 2, wherein: The generating includes generating the semantic meaning based on a relationship between a plurality of words in a sequence of the plurality of words.

6. The method according to claim 1 or 2, further comprising: The GMM is trained by using an attention mechanism layer.

7. A device for identifying a query, comprising: at least one memory configured to store program code; At least one processor is configured to read the program code and operate according to instructions of the program code to execute the method according to any one of claims 1 to 6.

8. A computer-readable medium storing instructions, the instructions comprising one or more instructions which, when executed by one or more processors of a device, cause the one or more processors to perform the method according to any one of claims 1 to 6.

9. An apparatus for identifying a query, comprising: A receiving module, configured to receive a query including a sequence of multiple words; A conversion module, used for converting the sequence of the plurality of words into a sequence of a plurality of feature vectors; A generation module, for generating semantic meaning of the sequence of the plurality of feature vectors using a bidirectional long short-term memory (LSTM), a following attention mechanism and a softmax layer; as well as A regularization module is used to regularize the LSTM based on the semantic meaning of the sequence of the multiple feature vectors generated using a Gaussian mixture model GMM, wherein the regularized LSTM includes: the GMM classifies the labeled part of the feature vector based on minimizing the softmax cross entropy loss between the predicted probability and the true label, and minimizing the negative log-likelihood of the GMM.

Images