Intelligent Assistant Method

Abstract

A method for an intelligent assistant includes identifying one or more insertion points within an input containing text for providing additional information, and generating a subsequent utterance including at least a portion of the input and the additional information at the one or more insertion points to clarify or supplement the meaning of the input.

Description

Technical Field

[0001] One or more embodiments relate generally to virtual assistants, and specifically to generating subsequent utterances for virtual assistants. Background Art

[0002] Customers use voice-based personal assistants such as and To answer questions, solve problems, and perform tasks to save time, energy, and make their lives more convenient. User interaction with these personal assistants usually requires some back-and-forth communication, as most users do not specify every relevant detail in a single statement. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] For a more complete understanding of the nature and advantages of the embodiments and the preferred mode of use, reference should be made to the following detailed description read in conjunction with the accompanying drawings, wherein:

[0004] Figure 1 shows a schematic diagram of a communication system according to some embodiments;

[0005] Figure 2 A block diagram illustrating an architecture of a system including an electronic device including a smart assistant application according to some embodiments;

[0006] Figure 3 shows an example of identifying an insertion point and amount according to some embodiments;

[0007] Figure 4 An example of generating words for a specification according to some embodiments is shown;

[0008] Figure 5 shows an example of asking follow-up questions using details according to some embodiments;

[0009] Figure 6 A block diagram illustrating a process for generating functionally similar but semantically different paraphrases with additional detail according to some embodiments;

[0010] Figure 7 A block diagram illustrating a process for identifying an insertion point and determining an insertion amount according to some embodiments;

[0011] Figure 8 A block diagram illustrating a process for generating a specified number of words at a specific position in an expression according to some embodiments;

[0012] Fig. 9 A block diagram illustrating a process for generating explicit follow-up questions according to some embodiments; and

[0013] Fig.10 is a high-level block diagram illustrating an information processing system including a computing system that implements one or more embodiments. DETAILED DESCRIPTION

[0014] One or more embodiments generally relate to identifying insertion points of an expression to provide additional information for generating a subsequent expression for an intelligent assistant. In one embodiment, a method for an intelligent assistant includes identifying one or more insertion points within an input containing text for providing additional information. A subsequent expression is generated that includes at least a portion of the input and the additional information at the one or more insertion points to clarify or supplement the meaning of the input.

[0015] In another embodiment, an electronic device includes a memory storing instructions. At least one processor executes the instructions including processing configured to identify one or more insertion points within an input containing text for providing additional information, and generate a subsequent expression including at least a portion of the input and the additional information at the one or more insertion points for clarifying or supplementing the meaning of the input.

[0016] In one embodiment, a non-transitory processor-readable medium includes a program that, when executed by a processor, performs a method comprising identifying one or more insertion points within an input containing text for providing additional information, and generating a subsequent expression including at least a portion of the input and the additional information at the one or more insertion points for clarifying or supplementing the meaning of the input.

[0017] These and other aspects and advantages of one or more embodiments will become apparent from the following detailed description, which, when taken in conjunction with the accompanying drawings, illustrates, by way of example, the principles of the one or more embodiments.

[0018] The following description is intended to illustrate the general principles of one or more embodiments and is not meant to limit the inventive concepts claimed herein. In addition, the specific features described herein may be used in combination with other described features in each of the various possible combinations and arrangements. Unless otherwise expressly defined herein, all terms should be given the broadest interpretation, including the meaning implied from the specification and the meaning understood by those skilled in the art and / or as defined in dictionaries, monographs, etc.

[0019] It should be noted that the term "at least one" refers to one or more of the following elements. For example, "at least one of a, b, c or a combination thereof" can be interpreted as "a", "b" or "c" alone; or "a" and "b" in combination, or "b" and "c" in combination; "a" and "c" in combination; or "a", "b" and "c" in combination.

[0020] One or more embodiments provide for identifying insertion points of an expression to provide additional information for generating a subsequent expression for an intelligent assistant. Some embodiments include a method for an intelligent assistant, the method comprising identifying one or more insertion points within an input containing text for providing additional information. A subsequent expression is generated that includes at least a portion of the input and the additional information at the one or more insertion points to clarify or supplement the meaning of the input.

[0021] Traditional personal assistants require advanced natural language understanding and dialogue management technologies to interact with users, understand and fulfill their intents. These traditional personal assistants do not generalize well to a large number of intents or a wide range of interest areas. The traditional approach to building a personal assistant is to build a core engine that extends the input language with metadata for downstream processing by a set of intent-specific service providers. This is called a skill application programming interface (API)-based approach. In the skill API-based approach, the core natural language understanding (NLU) engine extends the input language with metadata for downstream processing by a set of intent-specific service providers. Each of these service providers must implement downstream processing to handle the details of the intent and decide what questions to ask or what actions to perform. Intent service providers tend to adopt a "slot filling" approach. In the slot filling approach, each intent service provider creates a set of predefined attributes ("slots") that the user should provide for each intent. The service provider writes a question template for each attribute, designed to ask the user about the attribute, in which the value of an existing attribute can be referenced. A dialogue manager with a finite state machine or list processing system or other similar technology is used to select and ask one of these questions until values ​​for all attributes are provided. For example, a pizza delivery service would likely define the attributes of the location to be delivered to, the size of the pizza, and the toppings on the pizza.

[0022] To help understand the traditional approach, let’s assume the example input is “buy sausage pizza.” The intent classifier will select the pizza delivery intent and label sausage with a pizza topping tag. The dialog manager will then record that value for the pizza topping slot, check its list of unfilled slots, and select the next slot to fill. It might then select the “size” slot, and if it does, it will then decide to ask the associated question for that slot, “What size pizza would you like?” The process will continue for the remaining slots until the order is ready to be placed.

[0023] There are three problems with this traditional approach to dialogue management. First, it can only ask predefined follow-up questions for predefined attributes. If there is no template, there is no script to follow, and therefore no follow-up questions. If the template is made without a slot, then there will not be any follow-up questions about the slot. This traditional approach to dialogue management does not work for general cases or any unexpected cases. For example, "deliver sausage pizza and coke". The slot filling approach will not: discover the slot for the side dish and put the coke in it, or ask if you want the order with a breadstick side dish. The slot filling approach may cause confusion and: send the sentence to the wrong intent service provider, or put the coke on the pizza. The slot filling approach may just not ask for your pizza to have coke. All because the computer programmer who wrote the pizza delivery service forgot, or simply did not know about this situation. This problem will be more obvious for services that ordinary programmers do not use or are not familiar with.

[0024] Second, traditional dialogue management approaches lack the ability to rank slot values ​​and suggest the most appropriate choices in follow-up questions. Third, it takes a lot of effort to define all the properties for each intent / service and write out all the input examples needed to train the system to recognize user intents and label the slot values ​​with slot types. This is a serious problem because adding a new intent in the same domain, or adding a new slot type in an intent, or even changing just a few examples in an intent, is not a safe operation because it can change the way the system recognizes which intent to choose and which slot type to label. Everything must be tested again before operation. This is a major failure because there may be an infinite number of intents and each of them can be processed at a different level of detail. It is usually easier to recognize intents when the number of examples is large and these examples cover every relevant slot in every valid permutation. The amount of required examples increases with the number of skills, slot types, and slot values ​​in the system, as well as the structural language similarity and shared vocabulary of the intents.

[0025] With traditional personal assistants, there are usually multiple providers for any valuable service, and only one can be chosen. When letting third-party service providers fulfill intents themselves, this reduces the development cost of the personal assistant, but competing service providers with more example utterances with more examples of relevant attributes tend to beat out competitors who are not willing to take such steps. When there are multiple competing personal assistants, the same factors are still in play, but at a higher level.

[0026] In some embodiments, the system can not only ask follow-up questions about unspecified slots, but can also discover slots that were not originally used for that purpose in the example, discover slot values ​​for new slots, and even process those slot values ​​into the original input, or use the slot values ​​it discovered as possible suggested choices to ask questions about the slot. In one example embodiment, the system can respond to "Deliver sausage pizza and coke" with "Would you like me to deliver sausage pizza with breadsticks and coke?" because it determines that the additional information (e.g., breadsticks) is generally between pizza and coke.

[0027] One or more embodiments provide a personal assistant with a more powerful core engine and more useful skills, thereby providing more value to customers. According to some embodiments, the personal assistant can select the right skills / intents to provide a greater amount of skills. Some embodiments reduce the cumbersome burden placed on skill developers and increase their productivity and attract more developers, resulting in a more powerful personal assistant. In one or more embodiments, the core engine of the personal assistant is more capable, flexible, and versatile than traditional personal assistants. Some embodiments reduce the most cumbersome burdens in skill development and improve the ability of the core NLU engine to select the right skills.

[0028] Some embodiments identify locations in a natural language expression where additional relevant details can be inserted without changing the intent of the expression and with only minimal changes to the meaning of the expression. Alternative versions of natural language expressions can be generated using words or phrases that increase the meaning of the original expression. This includes adding relevant details, specifying (unstated) options, and adding constraints. Both the original expression and the alternative expression evoke the same type of intent, but with different meanings.

[0029] One or more embodiments generate follow-up questions that are intended to clarify the meaning of an expression by asking additional information such as related details, specifying unexpressed options, or constraints. Alternative versions of natural language expressions are generated, particularly of the type described above, for training systems that mark slots in expressions or determine the intent of an expression. Some embodiments generate additional details and insert them into sentences without disrupting the readability, flow, and general intent of the expression. Conventional systems may generate free-flowing text from an insertion point that is often tangential to the rest of the expression without generating text that links back to the rest of the expression; they generate the next most likely text. This raises several issues. The actual next word in the expression may be generated for insertion, resulting in a disfluent repetition that does not add information. Furthermore, if the next part of the expression is not related to the most likely topic, the text will not link back to the rest of the expression sensibly.

[0030] Some embodiments identify insertion points and determine the amount to insert, and generate a specified number of words at a specific location in an expression. Both of these techniques are features with unique capabilities that support other parts of the natural language interface to the personal assistant. One or more embodiments generate explicit follow-up questions for the natural language interface to the personal assistant that rely on the system to identify insertion points, and determine the amount to insert, because it is necessary to identify where additional details are typically located (missing) and determine how much additional details to add. Some embodiments can be used to identify when a follow-up question would be appropriate, and generate a specified number of words at a specific location in an expression, which can then provide new details for the follow-up question.

[0031] Figure 1 1 is a schematic diagram of a communication system 10 according to one embodiment. The communication system 10 may include a communication device (sending device 12) that initiates an outgoing communication operation and a communication network 110 that the sending device 12 may use to initiate and conduct communication operations with other communication devices within the communication network 110. For example, the communication system 10 may include a communication device (receiving device 11) that receives communication operations from the sending device 12. Although the communication system 10 may include multiple sending devices 12 and receiving devices 11, in Figure 1 Only one of each is shown in order to simplify the drawing.

[0032] Any suitable circuits, devices, systems, or combinations of these that are operable to create a communication network (e.g., wireless communication infrastructure including communication towers and telecommunication servers) may be used to create the communication network 110. The communication network 110 is capable of providing communications using any suitable communication protocol. In some embodiments, the communication network 110 may support, for example, traditional telephone lines, cable television, Wi-Fi (e.g., IEEE 802.11 protocol), High frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHz) communication systems, infrared, other relatively localized wireless communication protocols, or any combination thereof. In some embodiments, the communication network 110 can support wireless and cellular phones and personal email devices (e.g., ) protocol used. Such protocols may include, for example, GSM, GSM plus EDGE, CDMA, quad-band, and other cellular protocols. In another example, the remote communication protocol may include Wi-Fi and a protocol for making or receiving calls using VOIP, LAN, WAN, or other TCP-IP based communication protocols. When located within the communication network 110, the sending device 12 and the receiving device 11 may communicate on a bidirectional communication path such as path 13 or on two unidirectional communication paths. Both the sending device 12 and the receiving device 11 are capable of initiating communication operations and receiving initiated communication operations.

[0033] The sending device 12 and the receiving device 11 may include any suitable device for sending and receiving communication operations. For example, the sending device 12 and the receiving device 11 may include, but are not limited to, mobile phone devices, television systems, cameras, camcorders, devices with audio and video capabilities, tablet computers, wearable devices, smart devices, smart photo frames, and any other device capable of wireless communication (with or without the aid of wireless-enabled accessory systems) or via a wired path (e.g., using traditional telephone lines). The communication operations may include any appropriate form of communication, including, for example, data and control information, voice communication (e.g., telephone calls), data communication (e.g., emails, text messages, media messages), video communication, or a combination of these (e.g., video conferencing).

[0034] Figure 2 A functional block diagram of an architecture system 100 is shown, which can be used for a personal assistant to ask relevant follow-up questions and significantly reduce the number of The development time and cost of a personal assistant of the present invention is reduced, while using an electronic device 120 (e.g., a mobile phone device, a television (TV) system, a camera, a camcorder, a device with audio and video functions, a tablet computer, a tablet device, a wearable device, a smart device, a smart photo frame, a smart lighting, etc.) to increase the breadth and accuracy of natural language intent understanding capabilities. The sending device 12 ( Figure 1 ) and receiving device 11 may both include some or all of the features of electronic device 120. In one embodiment, electronic device 120 may include display 121, microphone 122, audio output 123, input mechanism 124, communication circuit 125, control circuit 126, camera 128, intelligent assistant application 129 (for generating functionally similar but semantically different interpretations with additional details, identifying insertion points and determining the amount to be inserted, generating a specified number of words at a specific position in the expression, asking clear follow-up questions and communicating with communication circuit 125 to obtain / provide its information with cloud or server 130; and may include but is not limited to the examples described below) and any other suitable components. In one embodiment, applications 1-N 127 are provided and can be obtained from cloud or server 130, communication network 110, etc., where N is a positive integer equal to or greater than 1.

[0035] In one embodiment, all applications employed by audio output 123, display 121, input mechanism 124, communication circuitry 125, and microphone 122 may be interconnected and managed by control circuitry 126. In one example, a handheld music player capable of streaming music to other tuning devices may be incorporated into electronic device 120.

[0036] In one embodiment, audio output 123 may include any suitable audio components for providing audio to a user of electronic device 120. For example, audio output 123 may include one or more speakers (e.g., mono or stereo speakers) built into electronic device 120. In some embodiments, audio output 123 may include an audio component that is remotely coupled to electronic device 120. For example, audio output 123 may include an audio component that can be coupled to electronic device 120 using a wire (e.g., using a jack) or wirelessly (e.g., Headphones or A headset, headphone or earbuds coupled to a communications device.

[0037] In one embodiment, display 121 may include any suitable screen or projection system for providing a display visible to a user. For example, display 121 may include a screen (e.g., an LCD screen, an LED screen, an OLED screen, etc.) incorporated into electronic device 120. As another example, display 121 may include a removable display or projection system for providing a display of content on a surface remote from electronic device 120 (e.g., a video projector). Display 121 may display content (e.g., information about communication operations or information about available media selections) under the direction of control circuitry 126.

[0038] In one embodiment, input mechanism 124 may be any suitable mechanism or user interface for providing user input or instructions to electronic device 120. Input mechanism 124 may take a variety of forms, such as buttons, a keypad, a dial, a click wheel, a mouse, a visual indicator, a remote control, one or more sensors (e.g., a camera or visual sensor, a light sensor, a proximity sensor, etc.), or a touch screen. Input mechanism 124 may include a multi-touch screen.

[0039] In one embodiment, the communication circuit 125 may be operable to connect to a communication network (e.g., Figure 1 The communication circuit 125 may be configured to communicate with other devices within the communication network 110 and transmit communication operations and media from the electronic device 120 to other devices within the communication network. The communication circuit 125 may be configured to use any suitable communication protocol, such as, for example, Wi-Fi (e.g., IEEE 802.11 protocol), High frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHz communication systems), infrared, GSM, GSM plus EDGE, CDMA, quad-band and other cellular protocols, VOIP, TCP-IP, or any other suitable protocol.

[0040] In some embodiments, the communication circuit 125 may be operable to create a communication network using any suitable communication protocol. For example, the communication circuit 125 may use a short-range communication protocol to create a short-range communication network to connect to other communication devices. For example, the communication circuit 125 may be operable to use a short-range communication protocol to create a short-range communication network to connect to other communication devices. The protocol creates a local communication network to connect the electronic device 120 with Headset coupling.

[0041] In one embodiment, the control circuit 126 can operate to control the operation and performance of the electronic device 120. The control circuit 126 may include, for example, a processor, a bus (e.g., for sending instructions to other components of the electronic device 120), a memory, a storage, or any other suitable component for controlling the operation of the electronic device 120. In some embodiments, the processor can drive the display and process inputs received from the user interface. The memory and storage may include, for example, cache, flash memory, ROM and / or RAM / DRAM. In some embodiments, the memory may be dedicated to storing firmware (e.g., for device applications such as operating systems, user interface functions, and processor functions). In some embodiments, the memory may be used to store information related to other devices with which the electronic device 120 performs communication operations (e.g., saving contact information related to communication operations or storing information related to different media types and media items selected by the user).

[0042] In one embodiment, the control circuit 126 can be used to perform operations of one or more applications implemented on the electronic device 120. Any suitable number or type of applications can be implemented. Although the following discussion will list different applications, it should be understood that some or all of the applications can be combined into one or more applications. For example, the electronic device 120 may include applications 1-N 127, including but not limited to: automatic speech recognition (ASR) applications, OCR applications, conversation applications, mapping applications, media applications (e.g., QuickTime, MobileMusic.app, or MobileVideo.app), social networking applications (e.g., The electronic device 120 may include one or more applications operable to perform communication operations. For example, the electronic device 120 may include a messaging application, an email application, a voicemail application, an instant messaging application (e.g., for chatting), a video conferencing application, a fax application, or any other suitable application for performing any suitable communication operation.

[0043] In some embodiments, the electronic device 120 may include a microphone 122. For example, the electronic device 120 may include a microphone 122 to allow a user to send audio (e.g., voice audio) for voice control and navigation of the applications 1-N 127 during a communication operation, or as a means of establishing a communication operation, or as an alternative to using a physical user interface. The microphone 122 may be incorporated into the electronic device 120, or may be remotely coupled to the electronic device 120. For example, the microphone 122 may be incorporated into a wired headset, the microphone 122 may be incorporated into a wireless headset, the microphone 122 may be incorporated into a remote control, etc.

[0044] In one embodiment, camera module 128 includes one or more camera devices that include functionality for capturing still and video images, editing functionality, communication interoperability for sending, sharing, etc., photos / videos, and the like.

[0045] In one embodiment, the electronic device 120 may include any other components suitable for performing communication operations. For example, the electronic device 120 may include a power supply, a port or interface for coupling to a host device, an auxiliary input mechanism (e.g., an ON / OFF switch), or any other suitable component.

[0046] Figure 3 An example 300 of identifying insertion points and amounts (insertion amounts) according to some embodiments is shown. In example 300, the expression "buy me a sausage pizza" is used and includes insertion points 310, 320, and 330. In some embodiments, an utterance is spoken, by a microphone (e.g., Figure 1 The input is captured by a microphone 122 of the electronic device 120 and converted to text (e.g., using a speech-to-text recognition process by an electronic device 120, a cloud / server 130, or a combination thereof). In other embodiments, the input is a text expression. In some embodiments, for each sequential word pair in the expression, a basic score is determined by a process by a cloud or server 130, an intelligent assistant 129 using a processor and memory of an electronic device 120, or a combination thereof. The basic score measures the likelihood that two words appear in that order in a sequence pair in normal use. In some embodiments, the process measures the likelihood that two words appear in that order in normal use, but at various short distances from each other. The process also scores the insertion quality based on the score on the distance D divided by the basic score. If the ratio is greater than a threshold (e.g., a number selected by an artificial intelligence (AI) programmer, a number selected based on processing speed, etc.), the process identifies the position between the two words as an insertion point and sets the amount inserted to the distance D.

[0047] In example 300, for a distance D of 1 (reference 311), the pair of "buy" and "me" results in buy_me / buy me>1. The pair of "a" and "sausage" results in a_sausage / sausage>1. The pair of "sausage" and "pizza" results in sausage_pizza / sausage pizza<1. For a distance D of 2 (reference 312), the pair of "buy" and "me" results in buy__me / buy me>1. The pair of "a" and "sausage" results in a__sausage / a sausage>1. The pair of "sausage" and "pizza" results in sausage__pizza / sausage pizza<1. For a distance D of 3 (reference 313), the pair of "buy" and "me" results in buy___me / buy me<1. The pair of "a" and "sausage" results in a___sausage / a sausage>1. The pair "sausage" and "pizza" results in sausage___pizza / sausage pizza>1. Thus, insertion point 310 can insert 1 or 2 words; insertion point 320 can insert 1, 2, or 3 words; and insertion point 330 can insert 3 words. Figure 7 Describes more details for identifying the insertion point and amount.

[0048] Figure 4 An example 400 of generating words for a specification according to some embodiments is shown. Figure 3 ), in example 400, the preprocessing portion includes training a generative language model to predict forward probabilities, and training a generative language model to predict backward probabilities. An expression (e.g., "Buy me a sausage pizza") is represented as a graph including nodes. The process then creates nodes for insertion points, where words must be generated at the insertion points. The nodes are inserted into the graph using the forward and backward models to generate edge weights. The process ensures that the path connects from the start point to the end point and continues through a specified number of insertion nodes at the insertion point. The paths are scored using the forward and backward scores from the model, the top X best scoring paths are selected, and the associated words are returned. In some embodiments, the generative language model can be personalized. Simple personalized language model embodiments may include: training personal natural language data created by users, such as their emails and other recorded conversations; updating an existing language model by performing additional training iterations on personal language data; using a collection of personal and general models; mixing model parameters of personal models and general models; using model fusion with personal and general models, etc.

[0049] In example 400, for insertion point 310, associated words 410 may include the following example words (ie, 1 or 2 words): “one get,” “and get,” “it for,” “to get,” “to make,” “they give,” “andgive,” “and,” “make,” “and,” “bring,” and “to bring.” For insertion point 320, associated words 420 may include example words (i.e., 1 word): “free”, “turkey”, “chicken”, “new”, “small”, “delicious”, “little”, “large”, and “Italian”; associated words 421 may include words (i.e., 2 words): “slice of”, “piece of”, “chicken and”, “cheese and”, “bowl of”, “pepperoni and”, “side of”, “turkey and”, “sausage and”, and “bag of”; associated words 422 may include example words (i.e., 3 words): “slice of the”, “pizza and a”, “sample of the”, “piece of the”, “side of the”, “bowl of the,” “bite of the,” “bag of the,” “sandwich and a,” and “salad and a.”For insertion point 330, associated words 430 may include example words (i.e., 3 words): "sandwich and a," "and a cheese," "pizza or a," "sandwich or a," "and cheese and," "and goat cheese," "and gluten free," "salad and a," "and a white," "instead of a," "egg and cheese," "ham and cheese," "instead of the," and "and thin crust." Reference is made below. Figure 8 Describes more details for generating words for the specification.

[0050] Figure 5 An example 500 of generating follow-up questions using details according to some embodiments is shown. In some embodiments, the process of 500 includes: identifying an insertion point; generating a specified number of words at a specific position in the expression (the generated words are details that need to be clarified); and generating a clear question using the details. In one or more embodiments, the generation of clear questions may include partitioning the generated details to present a choice between similar alternatives (e.g., separate alternatives with the term "or"); general templates, such as "did you mean X, Y or Z?"; and general templates plus details in context: keeping the words around the insertion point 510 next to the generated details (e.g., you can ask to confirm a detail or use "or" to indicate a choice). As a result of the partitioning step 520, example partitions of words 521 may include: small, little, and large; partitions of words 522 may include turkey, chicken, and Italian, and other words 523 may include: free, new, and delicious. The partitioning of word 521 maintains the order of word 521 , wherein the partitions ( 522 and 523 ) are considered subsets of word 521 .

[0051] In some embodiments, for the next step 530, detailed questions are asked using the partition words as an option. The insertion context words in segment 540 are copied and the words for the partition are inserted at 545. In this example, the words from cluster 521 (small, smaller, and large) are inserted after "A". At 550, the word "or" and a comma are inserted between the words in the partition(s). The result is "A small, little or large sausage pizza". Refer to the following Fig. 9 Descriptions are used to generate more details for follow-up questions using the details.

[0052] Figure 6 A block diagram of a process 600 for generating a functionally similar but semantically different interpretation with additional details according to some embodiments is shown. In box 610, a word is marked as a content or non-content word. In some embodiments, non-content words can be identified by running a part-of-speech tagger process. Closed class words are usually non-content words and can be identified by their part-of-speech tags (prepositions, determiners, etc.). All other words are considered content words. In box 620, for all content words, a set of candidate replacement words is generated using a process that measures the functional similarity between two words in the same context. In box 630, potential combinations of candidate words are scored, and scores are used to remove bad combinations. In some embodiments, the functional similarity scores of each selected candidate are multiplied. A threshold is used to determine whether the combined score has a value large enough to be acceptable. The threshold can be set to a predetermined level of functional similarity. Alternatively, in some embodiments, a set of rules are used to set the threshold to select the value of the given functional similarity score for each candidate. An example rule may be to set the threshold to the functional similarity score that has the worst suggested functionally similar word from a set of best suggested alternative words from all open class words.The process 600 then generates additional details for the combination based on blocks 640, 650, and 660 and inserts them into the combination as additional phrases.

[0053] In some embodiments, additional details are generated for each combination in block 630. In one embodiment, the additional details are generated on the original expression and reused by inserting them into the same insertion point in the combination generated in block 630. In block 640, the insertion point is identified and the insertion amount is determined. In block 650, a specified number of words are generated at a specific location in the expression. In block 660, the generated words are inserted at the insertion point identified in block 640.

[0054] Figure 7A block diagram of a process 700 for identifying insertion points and determining insertion amounts according to some embodiments is shown. In some embodiments, for each sequential word pair in an expression (e.g., when the expression is spoken, the microphone (e.g., Figure 1 The process 700 measures the likelihood of two words appearing in that order in normal use, but at various short distances from each other. In some embodiments, the process 700 uses the original corpus n-gram counts. In box 720, the process 700 measures the likelihood of two words appearing in that order in normal use, but at various short distances from each other. In some embodiments, the process 700 measures when there are 1, 2, or 3 words between a pair of words. These scores are scores with a distance of D, where D is the number of words between the pair of words. In box 730, the process 700 scores insertion quality = score with a distance of D / basic score. In box 740, if the ratio is greater than a threshold, the position between the two words is identified as an insertion point, and the insertion amount is set to the distance D.

[0055] Figure 8 A block diagram of a process 800 for generating a specified number of words at a specific position in an expression according to some embodiments is shown. Given an expression (e.g., when the expression is spoken, a microphone (e.g., Figure 1The microphone 122 of the electronic device 120 is captured and converted into text (for example, using speech-to-text recognition processing by the electronic device 120, the cloud / server 130, or a combination thereof), and the requirement to generate a specific number of words at a specific position, the process 800 generates a set of words that can be inserted into the corresponding position. In some embodiments, the expression can be a text expression. Here, "X" is used to represent the number of unique sets of words that are expected to be generated. In box 810, the process 800 trains a language model to predict the probability of the next word given the previous word. The processing in box 810 is called a forward model. In some embodiments, the forward model is an n-gram model. In one or more embodiments, the forward model is a recursive neural network (RNN) with a long short-term memory block. In box 820, the process 800 trains a language model to predict the probability of the previous word given the subsequent word. The processing in box 820 is called a backward model. In some embodiments, the backward model is an n-gram model. In one or more embodiments, the backward model is an RNN with a long short-term memory block. In box 830, the process 800 represents the expression as a graph. In some embodiments, start and stop nodes are added in front and behind the expression, and each word is linked to its successor by an edge. In frame 840, process 800 creates nodes for each word that must be generated. In frame 850, process 800 places the node from frame 840 at the specified insertion point in the figure. In some embodiments, forward and backward models are used to generate and score the edge weights that link these additional nodes to the graph. Nodes should be inserted so that there is a path from start to finish, and the number of nodes that each path passes through is equal to the number of words specified at the insertion point. Words are associated with each edge. In some embodiments, in frame 860, the probability of generating the word using the generative model is used as the weight on the edge, and the weight is used to score the path. Some embodiments use the score combination from both the forward model and the backward model for edge scoring. One or more embodiments combine these scores using addition, and some embodiments combine the scores using multiplication. One or more embodiments use forward scoring for the edge in the first half of the insertion, and use backward scoring for the edge in the second half of the insertion. In some embodiments, particle filtering techniques are used to maintain only a subset of nodes and edges to be selected in box 860. In box 860, process 800 scores the paths using the forward and backward scores from the model and selects the top X best scoring paths. The associated words are returned for insertion. In some embodiments, dynamic programming is used to select the best path. In one or more embodiments, particle filtering techniques are used to determine the best path.

[0056] Fig. 9 A block diagram of a process 900 for generating explicit follow-up questions according to some embodiments is shown. In block 910, the process 900 uses the process 700 ( Figure 7 ) to identify the insertion point and determine the amount to be inserted. In block 920, process 900 uses process 800 ( Figure 8 ) generates a specified number of words at a specific position in the expression. These are the details that need to be made clear. In box 930, process 900 formulates a clear question asking which detail is intended to be used. In some embodiments, a general template can be used, in which additional details are inserted into the general question, such as "Did you mean X, Y, or Z?" In one or more embodiments, the text around the insertion point in the template is used to provide context. A fixed number of words around the insertion point can be selected, the entire sentence (plus insertion) can be selected, or a rule can be used to select a subsection of the sentence based on the parser output. In some embodiments, subsections of the sentence are used. The end of the subsection is the end of the sentence, or the first closed class word encountered from the insertion point (non-inclusive). The beginning of the subsection is the last closed class word encountered when coming from the insertion point (inclusive).

[0057] Fig.10 1000 includes one or more processors 1011 (e.g., ASICs, CPUs, etc.), and may further include an electronic display device 1012 (for displaying graphics, text, and other data), a main memory 1013 (e.g., random access memory (RAM), a cache device, etc.), a storage device 1014 (e.g., a hard drive), a removable storage device 1015 (e.g., a removable storage drive, a removable memory, a tape drive, an optical drive, a computer-readable medium having stored therein computer software and / or data), a user interface device 1016 (e.g., a keyboard, a touch screen, a keypad, a pointing device), and a communication interface 1017 (e.g., a modem, a wireless transceiver (such as Wi-Fi, a cellular network), a network interface (e.g., an Ethernet card), a communication port or a PCMCIA slot and card).

[0058] The communication interface 1017 allows software and data to be transferred between the computer system and external devices via the Internet 1050, mobile electronic devices 1051, servers 1052, networks 1053, etc. The system 1000 also includes a communication infrastructure 1018 (e.g., a communication bus, crossbar switch, or network) to which the above-mentioned devices 1011 to 1017 are connected.

[0059] Information transmitted via communication interface 1017 may be in the form of signals, such as electronic, electromagnetic, optical, or other signals capable of being received by communication interface 1017 via a signal-carrying communication link, and may be implemented using wire or cable, optical fiber, a telephone line, a cellular telephone link, a radio frequency (RF) link, and / or other communication channels.

[0060] In one implementation of one or more embodiments in a mobile wireless device (e.g., a mobile phone, a tablet computer, a wearable device, etc.), the system 1000 also includes an image capture device 1020, such as a camera 128 ( Figure 2 ), and an audio capture device 1019, such as a microphone 122 ( Figure 2 ). The system 1000 may further include application processing or processors, such as MMS 1021, SMS 1022, email 1023, social network interface (SNI) 1024, audio / video (AV) player 1025, web browser 1026, image capture 1027, etc.

[0061] In one embodiment, the system 1000 includes an intelligent agent process 1030 that can implement communication with respect to the intelligent assistant application 129 ( Figure 2 ) and the process 600 ( Figure 6 ), process 700( Figure 7 ), process 800( Figure 8 ) and process 900( Fig. 9 In one embodiment, the intelligent agent process 1030 together with the operating system 1029 can be implemented as executable code residing in the memory of the system 1000. In another embodiment, the intelligent agent process 1030 can be provided in hardware, firmware, etc.

[0062] In one embodiment, the main memory 1003 , the storage device 1004 , and the removable storage device 1005 , each alone or in any combination, may store instructions of the above embodiments that may be executed by one or more processors 1001 .

[0063] As known to those skilled in the art, according to the above architecture, the aforementioned example architecture described above can be implemented in a variety of ways, such as program instructions executed by a processor, as a software module, microcode, as a computer program product on a computer-readable medium, as an analog / logic circuit, as an application-specific integrated circuit, as firmware, as a consumer electronic device, AV device, wireless / wired transmission unit, wireless / wired reception unit, network, multimedia device, etc. In addition, the embodiment of the architecture can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment containing both hardware and software elements.

[0064] One or more embodiments have been described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to one or more embodiments. Each frame or combination thereof of such diagrams / charts can be implemented by computer program instructions. When the computer program instructions are provided to the processor, the machine program produces a machine so that the instructions executed by the processor create a device for implementing the function / operation specified in the flowchart and / or block diagram. Each frame in the flowchart / block diagram can represent the hardware and / or software module or logic that implements one or more embodiments. In alternative embodiments, the functions indicated in the frame may not occur in the order indicated in the figure, concurrently, etc.

[0065] The terms "computer program medium", "computer usable medium", "computer readable medium" and "computer program product" are generally used to refer to media such as main memory, secondary memory, removable storage drive, hard disk installed on hard disk drive. These computer program products are devices for providing software to computer systems. The computer readable medium allows the computer system to read data, instructions, messages or message packets, and other computer readable information from the computer readable medium. For example, computer readable media can include non-volatile memory such as floppy disks, ROM, flash memory, disk drive memory, CD-ROM and other permanent storage. For example, it is useful for transferring information (such as data and computer instructions) between computer systems. Computer program instructions can be stored in a computer readable medium, which can instruct a computer, other programmable data processing device or other equipment to operate in a specific manner so that the instructions stored in the computer readable medium generate an article of manufacture including instructions, which implement the functions / actions specified in the flowchart and / or block diagram.

[0066] The computer program instructions representing the block diagrams and / or flow charts of this article can be loaded onto a computer, a programmable data processing device, or a processing device so that a series of operations performed thereon generate a computer-implemented process. The computer program (i.e., computer control logic) is stored in a main memory and / or an auxiliary memory. The computer program can also be received via a communication interface. Such a computer program enables a computer system to perform the features of the embodiments discussed herein when executed. Specifically, the computer program enables a processor and / or a multi-core processor to perform the features of a computer system when executed. Such a computer program represents a controller of a computer system. A computer program product comprises a tangible storage medium that can be read by a computer system and stores instructions executed by a computer system to perform the method of one or more embodiments.

[0067] Although the embodiments have been described with reference to certain embodiment versions; however, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.

Claims

1. A method for an intelligent assistant, the method comprising: identifying, by the electronic device, an insertion point between at least one sequential pair of words within the input comprising text at which at least one word is to be inserted; The electronic device obtains a plurality of associated words to be inserted into the insertion point; grouping, by the electronic device, the plurality of associated words into one or more sets; as well as generating, by the electronic device, a question by inserting at least one associated word included in one of the one or more sets into an insertion point; Wherein, identifying the insertion point comprises: identifying a first probability that the at least one sequential word pair is used sequentially; identifying a second probability that a word distance exists between the at least one sequential word pair to be used sequentially; and identifying an insertion point based on the first probability and the second probability, and The generation problem includes: representing the text as a graphic; generating a plurality of nodes corresponding to the plurality of associated words; At least one node to be connected to the graph is determined among the plurality of nodes by using a forward score of a forward model that predicts the probability of a next word considering previous words for a first half of an insertion point and a backward score of a backward model that predicts the probability of previous words considering subsequent words for a second half of an insertion point.

2. The method of claim 1 further comprising identifying the number of the at least one word to be inserted into the insertion point, wherein Obtaining the plurality of associated words is based on the number of the at least one word to be inserted into the insertion point.

3. The method according to claim 2, in, A quantity of the at least one word to be inserted into the insertion point is identified based on the first probability and the second probability.

4. The method according to claim 3, wherein: Identifying the insertion point and the number of the at least one word to be inserted into the insertion point based on the first probability and the second probability includes: identifying a ratio of the first probability and the second probability; and When the ratio is greater than a predetermined value, a position between the at least one sequential word pair is determined as an insertion point, and the word distance is determined as the number of the at least one word to be inserted into the insertion point.

5. The method according to claim 1, wherein: Identifying the plurality of associated words further comprises: generating a plurality of paths by inserting the plurality of nodes into insertion points; scoring the plurality of paths using a forward language model and a backward language model; and The plurality of associated words are identified based on the scores.

6. The method according to claim 1, further comprising: Classify words in a text as content words or non-content words; Generate combinations of candidate words to replace content words; Scoring combinations; as well as Based on the scores, combinations with scores below a threshold are removed from the combinations.

7. An electronic device comprising: a memory for storing instructions; as well as At least one processor configured to execute instructions to: identifying an insertion point between at least one sequential pair of words within an input comprising text at which at least one word is to be inserted; Obtaining a plurality of associated words to be inserted into the insertion point; grouping the plurality of associated words into one or more sets; as well as generating a question by inserting at least one associated word included in one of the one or more sets into an insertion point; Wherein, in order to identify the insertion point, the at least one processor is further configured to execute instructions to: identifying a first probability that the at least one sequential word pair is used sequentially; identifying a second probability that a word distance exists between the at least one sequential word pair to be used sequentially; and identifying an insertion point based on the first probability and the second probability, and The generation problem includes: representing the text as a graphic; generating a plurality of nodes corresponding to the plurality of associated words; At least one node to be connected to the graph is determined among the plurality of nodes by using a forward score of a forward model that predicts the probability of a next word considering previous words for a first half of an insertion point and a backward score of a backward model that predicts the probability of previous words considering subsequent words for a second half of an insertion point.

8. The electronic device according to claim 7, wherein: The at least one processor is further configured to identify a quantity of the at least one word to be inserted into the insertion point, In which, in order to obtain the multiple associated words to be inserted into the insertion point, the at least one processor is also configured to execute instructions to obtain the multiple associated words to be inserted into the insertion point based on the number of the at least one word to be inserted into the insertion point.

9. The electronic device according to claim 8, in, A quantity of the at least one word to be inserted into the insertion point is identified based on the first probability and the second probability.

10. The electronic device according to claim 9, wherein: To identify the insertion point and the number of the at least one word to be inserted into the insertion point based on the first probability and the second probability, the at least one processor is further configured to execute instructions to: identifying a ratio of the first probability to the second probability; as well as When the ratio is greater than a predetermined value, a position between the at least one sequential word pair is determined as an insertion point, and the word distance is determined as the number of the at least one word to be inserted into the insertion point.

11. The electronic device according to claim 7, wherein: To identify the plurality of associated words, the at least one processor is further configured to execute instructions to: generating a plurality of paths by inserting the plurality of nodes into insertion points; scoring the plurality of paths using a forward language model and a backward language model; as well as The plurality of associated words are identified based on the scores.

12. The electronic device according to claim 7, wherein: The at least one processor is further configured to execute instructions to: Classify words in a text as content words or non-content words; Generate combinations of candidate words to replace content words; Scoring combinations; as well as Based on the scores, combinations with scores below a threshold are removed from the combinations.

13. A computer readable medium comprising a program which, when executed by a processor, performs the method of any one of claims 1 to 6.

Images