Spoken language control system

Abstract

System and techniques for processing spoken language to use as input to a control system are described herein. After an utterance is obtained from a user in a general language, a generative neural network model is invoked on the utterance to transform the utterance into a phrase that conforms to a domain-specific language. The domain specific language phrase is provided a control system that accepts phrases of the domain-specific language as input and controls a device based on the input.

Description

SPOKEN LANGUAGE CONTROL SYSTEMCLAIM OF PRIORITY

[0001] This patent application claims the benefit of priority, under 35 U.S.C. § 119, to United States Provisional Application Serial No. 63 / 516,404, titled ‘PARAPHRASE PROJECTOR FOR LANGUAGE INTERFACE AND RELATED SYSTEM” and filed on July 28, 2023, and also claims priority to United States Provisional Application Serial No. 63 / 531,167, titled ‘‘PARAPHRASE PROJECTOR FOR LANGUAGE INTERFACE AND RELATED SYSTEM” and filed on August 7, 2023, the entirety of all are hereby incorporated by reference herein.TECHNICAL FIELD

[0002] Embodiments described herein generally relate to human machine interfaces for equipment control systems and more specifically to a spoken language control system.BACKGROUND

[0003] Machine control systems are used in automating or optimizing manufacturing processes. These systems can encompass a variety of technologies, including programmable logic controllers (PLCs), distributed control systems (DCS), and supervisory control and data acquisition (SCAD A) systems. PLCs are ruggedized digital computers used for automation of electromechanical processes, providing real-time responses and flexibility’ in controlling machinery. DCS systems offer decentralized control by distributing the control functions across multiple controllers to enhance reliability and performance. SCADA systems provide a higher-level overview and control, enabling operators to monitor and manipulate industrial processes from a central location, often integrating with PLCs and DCS. These control systems use a range of sensors, actuators, and human-machine interfaces (HMIs) to ensure precise operation, safety, and efficiency.

[0004] HMIs are used in many industrial and consumer applications, serving as the bridge between users and these often complex systems. HMIs encompass a range of devices or software applications that seek to provide intuitive, user-friendly interfaces for monitoring or controlling machinery and processes. These interfaces can include touchscreens, graphical displays, keypads, and voice control systems among others. Voice control technology — which can include technologies such as natural language processing (NLP) or artificial intelligence (Al) — enables operators to interact with machines through spoken commands, enhancing accessibility and efficiency, particularly in environments where manual control is challenging or impractical. Integration of voice control with HMIs can improve operational flexibility, reduce the need for extensive training, and enable hands-free operation.

[0005] Generative Al, such as Large Language Models (LLMs), enables machines to generate human-like text and respond to semantic inputs. LLMs are usually trained on extensive datasets made up of diverse textual information to produce coherent, contextually relevant responses across a wide range of topics. These models use deep learning techniques, such as transformer architectures, to capture subtle, and sometimes very' complex, patterns in language. The result is a quantification of many nuances in human communication that can be leveraged for a variety of tasks. These tasks can include content creation, language translation, summarization, or conversational agents. The ability7of LLMs to process semantic inputs enables them to quantify (e.g., interpret) meaning and intent behind user queries, enabling the model to provide accurate and context- aware responses.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. Like numerals having different letter suffixes may represent different instances of similar components. The drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document.

[0007] FIG. 1 is a block diagram of an example of an environment including a system for spoken language control system, according to an embodiment.

[0008] FIG. 2 illustrates an example of an architecture for spoken language control, according to an embodiment.

[0009] FIG. 3 illustrates an example of a prompt sequence being used to generate a next word from a generative Al model, according to an embodiment.

[0010] FIG. 4 illustrates an example of a prompt sequence being used to generate a sequence of words from a generative Al model, according to an embodiment.

[0011] FIG. 5 illustrates an example of a prompt sequence being used to generate a next word in a domain specific language from a generative Al model, according to an embodiment.

[0012] FIG. 6 illustrates an example of a prompt sequence being used to generate a next word and a subject classification from a generative Al model, according to an embodiment.

[0013] FIG. 7 illustrates a flow diagram of an example of a method 700 for spoken language control system, according to an embodiment.

[0014] FIG. 8 is a block diagram illustrating an example of a machine upon which one or more embodiments may be implemented.DETAILED DESCRIPTION

[0015] Speech based HMIs have been around for some time. Generally, these systems include a speech recognition facility to transform audio signals captured by a microphone, or the like, to text. Traditionally, this text was matched against commands for a given control system. These commands are a form a domain-specific language (DSL) that is often a very' small subset of the language of the user. Variances in command form, or synonymous meanings in the DSL, generally must be anticipated by the DSL designers and incorporated into the DSL itself. Thus, users must generally' know the DSL and accurately apply the DSL when giving spoken language commands to a control system. Such a system can be called a semantic interpreter. Generally, the semantic interpreter has a defined (e.g., user defined) domain specific grammar and vocabulary. Sentences are parsed into the DSL and a semantic engine parsed sentences to programmatic intents. These systems tend to be precise (e.g., the command given is the command carried out) and computationally efficient.

[0016] An issue can arise in traditional spoken language (e.g., voice) systems in providing reliable interpretation of user commands when a user is unaware of the DSL vocabulary or grammar. That is, in an environment where an untrained user can use natural language (e.g., a general language model such as standard American English), including idiomatic expressions, it is difficult for the control system to interpret the user's utterance and provide a reliable (e.g., correct) response in control (e.g.. manipulating an actuator) or response (e.g., providing a synthesized voice answer to a query).

[0017] Contemporary' generative Al models can provide sophisticated responses to a large variety of user queries. However, these models also produce great variety in their responses, such that output from the generative Al model can be different given the same inputs at different times, and can also be untrue (e.g., a hallucination by the generative Al model). Thus, while the semantic meaning of the input is generally addressed appropriately in the response, the variance that is problematic for natural language input to HMIs can still be present. Another issue that can arise is in verifying whether the system correctly interpreted the query or command. Generally, there is difficulty in determining how the generative Al model arrived at a given response. Consider, if one were speaking with an automated teller from a bank, one might want to be able to ask it how much money was in a bank account and have the automated teller reply with the correct amount. The person may also wish to withdraw cash from the automated teller and expect that the correct amount is withdrawn and given to the person. In these cases, it is generally unacceptable if the system is unreliable and sometimes answers one's query or command incorrectly (e.g.. by ‘’hallucinating” an incorrect amount of money).

[0018] Computational resource consumption is another issue that can arise with current generative Al models. These models can have hundreds of billions of parameters (e.g.. inter-neuronal weights or biases), run on hardware that uses a lot of power and space, and provides responses on the order of seconds. In contrast, machine controls in industrial applications, vehicles (e.g., in a car), or a mobile phone generally do not have the power or the space to host these models locally. Moreover, when consider voice control for something like a vehicle, sub-second response times are likely necessary for the interface to be useful to a user.

[0019] To address the issues noted above — e.g., to provide the ability to accept input (e.g.. queries) using a wide range of natural language expressions without user understanding of a particular voice command set — a spoken language control system can be used in which an utterance from a general language model is transformed by a specialized generative Al model into the DSL that can then be efficiently parsed into commands (e.g., via a semantic interpreter with a context free grammar) for the control system. This structure combines the benefits of generative Al models to accept a wide set of vocabulary, grammar, and expressions while providing a computationally efficient implementation that produces reliable and consistent control inputs. This efficient generative Al model, called a paraphrase projector in this document, is trained to produce valid output in the DSL. This restricted output range enables a comparatively small number of parameters (e.g., twenty million or fewer), which then can be processed more quickly on more limited hardware. Further, when the DSL is in the same language as the speaker (e.g., the DSL is a proper subset of a general language), the speaker can be given an opportunity to verify the command before execution. Even if the speaker is speaking in a language that is different language than the DSL, the translation from the DSL output, back to the user’s tongue, can be performed to enable this verification. Additional details and examples are provided below.

[0020] FIG. 1 is a block diagram of an example of an environment including a system 105 for spoken language control system, according to an embodiment. The system 105 includes processing circuitry 110, storage 120 (e.g., power-stable storage such as a hard drive, solid state drive, etc.), and memory 115. The memory 115 is generally used to maintain running state information for the system 105 that is generally discarded between system power cycles or restarts. The memory 115 and the storage 120 are both forms of computer readable media. The processing circuitry 110 or software residing in the memory 1 15 or storage 120 executing on the processing circuitry 110 configure the system 105 to perform various operations when in operation. As illustrated, the system 105 is embedded in a vehicle 125, but any other machine subject to control can host the system 105.

[0021] The system 105 is illustrated as implementing a voice-to- command pipeline 130.The pipeline 130 includes a voice capture component 135(e.g., a microphone) configured to sense sound and produce an audio signal. The pipeline 130 includes a speech- to-text system 140 that is configured to take the audio signal and convert that signal to text. The pipeline 130 includes a generative Al model 145 (e.g., a paraphrase projector or generative neural network) configured to transform the text into a DSL phrase. The pipeline include a semantic interpreter 150 that is configured to accept a valid DSL phrase and output a command. The pipeline 130 includes control circuitry 155 that is configured to accept the command and produce some result (e g., open the car windows).

[0022] To implement the spoken language control system, the processing circuitry’ 110 is configured to obtain (e.g., receive, retrieve, etc.) an utterance from a user 160. Here, the utterance is in a general language. The term general language refers to a language, such as English, French, Chinese, etc., used for natural language activities (e.g., conversations between people). In an example, the utterance is text created from the speech-to-text system 140 (e g., or other speech recognition system) to convert the signal from the voice capture component 135 to the text.

[0023] The processing circuitry' 110 is configured to invoke the generative Al model 145 on the utterance to transform the utterance into a phrase that conforms to a DSL. In an example, the DSL is a proper subset of the general language. In this last example, the DSL is a proper subset of the general language when using vocabulary and grammar from the general language. In any case, however, the general language is larger than the DSL. It is this size difference than enables optimizations to a traditional generative Al model used to run the transformation on embedded systems such as those illustrated here.

[0024] In an example, the generative Al model 145 uses a transformer architecture. A transformer architecture is based on a sequence-to-sequence framework using self-attention mechanisms. Generally, a transformer includes an encoder-decoder structure, where the encoder processes an input sequence and generates context-aware representations, and the decoder generates the output sequence by attending to these representations and previously generated tokens (output pieces, such as words, parts of words, etc ). Self-attention layers enable the model to weigh the importance of different words in a sequence, regardless of their position, enabling efficient handling of long-rangedependencies. The architecture can use multi-head attention to capture diverse linguistic patterns and position-wise feed-forward networks for non-linear transformations, with residual connections and layer normalization facilitating stable training. This design enables parallel processing and improved scalability compared to traditional recurrent neural networks. In an example, the generative neural network model has fewer than thirty million parameters. This example is the result of the general language to DSL training of the generative Al model 145. Again, because the DSL is a much more limited range than the general language, the number of parameters needed to accurately output tokens in the DSL is very small compared to general purpose generative Al models that can range into the hundreds of billions of parameters.

[0025] In an example, invoking the generative Al model 145 on the utterance includes iteratively building the phrase. Here, an iteration (e.g., each iteration) produces a token based on a current context. This current context includes the utterance and tokens from previous iterations. In a transformer architecture, the model is trained to produce a most likely next token given the prompt and tokens produced from previous iterations. The token could be a work, letter, or any other element in the range over which the model was trained. In a naive implementation, the generation of one token involve a pass through the feed-forward network of the transformer. As a token is produced, the token is added to the input context and another pass through the feedforward network occurs to create the next token. As shown below, this iterative nature can provide points of optimization in the pipeline 130.

[0026] In an example, the generative Al model 145 includes a target output in addition to phrase output. This target output provides an indication as to whether the utterance is directed to a subject of the system 105. The target output operates in addition to the token generative output. The purpose is to determine whether the utterance is actually directed to the vehicle 125 — the subject of the system, for example — or whether the utterance is part of a conversation between passengers in the vehicle 125. Due to the iterative nature of the DSL phrase generation, the process can be stopped early if, for example, the target output indicates that the utterance is not directed to a subject of the system 105. Because the target output is another set of neuronal connections in the generative Al model 145, a first token in the DSL phrase can be generatedand discarded when the target output indicates that the utterance is not relevant for the system 105. Accordingly, in an example, the processing circuitry 110 is configured to obtain a second utterance from the user 160 and to invoke the generative Al model 145 on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the system 105. In this case, the processing circuitry 110 is configured to refrain from executing additional iterations of the generative Al model 145 on the second utterance to cease processing of the second utterance.

[0027] In an example, the processing circuitry 110 is configured to compare, at an iteration, a current state of the phrase with valid phrases in the DSL to determine a set of valid next tokens for a next token on a next iteration. Then, the output of the generative Al model can be limited to the set of valid next tokens for the next iteration. This example enables even greater determinism by actively eliminating a token sequence early to address a circumstance where, for example, a highest probability’ token does not match a valid DSL phrase. Moreover, because the invalid token is not added to the output during the iterations, no iterative processing occurs based on the invalid DSL phrase.

[0028] In an example, the operations of the method 700 can include comparing, at an iteration, a current state of the phrase with valid phrases in the DSL to determine a single valid phrase and using the single valid phrase as the phrase without processing further iterations (e.g., ceasing future iterations with respect to the phrase generation). Whereas the previous example illustrated a way to ensure that any future iterations are producing valid DSL phrase output, this example addresses a situation where the generation of the remaining tokens of the DSL phrase is unnecessary7because there is only one valid DSL phrase left given the previously generated tokens. Here, the iterative processing of tokens by the generative Al model 145 can be stopped and the single phrase produced (e.g., selected and forwarded) as the output of the generative Al model 145.

[0029] The processing circuitry7110 is configured to provide the phrase from the generative Al model 145 to a control system configured to accept phrases of the DSL as input. Here, the control system is configured to control a device (such as the vehicle 125, an industrial machine, a mobile phone, etc.) based on the input. In an example, the control system includes the semanticinterpreter 150 configured to accept valid phrases from the DSL and produce a set of operating parameters. In an example, the set of operating parameters include identification of an interface and parameters that apply to that interface. In this configuration, the semantic interpreter 150 produces commands that something like the control circuitry 155 interprets and executes. These commands can take the form of structured data, such as a Javascript Object Notation (JSON) file that identifies a function and parameters, or the commands can include executable code, a register loading that is executed by the control circuitry' 155, etc. The control circuitry' 155, or other parts of the control system, then cause an actuator to operate (e.g., window motors to engage, a speaker to produce sound, etc.) or other computer effect (e.g.. send a communication using a transceiver) to be performed.

[0030] The generative Al model 145 can be trained to reduce parameters and operate more efficiently in a resource constrained environment as is often the case in device or edge applications. The training and optimizations take advantage of several aspects of transforming a phrase in a general language to a corresponding phrase in the DSL. Many tasks by general LLMs can involve one- to-many or many-to-many relationships between input context (e.g., the prompt) and output. However, the many-to-one nature of the present application for the generative Al model 145 enables many fewer parameters to accurately perform the transformation. The training of such a model, though involves a dataset that is not generally available, in part, because the DSL is a special purpose design for an application. Because the training of these types of models often uses vast amounts of training data, an issue can arise in finding appropriate training data sets. Here, a general purpose LLM can be useful in generating the training data by accepting DSL phrases as input and generating general language model variations (e.g., a one-to-many or a few-to-many relationship). These phrases can then be used to train the generative Al model 145 that will be embedded in the system 105. Specifically, in an example, some processing circuitry (e.g.. an unillustrated server, workstation, etc.) can generate a training data set by using a second phrase that conforms to the DSL as input to a second generative Al model configured to output a third phrase that is in the general language and exclusive of the DSL. In an example, the second (e.g., general LLM) Al modelhas more than one hundred and twenty million parameters in contrast to the twenty million or so parameters of the generative Al model 145.

[0031] Also, given the present context, the operation of the generative Al model 145 can be improved by observing that the output is “causal.” Generally, a forward pass involves passing input data through a similar model to obtain the output. For each new token generated, the model performs a forward pass to predict the next token in the sequence. A causal neural network, such as the generative Al model 145 in this example, ensures that information flows only from earlier tokens to later tokens. This design means that the prediction of the current token depends only on previous tokens, not on subsequent tokens, maintaining a sequential generation process. In an example, instead of recomputing all previous activations (e.g., intermediate outputs of the neural network) from scratch for each new token generated, the generative Al model 145 is configured to cache activations. Here, when generating the next token, the generative Al model 145 reuses the previously computed activations for the existing context and tokens instead of performing feed-forward iterations, resulting in a significant reduction in resource use while reducing the time used in creating the DSL phrase.

[0032] In an example, the generative Al model 145 is configured to use an integer quantization. Quantization reduces the precision of numbers used in computations within a neural network, typically converting floating-point numbers, which require more computational resources and memory, to integer values. This conversion enables computations to be performed using integer arithmetic, which is generally faster and less resource-intensive. For transformer models, which require significant computational resources for training and inference, quantization reduces the computational load and memory usage, making it feasible to deploy these models on devices with limited hardware capabilities, such as mobile phones or edge devices. This technique allows for more efficient computation without significantly compromising the model's performance.

[0033] FIG. 2 illustrates an example of an architecture for spoken language control, according to an embodiment. When the user 205 speaks, the sound is captured by a speech recognition system 210 (e.g., speech-to-text system) and converted to text, possibly with some errors. In an example, insteadof speaking, the user can enter the text (e.g., via a keyboard). In an example, a domain classifier (e.g., target output) can be used to determine whether the input speech was intended for the application (e.g., directed to the machine), or was, for example, background speech, such as a conversation between passengers of a vehicle. In an example, the domain classifier can be a distinct element upstream of the paraphrase projector 215 or may share parts of a neural network used to implement the paraphrase projector 215. The domain classifier provides an output that indicates whether the input was in-domain or out-of-domain. If the domain classifier indicates that the input speech was background speech, the paraphrase projector 215 need not generate nor computationally determine an output, saving resources (e.g., time, energy, hardware use. etc.).

[0034] The paraphrase projector 215 (e.g., the generative Al model 145 from FIG. 1) is configured to accept a wide domain of text (e.g., a large vocabulary7, idioms, or input errors among other things) and convert it to a valid DSL phrase (e.g., clean, structured. English sentences that follow a defined grammar).

[0035] The semantic interpreter 220 is configured to converts the DSL phrase into some language-independent logical form. In an example, the logical form are Prolog clauses or JSON expressions.

[0036] The execution engine 225 is configured to accept the logical form as input (e.g., expressions in the logical form) and convert it to executable commands. For example, received expressions can be compared (e.g., matched) to content of a database of facts or rules (e.g., a knowledge base). The matched record can be augmented (e.g., updating data based on a statement of fact), used to answer a query, or to generate a set of instructions to execute a command. The instructions can be applied to the application programming interface 230 (API) to execute the command. In an example, the API 230 is configured to update the knowledge engine with the state of the system through returned data. For example, the API 230 for a robot can accept instructions to move the arms or wheels of the robot and return the state of these features after activation.

[0037] In an example, the execution engine 225 is configured to output text to inform the user 205 of the response of the system. In an example, the textoutput can be converted to speech using the text-to-speech component 235 and transmitted to the user 205 using the speaker 240.

[0038] The following illustrate examples of using the architecture to control features of an automobile, such as windows, doors, lights, or climate control. These examples illustrate that the user 205 can speak commands that can be outside of the DSL (e.g.. structured English) that the semantic interpreter 220 understands. However, those commands are converted by the paraphrase projector 215 into theDSL understood by the semantic interpreter 220. For example, if the semantic interpreter 220 understands the DSL command “lock all rear doors.'’ If the user says the general language model sentence “I want to lock the doors in the back” (notice that this is technically a declarative statement of fact rather than a command) the paraphrase projector 215 converts this to the DSL command “lock all rear doors.” The following illustrate other examples. The “Query” is the utterance of the user 205, the “Interpretation” is the DSL produced by the paraphrase projector 215, and the “SLU” is the spoken language understanding (e.g., action) by the execution engine 225 :Query: “I want to lock the doors in the back.” Interpretation: “lock all rear doors” SLU response: locking all rear doorsQuery: “Would you roll up all the windows please?” Interpretation: “close all windows” SLU response: closing the windowsQuery: “If any doors are unlocked, would you go ahead and lock them?” Interpretation: lock all unlocked doorsSLU response: locking all front doors (on car that had front doors unlocked)Query: “I think I forgot to lock the trunk.Interpretation: lock the trunkSLU response: locking the trunkOther examples, can include controlling headlights or climate control (e.g., fan speed, temperature, etc.) using idiomatic expressions such as "let's get some air in here” to trigger opening of a window), “can you turn the heat down a bit” to trigger a reduction in a temperature set point, or “it’s getting dark out” to trigger turning the headlights on, among others.

[0039] In an example, the paraphrase projector 215 includes a small (e.g., embedded or edge computing) instance of an LLM. Here small refers to relative parameter counts, where a general chatbot LLM can have hundreds of billions or trillions of parameters, a small LLM has tens of millions (e.g., less than twenty million) parameters. To create the paraphrase projector LLM, a pretrained small LLM can be used as a starting point. For example, in the context of the English language, a small LLM can start as an LLM trained to predict the next word of English text from an English context (e.g., input). This training could include training from Spanish to English, or German to Chinese, etc.. The particular languages do not matter assuming appropriate training data, such as the European Parliament corpora that provides matching translations of a wide variety of text between European languages.

[0040] The small LLM can then be “fine-tuned” by additional training to convert between the general language model (e.g., English generally) and the DSL (e.g., structured English) using a fine-tuning data set. In an example, the fine-tuning data set can be obtained by using a general purpose LLM with requests to accept paraphrases of DSL sentences that the semantic interpreter 220 will accept as input and provide general language expressions as output. The result is a set of equivalent pairs (the input with several outputs) that can be used for the fine-tuning training. Consider the DSL phrase “lock the trunk.” The general purpose LLM can provide alternatives, each of which can make a new training pair for “lock the trunk,” such as “make sure the trunk is locked,” “can you please lock the trunk,” “secure the trunk,” go ahead and lock the trunk for me thanks,” or “lock up the trunk.” for “closing the windows,” the following could useful training pairs generated by the general purpose LLM: close the windows, shut down the windows, seal off the windows, cease the windows, finish the window s, or turn off the windows.

[0041] In generating these alternative expressions, prompts used to the general purpose LLM can have different effects. The purpose is to producenatural and correct paraphrases. Basic prompts often produce unrealistic outputs. Good prompts generally provide examples of good and bad outputs and provide the context of the task. For example, the prompt can include the following context, “it's important that your proposed paraphrases be natural and conversational while containing a useful level of variety,” “don't overdo it with synonyms unless they would really be used in natural speech,” “opening something and unlocking something are not interchangeable,” “don't say the word 'car' too much.”

[0042] FIG. 3 illustrates an example of a prompt sequence 305 being used to generate a next word 315 from a generative Al model 310, according to an embodiment. In general, the generative Al model 310 accepts the sequence of text, the prompt sequence 305 or prompt, and produces the next word 315 as output. This output is often called a prediction. As illustrated, and is typical, the output includes a metric (e.g., a score, probability, etc.) that indicates confidence in the next word being correct. With respect to the examples illustrated in FIG. 1 and FIG. 2. the prompt sequence 305 is a general language model utterance and the output is a token such as a word) in a DSL.

[0043] FIG. 4 illustrates an example of a prompt sequence 405 being used to generate a sequence of words 420 from a generative Al model 410, according to an embodiment. Here, the iterative process of generative Al models is shown by the addition of the token “open” (from the first next word 415) to the prompt sequence 405 to produce the sequence of words 420. These sequence of words 420 and again input into the generative Al model 410 to produce another next word 425 to add to the sequence of words until a termination condition (e.g., a termination token, token limit reached, or iterative processing is terminated as described above).

[0044] FIG. 5 illustrates an example of a prompt sequence 505 being used to generate a next word 515 in a DSL from a generative Al model 510, according to an embodiment. Here, guided inference can be used. Because the DSL is structure and with a finite and reasonably small vocabulary, it is efficient to match outputs to are valid commands. Thus, when an iteration occurs, the next word 515 possibilities that are invalid can be omitted from the output. As illustrated, even though “front” is the top next word 515, “front” can be eliminated at this iteration because is invalid for this given DSL. Guidedinference prevents future iterations that cannot be correct, can stop iterations altogether if there is no possible continuation (e.g.. all next word 515 possibilities are masked), or can skip future iterations if there is a single valid continuation.

[0045] This last example can be for the whole phrase, or for segments of the DSL phrase. For example, if the DSL had two commands to secure the vehicle but the commands varied only in the last word, then the single possible continuance could skip interim iterations until the variance. Consider the DSL commands “lock all doors” and “lock all doors including the trunk.” Assuming that no other DSL commands start with “lock,” then the generation of “lock” as the first next word 515 enables the lookup of “all doors” to be added to the output instead of additional iterations by the generative Al model 510. However, the generative Al model 510, in this case, can take over to determine if that is the end of the command based on the prompt sequence 505 or whether the next word 515 is “including” in which case the second DSL command can be immediately selected and the generative Al model 510 iterations can be terminated.

[0046] FIG. 6 illustrates an example of a prompt sequence 605 being used to generate a next word 615 and a subject classification 620 from a generative Al model 610. according to an embodiment. The subject classification 620 — also referred to a target output or a domain classifier — essentially provides a Boolean (e.g., true or false, yes or no) output as to whether the prompt sequence is directed as a subject of the control system. For example, if a vehicle is subject to a control system (e.g., voice HMI of the vehicle), then the subject classification provides an indication as to whether the utterance of the prompt sequence 605 is directed to the vehicle or elsewhere. As illustrated, the generative Al model 610 is trained to produce the subject classification 620 along with the next word 615 at each iteration. Thus, if the subject classification 620. at the first iteration, indicates that the utterance is not directed to the subject, then the processing of the prompt sequence 605 can terminate immediately. This eliminates consuming resources trying to interpret utterances that are, for example, conversations between people in the vehicle or elsewhere (e.g., a passenger talking to someone on a mobile phone).

[0047] FIG. 7 illustrates a flow diagram of an example of a method 700 for spoken language control system, according to an embodiment. The operations of the method 700 are performed by computer hardware, such as that described above or below (e.g., processing circuitry).

[0048] At operation 705, an utterance from a user is obtained. Here, the utterance is in a general language model. In an example, the utterance is text created from a speech recognition system to convert a signal from a microphone to the text.

[0049] At operation 710, a generative neural network model is invoked on the utterance to transform the utterance into a phrase that conforms to a DSL. In an example, the DSL is a proper subset of the general language model.

[0050] In an example, the generative neural network model uses a transformer architecture. In an example, the generative neural network model has fewer than thirty million parameters.

[0051] In an example, invoking the generative neural network model on the utterance includes iteratively building the phrase. Here, an iteration (e.g., each iteration) produces a token based on a current context. This current context includes the utterance and tokens from previous iterations. In an example, the generative neural network model includes a target output in addition to phrase output. This target output provides an indication as to whether the utterance is directed to a subject of the control system. In an example, the operations of the method 700 can include obtaining a second utterance from the user and invoking the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system. In this case, the operations of the method 700 include refraining from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

[0052] In an example, the operations of the method 700 include comparing, at an iteration, a current state of the phrase with valid phrases in the DSL to determine a set of valid next tokens for a next token on a next iteration. Then, the output of the generative neural network model can be limited to the set of valid next tokens for the next iteration. In an example, the operations of the method 700 can include comparing, at an iteration, a current state of the phrasewith valid phrases in the DSL to determine a single valid phrase and using the single valid phrase as the phrase without processing further iterations (e.g., ceasing future iterations with respect to the phrase generation).

[0053] At operation 715, the phrase is provided to a control system configured to accept phrases of the DSL as input. The control system is configured to control a device based on the input. In an example, the control system includes a semantic interpreter configured to accept valid phrases from the DSL and produce a set of operating parameters. In an example, the set of operating parameters include identification of an interface and parameters that apply to that interface.

[0054] The operations of the method 700 can include generating a training data set by using a second phrase that conforms to the DSL as input to a second generative neural network model configured to output a third phrase that is in the general language model and exclusive of the DSL. The operations can also include training the generative neural network model using the training data set. In an example, the second generative neural network model has more than one hundred and twenty million parameters.

[0055] FIG. 8 illustrates a block diagram of an example machine 800 upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform. Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms in the machine 800. Circuitry (e.g., processing circuitry) is a collection of circuits implemented in tangible entities of the machine 800 that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time. Circuitries include members that may, alone or in combination, perform specified operations when operating. In an example, hardware of the circuitry may be immutably designed to cany7out a specific operation (e.g., hardwired). In an example, the hardware of the circuitry may include variably connected physical components (e.g.. execution units, transistors, simple circuits, etc.) including a machine readable medium physically modified (e g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embeddedhardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, in an example, the machine readable medium elements are part of the circuitry or are communicatively coupled to the other components of the circuitry when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuitry. For example, under operation, execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry. or by a third circuit in a second circuitry at a different time. Additional examples of these components with respect to the machine 800 follow.

[0056] In alternative embodiments, the machine 800 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deploy ment, the machine 800 may operate in the capacity of a server machine, a client machine, or both in server-client network environments. In an example, the machine 800 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment. The machine 800 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations.

[0057] The machine (e.g., computer system) 800 may include a hardware processor 802 (e.g.. a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 804, a static memory (e.g., memory or storage for firmware, microcode, a basic- input-output (BIOS), unified extensible firmware interface (UEFI), etc.) 806, and mass storage 808 (e.g., hard drives, tape drives, flash storage, or other block devices) some or all of which may communicate with each other via an interlink (e.g., bus) 830. The machine 800 may further include a display unit 810, analphanumeric input device 812 (e.g., a keyboard), and a user interface (UI) navigation device 814 (e.g., a mouse). In an example, the display unit 810, input device 812 and UI navigation device 814 may be a touch screen display. The machine 800 may additionally include a storage device (e.g., drive unit) 808, a signal generation device 818 (e.g., a speaker), a network interface device 820, and one or more sensors 816. such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensor. The machine 800 may include an output controller 828, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

[0058] Registers of the processor 802, the main memory 804, the static memory 806, or the mass storage 808 may be, or include, a machine readable medium 822 on which is stored one or more sets of data structures or instructions 824 (e.g.. software) embodying or utilized by any one or more of the techniques or functions described herein. The instructions 824 may also reside, completely or at least partially, within any of registers of the processor 802, the main memory' 804, the static memory 806, or the mass storage 808 during execution thereof by the machine 800. In an example, one or any combination of the hardware processor 802, the main memory 804, the static memory 806, or the mass storage 808 may constitute the machine readable media 822. While the machine readable medium 822 is illustrated as a single medium, the term ‘‘machine readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store the one or more instructions 824.

[0059] The term ‘’machine readable medium” may include any medium that is capable of storing, encoding, or carry ing instructions for execution by the machine 800 and that cause the machine 800 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Nonlimiting machine readable medium examples may include solid-state memories, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon based signals, sound signals, etc.). In an example, a non-transitory machine readable medium comprises a machine readable medium with aplurality of particles having invariant (e.g., rest) mass, and thus are compositions of matter. Accordingly, non-transitory machine-readable media are machine readable media that do not include transitory propagating signals. Specific examples of non-transitory machine readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory’ (EPROM), Electrically Erasable Programmable Read-Only Memory’ (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0060] In an example, information stored or otherwise provided on the machine readable medium 822 may be representative of the instructions 824, such as instructions 824 themselves or a format from which the instructions 824 may be derived. This format from which the instructions 824 may be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), packaged instructions (e.g., split into multiple packages), or the like. The information representative of the instructions 824 in the machine readable medium 822 may be processed by processing circuitry’ into the instructions to implement any of the operations discussed herein. For example, deriving the instructions 824 from the information (e.g., processing by the processing circuitry’) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e g., dynamically or statically linking), encoding, decoding, encrypting, unencrypting, packaging, unpackaging, or otherwise manipulating the information into the instructions 824.

[0061] In an example, the derivation of the instructions 824 may include assembly, compilation, or interpretation of the information (e.g., by the processing circuitry ) to create the instructions 824 from some intermediate or preprocessed format provided by the machine readable medium 822. The information, when provided in multiple parts, may be combined, unpacked, and modified to create the instructions 824. For example, the information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or several remote servers. The source code packages may be encrypted when in transit over a network and decry pted, uncompressed, assembled (e.g., linked) if necessary, and compiled or interpreted (e.g., into aI i bran-, stand-alone executable etc.) at a local machine, and executed by the local machine.

[0062] The instructions 824 may be further transmitted or received over a communications network 826 using a transmission medium via the network interface device 820 utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), LoRa / LoRaWAN, or satellite communication networks, mobile telephone networks (e.g., cellular networks such as those complying with 3G, 4G LTE / LTE-A, or 5G standards), Plain Old Telephone (POTS) networks, and wireless data networks (e.g.. Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards known as Wi-Fi®, IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, among others. In an example, the network interface device 820 may include one or more physical jacks (e.g., Ethernet, coaxial, or phonejacks) or one or more antennas to connect to the communications netw ork 826. In an example, the netw ork interface device 820 may include a plurality' of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine 800, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software. A transmission medium is a machine readable medium.Additional Notes & Examples

[0063] Example 1 is an apparatus for a spoken language control system, the apparatus comprising: an interface configured to obtain an utterance from a user, the utterance being in a general language; and processing circuitry^ configured to: invoke a generative neural network model on the utterance to transform the utterance into a phrase that conforms to a domain-specific language; and provide the phrase to a control system configured to: acceptphrases of the domain-specific language as input; and control a device based on the input.

[0064] In Example 2, the subject matter of Example 1, wherein the domain-specific language is a proper subset of the general language.

[0065] In Example 3, the subject matter of any of Examples 1-2, wherein the generative neural network model uses a transformer architecture.

[0066] In Example 4, the subject matter of Example 3, wherein the generative neural network model has fewer than thirty million parameters.

[0067] In Example 5, the subject matter of any of Examples 1-4, comprising second processing circuitry is configured to: generate a training data set by using a second phrase that conforms to the domain-specific language as input to a second generative neural network model configured to output a third phrase that is in the general language and exclusive of the domain-specific language; and training the generative neural network model using the training data set.

[0068] In Example 6, the subject matter of Example 5, wherein the second generative neural network model has more than one hundred and twenty million parameters.

[0069] In Example 7, the subject matter of any of Examples 1-6, wherein, to invoke the generative neural network model on the utterance, the processing circuitry is configured to iteratively build the phrase, each iteration producing a token based on a current context, the current context including the utterance and tokens from previous iterations.

[0070] In Example 8, the subject matter of Example 7, wherein the generative neural network model includes a target output in addition to phrase output, the target output providing an indication as to whether the utterance is directed to a subject of the control system.

[0071] In Example 9, the subject matter of Example 8, wherein the processing circuitry is configured to: obtain, from the interface, a second utterance from the user; invoke the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system; and refrain from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

[0072] In Example 10, the subject matter of any of Examples 7-9, wherein the processing circuitry is configured to: compare, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a set of valid next tokens for a next token on a next iteration; and limit output of the generative neural network model for the next iteration to the set of valid next tokens.

[0073] In Example 11. the subject matter of Example 10, wherein the processing circuitry is configured to: compare, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a single valid phrase; cease future iterations; and use the single valid phrase as the phrase.

[0074] In Example 12, the subject matter of any of Examples 1-1 1, wherein the utterance is text created from a speech recognition system to convert a signal from a microphone to the text.

[0075] In Example 13, the subject matter of any of Examples 1-12. wherein the control system includes a semantic interpreter configured to accept valid phrases from the domain-specific language and produce a set of operating parameters.

[0076] In Example 14, the subject matter of Example 13, wherein the set of operating parameters include identification of an interface and parameters that apply to that interface.

[0077] Example 15 is a method for a spoken language control system, the method comprising: obtaining an utterance from a user, the utterance being in a general language; invoking a generative neural network model on the utterance to transform the utterance into a phrase that conforms to a domainspecific language; and providing the phrase to a control system configured to: accept phrases of the domain-specific language as input; and control a device based on the input.

[0078] In Example 16. the subject matter of Example 15, wherein the domain-specific language is a proper subset of the general language.

[0079] In Example 17, the subject matter of any of Examples 15-16, wherein the generative neural network model uses a transformer architecture.

[0080] In Example 18, the subject matter of Example 17, wherein the generative neural network model has fewer than thirty million parameters.

[0081] In Example 19, the subject matter of any of Examples 15-18, comprising: generating a training data set by using a second phrase that conforms to the domain-specific language as input to a second generative neural network model configured to output a third phrase that is in the general language and exclusive of the domain-specific language; and training the generative neural network model using the training data set.

[0082] In Example 20. the subject matter of Example 19, wherein the second generative neural network model has more than one hundred and twenty million parameters.

[0083] In Example 21, the subject matter of any of Examples 15-20, wherein invoking the generative neural network model on the utterance includes iteratively building the phrase, each iteration producing a token based on a current context, the current context including the utterance and tokens from previous iterations.

[0084] In Example 22, the subject matter of Example 21, wherein the generative neural network model includes a target output in addition to phrase output, the target output providing an indication as to whether the utterance is directed to a subject of the control system.

[0085] In Example 23, the subject matter of Example 22, comprising: obtaining a second utterance from the user; invoking the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system; and refraining from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

[0086] In Example 24, the subject matter of any of Examples 21-23, comprising: comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a set of valid next tokens for a next token on a next iteration; and limiting output of the generative neural network model for the next iteration to the set of valid next tokens.

[0087] In Example 25, the subject matter of Example 24, comprising: comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a single valid phrase; ceasing future iterations; and using the single valid phrase as the phrase.

[0088] In Example 26, the subject matter of any of Examples 15-25, wherein the utterance is text created from a speech recognition system to convert a signal from a microphone to the text.

[0089] In Example 27, the subject matter of any of Examples 15-26, wherein the control system includes a semantic interpreter configured to accept valid phrases from the domain-specific language and produce a set of operating parameters.

[0090] In Example 28, the subject matter of Example 27, wherein the set of operating parameters include identification of an interface and parameters that apply to that interface.

[0091] Example 29 is a machine readable media including instructions for a spoken language control system, the instructions, when executed by processing circuitry', cause the processing circuitry to perform operations comprising: obtaining an utterance from a user, the utterance being in a general language; invoking a generative neural network model on the utterance to transform the utterance into a phrase that conforms to a domain-specific language; and providing the phrase to a control system configured to: accept phrases of the domain-specific language as input; and control a device based on the input.

[0092] In Example 30. the subject matter of Example 29, wherein the domain-specific language is a proper subset of the general language.

[0093] In Example 31, the subject matter of any of Examples 29-30, wherein the generative neural network model uses a transformer architecture.

[0094] In Example 32, the subject matter of Example 31, wherein the generative neural network model has fewer than thirty million parameters.

[0095] In Example 33, the subject matter of any of Examples 29-32, wherein the operations comprise: generating a training data set by using a second phrase that conforms to the domain-specific language as input to a second generative neural network model configured to output a third phrase that is in the general language and exclusive of the domain-specific language; and training the generative neural network model using the training data set.

[0096] In Example 34, the subject matter of Example 33, wherein the second generative neural network model has more than one hundred and twenty million parameters.

[0097] In Example 35, the subject matter of any of Examples 29-34, wherein invoking the generative neural network model on the utterance includes iteratively building the phrase, each iteration producing a token based on a current context, the current context including the utterance and tokens from previous iterations.

[0098] In Example 36, the subject matter of Example 35, wherein the generative neural network model includes a target output in addition to phrase output, the target output providing an indication as to whether the utterance is directed to a subject of the control system.

[0099] In Example 37, the subject matter of Example 36, wherein the operations comprise: obtaining a second utterance from the user; invoking the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system; and refraining from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

[0100] In Example 38, the subject matter of any of Examples 35-37, wherein the operations comprise: comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a set of valid next tokens for a next token on a next iteration; and limiting output of the generative neural network model for the next iteration to the set of valid next tokens.

[0101] In Example 39, the subject matter of Example 38, wherein the operations comprise: comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a single valid phrase; ceasing future iterations; and using the single valid phrase as the phrase.

[0102] In Example 40, the subject matter of any of Examples 29-39, wherein the utterance is text created from a speech recognition system to convert a signal from a microphone to the text.

[0103] In Example 41, the subject matter of any of Examples 29-40, wherein the control system includes a semantic interpreter configured to accept valid phrases from the domain-specific language and produce a set of operating parameters.

[0104] In Example 42, the subject matter of Example 41, wherein the set of operating parameters include identification of an interface and parameters that apply to that interface.

[0105] Example 43 is a system for a spoken language control system, the system comprising: means for obtaining an utterance from a user, the utterance being in a general language; means for invoking a generative neural network model on the utterance to transform the utterance into a phrase that conforms to a domain-specific language; and means for providing the phrase to a control system configured to: accept phrases of the domain-specific language as input; and control a device based on the input.

[0106] In Example 44, the subject matter of Example 43, wherein the domain-specific language is a proper subset of the general language.

[0107] In Example 45, the subject matter of any of Examples 43-44, wherein the generative neural network model uses a transformer architecture.

[0108] In Example 46, the subject matter of Example 45, wherein the generative neural network model has fewer than thirty million parameters.

[0109] In Example 47, the subject matter of any of Examples 43-46, comprising: means for generating a training data set by using a second phrase that conforms to the domain-specific language as input to a second generative neural network model configured to output a third phrase that is in the general language and exclusive of the domain-specific language; and means for training the generative neural network model using the training data set.

[0110] In Example 48, the subject matter of Example 47, wherein the second generative neural network model has more than one hundred and twenty million parameters.

[0111] In Example 49, the subject matter of any of Examples 43-48, wherein the means for invoking the generative neural network model on the utterance include means for iteratively building the phrase, each iteration producing a token based on a current context, the current context including the utterance and tokens from previous iterations.

[0112] In Example 50, the subject matter of Example 49, wherein the generative neural network model includes a target output in addition to phrase output, the target output providing an indication as to whether the utterance is directed to a subject of the control system.

[0113] In Example 51, the subject matter of Example 50, comprising: means for obtaining a second utterance from the user; means for invoking the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system; and means for refraining from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

[0114] In Example 52, the subject matter of any of Examples 49-51, comprising: means for comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a set of valid next tokens for a next token on a next iteration; and means for limiting output of the generative neural network model for the next iteration to the set of valid next tokens.

[0115] In Example 53, the subject matter of Example 52, comprising: means for comparing, at an iteration, a cunent state of the phrase with valid phrases in the domain-specific language to determine a single valid phrase; means for ceasing future iterations; and means for using the single valid phrase as the phrase.

[0116] In Example 54, the subject matter of any of Examples 43-53, wherein the utterance is text created from a speech recognition system to convert a signal from a microphone to the text.

[0117] In Example 55, the subject matter of any of Examples 43-54, wherein the control system includes a semantic interpreter configured to accept valid phrases from the domain-specific language and produce a set of operating parameters.

[0118] In Example 56, the subject matter of Example 55, wherein the set of operating parameters include identification of an interface and parameters that apply to that interface.

[0119] Example 57 is at least one machine-readable medium including instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement of any of Examples 1-56.

[0120] Example 58 is an apparatus comprising means to implement of any of Examples 1-56.

[0121] Example 59 is a system to implement of any of Examples 1-56.

[0122] Example 60 is a method to implement of any of Examples 1-56.

[0123] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments that may be practiced. These embodiments are also referred to herein as ‘‘examples.'’ Such examples may include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0124] All publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated reference(s) should be considered supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.

[0125] In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In this document, the term “or” is used to refer to a nonexclusive or. such that “A or B” includes “A but not B.” “B but not A,” and “A and B,” unless otherwise indicated. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and "wherein." Also, in the following claims, the terms “including” and “comprising” are open-ended, that is. a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0126] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspectsthereof) may be used in combination with each other. Other embodiments may be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is to allow the reader to quickly ascertain the nature of the technical disclosure and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may he in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment. The scope of the embodiments should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

CLAIMS1. An apparatus for a spoken language control system, the apparatus comprising: an interface configured to obtain an utterance from a user, the utterance being in a general language; and processing circuitry configured to: invoke a generative neural network model on the utterance to transform the utterance into a phrase that conforms to a domain-specific language; and provide the phrase to a control system configured to: accept phrases of the domain-specific language as input; and control a device based on the input.

2. The apparatus of claim 1, wherein the domain-specific language is a proper subset of the general language.

3. The apparatus of claim 1, comprising second processing circuitry is configured to: generate a training data set by using a second phrase that conforms to the domain-specific language as input to a second generative neural network model configured to output a third phrase that is in the general language and exclusive of the domain-specific language; and training the generative neural network model using the training data set.

4. The apparatus of claim 1, wherein, to invoke the generative neural network model on the utterance, the processing circuitry is configured to iteratively build the phrase, each iteration producing a token based on a current context, the current context including the utterance and tokens from previous iterations.

5. The apparatus of claim 4, wherein the generative neural network model includes a target output in addition to phrase output, the target output providing an indication as to whether the utterance is directed to a subject of the control system.

6. The apparatus of claim 5, wherein the processing circuitry is configured to: obtain, from the interface, a second utterance from the user; invoke the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system; and refrain from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

7. The apparatus of claim 4, wherein the processing circuitry is configured to: compare, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a set of valid next tokens for a next token on a next iteration; and limit output of the generative neural network model for the next iteration to the set of valid next tokens.

8. The apparatus of claim 7, wherein the processing circuitry is configured to: compare, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a single valid phrase; cease future iterations; and use the single valid phrase as the phrase.

9. The apparatus of claim 1 , wherein the control system includes a semantic interpreter configured to accept valid phrases from the domain-specific language and produce a set of operating parameters.

10. The apparatus of claim 9, wherein the set of operating parameters include identification of a second interface and parameters that apply to that second interface.

11. A machine readable media including instructions for a spoken language control system, the instructions, when executed by processing circuitry, cause the processing circuitry7to perform operations comprising: obtaining an utterance from a user, the utterance being in a general language; invoking a generative neural network model on the utterance to transform the utterance into a phrase that conforms to a domain-specific language; and providing the phrase to a control system configured to: accept phrases of the domain-specific language as input; and control a device based on the input.

12. The machine readable media of claim 11, wherein the domainspecific language is a proper subset of the general language.

13. The machine readable media of claim 1 1 , wherein the operations comprise: generating a training data set by using a second phrase that conforms to the domain-specific language as input to a second generative neural network model configured to output a third phrase that is in the general language and exclusive of the domain-specific language; and training the generative neural network model using the training data set.

14. The machine readable media of claim 11, wherein invoking the generative neural network model on the utterance includes iteratively building the phrase, each iteration producing a token based on a current context, the current context including the utterance and tokens from previous iterations.

15. The machine readable media of claim 14, wherein the generative neural network model includes a target output in addition to phrase output, the target output providing an indication as to whether the utterance is directed to a subject of the control system.

16. The machine readable media of claim 15. wherein the operations comprise: obtaining a second utterance from the user; invoking the generative neural network model on the second utterance to produce the target output in a first iteration to determine that the second utterance is not directed to the subject of the control system; and refraining from executing additional iterations of the generative neural network model on the second utterance to cease processing of the second utterance.

17. The machine readable media of claim 14, wherein the operations comprise: comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a set of valid next tokens for a next token on a next iteration; and limiting output of the generative neural netw ork model for the next iteration to the set of valid next tokens.

18. The machine readable media of claim 17, wherein the operations comprise: comparing, at an iteration, a current state of the phrase with valid phrases in the domain-specific language to determine a single valid phrase; ceasing future iterations; and using the single valid phrase as the phrase.

19. The machine readable media of claim 11. wherein the control system includes a semantic interpreter configured to accept valid phrases from the domain-specific language and produce a set of operating parameters.

20. The machine readable media of claim 19, wherein the set of operating parameters include identification of an interface and parameters that apply to that interface.

Images