LLM based universal integration system

Abstract

A method includes the use of a source mapper that is configured to operate in a source vehicle to enable training of a target mapper that is configured to operate in a target vehicle. The source mapper maps an utterance into a source command that controls a source-vehicle device in the source vehicle and the target mapper maps the same utterance into a target command that controls a target-vehicle device in the target vehicle. The source command and target command are commands from a source command-set and a target-command set, respectively, with the target-command set differing from the source-command set.

Description

LLM BASED UNIVERSAL INTEGRATION SYSTEMCROSS-REFERENCE TO RELATED APPLICATIONS[00011 This application claims the benefit of U.S. provisional application Serial No. 63 / 663,768 filed June 25, 2024, the disclosure of which is hereby incorporated in its entirety by reference herein.TECHNICAL FIELD

[0002] Aspects of the disclosure relate to LLM based universal integration systems.BACKGROUND

[0003] A typical vehicle has numerous subsystems that a user may wish to control. Examples include a climate-control subsystem, an infotainment subsystem, and a navigation subsystem.

[0004] In the early days of automotive engineering, a user could press buttons or turn knobs to control these various subsystems. As vehicles evolved, it became usual for these subsystems to be computer-controlled. A typical subsystem would thus present an application-program interface to a vehicle’s control computer. This computer could then control the subsystem by issuing appropriate commands using the application-program interface. These commands typically took the form of API function calls.

[0005] The actual function calls are the product of a programmer who designed the subsystem. These function calls usually take the form of the name of the function and the logical names of arguments to that function. However, since these names are up to a programmer, the software that makes these calls must somehow know what they are.

[0006] As the industry continues to evolve, the population of these function calls changes over time. This means that the vehicle’s control computer must constantly be kept up-to-date. Inaddition, different vehicle makes and models potentially require different function calls. This means that what works in one vehicle may not work in another.SUMMARY100071 In one aspect, the invention features a method that includes using a source mapper that is configured to operate in a source vehicle to enable training of a target mapper that is configured to operate in a target vehicle. The source mapper an utterance into a source command that controls a source-vehicle device in the source vehicle and the mapper maps the utterance into a target command that controls a target-vehicle device in the target vehicle. The source and target commands are from a source command-set and a target command-set, respectively, the sourcecommand set being different from the target-command set.

[0008] Practices of the invention include those in which using the source mapper to enable training of the target mapper includes using the source mapper to train a large language model to associate utterances with source commands from the source command-set, those in which using the source mapper to enable training of the target mapper includes using a large language model to associate utterances with source commands from the source command-set, those in which using the source mapper to enable training of the target mapper includes training a large language model that has been trained associate utterances with source commands from the source command-set to associate the utterances with target commands from the target-command set instead of the sourcecommand set, those in which using the source mapper to enable training of the target mapper includes using a large language model to associate the utterances with target commands from the target-command set instead of the source-command set, and those in which using the source mapper to enable training of the target mapper includes using information provided by the source mapper for generating training data and using the training data to train the target mapper to associate utterances with target commands from the target-command set.

[0009] Further practices include those in which using the source mapper to enable training of the target mapper includes using the source mapper to train a large language model to associateutterances with source commands from the source command-set, training the large language model to associate utterances with target commands from the target-command set instead of with source commands form the source command-set, using the large language model, which has been trained to associated utterances with target commands, to generate training data, and using the training data to train the target mapper.

[0010] Still other practices include those in which using the source mapper to enable training of the target mapper includes using a large language model to associate utterances with source commands from the source command-set, using the large language model to associate utterances with target commands from the target-command set instead of with source commands form the source command-set, and using the large language model to generate training data, and using the training data to train the target mapper.[00111 In some practices, the source command is a source API call and the target command is a target API call.

[0012] Among the practices are those in which the source mapper is configured to provide a first argument, the target mapper is configured to provide a second argument, the first argument is an argument for the source command, and the target mapper is configured to provide an argument for a target command from the target-command set.

[0013] In other practices, the source vehicle includes a source-vehicle bus and the target vehicle includes a target-vehicle bus. In such practices, the source command is a constituent of a packet that is addressed to the source device and that reaches the source device via the source-vehicle bus and the target command is a constituent of a packet that is addressed to the target device and that reaches the target device via the target-vehicle bus.

[0014] In still other practices, the source command-set includes a source software- library of source functions and the target command-set includes a target software-library of target functions. In such practices, he source command includes a call to a source function from the source softwarelibrary and the target command includes a call to a target function from the target software-library.

[0015] In still other practices, the source mapper includes an integration layer that receives a semantic representation of the utterance and maps the semantic representation to the source command.

[0016] Also among the practices of the invention are those in which the source mapper includes a semantic layer that provides a semantic representation that correspond to the utterance.|0017| In still other practices, using the source mapper to enable training of a target mapper includes using a prompt augmenter to train a model to map utterances to corresponding commands from the source-command set source, the model being a large language model. In such practices, using the prompt augmenter includes augmenting a prompt provided to the model with the sourcecommand set.10018] Other practices include those in which using the source mapper to enable training of a target mapper includes training a model to map the utterances to target commands from the targetcommand set using a prompt augmenter to augment prompts provided to the model with the targetcommand set. In such practices, it is usual for the model to be a large language model.

[0019] In some practices, using the source mapper to enable training of the target mapper includes using information from the source mapper to train a model to map utterances to corresponding source commands from the source-command set includes using a prompt augmenter to augment prompts provided to the model with the source- command set and with semantic representations and further training the model to map the utterances to corresponding target commands from the target-command set by using the prompt augmenter to augment prompts provided to the model with the semantic representations and the target-command set.

[0020] Some practices of the method also include using information from the source mapper to train a model to associate the utterance with a corresponding command from the target command set, using the model to generate training data, and using the training data to construct a target integration-layer that is configured to map a semantic representation to a target command from the target-command set. In such practices, the target integration-layer is configured for operation within a constraint imposed by the target vehicle.

[0021] Still other practices include generating a training set for use in training the target mapper. In such practices, generating the training set includes providing augmented prompts to a model to cause the model to generate the training data. These augmented prompts are augmented with the target-command set.BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG. 1 illustrates a block diagram of an example LLM based universal integration system having a source vehicle and a target vehicle;

[0023] FIG. 2 illustrates a block diagram of a source mapper in the source vehicle of FIG. 1;

[0024] FIG. 3 illustrates a block diagram of an example process for generating a generalized mapper using the source commands;

[0025] FIG. 4 illustrates a block diagram of an example process for creating training data for the target commands based on the generalized mapper; and

[0026] FIG. 5 illustrates a block diagram of an example system using the training data of FIG. 4 to build an integration layer for the target mapper in the target vehicle.DETAILED DESCRIPTION

[0027] As required, detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis forteaching one skilled in the art to variously employ the present invention.

[0001] FIG. 1 illustrates a block diagram of an example LLM based universal integration system 2 having a source vehicle 10 and a target vehicle 24. The vehicles 10, 24 may includevarious types of passenger vehicles, such as crossover utility vehicle (CUV), sport utility vehicle (SUV), truck, recreational vehicle (RV), boat, plane or other mobile machine for transporting people or goods. Further, the vehicle 104 may be autonomous, partially autonomous, self-driving, driverless, or driver-assisted vehicles. The vehicle 104 may be an electric vehicle (EV), such as a battery electric vehicle (BEV), plug-in hybrid electric vehicle (PHEV), hybrid electric vehicle (HE Vs), etc.

[0002] The vehicles 10, 24 may be configured to include various types of components, processors, and memory, and may communicate with a communication network. The communication network may be referred to as a “cloud” and may involve data transfer via wide area and / or local area networks, such as the Internet, Global Positioning System (GPS), cellular networks, Wi-Fi, Bluetooth, etc. The communication network 110 may provide for communication between the vehicles 10, 24 and an external or remote server and / or database, as well as other external applications, systems, vehicles, etc. This communication network may provide navigation, music or other audio, program content, marketing content, internet access, speech recognition, cognitive computing, artificial intelligence, to the vehicles 10, 24.

[0003] Each vehicle 10, 24 may include a processor 15 and may perform functions and methods described herein. The system may also include speech interfaces, which includes a speech recognition system and natural language understanding system, each to identify words or phrases and aid in interpretating an utterance. Artificial intelligence may be used to continually refine and replicate certain scenarios and processes herein. The remote server and the database may include one or more computer hardware processors coupled to one or more computer storage devices for performing steps of one or more methods as described herein and may enable the vehicles 10, 24 to communicate and exchange information and data with systems and subsystems external to the vehicles 10, 24 and local to or onboard the vehicles 10, 24. The vehicles 10, 24 may include one or more additional processors configured to perform certain instructions, commands and other routines as described herein. Internal vehicle networks may also be included, such as a vehicle controller area network (CAN), an Ethernet network, and a media oriented system transfer (MOST), etc. The internal vehicle networks may allow the processor to communicate with other vehicle systems, such as a vehicle modem, a GPS module and / or Global System for MobileCommunication (GSM) module configured to provide current vehicle location and heading information, and various vehicle electronic control units (ECUs) configured to corporate with the processor 106.

[0004] The processor 15 may execute instructions for certain vehicle applications, including navigation, infotainment, climate control, etc. Instructions for the respective vehicle systems may be maintained in a non-volatile manner using a variety of types of computer-readable storage medium. The computer-readable storage medium (also referred to herein as memory, or storage) includes any non-transitory medium (e.g., a tangible medium) that participates in providing instructions or other data that may be read by the processor 15. Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and / or technologies, including, without limitation, and either alone or in combination, Java, C, C++, C#, Objective C, Fortran, Pascal, Java Script, Python, Perl, and PL / structured query language (SQL).

[0005] Certain examples of these processes may include navigation system outputs (e.g., time sensitive directions for a driver), incoming text messages converted to output speech, vehicle status outputs, and the like, e.g., output from a local or onboard storage medium or system. In some examples, the processing system provides input / output control functions with respect to one or more electronic devices, such as a heads-up-display (HUD), vehicle display, and / or mobile device of the driver or passenger, sensors, cameras, etc.

[0006] The vehicles 10, 24 may include a wireless transceiver, such as a BLUETOOTH module, a ZIGBEE transceiver, a Wi-Fi transceiver, an IrDA transceiver, a radio frequency identification (RFID) transceiver, etc.) configured to communicate with compatible wireless transceivers of various user devices, as well as with the communication network 110.[0007)

[0008] The vehicles 10, 24 may include various sensors and input devices. For example, the vehicles 10, 24 may include at least one microphone 17. The microphone 17 may be configured receive audio signals from within the vehicle cabin, such as acoustic utterances including spokenwords, phrases, or commands from a user. The microphone 17 may also include an audio input configured to provide audio signal processing features, including amplification, conversions, data processing, etc., to the processor 15.|0009] The microphone 17 may be used for other vehicle features such as active noise cancelation, hands-free interfaces, etc. The microphone 17 may facilitate speech recognition from audio received via the microphone 17 according to grammar associated with available commands, and voice prompt generation. The at least one microphone 17 may include a plurality of microphones 17 arranged throughout the vehicle cabin. The microphone 17 may be configured to receive audio signals from the vehicle cabin. These audio signals may include occupant utterances, sounds, singing, percussion noises, etc.

[0010] The vehicles 10, 24 may include an audio system having audio playback functionality through vehicle loudspeakers or headphones. The audio playback may include audio from sources such as a vehicle radio, including satellite radio, decoded amplitude modulated (AM) or frequency modulated (FM) radio signals, and audio signals from compact disc (CD) or digital versatile disk (DVD) audio playback, streamed audio from a mobile device, commands from a navigation system, etc.

[0011] The vehicles 10, 24 may include various displays and user interfaces, including HUDs, center console displays, steering wheel buttons, etc. Touch screens may be configured to receive user inputs. Visual displays may be configured to provide visual outputs to the user. In one example, the display may provide lyrics or other to the vehicle occupant.

[0012] The source vehicle 10 may have a source mapper 12 that is configured to receive an utterance 14 from an occupant 16. The utterance 14 may be received via the microphone 17. The source mapper 12 may receive the utterance 14 and infer the occupant’s intent. If appropriate, the source mapper 12 selects an appropriate command 18 (i.e., a “source command”) from a sourcecommand set 20. This selected command 18 is then provided to whichever device 22 the occupant 16 intends to control. The source mapper 12 may be integrated within a memory in communication with the processor 15.

[0013] FIG. 1 also shows a target vehicle 24 having a target mapper 26 that functions much like the source mapper 12. The target mapper 26 receives an utterance 14 from an occupant 16. Like the source mapper 12, the target mapper 26 infers the occupant’s intent and uses it to select an appropriate command 18. However, in this case, the command 18 (i.e., a “target command”) is selected from a target-command set 28. The target command set 28 may be stored in a memory. This target- command set 28 differs from the source-command set 20. This selected target command 18 is again provided to whichever device 22 that the occupant 16 wishes to control.

[0014] Traditionally, because the source-command set 20 and the target-command set 28 differ, the same intent will quite likely result in the source mapper 12 and the target mapper 26 providing different commands 18. This means that one cannot just use the source mapper 12 in the target vehicle 24. Instead, the usual practice is to create a bespoke target mapper 26 to match the targetcommand set 28.

[0015] The method described herein provides a way to leverage an existing a source mapper 12 to ease the burden of having to create a bespoke target mapper 26.

[0016] FIG. 2 illustrates a block diagram of the source mapper 12 in the source vehicle 10 of FIG. 1. , The source mapper 12 may include a semantic- representation set 30. Within this semantic-representation set 30 there exist numerous semantic representations 32. Each such semantic representation 32 communicates an intent that corresponds to that expressed by one or more utterances 14.

[0017] In general, there will be many ways for an occupant 16 to express an intent to control a device 22. To address this difficulty, the source mapper 12 also includes a semantic layer 34. The semantic layer 34 receives an utterance 14 from the occupant 16. It then determines which sematic representation 32 in the semantic-representation set 30 corresponds to that utterance 14.

[0018] The semantic layer 34 does not know the command 18 that would correspond to a particular semantic representation 32. This knowledge resides in an integration layer36. Accordingly, the semantic layer 34 provides its selection of a semantic representation 32 to the integration layer 36.

[0019] Upon receiving the semantic representation 32, the integration layer 36 selects a corresponding command 18 from the source-command set 20 and provides that command 18 with any relevant values or parameters that the command 18 may require for execution. It then communicates that selected command 18 to whichever device 22 the occupant 16 intends to control.

[0020] In some cases, it is desirable to use the source mapper 12 as a basis for creating the target mapper 26. One way to do so is to use a training set that pairs each semantic representation 32 in the semantic-representation set 30 with a corresponding command 18 from the target-command set 28. The matched pairs that result can then be provided to any of several known machinelearning techniques to implement a target mapper 26. One such mapping arises from a decision tree, the details of which emerge during training with a training set. Since the mapping to be used in a vehicle 10, 24 it is preferable to provide one that, during operation, makes frugal use of the vehicle’s computational resources.

[0021] Because the source-command set 20 would already have existed for some time, there would already exist a considerable body of training data for it. This training data comprises a set of triplets that define associations between utterances 14, semantic representations 32 thereof, and those commands from the source-command set 20 that correspond to those semantic representations 32. The target-command set 28, however, would most likely be newer.

[0022] Instead of following the direct path of developing new training data, the system may leverage the existing integration layer 36 to avoid the need to develop new training data for the target-command set 28.[0023 The process of leveraging the existing source mapper 12 of FIG. 2 to create a suitable target mapper 26 arises from observation of a synergistic coincidence, one related to the semantic representations 32 and the other related to the commands 18 in both the source-command set 20 and the target-command set 28.

[0024] The first observation is that a semantic representation 32 contains enough features of a natural language to make it possible for a human being to read them and come away with a fairly good idea of what each one is intended to communicate.

[0025] The second observation is that commands 18 and their surrounding infrastructure, such as documentation, are rife with features of a natural language. For example, in many cases, the commands 18 themselves and even their arguments are named in a way that suggests their purpose. Thus, the commands 18 in both source- command set 20 and the target-command set 28, while not exactly being expressed in the form of a natural language, also have enough features of a natural language so that one who inspects them can have a fairly good idea of what they are intended to do.

[0026] A large language model (LLM) (hereafter referred to as a “model” for brevity) typically operates with natural language. But it turns out that both semantic representations 32 and commands 18 are close enough to being natural languages that a model can still create associations between them in much the same way that it does with natural language expressions.

[0027] Referring now to FIG. 3, the first step in leveraging an existing integration layer 36 is that of creating a generalized mapper 40. The generalized mapper 40 essentially duplicates the function of the source mapper 12 but using different underlying machinery, i.e., a model 38 instead of the architecture shown in FIG. 2. As described below, this will ultimately allow it to be generalized to other command sets, and in particular, the target-command set 28.

[0028] The process of creating the generalized mapper 40 relies on a prompt augmenter 42. The prompt augmenter 42 augments prompts derived from utterances 14 with the source-command set 20 and the semantic-representation set 30. Once trained in this manner, the generalized mapper 40 is able to reliably map an utterance 14 to an appropriate command 18 from the source-command set 20.

[0029] An optional part of this first step is to fine tune the generalized mapper 40. This is carried out with a small training set that consists of utterances 14, semantic representations 32 that correspond to the utterances 14, and corresponding commands 18 from the target-command set 28. This step also relies on the prompt augmenter 42, which augments the prompts using the targetcommand set 28 instead of the source-command set 20. In some embodiments, this is achieved by fine tuning the generalized mapper 40 for the target command set 28.

[0030] Upon completion of the first step, it becomes possible to carry out a second step that includes synthesizing a training set 44 for the target-command set 28. Referring now to FIG. 4, this is carried out in a manner similar to the first step, except that the prompt augmenter 42 now augments the prompt with the target-command set 28 instead of the source-command set 20. The utterances 14 and the semantic representations 32 from the first step are reused for this second step.

[0031] The result of this second step is a generalized mapper 40 that can receive an utterance 14 and reliably output the correct command 18 from the target-command set 28. In principle, one could use this generalized mapper 40 directly in the target vehicle 24. However, because of hardware constraints in a typical target vehicle 24, it is more useful to use the generalized mapper 40 to create training data 44 that can then be used to build a lightweight integration layer 36 for use in the target mapper 26.

[0032] The third and final step is to therefore to use the training data 44 and a trainer 46 to build a lightweight integration layer 36 that maps a semantic representation 32 to a command 18 and that can be used within the constraints imposed by the target vehicle 24. As noted above, a trainer 46 can use any one of several machine learning systems provided a suitably large training set exists. And now, thanks to the strategic use of prompt augmentation in the preceding steps and the availability of the source mapper 12, such a training set 44 now exists.

[0033] As shown in FIG. 4 and FIG. 5, in some examples, a lightweight integration layer 36 suitable for use in a target vehicle 24 with limited computational resources includes a decision tree that has been built using pairs formed by semantic representations 32 and their corresponding commands. In other embodiments, the lightweight integration layer 36 is one built by using an utterance 14 as an input in conjunction with a corresponding semantic representation 32. The resulting lightweight integration layer 36 can then be combined with the same semantic layer 34 as the source mapper 12 to form the target mapper 26.|0034| In the foregoing discussion, the commands 18 include API calls in a form in which they are typically provided in a software library. In those vehicles with many devices 22 to be controlled, it is common to have a vehicle bus that carries packets addressed to the various devices22. Such a bus is often called a “controller-area-network bus.” In such cases, it is possible for a command 18 to correspond to a packet that is addressed to a particular device 22 and that carries a command 18 as its payload. It should therefore be apparent that the foregoing method is agnostic to the nature of the command 18 itself and is applicable to commands that take the form of a higher- level call to a function in a software library and also to a lower-level bus message.

[0035] While an automotive system is discussed in detail here, other applications may be appreciated. For example, similar functionally may also be applied to other, non-automotive cases, e.g., for augmented reality or virtual reality cases with smart glasses, phones, eye trackers in living environment, etc. While the terms “user” is used throughout, this term may be interchangeable with others such as speaker, occupant, etc.

[0036] While examples are described herein, other vehicle systems may be included and contemplated. Although not specifically shown, the vehicle may include on-board automotive processing units that may include an infotainment system that includes a head unit and a processor and a memory. The infotainment system may interface with a peripheral-device set that includes one or more peripheral devices, such as microphones, loudspeakers, the haptic elements, cabin lights, cameras, the projector and pointer, etc. The head unit may execute various applications such as a speech interface and other entertainment applications. Other processing include text to speech, a recognition module, etc. These systems and modules may respond to user commands and requests.10037] Computing devices described herein generally include computer-executable instructions, where the instructions may be executable by one or more computing devices such as those listed above. Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and / or technologies, including, without limitation, and either alone or in combination, Java™, C, C++, C#, Visual Basic, Java Script, Perl, etc. In general, a processor (e.g., a microprocessor) receives instructions, e.g., from a memory, a computer-readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data may be stored and transmitted using a variety of computer-readable media.

[0038] While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.

Claims

WHAT IS CLAIMED IS:

1. A method, comprising: receiving a user utterance at a source vehicle; mapping the utterance to a source command that controls a source-vehicle device in the source vehicle; training a target mapper at a target vehicle based on the mapping of the utterance to the source command; and mapping the utterance to a target command that controls a target-vehicle device in the target vehicle based on the target mapper, wherein the source command is command from a source command-set, wherein the target command is a command from a target command-set, and wherein the source command-set differs from the target-command-set.

2. The method of claim 1, wherein the training of the target mapper includes using the source mapper to train a large language model to associate utterances with source commands from the source command- set.

3. The method of claim 1, wherein the training of the target mapper includes using a large language model to associate utterances with source commands from the source command-set.

4. The method of claim 1, wherein the training of the target mapper includes training a large language model associates utterances with source commands from the source command-set to associate the utterances with target commands from the target-command set.

5. The method of claim 1, wherein the training of the target mapper includes using a large language model to associate the utterances with target commands from the targetcommand set.

6. The method of claim 1, wherein the training the target mapper includes using information provided by the source mapper for generating training data and using the training data to train the target mapper to associate utterances with target commands from the target command-set.

7. The method of claim 1, wherein the training of the target mapper includes using the source mapper to train a large language model to associate utterances with source commands from the source command- set, training the large language model to associate utterances with target commands from the target-command set instead of with source commands form the source command-set, and using the large language model, which has been trained to associated utterances with target commands, to generate training data, and using the training data to train the target mapper.

8. The method of claim 1, wherein the training of the target mapper includes using a large language model to associate utterances with source commands from the source command-set, using the large language model to associate utterances with target commands from the target-command set instead of with source commands form the source command-set, and using the large language model to generate training data, and using the training data to train the target mapper.

9. The method of claim 1, wherein the source command is a source API call and the target command is a target API call.

10. The method of claim 1, wherein the source mapper is configured to provide a first argument, wherein the target mapper is configured to provide a second argument, wherein the first argument is an argument for the source command, and wherein the target mapper is configured to provide an argument for a target command the target-command set.

11. The method of claim 1 , wherein the source vehicle includes a source-vehicle bus, wherein the target vehicle comprises a target-vehicle bus, wherein the source command is a constituent of a packet that is addressed to the source device and that reaches the source device viathe source-vehicle bus, and wherein the target command is a constituent of a packet that is addressed to the target device and that reaches the target device via the target- vehicle bus.

12. The method of claim 1, wherein the source command-set includes a source software-library of source functions, wherein the target command-set includes a target softwarelibrary of target functions, wherein the source command includes a call to a source function from the source software-library, and wherein the target command comprises a call to a target function from the target software-library.

13. The method of claim 1, wherein the source mapper includes an integration layer that receives a semantic representation of the utterance and maps the semantic representation to the source command.

14. The method of claim 1, wherein the source mapper includes a semantic layer that provides a semantic representation that correspond to the utterance.

15. The method of claim 1, wherein the training of the target mapper includes using a prompt augmenter to train a model to map utterances to corresponding commands from the source-command set source, the model being a large language model, wherein using the prompt augmenter includes augmenting a prompt provided to the model with the source- command set.

16. The method of claim 1, wherein the training of the target mapper includes training a model to map the utterances to target commands from the target-command set using a prompt augmenter to augment prompts provided to the model with the target-command set, the model being a large language model.

17. The method of claim 1, wherein the training of the target mapper includes using information from the source mapper to train a model to map utterances to corresponding source commands from the source- command set comprises using a prompt augmenter to augment prompts provided to the model with the source-command set and with semantic representations and further training the model to map the utterances to corresponding target commands from thetarget-command set by using the prompt augmenter to augment prompts provided to the model with the semantic representations and the target-command set.

18. The method of claim 1, further comprising using information from the source mapper to train a model to associate the utterance with a corresponding command from the target command set, using the model to generate training data, and using the training data to construct a target integration-layer that is configured to map a semantic representation to a target command from the target-command set, wherein the target integration-layer is configured for operation within a constraint imposed by the target vehicle.

19. The method of claim 1, further comprising generating a training set for use in training the target mapper, wherein generating the training set comprises providing augmented prompts to a model to cause the model to generate the training data, wherein the augmented prompts are augmented with the target- command set.

Images