A combined vehicle control, communication and interactive assistant system employing ai multimodal large language models.

GB2629565APending Publication Date: 2024-11-06DWAIN BARNES
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
GB2023006187
Authority / Receiving Office
GB · GB
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-06

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

An audio plain language conversational driven combined vehicle control, communication and interactive assistant system. The vehicle system includes a computer 1 with a controller with means to run and instruct at least one fine-tuned multimodal large language model (LLM), for example Dolly 2.0. The vehicle system also includes: an audio input device, e.g. microphone 2 or Bluetooth (RTM) speaker; an audio output device, e.g. loudspeaker 3; connections to vehicle systems 4; and a mobile data SIM 5 with 4G or 5G connectivity to the Internet 6. The system is able to output conversational responses to user speech, voice requests and voice control – i.e. it generates natural language responses, for example from an artificial intelligence (AI) model located in the cloud.
Need to check novelty before this filing date? Find Prior Art

Claims

1. An audio plain language conversational driven combined vehicle control, communication and interactive assistant system comprising an integrated computer with controller with means to run and instruct at least one fine-tuned multimodal large language models with said LLM as the controller and numerous expert models as collaborative executors, said system further comprises microphone or audio input device, said system further comprises loudspeaker(s) or audio output device(s) with conversational output said system further comprises mobile data SIM with 4G and / or 5G connectivity to Internet said system also comprises interface(s) to vehicle systems.

2. A system as in claim (1) above except wherein said microphone or audio input device is the existing Bluetooth linked audio input device or microphone in the vehicle and wherein said loudspeaker(s) or audio output device is the existing Bluetooth linked loudspeaker(s) or audio output device (s) and wherein 4 is interface or interfaces with all vehicle systems and Bluetooth.

3. A system as in claims 1 or 2 above wherein said vehicle systems are any or all of such as GPS unit, vehicle ECU, vehicle infotainment system, and any or all of vehicle's sensors.

4. A system as in any of claims 1-3 herein above wherein the said fine -tuned model is Dolly V2.

5. System as in any of claims 1-4 above wherein said controller running LLM's is running on computer integral to vehicle.

6. System as in any of claims 1-4 above wherein said controller harnesses the power of the cloud computing base and is externally linked to low power computer in the vehicle.

Citation Information

Patent Citations

  • Chat type multi-round dialogue method and system based on semantic gate, medium and equipment

    CN114048296A

  • Vehicle personal assistant

    US20140136013A1

  • Chatbot Integrating Derived User Intent

    US20190188590A1

  • Two-way in-vehicle virtual personal assistant

    US20200211553A1

  • A voice assistant system for a vehicle cockpit system

    US20210358496A1