System and method for intelligently defining scene of large model

Abstract

The invention discloses a large model intelligent definition scene system and method, and belongs to the technical field of Internet of Vehicles, and the system comprises a voice receiving system ASR, a scene card intelligent definition large model, a scene card definition execution module and large model training. A new scene card json is automatically generated and an existing scene card is recognized for modification and updating, a large model intelligently recognizes a newly defined user scene, a scene card definition execution module converts a text json into a script executed by a vehicle, scene card display and voice confirmation are performed on a vehicle display screen, and the scene card is created and recorded after confirmation. Performing large model training after recording; the scene intelligent definition method based on the large model is constructed, and personalized scene services needed by the user can be intelligently defined through historical voice communication of the user.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Vehicles, and specifically relates to a large-model intelligent scene definition system and method. Background Art

[0002] Currently, car owners passively receive scenario-based services and find it difficult to participate in scenario definition. They are not clear about the car’s data and services. Large-model intelligent services are needed to actively and intelligently define scenarios to meet users’ growing demand for personalized scenarios.

[0003] With the improvement of Internet of Vehicles capabilities and vehicle intelligence, users have higher requirements for vehicle intelligence. Users hope to have personalized scenario services. The scenario services defined by car manufacturers cannot meet the individual needs of users, and scene customization functions are required. The current customization functions have simple content, single services, and complex operations. More importantly, the intelligent capabilities of scene definition are insufficient. Summary of the invention

[0004] In view of the above problems, the present invention provides a large-model intelligent scene definition system and method, including a voice receiving system ASR, a scene card intelligent definition large model, a scene card definition execution module and large model training; the scene card intelligent definition large model includes semantic understanding, context judgment, and instruction intention recognition; the scene card definition execution module includes scene json parsing, protocol commands and vehicle scene execution scripts; the large model training includes training data input, SFT fine-tuning, and large model deployment and online; in voice communication, the scene card intelligent definition large model actively identifies changes in user application scenarios, automatically generates new scene card json and identifies existing scene cards for modification and update, the large model intelligently identifies newly defined user scenarios, the scene card definition execution module converts the text json into a script executed by the vehicle, displays the scene card on the vehicle display screen and performs voice confirmation, creates and records the scene card after confirmation, and then trains the large model after recording; the present invention constructs a scene intelligent definition method based on a large model, which can intelligently define the personalized scene services required by the user through the user's historical voice communication.

[0005] The technical solution of the present invention is as follows: a large model intelligent scene definition system, comprising:

[0006] The speech receiving system ASR (Automatic Speech Recognition), the scene card intelligently defines the big model, the scene card defines the execution module and the big model training; the scene card intelligently defines the big model including semantic understanding, context judgment, and command intention recognition; the scene card defines the execution module including scene json (JavaScript Object Notation, a lightweight data exchange format) parsing, protocol commands and vehicle scene execution scripts; the big model training includes training data input, SFT (Supervised Fine-Tun-ing) fine-tuning, and big model deployment and launch.

[0007] Furthermore, the voice receiving system ASR can obtain voice data ASR, parse and reconstruct the ASR to generate a large model prompt ("prompt word" to prompt the AI ​​model with the context of input information and parameter information of the input model).

[0008] Furthermore, semantic understanding, when processing data, not only learns the basic structure of the language, but also learns how to correctly interpret and apply these language elements according to the context;

[0009] Contextual judgment can improve the model's ability to understand text and maintain coherence and understanding when dealing with complex contexts;

[0010] Command intent recognition can identify vehicle scene commands, confirm whether it is a new scene or a change to an existing scene, and then generate a json for scene creation / modification.

[0011] Furthermore, the scene json parsing can parse the scene json and construct a scene card, including the scene name, service name, and service command;

[0012] Protocol commands can construct service commands that the vehicle executes;

[0013] The vehicle scenario execution script includes HMI (abbreviation of Human Machine Interface, HMI is the medium for interaction and information exchange between the system and the user, which realizes the conversion between the internal form of information and the form acceptable to humans) display text, voice TTS (abbreviation of Text To Speech, i.e. "from text to speech") and vehicle execution service commands.

[0014] Furthermore, the training data input includes: LLM (abbreviation for "Large Language Model") output json, historical instruction accumulation, instruction definition set, question-answer pairs and scenario instructions.

[0015] Furthermore, SFT fine-tuning is specifically to create a new neural network model that copies all model designs and parameters of the pre-trained model except the output layer. Because the output layer is closely related to the label of the source dataset, an output layer is added to the target model, whose size is the number of categories in the target dataset, and the parameters of this layer are randomly initialized. When training on the target dataset, the output layer is trained from scratch, and the parameters of the remaining layers are adjusted based on the parameters of the source model.

[0016] A large model intelligent scene definition method, specifically:

[0017] During voice communication, the scene card intelligently defines the big model to actively identify changes in user application scenarios, automatically generate new scene card json, and identify existing scene cards for modification and update. The big model intelligently identifies newly defined user scenarios. The scene card definition execution module converts the text json into a script executed by the vehicle, displays the scene card on the vehicle display screen and performs voice confirmation. After confirmation, the scene card is created and recorded, and then trained with the big model.

[0018] The beneficial effects of the present invention are:

[0019] The present invention constructs a scene intelligent definition method based on a large model, which can intelligently define the personalized scene services required by the user through the user's historical voice communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention provides intelligent definition of scene cards and large model training intentions. DETAILED DESCRIPTION

[0021] It should be noted that in the description of the present invention, the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", "clockwise", "counterclockwise", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction.

[0022] In the present invention, unless otherwise clearly specified and limited, the terms "set", "install", "connect", "connect" and the like should be understood in a broad sense. For example, "fixed" can be a fixed connection, a detachable connection, or an integral connection; the connection can be a mechanical connection or an electrical connection; the connection can be a direct connection or an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0023] During the voice communication between the car owner and the user, the scene card intelligent definition big model actively identifies the changes in the user's application scene (new definition and scene modification), automatically generates a new scene card json and identifies the existing scene card for modification and update. For example: the user says "I'm cold", "Please turn on the air conditioner for me", "Adjust the temperature to 24 degrees", and the big model intelligently identifies the newly defined user scene. The scene card definition execution module converts the text json into a script executed by the vehicle (including scene display HMI, vehicle control instructions, music and other services), displays the scene card on the vehicle display screen and performs voice confirmation, for example: "Owner, to provide you with convenient scene services, when you say "I'm cold" in the future, I will actively turn on the air conditioner for you and adjust the air conditioner temperature to 24°. Do you want to create this smart scene card for you?" After the user agrees, the scene card is created and recorded.

[0024] The large model is trained with SFT through the large car model base model. The training data input includes: LLM output json, historical instruction accumulation, instruction definition set, question and answer pairs, scenario instructions, etc.

[0025] Data input:

[0026] Car owner voice data

[0027] Get the ASR voice data of the car owner, parse and reconstruct the ASR to generate a large model prompt

[0028] Scene card intelligent definition of large model:

[0029] Semantic Understanding

[0030] When processing data, the big model not only learns the basic structure of the language, but also learns how to correctly interpret and apply these language elements according to the context and context. This means that the model can understand that the same word or phrase may have different meanings in different situations, so as to process natural language more accurately, that is, it can understand the language semantics of the car owner.

[0031] Context judgment

[0032] Context plays a vital role, not only improving the model's ability to understand text, but also maintaining coherence and understanding when dealing with complex contexts. However, optimizing the use of context and effectively using technologies such as vector databases to improve the ability of large models to process inputs can also make scene judgments based on the content of the owner's user context.

[0033] Command intent recognition

[0034] Identify the vehicle scene command to confirm whether it is a new scene or a change to an existing scene, and then generate the scene creation / modification json.

[0035] Scenario card definition execution module

[0036] Scene json parsing

[0037] Parse the scene json and build the scene card, including the scene name, service name, service command, etc.

[0038] Protocol Commands

[0039] Construct service commands executed by the vehicle through vehicle protocol commands.

[0040] Vehicle scene execution script

[0041] Generate scripts that can be executed by the vehicle, including HMI display text, voice TTS, and vehicle execution service commands, etc.

[0042] Scenario card intelligent definition of large model training:

[0043] SFT Data Preparation

[0044] The training data input includes: LLM output json, historical instruction accumulation, instruction definition set, question and answer pairs, scenario instructions, etc., with no less than 10,000 sample data.

[0045] SFT fine-tuning

[0046] Based on the pre-trained model, the model is fine-tuned using labeled data to adapt it to a specific task or domain. During the SFT process, a new neural network model (target model) is created that copies all the model designs and parameters of the pre-trained model (source model) except the output layer. The output layer is closely related to the labels of the source dataset. An output layer is added to the target model, with a size equal to the number of categories in the target dataset, and the parameters of the layer are randomly initialized. When training on the target dataset, the output layer is trained from scratch, while the parameters of the remaining layers are fine-tuned based on the parameters of the source model.

[0047] Large model deployment online

[0048] The newly generated large model is deployed online after testing and verification

[0049] After starting the ignition, the driver communicates by voice, such as: "Help me open the window", "Help me adjust the seat", "Help me call the radio".

[0050] The scene card intelligent definition model recognizes user intent through context and determines: This is a personalized scene before the user drives. Analyze whether there is a historical scene. If not, create a new "welcome scene" and encapsulate the scene card for the user's instruction service after getting on the car to generate a new welcome scene json. If it already exists, modify the scene based on the user's recent communication.

[0051] The scene card definition execution module parses the scene json and translates the corresponding vehicle control, ecological service and other related protocols according to the vehicle model to generate a scene card executable by the vehicle, including HMI display, voice TTS, service timing, service instructions, etc.

[0052] The scene card is displayed on the vehicle screen, and voice communication is carried out, for example: "Master, we provide you with a convenient welcome scene service. When you get in the car in the future, I will take the initiative to open the window, adjust the seat, and turn on the radio for you. Do you want to create this welcome scene card for you?

[0053] After the user confirms, we will create a welcome scene card and save it.

[0054] The above is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent replacement and improvement made by any person skilled in the art within the technical scope disclosed by the present invention and within the spirit and principle of the present invention shall be covered within the protection scope of the present invention. At the same time, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0055] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

Claims

1. A large-model intelligent scene definition system, characterized in that: include: Voice receiving system ASR, scene card intelligent definition big model, scene card definition execution module and big model training; scene card intelligent definition big model includes semantic understanding, context judgment, and command intention recognition; scene card definition execution module includes scene json parsing, protocol commands and vehicle scene execution scripts; Large model training includes training data input, SFT fine-tuning, and large model deployment.

2. A large-model intelligent scene definition system according to claim 1, characterized in that: The voice receiving system ASR can obtain voice data ASR, parse and reconstruct the ASR to generate a large model prompt.

3. A large-model intelligent scene definition system according to claim 1, characterized in that: Semantic understanding, when processing data, not only learns the basic structure of language, but also learns how to correctly interpret and apply these language elements based on context and language. Contextual judgment can improve the model's ability to understand text and maintain coherence and understanding when dealing with complex contexts; Command intent recognition can identify vehicle scene commands, confirm whether it is a new scene or a change to an existing scene, and then generate a json for scene creation / modification.

4. A large-scale intelligent scene definition system according to claim 1, characterized in that: The scene json parsing can parse the scene json and construct the scene card, including the scene name, service name, and service command; the protocol command can construct the service command executed by the vehicle; The vehicle scenario execution script includes HMI display text, voice TTS, and vehicle execution service commands.

5. A large-model intelligent scene definition system according to claim 1, characterized in that: The training data input includes: LLM output json, historical instruction accumulation, instruction definition set, question-answer pairs, and scenario instructions.

6. A large-model intelligent scene definition system according to claim 1, characterized in that: SFT fine-tuning specifically creates a new neural network model that copies all model designs and parameters of the pre-trained model except the output layer. Because the output layer is closely related to the label of the source dataset, an output layer is added to the target model, whose size is the number of categories in the target dataset, and the parameters of the layer are randomly initialized. When training on the target dataset, the output layer is trained from scratch, and the parameters of the remaining layers are adjusted based on the parameters of the source model.

7. A large model intelligent scene definition method, characterized in that: Specifically: During voice communication, the scene card intelligently defines a large model that actively identifies changes in user application scenarios, automatically generates new scene card json, and identifies existing scene cards for modification and update. The large model intelligently identifies newly defined user scenarios. The scene card definition execution module converts the text json into a script executed by the vehicle, displays the scene card on the vehicle display screen, and performs voice confirmation. After confirmation, the scene card is created and recorded, and then trained with the large model.

Images