Vehicle-mounted intelligent scene generation method and device based on large model, equipment and storage medium

Abstract

The invention discloses a vehicle-mounted intelligent scene generation method and device based on a large model, equipment and a storage medium. The method comprises the following steps: receiving user expression information input by a user, and outputting user intention information according to the user expression information by utilizing a preset large language model; a new vehicle scene card is determined to be generated according to the user intention information, or a target vehicle scene card matched with the user intention information is determined by using a knowledge graph, and the knowledge graph comprises historical vehicle scene cards; and controlling the current vehicle according to the execution action in the new vehicle scene card or the target vehicle scene card. According to the technical scheme of the embodiment of the invention, the new scene card is generated or the matched historical scene card is found, so that the problem that the scene card lacks flexibility and individuation is solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a large model-based vehicle-mounted intelligent scene generation method, device, equipment and storage medium. Background Art

[0002] With the development of the automobile industry, especially the rise of intelligent connected car technology, the intelligence of vehicle systems has become the focus of the industry. Contemporary cars are no longer just a means of transportation, they are transforming into intelligent mobile spaces that integrate multimedia entertainment, navigation, communication and safety assistance.

[0003] At present, traditional in-vehicle intelligent systems generally have the problem of poor interactive experience. For example, the scene modes provided by existing in-vehicle intelligent systems are often relatively fixed and lack the ability to dynamically adjust according to user preferences and real-time needs. At the same time, during the human-computer interaction process, users usually need to communicate with the system in a fixed command format. All of these limit the user experience and lead to a decrease in user satisfaction. Summary of the invention

[0004] The present invention provides a large-model-based vehicle-mounted intelligent scene generation method, device, equipment and storage medium to solve the problem of poor interaction experience between the vehicle and the user.

[0005] In a first aspect, the present invention provides a method for generating an in-vehicle intelligent scene based on a large model, comprising:

[0006] Receiving user expression information input by a user, and outputting user intention information according to the user expression information using a preset large language model;

[0007] Determine and generate a new vehicle scene card according to the user intention information, or determine a target vehicle scene card matching the user intention information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards;

[0008] The current vehicle is controlled according to the execution action in the new vehicle scenario card or the target vehicle scenario card.

[0009] In a second aspect, the present invention provides a vehicle-mounted intelligent scene generation device based on a large model, comprising:

[0010] A user intention recognition module is used to receive user expression information input by a user, and output user intention information according to the user expression information using a preset large language model;

[0011] A vehicle scene card determination module, used to determine and generate a new vehicle scene card according to the user intention information, or to determine a target vehicle scene card matching the user intention information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards;

[0012] The vehicle control module is used to control the current vehicle according to the execution action in the new vehicle scene card or the target vehicle scene card.

[0013] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0014] at least one processor;

[0015] and a memory communicatively coupled to the at least one processor;

[0016] Among them, the memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the large model-based in-vehicle intelligent scene generation method of the first aspect mentioned above.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, which are used to enable a processor to implement the large model-based in-vehicle intelligent scene generation method of the first aspect when executed.

[0018] The in-vehicle intelligent scene generation solution based on a large model provided by the present invention receives user expression information input by a user, outputs user intent information based on the user expression information using a preset large language model, determines to generate a new vehicle scene card based on the user intent information, or determines a target vehicle scene card matching the user intent information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards, and controls the current vehicle based on the execution actions in the new vehicle scene card or the target vehicle scene card. By adopting the above technical solution, the natural language processing capability of the in-vehicle system is enhanced by using a large language model, and the user intent is accurately identified. Then, the user intent and knowledge graph technology are used to improve the accuracy of scene recognition, so that the system can better serve users. By generating new scene cards or finding matching historical scene cards, the problem of lack of flexibility and personalization in scene generation is solved, providing a more intelligent and personalized scene generation solution, and improving the user's interactive experience.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a method for generating a vehicle-mounted intelligent scene based on a large model according to Embodiment 1 of the present invention;

[0022] Figure 2 is a flow chart of a method for generating a vehicle-mounted intelligent scene based on a large model according to Embodiment 2 of the present invention;

[0023] Figure 3 is a structural schematic diagram of a vehicle-mounted intelligent scene generation device based on a large model according to Embodiment 3 of the present invention;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In the description of the present invention, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are an "or" relationship. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1 A flowchart of a method for generating an in-vehicle intelligent scene based on a large model is provided for the first embodiment of the present invention. This embodiment is applicable to the situation where a vehicle is controlled by using an in-vehicle intelligent scene. The method can be executed by an in-vehicle intelligent scene generating device based on a large model. The in-vehicle intelligent scene generating device based on a large model can be implemented in the form of hardware and / or software. The in-vehicle intelligent scene generating device based on a large model can be configured in an electronic device in the vehicle. The electronic device can be composed of two or more physical entities or one physical entity.

[0029] like Figure 1 As shown, the first embodiment of the present invention provides a method for generating a vehicle-mounted intelligent scene based on a large model, which specifically includes the following steps:

[0030] S101, receiving user expression information input by a user, and using a preset large language model to output user intention information according to the user expression information.

[0031] In this embodiment, user expression information input by the user may be received first, such as voice information and / or text information, etc. Then, the user expression information is processed using a pre-trained large language model to obtain user intention information.

[0032] S102. Generate a new vehicle scenario card according to the user intention information, or use a knowledge graph to determine a target vehicle scenario card that matches the user intention information, wherein the knowledge graph includes historical vehicle scenario cards.

[0033] In this embodiment, the knowledge graph can be used to determine whether there is a target vehicle scene card that matches the user's intention information in the historical vehicle scene card. If not, a new vehicle scene card can be generated based on the user's intention information. The knowledge graph can include the association relationship between multiple historical vehicle scene cards and user's intention information. The vehicle scene card can include execution actions, such as opening a window.

[0034] S103: Control the current vehicle according to the execution action in the new vehicle scene card or the target vehicle scene card.

[0035] The technical solution of the embodiment of the present invention utilizes a large language model to enhance the natural language processing capability of the vehicle-mounted system, accurately identifies user intent, and then utilizes the user intent and knowledge graph technology to improve the accuracy of scene recognition, allowing the system to better serve users. By generating new scene cards or finding matching historical scene cards, the problem of lack of flexibility and personalization in scene generation is solved, providing a more intelligent and personalized scene generation solution and improving the user's interactive experience.

[0036] Optionally, before receiving the user expression information input by the user, it also includes: generating a historical vehicle scene card by presetting the constituent elements of the vehicle scene card, wherein the constituent elements include a scene name, a scene description, a scene category, an execution action and a trigger condition.

[0037] Specifically, a scene card template library can be pre-established to pre-define a variety of vehicle scene cards. The components of a vehicle scene card include: scene name, scene description, scene category, execution action, and trigger condition.

[0038] For example, the scene names can be "work mode" and "lunch break mode". The scene description is a brief description of each scene to help users understand its function. For example: "Create an efficient and focused environment to help users maintain clear thinking and a good state" and "Create a comfortable and quiet environment to help users relax quickly, enjoy a short break, and effectively restore their body and mind". The scene category is a classification based on the function and application field of the scene. For example: "Car skills", "Emotional experience", "Comfortable space" and "Efficient travel". The execution action is the specific execution action corresponding to each scene. For example: "Open the window", "Turn off the air conditioner", "Adjust the light and turn on the fragrance sprayer", "Adjust the wave effect of the seat" and "Turn on the air circulation system and play QQ music, and play today's recommended playlists". The trigger condition is the condition for the scene to be triggered. For example, the trigger condition of "work mode" is: "Start time: 07:00, end time: 10:00, repeated days: statutory working days".

[0039] Optionally, before receiving the user expression information input by the user, it also includes: for each historical vehicle scene card, determining the association relationship between the constituent elements of the current historical vehicle scene card, and generating triples according to the association relationship; and generating a knowledge graph using the triples.

[0040] Specifically, entities (i.e., constituent elements) and their associations can be extracted from historical vehicle scenario cards to generate triples. The knowledge graph of the scenario card can be created using the triples. Triples are a common way to represent knowledge graphs, with the format of (node ​​1, edge, node 2). The triples of scenario cards are represented as (main entity, relationship, guest entity). Each triple consists of two entities and corresponding relationships. Both entities and relationships have descriptive concepts and attributes, where concepts refer to the types and categories of entities or relationships, and attributes refer to their intrinsic parameters, characteristics, and properties. The knowledge graph is composed of multiple triples connected to each other.

[0041] Exemplarily, the relationships between entities include the following four types:

[0042] 1) BELONGS_TO (belongs to) indicates the category to which the entity (such as scene name) belongs;

[0043] 2) DESCRIBED_AS() indicates the detailed description of the entity (such as the scene name);

[0044] 3) HAS_ACTION() indicates the execution action contained in the entity (such as scene name);

[0045] 4) HAS_TRIGGER indicates the trigger condition contained in the entity (such as the scene name).

[0046] The triples include:

[0047] (Scene name, belongs to, category), (Scene name, description is, description), (Description, belongs to, category), (Scene name, contains execution action, execution action), (Scene name, contains trigger condition, trigger condition), (execution action, description is, execution action), (trigger condition, description is, trigger condition), etc. Among them, if the trigger condition is a complex condition or contains multiple sub-conditions, or the execution action is a complex action or contains multiple sub-actions, it needs to be further refined. For example, if the user expression information input by the user is "get in the car", then there can be two corresponding sub-conditions "Driver's seat: occupied" and "Ignition status: turned on".

[0048] For example, a specific vehicle scenario card can be represented as:

[0049] 1) Scene name: Lunch break mode;

[0050] 2) Scene description: Create a comfortable and quiet environment to help users relax quickly, enjoy a short break, and effectively restore their body and mind;

[0051] 3)Scene category: comfortable space;

[0052] 4) Execute actions: close the car window, turn on the fragrance sprayer, adjust the seat's wave effect, and turn on the air circulation system;

[0053] 5) Trigger condition: time is between 12:00 and 13:00.

[0054] The corresponding list of triples is:

[0055] 1)(Lunch break mode, BELONGS_TO, comfortable space);

[0056] 2)(Lunch break mode, DESCRIBED_AS, creates a comfortable and quiet environment to help users relax quickly, enjoy a short rest, and effectively restore their body and mind);

[0057] 3) (Create a comfortable and quiet environment to help users relax quickly, enjoy a short rest, and effectively restore their body and mind, BELONGS_TO, comfortable space);

[0058] 4) (Lunch break mode, HAS_ACTION, close the car window);

[0059] 5) (Lunch break mode, HAS_ACTION, turn on the fragrance sprayer);

[0060] 6) (Lunch break mode, HAS_ACTION, adjust the seat fluctuation effect);

[0061] 7) (Lunch break mode, HAS_ACTION, turn on the air circulation system).

[0062] Among them, the graph database Neo4j can be used to build a relational model and store these entities and relationships in the form of a graph.

[0063] Optionally, the using of a preset large language model to output user intention information based on the user expression information includes: using a preset large language model to mine user intention information from the user expression information based on a first preset prompt, wherein the first preset prompt includes user intention generation guidance information, and the user intention information includes vehicle scene card creation requirement information or emotional expression information; wherein, generating a new vehicle scene card based on the user intention information, or using a knowledge graph to determine a target vehicle scene card that matches the user intention information, includes: if the user intention information includes vehicle scene card creation requirement information, then generating a new vehicle scene card based on the vehicle scene card creation requirement information; if the user intention information includes emotional expression information, then using a knowledge graph to determine a target vehicle scene card that matches the user intention information.

[0064] Specifically, a large model (such as Qwen or GLM, etc.) can be used to identify the user's input user expression information, which can identify the user's demand type (i.e., user intention information). Before entering the large model, the user expression information can also be preprocessed. The specific operations include: preprocessing the user expression information input by the user, including text cleaning, removing irrelevant characters (such as extra spaces and special symbols), standardization (unifying letter case and converting common abbreviations, etc.) and word segmentation. For example, the user inputs "the air outside is good or bad", and the output after preprocessing is: "outside", "air", "good", "bad".

[0065] Then, the preprocessed user expression information is input into the big model, and the pre-defined first preset prompt is used to assist the big model in determining the type of user demand. This setting can increase the accuracy of the big model output. The types include scene card creation categories (i.e., vehicle scene card creation demand information) and emotional expression categories (i.e., emotional expression information). The first preset prompt may include user intention generation guidance information such as input and output examples of the big model. Among them, the vehicle scene card creation demand information includes the user's output direct vehicle control instructions, and the emotional expression information includes the user's implicit vehicle control requirements (indirect vehicle control instructions), such as information expressing emotions or feelings.

[0066] Next, after identifying the user's intention information, if the user's intention information includes the vehicle scene card creation requirement information, a new vehicle scene card is generated according to the vehicle scene card creation requirement information. If the user's intention information includes emotional expression information, the knowledge graph is used to determine the target vehicle scene card that matches the user's intention information.

[0067] Exemplarily, the first preset prompt may be expressed as:

[0068] 1. You are an expert in in-vehicle intelligent scene recognition;

[0069] 2. Define the following types of user needs:

[0070] - Create scene cards (e.g., when I get in the car, the air conditioner will automatically turn on and music will play) - Emotional expression (e.g., I want to take a nap)

[0071] 3. Define the following scene categories:

[0072] -Car use skills

[0073] -Emotional experience

[0074] - Comfortable space

[0075] - Efficient travel

[0076] 4. The requirement types and scenario categories in the returned results only return those that have been defined above;

[0077] 5. Based on the user's input, determine the user's demand type and scenario category, rewrite the user input into a form that is more in line with the scenario description, and extract the keywords and phrases entered by the user

[0078] 6. The returned results are output in JSON format

[0079] 7. Example:

[0080] - Input: "When I get in the car, automatically turn on the air conditioner and play music"

[0081] - Output: {"Requirement type":"Create scenario card","Scenario category":"Car use skills","Keywords and phrases":["Get in the car","Turn on the air conditioner","Play music"],"Rewritten query":"When the user gets in the car, automatically turn on the air conditioner and play music"}

[0082] - Input: "I want to take a nap"

[0083] - Output: {"Requirement type":"emotional expression","Scene category":"comfortable space","Keywords and phrases":["want to sleep"],"Rewritten query":"I feel a little sleepy and hope to have a quiet and comfortable environment to take a nap"}

[0084] question:

[0085] Which of the following two categories does this user query belong to: 1. Creating a scene card 2. Emotional expression; determine the scene category, rewrite the user input into a form that is more in line with the scene description, and extract keywords and phrases.

[0086] Embodiment 2

[0087] Figure 2A flowchart of a large-model-based in-vehicle intelligent scene generation method provided in Embodiment 2 of the present invention. The technical solution of the embodiment of the present invention is further optimized on the basis of the above-mentioned optional technical solutions, and a specific method of controlling a vehicle using an in-vehicle intelligent scene is given.

[0088] Optionally, the determining to generate a new vehicle scene card according to the user intention information includes: if the user intention information includes vehicle scene card creation requirement information, extracting first target information from the user expression information using a natural language processing model, wherein the first target information is related to the constituent elements of the historical vehicle scene card; generating an initial vehicle scene card according to the first target information, keywords and phrases using the preset large language model and the second preset prompt, wherein the user intention information also includes the keywords and the phrases, and the second preset prompt includes vehicle scene card generation guidance information; generating a new vehicle scene card by calculating the similarity between the initial vehicle scene card and the historical vehicle scene card in the knowledge graph. The advantage of such a setting is that, based on the RAG (Retrieval-Augmented Generation) idea, when the user intention information includes vehicle scene card creation requirement information, the initial vehicle scene card is generated using the large model, and the new vehicle scene card is accurately generated by similarity calculation, realizing automatic generation of personalized scene cards according to the actual needs of the user, ensuring that the generated scene card not only meets the user's needs, but also has high accuracy and personalization.

[0089] Optionally, the method of using the knowledge graph to determine the target vehicle scene card that matches the user intention information includes: if the user intention information includes emotional expression information, extracting second target information from the user expression information and the rewritten sentence using a natural language processing model, wherein the second target information is related to the constituent elements of the historical vehicle scene card, and the user intention information also includes the rewritten sentence; determining the target vehicle scene card that matches the user intention information by performing similarity calculation on the second target information, keywords and phrases, and the historical vehicle scene cards in the knowledge graph, wherein the user intention information also includes the keywords and the phrases. The advantage of such a setting is that when the user intention information includes emotional expression information, the use of a large model and similarity calculation enhances the natural language processing capability, improves the accuracy of scene recognition, promotes the high personalization and intelligence of the on-board intelligent system, and improves the convenience and comfort of user interaction with the vehicle.

[0090] like Figure 2 As shown, a method for generating an in-vehicle intelligent scene based on a large model provided in the second embodiment of the present invention specifically includes the following steps:

[0091] S201. Generate a historical vehicle scene card by presetting the constituent elements of the vehicle scene card; for each historical vehicle scene card, determine the association relationship between the constituent elements of the current historical vehicle scene card, and generate a triple according to the association relationship; generate a knowledge graph using the triple; wherein the constituent elements include a scene name, a scene description, a scene category, an execution action, and a trigger condition.

[0092] S202, receiving user expression information input by the user, and using a preset large language model to mine user intention information from the user expression information according to a first preset prompt, wherein the first preset prompt includes user intention generation guidance information, and the user intention information includes vehicle scene card creation requirement information or emotion expression information.

[0093] S203 , determining whether the user intention information includes vehicle scene card creation requirement information, if so, executing step 204 , if not, executing step 207 .

[0094] S204. Extracting first target information from the user expression information using a natural language processing model, wherein the first target information is related to constituent elements of the historical vehicle scene card.

[0095] Specifically, if the user intent information includes vehicle scene card creation requirement information, a natural language processing model (such as a SpaCy model) can be used to extract entity words or phrases related to the constituent elements of the historical vehicle scene card from the user expression information, that is, the first target information.

[0096] S205. Generate an initial vehicle scenario card according to the first target information, keywords and phrases by using the preset large language model and the second preset prompt, wherein the user intent information also includes the keywords and the phrases, and the second preset prompt includes vehicle scenario card generation guidance information.

[0097] Specifically, a second preset prompt may be pre-set to assist the preset large language model in generating an initial vehicle scene card according to the first target information and the keywords and phrases in the user intent information. The second preset prompt includes vehicle scene card generation guidance information such as vehicle scene card generation examples.

[0098] Exemplarily, the second preset prompt may be expressed as:

[0099] 1. You are an expert in in-vehicle intelligent scene recognition;

[0100] 3. Define the following scene categories:

[0101] -Car use skills

[0102] -Emotional experience

[0103] - Comfortable space

[0104] - Efficient travel

[0105] 4. The scene categories in the returned results only return those defined above;

[0106] 6. The returned results are output in JSON format

[0107] 7. Example of scene card:

[0108] - Example 1:

[0109] Scene name: Lunch break mode

[0110] Scene description: Create a comfortable and quiet environment to help users relax quickly, enjoy a short break, and effectively restore their body and mind

[0111] Scene category: Comfortable space

[0112] Trigger conditions:

[0113] Actions: Close the car window, turn on the fragrance sprayer, adjust the seat's wave effect, turn on the air circulation system

[0114] - Example 2:

[0115] Scene name: Poor air quality mode

[0116] Scenario description: When the outside air quality is poor, close all windows and sunroof, start the air purification system, and protect the air quality inside the car

[0117] Scenario category: Car use skills

[0118] Trigger conditions:

[0119] Action: Close all windows, close the sunroof, start the air purification system

[0120] ##question

[0121] Generate a scenario card based on the user's input and the keywords and phrases proposed

[0122] ##enter

[0123] {user_input}

[0124] ## Output

[0125] {

[0126] "Scene Name":"",

[0127] "Scene description":"",

[0128] "Scene Category":"",

[0129] "Execute action":[],

[0130] "Trigger Condition":[]

[0131] }

[0132] For example, when the first target information input into the preset large language model is: "When I get in the car, the air conditioner is automatically turned on and music is played", the keywords and phrases are: "get in the car", "turn on the air conditioner", "play music". The initial vehicle scene card output by the preset large language model is: {"scene name":"welcome mode","scene description":"When the user gets in the car, the vehicle automatically adjusts to a comfortable initial state, including turning on the air conditioner to adjust the temperature, and playing music at the same time, to provide the user with a pleasant riding experience.","scene category":"efficient travel","execution action":["turn on the air conditioner","play music"],"trigger condition":["get in the car"]}.

[0133] S206 , generating a new vehicle scene card by calculating the similarity between the initial vehicle scene card and the historical vehicle scene card in the knowledge graph. Execute step 211 .

[0134] Specifically, a similarity calculation can be performed on the contents of the initial vehicle scene card and the historical vehicle scene cards in the knowledge graph, and the target content in the historical vehicle scene card with the highest similarity is written into the initial vehicle scene card, such as the content corresponding to the execution action, to obtain a new vehicle scene card.

[0135] Furthermore, the method of generating a new vehicle scene card by performing similarity calculation between the initial vehicle scene card and the historical vehicle scene cards in the knowledge graph includes: performing similarity calculation between the trigger conditions and execution actions in the constituent elements of the initial vehicle scene card and the trigger conditions and execution actions in the constituent elements of the historical vehicle scene cards in the knowledge graph to obtain a first calculation result; and replacing the trigger conditions and execution actions in the initial vehicle scene card with the trigger conditions and execution actions corresponding to the highest first calculation result to generate a new vehicle scene card.

[0136] Optionally, the similarity calculation includes TF-IDF similarity calculation and BERT similarity calculation.

[0137] Specifically, the trigger conditions and execution actions in the initial vehicle scene card can be similarly calculated with the trigger conditions and execution actions in the knowledge graph, and the trigger conditions and execution actions of the historical vehicle scene card with the highest similarity in the knowledge graph can be determined as the best match. The trigger conditions and execution actions in the initial vehicle scene card are replaced with the trigger conditions and execution actions corresponding to the highest similarity to generate a new vehicle scene card. Among them, TF-IDF cosine similarity is usually used for statistical-based text representation, while BERT is a pre-trained model based on deep learning that can provide richer and more semantic text representation. Combining the two to calculate text similarity can comprehensively utilize statistical information and semantic information to improve the accuracy and robustness of similarity calculation.

[0138] Exemplarily, the method of determining the first calculation result includes:

[0139] First calculation result = α*TF-IDF similarity calculation result + (1-α)*BERT similarity calculation result

[0140] Among them, α is a preset weight parameter, such as being set to 0.4.

[0141] S207, determining whether the user intention information includes emotion expression information, if so, executing step 208, if not, executing step 202.

[0142] Specifically, if the user intention information does not include emotion expression information and vehicle scenario card creation requirement information, step 202 may be executed to re-receive the user expression information input by the user.

[0143] S208. Extracting second target information from the user expression information and the rewritten sentence using a natural language processing model, wherein the second target information is related to constituent elements of the historical vehicle scenario card, and the user intention information also includes the rewritten sentence.

[0144] Specifically, if the user intention information includes emotional expression information, a natural language processing model (such as a SpaCy model) can be used to extract entity words or phrases related to the constituent elements of the historical vehicle scene card from the user expression information and the rewritten sentence, i.e., the second target information. The rewritten sentence is a rewritten sentence of the user expression information, which is more standardized, more fluent, and more suitable for large model processing.

[0145] S209. Determine a target vehicle scene card that matches the user intent information by calculating similarity between the second target information, keywords and phrases, and historical vehicle scene cards in the knowledge graph, wherein the user intent information also includes the keywords and the phrases.

[0146] Specifically, the similarity between the second target information, keywords and phrases, and the content in the historical vehicle scene cards in the knowledge graph can be calculated, and the historical vehicle scene card with the highest similarity can be determined as the target vehicle scene card that matches the user intention information.

[0147] Furthermore, the method of determining a target vehicle scene card that matches the user intention information by performing similarity calculation on the second target information, keywords and phrases, and historical vehicle scene cards in the knowledge graph includes: performing similarity calculation on the scene descriptions in the historical vehicle scene cards in the knowledge graph with the second target information, keywords and phrases, respectively, to obtain a second calculation result; and determining the historical vehicle scene card corresponding to the highest second calculation result as the target vehicle scene card that matches the user intention information.

[0148] Specifically, the historical vehicle scene cards in the knowledge graph can be traversed to calculate the similarity between each scene description under each scene category and the second target information, keywords and phrases. The historical vehicle scene card corresponding to the scene description with the highest similarity is determined as the target vehicle scene card matching the user intention information.

[0149] S210: Control the current vehicle according to the execution action in the target vehicle scene card.

[0150] Specifically, before executing step 210, the target vehicle scene card may be displayed to allow the user to confirm and receive the user's feedback information. After the user confirms, the current vehicle may be automatically controlled according to the execution action in the target vehicle scene card.

[0151] S211. Control the current vehicle according to the execution action in the new vehicle scene card.

[0152] Specifically, before executing step 211, a new vehicle scene card may be displayed to allow the user to confirm and receive user feedback information. After the user confirms, the new vehicle scene card may be saved and updated to the knowledge graph to obtain an updated knowledge graph. In this way, the knowledge graph may be continuously optimized and expanded to provide more intelligent and personalized services based on the user's description.

[0153] The in-vehicle intelligent scene generation method based on a big model provided by an embodiment of the present invention is based on the RAG (Retrieval-Augmented Generation) idea. When the user intention information includes the vehicle scene card creation requirement information, the big model is used to generate the initial vehicle scene card, and the new vehicle scene card is accurately generated through similarity calculation, thereby realizing the automatic generation of personalized scene cards according to the actual needs of the user, ensuring that the generated scene cards not only meet the user needs but also have high accuracy and personalization. When the user intention information includes emotional expression information, the big model and similarity calculation are used to enhance the natural language processing capability, improve the accuracy of scene recognition, promote the high personalization and intelligence of the in-vehicle intelligent system, and enhance the convenience and comfort of user interaction with the vehicle.

[0154] Embodiment 3

[0155] Figure 3 This is a schematic diagram of the structure of a vehicle-mounted intelligent scene generation device based on a large model provided in the third embodiment of the present invention. Figure 3 As shown, the device includes: a user intention recognition module 301, a vehicle scene card determination module 302 and a vehicle control module 303, wherein:

[0156] A user intention recognition module is used to receive user expression information input by a user, and output user intention information according to the user expression information using a preset large language model;

[0157] A vehicle scene card determination module, used to generate a new vehicle scene card according to the user intention information, or to determine a target vehicle scene card matching the user intention information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards;

[0158] The vehicle control module is used to control the current vehicle according to the execution action in the new vehicle scene card or the target vehicle scene card.

[0159] The large-model-based in-vehicle intelligent scene generation device provided in the embodiment of the present invention utilizes a large language model to enhance the natural language processing capability of the in-vehicle system, accurately identifies user intent, and then utilizes the user intent and knowledge graph technology to improve the accuracy of scene recognition, allowing the system to better serve users. By generating new scene cards or finding matching historical scene cards, it solves the problem of lack of flexibility and personalization in scene generation, provides a more intelligent and personalized scene generation solution, and enhances the user's interactive experience.

[0160] Optionally, the device further comprises:

[0161] The historical scene card generation module is used to generate a historical vehicle scene card by presetting the constituent elements of the vehicle scene card before receiving the user expression information input by the user, wherein the constituent elements include the scene name, scene description, scene category, execution action and trigger condition.

[0162] Optionally, the device further comprises:

[0163] A triplet generation module is used to determine, for each historical vehicle scene card, the association relationship between the components of the current historical vehicle scene card before receiving the user expression information input by the user, and generate a triplet according to the association relationship;

[0164] A knowledge graph generation module is used to generate a knowledge graph using the triples.

[0165] Optionally, the user intention recognition module includes:

[0166] A user intention recognition unit is used to use a preset large language model to mine user intention information from the user expression information according to a first preset prompt, wherein the first preset prompt includes user intention generation guidance information, and the user intention information includes vehicle scene card creation requirement information or emotion expression information.

[0167] Optionally, the vehicle scene card determination module includes:

[0168] A new scene card determination unit, configured to generate a new vehicle scene card according to the vehicle scene card creation requirement information if the user intention information includes the vehicle scene card creation requirement information;

[0169] The first target scene card generating unit is used to determine a target vehicle scene card matching the user intention information by using a knowledge graph if the user intention information includes emotion expression information.

[0170] Optionally, the vehicle scene card determination module includes:

[0171] A first target information extraction unit, configured to extract first target information from the user expression information by using a natural language processing model if the user intention information includes vehicle scene card creation requirement information, wherein the first target information is related to a component element of the historical vehicle scene card;

[0172] an initial scene card generating unit, configured to generate an initial vehicle scene card according to the first target information, keywords and phrases by using the preset large language model and the second preset prompt, wherein the user intention information also includes the keywords and the phrases, and the second preset prompt includes vehicle scene card generation guidance information;

[0173] A new scene card generating unit is used to generate a new vehicle scene card by calculating the similarity between the initial vehicle scene card and the historical vehicle scene cards in the knowledge graph.

[0174] Furthermore, the method of generating a new vehicle scene card by performing similarity calculation between the initial vehicle scene card and the historical vehicle scene cards in the knowledge graph includes: performing similarity calculation between the trigger conditions and execution actions in the constituent elements of the initial vehicle scene card and the trigger conditions and execution actions in the constituent elements of the historical vehicle scene cards in the knowledge graph to obtain a first calculation result; and replacing the trigger conditions and execution actions in the initial vehicle scene card with the trigger conditions and execution actions corresponding to the highest first calculation result to generate a new vehicle scene card.

[0175] Optionally, the vehicle scene card determination module includes:

[0176] A second target information extraction unit is used to extract second target information from the user expression information and the rewritten sentence using a natural language processing model if the user intention information includes emotional expression information, wherein the second target information is related to the constituent elements of the historical vehicle scene card, and the user intention information also includes the rewritten sentence;

[0177] The second target scenario card generating unit is used to determine a target vehicle scenario card that matches the user intention information by calculating similarity between the second target information, keywords and phrases, and historical vehicle scenario cards in the knowledge graph, wherein the user intention information also includes the keywords and the phrases.

[0178] Furthermore, the method of determining a target vehicle scene card that matches the user intention information by performing similarity calculation on the second target information, keywords and phrases, and historical vehicle scene cards in the knowledge graph includes: performing similarity calculation on the scene descriptions in the historical vehicle scene cards in the knowledge graph with the second target information, keywords and phrases, respectively, to obtain a second calculation result; and determining the historical vehicle scene card corresponding to the highest second calculation result as the target vehicle scene card that matches the user intention information.

[0179] The large-model-based in-vehicle intelligent scene generation device provided in an embodiment of the present invention can execute the large-model-based in-vehicle intelligent scene generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0180] Embodiment 4

[0181] Figure 4A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0182] like Figure 4 As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0183] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0184] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 41 executes the various methods and processes described above, such as the vehicle-mounted intelligent scene generation method based on a large model.

[0185] In some embodiments, the vehicle-mounted intelligent scene generation method based on the large model can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the vehicle-mounted intelligent scene generation method based on the large model described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the vehicle-mounted intelligent scene generation method based on the large model by any other appropriate means (e.g., by means of firmware).

[0186] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0187] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0188] The computer device provided above can be used to execute the large model-based in-vehicle intelligent scene generation method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0189] Embodiment 5

[0190] In the context of the present invention, a computer-readable storage medium may be a tangible medium, and the computer-executable instructions, when executed by a computer processor, are used to perform a method for generating a vehicle-mounted intelligent scene based on a large model, the method comprising:

[0191] Receiving user expression information input by a user, and outputting user intention information according to the user expression information using a preset large language model;

[0192] Generate a new vehicle scene card according to the user intention information, or determine a target vehicle scene card matching the user intention information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards;

[0193] The current vehicle is controlled according to the execution action in the new vehicle scenario card or the target vehicle scenario card.

[0194] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by an instruction execution system, device or equipment or used with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0195] The computer device provided above can be used to execute the large model-based in-vehicle intelligent scene generation method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0196] It is worth noting that in the above-mentioned embodiment of the large-model-based in-vehicle intelligent scene generation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0197] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for generating an in-vehicle intelligent scene based on a large model, characterized in that: include: Receiving user expression information input by a user, and outputting user intention information according to the user expression information using a preset large language model; Determine and generate a new vehicle scene card according to the user intention information, or determine a target vehicle scene card matching the user intention information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards; The current vehicle is controlled according to the execution action in the new vehicle scenario card or the target vehicle scenario card.

2. The method according to claim 1, characterized in that: Before receiving the user expression information input by the user, the method further includes: A historical vehicle scenario card is generated by presetting constituent elements of the vehicle scenario card, wherein the constituent elements include a scenario name, a scenario description, a scenario category, an execution action, and a trigger condition.

3. The method according to claim 2, characterized in that Before receiving the user expression information input by the user, the method further includes: For each historical vehicle scene card, determining the association relationship between the components of the current historical vehicle scene card, and generating a triplet according to the association relationship; A knowledge graph is generated using the triples.

4. The method according to claim 1, characterized in that: The using a preset large language model to output user intention information according to the user expression information includes: Mining user intention information from the user expression information according to a first preset prompt using a preset large language model, wherein the first preset prompt includes user intention generation guidance information, and the user intention information includes vehicle scene card creation requirement information or emotion expression information; Wherein, the determining to generate a new vehicle scene card according to the user intention information or determining a target vehicle scene card matching the user intention information by using a knowledge graph includes: If the user intention information includes vehicle scene card creation requirement information, generating a new vehicle scene card according to the vehicle scene card creation requirement information; If the user intention information includes emotion expression information, the knowledge graph is used to determine the target vehicle scene card that matches the user intention information.

5. The method according to claim 1, characterized in that The determining and generating a new vehicle scenario card according to the user intention information includes: If the user intention information includes vehicle scene card creation requirement information, extracting first target information from the user expression information using a natural language processing model, wherein the first target information is related to a component element of the historical vehicle scene card; Using the preset large language model and the second preset prompt, generating an initial vehicle scenario card according to the first target information, keywords and phrases, wherein the user intent information also includes the keywords and the phrases, and the second preset prompt includes vehicle scenario card generation guidance information; A new vehicle scene card is generated by calculating the similarity between the initial vehicle scene card and the historical vehicle scene cards in the knowledge graph.

6. The method according to claim 5, characterized in that The generating of a new vehicle scene card by calculating the similarity between the initial vehicle scene card and the historical vehicle scene card in the knowledge graph comprises: Calculate the similarity between the trigger conditions and execution actions in the constituent elements of the initial vehicle scenario card and the trigger conditions and execution actions in the constituent elements of the historical vehicle scenario card in the knowledge graph, respectively, to obtain a first calculation result; The trigger condition and the execution action corresponding to the highest first calculation result are used to replace the trigger condition and the execution action in the initial vehicle scenario card to generate a new vehicle scenario card.

7. The method according to claim 1, characterized in that The step of using the knowledge graph to determine a target vehicle scenario card that matches the user intention information includes: If the user intention information includes emotional expression information, extracting second target information from the user expression information and the rewritten sentence using a natural language processing model, wherein the second target information is related to the constituent elements of the historical vehicle scene card, and the user intention information also includes the rewritten sentence; By calculating the similarity between the second target information, keywords and phrases, and the historical vehicle scene cards in the knowledge graph, a target vehicle scene card matching the user intent information is determined, wherein the user intent information also includes the keywords and the phrases.

8. The method according to claim 7, characterized in that The method of calculating similarity between the second target information, keywords and phrases, and historical vehicle scene cards in the knowledge graph to determine a target vehicle scene card that matches the user intention information includes: Calculate the similarity between the scene description in the historical vehicle scene card in the knowledge graph and the second target information, keywords and phrases to obtain a second calculation result; The historical vehicle scenario card corresponding to the highest second calculation result is determined as the target vehicle scenario card matching the user intention information.

9. The method according to any one of claims 5 to 8, characterized in that: The similarity calculation includes TF-IDF similarity calculation and BERT similarity calculation.

10. A vehicle-mounted intelligent scene generation device based on a large model, characterized in that: include: A user intention recognition module is used to receive user expression information input by a user, and output user intention information according to the user expression information using a preset large language model; A vehicle scene card determination module, used to determine and generate a new vehicle scene card according to the user intention information, or to determine a target vehicle scene card matching the user intention information using a knowledge graph, wherein the knowledge graph includes historical vehicle scene cards; The vehicle control module is used to control the current vehicle according to the execution action in the new vehicle scene card or the target vehicle scene card.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the large model-based in-vehicle intelligent scene generation method described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the large model-based vehicle-mounted intelligent scene generation method described in any one of claims 1 to 9 when executed.

Images