Vehicle control method and device, electronic equipment, storage medium and program product

Abstract

The application provides a vehicle control method, device, electronic equipment, storage medium and program product. The method comprises: obtaining user text; based on a large language model, generating a calling instruction corresponding to the user text under the constraint of vehicle control tool definition and slot description information, the vehicle control tool definition comprising an intention identifier, function description text and parameter information of the vehicle control tool, and the slot description information corresponding to the parameter information; and calling a corresponding vehicle control tool based on the calling instruction to control the vehicle. The method, device, electronic equipment, storage medium and program product provided by the application utilize the function calling capability of the large language model to accurately convert non-standard expressions of the user into calling instructions under the constraint of the vehicle control tool definition and the slot description information, thereby greatly improving the degree of freedom of vehicle-mounted voice interaction. Moreover, for newly added vehicle-mounted control functions, only the vehicle-mounted tool definition and the slot description information need to be updated, without modifying the underlying logic, thereby greatly reducing the maintenance cost.

Description

Technical Field

[0001] This invention relates to the field of smart cockpit technology, and more particularly to a vehicle control method, device, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of intelligent vehicle technology, in-vehicle voice interaction has become one of the core functions of intelligent cockpits.

[0003] Currently, in-vehicle voice interaction mainly relies on fixed command templates and rule engines. Due to this over-reliance on static rules, the naturalness of in-vehicle voice interaction is low, and it is difficult to recognize vague expressions from users in everyday scenarios. Furthermore, for each new vehicle control function added, technicians need to write a large number of rules and templates, resulting in high maintenance costs and poor scalability. Summary of the Invention

[0004] This invention provides a vehicle control method, device, electronic device, storage medium, and program product to address the shortcomings of poor semantic understanding and high maintenance costs in in-vehicle voice interaction in related technologies.

[0005] This invention provides a vehicle control method, comprising: Get user text; Based on a large language model, under the constraints of vehicle control tool definition and slot description information, a call command corresponding to the user text is generated. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool. The slot description information corresponds to the parameter information. The vehicle is controlled by invoking the corresponding vehicle control tool based on the invocation command.

[0006] According to a vehicle control method provided by the present invention, the step of generating a call instruction corresponding to the user text based on a large language model, under the constraints of vehicle control tool definition and slot description information, includes: Based on the large language model, the user text is classified as a control intent. Based on the large language model, combined with the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated.

[0007] According to a vehicle control method provided by the present invention, the step of classifying the user text for control intent based on the large language model includes: Based on the large language model, the user text is classified into control intents under the constraints of the definition of rejection intent; The definition of the denial intent includes a scenario description text that is not related to vehicle control intent.

[0008] According to a vehicle control method provided by the present invention, the step of generating a call instruction corresponding to the user text based on the large language model, combined with the result of the control intent classification, the vehicle control tool definition, and the slot description information, includes: Based on the large language model, combined with the set of available processing strategies, the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated. The set of available processing strategies includes at least one of the following: synonym mapping strategy, intent decomposition strategy, context fusion strategy, and parameter inference strategy.

[0009] According to a vehicle control method provided by the present invention, the step of generating a call instruction corresponding to the user text based on a large language model, under the constraints of vehicle control tool definition and slot description information, includes: At least one of the following: interaction history information, vehicle information, environmental information, and user information, as well as the user text, is input into the large language model to obtain the calling instructions output by the large language model under the constraints of the vehicle control tool definition and the slot description information.

[0010] A vehicle control method according to the present invention further includes: Obtain the execution result of the invocation instruction; Based on the large language model, and combined with the execution result, interactive text or a new round of calling instructions are generated under the constraints of the vehicle control tool definition and the slot description information.

[0011] According to a vehicle control method provided by the present invention, the parameter information includes at least one of the parameter type, parameter description text, and parameter necessity of the parameters required by the vehicle control tool; The slot description information includes at least one of the following: the slot identifier, slot class, and slot enumeration list for the slot corresponding to the parameters required by the vehicle control tool.

[0012] According to a vehicle control method provided by the present invention, the step of obtaining user text includes: Receive user text sent from the vehicle terminal, wherein the user text is sent when it is determined to be a complex intent instruction.

[0013] The present invention also provides a vehicle control device, comprising: The acquisition unit is used to acquire user text; The generation unit is used to generate the call instruction corresponding to the user text based on the large language model and under the constraints of the vehicle control tool definition and slot description information. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool, and the slot description information corresponds to the parameter information. The control unit is used to invoke the corresponding vehicle control tools to control the vehicle based on the invocation command.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle control method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle control method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the vehicle control method as described above.

[0017] The vehicle control method, device, electronic device, storage medium, and program product provided by this invention utilize the semantic understanding and function call capabilities of a large language model. Under the constraints of structured vehicle control tool definitions and slot description information, it generates call commands representing the user's control intent from text. This allows non-standard expressions uttered by the user to be accurately converted into call commands for vehicle control tools, greatly improving the freedom of in-vehicle voice interaction. The application of structured vehicle control tool definitions and slot description information helps the large language model perform accurate semantic alignment and parameter inference, effectively avoiding the hallucination output and command drift problems of large models in complex vehicle control scenarios. This significantly improves the understanding accuracy and execution security of in-vehicle voice interaction under complex linguistic phenomena such as multi-intent parallelism, referential resolution, and negation expressions. Furthermore, for newly added vehicle control functions, only the vehicle tool definitions and slot description information need to be updated, without modifying the underlying logic, greatly reducing maintenance costs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the vehicle control method provided by the present invention.

[0020] Figure 2 This is the second flowchart of the vehicle control method provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the vehicle control device provided by the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] All actions involving the acquisition of signal information or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the device is located, and with the authorization granted by the owner of the device.

[0025] In the field of smart cockpit and in-vehicle voice interaction technology, accurately understanding and executing the user's vehicle control intentions is key to improving the user interaction experience. Traditional in-vehicle voice interaction solutions typically adopt a pipeline architecture that combines ASR (Automatic Speech Recognition) and NLU (Natural Language Understanding), and its core logic is static matching based on predefined grammar rules and entity dictionaries.

[0026] Semantic understanding, as an important means of measuring the intelligence of an interactive system, is mainly achieved by parsing the user's input text and determining whether it conforms to a specific instruction format. In this case, usually only instruction text that conforms to a specific instruction format can trigger vehicle functions, such as "turn on the air conditioner" or "turn up the volume." If the user's expression does not conform to the preset template, the instruction is often not recognized or is judged as invalid.

[0027] This rule-based and template-based interaction approach falls short when faced with complex and ever-changing natural language scenarios.

[0028] Firstly, users need to memorize specific command formats and cannot express their needs using everyday natural language. For example, vague expressions such as "I'm a little hot" or "I feel a little stuffy" cannot be recognized, resulting in low naturalness of in-vehicle voice interaction. Furthermore, since such interaction solutions are mostly based on keyword matching or grammatical rules, they are difficult to understand contextual semantics, process multiple intentions in parallel, and resolve complex linguistic phenomena such as referential relationships. They are also insufficient in processing user voices that do not contain objects of control and only express subjective feelings, and cannot cover the diversity of users' everyday natural language.

[0029] Secondly, when faced with complex control requirements such as multi-step reasoning, scene linkage, and parameter inference, rule-based and template-based solutions lack sufficient processing capabilities and cannot effectively handle complex scenarios.

[0030] Thirdly, due to the lack of a standardized description format for vehicle control functions, various semantic understanding models, including LLM (Large Language Model), cannot automatically understand and invoke vehicle control functions through structured tools. This results in the need to manually write a large number of rules when adding new vehicle control functions, leading to high maintenance costs and poor scalability.

[0031] To address the above problems, embodiments of the present invention provide a vehicle control method. Figure 1 This is one of the flowcharts illustrating the vehicle control method provided by the present invention, such as... Figure 1 As shown, the method includes: Step 110: Obtain user text.

[0032] Specifically, user text refers to text expressed by a user using natural language. User text may contain vehicle control intentions or may not contain vehicle control intentions and may only be casual conversation. In in-vehicle voice interaction scenarios, the user's voice signal can be collected through an in-vehicle microphone array, and speech recognition technology can be used to convert the speech signal into text in real time, thereby obtaining the user text.

[0033] The user text here can be instruction text that conforms to a specific instruction format, such as "turn on the air conditioner" or "turn up the volume", or it can be descriptive text that contains a controlled object but is vaguely expressed, such as "the car windows are all fogged up", or it can be descriptive text that does not contain a controlled object and requires inference of the user's needs from their perception, such as "I'm a little hot" or "I feel a little stuffy". This embodiment of the invention does not make specific limitations on this.

[0034] Step 120: Based on the large language model, generate the call command corresponding to the user text under the constraints of the vehicle control tool definition and slot description information. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool. The slot description information corresponds to the parameter information.

[0035] Specifically, a Large Language Model (LLM) refers to a deep learning model with massive parameters and large-scale pre-training. LLMs possess powerful natural language understanding, logical reasoning, and function call capabilities, enabling them to understand complex contextual semantics and select appropriate external tools based on instructions. In this embodiment of the invention, the function call capability of the LLM can be used to invoke various vehicle control tools.

[0036] Here, vehicle control tools refer to callable entities that utilize the function call capabilities of a large language model. Vehicle control tools are predefined, digitally encapsulated modules with specific vehicle control functions. At the semantic level, vehicle control tools abstract the vehicle's underlying execution logic into standard interfaces that the large language model can understand, recognize, and call. For example, vehicle control tools may include vehicle body domain control (carControl), air conditioning control (airControl), command execution (cmd), music (MusicX), navigation (mapU), etc. Furthermore, vehicle control tools may include seat control, air conditioning control, refrigerator control, interior and exterior lighting control, energy management, audio control, rear-seat screen control, driving modes, connectivity functions, system applications, wireless charging, system settings, screen brightness, fragrance control, rearview mirror control, sunroof control, HUD control, trunk control, scene modes, vehicle lock control, wiper control, navigation control, multimedia control, and various intelligent agent controls, etc.

[0037] The vehicle control tool definition is a structured description of a vehicle control tool, designed to facilitate accurate understanding and invocation of the tool by large language models. For each vehicle control tool, the definition includes three parts: intent identifier, function description text, and parameter information.

[0038] The intent identifier is used to uniquely identify the intent name of the vehicle control function. For example, for an air conditioning control tool, the intent identifier could be airConditionerControl.

[0039] Functional description text is a natural language description of the applicable scenarios, functional effects, and semantic boundaries of a vehicle control tool. It is crucial for the large language model to read and understand. By understanding the functional description text of each vehicle control tool, the large language model can determine whether each tool can meet the user's vehicle control needs. For example, for an air conditioning control tool, the functional description text could be "used to adjust the interior temperature, turn the air conditioning on and off, adjust the fan speed, or defrost mode."

[0040] Parameter information defines the structured parameters that are required or optional when calling the vehicle control function. Parameter information may include parameter type, parameter description, and whether it is required. The parameter type defines the data format; for example, it can be a string, a number, or an enumeration. The parameter description is a natural language explanation of the parameter's specific meaning, such as "target temperature value." The "required" field indicates whether the parameter is mandatory or optional when executing the command.

[0041] In some embodiments, the vehicle control tool definition may also include a service name, which is the name of the service module to which the vehicle control tool belongs.

[0042] Slot description information is a standardized description format for the parameters required by vehicle control tools. The purpose of setting slot description information is to help large language models accurately extract the parameter slots needed to call vehicle control tools. Slot description information includes at least one of the following: slot name, slot type, and slot enumeration list. Specifically, the slot name (name) is the unique identifier of the slot; the slot type (type) is the data type of the slot, such as string, number, enumeration, etc.; the slot enumeration list (list_of_slots) includes a semantic description of each enumeration value (slots), used to constrain the range of slot values ​​and help large language models understand the meaning of each enumeration value.

[0043] In this embodiment of the invention, the vehicle control tool definition specifies the semantics of the functions implemented by the vehicle control tool, while the slot description information specifies the parameter details and value constraints required to implement the function. The vehicle control tool definition and the slot description information together constitute a structured tool manual, thereby exposing the vehicle's control capabilities to the large language model.

[0044] After obtaining the user text, it can be input into a large language model, along with vehicle control tool definitions and slot descriptions. The large language model can then generate invocation commands that reflect the explicit or implicit vehicle control intentions conveyed in the user text, constrained by these definitions and descriptions. For example, the large language model can understand the functions of each vehicle control tool through its definitions, thereby matching one or more suitable tools to the explicit or implicit vehicle control intentions conveyed in the user text. Based on this, and constrained by the slot descriptions, it can extract or infer the specific parameter values ​​required to invoke the matched vehicle control tools from the user text, thereby generating invocation commands to invoke these tools.

[0045] Specifically, providing the vehicle control tool definition and slot description information to the large language model can be achieved by carrying the vehicle control tool definition and slot description information when calling the function call interface of the large language model; or it can be achieved by configuring the vehicle control tool definition and slot description information to the tool context of the large language model. This embodiment of the invention does not specifically limit this.

[0046] Therefore, the invocation command output by the large language model can be obtained. This invocation command contains a defined intent identifier and pre-filled slot parameters, which can be used to invoke vehicle control tools.

[0047] In this embodiment of the invention, by applying structured vehicle control tool definitions and slot description information, combined with the powerful semantic understanding and function call capabilities of a large language model, non-standard expressions spoken by the user can be accurately converted into call commands for vehicle control tools, thereby greatly improving the freedom of in-vehicle voice interaction. Furthermore, for newly added in-vehicle control functions, only the vehicle tool definitions and slot description information need to be updated, without modifying the underlying logic, significantly reducing maintenance costs.

[0048] Step 130: Based on the invocation command, invoke the corresponding vehicle control tool to control the vehicle.

[0049] Specifically, after receiving the call command output by the large language model, the corresponding vehicle control tool can be directly invoked using the call command, thereby achieving vehicle control. For example, the corresponding vehicle control tool can be determined based on the intent identifier in the call command. By inputting the call command into the corresponding vehicle control tool, vehicle control actions such as adjusting the temperature, opening the sunroof, or playing music can be completed, thereby achieving precise control of the vehicle.

[0050] In the method provided by this invention, the semantic understanding and function call capabilities of a large language model are utilized to generate call commands representing the user's control intent under the constraints of structured vehicle control tool definitions and slot description information. This allows non-standard expressions uttered by the user to be accurately converted into call commands for vehicle control tools, greatly improving the freedom of in-vehicle voice interaction. The application of structured vehicle control tool definitions and slot description information helps the large language model perform accurate semantic alignment and parameter inference, effectively avoiding the hallucination output and command drift problems of large models in complex vehicle control scenarios. This significantly improves the understanding accuracy and execution security of in-vehicle voice interaction under complex linguistic phenomena such as multi-intent parallelism, referential resolution, and negation expressions. Furthermore, for newly added vehicle control functions, only the vehicle tool definitions and slot description information need to be updated, without modifying the underlying logic, greatly reducing maintenance costs.

[0051] Based on any of the above embodiments, in step 120, generating the invocation instruction corresponding to the user text based on the large language model, under the constraints of vehicle control tool definition and slot description information, includes: Based on the large language model, the user text is classified as a control intent. Based on the large language model, combined with the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated.

[0052] Specifically, after the user text is input into the large language model, the large language model can classify the control intent based on the user text. Here, the large language model classifies the control intent based on the user text, specifically by classifying the form of the vehicle control intent reflected in the user text.

[0053] In some embodiments, the control intent classification may include the following four intent types: precise, vague, free, and scenario-based.

[0054] In this context, "precise" refers to situations where the user uses standard, clear vehicle control commands, such as user text containing commands like "turn on the air conditioner" or "increase the volume to 50%." "Precise" also refers to situations where the user text directly carries a clear vehicle control intention and parameters. For control intentions categorized as "precise," the corresponding command can be directly matched to and executed from the user text.

[0055] Semi-natural expression refers to situations where user text contains specific control objects but uses non-standard, vague expressions, such as "the car windows are all fogged up" or "turn on the function that brakes when you release the accelerator." For this type of intent, the control intent and parameters need to be inferred by combining auxiliary elements. These auxiliary elements can include one or more of the following: context, modal particles, degree adverbs, user feelings, function status, usage scenarios, and descriptions of the vehicle's internal and external environment.

[0056] Fully natural expression refers to situations where user text does not contain a clear object of control, but only describes subjective feelings, states, or needs, such as "I'm a little hot" or "I feel a little stuffy." For this type of intent, large language models need to use semantic reasoning to achieve a semantic mapping from the user's feelings to specific vehicle control actions.

[0057] Scenario-based communication refers to a user's text describing a life scenario or situational intent, such as "getting ready to go home." Such intents typically correspond to multiple linked control actions, rather than a single function call. For example, the user text "getting ready to go home" might trigger multiple control actions such as navigating home, adjusting the seat massage, or playing soothing music.

[0058] The results of control intent classification determine the inference strategy and parameter extraction logic of the large language model when generating invocation commands. Based on different control intent classification results, the large language model can dynamically adjust the way it uses the vehicle control tool definition and slot description information, thereby generating invocation commands corresponding to the user text.

[0059] For example, when the user text is precise, the large language model can use a direct mapping strategy to generate invocation commands. Specifically, since the intent is clear, the large language model can directly extract the control intent from the user text and match it with the functions described in the vehicle control tool definition. It can also directly extract numerical values ​​or states from the user text and compare them with the slot types and required fields in the slot description information, thereby quickly generating high-confidence invocation commands.

[0060] For cases where user text is ambiguous, large language models can employ parameter inference strategies. Specifically, large language models can refer to the function description text defined in vehicle control tools and utilize parameter translation or fuzzy command inference capabilities to map non-standard words in user text to standard parameter values ​​defined in slot description information. For example, "full of fog" can be inferred as "defrost mode = on".

[0061] For user-generated text that is freely spoken, the large language model can employ a semantic mapping strategy. Specifically, the large language model can traverse the functional description text of each vehicle control tool in the vehicle control tool definition to find the vehicle control tool that best matches the user's experience. For example, when it recognizes "I'm a little hot," it can use logical reasoning to match the air conditioning control tool and then automatically generate a cooling command based on the current vehicle temperature.

[0062] For user text that is context-dependent, the large language model can employ a multi-tool orchestration strategy. That is, user text can trigger the large language model to invoke a combination of multiple vehicle control tools. Specifically, the large language model can generate a task sequence containing multiple intent identifiers by understanding the usage constraints of each vehicle control tool in its definition; for example, it can simultaneously generate multiple invocation commands such as navigation, seat massage, and music playback.

[0063] In the method provided in the embodiments of the present invention, by classifying the user text into control intents, the intent understanding and parameter inference are performed based on the results of the control intent classification, thereby improving the reliability and accuracy of the generation of calling instructions.

[0064] In related technologies, under unsafe conditions, in-vehicle voice interaction systems are prone to misidentifying non-vehicle control intentions such as pure environmental descriptions, psychological feelings, or interpersonal dialogues as vehicle control intentions, leading to vehicle malfunctions and directly affecting the safety of interaction during driving. To address this issue, based on any of the above embodiments, step 120, which involves classifying the user text for control intentions based on the large language model, includes: Based on the large language model, the user text is classified into control intents under the constraints of the definition of rejection intent; The definition of the denial intent includes a scenario description text that is not related to vehicle control intent.

[0065] Specifically, the definition of rejected intent refers to a set of pre-defined structured definitions used to describe non-vehicle control command scenarios or illegal control requests. The definition of rejected intent provides a clear negative constraint for the large language model, enabling it to distinguish between genuine vehicle control intent and non-vehicle control intent. Therefore, when classifying control intent based on user text, the large language model will not incorrectly identify user text that does not contain vehicle control intent and is merely casual conversation into any control intent category, thereby avoiding erroneous vehicle operation.

[0066] The definition of rejection intent includes scenario description text for various non-vehicle control intents. Here, scenario description text is used to describe various scenarios in which user text does not contain vehicle control intent. For example, the definition of rejection intent may include purely environmental description text, text expressing psychological feelings, interpersonal dialogue text, query text outside the vehicle control domain, safety-sensitive / unauthorized command text, erroneous / ambiguous command text, noise and invalid text, etc.

[0067] Among them, pure environmental description text refers to user text that only describes the objective environment outside the vehicle but does not contain any control intent, such as "There's a garbage truck outside," which only describes the external scene and does not contain any intent to control the windows. Psychological expression text refers to user text that only expresses emotions or states, such as "I'm in a bad mood" or "I feel very tired." Interpersonal dialogue text refers to user text that is a conversation between the user and other passengers in the vehicle, such as "Don't open the window, your dad will scold you." Non-vehicle control domain query text refers to user text that requests or queries a function that the vehicle will execute, or the content requested or queried by the user is not related to vehicle control, such as weather queries or encyclopedia questions. Safety-sensitive / unauthorized instruction text refers to user text involving instructions that may endanger driving safety or operations that the current user is not authorized to perform, such as requesting to open the trunk while driving. Error / ambiguous instruction text refers to user text where the intent and parameter combination is unreasonable, such as "Reverse the air conditioner," or user text containing unresolved semantic ambiguity, such as user text with multiple possible interpretations that cannot be resolved through context. Noise and invalid text refer to user text that is mistakenly triggered by voice noise, repeated invalid commands, incomplete commands, cross-domain mixed intents, negative confirmation rejections, test expressions, etc. For example, user text that is misidentified text that is not background music or noise and is not voice content; user text that is a repeated command when the function it intends to achieve is already in the target state; user text that is an incomplete expression that lacks key information and cannot be inferred; user text that is a mixed expression that contains both vehicle control and non-vehicle control intents; user text that is an instruction that the user has explicitly stated does not need to be executed; and user text that is a non-real control intent of the user testing the system's capabilities.

[0068] After receiving user text, the large language model can determine whether the user text falls into the category of rejection by understanding the scenario description text in the rejection intent definition. For example, when a user says "There is a garbage truck outside," the model recognizes this as a rejection scenario description that conforms to the "pure environment description" rather than a tool description of "window control." Therefore, it does not classify "There is a garbage truck outside" as a control intent or generate subsequent call instructions.

[0069] In the method provided in this embodiment of the invention, a complete rejection intention definition is constructed by the scene description text of non-vehicle control intention. Under the constraint of the rejection intention definition, the large language model can effectively and reliably reject non-vehicle control intentions, thereby avoiding the vehicle from executing incorrect control actions due to misidentification and ensuring driving safety.

[0070] Based on any of the above embodiments, in step 120, generating the invocation command corresponding to the user text based on the large language model, combined with the result of the control intent classification, the vehicle control tool definition, and the slot description information, includes: Based on the large language model, combined with the set of available processing strategies, the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated. The set of available processing strategies includes at least one of the following: synonym mapping strategy, intent decomposition strategy, context fusion strategy, and parameter inference strategy.

[0071] Specifically, the available processing strategy set refers to a predefined set of logical parsing rules and inference operators used to handle complex language phenomena in vehicles. The available processing strategy set aims to provide a standardized set of processing strategies for large language models. Thus, when faced with user text input in non-standard, incomplete, or complex natural language forms, the large language model can select one or more strategies from the available processing strategy set based on the user text itself and / or the results of control intent classification, to accurately convert the user text into invocation instructions that conform to the vehicle control protocol.

[0072] Furthermore, the set of available processing strategies includes at least one of the following: synonym mapping strategy, intent decomposition strategy, context fusion strategy, and parameter inference strategy.

[0073] The synonym mapping strategy is used to accurately map non-standard words in user text to standard entity names or action commands required by the vehicle control tool definition. Here, non-standard words can include colloquialisms, abbreviations, dialectal expressions, etc. The synonym mapping strategy includes at least one of noun translation and verb translation. Noun translation refers to mapping non-standard nouns in user text to standard vehicle control entities, such as mapping "cooling air" to "air conditioning"; verb translation refers to mapping non-standard verbs in user text to standard vehicle control actions, such as mapping "adjust" to "increase". The application of the synonym mapping strategy can eliminate ambiguity caused by the diversity of natural language and ensure the uniqueness of intent recognition.

[0074] Intent decomposition strategies are used to break down complex, long sentences containing multiple control intents into multiple independent, executable sequences of sub-intents. Here, intent decomposition strategies include at least one of parallel sentences, segmented sentences, sequential sentences, and scene translation. Parallel sentences handle user text containing multiple parallel vehicle control intents in a single expression, such as breaking "turn on the air conditioning and music" into two actions; segmented sentences integrate scattered information and handle instructions expressed in segments, such as "adjust the air conditioning temperature to 26 degrees"; sequential sentences handle multi-step instructions with a sequential order, such as "close the windows first, then turn on the air conditioning"; scene translation maps the scene description in the user text to a set of vehicle control actions, such as "turn on sleep mode" which can be mapped to the seat adjustment tool - recline, the window tool - close, and the media tool - play white noise. Intent decomposition strategies can avoid missing instructions or incorrect execution order when handling multiple concurrent intents and logical sequences.

[0075] Context fusion strategies are used to combine contextual information such as historical dialogue records or the current execution state to perform semantic completion or referential resolution on user text. Context fusion strategies include at least one of context overriding and context referential resolution. Context overriding handles situations where user text overrides or modifies parameters from the previous round of dialogue; for example, correcting preceding parameters when the user text is "No, lower it a little." Context referential resolution resolves pronouns and referential relationships in user text; for example, in "lower it," "it" refers to the previously mentioned "wind speed." The application of context fusion strategies enables in-vehicle voice interaction based on large language models to support natural multi-turn continuous dialogue.

[0076] Parameter inference strategies are used to infer specific control parameters based on preset logic, vehicle status, or user perception when the user does not provide specific numerical values ​​or the expression is vague. Parameter inference strategies can include at least one of parameter translation, parameterless inference, negation sentences, and fuzzy command inference. Specifically, parameter translation maps fuzzy parameters in user text to specific numerical values, for example, mapping "a little bigger" to "+10%"; parameterless inference infers reasonable parameters based on the current state and context when the user text does not provide parameters; negation sentences handle user text containing negative words, for example, mapping "don't be too cold" to "raise the temperature by 2℃"; and fuzzy command inference performs semantic inference on completely vague user text, for example, inferring "I'm a little hot" as "lower the air conditioning temperature". The application of parameter inference strategies achieves a leap from vague perceptual descriptions to concrete quantitative execution, greatly improving the intelligence level of in-vehicle voice interaction.

[0077] The method provided in this invention effectively solves the problem of complex language phenomena such as reference, negation, and sequence, which are difficult to handle in traditional natural language processing scenarios, by breaking down the complex understanding process into targeted strategy applications. Therefore, even when faced with ambiguous user expressions, the large language model can provide reasonable invocation commands, greatly improving the freedom and reliability of in-vehicle voice interaction.

[0078] Based on any of the above embodiments, in step 120, generating the invocation instruction corresponding to the user text based on the large language model, under the constraints of vehicle control tool definition and slot description information, includes: At least one of the following: interaction history information, vehicle information, environmental information, and user information, as well as the user text, is input into the large language model to obtain the calling instructions output by the large language model under the constraints of the vehicle control tool definition and the slot description information.

[0079] Specifically, interaction history information can include at least one of dialogue history and execution history. For example, if a user first says "set the air conditioner to 26 degrees" and then immediately says "a little higher", then "set the air conditioner to 26 degrees" can be used as the interaction history information for "a little higher". Combining the interaction history information, it can be determined that the temperature is being increased from 26 degrees.

[0080] Vehicle information refers to the current physical status data of a vehicle. Vehicle information can include the on / off status of various hardware components of the vehicle (window opening, light status), current values ​​(current wind speed, current volume level), and fault status, etc. For example, vehicle information can include the on / off status of windows, the on / off status of lights, the current temperature and current wind speed of the air conditioner, and the current volume level of the audio system.

[0081] Environmental information refers to sensor data related to the inside and outside of the vehicle or the external environment. Environmental information can include outdoor temperature, weather conditions, air quality inside the vehicle, current vehicle location, and road conditions.

[0082] Understandably, combining vehicle information with environmental information can reflect the vehicle's internal and external status.

[0083] User information refers to a user's personalized information, which can include user profiles, user behavior, and user habits. For example, user information may include a user's preferred temperature, frequently listened-to music style, and commonly used navigation addresses.

[0084] At least one of the aforementioned interaction history information, vehicle information, environmental information, and user information can be input into the large language model along with the user text. Therefore, at least one of these information can provide the large language model with real-time contextual information, thus providing a reference for the large language model to perform semantic understanding and parameter inference based on the user text.

[0085] In the method provided in this embodiment of the invention, by inputting at least one of the following—interaction history information, vehicle information, environmental information, and user information—along with user text into the large language model, the large language model is transformed from a pure text processing tool into an intelligent agent with scene awareness capabilities. This greatly improves the reliability and accuracy of the call command output based on the large language model, thereby ensuring the reliability and accuracy of vehicle control.

[0086] Based on any of the above embodiments, the vehicle control method further includes: Obtain the execution result of the invocation instruction; Based on the large language model, and combined with the execution result, interactive text or a new round of calling instructions are generated under the constraints of the vehicle control tool definition and the slot description information.

[0087] Specifically, in practical applications, vehicle control is often not an isolated, single action, but a dynamic process of issuing a command, receiving feedback on the result, and making further decisions. In this embodiment of the invention, the execution result of the command can be obtained, and secondary reasoning can be performed using a large language model, thereby achieving a more intelligent interactive experience.

[0088] The execution result of the call instruction can include whether the execution was successful or failed, or it can include the current status parameter after the call instruction is completed, such as the sunroof opening / closing status after the sunroof is opened.

[0089] After obtaining the execution result of the invocation command, the result can be input into the large language model. The large language model can then combine the execution result and interaction history information to generate interactive text that conforms to natural language logic, or generate an invocation command for the next round of control that matches the user's vehicle control intent conveyed in the text.

[0090] For example, if a user's text is "I'm a little hot," the large language model outputs a command to invoke the air conditioning plugin to lower the temperature to 20°C. The execution result of this command is that the air conditioning has been turned on and started cooling. This result is then input into the large language model, which outputs the interactive text "The air conditioning has been turned on and set to 20 degrees Celsius for you. It's very sunny now; would you like us to close the sunshade as well?"

[0091] For example, if a user's text is "Check the car windows for me. If they're not closed properly, close them all and then turn on the rapid cooling mode," the large language model outputs a first-round command to check the window status. Then, it receives the execution result of this command, which is "The left rear window is open." This execution result is input into the large language model, which determines that the result matches the condition "not closed properly" in the user's text. Under the constraints defined by the vehicle control tool, it generates a second-round command to close the window. After receiving the execution result that the window is successfully closed, the large language model generates a third-round command to adjust the air conditioning to rapid cooling mode.

[0092] In the method provided in this embodiment of the invention, by feeding back the execution result of the calling instruction to the large language model, it is effectively ensured that in continuous dialogue or chained execution scenarios, the large language model will not deviate from the actual capabilities of the vehicle due to the complexity of the context.

[0093] Furthermore, the method provided in this embodiment of the invention can be applied to multi-round instruction arrangement in complex control scenarios, thereby realizing the linkage control of multiple vehicle control tools.

[0094] Based on any of the above embodiments, the parameter information includes at least one of the parameter type, parameter description text, and parameter necessity of the parameters required by the vehicle control tool; The slot description information includes at least one of the slot identifier, slot type, and slot enumeration list for the slot corresponding to the parameters required by the vehicle control tool.

[0095] Specifically, parameter information refers to the definition of structured parameters that are required or optional when invoking vehicle control functions. Parameter information may include parameter type, parameter description, and parameter necessity. The parameter type defines the data format; for example, a parameter type can be a string, a number, or an enumeration. The parameter description is a natural language explanation of the parameter's specific meaning, such as "target temperature value." Parameter necessity indicates whether the parameter is required or optional when executing the command.

[0096] Correspondingly, slot description information is a standardized description format for the parameters required by vehicle control tools. Slot description information includes at least one of the following: slot name, slot type, and slot enumeration list. Specifically, the slot name (name) is the unique identifier of the slot; the slot type (type) is the data type of the slot, such as string, number, enumeration, etc.; the slot enumeration list (list_of_slots) includes a semantic description of each enumeration value (slots), used to constrain the range of slot values ​​and help the large language model understand the meaning of each enumeration value.

[0097] Based on any of the above embodiments, step 110, obtaining user text, includes: Receive user text sent from the vehicle terminal, wherein the user text is sent when it is determined to be a complex intent instruction.

[0098] Specifically, assuming that the execution subject of the vehicle control method provided in this embodiment of the invention is the server, then obtaining user text can specifically be receiving user text sent by the vehicle.

[0099] The vehicle can directly collect user voice and perform speech recognition to obtain user text. Based on this, the vehicle can also classify the user text into simple intent commands or complex intent commands. Here, simple intent commands refer to simple, high-frequency vehicle control commands; complex intent commands refer to vehicle control commands that require semantic reasoning, are ambiguous in meaning, and are complex in expression.

[0100] When the user text is a simple intent command, the vehicle can process the command directly locally without transmitting the user text to the server. As a result, the response speed of vehicle control based on simple intent commands is guaranteed.

[0101] For cases where user text represents complex intent commands, the vehicle does not process the commands locally but instead transmits the user text to the server. The server receives the user text from the vehicle and uses a large language model to perform semantic understanding and parameter inference, thereby obtaining the command output by the large language model. This command is then returned to the vehicle, which uses it for vehicle control. This ensures the reliability and accuracy of vehicle control based on complex intent commands.

[0102] In the method provided in this embodiment of the invention, by classifying whether the user text is a complex intent instruction, a collaborative mechanism between the vehicle end and the server end in the vehicle control scenario is realized. This ensures response speed while giving full play to the complex reasoning ability of the server end model, thereby ensuring high accuracy and low latency in vehicle control.

[0103] Based on any of the above embodiments Figure 2 This is the second flowchart illustrating the vehicle control method provided by this invention. For example... Figure 2 As shown, the method includes: The vehicle acquires user input. This user input can be voice or text obtained through speech recognition. The vehicle then inputs this user input into a locally deployed traditional semantic intervention system, thereby performing semantic intervention based on the user input. This semantic intervention may include determining whether the user input is a simple or complex intent command.

[0104] Based on this, when the user input is a simple intent command, the vehicle directly processes the user input; when the user input is a complex intent command, the vehicle sends the user text to the server.

[0105] Deploy a large language model on the server side. After receiving user text, the server can input the user text, along with interaction history, vehicle information, environmental information, and user information, into the large language model. The large language model can combine the above inputs with the tool manual containing vehicle control tool definitions and slot descriptions to perform semantic understanding and parameter reasoning on the user text. This allows it to determine the vehicle control tool to be invoked from various vehicle control tools, including contextual tools, semantic tools, and casual conversation tools, and generate invocation instructions for invoking the vehicle control tool.

[0106] Based on this, vehicle control tools can be invoked using command calls, thereby achieving vehicle control. Furthermore, the execution results of the vehicle control tools can be fed back to the large language model, thereby enabling multi-round vehicle control or outputting control results and sending them to the vehicle.

[0107] Here, the large language model can be understood as an intelligent dialogue manager that supports function call mechanisms. In addition to generating call instructions, it is responsible for implementing functions such as dialogue state tracking, context management, and multi-turn dialogue processing.

[0108] In the method provided by this invention, the semantic understanding and function call capabilities of a large language model are utilized to generate call commands representing the user's control intent under the constraints of structured vehicle control tool definitions and slot description information. This allows non-standard expressions uttered by the user to be accurately converted into call commands for vehicle control tools, greatly improving the freedom of in-vehicle voice interaction. The application of structured vehicle control tool definitions and slot description information helps the large language model perform accurate semantic alignment and parameter inference, effectively avoiding the hallucination output and command drift problems of large models in complex vehicle control scenarios. This significantly improves the understanding accuracy and execution security of in-vehicle voice interaction under complex linguistic phenomena such as multi-intent parallelism, referential resolution, and negation expressions. Furthermore, for newly added vehicle control functions, only the vehicle tool definitions and slot description information need to be updated, without modifying the underlying logic, greatly reducing maintenance costs.

[0109] The vehicle control device provided by the present invention is described below. The vehicle control device described below can be referred to in correspondence with the vehicle control method described above.

[0110] Figure 3 This is a structural schematic diagram of the vehicle control device provided by the present invention, as shown below. Figure 3 As shown, the device includes: Acquisition unit 310 is used to acquire user text; The generation unit 320 is used to generate the call instruction corresponding to the user text based on the large language model and under the constraints of the vehicle control tool definition and slot description information. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool, and the slot description information corresponds to the parameter information. The control unit 330 is used to invoke the corresponding vehicle control tool to control the vehicle based on the invocation command.

[0111] In the device provided in this embodiment of the invention, the semantic understanding and function call capabilities of a large language model are utilized to generate call commands representing the user's control intent, under the constraints of structured vehicle control tool definitions and slot description information. This allows non-standard expressions uttered by the user to be accurately converted into call commands for vehicle control tools, greatly improving the freedom of in-vehicle voice interaction. The application of structured vehicle control tool definitions and slot description information helps the large language model perform accurate semantic alignment and parameter inference, effectively avoiding the hallucination output and command drift problems of large models in complex vehicle control scenarios. This significantly improves the understanding accuracy and execution security of in-vehicle voice interaction under complex linguistic phenomena such as multi-intent parallelism, referential resolution, and negation expressions. Furthermore, for newly added vehicle control functions, only the vehicle tool definitions and slot description information need to be updated, without modifying the underlying logic, greatly reducing maintenance costs.

[0112] Based on any of the above embodiments, the generating unit is specifically used for: Based on the large language model, the user text is classified as a control intent. Based on the large language model, combined with the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated.

[0113] Based on any of the above embodiments, the generating unit is specifically used for: Based on the large language model, the user text is classified into control intents under the constraints of the definition of rejection intent; The definition of the denial intent includes a scenario description text that is not related to vehicle control intent.

[0114] Based on any of the above embodiments, the generating unit is specifically used for: Based on the large language model, combined with the set of available processing strategies, the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated. The set of available processing strategies includes at least one of the following: synonym mapping strategy, intent decomposition strategy, context fusion strategy, and parameter inference strategy.

[0115] Based on any of the above embodiments, the generating unit is specifically used for: At least one of the following: interaction history information, vehicle information, environmental information, and user information, as well as the user text, is input into the large language model to obtain the calling instructions output by the large language model under the constraints of the vehicle control tool definition and the slot description information.

[0116] Based on any of the above embodiments, the generating unit is further configured to: Obtain the execution result of the invocation instruction; Based on the large language model, and combined with the execution result, interactive text or a new round of calling instructions are generated under the constraints of the vehicle control tool definition and the slot description information.

[0117] Based on any of the above embodiments, the parameter information includes at least one of the parameter type, parameter description text, and parameter necessity of the parameters required by the vehicle control tool; The slot description information includes at least one of the slot identifier, slot type, and slot enumeration list for the slot corresponding to the parameters required by the vehicle control tool.

[0118] Based on any of the above embodiments, the acquisition unit is specifically used for: Receive user text sent from the vehicle terminal, wherein the user text is sent when it is determined to be a complex intent instruction.

[0119] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a vehicle control method, which includes: Get user text; Based on a large language model, under the constraints of vehicle control tool definition and slot description information, a call command corresponding to the user text is generated. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool. The slot description information corresponds to the parameter information. The vehicle is controlled by invoking the corresponding vehicle control tool based on the invocation command.

[0120] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the vehicle control method provided by the above methods, the method comprising: Get user text; Based on a large language model, under the constraints of vehicle control tool definition and slot description information, a call command corresponding to the user text is generated. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool. The slot description information corresponds to the parameter information. The vehicle is controlled by invoking the corresponding vehicle control tool based on the invocation command.

[0122] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the vehicle control methods provided by the methods described above, the method comprising: Get user text; Based on a large language model, under the constraints of vehicle control tool definition and slot description information, a call command corresponding to the user text is generated. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool. The slot description information corresponds to the parameter information. The vehicle is controlled by invoking the corresponding vehicle control tool based on the invocation command.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle control method, characterized in that, include: Get user text; Based on a large language model, under the constraints of vehicle control tool definition and slot description information, a call command corresponding to the user text is generated. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool. The slot description information corresponds to the parameter information. The vehicle is controlled by invoking the corresponding vehicle control tool based on the invocation command.

2. The vehicle control method according to claim 1, characterized in that, The process of generating the invocation command corresponding to the user text based on a large language model, under the constraints of vehicle control tool definition and slot description information, includes: Based on the large language model, the user text is classified as a control intent. Based on the large language model, combined with the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated.

3. The vehicle control method according to claim 2, characterized in that, The process of classifying the user text for control intent based on the large language model includes: Based on the large language model, the user text is classified into control intents under the constraints of the definition of rejection intent; The definition of the denial intent includes a scenario description text that is not related to vehicle control intent.

4. The vehicle control method according to claim 2, characterized in that, Based on the large language model, and combining the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated, including: Based on the large language model, combined with the set of available processing strategies, the results of the control intent classification, the vehicle control tool definition, and the slot description information, the call command corresponding to the user text is generated. The set of available processing strategies includes at least one of the following: synonym mapping strategy, intent decomposition strategy, context fusion strategy, and parameter inference strategy.

5. The vehicle control method according to claim 1, characterized in that, The process of generating the invocation command corresponding to the user text based on a large language model, under the constraints of vehicle control tool definition and slot description information, includes: At least one of the following: interaction history information, vehicle information, environmental information, and user information, as well as the user text, is input into the large language model to obtain the calling instructions output by the large language model under the constraints of the vehicle control tool definition and the slot description information.

6. The vehicle control method according to claim 1, characterized in that, Also includes: Obtain the execution result of the invocation instruction; Based on the large language model, and combined with the execution result, interactive text or a new round of calling instructions are generated under the constraints of the vehicle control tool definition and the slot description information.

7. The vehicle control method according to any one of claims 1 to 6, characterized in that, The parameter information includes at least one of the following: parameter type, parameter description text, and parameter necessity of the parameters required by the vehicle control tool; The slot description information includes at least one of the slot identifier, slot type, and slot enumeration list for the slot corresponding to the parameters required by the vehicle control tool.

8. The vehicle control method according to any one of claims 1 to 6, characterized in that, The acquisition of user text includes: Receive user text sent from the vehicle terminal, wherein the user text is sent when it is determined to be a complex intent instruction.

9. A vehicle control device, characterized in that, include: The acquisition unit is used to acquire user text; The generation unit is used to generate the call instruction corresponding to the user text based on the large language model and under the constraints of the vehicle control tool definition and slot description information. The vehicle control tool definition includes the intent identifier, function description text and parameter information of the vehicle control tool, and the slot description information corresponds to the parameter information. The control unit is used to invoke the corresponding vehicle control tools to control the vehicle based on the invocation command.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the vehicle control method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle control method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the vehicle control method as described in any one of claims 1 to 8.

Images