Remote vehicle interaction method, apparatus and device, and storage medium

Abstract

A remote vehicle interaction method, apparatus and device, and a storage medium. The method comprises: receiving natural language text sent by a user terminal; by means of a pre-configured first large language model, performing recognition on the natural language text, so as to obtain a first user intention; by means of a pre-configured second large language model, determining control information on the basis of the first user intention; and controlling a vehicle to execute a control operation on the basis of the control information, and sending to the user terminal a reply corresponding to the natural language text.

Description

Off-vehicle interaction methods, devices, equipment and storage media Cross-reference to related applications This application claims priority to Chinese patent application No. 202510471908.9, filed on April 15, 2025, which is incorporated herein by reference. Technical Field

[0001] This disclosure pertains to the field of vehicle technology, and particularly relates to an off-vehicle interaction method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of vehicle technology, vehicles have become an indispensable tool for people's daily travel. To meet the growing demand for intelligent and connected vehicles, vehicles are equipped with an increasing number of human-vehicle interaction functions. Among these, off-vehicle interaction, as a crucial function, allows users to interact with the vehicle remotely via a user terminal when they are away from the vehicle. However, off-vehicle interaction functions in related technologies are usually limited to single commands and feedback, resulting in relatively poor human-vehicle interaction. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for off-vehicle interaction, which can improve the interactivity between people and vehicles in off-vehicle interaction functions.

[0004] In a first aspect, embodiments of this disclosure provide an off-vehicle interaction method, the method comprising: receiving natural language text sent by a user terminal; recognizing the natural language text using a pre-configured first major language model to obtain a first user intent; determining control information based on the first user intent using a pre-configured second major language model, wherein the control information is associated with the natural language text; controlling a vehicle to perform a control operation based on the control information; and sending a response corresponding to the natural language text to the user terminal, wherein the response is used to indicate the control result of the control operation.

[0005] Secondly, embodiments of this disclosure also provide an off-vehicle interaction device, comprising: a text receiving module configured to receive natural language text sent by a user terminal; an intent recognition module configured to identify a first user intent through a pre-configured first large language model; a control information determination module configured to determine control information based on the first user intent through a pre-configured second large language model, wherein the control information is associated with the natural language text; and a control module configured to control a vehicle to perform control operations based on the control information, and to send a response corresponding to the natural language text to the user terminal, wherein the response is used to indicate the control result of the control operation.

[0006] Thirdly, embodiments of this disclosure also provide an off-vehicle interaction device, the off-vehicle interaction device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the off-vehicle interaction method as described in any of the first aspects.

[0007] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the vehicle-to-vehicle interaction method as described in any one of the first aspects.

[0008] Fifthly, embodiments of this disclosure also provide a computer program product, wherein instructions in the computer program product, when executed by the processor of the off-vehicle interaction device, cause the off-vehicle interaction device to implement the off-vehicle interaction method as described in any one of the first aspects.

[0009] In this embodiment, natural language text sent by a user terminal is received; the natural language text is input into a pre-configured first large language model, and a first user intent is identified through the first large language model; the vehicle is controlled to perform control operations based on the control information; and a response corresponding to the natural language text is sent to the user terminal. Thus, the large language model can more intelligently and accurately identify the user's target intent and determine the corresponding control information, thereby enabling more flexible control of the vehicle to perform the user's required control operations and provide corresponding responses, rather than being limited to single instructions and feedback, thereby improving the interactivity between humans and vehicles. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments of this disclosure will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 is a flowchart illustrating an embodiment of the off-vehicle interaction method provided in this disclosure.

[0012] Figure 2 is a flowchart illustrating the intent processing operation in an embodiment of the off-vehicle interaction method provided in this disclosure.

[0013] Figure 3 is a flowchart illustrating an application example of the off-vehicle interaction method provided in this disclosure.

[0014] Figure 4 is a structural schematic diagram of an embodiment of the off-vehicle interaction device provided in this disclosure.

[0015] Figure 5 is a structural schematic diagram of an embodiment of the off-vehicle interaction device provided in this disclosure. Detailed Implementation

[0016] The features and exemplary embodiments of various aspects of this disclosure will now be described in detail. To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, will provide a further detailed description of this disclosure. It should be understood that the specific embodiments described herein are intended only to explain this disclosure and not to limit it. For those skilled in the art, this disclosure can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this disclosure by illustrating examples.

[0017] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0018] Figure 1 is a flowchart illustrating an embodiment of the vehicle-to-vehicle interaction method provided in this disclosure. The vehicle-to-vehicle interaction method of this disclosure can be applied to vehicle-to-vehicle interaction devices, which can be vehicles or electronic devices. The electronic device can be an in-vehicle terminal or a terminal device, server (e.g., a cloud server, etc., independent of the vehicle). As shown in Figure 1, the vehicle-to-vehicle interaction method includes, but is not limited to, the following steps S101 to S104.

[0019] Step S101: Receive natural language text sent by the user terminal.

[0020] In this step, if a user needs to interact with the vehicle through their user terminal after leaving the vehicle, the user can send the aforementioned natural language text to the off-vehicle interaction device through the user terminal.

[0021] The aforementioned target natural language can be natural language text obtained by receiving a preset operation input by the user based on the off-vehicle interaction interface when the user terminal displays the off-vehicle interaction interface, and obtaining the natural language text based on the preset operation. The preset operation can be at least one of voice operation, air gesture operation, and touch operation.

[0022] For example, the user terminal could receive voice input from the user when triggering the voice input control in the vehicle exit interaction interface (e.g., the user inputs "open the car window"), perform voice content recognition on the voice, and generate natural language text corresponding to the voice; or, the user terminal could receive natural language text input by the user through touch operation in the text input component of the vehicle exit interaction interface, and so on.

[0023] Step S102: Recognize the natural language text using a pre-configured first large language model to obtain the first user intent.

[0024] In this step, the aforementioned first large language model can be any pre-trained model capable of recognizing user intent based on natural language text. For example, the aforementioned first large language model can include at least one of the large language models such as Qwen, ChatGPT, and Llama.

[0025] Training the first large language model can include: obtaining a training set, which includes a certain number of training samples, each training sample including a preset natural language text and a user intent label, the user intent label being a manually labeled user intent corresponding to the preset natural language text; inputting the training samples in the training set into the initial large language model, iteratively training the initial large model to obtain the first large language model, wherein, during the training process, the preset natural language text is used as the input of the initial large language model, and the initial large language model adjusts the network parameters according to the loss between its output and the user intent label until the loss meets a preset condition (e.g., tends to be stable and unchanged).

[0026] It should be noted that the aforementioned first user intent may contain only one user intent. For example, if the natural language text is "open the car window", the first user intent may only include the intent to open the car window. Of course, the aforementioned first user intent may also contain multiple user intents. For example, if the natural language text is "open the car window and air conditioner", the first user intent may include the intent to open the car window and the intent to turn on the air conditioner.

[0027] Step S103: Determine control information based on the first user intent using a pre-configured second language model, wherein the control information is associated with the natural language text.

[0028] In this step, the aforementioned second major language model can be any pre-trained model capable of predicting vehicle control information based on user intent. For example, the aforementioned second major language model can also include at least one of the major language models such as Generalized Questions (e.g., Qwen2.5-7B), ChatGPT, and Llama.

[0029] It should be noted that the second language model mentioned above and the first language model mentioned above can be the same large language model, that is, the large language model can both recognize user intent and predict vehicle control information; or, the two can be different large language models, and the training process of the second language model can be similar to that of the first language model, which will not be elaborated here.

[0030] The aforementioned control information can be information that enables the vehicle to perform any control operation. For example, if the first user intent includes the intent to open the window, the control information may include the window opening degree; or, if the first user intent also includes the intent to turn on the air conditioning, the control information may also include the air conditioning temperature, and so on.

[0031] Step S104: Control the vehicle to perform control operations based on the control information, and send a response corresponding to the natural language text to the user terminal, wherein the response is used to indicate the control result of the control operation.

[0032] In this step, the off-vehicle interaction device can generate a control command carrying the aforementioned control information based on the control information, and control the vehicle to perform corresponding control operations based on the control command.

[0033] For example, if the control information includes the opening degree of the window, a window control command including the window opening degree can be generated, and the window opening degree corresponding to the window opening degree can be controlled by the window control command; or, if the control information also includes the air conditioning temperature, an air conditioning control command including the air conditioning temperature can be generated, and the air conditioning can be controlled to turn on to the air conditioning temperature.

[0034] When the aforementioned vehicle-to-vehicle interaction system completes a control operation, it can generate a response corresponding to natural language text based on the control result of the operation, and send the response to the user terminal so that the user terminal can output the response, thereby realizing the vehicle-to-vehicle interaction function.

[0035] The aforementioned user terminal outputs a response, which may be displayed on the vehicle exit interaction interface when the user terminal displays such an interface; or the response may be played by the user terminal via voice broadcast.

[0036] In this embodiment, natural language text sent by a user terminal is received; the natural language text is input into a pre-configured first large language model, and a first user intent is identified through the first large language model; the vehicle is controlled to perform control operations based on the control information; and a response corresponding to the natural language text is sent to the user terminal. Thus, the large language model can more intelligently and accurately identify the user's target intent and determine the corresponding control information, thereby enabling more flexible control of the vehicle to perform the user's required control operations and provide corresponding responses, rather than being limited to single instructions and feedback, thereby improving the interactivity between humans and vehicles.

[0037] In some implementations, before inputting the natural language text into a pre-configured first large language model and identifying the user intent through the first large language model, the method further includes: querying a preset Frequently Asked Questions (FAQ) database based on the natural language text to obtain query results; and if the query results indicate that a second user intent matching the natural language text has been found, determining control information based on the second user intent.

[0038] If the query result indicates that no user intent matching the natural language text is found, the natural language text is input into a pre-configured first large language model, and the first user intent is identified through the first large language model.

[0039] In this embodiment, user intent can be queried through the FAQ database first. If a second user intent matching the natural language text is found in the FAQ database, control information is determined based on the second user intent. If no matching user intent is found in the FAQ database, the first user intent is identified using the first large language model. This ensures the efficiency of FAQ intent recognition and the flexibility of the large language model, and ensures the stability and accurate response of intent recognition in different scenarios.

[0040] Based on natural language text, in a preset frequent question and answer database, the user intent that matches the natural language text can be queried. This can be done by the off-vehicle interaction device extracting keywords from the natural language text and querying the FAQ database for user intent that matches the extracted keywords.

[0041] For example, assuming the FAQ database contains user intent data for "charging station search", when the natural language text is "Where are the charging stations nearby?", the off-vehicle interaction device can use the keyword "charging station" to search for "charging station search" as a second user intent.

[0042] It should be noted that, in the case of determining the second user intent, the off-vehicle interaction device can further determine control information based on the second user intent through the aforementioned second language model, and control the vehicle to perform corresponding control operations and send responses corresponding to natural language text based on the control information.

[0043] Alternatively, the FAQ database may also contain a mapping between user intents and control commands. When the intent of the second user is determined, the off-vehicle interaction device may determine the control command corresponding to the second user based on the FAQ database, and control the vehicle to perform control operations and send responses corresponding to natural language text based on the control command.

[0044] For example, if the above query finds "charging station query" as the second user intent, the off-vehicle interaction system can further determine the battery level query command through the FAQ database. Based on this command, the system controls the vehicle to query the battery level and sends a response based on the queried battery level. If the battery level is less than 30%, the system replies to the user terminal, "Hey, low battery! Don't worry, a nearby charging station has been arranged for you!" and launches the charging station mini-program; otherwise, the system replies to the user terminal, "A nearby charging station has been found for you."

[0045] In some implementations, the step of identifying the first user intent through the first large language model includes: when the natural language text is natural language text in a multi-turn dialogue process, obtaining dialogue information of at least one round of historical dialogue associated with the natural language text in the multi-turn dialogue; and identifying the dialogue information of the at least one round of historical dialogue and the natural language text by the first large language model to obtain the first user intent.

[0046] In this embodiment, during multi-turn dialogue, the off-vehicle interaction device can obtain dialogue information of at least one round of historical dialogue associated with the natural language text of the current round of dialogue through the first large language model, and identify the dialogue information of at least one round of historical dialogue and natural language text to obtain the first user intent. Thus, based on context awareness, it can understand the semantic changes and associations of the user in continuous dialogue, more accurately identify the user intent, and provide control commands that are more in line with actual needs.

[0047] In this embodiment, the aforementioned multi-turn dialogue can be understood as the user terminal sending natural language text to the off-vehicle interaction device multiple times in succession, and the off-vehicle interaction device performing corresponding control operations and providing corresponding responses for each natural language text response.

[0048] For example, the aforementioned multi-round dialogue may include: in the first round of dialogue, the user terminal sends the natural language text "Please help turn on the seat heating," the off-vehicle interaction device determines the user's intention to "turn on the seat heating function," and controls the vehicle to perform the seat heating operation and reply with the corresponding response; in the second round of dialogue, the user terminal sends the natural language text "The same goes for the windows," the off-vehicle interaction device determines the user's intention to "open the windows," and controls the vehicle to perform the window opening operation and reply with the corresponding response; in the third round of dialogue, the user terminal sends the natural language text "The same goes for the air conditioning."

[0049] The aforementioned acquisition of at least one round of historical dialogue associated with the natural language text can be achieved by determining the dialogue of a predetermined number of rounds preceding the natural language text as the at least one round of historical dialogue. For example, the predetermined number of rounds can be 3 rounds, meaning the first 3 rounds of dialogue are determined as historical dialogue. Each round of historical dialogue can include natural language text and the corresponding user intent.

[0050] The above-mentioned identification of the dialogue information and natural language text of the at least one round of historical dialogue by the first large language model can be achieved by combining the natural language text and natural language text in the at least one round of historical dialogue, and inputting the combined natural language text into the first large language model, which then outputs the target user intent; or, it can be achieved by inputting the user intent and natural language text of the at least one round of historical dialogue into the first large language model, which then outputs the target user intent. There is no limitation on this.

[0051] For example, if the user terminal sends the natural language text "The air conditioning is the same" in the third round of dialogue, the off-vehicle interaction can identify the user intent of "turn on the air conditioning" corresponding to the natural language text "The air conditioning is the same" based on the natural language text or user intent in the first two rounds of dialogue, and control the vehicle to perform the air conditioning operation and reply with the corresponding response.

[0052] In some implementations, before determining control information based on the first user intent using a pre-configured second language model, the method further includes: matching the first user intent with a preset special user intent to obtain a first matching result; and, if the first matching result indicates that the first user intent matches a target special user intent, performing an intent processing operation corresponding to the target special user intent to obtain a third user intent, wherein the third user intent is more consistent with the actual operating logic of the vehicle than the first user intent.

[0053] The step of determining control information based on the first user intent using a pre-configured second major language model includes: determining control information based on the third user intent using a pre-configured second major language model.

[0054] In this embodiment, the off-vehicle interaction device can perform intent processing operations on the identified first user intent for specific user intents, so that the processed first user intent is more in line with the actual operation logic of the vehicle, thereby improving the accuracy of the processed first user intent.

[0055] The aforementioned target-specific user intents can be pre-set user intents, which may include user intents that are relatively complex to perform or whose intents are not very obvious. For example, the aforementioned target-specific user intents may include user intents such as "schedule vehicle preparation," "rapid heating," or "rapid cooling."

[0056] The above-mentioned execution of intent processing operations corresponding to the target special user intent to obtain the processed first user intent may be based on the configuration of corresponding intent processing operation strategies for each target special user intent. The intent processing operation strategy is used to perform intent processing operations on the first user intent, so that the processed third user intent is more in line with the actual operation logic of the vehicle.

[0057] For example, there could be preset user intents that are more in line with actual operation logic, configured for each target-specific user intent. If the first user intent is determined to be a target-specific user intent, the first user intent can be replaced with a preset user intent that corresponds to the target user intent, and so on.

[0058] In some implementations, the execution of the intent processing operation corresponding to the target specific user intent includes: determining at least one user sub-intent corresponding to the target specific user intent; matching the natural language text with each of the user sub-intents to obtain a second matching result, the second matching result being used to indicate whether there is a preset text in the natural language text corresponding to the user sub-intent; based on the second matching result, removing the target user sub-intent from the at least one user sub-intent to obtain a third user intent, wherein the target user sub-intent is a user sub-intent whose corresponding preset text does not exist in the natural language text.

[0059] In this embodiment, the off-vehicle interaction device can take at least a portion of the user sub-intents after removing the target user sub-intent as the third user intent, thereby enabling the removal of user sub-intents that are not related to natural language text from the at least one user sub-intents, making the third user intent more reasonable.

[0060] The determination of at least one user sub-intention corresponding to the target specific user intention can be achieved by splitting the target specific intention to obtain the at least one user sub-intention. Specifically, the off-vehicle interaction device may have pre-configured user sub-intentions corresponding to each specific user intention. When it is determined that the first user intention matches the target specific user intention, the off-vehicle interaction device determines the user sub-intention corresponding to the target specific user intention as the at least one user sub-intention.

[0061] For example, if the first user intent mentioned above matches the special user intent "reserve a car", and "reserve a car" is pre-configured with two corresponding user sub-intents "navigation, I want to get in the car", then the two user sub-intents "navigation, I want to get in the car" corresponding to "reserve a car" can be obtained.

[0062] The above-mentioned method of removing the target user sub-intent from the at least one user sub-intent based on the second matching result to obtain the third user intent can be as follows: if the second matching result indicates that there is a preset text corresponding to the target user sub-intent that does not exist in the natural language text, then the target user sub-intent is removed from the at least one user sub-intent, and the remaining user sub-intent after removing the at least one user sub-intent is determined as the third user intent; of course, if the second matching result indicates that there is no target user sub-intent, then at least one user sub-intent is determined as the third user intent.

[0063] For example, as shown in Figure 2, if the intent of "Reservation for Car Preparation" is split into two user sub-intents, "Navigation, I want to get in the car", the off-vehicle interaction device can first determine whether the natural language text contains the text "location" related to the user sub-intent "Navigation". If the natural language text contains the text "location", the user sub-intent "Navigation" is removed. If the natural language text does not contain the text "location", it further determines whether the natural language text contains the text "car" related to the user sub-intent "I want to get in the car". If the natural language text contains the text "car", the user sub-intent "I want to get in the car" is removed. If the natural language text also does not contain the text "car", the result "Navigation, I want to get in the car" (i.e., the third user intent) is output.

[0064] In some implementations, determining control information based on the first user intent includes: matching a target entity keyword corresponding to the first user intent in a preset entity keyword set, the preset entity keyword set including at least one preset entity keyword, and each preset entity keyword being pre-configured with a corresponding prompt, the prompt including a slot corresponding to the corresponding preset entity keyword; and extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword.

[0065] The control vehicle performs control operations based on the control information, including: controlling the vehicle to perform control operations based on the first slot information.

[0066] In this embodiment, by determining the target entity keyword corresponding to the first user intent, a prompt corresponding to the target entity keyword is obtained. Then, based on the obtained prompt, the first slot information of the slot corresponding to the target entity keyword is extracted as the first slot information of the first user intent. The vehicle is controlled to perform control operations through the first slot information of the first user intent, thereby realizing the accurate determination of control information for controlling the vehicle to perform control operations through a large language model.

[0067] The aforementioned set of preset entity keywords can be pre-configured with at least one preset entity keyword. This preset entity keyword can be standardized terminology in vehicle control. For example, a set of preset entity keywords that can be matched can be defined using a regular expression `pattern`, forming the following preset entity keyword set: (steering wheel | fuel level | driving range | air conditioning | seat | trunk | air purification | casual conversation | get in the car | honk | ventilation | find car | unlock | lock car | car lock | tire pressure | interior temperature | battery level | battery | defrost | car door | car window | flashing lights | light show | navigation | location | delete, modify, search).

[0068] The above-mentioned matching of target entity keywords corresponding to the first user intent in the preset entity keyword set can be based on the semantics of each preset entity keyword and the first user intent for association matching; or it can be based on the matching of the first user intent with each preset entity keyword through preset mapping rules.

[0069] It should be noted that the first user intent mentioned above can match at least one target entity keyword. When the first user intent matches multiple target entity keywords, multiple synonyms among the multiple target entity keywords can be merged. For example, if "unlock", "lock car", and "car lock" are matched, they can be uniformly mapped to "unlock car". In addition, alias conversion between preset entity keywords can be performed according to preset special mapping words. For example, "car interior temperature" can be converted to "temperature", "trunk" can be converted to "rear door", and "chat" can be converted to "other", etc.

[0070] Of course, if the first user intent mentioned above does not match the preset entity keyword, the second language model mentioned above can also perform a fallback, for example, returning "other" when no keyword is hit or the match is abnormal.

[0071] In this embodiment, each preset entity keyword is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the preset entity keyword. In a large language model, a slot is a structured field used to represent key information that needs to be extracted in a specific task. The design of the slot schema determines the type and format of information that the model needs to extract.

[0072] The aforementioned prompt can be natural language text that only indicates the current round of dialogue. In this case, the second major language model can be the first slot information that extracts the first user intent based on the natural language text of the prompt.

[0073] For example, for the preset entity keyword "delete, modify, query", the configuration can be as follows: the slot schema is preset to: {'command':"}, where the command value is limited to "query, delete, modify", clearly defining the user's operation command type for vehicle-related information and providing precise operation guidance for vehicle data management. System_prompt (i.e., the prompt) = "You are an intelligent agent, and your task is to determine whether the user input belongs to the category of 'query', 'modify', or 'delete'."

[0074] In some implementations, the prompt also includes indication information for indicating historical conversations.

[0075] The step of extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword includes: extracting the first slot information of the first user intent from the historical dialogue associated with the natural language text and the natural language text based on the prompt corresponding to the target entity keyword.

[0076] In this embodiment, by indicating the historical dialogue in the prompt, the second language model can extract the first slot information from the natural language text of the historical dialogue and the current dialogue based on the prompt, thereby making the extracted first slot information more accurate.

[0077] The aforementioned prompts may also include instruction information for indicating historical dialogues, which may be any field or identifier describing at least one round of historical dialogue.

[0078] For example, for the preset entity keyword "delete, modify, query", the configuration can be as follows: the slot schema is preset to: {'command':"}, where the command value is limited to "query, delete, modify", clearly defining the user's operation command type for vehicle-related information and providing precise operation guidance for vehicle data management. System_prompt (i.e., the prompt) = "You are an intelligent agent. Your task is to determine whether the user input belongs to the category of 'query,' 'modify,' or 'delete' based on the historical context." In this way, the second language model can extract the first slot information of "query," "modify," or "delete" from historical dialogues and natural language text.

[0079] For the preset entity keyword "car window", the configuration can be as follows: the slot schema is designed as: {'value':"}, focusing on extracting information about the car window opening from the user input. Considering the diversity of user expression habits, this submodule can flexibly handle car window opening descriptions containing Chinese or a mixture of Chinese and English, accurately obtaining the specific parameters for car window control. System_prompt = "You are an intelligent agent. Your task is to extract the car window opening from the user input based on historical context. The car window opening can contain both Chinese characters and Arabic numerals." In this way, the second language model can extract the first slot information of the car window opening from historical dialogues and natural language text.

[0080] For the preset entity keyword "seat", the configuration can be as follows: the slot schema is designed as: {'position':”, 'function':”}, corresponding to seat position adjustment and function selection, respectively. Through detailed analysis of user input, the specific position information of seat adjustment and the required functions, such as heating and ventilation, are extracted to achieve personalized control of the seat system. System_prompt = "You are an intelligent agent, and your task is to extract the seat position and seat function from the user input based on historical context." In this way, the second language model can extract the first slot information of seat position and seat function from historical dialogue and natural language text.

[0081] For the preset entity keyword "air conditioner", the configuration can be as follows: Use the slot schema {'temp':”} to extract air conditioner temperature settings from user input. This also supports temperature values ​​expressed in Chinese and mixed Chinese and English, ensuring the air conditioning system can accurately respond to the user's temperature adjustment needs. System_prompt = "You are an intelligent agent. Your task is to extract the air conditioner temperature from the user input based on historical context. The air conditioner temperature can contain both Chinese characters and Arabic numerals." In this way, the second language model can extract the first slot information of the air conditioner temperature from historical dialogues and natural language text, and so on.

[0082] In some implementations, the first user intent includes a preset special user intent for booking a vehicle, and the slot for the prompt corresponding to the booking intent includes the location and time.

[0083] The step of extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword includes: extracting the target location and target time of the car reservation intent based on the prompt corresponding to the car reservation entity keyword.

[0084] The control operation of the vehicle based on the first slot information includes: based on the first slot information, controlling the vehicle to generate a target reservation vehicle preparation event; based on the target reservation vehicle preparation event, instructing the vehicle to start the vehicle in advance to perform a preset operation before the target time arrives, so that the user can travel to the target location by taking the vehicle at the target time; wherein the preset operation includes generating a navigation route with the target location as the destination.

[0085] In this embodiment, by configuring slots such as location and time for the prompts corresponding to the keywords of the vehicle reservation entity, the large language model can extract the slot information of the user's intention to reserve a vehicle, thereby accurately controlling the vehicle to realize the interaction of the reservation function.

[0086] When the vehicle departure interaction device recognizes the intention to reserve a vehicle, it can determine the reservation vehicle preparation entity keyword corresponding to the reservation vehicle preparation intention, obtain the prompt of the reservation vehicle preparation entity keyword, and then extract the target location and target time of the reservation vehicle preparation intention through the prompt of the reservation vehicle preparation entity keyword.

[0087] The time slot in the prompt message corresponding to the aforementioned vehicle reservation intention can be used to clarify when the user needs to use the vehicle. Specifically, the time can include the date, time period, and specific time (also known as a "time point").

[0088] For example, for the preset entity keyword "[Reservation for Vehicle Preparation]", the configuration can be as follows: the slot schema is designed as: {'location':”,'dayparts':”,'time':”,'date':”}, comprehensively covering all the key information elements required for reservation for vehicle preparation. It accurately extracts parameters such as location, time period, specific time, and date from user input, providing a complete basis for vehicle reservation preparation. System_prompt = "Reservation for Vehicle Preparation: The intelligent agent's task is to combine historical context and user input to comprehensively extract key information such as location, dayparts, time, and date for reservation for vehicle preparation, ensuring the accurate triggering of the reservation function." Thus, the second language model can extract the target location, target time period, target specific time, and target date from historical dialogues and natural language text. In this way, the off-vehicle interaction device can control the vehicle to generate corresponding target reservation for vehicle preparation events based on the target location, target time period, target specific time, and target date.

[0089] It should be noted that the above-mentioned preset operations may include generating a navigation route with the target location as the destination; or, the above-mentioned preset operations may also include at least one of starting the vehicle, turning on the air conditioner, opening the windows, and adjusting the seat.

[0090] In some implementations, after extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword, the method further includes: storing the first slot information of the first user intent into a target data structure.

[0091] The control of the vehicle to perform control operations based on the first slot information includes: controlling the vehicle to perform control operations based on the target data structure.

[0092] In this embodiment, after extracting the first slot information of the first user intent, the off-vehicle interaction device can store the extracted first slot information into the target data structure and execute vehicle control operations based on the target data structure. This enables the integration of the first slot information into the target data structure, providing a unified and orderly data foundation for subsequent analysis, decision-making, or other processing steps, which helps to improve processing efficiency and accuracy.

[0093] The target data structure mentioned above can be any data structure that can cache data; for example, it can be a forerunner data structure.

[0094] In some embodiments, controlling the vehicle to perform control operations based on the first slot information includes: post-processing the first slot information to obtain post-processed second slot information; and controlling the vehicle to perform control operations based on the second slot information.

[0095] In this embodiment, by post-processing the first slot information, the second slot information obtained after post-processing is more accurate, thereby further improving the accuracy of vehicle control operations.

[0096] It should be noted that when storing slot information through the target data structure, the above-mentioned post-processing of the first slot information to obtain the post-processed second slot information can be performed before storing the slot information in the target data structure, that is, after obtaining the second slot information through post-processing, the second slot information is stored in the target data structure; or, it can be performed after storing the first slot information in the target data structure, and the post-processing of the first slot information is performed. There is no limitation here.

[0097] The post-processing of the first slot information described above may include at least one of the following: slot value mapping, standardization, integration, and deduplication.

[0098] For example, assuming the target entity keyword corresponding to the first user intent is "car window", after the second language model extracts the slot information of the window opening value, the following post-processing can be performed: Slot value mapping generates two types of results based on the extracted value: Original value mapping: result_value = value if value is not None else "" Opening degree mapping: Classified based on predefined business rules: If value == "a little", then degree = "a little"; If value is None, then degree = "all"; Other cases (such as the specific value "30%)", then degree = "specific opening degree"; Finally, result_degree = degree is generated. Result integration stores result_value and result_degree into the result container forerunner dictionary and returns the updated context object.

[0099] For example, assuming the target entity keyword corresponding to the first user intent is "seat," after the second language model extracts the seat position and function slot information, the following post-processing can be performed: First, check if the value corresponding to the 'position' key in the result exists. If it exists, assign it to the position variable; if it does not exist, assign the value 'none' to position to clearly indicate that no valid position information was extracted. To make the position information representation more uniform and standardized, and to facilitate subsequent analysis and integration, the value of position is further judged. If position is 'main rear' or 'secondary rear', it is uniformly converted to 'back row' to achieve a standardized representation of position information.

[0100] For example, assuming the target entity keyword corresponding to the first user intent is "reservation of car service", after the second language model extracts the slot information of location, time period, specific time, and date, the following post-processing can be performed:

[0101] 1) Post-processing of navigation address information.

[0102] By receiving two parameters, forerunner and slots_info, the system extracts location information from slots_info, formats it into a specific nested dictionary structure, and then updates the result_loc attribute of the forerunner object. If the location information is empty, an empty dictionary is assigned. Finally, the updated forerunner object is returned, providing a unified and standardized data format for subsequent navigation-related operations, ensuring data consistency and availability, and supporting the implementation and expansion of subsequent navigation functions.

[0103] 2) Time and date post-processing workflow

[0104] 1. Date Completion Processing: When detecting date strings containing "month" but missing "day", such as "May 3", the regular expression `(\d+)month\s(\d+)` is used for precise matching to ensure that other formats are not affected. The "day" is then added to the end of the successfully matched string to standardize it as "May 3".

[0105] 2. Date Standardization and Special Periodic Handling: Workdays: If the date information contains both "workday" and "every," it is converted to "Monday to Friday." If it only contains "workday" without "every," it is considered a regular workday, i.e., "Monday to Friday." Weekends: If the date information contains both "weekend" and "every," it is converted to "Saturday and Sunday." If it only contains "weekend," it is a regular Saturday and Sunday. Standardized Terminology: Replace "day" with "day," for example, "Sunday" is replaced with "Sunday," standardizing date expression. Replace "week of the week" with "week," for example, "Wednesday" is replaced with "Wednesday," ensuring terminology consistency. Periodic Identifiers: If the date information contains periodic identifiers such as "every" or "the week after next," these identifiers are retained; for example, "Wednesday" is directly retained as a periodic identifier. For combined date information containing keywords such as "week" or "week of the week," it is split into a list; for example, "next Wednesday and Thursday" is split into ["next Wednesday", "next Thursday"] for easier subsequent processing. Relative date processing: Directly extract relative date terms such as "today", "tomorrow", and "the day after tomorrow" without additional conversion, preserving their relative meaning.

[0106] 3. Key Processing Logic for Time Standardization: **Removing Irrelevant Words:** Remove non-time words such as "today," "tomorrow," and "immediately." For example, change "tomorrow afternoon 5 o'clock" to "afternoon 5 o'clock" to accurately extract time information. **Correcting Duplicate Words:** Correct duplicate words for erroneous inputs like "8 o'clock in the morning" to standardize it as "8 o'clock in the morning." **Separating Date Interference:** If the time field mistakenly contains date information (such as "Saturday 5 o'clock"), after ensuring the `date` field has been correctly extracted, attempt to remove the date portion from the time field to avoid interference.

[0107] 4. Time Period Extraction: The extraction logic uses the regular expression

pattern=r'(morning|noon|evening|morning|afternoon|daytime|night|early morning|late night|evening)'

[0108] 5. Key Steps for Data Integration and Deduplication: 1. Deduplication: Perform deduplication operations on the date and time lists using `list(set(...))` to remove duplicate date and time information, ensuring data uniqueness. 2. Field Update: Write the processed date (`result_date`), time (`result_time`), and time period (`result_dayparts`) into a `forerunner` object, completing data integration and updating.

[0109] 6. Periodic Task Judgment Logic: If the `date` field is not empty and the first element contains "every" (e.g., "every Wednesday"), it is judged as a periodic task, and `result_ifcycle = True` is set; otherwise, if the above conditions are not met, it is marked as a non-periodic task, and `result_ifcycle = False`. The final output, the `forerunner` object, contains the following fields: `result_date`: a normalized list of dates, such as `["every Wednesday"]`. `result_time`: a cleaned list of times, such as `["5 o'clock"]`. `result_dayparts`: a list of time periods, such as `["afternoon"]`. `result_ifcycle`: a periodic task identifier, with a value of `True` or `False`.

[0110] Through the coordinated operation of the above modules, this intelligent vehicle interaction system can achieve comprehensive and in-depth processing of user input, accurately understand user intent, efficiently execute corresponding instructions, and provide users with a convenient and intelligent interactive experience.

[0111] In some embodiments, post-processing the first slot information in the target data structure to obtain the post-processed target data structure includes: generating a periodic identifier corresponding to the periodic field when the first slot information includes a periodic field, wherein the second slot information includes the periodic identifier.

[0112] The control operation of the vehicle based on the second slot information includes: controlling the vehicle to perform the control operation based on the second slot information according to the period corresponding to the periodic identifier.

[0113] In this embodiment, a periodic identifier corresponding to the periodic field in the first slot information is generated and added to the first slot information to form the second slot information, thereby enabling the vehicle to perform periodic control operations based on the periodic identifier in vehicle control.

[0114] For example, if the forerunner dictionary structure contains slot information for the time corresponding to "reservation for vehicle standby," and that time includes a field for "Wednesday," then a periodic identifier for "Wednesday" is generated. In this way, the vehicle interaction device can control the vehicle to perform control operations corresponding to the user's reservation for vehicle standby every Wednesday based on the periodic identifier for "Wednesday."

[0115] In some implementations, the method further includes: if the first slot information corresponding to the first user intent is not extracted through the second language model, generating a preset character identifier; and in response to the preset character identifier, sending the first catch-all response to the user terminal.

[0116] In this embodiment, if the second language model fails to extract the first slot information, the off-vehicle interaction device can generate a preset character identifier and, in response to the preset character identifier, send a first fallback response to the user terminal, thereby ensuring the continuity of the processing flow and improving the user interaction experience.

[0117] For example, assuming the target entity keyword corresponding to the first user intent is "car window", if the second language model does not extract the slot information of the car window opening value, then perform null initialization value = None (i.e., preset character identifier) ​​and store None in the forerunner data structure. The off-vehicle device can provide a corresponding fallback response based on this None.

[0118] In some implementations, after identifying the first user intent through the first large language model, the method further includes: determining a target confidence level that associates the first large language model with the first user intent; if the target confidence level is less than a confidence level threshold, obtaining a second catch-all response corresponding to the natural language text; and sending the second catch-all response to the user terminal.

[0119] In this embodiment, if the first language model fails to recognize the correct first user intent (i.e., the target confidence level is less than the confidence level threshold), the off-vehicle interaction device can send a second fallback response to the user terminal, thereby ensuring the continuity of the processing flow and improving the user interaction experience.

[0120] To facilitate understanding of the off-vehicle interaction method in this embodiment, an application example of the off-vehicle interaction method is provided here, as shown in Figure 3. The specific processing procedure is as follows: Upon receiving a user request (i.e., a request sent by the user terminal carrying natural language text), the natural language text in the user request is subjected to FAQ matching (i.e., querying a preset frequent question and answer database to find the user intent that matches the natural language text); If the FAQ match matches the user intent (i.e., the second user intent), the FAQ matching result (carrying the second user intent) is returned, and the vehicle is controlled to perform control operations and provide feedback based on the FAQ matching result; If the FAQ does not match the user intent, multi-turn intent recognition is performed on the natural language text in the user request, i.e., user intent recognition is performed based on multi-turn historical dialogue and natural language text using a large language model, and the recognized user intents are generated into an intent list; The intent list is preprocessed, such as deduplication; It is determined whether the preprocessed intent list contains a special intent [reservation for vehicle preparation] (i.e., the target special user intent); If the intent list contains the special intent "[Reservation for Car Standby]", then the special intent "[Reservation for Car Standby]" is split into two sub-intents "[Navigation, Car Use]". Intent processing is performed on the two sub-intents "[Navigation, Car Use]", and the slot information of the sub-intents after intent processing is extracted, including the slot information of time and address. The slot information of time and address is then post-processed, such as time and address standardization. After post-processing, reservation for car standby parameters are generated and stored in the target data structure (such as the forerunner data structure) for slot integration, and a structured result is returned. If the intent list does not contain the special intent "[Reservation for Car Standby]", then if the intent list contains user intents corresponding to at least one of the preset entity keywords "[Car Window]", "[Air Conditioning]", "[Navigation]", and "[Seat]", intent processing is performed on the user intents corresponding to at least one entity keyword to obtain the corresponding slot information. The slot information is then stored in the target data structure (such as the forerunner data structure) for slot integration, and a structured result is returned.

[0121] The vehicle-to-vehicle interaction method provided in this disclosure can be executed by a vehicle-to-vehicle interaction device. This disclosure uses the execution of the vehicle-to-vehicle interaction method by a vehicle-to-vehicle interaction device as an example to illustrate the vehicle-to-vehicle interaction device provided in this disclosure.

[0122] Figure 4 is a schematic diagram of the structure of the off-vehicle interaction device 400 provided in an embodiment of this disclosure. As shown in Figure 4, the off-vehicle interaction device 400 of this disclosure includes: a text receiving module 401, configured to receive natural language text sent by a user terminal; an intent recognition module 402, configured to recognize the natural language text using a pre-configured first large language model to obtain a first user intent; a control information determination module 403, configured to determine control information based on the first user intent using a pre-configured second large language model, wherein the control information is associated with the natural language text; and a control module 404, configured to control the vehicle to perform control operations based on the control information, and to send a response corresponding to the natural language text to the user terminal, wherein the response is used to indicate the control result of the control operation.

[0123] In some embodiments, the device 400 further includes: a database query module configured to: perform a query in a preset frequent question and answer database based on the natural language text to obtain query results; a control information determination module 403 further configured to: determine control information based on the second user intent if the query result indicates that a second user intent matching the natural language text has been found; and an intent recognition module 402 specifically configured to: input the natural language text into a pre-configured first large language model if the query result indicates that a user intent matching the natural language text has not been found, and recognize the first user intent through the first large language model.

[0124] In some implementations, the intent recognition module 402 is specifically configured to: when the natural language text is natural language text in a multi-turn dialogue process, obtain dialogue information of at least one round of historical dialogue associated with the natural language text in the multi-turn dialogue; and use the first large language model to recognize the dialogue information of the at least one round of historical dialogue and the natural language text to obtain a first user intent.

[0125] In some embodiments, the device 400 further includes: a special intent matching module, configured to: match the first user intent with a preset special user intent to obtain a first matching result; an intent processing module, configured to: when the first matching result indicates that the first user intent matches a target special user intent, perform an intent processing operation corresponding to the target special user intent to obtain a third user intent, wherein the third user intent is more consistent with the actual operating logic of the vehicle than the first user intent; and a control information determination module 403, specifically configured to: determine control information based on the third user intent using a pre-configured second language model.

[0126] In some implementations, the intent processing module is specifically configured to: determine at least one user sub-intent corresponding to the target specific user intent; match the natural language text with each of the user sub-intents to obtain a second matching result, the second matching result being used to indicate whether there is a preset text in the natural language text corresponding to the user sub-intent; based on the second matching result, remove the target user sub-intent from the at least one user sub-intent to obtain a third user intent, wherein the target user sub-intent is a user sub-intent whose corresponding preset text does not exist in the natural language text.

[0127] In some embodiments, the control information determination module 403 is specifically configured to: match a target entity keyword corresponding to the first user intent in a preset entity keyword set, the preset entity keyword set including at least one preset entity keyword, and each preset entity keyword having a corresponding prompt, the prompt including a slot corresponding to the corresponding preset entity keyword; extract the first slot information of the first user intent based on the prompt corresponding to the target entity keyword; and the control module 404 is specifically configured to: control the vehicle to perform a control operation based on the first slot information.

[0128] In some implementations, the prompt also includes indication information for indicating historical dialogues. The control information determination module 403 is specifically configured to: extract the first slot information of the first user intent from historical dialogues associated with the natural language text and the natural language text based on the prompt corresponding to the target entity keyword.

[0129] In some implementations, the first user intent includes a pre-set special user intent for vehicle reservation, and the slot for the prompt corresponding to the vehicle reservation intent includes location and time; the control information determination module 403 is specifically configured to: extract the target location and target time of the vehicle reservation intent based on the prompt corresponding to the vehicle reservation entity keyword; the control module 404 is specifically configured to: control the vehicle to generate a target vehicle reservation event based on the first slot information, and instruct the vehicle to start the vehicle in advance to perform a preset operation before the target time arrives, so that the user can travel to the target location by taking the vehicle at the target time, the preset operation including generating a navigation route with the target location as the destination.

[0130] In some embodiments, the device 400 further includes: a storage module configured to store the first slot information of the first user intent into a target data structure; and a control module 404 specifically configured to control the vehicle to perform control operations based on the target data structure.

[0131] In some embodiments, the control module 404 is specifically configured to: post-process the first slot information to obtain post-processed second slot information; and control the vehicle to perform control operations based on the second slot information.

[0132] In some embodiments, the control module 404 is specifically configured to: generate a periodic identifier corresponding to the periodic field when the first slot information includes a periodic field, wherein the second slot information includes the periodic identifier; and control the vehicle to perform control operations based on the second slot information according to the period corresponding to the periodic identifier.

[0133] In some embodiments, the device 400 further includes: an identifier generation module, configured to generate a preset character identifier if the first slot information corresponding to the first user intent is not extracted through the second language model; and a first fallback module, configured to send a first fallback response to the user terminal in response to the preset character identifier.

[0134] In some embodiments, the device 400 further includes: a confidence determination module, configured to: determine a target confidence level associated with the first large language model and the first user intent; a fallback response acquisition module, configured to: acquire a second fallback response corresponding to the natural language text when the target confidence level is less than a confidence threshold; and a fallback response sending module, configured to: send the second fallback response to the user terminal.

[0135] The vehicle-to-vehicle interaction device 400 provided in this embodiment can execute the technical solution shown in the above-described vehicle-to-vehicle interaction method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0136] Figure 5 is a schematic diagram of the hardware structure of the off-vehicle interaction device provided in an embodiment of this disclosure.

[0137] The vehicle-to-everything (V2X) interaction device may include a processor 501 and a memory 502 storing computer program instructions. The V2X interaction device may be an electronic device or a vehicle.

[0138] Specifically, the processor 501 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this disclosure.

[0139] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0140] In some embodiments, memory 502 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the off-vehicle interaction method according to this disclosure.

[0141] The processor 501 implements the off-vehicle interaction method in the above embodiments by reading and executing computer program instructions stored in the memory 502.

[0142] In one example, the off-vehicle interaction device may also include a communication interface 503 and a bus 510. As shown in Figure 5, the processor 501, memory 502, and communication interface 503 are connected via the bus 510 and communicate with each other.

[0143] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this disclosure.

[0144] Bus 510 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this disclosure, this disclosure contemplates any suitable bus or interconnect.

[0145] The off-vehicle interaction device can execute the off-vehicle interaction method in the embodiments of this disclosure, thereby realizing the off-vehicle interaction method and device described in conjunction with Figures 1 to 4.

[0146] In addition, in conjunction with the vehicle-to-vehicle interaction method in the above embodiments, this disclosure also provides a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement the vehicle-to-vehicle interaction method in the above embodiments.

[0147] In conjunction with the vehicle-to-vehicle interaction method in the above embodiments, this disclosure also provides a computer program product, wherein the instructions in the computer program product, when executed by the processor of the vehicle-to-vehicle interaction device, enable the vehicle-to-vehicle interaction device to implement the vehicle-to-vehicle interaction method in the above embodiments.

[0148] Vehicles can be private cars, such as sedans, SUVs, MPVs, or pickup trucks. Vehicles can also be commercial vehicles, such as vans, buses, small trucks, or large semi-trailers. Vehicles can be either gasoline-powered or new energy vehicles. When a vehicle is a new energy vehicle, it can be a hybrid or a pure electric vehicle.

[0149] It should be clarified that this disclosure is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this disclosure is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this disclosure.

[0150] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this disclosure are programs or code segments used to perform desired tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0151] It should also be noted that the exemplary embodiments mentioned in this disclosure describe methods or systems based on a series of steps or apparatus. However, this disclosure is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0152] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods (systems) and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0153] The above description is merely a specific embodiment of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the protection scope of this disclosure.

Claims

1. A method for off-vehicle interaction, comprising: Receive natural language text sent by user terminals; The first user intent is obtained by recognizing the natural language text using a pre-configured first major language model. Control information is determined based on the first user intent using a pre-configured second language model, wherein the control information is associated with the natural language text; The vehicle is controlled to perform control operations based on the control information, and, Send a response corresponding to the natural language text to the user terminal, wherein the response is used to indicate the control result of the control operation.

2. The method of claim 1, wherein, Before recognizing the natural language text using the first large language model, the process also includes: Based on the natural language text, a query is performed in a preset frequent question and answer database to obtain the query results; If the query result indicates that a second user intent is found that matches the natural language text, control information is determined based on the second user intent. If the query results indicate that no user intent matching the natural language text was found, the natural language text is input into the first large language model.

3. The method of claim 1, wherein, The step of obtaining the first user intent by recognizing the natural language text through the first large language model includes: When the natural language text is natural language text in a multi-turn dialogue process, obtain the dialogue information of at least one round of historical dialogue associated with the natural language text in the multi-turn dialogue; The first user intent is obtained by recognizing the dialogue information of the at least one round of historical dialogue and the natural language text using the first large language model.

4. The method of claim 1, wherein, Before the controlled vehicle performs a control operation based on the control information, the method further includes: The first user intent is matched with a preset special user intent to obtain a first matching result; If the first matching result indicates that the first user intent matches the target specific user intent, an intent processing operation corresponding to the target specific user intent is executed to obtain a third user intent, wherein the third user intent is more consistent with the actual operation logic of the vehicle than the first user intent.

5. The method of claim 4, wherein, The step of determining control information based on the first user intent using a pre-configured second language model includes: Control information is determined based on the third user intent using a pre-configured second language model.

6. The method of claim 4, wherein, The execution of the intent processing operation corresponding to the target specific user intent includes: Determine at least one user sub-intent that corresponds to the target specific user intent; The natural language text is matched with the at least one user sub-intention to obtain a second matching result, wherein the second matching result is used to indicate whether there is preset text in the natural language text that corresponds to the at least one user sub-intention; Based on the second matching result, the target user sub-intent is removed from the at least one user sub-intent to obtain a third user intent, wherein the target user sub-intent is a user sub-intent whose corresponding preset text does not exist in the natural language text.

7. The method of claim 1, wherein, The step of determining control information based on the first user intent includes: Match the target entity keyword corresponding to the first user intent in the preset entity keyword set, wherein the preset entity keyword set includes at least one preset entity keyword, and each of the at least one preset entity keyword is pre-configured with a corresponding prompt, the prompt including a slot corresponding to the corresponding preset entity keyword; Based on the prompts corresponding to the target entity keywords, the first slot information of the first user intent is extracted.

8. The method of claim 7, wherein, The controlled vehicle performs control operations based on the control information, including: The vehicle is controlled to perform control operations based on the first slot information.

9. The method of claim 7, wherein, The prompt also includes instructions for indicating historical conversations. The step of extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword includes: Based on the prompts corresponding to the target entity keywords, the first slot information of the first user intent is extracted from the historical dialogues associated with the natural language text and the natural language text.

10. The method of claim 8, wherein, The first user intent includes the preset special user intent of booking a vehicle, and the slot for the prompt corresponding to the booking a vehicle includes the location and the time. The step of extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword includes: Based on the prompts corresponding to the keywords of the vehicle reservation entity, the target location and target time of the vehicle reservation intention are extracted.

11. The method of claim 10, wherein, The control operation of the vehicle based on the first slot information includes: Based on the first slot information, control the vehicle to generate a target reservation vehicle preparation event; Based on the target reservation vehicle preparation event, the vehicle is instructed to start in advance and perform preset operations before the target time arrives, so that the user can travel to the target location by taking the vehicle at the target time. The preset operations include generating a navigation route with the target location as the destination.

12. An off-vehicle interaction device, comprising: The text receiving module is configured to receive natural language text sent by the user terminal. The intent recognition module is configured to: recognize the natural language text using a pre-configured first large language model to obtain the first user intent; The control information determination module is configured to: determine control information based on the first user intent using a pre-configured second language model, wherein the control information is associated with the natural language text; The control module is configured to: control the vehicle to perform control operations based on the control information, and send a response corresponding to the natural language text to the user terminal, wherein the response is used to indicate the control result of the control operation.

13. A vehicle-unaware interaction device comprising: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the off-vehicle interaction method as described in any one of claims 1-10.

14. A computer-readable storage medium storing computer program instructions; The computer program instructions, when executed by the processor, implement the off-car interaction method according to any one of claims 1-10.

Images