Off-car interaction method, device and equipment and storage medium

Abstract

The invention discloses an off-vehicle interaction method and device, equipment and a storage medium. The method comprises the following steps: receiving a target natural language text sent by a user terminal; inputting the target natural language text into a pre-configured first large language model, and identifying through the first large language model to obtain a first user intention; controlling the vehicle to execute control operation based on the control information; and sending a target answer language corresponding to the target natural language text to the user terminal. In this way, the interactivity between people and the vehicle can be improved.

Description

Technical Field

[0001] The present application belongs to the field of vehicle technology, and in particular relates to an off-vehicle interaction method, device, equipment and storage medium. Background Art

[0002] With the rapid development of vehicle technology, vehicles have become an indispensable tool for daily commuting. To meet the growing demand for intelligent and connected vehicles, vehicles are increasingly equipped with a growing number of human-vehicle interaction features. Off-vehicle interaction, a key feature of vehicle human-vehicle interaction, allows users to interact with the vehicle through a user terminal while away from the vehicle, enabling remote control of the vehicle. However, off-vehicle interaction features in related technologies are typically limited to single commands and feedback, resulting in poor interactivity between the human and the vehicle. Summary of the Invention

[0003] The embodiments of the present application provide an off-vehicle interaction method, apparatus, device, and storage medium, which can improve the interactivity between people and vehicles in the off-vehicle interaction function.

[0004] In a first aspect, an embodiment of the present application provides an off-vehicle interaction method, the method comprising:

[0005] receiving a target natural language text sent by a user terminal;

[0006] Inputting the target natural language text into a preconfigured first language model, and obtaining a first user intention through recognition by the first language model;

[0007] determining, using a preconfigured second language model, control information based on the first user intent, wherein the control information is associated with the target natural language text;

[0008] The vehicle is controlled to perform a control operation based on the control information, and a target answer language corresponding to the target natural language text is sent to the user terminal, wherein the target answer language is used to indicate a control result of the control operation.

[0009] In some implementations, before inputting the target natural language text into a preconfigured first language model and identifying the user intent using the first language model, the method further includes:

[0010] In response to the target natural language text, searching a preset frequent question answering database for user intentions that match the target natural language text to obtain a query result;

[0011] In the case where the query result indicates that a second user intention matching the target natural language text is queried, determine control information based on the second user intention;

[0012] The inputting the target natural language text into a pre-configured first large language model and identifying a first user intention through the first large language model includes:

[0013] In the case where the query result indicates that no user intention matching the target natural language text is queried, input the target natural language text into a pre-configured first large language model, and identify a first user intention through the first large language model.

[0014] In some embodiments, the identifying a first user intention through the first large language model includes:

[0015] In the case where the target natural language text is a natural language text in a multi-round conversation, obtain the conversation information of at least one round of historical conversation associated with the target natural language text in the multi-round conversation;

[0016] Based on the conversation information of the at least one round of historical conversation and the target natural language text, the first large language model identifies the user intention to obtain a first user intention.

[0017] In some embodiments, before the method controls the vehicle to perform a control operation based on the control information, the method further includes:

[0018] Match the first user intention with a preset special user intention to obtain a first matching result;

[0019] In the case where the first matching result indicates that the first user intention matches a target special user intention, perform an intention processing operation corresponding to the target special user intention to obtain a third user intention, and the third user intention is more in line with the actual operation logic of the vehicle than the first user intention;

[0020] The determining control information based on the first user intention through a pre-configured second large language model includes:

[0021] Through a pre-configured second large language model, determine control information based on the third user intention.

[0022] In some embodiments, the performing an intention processing operation corresponding to the target special user intention includes:

[0023] Determine at least one user sub-intention corresponding to the target special user intention;

[0024] Matching the target natural language text with each of the user sub-intents to obtain a second matching result, where the second matching result is used to indicate whether there is a preset text corresponding to the user sub-intent in the target natural language text;

[0025] Based on the second matching result, the target user sub-intent in the at least one user sub-intent is eliminated to obtain a third user intent, wherein the target user sub-intent is a user sub-intent whose corresponding preset text does not exist in the target natural language text.

[0026] In some implementations, determining control information based on the first user intention includes:

[0027] Matching a target entity keyword corresponding to the first user intent in a preset entity keyword set, wherein the preset entity keyword set includes at least one preset entity keyword, and each preset entity keyword is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the corresponding preset entity keyword;

[0028] Extracting first slot information of the first user intent based on a prompt corresponding to the target entity keyword;

[0029] The controlling the vehicle to perform a control operation based on the control information includes:

[0030] The vehicle is controlled to perform a control operation based on the first slot information.

[0031] In some embodiments, the prompt also includes indication information for indicating historical conversations.

[0032] The extracting the first slot information of the first user intention based on the prompt corresponding to the target entity keyword includes:

[0033] Based on the prompt corresponding to the target entity keyword, the first slot information of the first user intention is extracted from the historical conversation associated with the target natural language text and the target natural language text.

[0034] In some embodiments, the first user intention includes an intention to reserve a vehicle among preset special user intentions, and the slot of the prompt corresponding to the intention to reserve a vehicle includes a location and a time;

[0035] The extracting the first slot information of the first user intention based on the prompt corresponding to the target entity keyword includes:

[0036] Extracting the target location and target time of the vehicle preparation intention based on the prompt corresponding to the vehicle preparation entity keyword;

[0037] The controlling the vehicle to perform a control operation based on the first slot information includes:

[0038] The vehicle is controlled to generate a target reservation and preparation event, and the target reservation and preparation event is used to: instruct the vehicle to start the vehicle in advance before the target time arrives to perform a preset operation, so that the user can take the vehicle to the target location at the target time. The preset operation includes generating a navigation route with the target location as the destination.

[0039] In some embodiments, after extracting the first slot information of the first user intent based on the prompt corresponding to the target entity keyword, the method further includes:

[0040] Storing the first slot information of the first user intention into a target data structure;

[0041] The controlling the vehicle to perform a control operation based on the first slot information includes:

[0042] The vehicle is controlled to perform a control operation based on the target data structure.

[0043] In some embodiments, controlling the vehicle to perform a control operation based on the first slot information includes:

[0044] Post-processing the first slot information to obtain post-processed second slot information;

[0045] The vehicle is controlled to perform a control operation based on the second slot information.

[0046] In some implementations, post-processing the first slot information to obtain the post-processed second slot information includes:

[0047] In a case where the first slot information includes a periodicity field, generating a periodicity identifier corresponding to the periodicity field, wherein the second slot information includes the periodicity identifier;

[0048] The controlling the vehicle to perform a control operation based on the second slot information includes:

[0049] According to the period corresponding to the periodic identifier, the vehicle is controlled to perform a control operation based on the second slot information.

[0050] In some embodiments, the method further comprises:

[0051] If the first slot information corresponding to the first user intention is not extracted through the second language model, generating a preset character identifier;

[0052] In response to the preset character identifier, a first catch-all answer is sent to the user terminal.

[0053] In some implementations, after identifying the first user intent using the first language model, the method further includes:

[0054] Determining a target confidence level associated with the first language model and the first user intent;

[0055] When the target confidence is less than a confidence threshold, obtaining a second fallback answer corresponding to the target natural language text;

[0056] Send the second catch-all answer to the user terminal.

[0057] In a second aspect, an embodiment of the present application further provides an off-vehicle interaction device, comprising:

[0058] A text receiving module, configured to receive a target natural language text sent by a user terminal;

[0059] an intention recognition module, configured to input the target natural language text into a preconfigured first language model, and obtain a first user intention through recognition using the first language model;

[0060] a control information determination module, configured to determine control information based on the first user intention using a preconfigured second language model, wherein the control information is associated with the target natural language text;

[0061] A control module is used to control the vehicle to perform a control operation based on the control information, and to send a target answer language corresponding to the target natural language text to the user terminal, wherein the target answer language is used to indicate a control result of the control operation.

[0062] In some embodiments, the device further comprises:

[0063] a database query module, configured to query a preset frequently asked questions database for user intentions that match the target natural language text in response to the target natural language text, and obtain a query result;

[0064] A control information determination module, configured to determine control information based on a second user intent when the query result indicates that the query obtains a second user intent that matches the target natural language text;

[0065] The intention recognition module is specifically used to:

[0066] In the case where the query result indicates that no user intention matching the target natural language text is found, input the target natural language text into a pre-configured first large language model, and identify a first user intention through the first large language model.

[0067] In some embodiments, the intention recognition module is specifically configured to:

[0068] In the case where the target natural language text is a natural language text in a multi-round conversation, obtain the conversation information of at least one round of historical conversation associated with the target natural language text in the multi-round conversation;

[0069] The first large language model identifies the user intention based on the conversation information of the at least one round of historical conversation and the target natural language text, and obtains a first user intention.

[0070] In some embodiments, the device further includes:

[0071] A special intention matching module, configured to match the first user intention with a preset special user intention to obtain a first matching result;

[0072] An intention processing module, configured to perform an intention processing operation corresponding to the target special user intention in the case where the first matching result indicates that the first user intention matches the target special user intention, and obtain a third user intention, where the third user intention is more in line with the actual operation logic of the vehicle than the first user intention;

[0073] The control information determination module is specifically configured to:

[0074] Determine control information based on the third user intention through a pre-configured second large language model.

[0075] In some embodiments, the intention processing module is specifically configured to:

[0076] Determine at least one user sub-intention corresponding to the target special user intention;

[0077] Match the target natural language text with each of the user sub-intentions to obtain a second matching result, where the second matching result is used to indicate whether there is a preset text corresponding to the user sub-intention in the target natural language text;

[0078] Based on the second matching result, eliminate the target user sub-intention from the at least one user sub-intention, and obtain a third user intention, where the target user sub-intention is a user sub-intention for which the corresponding preset text does not exist in the target natural language text.

[0079] In some implementations, the control information determination module is specifically configured to:

[0080] Matching a target entity keyword corresponding to the first user intent in a preset entity keyword set, wherein the preset entity keyword set includes at least one preset entity keyword, and each preset entity keyword is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the corresponding preset entity keyword;

[0081] Extracting first slot information of the first user intent based on a prompt corresponding to the target entity keyword;

[0082] The control module is specifically used to:

[0083] The vehicle is controlled to perform a control operation based on the first slot information.

[0084] In some embodiments, the prompt also includes indication information for indicating historical conversations.

[0085] The control information determination module is specifically configured to:

[0086] Based on the prompt corresponding to the target entity keyword, the first slot information of the first user intention is extracted from the historical conversation associated with the target natural language text and the target natural language text.

[0087] In some embodiments, the first user intention includes an intention to reserve a vehicle among preset special user intentions, and the slot of the prompt corresponding to the intention to reserve a vehicle includes a location and a time;

[0088] The control information determination module is specifically configured to:

[0089] Extracting the target location and target time of the vehicle preparation intention based on the prompt corresponding to the vehicle preparation entity keyword;

[0090] The control module is specifically used to:

[0091] The vehicle is controlled to generate a target reservation and preparation event, and the target reservation and preparation event is used to: instruct the vehicle to start the vehicle in advance before the target time arrives to perform a preset operation, so that the user can take the vehicle to the target location at the target time. The preset operation includes generating a navigation route with the target location as the destination.

[0092] In some embodiments, the device further comprises:

[0093] a storage module, configured to store the first slot information of the first user intention into a target data structure;

[0094] The control module is specifically used to:

[0095] The vehicle is controlled to perform a control operation based on the target data structure.

[0096] In some embodiments, the control module is specifically configured to:

[0097] Post-processing the first slot information to obtain post-processed second slot information;

[0098] The vehicle is controlled to perform a control operation based on the second slot information.

[0099] In some embodiments, the control module is specifically configured to:

[0100] In a case where the first slot information includes a periodicity field, generating a periodicity identifier corresponding to the periodicity field, wherein the second slot information includes the periodicity identifier;

[0101] According to the period corresponding to the periodic identifier, the vehicle is controlled to perform a control operation based on the second slot information.

[0102] In some embodiments, the device further comprises:

[0103] an identifier generating module, configured to generate a preset character identifier if the first slot information corresponding to the first user intention is not extracted by the second language model;

[0104] The first fallback module is configured to send a first fallback answer to the user terminal in response to the preset character identifier.

[0105] In some embodiments, the device further comprises:

[0106] a confidence determination module, configured to determine a target confidence level associated with the first language model and the first user intent;

[0107] A fallback answer acquisition module, configured to acquire a second fallback answer corresponding to the target natural language text when the target confidence is less than a confidence threshold;

[0108] The catch-all answer sending module is configured to send the second catch-all answer to the user terminal.

[0109] In a third aspect, an embodiment of the present application further provides an off-vehicle interaction device, the off-vehicle interaction device comprising: a processor and a memory storing computer program instructions;

[0110] When the processor executes the computer program instructions, the off-vehicle interaction method as described in any one of the first aspects is implemented.

[0111] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the off-vehicle interaction method as described in any one of the first aspects is implemented.

[0112] In a fifth aspect, an embodiment of the present application further provides a computer program product. When the instructions in the computer program product are executed by a processor of an off-vehicle interaction device, the off-vehicle interaction device implements the off-vehicle interaction method as described in any one of the first aspects.

[0113] In an embodiment of the present application, a target natural language text is received from a user terminal; the target natural language text is input into a preconfigured first large language model, and a first user intent is identified by the first large language model; the vehicle is controlled to perform a control operation based on the control information; and a target response corresponding to the target natural language text is sent to the user terminal. In this way, the large language model can more intelligently and accurately identify the user's target user intent and determine the control information corresponding to the target user intent, thereby more flexibly controlling the vehicle to perform the user's desired control operation and providing the user with the corresponding target response, rather than being limited to a single instruction and feedback, thereby enhancing the interactivity between the vehicle and the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0115] Figure 1 is a flowchart of an embodiment of the off-vehicle interaction method provided by the present application;

[0116] Figure 2 is a flowchart of an intention processing operation in an embodiment of the off-vehicle interaction method provided by this application;

[0117] Figure 3 This is a flowchart of an application example in an embodiment of the off-vehicle interaction method provided in this application;

[0118] Figure 4 is a structural diagram of an embodiment of the off-vehicle interaction device provided by the present application;

[0119] Figure 5 It is a structural diagram of an embodiment of the off-vehicle interaction device provided in this application. Detailed Embodiments

[0120] The features and exemplary embodiments of various aspects of the present application will be described in detail below. To make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application and not to limit the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0121] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, elements defined by the statement "including..." do not exclude the presence of additional identical elements in the process, method, article or device including the said elements.

[0122] Figure 1 is a schematic flowchart of an embodiment of the off-vehicle interaction method provided by the present application. The off-vehicle interaction method of the present application can be applied to an off-vehicle interaction device, which can be a vehicle or an electronic device. The electronic device can be an in-vehicle terminal or a terminal device independent of the vehicle, a server (such as a cloud server, etc.). As Figure 1 shown, the off-vehicle interaction method includes but is not limited to the following steps S101 to S104.

[0123] Step S101: Receive the target natural language text sent by the user terminal.

[0124] In this step, when the user of the user terminal leaves the vehicle, if the user needs to interact with the vehicle through the user terminal, the user can send the above natural language text to the off-vehicle interaction device through the user terminal.

[0125] The above target natural language can be the natural language text obtained by receiving a preset operation input by the user based on the off-vehicle interaction interface when the off-vehicle interaction interface is displayed on the user terminal. Among them, the preset operation can be at least one of a voice operation, an air gesture operation, and a touch operation.

[0126] For example, the user terminal may receive the voice input by the user when the voice input control in the off-vehicle interaction interface is triggered (for example, the user inputs the voice of "open the car window", etc.), perform voice content recognition on the voice, and generate a target natural language text corresponding to the voice; or, the user terminal may receive the target natural language text input by the user in the text input component in the off-vehicle interaction interface through touch operation, and so on.

[0127] Step S102: input the target natural language text into a preconfigured first language model, and obtain a first user intention through recognition using the first language model.

[0128] In this step, the first language model can be any pre-trained model that can recognize user intent based on natural language text. For example, the first language model can include at least one of the large language models such as Qwen, ChatGPT, and Llama.

[0129] The training of the above-mentioned first large language model may include: obtaining a training set, which includes a certain number of training samples, each training sample includes a preset natural language text and a user intent label, and the user intent label can be a manually labeled user intent corresponding to the preset natural language text; inputting the training samples in the training set into the initial large language model, iteratively training the initial large model to obtain the first large language model, wherein, during the training process, the preset natural language text is used as the input of the initial large language model, and the initial large language model adjusts the network parameters according to the loss between its output result and the user intent label until the loss meets the preset conditions (for example, tends to be stable, etc.).

[0130] It should be noted that the above-mentioned first user intention may include only one user intention. For example, when the target natural language text is "open the car window", the first user intention only includes the intention to open the car window; of course, the above-mentioned first user intention may also include multiple user intentions. For example, when the target natural language text is "open the car window and air conditioning", the first user intention may include the intention to open the car window and the intention to turn on the air conditioning.

[0131] Step S103: Determine control information based on the first user intention using a preconfigured second language model, where the control information is associated with the target natural language text.

[0132] In this step, the second language model can be any pre-trained model that can predict vehicle control information based on user intent. For example, the second language model can include at least one of Tongyi Qianwen (e.g., Qwen2.5-7B), ChatGPT, and Llama.

[0133] It should be noted that the above-mentioned second largest language model and the above-mentioned first largest language model can be the same large language model, that is, the large language model can not only realize the recognition of user intentions, but also realize the prediction of vehicle control information; or, the two can also be different large language models, and the training process of the second largest language model can be similar to the training process of the first largest language model, which will not be repeated here.

[0134] The control information may be information that enables the vehicle to perform any control operation. For example, if the first user intent includes opening a window, the control information may include the window opening degree; or, if the first user intent also includes turning on the air conditioner, the control information may also include the air conditioner temperature, etc.

[0135] Step S104: Control the vehicle to perform a control operation based on the control information, and send a target answer language corresponding to the target natural language text to the user terminal, wherein the target answer language is used to indicate a control result of the control operation.

[0136] In this step, the off-vehicle interaction device can generate a control instruction carrying the above control information based on the control information, and control the vehicle to perform corresponding control operations based on the control instruction.

[0137] For example, when the above-mentioned control information includes the opening of the window, a window control instruction including the window opening can be generated, and the window can be controlled to open to an opening corresponding to the window opening through the window control instruction; or, when the above-mentioned control information also includes the air-conditioning temperature, an air-conditioning control instruction including the air-conditioning temperature can also be generated, and the air-conditioning can be controlled to open to the above-mentioned air-conditioning temperature through the air-conditioning control instruction.

[0138] When the off-vehicle interaction system completes the control operation, it can generate a target response corresponding to the target natural language text according to the control result of the control operation, and send the target response to the user terminal so that the user terminal outputs the target response, thereby realizing the off-vehicle interaction function.

[0139] The above-mentioned user terminal outputs the target answer language, which can be when the user terminal displays an off-vehicle interaction interface, by displaying the corresponding text of the target answer language in the off-vehicle interaction interface; or, the user terminal can also play the target answer language through voice broadcast.

[0140] In the embodiments of the present application, by receiving the target natural language text sent by the user terminal; inputting the target natural language text into a pre-configured first large language model, and obtaining a first user intention through recognition by the first large language model; controlling the vehicle to execute a control operation based on the control information; and sending a target response corresponding to the target natural language text to the user terminal. In this way, through the large language model, the target user intention of the user can be identified more intelligently and accurately, and the control information corresponding to the target user intention can be determined, so that the vehicle can be more flexibly controlled to execute the control operations required by the user and feedback the corresponding target response to the user, rather than being limited to a single instruction and feedback, thereby enhancing the interaction between the vehicle and the user.

[0141] In some embodiments, before inputting the target natural language text into a pre-configured first large language model and obtaining a user intention through recognition by the first large language model, it further includes:

[0142] In response to the target natural language text, query in a preset Frequently Asked Questions (FAQ) database for a user intention matching the target natural language text to obtain a query result;

[0143] In the case where the query result indicates that a second user intention matching the target natural language text is found, determine control information based on the second user intention;

[0144] The step of inputting the target natural language text into a pre-configured first large language model and obtaining a first user intention through recognition by the first large language model includes:

[0145] In the case where the query result indicates that no user intention matching the target natural language text is found, input the target natural language text into a pre-configured first large language model and obtain a first user intention through recognition by the first large language model.

[0146] In this embodiment, the user intention can be first queried through the FAQ database. In the case where a second user intention matching the target natural language text is found in the FAQ database, determine control information based on the second user intention; and in the case where no matching user intention is found in the FAQ database, use the first large language model to identify the first user intention, so as to ensure the efficiency of FAQ intention recognition and the flexibility of the large language model, and ensure the stability and accurate response of intention recognition in different scenarios.

[0147] The above-mentioned response to the target natural language text queries the user intention that matches the target natural language text in the preset frequent question answering database to obtain the query result. The off-vehicle interactive device may extract keywords from the target natural language text and query the user intention that matches the extracted keywords in the FAQ database.

[0148] For example, assuming that the FAQ database has preset user intent data of [charging pile query], when the target natural language text is "Where are the charging piles nearby", the off-vehicle interactive device can query [charging pile query] as the second user intent through the keyword "charging pile".

[0149] It should be noted that, when determining the second user intention, the off-vehicle interaction device can further determine the control information based on the second user intention through the above-mentioned second language model, and control the vehicle to perform corresponding control operations and send target response language corresponding to the natural language text based on the control information.

[0150] Alternatively, the FAQ database may also be configured with a correspondence between user intentions and control instructions. When the second user intention is determined, the off-vehicle interaction device may determine the control instructions corresponding to the second user based on the FAQ database, and control the vehicle to perform control operations and send target answer words corresponding to the natural language text based on the control instructions.

[0151] For example, when the above query finds [Charging pile query] as the second user intention, the off-vehicle interaction system can further determine the power query instruction through the FAQ database, control the vehicle to query the power based on the power query instruction, and send a reply based on the queried power. If the power is less than 30%, the user terminal will be replied with "Hey, the power is low! Don't worry, a nearby charging pile has been arranged for you!", and the charging pile applet will be launched; otherwise, the user terminal will be replied with "A nearby charging pile has been found for you."

[0152] In some implementations, identifying the first user intent using the first language model includes:

[0153] When the target natural language text is a natural language text in a multi-round dialogue process, obtaining dialogue information of at least one historical dialogue round associated with the target natural language text in the multi-round dialogue;

[0154] The first language model identifies the user intent based on the dialogue information of the at least one round of historical dialogue and the target natural language text to obtain a first user intent.

[0155] In this embodiment, during the multi-turn conversation process, the off-vehicle interaction device can obtain the conversation information of at least one round of historical conversation associated with the target natural language text of this round of conversation through the first large language model, and identify the user intention based on the conversation information of at least one round of historical conversation and the target natural language text to obtain the first user intention. Thus, based on context awareness, the semantic changes and associations of the user in the continuous conversation can be understood, the user intention can be more accurately identified, and then a control command more in line with the actual needs can be provided.

[0156] In this embodiment, the above multi-turn conversation can be understood as the user terminal continuously sending natural language texts to the off-vehicle interaction device multiple times respectively, and the off-vehicle interaction device responds to each natural language text, that is, performs corresponding control operations and gives corresponding response messages.

[0157] For example, the above multi-turn conversation can include: the user terminal sends the natural language text "Please help me turn on the seat heating" in the first round of conversation, the off-vehicle interaction device determines the user intention of [turning on the seat heating function], and controls the vehicle to perform the seat heating operation and reply with the corresponding response message; the user terminal sends the natural language text "The same goes for the window" in the second round of conversation, the off-vehicle interaction device determines the user intention of [opening the window], and controls the vehicle to perform the window opening operation and reply with the corresponding response message; the user terminal sends the target natural language text "The same goes for the air conditioner" in the third round of conversation.

[0158] The above-mentioned obtaining of at least one round of historical conversation associated with the target natural language text can be to determine the previous preset number of rounds of conversation before the target natural language text as the above-mentioned at least one round of historical conversation. For example, the preset number of rounds can be preset to 3 rounds, that is, the conversations of the previous 3 rounds are determined as historical conversations. Among them, each round of the above historical conversation can include the natural language text and the corresponding user intention.

[0159] The above-mentioned identification of the user intention by the first large language model based on the conversation information of at least one round of historical conversation and the target natural language text can be to combine the natural language texts in at least one round of historical conversation and the target natural language text, and input the combined natural language text into the first large language model, and the first large language model outputs the target user intention; or, it can also be to input the user intention of at least one round of historical conversation and the target natural language text into the first large language model, and the first large language model outputs the target user intention, which is not limited here.

[0160] For example, when the user terminal sends the target natural language text "The same goes for the air conditioner" in the third round of conversation, the off-vehicle interaction can identify the user intention of [Turn on the air conditioner] corresponding to the target natural language text "The same goes for the air conditioner" according to the natural language text or user intention in the previous two rounds of conversation, and control the vehicle to execute the air conditioner turning-on operation and reply with the corresponding response.

[0161] In some embodiments, before determining the control information based on the first user intention through the pre-configured second large language model, the method further includes:

[0162] Match the first user intention with a preset special user intention to obtain a first matching result;

[0163] When the first matching result indicates that the first user intention matches the target special user intention, perform an intention processing operation corresponding to the target special user intention to obtain a third user intention, and the third user intention is more in line with the actual operation logic of the vehicle than the first user intention;

[0164] Determining the control information based on the first user intention through the pre-configured second large language model includes:

[0165] Determine the control information based on the third user intention through the pre-configured second large language model..

[0166] In this embodiment, the off-vehicle interaction device can perform an intention processing operation on the identified first user intention for special user intentions, so that the processed first user intention is more in line with the actual operation logic of the vehicle, thereby improving the accuracy of the processed first user intention.

[0167] The above-mentioned target special user intentions can be some pre-set user intentions, which can be some user intentions with relatively complex execution operations or less obvious intentions, etc. For example, the above-mentioned target special user intentions can include user intentions such as [Reserve vehicle preparation], [Rapid heating], or [Rapid cooling].

[0168] Performing the above-mentioned intention processing operation corresponding to the target special user intention to obtain the processed first user intention can be that corresponding intention processing operation strategies are configured for each target special user intention, and the first user intention is subjected to an intention processing operation through this intention processing operation strategy, so that the obtained third user intention is more in line with the actual operation logic of the vehicle.

[0169] For example, a preset user intention that is more in line with actual operation logic can be configured for each target special user intention. When the first user intention is determined to be the target special user intention, the first user intention can be replaced by a preset user intention corresponding to the target user intention, and so on.

[0170] In some implementations, performing an intent processing operation corresponding to the target special user intent includes:

[0171] Determining at least one user sub-intent corresponding to the target special user intent;

[0172] Matching the target natural language text with each of the user sub-intents to obtain a second matching result, where the second matching result is used to indicate whether there is a preset text corresponding to the user sub-intent in the target natural language text;

[0173] Based on the second matching result, the target user sub-intent in the at least one user sub-intent is eliminated to obtain a third user intent, wherein the target user sub-intent is a user sub-intent whose corresponding preset text does not exist in the target natural language text.

[0174] In this embodiment, the off-vehicle interaction device can use at least part of the user sub-intention after removing the target user sub-intention from at least one user sub-intention as the third user intention, thereby eliminating the user sub-intentions that are irrelevant to the target natural language text in the at least one user sub-intention, making the third user intention more reasonable.

[0175] Determining the at least one user sub-intent corresponding to the target special user intent may involve decomposing the target special intent to obtain the at least one user sub-intent. Specifically, the off-vehicle interaction device may be pre-configured with user sub-intents corresponding to various special user intents. Upon determining that the first user intent matches the target special user intent, the off-vehicle interaction device may determine the user sub-intent corresponding to the target special user intent as the at least one user sub-intent.

[0176] For example, if the above-mentioned first user intention matches the special user intention [make a reservation for a car], and [make a reservation for a car] is pre-configured with two corresponding user sub-intents [navigate, I want to get in the car], then the two user sub-intents [navigate, I want to get in the car] corresponding to [make a reservation for a car] are split.

[0177] Based on the second matching result, removing the target user sub-intention from the at least one user sub-intention to obtain a third user intention may be that if the second matching result indicates that a preset text corresponding to the target user sub-intention does not exist in the target natural language text, the target user sub-intention is removed from the at least one user sub-intention, and the remaining user sub-intention after removing the at least one user sub-intention is determined as the third user intention; of course, if the second matching result indicates that there is no target user sub-intention, the at least one user sub-intention is determined as the third user intention.

[0178] For example, as Figure 2 shown, if the intention of [Reservation for Vehicle Preparation] is split into two user sub-intentions [Navigation, I want to get in the car], the vehicle departure interaction device can first determine whether the target natural language text contains the text "location" related to the user sub-intention [Navigation]. If the target natural language text contains the text "location", the user sub-intention [Navigation] is removed; if the target natural language text does not contain the text "location", it is further determined whether the target natural language text contains the text "car" related to the user sub-intention [I want to get in the car]. If the target natural language text contains the text "car", the user sub-intention [I want to get in the car] is removed; if the target natural language text does not contain the text "car" either, the result [Navigation, I want to get in the car] (i.e., the third user intention) is output.

[0179] In some embodiments, determining the control information based on the first user intention includes:

[0180] Matching a target entity keyword corresponding to the first user intention in a preset entity keyword set, where the preset entity keyword set includes at least one preset entity keyword, and each of the preset entity keywords is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the corresponding preset entity keyword;

[0181] Extracting first slot information of the first user intention based on the prompt corresponding to the target entity keyword;

[0182] Controlling the vehicle to perform a control operation based on the control information includes:

[0183] Controlling the vehicle to perform a control operation based on the first slot information.

[0184] In this embodiment, by determining the target entity keyword corresponding to the first user intention, a prompt corresponding to the target entity keyword is obtained. Then, based on the obtained prompt, the first slot information of the slot corresponding to the target entity keyword is extracted as the first slot information of the first user intention, and the vehicle is controlled to execute a control operation through the first slot information of the first user intention, so as to accurately determine the control information for controlling the vehicle to execute the control operation through the large language model.

[0185] The above preset entity keyword set can be preconfigured with at least one preset entity keyword, and the preset entity keyword can be a standardized term in vehicle control. For example, the preset entity keyword that can be matched can be defined through the regular expression `pattern` to form the following preset entity keyword set:

[0186] (Steering wheel|Fuel quantity|Endurance mileage|Air conditioner|Seat|Trunk|Air purification|Chat|I want to get in the car|Honk|Ventilate|Find the car|Unlock|Lock the car|Car lock|Tire pressure|Interior temperature|Battery level|Battery|Defrost|Door|Window|Flash lights|Light show|Navigation|Location|Delete / Modify / Query)

[0187] Matching the target entity keyword corresponding to the first user intention in the above preset entity keyword set can be based on the semantic association between each preset entity keyword and the first user intention; or the first user intention can be matched with each preset entity keyword through a preset mapping rule.

[0188] It should be noted that the above first user intention can match at least one target entity keyword. In the case where the first user intention matches multiple target entity keywords, multiple synonymous keywords among the multiple target entity keywords can be merged. For example, if "Unlock", "Lock the car", and "Car lock" are matched, they can be uniformly mapped to "Unlock the car"; in addition, alias conversion between preset entity keywords can also be performed according to preset special mapping words. For example, "Interior temperature" is converted to "Temperature", "Trunk" is converted to "Rear door", "Chat" is converted to "Other", etc.

[0189] Of course, in the case where the above first user intention does not match the preset entity keyword, the above second large language model can also perform a fallback process. For example, when the keyword is not hit or the match is abnormal, "Other" is returned.

[0190] In this implementation, each pre-configured entity keyword is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the corresponding pre-configured entity keyword. In a large language model, a slot is a structured field used to represent the key information to be extracted for a specific task. The design of the slot schema determines the type and format of information the model needs to extract.

[0191] The above prompt may only indicate the target natural language text in this round of conversation. At this time, the second largest language model may be the first slot information of the first user intention extracted from the target natural language text of the prompt.

[0192] For example, for the preset entity keyword [Delete, Modify, Query], the configuration can be as follows: The slot schema is preset to: {'command':"}, where the command value is limited to [Query, Delete, Modify]. This clarifies the user's operation instruction type for vehicle-related information and provides precise operational guidance for vehicle data management. System_prompt (i.e., prompt) = "You are an intelligent agent. Your task is to determine whether the user input belongs to the category of "Query", "Modify", or "Delete". "

[0193] In some embodiments, the prompt also includes indication information for indicating historical conversations.

[0194] The extracting the first slot information of the first user intention based on the prompt corresponding to the target entity keyword includes:

[0195] Based on the prompt corresponding to the target entity keyword, the first slot information of the first user intention is extracted from the historical conversation associated with the target natural language text and the target natural language text.

[0196] In this embodiment, by indicating the historical conversation in the prompt, the second largest language model can extract the first slot information from the historical conversation and the target natural language text of the current round of conversation based on the prompt, so that the extracted first slot information is more accurate.

[0197] The prompt may further include indication information for indicating a historical conversation, and the indication information may be any field or identifier that describes at least one round of historical conversation.

[0198] For example, for the preset entity keyword ["Delete, Modify, Query"], the configuration can be as follows: The slot schema is preset as: {'command': ""}, where the value of command is limited to ["Query", "Delete", "Modify"], which clarifies the type of operation instruction for the vehicle-related information of the user and provides accurate operation guidance for vehicle data management. System_prompt (i.e., the prompt) = "You are an intelligent agent, and your task is to determine which category of 'Query', 'Modify', or 'Delete' the user input belongs to based on the historical context." In this way, the second large language model can extract the first slot information of 'Query', 'Modify', or 'Delete' from the historical conversation and the target natural language text;

[0199] For the preset entity keyword ["Window"], the configuration can be as follows: The slot schema is designed as: {'value': ""}, which focuses on extracting the information about the window opening degree in the user input. Considering the diversity of user expression habits, this sub-module can flexibly process the descriptions of window opening degrees that contain Chinese or both Chinese and English mixed, and accurately obtain the specific parameters for window control. System_prompt = "You are an intelligent agent, and your task is to extract the window opening degree in the user input from the historical context. The window opening degree can contain Chinese or both Arabic numerals at the same time." In this way, the second large language model can extract the first slot information of the window opening degree from the historical conversation and the target natural language text;

[0200] For the preset entity keyword ["Seat"], the configuration can be as follows: The slot schema is designed as: {'position': "", 'function': ""}, which correspond to the seat position adjustment and function selection respectively. By carefully analyzing the user input, extract the specific position information for seat adjustment and the functions to be enabled, such as heating, ventilation, etc., to achieve personalized control of the seat system. System_prompt = "You are an intelligent agent, and your task is to extract the seat position and seat function in the user input from the historical context." In this way, the second large language model can extract the first slot information of the seat position and seat function from the historical conversation and the target natural language text;

[0201] For the preset entity keyword [air conditioner], the configuration can be as follows: Use the slot schema {'temp':"} to extract the air conditioner temperature setting information from the user input. It also supports temperature values expressed in Chinese and mixed Chinese and English, ensuring that the air conditioning system can accurately respond to the user's temperature adjustment needs. System_prompt = "You are an intelligent agent. Your task is to extract the air conditioner temperature from the user input based on the historical context. The air conditioner temperature can contain both Chinese characters and Arabic numerals." In this way, the second language model can extract the first slot information of the air conditioner temperature from the historical conversation and the target natural language text, and so on.

[0202] In some embodiments, the first user intention includes an intention to reserve a vehicle among preset special user intentions, and the slot of the prompt corresponding to the intention to reserve a vehicle includes a location and a time;

[0203] The extracting the first slot information of the first user intention based on the prompt corresponding to the target entity keyword includes:

[0204] Extracting the target location and target time of the vehicle preparation intention based on the prompt corresponding to the vehicle preparation entity keyword;

[0205] The controlling the vehicle to perform a control operation based on the first slot information includes:

[0206] The vehicle is controlled to generate a target reservation and preparation event, and the target reservation and preparation event is used to: instruct the vehicle to start the vehicle in advance before the target time arrives to perform a preset operation, so that the user can take the vehicle to the target location at the target time. The preset operation includes generating a navigation route with the target location as the destination.

[0207] In this embodiment, by configuring slots such as location and time for prompts corresponding to the keywords of the reservation vehicle entity, the large language model can extract the slot information of the user's reservation vehicle intention, and then accurately control the vehicle to realize the interaction of the reservation vehicle function.

[0208] When the off-vehicle interaction device recognizes the intention to make a reservation for the car, the off-vehicle interaction device can determine the reservation for the car entity keyword corresponding to the intention to make a reservation for the car, and obtain the prompt of the reservation for the car entity keyword, and then extract the target location and target time of the reservation for the car through the prompt of the reservation for the car entity keyword.

[0209] The time slot in the prompt corresponding to the above-mentioned intention of reserving a vehicle in advance can be used to clarify the information about when the user needs to use the vehicle. Specifically, the above-mentioned time can include date, time period, and specific time (which can also be called "time point"), etc.

[0210] For example, for the preset entity keyword [Reserving a vehicle in advance], the configuration can be as follows: The slot schema is designed as {'location': "", 'dayparts': "", 'time': "", 'date': ""}, comprehensively covering all key information elements required for reserving a vehicle in advance. Accurately extract parameters such as the location, time period, specific time, and date in the user input, providing a complete basis for the reservation preparation of the vehicle. System_prompt = "Reserving a vehicle in advance: It is stipulated that the task of the intelligent agent is to comprehensively extract key information for reserving a vehicle in advance, such as location (place), dayparts (time period), time (specific time), and date, by combining historical context with the user input, to ensure the accurate triggering of the reservation function.", in this way, the second large language model can extract the target location, target time period, target specific time, and target date from the historical conversation and the target natural language text. In this way, the off-vehicle interaction device can control the vehicle to generate a corresponding target event of reserving a vehicle in advance based on the target location, target time period, target specific time, and target date.

[0211] It should be noted that the above-mentioned preset operations can include generating a navigation route with the target location as the destination; or, the above-mentioned preset operations can also include at least one of starting the vehicle, turning on the air conditioner, opening the window, and adjusting the seat.

[0212] In some embodiments, after extracting the first slot information of the first user intention based on the prompt corresponding to the target entity keyword, it further includes:

[0213] Storing the first slot information of the first user intention into the target data structure;

[0214] Controlling the vehicle to perform control operations based on the first slot information includes:

[0215] Controlling the vehicle to perform control operations based on the target data structure.

[0216] In this embodiment, after extracting the first slot information of the first user intention, the off-vehicle interaction device can store the extracted first slot information into the target data structure and perform control operations on the vehicle based on the target data structure, so as to integrate the first slot information into the target data structure, providing a unified and orderly data basis for subsequent analysis, decision-making, or other processing links, which helps to improve processing efficiency and accuracy.

[0217] The above target data structure can be any data structure capable of caching data. For example, the data structure can be a forerunner data structure.

[0218] In some embodiments, controlling the vehicle to perform a control operation based on the first slot information includes:

[0219] Performing post-processing on the first slot information to obtain second slot information after post-processing;

[0220] Controlling the vehicle to perform a control operation based on the second slot information.

[0221] In this embodiment, by performing post-processing on the first slot information, the second slot information obtained by post-processing is more accurate, further improving the accuracy of the vehicle's control operation.

[0222] It should be noted that in the case of storing slot information through the target data structure, the above post-processing of the first slot information to obtain the second slot information after post-processing can be performed before storing the slot information in the target data structure, that is, after obtaining the second slot information by post-processing, storing the second slot information in the target data structure; or, it can also be performed after storing the first slot information in the target data structure. This is not limited herein.

[0223] The above post-processing of the first slot information can be at least one of slot value mapping, standardization, integration, and deduplication of the first slot information.

[0224] For example, assuming that the target entity keyword corresponding to the first user intention is [window], after the second large language model extracts the slot information of the window opening value, the following post-processing can be performed:

[0225] Slot value mapping

[0226] Generate two types of results based on the extracted value:

[0227] Original value mapping:

[0228] result_value = value if value is not None else ""

[0229] Opening degree mapping:

[0230] Classify based on predefined business rules:

[0231] If value == "one o'clock", then degree = "one o'clock";

[0232] If value is None, then degree = "all";

[0233] In other cases (such as a specific value "30%"), degree = "specific opening";

[0234] Finally, result_degree = degree is generated.

[0235] Results integration

[0236] Store result_value and result_degree into the result container forerunner dictionary and return the updated context object.

[0237] For another example, assuming the target entity keyword corresponding to the first user intent is [seat], after the second language model extracts the slot information of the seat position and seat function, the following post-processing can be performed:

[0238] First, check whether the value corresponding to the 'position' key in the result exists. If so, assign it to the position variable. If not, assign position to 'None' to clearly indicate that no valid position information was extracted. To standardize the representation of position information and facilitate subsequent analysis and integration, further check the position value. If position is 'Primary Back' or 'Secondary Back', convert it to 'Back Row' to achieve standardized representation of position information.

[0239] For another example, suppose the target entity keyword corresponding to the first user intent is [book a car], after the second language model extracts the slot information of location, time period, specific time, and date, the following post-processing can be performed:

[0240] 1) Post-processing of navigation address information.

[0241] By receiving two parameters, forerunner and slots_info, extracting location information from slots_info, formatting it into a specific nested dictionary structure, and then updating it to the result_loc attribute of the forerunner object. If the location information is empty, an empty dictionary is assigned, and finally the updated forerunner object is returned. This provides a unified and standardized data format for subsequent navigation-related operations, ensuring data consistency and availability to support the subsequent implementation and expansion of navigation functions.

[0242] 2) Time and date post-processing process

[0243] 1. Date completion processing

[0244] When detecting a date string that contains the month but lacks the day, such as "May 3rd," use the regular expression `(\d+)月\s(\d+)` to accurately match this format, ensuring that other formats are not accidentally detected. The "day" is added to the end of the matching string to standardize it to "May 3rd."

[0245] 2. Date standardization

[0246] Special cycle processing:

[0247] Working day: If the date information contains both "working day" and "every", it will be converted to "every Monday to Friday"; if it only contains "working day" without "every", it will be regarded as an ordinary working day, that is, "Monday to Friday".

[0248] Weekend: If the date information contains both "weekend" and "every", it will be converted to "every Saturday and Sunday"; if it only contains "weekend", it will be an ordinary Saturday and Sunday.

[0249] Unified terminology processing:

[0250] Replace "天" with "日" uniformly, for example, replace "Sunday" with "当时", to standardize date expression.

[0251] Replace "week" with "week", for example, replace "Wednesday" with "Wednesday" to ensure terminology consistency.

[0252] Cycle identification processing:

[0253] If the date information contains period identifiers such as "every" or "next week", these identifiers are retained. For example, "every Wednesday" is directly retained as the period identifier.

[0254] For combined date information containing keywords such as "week" and "day of the week", split it into a list. For example, "next Wednesday and Thursday" is split into ["next Wednesday", "next Thursday"] to facilitate subsequent processing.

[0255] Relative date processing: Directly extract relative date terms such as "today", "tomorrow", and "the day after tomorrow" without additional conversion, maintaining their relative meaning.

[0256] 3. Time standardization

[0257] Key processing logic:

[0258] Clean up irrelevant words: Remove non-time words such as "today", "tomorrow", and "immediately". For example, clean up "tomorrow at five o'clock in the afternoon" to "five o'clock in the afternoon" to accurately extract time information.

[0259] Fix repeated words: For incorrect inputs like "8 o'clock in the morning", correct the repeated words and standardize them to "8 o'clock in the morning".

[0260] Separate date interference: If the time field accidentally contains date information (such as "5 o'clock on Saturday"), on the premise that the `date` field has been correctly extracted, try to remove the date part from the time field to avoid interference.

[0261] 4. Time period extraction

[0262] Extraction logic:

[0263] Use regular expression

pattern=r'(morning|noon|evening|forenoon|afternoon|daytime|night|dawn|midnight|dusk)'

[0264] 5. Data integration and deduplication

[0265] Key steps:

[0266] 1. Deduplication processing: Deduplicate the date list and time list respectively through `list(set(...))` to remove duplicate date and time information and ensure the uniqueness of the data.

[0267] 2. Field update: Write the processed date (`result_date`), time (`result_time`), and time period (`result_dayparts`) into the `forerunner` object to complete data integration and update.

[0268] 6. Periodic task judgment

[0269] Judgment logic:

[0270] If the `date` field is not empty and the first element contains "every" (such as "every Wednesday"), then it is judged as a periodic task, and mark `result_ifcycle = True`; otherwise, if the above conditions are not met, it is marked as a non-periodic task,

[0271] `result_ifcycle = False`.

[0272] Final output result, the `forerunner` object contains the following fields:

[0273] `result_date`: The standardized date list, such as `["every Wednesday"]`.

[0274] `result_time`: The list of times after cleaning, e.g., `["five o'clock"]`.

[0275] `result_dayparts`: The list of time periods, e.g., `["afternoon"]`.

[0276] `result_ifcycle`: The periodic task identifier, with values of `True` or `False`.

[0277] Through the collaborative operation of the above modules, this intelligent vehicle interaction system can achieve all-round and in-depth processing of user input, accurately understand user intentions, efficiently execute corresponding instructions, and provide users with a convenient and intelligent interaction experience.

[0278] In some embodiments, the post-processing of the first slot information in the target data structure to obtain the target data structure after post-processing includes:

[0279] When the first slot information includes a periodic field, generating a periodic identifier corresponding to the periodic field, where the second slot information includes the periodic identifier;

[0280] The controlling the vehicle to perform a control operation based on the second slot information includes:

[0281] Controlling the vehicle to perform a control operation based on the second slot information according to the period corresponding to the periodic identifier.

[0282] In this embodiment, by generating a periodic identifier corresponding to the periodic field in the first slot information and adding it to the first slot information to form the second slot information, the vehicle can be controlled to perform control operations periodically based on the periodic identifier in vehicle control.

[0283] For example, when the slot information corresponding to

Reservation for Vehicle Preparation

[0284] In some embodiments, the method further includes:

[0285] If the first slot information corresponding to the first user intention is not extracted by the second large language model, generating a preset character identifier;

[0286] In response to the preset character identifier, sending the first fallback response to the user terminal.

[0287] In this embodiment, when the second largest language model fails to extract the first slot information, the off-vehicle interaction device can generate a preset character identifier and send a first catch-all answer to the user terminal in response to the preset character identifier, thereby ensuring the continuity of the processing flow and improving the user interaction experience.

[0288] For example, assuming that the target entity keyword corresponding to the first user intent is [car window], if the second language model does not extract the slot information of the car window opening value, then the empty value initialization value = None (i.e., the preset character identifier) is executed, and None is stored in the forerunner data structure. The off-vehicle device can feedback the corresponding fallback answer based on None.

[0289] In some implementations, after identifying the first user intent using the first language model, the method further includes:

[0290] Determining a target confidence level associated with the first language model and the first user intent;

[0291] When the target confidence is less than a confidence threshold, obtaining a second fallback answer corresponding to the target natural language text;

[0292] Send the second catch-all answer to the user terminal.

[0293] In this embodiment, when the first language model cannot recognize the correct first user intention (i.e., the target confidence is less than the confidence threshold), the off-vehicle interaction device can send a second fallback answer to the user terminal, thereby ensuring the continuity of the processing flow and improving the user interaction experience.

[0294] To facilitate understanding of the off-vehicle interaction method in the embodiment of the present application, an application example of the off-vehicle interaction method is provided here, such as Figure 3 The specific processing process is as follows:

[0295] Upon receiving a user request (i.e., a request sent by a user terminal carrying a target natural language text), performing FAQ matching on the target natural language text in the user request (i.e., searching a preset frequently asked questions database for user intentions that match the target natural language text);

[0296] If the FAQ matches the user intent (i.e., the second user intent), the FAQ matching result (carrying the second user intent) is returned, and the vehicle is controlled to perform control operations and feedback answer language based on the FAQ matching result;

[0297] If the user intention is not recognized in the FAQ, multi-round intention recognition is performed on the target natural language text in the user request, that is, through a large language model, user intention recognition is performed based on multi-round historical conversations and the target natural language text, and the recognized user intentions are generated into an intention list;

[0298] Preprocess the intention list, such as removing duplicates;

[0299] Determine whether the preprocessed intention list contains the special intention [Reservation for Vehicle Preparation] (i.e., the target special user intention);

[0300] If it is determined that the special intention [Reservation for Vehicle Preparation] is included in the intention list, then split the special intention [Reservation for Vehicle Preparation] into two sub-intentions [Navigation, Vehicle Usage], perform intention processing operations on the two sub-intentions [Navigation, Vehicle Usage], extract the slot information of the sub-intentions after the intention processing operations, including the slot information of time and address, and perform post-processing on the slot information of time and address, such as standardizing time and address. After post-processing, generate reservation for vehicle preparation parameters, and store the reservation for vehicle preparation parameters in a target data structure (such as a forerunner data structure) for slot integration, and return a structured result;

[0301] If it is determined that the special intention [Reservation for Vehicle Preparation] is not included in the intention list, then in the case where the intention list contains user intentions corresponding to at least one entity keyword among the preset entity keywords [Window], [Air Conditioner], [Navigation], and [Seat], perform intention processing operations on the user intentions corresponding to at least one entity keyword, obtain the corresponding slot information, and store the slot information in a target data structure (such as a forerunner data structure) for slot integration, and return a structured result.

[0302] In the off-vehicle interaction method provided by the embodiments of the present application, the execution subject can be an off-vehicle interaction device. In the embodiments of the present application, taking the off-vehicle interaction device executing the off-vehicle interaction method as an example, the off-vehicle interaction device provided by the embodiments of the present application is described.

[0303] Figure 4 It is a schematic structural diagram of the off-vehicle interaction device 400 provided by the embodiments of the present application. As Figure 4 shown, the off-vehicle interaction device 400 of the present application includes:

[0304] A text receiving module 401, configured to receive the target natural language text sent by the user terminal;

[0305] An intention recognition module 402, configured to input the target natural language text into a pre-configured first large language model, and recognize a first user intention through the first large language model;

[0306] a control information determination module 403, configured to determine control information based on the first user intention using a preconfigured second language model, wherein the control information is associated with the target natural language text;

[0307] The control module 404 is used to control the vehicle to perform a control operation based on the control information, and to send a target answer language corresponding to the target natural language text to the user terminal, wherein the target answer language is used to indicate a control result of the control operation.

[0308] In some embodiments, the apparatus 400 further comprises:

[0309] a database query module, configured to query a preset frequently asked questions database for user intentions that match the target natural language text in response to the target natural language text, and obtain a query result;

[0310] A control information determination module 403 is configured to determine control information based on a second user intent when the query result indicates that the query obtains a second user intent that matches the target natural language text;

[0311] The intention recognition module 402 is specifically configured to:

[0312] If the query result indicates that no user intent matching the target natural language text is found, the target natural language text is input into a preconfigured first language model, and a first user intent is identified by the first language model.

[0313] In some implementations, the intention recognition module 402 is specifically configured to:

[0314] When the target natural language text is a natural language text in a multi-round dialogue process, obtaining dialogue information of at least one historical dialogue round associated with the target natural language text in the multi-round dialogue;

[0315] The first language model identifies the user intent based on the dialogue information of the at least one round of historical dialogue and the target natural language text to obtain a first user intent.

[0316] In some embodiments, the apparatus 400 further comprises:

[0317] a special intent matching module, configured to match the first user intent with a preset special user intent to obtain a first matching result;

[0318] an intent processing module configured to, when the first matching result indicates that the first user intent matches a target special user intent, perform an intent processing operation corresponding to the target special user intent to obtain a third user intent, the third user intent being more consistent with an actual operating logic of the vehicle than the first user intent;

[0319] The control information determination module 403 is specifically configured to:

[0320] The control information is determined based on the third user intention by using a preconfigured second language model.

[0321] In some embodiments, the intention processing module is specifically configured to:

[0322] Determining at least one user sub-intent corresponding to the target special user intent;

[0323] Matching the target natural language text with each of the user sub-intents to obtain a second matching result, where the second matching result is used to indicate whether there is a preset text corresponding to the user sub-intent in the target natural language text;

[0324] Based on the second matching result, the target user sub-intent in the at least one user sub-intent is eliminated to obtain a third user intent, wherein the target user sub-intent is a user sub-intent whose corresponding preset text does not exist in the target natural language text.

[0325] In some implementations, the control information determination module 403 is specifically configured to:

[0326] Matching a target entity keyword corresponding to the first user intent in a preset entity keyword set, wherein the preset entity keyword set includes at least one preset entity keyword, and each preset entity keyword is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the corresponding preset entity keyword;

[0327] Extracting first slot information of the first user intent based on a prompt corresponding to the target entity keyword;

[0328] The control module 404 is specifically configured to:

[0329] The vehicle is controlled to perform a control operation based on the first slot information.

[0330] In some embodiments, the prompt also includes indication information for indicating historical conversations.

[0331] The control information determination module 403 is specifically configured to:

[0332] Based on the prompt corresponding to the target entity keyword, the first slot information of the first user intention is extracted from the historical conversation associated with the target natural language text and the target natural language text.

[0333] In some embodiments, the first user intention includes an intention to reserve a vehicle among preset special user intentions, and the slot of the prompt corresponding to the intention to reserve a vehicle includes a location and a time;

[0334] The control information determination module 403 is specifically configured to:

[0335] Extracting the target location and target time of the vehicle preparation intention based on the prompt corresponding to the vehicle preparation entity keyword;

[0336] The control module 404 is specifically configured to:

[0337] The vehicle is controlled to generate a target reservation and preparation event, and the target reservation and preparation event is used to: instruct the vehicle to start the vehicle in advance before the target time arrives to perform a preset operation, so that the user can take the vehicle to the target location at the target time. The preset operation includes generating a navigation route with the target location as the destination.

[0338] In some embodiments, the apparatus 400 further includes:

[0339] a storage module, configured to store the first slot information of the first user intention into a target data structure;

[0340] The control module 404 is specifically configured to:

[0341] The vehicle is controlled to perform a control operation based on the target data structure.

[0342] In some implementations, the control module 404 is specifically configured to:

[0343] Post-processing the first slot information to obtain post-processed second slot information;

[0344] The vehicle is controlled to perform a control operation based on the second slot information.

[0345] In some implementations, the control module 404 is specifically configured to:

[0346] In a case where the first slot information includes a periodicity field, generating a periodicity identifier corresponding to the periodicity field, wherein the second slot information includes the periodicity identifier;

[0347] Control the vehicle to perform a control operation based on the second slot information according to the period corresponding to the periodic identifier.

[0348] In some embodiments, the apparatus 400 further includes:

[0349] An identifier generation module, configured to generate a preset character identifier if the first slot information corresponding to the first user intention is not extracted by the second large language model;

[0350] A first fallback module, configured to send a first fallback response to the user terminal in response to the preset character identifier.

[0351] In some embodiments, the apparatus 400 further includes:

[0352] A confidence determination module, configured to determine a target confidence level associated with the first user intention by the first large language model;

[0353] A fallback response acquisition module, configured to acquire a second fallback response corresponding to the target natural language text when the target confidence level is less than a confidence threshold;

[0354] A fallback response sending module, configured to send the second fallback response to the user terminal.

[0355] The off-vehicle interaction apparatus 400 provided by the embodiments of the present application can execute the technical solutions shown in the above embodiments of the off-vehicle interaction method, and its implementation principle and beneficial effects are similar, and will not be described in detail here.

[0356] Figure 5 It is a schematic hardware structure diagram of an off-vehicle interaction device provided by an embodiment of the present application.

[0357] The off-vehicle interaction device may include a processor 501 and a memory 502 storing computer program instructions. The off-vehicle interaction device may be an electronic device or a vehicle.

[0358] Specifically, the above-mentioned processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0359] The memory 502 may include a mass storage for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 502 may include removable or non-removable (or fixed) media. Where appropriate, the memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, the memory 502 is a non-volatile solid-state memory.

[0360] In some embodiments, the memory 502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the off-vehicle interaction method according to the present application.

[0361] The processor 501 reads and executes the computer program instructions stored in the memory 502 to implement the off-vehicle interaction method in the above embodiments.

[0362] In one example, the off-vehicle interaction device may further include a communication interface 503 and a bus 510. Among them, as Figure 5 shown, the processor 501, the memory 502, and the communication interface 503 are connected through the bus 510 to complete the communication with each other.

[0363] The communication interface 503 is mainly used to implement the communication between the various modules, devices, units, and / or devices in the embodiments of the present application.

[0364] The bus 510 includes hardware, software, or both, and couples the components of the on-line data flow metering device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, the bus 510 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0365] The off-vehicle interaction device can execute the off-vehicle interaction method in the embodiments of the present application, so as to implement the off-vehicle interaction method and device described in combination with Figures 1 to 4 the off-vehicle interaction method and device described.

[0366] In addition, in combination with the off-vehicle interaction method in the above embodiments, the embodiments of the present application also provide a computer storage medium to implement. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, the off-vehicle interaction method in the above embodiments is implemented.

[0367] In combination with the off-vehicle interaction method in the above embodiments, the embodiments of the present application also provide a computer program product. When the instructions in the computer program product are executed by the processor of the off-vehicle interaction device, the off-vehicle interaction device implements the off-vehicle interaction method in the above embodiments.

[0368] The vehicle can be a private car, such as a sedan, an SUV, an MPV, or a pickup truck, etc. The vehicle can also be an operating vehicle, such as a minivan, a bus, a small truck, or a large trailer, etc. The vehicle can be an oil vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it can be a hybrid vehicle or a pure electric vehicle.

[0369] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, the detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0370] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave over a transmission medium or a communication link. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0371] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems according to a series of steps or devices. However, the present application is not limited to the order of the above steps. That is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0372] The aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of methods (systems) and computer program products according to embodiments of the present application. It should be understood that each block in the flowchart and / or block diagram, and the combination of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing devices to produce a machine such that these instructions executed by the processor of the computer or other programmable data processing devices enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It can also be understood that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0373] As described above, the above is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A method for off-vehicle interaction, characterized in that Including: Receiving a target natural language text sent by a user terminal; Inputting the target natural language text into a pre-configured first large language model, and identifying a first user intention through the first large language model; Determining control information based on the first user intention through a pre-configured second large language model, where the control information is associated with the target natural language text; Controlling the vehicle to perform a control operation based on the control information, and sending a target response corresponding to the target natural language text to the user terminal, where the target response is used to indicate the control result of the control operation.

2. The method according to claim 1, characterized in that, Before inputting the target natural language text into the pre-configured first large language model and identifying the user intention through the first large language model, it further includes: In response to the target natural language text, querying in a preset frequently asked questions and answers database for a user intention matching the target natural language text to obtain a query result; In the case where the query result indicates that a second user intention matching the target natural language text is queried, determining control information based on the second user intention; The step of inputting the target natural language text into the pre-configured first large language model and identifying the first user intention through the first large language model includes: In the case where the query result indicates that no user intention matching the target natural language text is queried, inputting the target natural language text into the pre-configured first large language model and identifying the first user intention through the first large language model.

3. The method according to claim 1, characterized in that, The step of identifying the first user intention through the first large language model includes: In the case where the target natural language text is a natural language text in a multi-round conversation, obtaining the conversation information of at least one round of historical conversation associated with the target natural language text in the multi-round conversation; The first large language model identifies the user intention based on the conversation information of the at least one round of historical conversation and the target natural language text to obtain the first user intention.

4. The method according to claim 1, wherein Before controlling the vehicle to perform a control operation based on the control information, the method further includes: Matching the first user intention with a preset special user intention to obtain a first matching result; In the case where the first matching result indicates that the first user intention matches a target special user intention, performing an intention processing operation corresponding to the target special user intention to obtain a third user intention, where the third user intention is more in line with the actual operation logic of the vehicle than the first user intention; The step of determining control information based on the first user intention through the pre-configured second large language model includes: Determining control information based on the third user intention through the pre-configured second large language model.

5. The method according to claim 4, characterized in that, The step of performing the intention processing operation corresponding to the target special user intention includes: Determining at least one user sub-intention corresponding to the target special user intention; Match the target natural language text with each of the user sub-intents to obtain a second matching result, where the second matching result is used to indicate whether there is a preset text corresponding to the user sub-intent in the target natural language text; Based on the second matching result, eliminate the target user sub-intent among the at least one user sub-intents to obtain a third user intent, where the target user sub-intent is the user sub-intent for which the corresponding preset text does not exist in the target natural language text.

6. The method according to claim 1, characterized in that, The determining the control information based on the first user intent includes: Match a target entity keyword corresponding to the first user intent in a preset entity keyword set, where the preset entity keyword set includes at least one preset entity keyword, and each of the preset entity keywords is pre-configured with a corresponding prompt, and the prompt includes a slot corresponding to the corresponding preset entity keyword; Extract first slot information of the first user intent based on the prompt corresponding to the target entity keyword; The controlling the vehicle to perform a control operation based on the control information includes: Control the vehicle to perform a control operation based on the first slot information.

7. The method according to claim 6, wherein The prompt further includes indication information for indicating a historical conversation, The extracting first slot information of the first user intent based on the prompt corresponding to the target entity keyword includes: Extract first slot information of the first user intent from a historical conversation associated with the target natural language text and the target natural language text based on the prompt corresponding to the target entity keyword.

8. The method according to claim 6, characterized in that, The first user intent includes a reservation vehicle preparation intent among preset special user intents, and the slots of the prompt corresponding to the reservation vehicle preparation intent include a location and a time; The extracting first slot information of the first user intent based on the prompt corresponding to the target entity keyword includes: Extract a target location and a target time of the reservation vehicle preparation intent based on the prompt corresponding to the reservation vehicle preparation entity keyword; The controlling the vehicle to perform a control operation based on the first slot information includes: Control the vehicle to generate a target reservation vehicle preparation event, where the target reservation vehicle preparation event is used to: indicate that the vehicle starts in advance before the target time arrives to perform a preset operation to prepare for the user to travel to the target location in the vehicle at the target time, and the preset operation includes generating a navigation route with the target location as the destination.

9. An off-vehicle interaction device, characterized in that, Includes: A text receiving module, configured to receive a target natural language text sent by a user terminal; An intent recognition module, configured to input the target natural language text into a pre-configured first large language model, and recognize a first user intent through the first large language model; A control information determining module, configured to determine control information based on the first user intent through a pre-configured second large language model, where the control information is associated with the target natural language text; A control module, configured to control the vehicle to perform a control operation based on the control information, and send a target response corresponding to the target natural language text to the user terminal, where the target response is used to indicate the control result of the control operation.

10. A vehicle-off interaction device, characterized in that, The off-vehicle interaction device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the off-vehicle interaction method according to any one of claims 1-8 is implemented.

11. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium; When the computer program instructions are executed by the processor, the off-vehicle interaction method according to any one of claims 1-8 is implemented.

Images