Vehicle control method, server and computer readable storage medium

Abstract

The invention discloses a vehicle control method, a server and a computer readable storage medium. The method comprises the following steps: receiving a first voice request forwarded by a vehicle, current display content information of vehicle display parts and current perception information of the vehicle; and based on a preset model, according to the first voice request, the current display content information and the current perception information, determining a target execution action. And issuing the target execution action to the vehicle to control the vehicle to execute the target execution action. Thus, the target execution action needing to be executed is determined by analyzing the content of the first voice request, understanding the currently displayed content and the perception information of the vehicle, so that the driving convenience and safety are improved, meanwhile, distraction of a driver is reduced, and the overall user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a vehicle control method, a server and a computer-readable storage medium. Background Art

[0002] In the related art, human-computer interaction is performed with users through in-vehicle voice assistants to facilitate user operation. However, the design and training of in-vehicle voice assistants are mainly focused on general functions, which limits the depth and efficiency of interaction between users and in-vehicle voice assistants and affects user experience. Summary of the invention

[0003] The present application provides a vehicle control method, a server and a computer-readable storage medium.

[0004] The present application provides a vehicle control method, the method comprising:

[0005] Receiving a first voice request forwarded by a vehicle, current display content information of a display component of the vehicle, and current perception information of the vehicle;

[0006] Based on a preset model, determining a target execution action according to the first voice request, the currently displayed content information, and the currently perceived information;

[0007] The target execution action is sent to the vehicle to control the vehicle to execute the target execution action.

[0008] In this way, the server receives the first voice request forwarded by the vehicle, the current display content information of the vehicle's display components, and the current perception information of the vehicle. Then, based on the preset model, the server determines the target execution action according to the first voice request, the current display content information, and the current perception information. Finally, the server sends the target execution action to the vehicle to control the vehicle to execute the target execution action. In this way, by analyzing the content of the first voice request, understanding the currently displayed content and the vehicle's perception information, and determining the target execution action that needs to be executed, the convenience and safety of driving can be improved, while reducing the driver's distraction and improving the overall user experience.

[0009] In some implementations, determining a target execution action according to the first voice request, the currently displayed content information, and the currently perceived information includes:

[0010] Based on a preset task rewriting template, rewriting the currently displayed content information, the first voice request, and the currently perceived information to obtain first input information;

[0011] The target execution action is determined according to the first input information.

[0012] In this way, based on the preset task rewriting template, the server rewrites the current display content information, the first voice request and the current perception information to obtain the first input information. Then, the server determines the target execution action based on the first input information. In this way, through the semantic understanding of the first voice request, the analysis and interpretation of the display content, and the integration and interpretation of the perception information, the server can convert the original information into a form that the system can understand and process, that is, the first input information, so as to determine the target execution action to be performed.

[0013] In some implementations, determining the target execution action according to the first input information includes:

[0014] Analyzing and processing the first input information;

[0015] Determining a target execution task and a task type of the target execution task according to the result of the analysis and processing;

[0016] The target execution action is determined according to the task type, the target execution task and the currently displayed content information.

[0017] In this way, the server analyzes and processes the first input information. Then, the server determines the target execution task and the task type of the target execution task based on the results of the analysis and processing. Finally, the server determines the target execution action based on the task type, the target execution task and the currently displayed content information. In this way, by analyzing and processing the first input information and processing the results of the analysis and processing, determining the target execution task and the task type of the target execution task, and determining the target execution action, the user needs are understood and responded to, and the user experience is improved.

[0018] In some implementations, determining the target execution action according to the task type, the target execution task, and the currently displayed content information includes:

[0019] In the case where the task type is a first task type, the target execution action is determined according to the target execution task and the currently displayed content information, wherein the first task type is a task type with complete semantic information and a unique execution path.

[0020] In this way, when the task type is the first task type, the server determines the target execution action according to the target execution task and the currently displayed content information, wherein the first task type is a task type with complete semantic information and a unique execution path. In this way, when the task type is the first task type, the server can respond to user needs quickly and accurately, improve user satisfaction and loyalty, and provide users with better services.

[0021] In some implementations, determining the target execution action according to the task type, the target execution task, and the currently displayed content information includes:

[0022] In the case where the task type is a second task type, processing the target execution task and the current display content information to confirm missing slot information, the second task type being a task type with missing semantic information;

[0023] The target execution action is determined according to the missing slot information.

[0024] In this way, when the task type is the second task type, the server processes the target execution task and the current display content information, confirms the missing slot information, and the second task type is a task type with missing semantic information. Then, the server determines the target execution action based on the missing slot information. In this way, when the task type is the second task type, by confirming the missing slot information and requesting the user to provide relevant information, the server's questioning ability is improved, so that it can better understand user needs and improve the intelligence and interactivity of the system.

[0025] In some implementations, the processing of the target execution task and the currently displayed content information to confirm the missing slot information includes:

[0026] Based on the preset task database, according to the first input information, confirming the target preset task and the target preset task slot corresponding to the first input information;

[0027] The missing slot information is determined according to the first input information and the target preset task slot.

[0028] In this way, the server confirms the target preset task and the target preset task slot corresponding to the first input information based on the preset task database and the first input information. Then, the server determines the missing slot information based on the first input information and the target preset task slot. In this way, by using the preset task database, the system can confirm the target preset task and the target preset task slot corresponding to the first input information, thereby improving the accuracy of task processing.

[0029] In some implementations, determining the target execution action according to the missing slot information includes:

[0030] generating first inquiry information according to the missing slot information, and sending the inquiry information to the vehicle;

[0031] receiving a second voice request for the first follow-up information, the current display content information, and the current perception information forwarded by the vehicle;

[0032] Based on the preset model, the target execution action is generated according to the first voice request, the second voice request, the currently displayed content information and the current perception information.

[0033] In this way, the server generates a first follow-up message based on the missing slot information and sends the follow-up message to the vehicle. Next, the server receives the second voice request, the current display content information, and the current perception information forwarded by the vehicle for the first follow-up message. Finally, based on the preset model, the server generates a target execution action based on the first voice request, the second voice request, the current display content information, and the current perception information. In this way, by asking questions and collecting missing information, the system can understand and execute the user's tasks more accurately, thereby improving the accuracy of task completion.

[0034] In some implementations, determining the target execution action according to the task type, the target execution task, and the currently displayed content information includes:

[0035] In the case where the task type is the third task type, processing the target execution task and the current display content information, confirming the second inquiry information, and sending the second inquiry information to the vehicle, wherein the third task type is a task type with complete semantic information and a non-unique execution path;

[0036] receiving a third voice request for the second follow-up information, the current display content information, and the current perception information forwarded by the vehicle;

[0037] Based on the preset model, the target execution action is generated according to the first voice request, the third voice request, the currently displayed content information and the current perception information.

[0038] In this way, when the task type is the third task type, the server processes the target execution task and the currently displayed content information, confirms the second follow-up information, and sends the second follow-up information to the vehicle, wherein the third task type is a task type with complete semantic information and a non-unique execution path. Next, the server receives the third voice request for the second follow-up information, the currently displayed content information, and the current perception information forwarded by the vehicle. Finally, based on the preset model, the server generates a target execution action according to the first voice request, the third voice request, the currently displayed content information, and the current perception information. In this way, by processing the target execution task and the currently displayed content information and confirming the second follow-up information, the user's choice or preference can be obtained through interaction, thereby improving the flexibility of task completion and enhancing the user experience.

[0039] In certain embodiments, the method further comprises:

[0040] When the vehicle completes the target execution action, a preset feedback generation template is generated, and voice broadcast information is generated according to the current perception information. The voice broadcast information includes execution completion status information and / or action suggestions of the target execution action.

[0041] In this way, when the vehicle completes the target execution action, the server generates a template based on the preset feedback and generates voice broadcast information based on the current perception information. The voice broadcast information includes the execution completion status information and / or action suggestions of the target execution action. In this way, by providing real-time execution completion status information and action suggestions, the system can help the driver better deal with various situations during driving, thereby improving the user experience.

[0042] An embodiment of the present application provides a server, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned vehicle control method is implemented.

[0043] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the vehicle control method described above are implemented.

[0044] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0046] Figure 1 It is one of the flowcharts of the vehicle control method of certain embodiments of the present application;

[0047] Figure 2 is a schematic diagram of a processing flow of a voice request in certain embodiments of the present application;

[0048] Figure 3 This is a second flow chart of a vehicle control method according to certain embodiments of the present application;

[0049] Figure 4 This is a third flow chart of a vehicle control method according to certain embodiments of the present application;

[0050] Figure 5 This is a fourth flow chart of a vehicle control method according to certain embodiments of the present application;

[0051] Figure 6This is a fifth flow chart of a vehicle control method according to certain embodiments of the present application;

[0052] Figure 7 This is the sixth flow chart of the vehicle control method of certain embodiments of the present application;

[0053] Figure 8 This is the seventh flow chart of the vehicle control method of certain embodiments of the present application;

[0054] Fig. 9 This is a flowchart of a vehicle control method according to some embodiments of the present application;

[0055] Fig.10 This is the ninth flow chart of the vehicle control method of certain embodiments of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0057] In the field of smart cars, in-vehicle voice assistants have become an important technology to improve driving experience and safety. These assistants are usually able to understand and respond to users' voice commands and perform common functions such as navigation, phone calls, music playback, etc. However, although these functions provide convenience to users, the design and training of in-vehicle voice assistants often focus on these common functions, which to some extent limits the depth and efficiency of users' interaction with in-vehicle voice assistants, thus affecting the user experience.

[0058] First, because the in-car voice assistant is mainly trained for general functions and cannot understand or respond to more complex or specific commands from users, the in-car voice assistant has poor interactivity with users, resulting in a poor user experience.

[0059] Second, in-vehicle voice assistants lack the ability to process the vehicle's current perception information. For example, they may not be able to adjust the interaction method or provide more appropriate suggestions based on the vehicle's status information or changes in the driving environment.

[0060] Based on the above questions, please refer to Figure 1 , the embodiment of the present application provides a vehicle control method, the method comprising:

[0061] 01: receiving the first voice request forwarded by the vehicle, the current display content information of the vehicle display components, and the current perception information of the vehicle;

[0062] 02: Based on the preset model, determine the target execution action according to the first voice request, the current display content information and the current perception information;

[0063] 03: Send the target execution action to the vehicle to control the vehicle to execute the target execution action.

[0064] The embodiment of the present application also provides a server, including a memory and a processor. The vehicle control method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive a first voice request forwarded by the vehicle, current display content information of the vehicle display components, and current perception information of the vehicle. And based on a preset model, according to the first voice request, the current display content information and the current perception information, determine the target execution action. And send the target execution action to the vehicle to control the vehicle to execute the target execution action.

[0065] The embodiment of the present application also provides a vehicle control device. The vehicle control method of the embodiment of the present application can be implemented by the vehicle control device of the embodiment of the present application. Specifically, the vehicle control device includes a receiving module, a determining module and a sending module. The receiving module is used to receive a first voice request forwarded by the vehicle, the current display content information of the vehicle display components, and the current perception information of the vehicle. The determining module is used to determine a target execution action based on a preset model according to the first voice request, the current display content information and the current perception information. The sending module is used to send the target execution action to the vehicle to control the vehicle to perform the target execution action.

[0066] Specifically, the first voice request refers to the first voice command issued by the user through the in-vehicle voice assistant in a round of voice interaction, including a function control of the vehicle, information query or service request. For example, "Turn on the air conditioner", "What's the weather like today?" and "I need the nearest gas station".

[0067] The current display content information of the vehicle display component refers to the information content currently presented on the vehicle display screen or other display devices. For example, navigation-related content such as the current driving route, destination, estimated arrival time and traffic conditions, or entertainment system-related content such as music playlists, track information, radio station frequencies, etc., and communication-related content such as incoming call information, text message content, and contact lists. Display content information is an important part of the vehicle intelligent control system. The display content information can provide the driver with key driving information and entertainment services, and is also an important interface for interaction between the system and the user. It should be noted that in the implementation manner of the present application, the vehicle control method is described using the vehicle display screen as the vehicle display component, that is, the current display content information is a screenshot of the vehicle display screen.

[0068] Current perception information refers to the environmental status information of the current vehicle and the state perception information of the vehicle's onboard equipment functions collected by the vehicle through its sensors. Environmental status information includes data about the vehicle's surroundings, such as traffic conditions, road conditions, weather conditions, and geographic location. Traffic conditions refer to road traffic conditions collected by sensors such as cameras, radars, or laser radar (LiDAR), including the location and speed of other vehicles. Road conditions refer to information such as the slipperiness, potholes, and obstacles of the road. Weather conditions refer to weather information obtained through weather sensors or data exchange with external services, such as temperature, rainfall, wind speed, etc. Geographic location refers to the current location information of the vehicle obtained through positioning systems such as GPS. Onboard equipment function status perception information includes status data of the vehicle's internal systems and equipment, such as the status of the onboard entertainment system, the settings of the navigation system, the air conditioning configuration, the engine status, the battery power, the fuel level, and the tire pressure.

[0069] The target execution action refers to the specific operation determined and executed by the system based on the first voice request, the currently displayed content information and the currently perceived information, including click operation, sliding operation and typing operation. Click operation refers to the operation of clicking an icon or button, which is used to activate a specific function or service. For example, when the user says "open the music", the system may need to execute the operation of clicking the music application icon in the in-vehicle entertainment system. Sliding operation refers to the need to slide the screen or turn the page to view more information when the current page information list is collapsed or too long. For example, when the route information displayed by the navigation system is too long, the system may need to perform the operation of sliding the screen to display the complete route. Typing operation refers to the operation of clicking the search box and typing content, which is used to find specific information or services in the system. For example, when the user says "search for nearby restaurants", the system may need to execute the operation of clicking the search box and typing "nearby restaurants". Typing operation can also indicate the input and sending of text information in the in-vehicle system, such as sending text messages or making comments on social media. For example, when the user says "send a message to Zhang San", the system may need to execute the operation of opening the message application, selecting the contact Zhang San and opening the chat window.

[0070] The preset model refers to a pre-trained vision-language model (VLM) that can understand and process the relationship between images and text, allowing the machine to understand both visual content and natural language text.

[0071] See also Figure 2 The server receives a first voice request (i.e., a voice request) forwarded by the vehicle, current display content information of display components of the vehicle, and current perception information of the vehicle.

[0072] Next, the server uses this information as input and processes it based on a preset model, i.e., analyzing the content of the first voice request, understanding the currently displayed content information and the current perception information of the vehicle, and then determining the target execution action that needs to be performed.

[0073] Finally, the server sends the determined target execution action to the vehicle, and the system on the vehicle performs the corresponding operation after receiving the instruction.

[0074] In summary, in the vehicle control method and server provided in the embodiments of the present application, the server receives a first voice request forwarded by the vehicle, the current display content information of the vehicle display components, and the current perception information of the vehicle. Then, based on the preset model, the server determines the target execution action according to the first voice request, the current display content information, and the current perception information. Finally, the server sends the target execution action to the vehicle to control the vehicle to execute the target execution action. In this way, by analyzing the content of the first voice request, understanding the currently displayed content and the perception information of the vehicle, and determining the target execution action that needs to be executed, the convenience and safety of driving can be improved, while reducing the driver's distraction and improving the overall user experience.

[0075] See also Figure 3 In some embodiments, step 02 (determining a target execution action according to the first voice request, the currently displayed content information, and the currently perceived information) includes:

[0076] 021: rewriting the current displayed content information, the first voice request and the current perception information based on the preset task rewriting template to obtain first input information;

[0077] 022: Determine a target execution action according to the first input information.

[0078] In some embodiments, the determination module is further configured to rewrite the current display content information, the first voice request and the current perception information based on a preset task rewriting template to obtain first input information, and determine the target execution action according to the first input information.

[0079] In some embodiments, the processor is further configured to rewrite the current display content information, the first voice request and the current perception information based on a preset task rewriting template to obtain first input information, and determine a target execution action according to the first input information.

[0080] Specifically, the preset task rewriting template refers to a pre-designed template or rule for converting or mapping the user's voice request, current display content information, and current perception information into a task description that the system can understand and execute. In some embodiments, the preset task rewriting template is as follows:

[0081] Assume that you are an intelligent assistant. I will give you the following information and ask you to rewrite the content based on this information and output the rewriting task:

[0082] User Task: <Input Request>

[0083] Current perception information: <The current state of the vehicle’s environment and the state of the vehicle’s equipment>

[0084] Current display content information: <Screenshot of the vehicle display>

[0085] Please organize the above information into a task in concise language, within 20 words:

[0086] It should be noted that if the server generates a follow-up question, the server will also rewrite the second voice request issued by the user based on the follow-up question, that is, based on the preset task rewriting template, the current display content information, the first voice request, the current perception information and the second voice request are rewritten. Of course, the current display content information and the current perception information need to be reacquired when receiving the second voice request.

[0087] Rewriting refers to the process of reorganizing and expressing tasks based on current perceived information, current display content information, and user input requests.

[0088] Based on the preset task rewriting template, the server rewrites the current display content information, the first voice request and the current perception information to obtain the first input information, including the semantic understanding of the voice request, the analysis and interpretation of the display content, and the integration and interpretation of the perception information. Then, the server determines the target execution action according to the first input information.

[0089] In this way, through semantic understanding of the first voice request, analysis and interpretation of the displayed content, and integration and interpretation of the perceived information, the server can convert the original information into a form that the system can understand and process, namely, the first input information, in order to determine the target execution action that needs to be performed.

[0090] See also Figure 4 In some embodiments, step 022 (determining a target execution action according to the first input information) includes:

[0091] 0221: Analyze and process the first input information;

[0092] 0222: Determine the target execution task and the task type of the target execution task based on the results of the analysis and processing;

[0093] 0223: Determine the target execution action based on the task type, target execution task and current display content information.

[0094] In some embodiments, the determination module is further configured to analyze and process the first input information, determine the target execution task and the task type of the target execution task according to the result of the analysis and processing, and determine the target execution action according to the task type, the target execution task and the currently displayed content information.

[0095] In some embodiments, the processor is further configured to analyze and process the first input information, determine the target execution task and the task type of the target execution task according to the result of the analysis and processing, and determine the target execution action according to the task type, the target execution task and the currently displayed content information.

[0096] Specifically, the task type refers to the way to classify tasks according to the completeness of the semantic information requested by the user and the uniqueness of the execution path, including the first task type, the second task type and the third task type. Among them, the first task type is a task type with complete semantic information and a unique execution path. The second task type is a task type with missing semantic information. The third task type is a task type with complete semantic information and a non-unique execution path.

[0097] The analysis process refers to understanding the first input information to determine the needs and intentions of the user, that is, to determine the target execution task and the task type of the target execution task.

[0098] The server analyzes and processes the first input information. Then, the server determines the target execution task and the task type of the target execution task according to the result of the analysis and processing. For example, if the analysis result shows that the user requests navigation to a certain location, the target execution task may be "providing navigation services" and the task type may be "first task type".

[0099] Finally, the server determines the target execution action based on the task type, target execution task and current display content information.

[0100] For example, the driver says: "I need to go to gas station A", and the vehicle currently displays the navigation interface. At this time, the server receives the voice request and the currently displayed content information, and after rewriting, obtains the first input information: "The user requests navigation to gas station A and selects the nearest route, and the navigation interface is currently displayed." The server analyzes and processes the first input information, and determines that the target execution task is "providing navigation services" and the task type is "first task type." Finally, the server determines the target execution action as: "Type A gas station in the search box of the navigation interface and select the nearest route."

[0101] In this way, by analyzing and processing the first input information and processing the results of the analysis and processing, the target execution task and the task type of the target execution task are determined to determine the target execution action, thereby understanding and responding to user needs and improving user experience.

[0102] See also Figure 5 In some embodiments, step 0223 (determining the target execution action according to the task type, the target execution task and the currently displayed content information) includes:

[0103] 02231: When the task type is the first task type, determine the target execution action according to the target execution task and the currently displayed content information.

[0104] In some implementations, the determination module is further configured to determine the target execution action according to the target execution task and the currently displayed content information when the task type is the first task type.

[0105] In some implementations, the processor is further configured to determine a target execution action according to the target execution task and currently displayed content information when the task type is a first task type.

[0106] Specifically, the target execution action refers to the specific operation determined and executed by the system based on the user's request or the current context.

[0107] The server first determines the task type. If the task type is "first task type", that is, the task requirement is very clear and there is only one clear execution method. Then the server will determine the target execution task based on the first input information. For example, if the user request is "navigate to the nearest gas station", then the target execution task is "provide a route to the nearest gas station".

[0108] Next, perform image UI analysis on the current display content information (i.e., a screenshot of the vehicle display), and determine the target execution action based on the analysis results and the target execution task. That is, determine how to integrate the target execution task into the current display content. For example, if the user request is "navigate to the nearest gas station" and the current display content information is a song playback interface. Then the target execution actions are "close the song playback interface", "open the navigation program", and "type the gas station in the navigation interface".

[0109] In this way, when the task type is the first task type, the server can respond to user needs quickly and accurately, improve user satisfaction and loyalty, and provide users with better services.

[0110] See also Figure 6In some embodiments, step 0223 (determining the target execution action according to the task type, the target execution task and the currently displayed content information) includes:

[0111] 02232: When the task type is the second task type, the target execution task and the current display content information are processed to confirm the missing slot information;

[0112] 02233: Determine the target execution action based on the missing slot information.

[0113] In some embodiments, the determination module is further configured to process the target execution task and the current display content information to confirm the missing slot information when the task type is the second task type, and determine the target execution action according to the missing slot information.

[0114] In some embodiments, the processor is further configured to process the target execution task and the current display content information to confirm the missing slot information when the task type is the second task type, and determine the target execution action according to the missing slot information.

[0115] Specifically, missing slot information refers to information that the system identifies as needing to be further acquired or confirmed when processing a user request in order to fully understand and respond to the user's intent.

[0116] The server first determines the task type. If the task type is "second task type", that is, a task type with missing semantic information, the server will process the target execution task and the current display content information to confirm the missing slot information. For example, if the user request is "navigate to the nearest" but does not specify a specific destination, the missing slot information may be "destination".

[0117] The server determines the target execution action based on the missing slot information. For example, if the missing slot information is the "destination", the server will determine the target execution action based on the missing slot information and other data information after obtaining the "destination".

[0118] In this way, when the task type is the second task type, by confirming the missing slot information and requesting the user to provide relevant information, the server's inquiry capability is improved, so that the user's needs can be better understood and the intelligence and interactivity of the system can be improved.

[0119] See also Figure 7 In some implementations, step 02232 (processing the target execution task and the current display content information to confirm the missing slot information) includes:

[0120] 022321: based on the preset task database, according to the first input information, confirming the target preset task and the target preset task slot corresponding to the first input information;

[0121] 022322: Determine the missing slot information according to the first input information and the target preset task slot.

[0122] In some embodiments, the determination module is used to confirm the target preset task and the target preset task slot corresponding to the first input information based on the preset task database according to the first input information, and determine the missing slot information according to the first input information and the target preset task slot.

[0123] In some embodiments, the processor is further configured to confirm the target preset task and the target preset task slot corresponding to the first input information based on the preset task database and the first input information, and determine the missing slot information based on the first input information and the target preset task slot.

[0124] Specifically, the preset task database refers to a database that stores a variety of preset tasks and corresponding slot information, which is used to help the system understand and respond to user requests. For example, if the user's target task is "navigation", then the preset task slots may include "destination" (such as location A) and "route preference" (such as shortest distance or least traffic lights, etc.). If the user's target task is "send a message", then the preset task slots may include "sender" and "send content". If the user's target task is "find nearby [facility type]", then the preset task slots may include "facility type".

[0125] The server first confirms the target preset task corresponding to the input information based on the preset task database and the received first input information, and identifies the slots in the target preset task, which represent all necessary information required to complete the task.

[0126] The server then analyzes and determines which slot information is missing based on the first input information and the confirmed target preset task slots. For example, if the target preset task is "navigate to [destination]", and the first input information does not provide specific [destination] information, then [destination] is a missing slot. In some embodiments, the in-vehicle intelligent assistant is prompted by a certain template, as follows:

[0127] Assuming you are an agent planning assistant, please determine the execution type of the next action of the task based on the first input information given and the preset task slot. There are two execution types: "Ask again" and "Direct execution". The supplementary information is as follows:

[0128] 1. "Ask again" means that the current task lacks semantic information, the sentence information does not meet the task slot information, and the next step cannot be executed.

[0129] 2. "Direct execution" means that the current task is relatively clear, and the information in the sentence information can be used to extract the task slot information and determine how to execute the next step.

[0130] Mission: <First Input Information>

[0131] Task slot: <Default task slot>

[0132] Execution Type:

[0133] In this way, by using the preset task database, the system can confirm the target preset task and the target preset task slot corresponding to the first input information, thereby improving the accuracy of task processing.

[0134] See also Figure 8 In some embodiments, step 02233 (determining a target execution action based on missing slot information) includes:

[0135] 022331: Generate the first inquiry information according to the missing slot information, and send the inquiry information to the vehicle;

[0136] 022332: receiving a second voice request for the first follow-up information, current display content information, and current perception information forwarded by the vehicle;

[0137] 022333: Based on the preset model, generate a target execution action according to the first voice request, the second voice request, the current display content information and the current perception information.

[0138] In some embodiments, the determination module is used to generate a first inquiry message according to the missing slot information, and send the inquiry message to the vehicle. And receive a second voice request, currently displayed content information, and currently perceived information forwarded by the vehicle for the first inquiry message. And based on a preset model, generate a target execution action according to the first voice request, the second voice request, the currently displayed content information, and the currently perceived information.

[0139] In some embodiments, the processor is further configured to generate a first inquiry message according to the missing slot information, and send the inquiry message to the vehicle. And receive a second voice request, currently displayed content information, and currently perceived information forwarded by the vehicle for the first inquiry message. And based on a preset model, generate a target execution action according to the first voice request, the second voice request, the currently displayed content information, and the currently perceived information.

[0140] Specifically, the first follow-up information refers to the question that the system asks the user when processing the user's request in order to obtain the missing slot information or further clarify the user's intention. Suppose the user says: "Help me find a nearby restaurant." The system will recognize that this is a task of "finding nearby facilities" and find that it needs to further obtain the slot information of "facility type". Therefore, the system will generate the first follow-up information: "What type of restaurant do you want?" After the user answers, the system can obtain the complete slot information and then perform the corresponding operation.

[0141] The second voice request refers to the user's response to the system's follow-up questions or further instructions after the first voice request. For example, the user requests: Help me find a nearby restaurant. The system asks: What type of restaurant do you want? The user answers: I want to eat Chinese food. In this example, the user's first request is "Help me find a nearby restaurant", and the system obtains more information by asking "What type of restaurant do you want?" The user's answer "I want to eat Chinese food" is the second voice request, which provides the key information needed for the system to complete the task. It should be noted that the current display content information and current perception information received at this time are both the display content information and perception information re-acquired when the second voice request is received.

[0142] The server generates a first follow-up message based on the determined missing slot information. For example, if the missing slot information is "destination", the follow-up message may be "Where do you want to navigate to?" Then, the server sends the generated follow-up message to the vehicle. Then, the user will make a supplementary answer based on the follow-up message, that is, give a second voice request. Subsequently, the server receives the second voice request for the first follow-up message, the currently displayed content information, and the current perception information forwarded by the vehicle. Finally, based on the preset model, the server combines the first voice request, the second voice request, the currently displayed content information, and the current perception information to generate a target execution action. For example, if the user provides destination information in the second voice request, the server will generate instructions to navigate to the destination based on this information.

[0143] In this way, by asking questions and collecting missing information, the system can understand and perform the user's tasks more accurately, thereby improving the accuracy of task completion.

[0144] See also Fig. 9 In some embodiments, step 0223 (determining the target execution action according to the task type, the target execution task and the currently displayed content information) includes:

[0145] 02234: when the task type is the third task type, processing the target execution task and the current display content information, confirming the second inquiry information, and sending the second inquiry information to the vehicle;

[0146] 02235: receiving a third voice request for the second follow-up information, current display content information, and current perception information forwarded by the vehicle;

[0147] 02236: Based on the preset model, generate a target execution action according to the first voice request, the third voice request, the current display content information and the current perception information.

[0148] In some embodiments, the determination module is used to process the target execution task and the currently displayed content information when the task type is the third task type, confirm the second inquiry information, and send the second inquiry information to the vehicle. And receive the third voice request for the second inquiry information forwarded by the vehicle, the currently displayed content information, and the currently perceived information. And based on the preset model, according to the first voice request, the third voice request, the currently displayed content information, and the currently perceived information, generate a target execution action.

[0149] In some embodiments, the processor is further configured to process the target execution task and the currently displayed content information, confirm the second inquiry information, and send the second inquiry information to the vehicle when the task type is the third task type. And receive the third voice request for the second inquiry information, the currently displayed content information, and the currently perceived information forwarded by the vehicle. And based on the preset model, generate the target execution action according to the first voice request, the third voice request, the currently displayed content information, and the currently perceived information.

[0150] Specifically, the second follow-up information refers to the fact that when processing a user request, due to the existence of multiple possible execution paths, the system needs to ask the user further questions to determine which specific execution path should be taken. For example, the user requests: Help me find a nearby restaurant. The system asks: What type of restaurant do you want? The user answers: I want to eat Chinese food. The system will ask again: Do you want traditional Chinese food or modern Chinese food? In this example, the user's first request is "Help me find a nearby restaurant", and the system obtains more information through the first follow-up information "What type of restaurant do you want?" After the user answers "I want to eat Chinese food", the system finds that there are many different options under the Chinese food category, so the second follow-up information "Do you want traditional Chinese food or modern Chinese food?" is used to further clarify the user's specific needs.

[0151] When the task type is the third task type, the server processes the target execution task and the currently displayed content information, confirms the second query information, and sends the second query information to the vehicle, wherein the third task type is a task type with complete semantic information and a non-unique execution path. Next, the server receives the third voice request for the second query information, the currently displayed content information, and the currently perceived information forwarded by the vehicle. Finally, based on the preset model, the server generates the target execution action according to the first voice request, the third voice request, the currently displayed content information, and the currently perceived information.

[0152] In this way, by processing the target execution task and the current display content information, the second inquiry information is confirmed, so as to obtain the user's choice or preference through interaction, thereby improving the flexibility of task completion and enhancing the user experience.

[0153] See also Fig.10 In some embodiments, the method further comprises:

[0154] 04: When the vehicle completes the target execution action, a template is generated based on the preset feedback, and voice broadcast information is generated based on the current perception information.

[0155] In some embodiments, the generation module is used to generate a template based on preset feedback when the vehicle completes the target execution action, and generate voice broadcast information based on current perception information.

[0156] In some embodiments, the processor is also used to generate a template based on preset feedback and generate voice broadcast information based on current perception information when the vehicle completes the target execution action.

[0157] Specifically, the preset feedback generation template refers to a series of pre-designed, personalized voice feedback templates generated by the system after the target execution action is completed, based on the vehicle's current perception information and completed tasks. For example, user task: Help me navigate to Xiangshan and book a hotel nearby. Current perception information: The temperature outside the car is 10 degrees / the air conditioning and heating are turned on. Output voice broadcast information tts: You have successfully navigated to Xiangshan and booked a nearby hotel. The temperature outside the car is 10 degrees, the air conditioning and heating are turned on, and the temperature outside the car is relatively cold. If you are doing outdoor activities, remember to keep warm. In some embodiments, the preset feedback generation template is as follows:

[0158] Assuming you are a car computer intelligent assistant, please output the appropriate task report tts based on the following tasks and car computer status:

[0159] example:

[0160] User task: Help me navigate to Xiangshan and book a hotel nearby

[0161] Current sensor information: Outside temperature 10 degrees / air conditioning and heating on

[0162] Output tts: You have successfully navigated to Xiangshan and booked a nearby hotel. The temperature outside the car is 10 degrees, the air conditioning and heating are turned on, and the temperature outside the car is relatively cold. If you are doing outdoor activities, remember to keep warm.

[0163] User task: <input voice request>

[0164] Current perception information: <Enter current perception information>

[0165] Based on the above information, please output the report after the task is completed, with a lively language style and within 60 words:

[0166] Output tts:

[0167] The server first confirms that the vehicle has completed the previously issued target action. Then, based on the preset feedback generation template, the server generates voice broadcast information based on the current perception information to describe different execution completion states and provide action suggestions. For example, if the target action is to navigate to a certain location, the voice broadcast information may be "You have arrived at your destination. Please pay attention to the surrounding environment when parking."

[0168] In this way, by providing real-time execution completion status information and action suggestions, the system can help drivers better cope with various situations during driving, thereby improving the user experience.

[0169] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle control method described above are implemented.

[0170] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file or some intermediate form. Computer readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution medium.

[0171] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0172] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable requests for implementing specific logical functions or steps of a process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0173] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A vehicle control method, characterized in that: The method comprises: Receiving a first voice request forwarded by a vehicle, current display content information of a display component of the vehicle, and current perception information of the vehicle; Based on a preset model, determining a target execution action according to the first voice request, the currently displayed content information, and the currently perceived information; The target execution action is sent to the vehicle to control the vehicle to execute the target execution action.

2. The vehicle control method according to claim 1, characterized in that: The step of determining a target execution action according to the first voice request, the currently displayed content information, and the currently perceived information includes: Based on a preset task rewriting template, rewriting the currently displayed content information, the first voice request, and the currently perceived information to obtain first input information; The target execution action is determined according to the first input information.

3. The vehicle control method according to claim 2, characterized in that: The step of determining the target execution action according to the first input information includes: Analyzing and processing the first input information; Determining a target execution task and a task type of the target execution task according to the result of the analysis and processing; The target execution action is determined according to the task type, the target execution task and the currently displayed content information.

4. The vehicle control method according to claim 3, characterized in that: The step of determining the target execution action according to the task type, the target execution task and the currently displayed content information includes: In the case where the task type is a first task type, the target execution action is determined according to the target execution task and the currently displayed content information, wherein the first task type is a task type with complete semantic information and a unique execution path.

5. The vehicle control method according to claim 3, characterized in that: The step of determining the target execution action according to the task type, the target execution task and the currently displayed content information includes: In the case where the task type is a second task type, processing the target execution task and the current display content information to confirm missing slot information, the second task type being a task type with missing semantic information; The target execution action is determined according to the missing slot information.

6. The vehicle control method according to claim 5, characterized in that: The processing of the target execution task and the current display content information to confirm the missing slot information includes: Based on the preset task database, according to the first input information, confirming the target preset task and the target preset task slot corresponding to the first input information; The missing slot information is determined according to the first input information and the target preset task slot.

7. The vehicle control method according to claim 5, characterized in that: The step of determining the target execution action according to the missing slot information includes: generating first inquiry information according to the missing slot information, and sending the inquiry information to the vehicle; receiving a second voice request for the first follow-up information, the current display content information, and the current perception information forwarded by the vehicle; Based on the preset model, the target execution action is generated according to the first voice request, the second voice request, the currently displayed content information and the current perception information.

8. The vehicle control method according to claim 3, characterized in that: The step of determining the target execution action according to the task type, the target execution task and the currently displayed content information includes: In the case where the task type is the third task type, processing the target execution task and the current display content information, confirming the second inquiry information, and sending the second inquiry information to the vehicle, wherein the third task type is a task type with complete semantic information and a non-unique execution path; receiving a third voice request for the second follow-up information, the current display content information, and the current perception information forwarded by the vehicle; Based on the preset model, the target execution action is generated according to the first voice request, the third voice request, the currently displayed content information and the current perception information.

9. The vehicle control method according to claim 1, characterized in that: The method further comprises: When the vehicle completes the target execution action, a preset feedback generation template is generated, and voice broadcast information is generated according to the current perception information. The voice broadcast information includes execution completion status information and / or action suggestions of the target execution action.

10. A server, characterized in that: The server includes a processor and a memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the vehicle control method according to any one of claims 1 to 9 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images