Voice interaction method, server and readable storage medium

Abstract

The invention discloses a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program. The method comprises the following steps: receiving a current round voice request forwarded by a vehicle and current environment image coding information of a cabin; and determining target input information according to the current environment image coding information of the cockpit, the current round of voice request, the historical round of voice request and the natural language understanding result of the historical round of voice request. And according to the target input information, determining an intention label of the current round of voice request. And completing voice interaction according to the intention label. Therefore, the target input information is determined by combining the current environment image coding information of the cabin, the current round of voice request, the slot position identification result, the historical round of voice request and the understanding result thereof, intention identification is performed on the target input information, the weak intention voice request of the user is accurately identified, misunderstanding of the user request is reduced, and the user experience is improved. And the probability of error response is reduced, so that the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a voice interaction method, a server and a non-volatile computer-readable storage medium containing a computer program. Background Art

[0002] In the related technologies, in-vehicle intelligent systems usually use single-mode voice control for human-vehicle interaction. However, single-mode voice control has limited understanding and response capabilities for complex scenarios, and cannot distinguish whether the user is chatting with others or giving instructions to the system, which may cause erroneous replies or false rejections, resulting in poor user experience. Summary of the invention

[0003] The embodiments of the present invention provide a voice interaction method, a server and a non-volatile computer-readable storage medium containing a computer program.

[0004] The present application provides a voice interaction method, including:

[0005] Receive the current round of voice request and the current environment image encoding information of the cockpit forwarded by the vehicle;

[0006] Determine target input information according to the cockpit current environment image encoding information, the current round voice request, the historical round voice request and the natural language understanding result of the historical round voice request;

[0007] Determining the intent label of the current round of voice request according to the target input information;

[0008] Complete the voice interaction according to the intent label.

[0009] In this way, an embodiment of the present invention provides a voice interaction method. First, the current round voice request and the current environment image coding information of the cabin forwarded by the vehicle are received. Then, the target input information is determined based on the current environment image coding information of the cabin, the current round voice request, the historical round voice request and the natural language understanding results of the historical round voice request. Next, the intent label of the current round voice request is determined based on the target input information. Finally, according to the intent label, the voice interaction is completed. In this way, the target input information is determined by combining the current environment image coding information of the cabin, the current round voice request, the slot recognition results, the voice requests of the historical rounds and their understanding results, and the intent recognition is performed on the target input information, the user's weak intent voice request is accurately identified, the misunderstanding of the user's request is reduced, and the probability of incorrect response is reduced, thereby enhancing the user experience and making the interaction more natural and smooth.

[0010] In some embodiments, determining the target input information according to the cockpit current environment image encoding information, the current round voice request, the historical round voice request and the natural language understanding result of the historical round voice request includes:

[0011] The preset mark, the current round voice request, the historical round voice request and the application program interface corresponding to the historical round voice request are concatenated to obtain first temporary input information;

[0012] Performing position coding processing on the first temporary input information to determine second temporary input information;

[0013] Determining third temporary input information according to the round information of the current round voice request and the round information of the historical round voice request;

[0014] The first temporary input information, the second temporary input information, the third temporary input information and the cockpit current environment image coding information are spliced ​​to determine the target input information.

[0015] In this way, the server splices the preset mark, the current round voice request, the slot recognition result, the historical round voice request and the natural language understanding result to determine the first temporary input information. Next, the server performs position encoding processing on the first temporary input information to determine the second temporary input information. Then, the server determines the third temporary input information based on the round information of the current round voice request and the round information of the historical round voice request. Finally, the server splices the first temporary input information, the second temporary input information, the third temporary input information and the cockpit current environment image encoding information to determine the target input information. In this way, the server can more accurately judge the nature of the voice request by considering the current environment in the car, thereby providing a more appropriate response, improving the model's processing ability for sequence data, ensuring that the server can accurately capture the user's intentions and contextual information, and ensuring the coherence and accuracy of the conversation. This helps to improve the user experience of the in-vehicle voice interaction system and avoid misunderstanding the user's intentions.

[0016] In some implementations, completing the voice interaction according to the intention tag includes:

[0017] When the intent tag is the first intent tag, the current round of voice request is rejected.

[0018] In this way, when the intent tag is the first intent tag, the server rejects the current round of voice request. In this way, through the rejection process, the server can be prevented from providing an erroneous or irrelevant response, thereby avoiding misleading the user and improving the overall data quality.

[0019] In some implementations, completing the voice interaction according to the intention tag includes:

[0020] When the intent tag is the second intent tag, performing slot identification on the current round of voice request;

[0021] Performing application program interface prediction on the current round of voice request;

[0022] According to the result of the slot identification and the predicted application program interface, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

[0023] In this way, when the intent tag is the second intent tag, the server performs slot identification on the current round of voice requests. Next, the service performs application program interface prediction on the current round of voice requests. Finally, the server executes application program interface parameter filling according to the results of slot identification and the predicted application program interface, and outputs the execution result and sends it to the vehicle to complete the voice interaction. In this way, according to the results of slot identification, the predicted application program interface is selected to execute application program interface parameter filling. The server can understand and respond to the user's voice commands, and directly outputs the execution result and sends it to the vehicle to complete the voice interaction, thereby reducing the delay of the vehicle system, improving the response speed to user commands, and providing a convenient and natural interactive experience.

[0024] In some embodiments, the method of performing application program interface parameter filling according to the slot identification result and the predicted application program interface, and outputting the execution result to the vehicle to complete the voice interaction includes:

[0025] Determine a target parameter for slot filling according to the current round of voice request, the result of the slot identification, the predicted application program interface, and the predicted application program interface type;

[0026] According to the result of the slot identification and the target parameter, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

[0027] In this way, the server determines the target parameters for slot filling based on the current round of voice requests, the results of slot identification, the predicted API, and the predicted API type. Then, the server executes API parameter filling based on the results of slot identification and the target parameters, and outputs the execution results and sends them to the vehicle to complete the voice interaction. In this way, the predicted API can be selected to execute API parameter filling based on the results of slot identification and the target parameters, and the execution results can be directly output and sent to the vehicle to complete the voice interaction, which can reduce the latency of the vehicle system and improve the response speed to user commands.

[0028] In some implementations, completing the voice interaction according to the intention tag includes:

[0029] When the intent tag is the third intent tag and the current round of voice request is not a first round of voice request, performing slot identification on the current round of voice request;

[0030] According to the result of the slot identification and the application program interface of the historical round voice request, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

[0031] In this way, when the intent tag is the third intent tag and the current round of voice request is not the first round of voice request, the server performs slot identification on the current round of voice request. Then, the server executes the application interface parameter filling according to the result of slot identification and the application interface of the historical round of voice request, and outputs the execution result and sends it to the vehicle to complete the voice interaction. In this way, the use of an end-to-end architecture can reduce the latency of the vehicle system and improve the response speed to user commands. At the same time, the slot identification results of the user's voice request and the additional features of the application interface of the historical round of voice request are integrated, which can accurately identify the user's intentions and required parameters, and can call the correct application interface to perform the corresponding operations, thereby meeting the vehicle control requirements.

[0032] In some implementations, completing the voice interaction according to the intention tag includes:

[0033] When the intent tag is the third intent tag and the current round of voice request is the first round of voice request, performing slot identification on the current round of voice request;

[0034] Based on a preset database, performing application program interface prediction on the current round of voice request;

[0035] According to the result of the slot identification and the predicted application program interface, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

[0036] In this way, when the intent tag is the third intent tag and the current round of voice request is the first round of voice request, the server performs slot identification on the current round of voice request. Then, based on the preset database, the server predicts the application interface for the current round of voice request. Finally, the server performs application interface parameter filling based on the results of slot identification and the predicted application interface, and outputs the execution result to the vehicle to complete the voice interaction. In this way, based on the preset database, the application interface is predicted, and the server can obtain the commands or operations that need to be executed when the intent tag is the second intent tag, thereby performing human-computer interaction and providing a convenient and natural user experience.

[0037] In some implementations, completing the voice interaction according to the intention tag includes:

[0038] When the intent tag is the fourth intent tag, the current round of voice request is rejected.

[0039] In this way, when the intent tag is the fourth intent tag, the server rejects the current round of voice request. In this way, through the rejection process, the server can be prevented from providing an erroneous or irrelevant response, thereby avoiding misleading the user and improving the overall data quality.

[0040] The present invention further provides a server, wherein the server comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the voice interaction method described in any one of the above implementation modes is implemented.

[0041] The present invention also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the voice interaction method described in any one of the above implementations is implemented.

[0042] Additional aspects and advantages of the embodiments of the present invention will be given in part in the following description, and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1 This is one of the flow charts of the voice interaction method according to the implementation mode of the present application;

[0045] Figure 2 This is the second flow chart of the voice interaction method according to the implementation mode of the present application;

[0046] Figure 3is a flowchart of an example of a voice interaction method according to an embodiment of the present application;

[0047] Figure 4 This is the third flow chart of the voice interaction method according to the implementation mode of the present application;

[0048] Figure 5 This is the fourth flow chart of the voice interaction method according to the implementation mode of the present application;

[0049] Figure 6 This is the fifth flow chart of the voice interaction method according to the implementation mode of the present application;

[0050] Figure 7 This is the sixth flow chart of the voice interaction method according to the implementation mode of the present application;

[0051] Figure 8 This is the seventh flow chart of the voice interaction method according to the implementation mode of the present application;

[0052] Fig. 9 This is the eighth flow chart of the voice interaction method of the implementation mode of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present invention, and cannot be understood as limiting the embodiments of the present invention.

[0054] Among the technologies related to in-vehicle intelligent systems, traditional single-mode voice control is a common way of human-vehicle interaction. This method mainly relies on voice signals and converts the user's voice instructions into commands that the machine can understand through voice recognition technology. However, single-mode voice control has certain limitations in understanding and responding to user intentions, and cannot distinguish whether the user's intention is a weak intention, especially in a complex and dynamic in-vehicle environment.

[0055] For example, when the vehicle is driving, the interior environment may be very noisy, including road noise, music in the car, passengers talking, etc. These factors may interfere with the voice recognition system, causing it to be unable to accurately understand the user's instructions.

[0056] Secondly, in a unimodal voice control system, it is difficult to distinguish whether the user is chatting with other passengers in the car or giving instructions to the in-vehicle system. This may cause the system to incorrectly respond to commands not intended for it, or mistakenly ignore the user's real commands.

[0057] Due to the above challenges, single-mode voice control systems may experience false responses or false rejections. These errors can seriously affect the user experience and may even pose a safety hazard. False responses refer to incorrect execution of commands, while false rejections refer to incorrectly ignoring commands.

[0058] Weak intent refers to an intent that is not clear, complete, or direct enough from the user. This intent may be difficult for the system to accurately understand and respond to due to the user's vague words, missing information, or unclear context. For example, the user may say "I'm a little cold" instead of explicitly asking to adjust the air conditioning temperature; or the user may say "play a song" but do not provide a specific song title.

[0059] Based on the above questions, please refer to Figure 1 , the present application embodiment provides a voice interaction method, the method comprising:

[0060] 01: Receive the current round of voice request and the current environment image encoding information of the cabin forwarded by the vehicle;

[0061] 02: Determine the target input information based on the current environment image encoding information of the cockpit, the current round of voice requests, the historical round of voice requests, and the natural language understanding results of the historical round of voice requests;

[0062] 03: Determine the intent label of the current round of voice request based on the target input information;

[0063] 04: Complete voice interaction based on intent tags.

[0064] The embodiment of the present application also provides a server. The server includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive the current round voice request and the current environment image coding information of the cabin forwarded by the vehicle. And according to the natural language understanding results of the current environment image coding information of the cabin, the current round voice request, the historical round voice request and the historical round voice request, the target input information is determined. The processor is also used to determine the intent label of the current round voice request according to the target input information. And according to the intent label, the voice interaction is completed.

[0065] Specifically, the current round voice request refers to the voice instruction or request issued by the user in the current interaction round.

[0066] The current environment image coding information of the cockpit refers to the image data of the vehicle's internal cockpit. This data is converted into a coding format through image recognition and processing technology to facilitate analysis and understanding by the vehicle system. The photos taken by the camera in the car are encoded on the terminal and the coded information is uploaded to the cloud, such as: single-person car, multi-person car, and phone calls. In addition, to protect privacy, after the terminal obtains the scene in the car in real time, only the implicit coding is uploaded. The coding cannot be reversely decoded to obtain the original image information to ensure information confidentiality.

[0067] Slot recognition refers to extracting specific information fragments from the user's input, which are usually called "slots". Slots are usually key information required to complete a task or request, such as time, place, object, etc. Taking the user's voice request "What will the temperature be tomorrow" as an example, the slot information that can be obtained through slot recognition includes ["tomorrow" - date (Date)], that is, the slot information includes slot value and slot type, where "tomorrow" is the slot value and date (Date) is the slot type. Taking the user's voice request "Navigate to Zhongguancun" as an example, the slot information that can be obtained through slot recognition is ["Zhongguancun" - place name (Place)], where "Zhongguancun" is the slot value and place name (Place) is the slot type.

[0068] The slot recognition result refers to the named entity obtained by performing slot recognition on the voice request, such as the slot information ["tomorrow" - date (Date)] and slot information ["Zhongguancun" - place name (Place)] mentioned above.

[0069] The historical round voice request refers to the collection of voice instructions or requests issued by the user in the previous interaction round in the dialogue system. It should be noted that the implementation method of the present application takes the previous round voice request of the current round voice request as the historical round voice request to illustrate the voice interaction method of the implementation method of the present application. In some implementation methods, the historical round voice request can indicate more rounds of voice requests, which is not limited here.

[0070] The natural language understanding results of historical rounds of voice requests include the corresponding API call or operation of the previous round of voice requests, the user's original voice request in the previous round, and the named entities obtained from the user's previous round of voice requests.

[0071] The intent label refers to the classification obtained by the label generation model after performing intent recognition processing on the current round of voice requests, including the first intent label, the second intent label, the third intent label, and the fourth intent label.

[0072] Generative models refer to a class of models in the field of machine learning and artificial intelligence that can generate new data instances. Generative models are able to learn the underlying distribution of data and generate new data based on it. This means that they can not only generate new instances that are similar to the training data, but also generate completely new instances that do not exist in the training data. Generative models can be applied to a variety of tasks, such as image generation, text generation, audio synthesis, etc.

[0073] The label generation model refers to a generative model obtained by fine-tuning a pre-trained model such as BERT in the implementation of the present application. The BERT training model is a pre-trained language representation model that can understand text from both left to right and right to left, thereby capturing deep bidirectional contextual information in the text.

[0074] When the user speaks a certain sentence, the vehicle's voice recognition system receives the user's voice input and converts it into text information, which is then forwarded to the server. At the same time, the camera in the cockpit captures the image of the interior environment and generates image encoding information. For example, when the user says in the vehicle: "Please adjust the air conditioning temperature to 25 degrees", the voice recognition system will convert the voice input into text: "Please adjust the air conditioning temperature to 25 degrees". Or, the user says in the vehicle: "The air conditioning temperature is a bit low, do you want to adjust it to 25 degrees?" In the first voice request, the user's instructions are clear and the server needs to respond in time, while in the second voice request, the instructions are not clear enough and need to be recognized in combination with the current interior environment image.

[0075] Subsequently, the server determines the target input information based on the current environment image coding information of the cockpit, the current round of voice requests, the historical round of voice requests, and the natural language understanding results of the historical round of voice requests.

[0076] Next, the server determines the intent label of the current round of voice request based on the target input information. That is, the content of the voice request and the environmental image analysis results are combined to determine whether it is human-to-human interaction or human-to-computer interaction. For example, if the image shows a passenger talking to a part of the vehicle, the server may determine that this is a human-to-computer interaction scene. In the second voice request mentioned above, if the user speaks directly to the vehicle's onboard system, it is considered to be human-to-computer interaction and is in an intercom state. If there are other passengers in the car, it is considered that the user is having a conversation with other people and is in a chatting state.

[0077] Finally, the server completes the voice interaction based on the intent tag.

[0078] It should be noted that in some embodiments, the server is deployed in the cloud and communicates with the vehicle through a communication protocol. In some embodiments, if the vehicle has sufficient computing power, the server is deployed in the vehicle.

[0079] In this way, the target input information is determined by combining the current environment image coding information of the cockpit, the current round of voice requests, the slot recognition results, the voice requests of historical rounds and their understanding results, and the intent recognition is performed on the target input information to accurately identify the user's weak intention voice requests, reduce misunderstandings of user requests, and reduce the probability of incorrect responses, thereby improving the user experience and making the interaction more natural and smooth.

[0080] See also Figure 2 In some embodiments, step 02 (determining target input information according to the current round voice request, the slot recognition result, the historical round voice request, the natural language understanding result of the historical round voice request, and the first wake-up state information of the historical round voice request) includes:

[0081] 021: Concatenate the preset mark, the current round voice request, the slot recognition result, the historical round voice request and the natural language understanding result to determine the first temporary input information;

[0082] 022: Perform position coding processing on the first temporary input information to determine the second temporary input information;

[0083] 023: Determine third temporary input information according to the round information of the current round voice request and the round information of the historical round voice request;

[0084] 024: The first temporary input information, the second temporary input information, the third temporary input information and the cockpit current environment image coding information are spliced ​​to determine the target input information.

[0085] The processor is also used to perform splicing processing on the preset mark, the current round voice request, the slot recognition result, the historical round voice request and the natural language understanding result to determine the first temporary input information. And perform position coding processing on the first temporary input information to determine the second temporary input information. The processor is also used to determine the third temporary input information according to the round information of the current round voice request and the round information of the historical round voice request. And perform splicing processing on the first temporary input information, the second temporary input information, the third temporary input information and the cockpit current environment image coding information to determine the target input information.

[0086] Specifically, the preset token refers to a special token or symbol used for a specific purpose in the model input, including a CLS token (Class Token) and a special token.

[0087] The CLS tag is a special tag that is added to the beginning of each input sequence. It can aggregate the global information of the entire sequence and is often used in classification tasks.

[0088] Special Token refers to the start and end position of the slot. For example, if the user's voice output is "open the window", then the window will be <device> and< / device> Marking. <device> Represents the starting point of the device.< / device> Represents the endpoint of the device.

[0089] Position encoding refers to the process of position encoding the first temporary input information in the Transformer model according to the position of the slot recognition result in the current round of voice request to confirm the second temporary input information. For example, if the voice request is "open the window", then the position of the slot recognition result "window" in the voice request "open the window" is 2 to 3. Correspondingly, in the second temporary input information, the position of the slot recognition result "window" also needs to be 2 to 3.

[0090] The server concatenates the preset mark, the current round of voice request, the slot recognition result, the previous round of voice request, and the natural language understanding result of the previous round of voice request to form a comprehensive input information, namely the first temporary input information. This helps the server better understand the context of the entire conversation.

[0091] Next, in order to understand the positional relationship of each element in the input sequence, the server performs position coding processing on the first temporary input information determined after splicing to determine the second temporary input information, which helps to understand the relative or absolute position of the elements in the sequence.

[0092] Then, the server determines the third temporary input information according to the turn information of the current round of voice request and the previous round of voice request, so as to take into account the timing characteristics of the dialogue.

[0093] Finally, the server combines the first temporary input information, the second temporary input information, the third temporary input information and the current environment image encoding information of the cockpit to generate target input information, which will be used as input for intent recognition.

[0094] See also Figure 3In one example, the preset tag is "CLS", the current round voice request is "How is Haidilao", the slot recognition result is "[{Singer: Haidilao, pos: [1,3]}]", the previous round voice request is "What's delicious", and the natural language understanding result of the previous round voice request is "[last-api: ChatGeneral], [last-arguments: {delicious: delicious, pos: [10,12]}]". The preset tag, the current round voice request, the slot recognition result, the previous round voice request and the natural language understanding result of the previous round voice request are concatenated to obtain token-embedding, that is, the first temporary input information. Then, the server encodes the token-embedding according to the position of the voice request to obtain pos-embedding, that is, the second temporary input information. Then, the server encodes the token-embedding according to the position of the voice request to obtain the second temporary input information. Then, the server encodes the token-embedding according to the round information of the current round voice request and the previous round voice request. In this example, the current round of voice request is the second round of dialogue, the round information is 1, and the previous round of voice request is the first round of dialogue, the round information is 0. Then the wakeup-embedding, that is, the third temporary input information, is obtained. Finally, the server combines the first temporary input information, the second temporary input information, the third temporary input information and the current environment image encoding information of the cockpit to generate the target input information.

[0095] Subsequently, the label generation model analyzes the generated target input information and obtains an intent label, which can mark which intent label the current voice request belongs to among the first intent label, the second intent label, and the third intent label. In this example, the label generation model determines that the current voice request is "How is Haidilao?" as the first intent label.

[0096] Next, the server uses API prediction and API parameter filling to obtain "ChatGeneral", believing that the current voice request is more likely to be in a chat state, that is, a conversation between people, so the final NLU result is "Api-name: chatGeneral". The server sends this NLU result to the vehicle, and the vehicle completes the corresponding operations according to these instructions, thereby completing the voice interaction.

[0097] In this way, the server can more accurately judge the nature of the voice request by considering the current environment in the car, thereby providing a more appropriate response, improving the model's ability to process sequence data, ensuring that the server can accurately capture the user's intention and context information, and ensuring the coherence and accuracy of the conversation. This helps improve the user experience of the in-vehicle voice interaction system and avoid misunderstanding the user's intention.

[0098] See also Figure 4 In some implementations, step 04 (completing voice interaction according to the intent tag) includes:

[0099] 041: When the intent tag is the first intent tag, the current round of voice request is rejected.

[0100] The processor is also used to reject the current round of voice request when the intent tag is the first intent tag.

[0101] Specifically, the first intent tag is used to indicate that the server, after analyzing the content of the voice request and the environmental image, concludes that the user is in a human-to-human interaction state, that is, a chat state. For example, if the server receives a voice request, if the image shows that a passenger is talking to a part of the vehicle, the server may determine that this is a human-computer interaction scene. If the server receives a voice request, if the image shows that there is interaction between passengers, and the content of the voice request is not related to the vehicle function, the server may determine that this is a human-to-human interaction.

[0102] When the intent tag is the first intent tag, the server rejects the current round of voice request. That is, if it is determined to be a human-to-human interaction, the server may not take any action, or provide auxiliary information related to the content of the communication. Because in some cases, if the server cannot determine the user's intention, providing an incorrect response may be worse than providing no response. Rejection processing can avoid confusion or discomfort to users.

[0103] In this way, through rejection processing, the server can be prevented from providing erroneous or irrelevant responses, thereby avoiding misleading users and improving overall data quality.

[0104] See also Figure 5 In some implementations, step 04 (completing voice interaction according to the intent tag) includes:

[0105] 042: when the intent tag is the second intent tag, perform slot identification on the current round of voice request;

[0106] 043: Predict the application program interface for the current round of voice requests;

[0107] 044: Based on the slot recognition result and the predicted API, the API parameters are filled and the output execution result is sent to the vehicle to complete the voice interaction.

[0108] The processor is also used to perform slot identification on the current round of voice request when the intent tag is the second intent tag, and perform application program interface prediction on the current round of voice request, and execute application program interface parameter filling according to the slot identification result and the predicted application program interface, and output the execution result and send it to the vehicle to complete the voice interaction.

[0109] Specifically, the second intent tag means that after the server analyzes the content of the voice request and the environment image, it determines that the current state is the intercom state and the voice request given by the user is a complete voice request. For example, the voice request given by the user is: "I want to listen to song B by singer A".

[0110] When the intent tag is the second intent tag, the server performs slot identification on the current voice request to extract key information, such as time, location, and object. Slot identification helps the server understand the specific content of the user's request. Then, the server predicts the application program interface that the user may want to call based on the content and context of the current voice request.

[0111] For example, if the user request is about navigation, the server may predict that the user wants to use the map service API. Then, once the corresponding API is predicted, the server will fill in the parameters required by the API based on the results of slot identification.

[0112] For example, if the map service API is predicted, the server will fill in parameters such as the destination address. Finally, the server will execute the result output and send it to the vehicle to complete the voice interaction. For example, start the navigation system and provide route guidance.

[0113] In this way, according to the results of slot recognition, the predicted application interface is selected to execute the application interface parameter filling. The server can understand and respond to the user's voice commands, and directly output the execution results to the vehicle to complete the voice interaction, thereby reducing the delay of the vehicle system, improving the response speed to user commands, and providing a convenient and natural interactive experience.

[0114] See also Figure 6 In some embodiments, step 044 (performing application program interface parameter filling according to the slot identification result and the predicted application program interface, and outputting the execution result to the vehicle to complete the voice interaction) includes:

[0115] 0441: determining a target parameter for slot filling according to a current round of voice request, a result of slot recognition, a predicted application program interface, and a predicted application program interface type;

[0116] 0442: According to the slot recognition result and target parameters, execute the application interface parameter filling, output the execution result and send it to the vehicle to complete the voice interaction.

[0117] The processor is also used to determine the target parameters for slot filling according to the current round of voice request, the result of slot recognition, the predicted application interface and the predicted application interface type. And according to the result of slot recognition and the target parameter, the application interface parameter filling is executed, and the execution result is output and sent to the vehicle to complete the voice interaction.

[0118] Specifically, the server determines the target parameters for slot filling based on the current round of voice requests, slot recognition results, predicted APIs, and predicted API types. Then, the server performs API parameter filling based on the slot recognition results and target parameters, and outputs the execution results to the vehicle to complete the voice interaction.

[0119] According to the user's voice request "What will be the temperature tomorrow?", the result of slot recognition is: ["tomorrow" - date (Date)], the parameters of application interface 1 include two parameters "location" and "time", and the corresponding application interface type is "temperature query (Temperature Check)" type. It is then determined that the target parameter "tomorrow" in the result of slot recognition needs to be filled into application interface 1, which is "time". After filling "tomorrow" in the result of slot recognition into temperature query application interface 1, the query request can be executed accordingly to obtain tomorrow's temperature, thereby completing voice interaction.

[0120] For another example, the user's voice request is "navigate to Zhongguancun", and the result of slot recognition is: ["Zhongguancun" - place name (Place)], the parameters of application program interface 2 include two parameters, "departure place" and "destination", and the corresponding application program interface type is "navigation" type. It is then determined that the "Zhongguancun" in the result of slot recognition needs to be filled into the target parameter in application program interface 2 as "destination". After filling the "Zhongguancun" in the result of slot recognition into the navigation application program interface 2, the navigation task of navigating to Zhongguancun can be executed accordingly, thereby completing the voice interaction.

[0121] In this way, the predicted application interface can be selected to execute the application interface parameter filling according to the slot identification result and target parameters, and the execution result can be directly output and sent to the vehicle to complete the voice interaction, which can reduce the delay of the vehicle system and improve the response speed to user commands.

[0122] See also Figure 7 In some implementations, step 04 (completing voice interaction according to the intent tag) includes:

[0123] 045: when the intent tag is the third intent tag and the current round of voice request is not the first round of voice request, slot identification is performed on the current round of voice request;

[0124] 046: According to the slot recognition result and the API of the historical round voice request, the API parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

[0125] The processor is also used to perform slot identification on the current round of voice request when the intent tag is the third intent tag and the current round of voice request is not the first round of voice request. And according to the result of slot identification and the application program interface of the historical round of voice request, execute the application program interface parameter filling, output the execution result and send it to the vehicle to complete the voice interaction.

[0126] Specifically, in some embodiments, the judgment process of the third intent label can be as follows: First, the server analyzes the content of the voice request and the environmental image, and determines that the current state is the intercom state. Next, the label generation model first performs intent recognition processing on the data information corresponding to the current round of voice request in the input target input information, and determines that the current round of voice request is a weak intent. Finally, the data information corresponding to the current round of voice request, the data information corresponding to the previous round of voice request, and the natural language understanding results are subjected to intent recognition processing to determine that the current round of voice request is related to the previous round of voice request, and the previous round of voice request outputs the corresponding application program interface, so that the intent label can be confirmed as the first intent label. In other embodiments, there may be other confirmation processes, which are not limited here.

[0127] For example, the previous round of voice request is "I want to listen to Daoxiang", and the current round of voice request is "Cao Cao". Then there is only a single "Cao Cao", which may refer to the historical figure Cao Cao or Cao Cao in film and television dramas. The server cannot identify what information needs to be provided. However, based on the previous round of voice requests, it can be identified that the user is more likely to want to provide the song "Cao Cao". In this way, only looking at the current round of voice requests is a weak intent, but combined with the previous round of voice requests, it can be identified, which means that the intent label of the current round of voice requests is the first intent label.

[0128] The server first performs intent recognition on the user's voice request to determine the user's intent. In this scenario, if the identified intent tag is the first intent tag, the server will enter a specific processing flow. That is, the slot recognition results of the current round and the voice request information of the historical round are used to determine the application program interface (API) that needs to be called. The slot recognition results provide the location and content of the key information in the user request, while the historical round voice request provides the context information of the conversation. The server then performs application program interface parameter filling based on the slot recognition results and the application program interface of the historical round voice request. This means that the server will use the identified key information to fill in the parameters required for the API call. Finally, the server sends the execution results to the vehicle, and the vehicle completes the corresponding operations according to these instructions, thereby completing the voice interaction.

[0129] In this way, the adoption of an end-to-end architecture can reduce the latency of the vehicle system and improve the response speed to user commands. At the same time, the additional features of the application program interface that incorporates the slot recognition results of the user's voice requests and the historical rounds of voice requests can accurately identify the user's intentions and required parameters, and can call the correct application program interface to perform the corresponding operations, thereby meeting vehicle control requirements.

[0130] See also Figure 8 In some implementations, step 04 (completing voice interaction according to the intent tag) includes:

[0131] 047: when the intent tag is the third intent tag and the current round of voice request is the first round of voice request, slot identification is performed on the current round of voice request;

[0132] 048: Based on the preset database, perform application program interface prediction for the current round of voice request;

[0133] 049: Based on the slot recognition results and the predicted API, the API parameters are filled and the output execution results are sent to the vehicle to complete the voice interaction.

[0134] The processor is also used to perform slot identification on the current round of voice request when the intent tag is the third intent tag and the current round of voice request is the first round of voice request. And based on the preset database, perform application program interface prediction on the current round of voice request. And according to the result of slot identification and the predicted application program interface, execute application program interface parameter filling, output the execution result and send it to the vehicle to complete the voice interaction.

[0135] Specifically, the preset database refers to a pre-set database, in which, when the intent tag is the third intent tag and the current round of voice request is not the first round of voice request, in order to improve the user experience, the server may need to answer the user's voice request, but the user's voice request is unclear, so the server responds to the user's voice request based on the database.

[0136] First, the server identifies the intent tag of the current round of voice request. If the intent tag is the third intent tag, and the current round of voice request is not the first round of voice request, the next step is processed. Next, the server performs slot identification on the current round of voice request. The purpose of slot identification is to extract key information from the voice request, such as time, place, object, etc. Then, the server performs application interface parameter filling based on the results of slot identification and the application interface of historical rounds of voice requests. Subsequently, after the parameter filling is completed, the server will execute the API call and output the execution results. These results may be to start a function, adjust vehicle settings, or provide information. Finally, the server sends the execution results to the vehicle to complete the voice interaction. The vehicle's system or equipment will perform corresponding operations based on these results, such as adjusting the air conditioning temperature, starting navigation, etc.

[0137] In this way, based on the preset database, the application interface is predicted, and the server can obtain the command or operation to be executed when the intent tag is the second intent tag, thereby performing human-computer interaction and providing a convenient and natural user experience.

[0138] See also Fig. 9 In some implementations, step 04 (completing voice interaction according to the intent tag) includes:

[0139] 050: When the intent tag is the fourth intent tag, the current round of voice request is rejected.

[0140] The processor is also used to reject the current round of voice request when the intent tag is the fourth intent tag.

[0141] Specifically, in some embodiments, the judgment process of the fourth intent label can be as follows: First, the label generation model first performs intent recognition processing on the data information corresponding to the current round of voice request in the input target input information, and determines that the current round of voice request is a weak intent. Next, the data information corresponding to the current round of voice request, the data information corresponding to the previous round of voice request, and the natural language understanding result are subjected to intent recognition processing to determine that the current round of voice request is not associated with the previous round of voice request, or that the previous round of voice request did not output the corresponding application program interface, so that the intent label can be confirmed as the fourth intent label. In other embodiments, there may be other confirmation processes, which are not limited here.

[0142] For example, the previous round of voice request is "open the car window", and the current round of voice request is "Cao Cao". Then there is only a single "Cao Cao", which may refer to the historical figure Cao Cao or Cao Cao in film and television dramas. The server cannot identify it, and what information needs to be provided. And according to the previous round of voice request, it is found that the user's intention cannot be identified because the correlation is not strong. Or, the previous round of voice request is "Cao Cao", and the current round of voice request is "confession balloon". Then there is only a single "confession balloon", which the server cannot identify, and what information needs to be provided. And according to the previous round of voice request, it is found that the user's intention cannot be identified, and no other application interface is called. Or, in the current voice interaction cycle, the user only performed one round of voice interaction, that is, there is no historical voice request. And the current round of voice request is "Cao Cao", which may refer to the historical figure Cao Cao or Cao Cao in film and television dramas. The server cannot identify it, and what information needs to be provided. In this way, only looking at the current round of voice request is a weak intent, and it cannot be identified in combination with the previous round of voice request, it means that the intent label of the current round of voice request is the second intent label.

[0143] In this way, through rejection processing, the server can be prevented from providing erroneous or irrelevant responses, thereby avoiding misleading users and improving overall data quality.

[0144] The present application also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the one or more processors execute the method of the present application.

[0145] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file or some intermediate form. Computer readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution medium.

[0146] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc., mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to be considered as the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0147] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the art to which the embodiments of the present application belong.

[0148] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Receive the current round of voice request and the current environment image encoding information of the cockpit forwarded by the vehicle; Determine target input information according to the cockpit current environment image encoding information, the current round voice request, the historical round voice request and the natural language understanding result of the historical round voice request; Determining the intent label of the current round of voice request according to the target input information; Complete the voice interaction according to the intent label.

2. The method according to claim 1, characterized in that The determining target input information according to the cockpit current environment image encoding information, the current round voice request, the historical round voice request and the natural language understanding result of the historical round voice request includes: The preset mark, the current round voice request, the historical round voice request and the application program interface corresponding to the historical round voice request are concatenated to obtain first temporary input information; Performing position coding processing on the first temporary input information to determine second temporary input information; Determining third temporary input information according to the round information of the current round voice request and the round information of the historical round voice request; The first temporary input information, the second temporary input information, the third temporary input information and the cockpit current environment image coding information are spliced ​​to determine the target input information.

3. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the intention tag includes: When the intent tag is the first intent tag, the current round of voice request is rejected.

4. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the intention tag includes: When the intent tag is the second intent tag, performing slot identification on the current round of voice request; Performing application program interface prediction on the current round of voice request; According to the result of the slot identification and the predicted application program interface, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

5. The voice interaction method according to claim 4, characterized in that: The method of performing application program interface parameter filling according to the slot identification result and the predicted application program interface, and outputting the execution result to the vehicle to complete the voice interaction includes: Determine a target parameter for slot filling according to the current round of voice request, the result of the slot identification, the predicted application program interface, and the predicted application program interface type; According to the result of the slot identification and the target parameter, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

6. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the intention tag includes: When the intent tag is the third intent tag and the current round of voice request is not a first round of voice request, performing slot identification on the current round of voice request; According to the result of the slot identification and the application program interface of the historical round voice request, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

7. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the intention tag includes: When the intent tag is the third intent tag and the current round of voice request is the first round of voice request, performing slot identification on the current round of voice request; Based on a preset database, performing application program interface prediction on the current round of voice request; According to the result of the slot identification and the predicted application program interface, the application program interface parameter filling is executed, and the output execution result is sent to the vehicle to complete the voice interaction.

8. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the intention tag includes: When the intent tag is the fourth intent tag, the current round of voice request is rejected.

9. A server, characterized in that: The server includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the voice interaction method described in any one of claims 1 to 8 is implemented.

10. A non-volatile computer-readable storage medium containing a computer program, characterized in that: When the computer program is executed by one or more processors, the voice interaction method described in any one of claims 1 to 8 is implemented.

Images