Voice interaction method, device, equipment, chip, medium and product

Abstract

The invention provides a voice interaction method and device, equipment, a chip, a medium and a product, and is applied to the technical field of vehicle voice. The voice interaction method comprises the following steps: acquiring a voice instruction; contextual information of the voice instruction is obtained, the contextual information comprises current vehicle state information and historical dialogue information, and the historical dialogue information comprises multi-round dialogue information of the vehicle-mounted voice system in past time; according to the voice instruction and the context information, feedback information is generated, and the feedback information indicates whether vehicle operation indicated by the voice instruction is executed under the vehicle state information or not and reply content which is not repeated with the historical dialogue information; and outputting reply voice corresponding to the voice instruction according to the feedback information. In a vehicle-mounted voice interaction scene, the scheme improves the accuracy and individuation degree of the voice reply generated by the vehicle-mounted voice system.

Description

Technical Field

[0001] The present application is applied to the field of vehicle voice technology, and in particular relates to a voice interaction method, device, equipment, chip, medium and product. Background Art

[0002] With the development of intelligent technology, the in-vehicle voice system has become an important part of the vehicle. The personnel in the vehicle interact with the in-vehicle voice system to realize the voice operation of the vehicle functions.

[0003] Related technology: The vehicle-mounted voice system recognizes the input voice commands, and based on the recognition results, determines the vehicle operation indicated by the voice command in the predefined command execution logic; searches for the reply voice corresponding to the voice command in the predefined voice interaction template; and responds to the voice commands of the user in the vehicle by executing the vehicle operation and outputting the reply voice.

[0004] However, the above method is relatively simple and cannot generate personalized responses based on actual conditions. Summary of the invention

[0005] In order to solve the above problems, the present application proposes a voice interaction method, device, equipment, chip, medium and product, which can generate personalized responses according to the actual situation of the in-vehicle voice interaction scenario.

[0006] The first aspect of the present application provides a voice interaction method, comprising: obtaining a voice command; obtaining context information of the voice command, the context information including current vehicle status information and historical conversation information, the historical conversation information including multiple rounds of conversation information of a vehicle voice system in the past; generating feedback information based on the voice command and the context information, the feedback information indicating whether to execute the vehicle operation indicated by the voice command under the vehicle status information and reply content that is not repeated with the historical conversation information; and outputting a reply voice corresponding to the voice command based on the feedback information.

[0007] In some embodiments, generating feedback information according to the voice instruction and the context information includes: identifying a voice interaction intention according to the voice instruction; and generating the feedback information according to the voice interaction intention and the context information.

[0008] In some embodiments, identifying the voice interaction intention according to the voice instruction includes: recognizing the voice instruction to obtain recognized text; inputting the recognized text into a large language model; and performing semantic understanding and intent recognition on the recognized text in the large language model to obtain the voice interaction intention.

[0009] In some embodiments, generating the feedback information according to the voice interaction intention and the context information includes: inputting the context information into the large language model; generating a dialogue in the large language model according to the voice interaction intention and the context information, and obtaining the feedback information output by the large language model.

[0010] In some embodiments, generating the feedback information based on the voice interaction intention and the context information includes: filtering related information of the voice interaction intention in the context information, the related information including status information related to the voice interaction intention in the vehicle status information and conversation information related to the voice interaction intention in the historical conversation information; generating the feedback information based on the voice interaction intention and the related information.

[0011] In some embodiments, the vehicle status information is obtained by: acquiring the vehicle's driving information and / or attribute information of on-board objects on the vehicle, the on-board objects including on-board devices and / or on-board applications, the attribute information including identification information, currently supported vehicle functions, the current status of the vehicle functions, and whether the vehicle functions currently support voice operations; obtaining the vehicle status information based on the attribute information and / or the driving information.

[0012] In some embodiments, the context information also includes feature information of the voice instruction, and the feature information is obtained by performing feature extraction on the voice instruction to obtain the feature information, and the feature information includes at least one of the following: timbre feature, speaking speed feature, tone feature, and dialect type.

[0013] The second aspect of the present application provides a voice interaction device, including: a voice acquisition unit, used to acquire voice instructions; a context acquisition unit, used to acquire context information of the voice instructions, the context information includes current vehicle status information and historical dialogue information, and the historical dialogue information includes multiple rounds of dialogue information of the vehicle voice system in the past; a feedback generation unit, used to generate feedback information according to the voice instructions and the context information, the feedback information indicates whether the vehicle operation indicated by the voice instruction is executed under the vehicle status information and the reply content that is not repeated with the historical dialogue information; a voice output unit, used to output the reply voice corresponding to the voice instruction according to the feedback information.

[0014] The third aspect of the present application provides a vehicle-mounted device, including a memory and a processor; the memory is connected to the processor and is used to store programs; the processor is used to implement the voice interaction method as described in the first aspect or any embodiment of the first aspect by running the program in the memory.

[0015] The fourth aspect of the present application provides a chip, including a processor and a data interface, wherein the processor reads and runs a program stored in a memory through the data interface to execute the voice interaction method as described in the first aspect or any embodiment of the first aspect.

[0016] A fifth aspect of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the voice interaction method as described in the first aspect or any embodiment of the first aspect is implemented.

[0017] The sixth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the voice interaction method as described in the first aspect or any embodiment of the first aspect.

[0018] The present application proposes a voice interaction method, device, equipment, chip, medium and product, which combines voice commands and context information of voice commands to generate feedback information, and outputs a reply voice corresponding to the voice command based on the feedback information. The context information of the voice command includes the current vehicle status information, and the generated feedback information indicates whether to execute the vehicle operation indicated by the voice command under the current vehicle status information, so that the output reply voice conforms to the current vehicle status information; the context information of the voice command includes multiple rounds of dialogue information of the vehicle voice system in the past time, so that the output reply voice does not repeat the multiple rounds of dialogue information, thereby improving the personalization of the reply voice. Thus, a personalized reply is generated according to the actual situation during the vehicle voice interaction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of an implementation environment involved in an embodiment of the present application;

[0021] Figure 2 The following is a flow chart of the voice interaction method provided according to the embodiment of the present application. Figure 1 ;

[0022] Figure 3 The following is a flow chart of the voice interaction method provided according to the embodiment of the present application. Figure 2 ;

[0023] Figure 4A schematic diagram of the structure of a voice interaction device provided according to an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In the related technologies of in-vehicle voice interaction, the user's voice commands are responded to through predefined voice templates and static response mechanisms. This method has the following disadvantages: on the one hand, the output response voice is often relatively simple and fixed. For the repeated voice commands issued by the user, the in-vehicle equipment (such as the car computer) outputs the same response voice, which lacks flexibility and personalization; on the other hand, there is a lack of real-time tracking of the vehicle status, and the output response voice does not match the current status of the vehicle.

[0027] For example, in one scenario: when the user gives the first voice instruction "turn on the zero-gravity seat", the in-vehicle device replies "the zero-gravity seat has been turned on for you"; when the user gives the next voice instruction "turn on the zero-gravity seat", the in-vehicle device replies "the zero-gravity seat has been turned on for you" without considering the current state of the seat. In another scenario: due to differences in equipment and functional configurations of different models, some models are not equipped with zero-gravity seats. When the user gives the voice instruction "turn on the zero-gravity seat", the in-vehicle device does not reply to the user, causing trouble to the user. In another scenario: when the vehicle is in certain states, some operations that may pose a safety hazard to the vehicle or the people in the vehicle are not allowed. If the user's voice instruction requires the execution of an operation that poses a safety hazard, the in-vehicle device may not reply to the user or operate according to the user's voice instruction regardless of the current state of the vehicle.

[0028] As a result, in the in-vehicle voice interaction, it is impossible to generate personalized responses for the user's voice commands based on the actual situation, resulting in the user's failure to feel the intelligent adaptability of the in-vehicle voice system in the actual interaction scenario, resulting in a poor user experience.

[0029] To solve the above problems, the embodiments of the present application provide a voice interaction method, device, equipment, chip, medium and product, which are applied to the field of in-vehicle voice technology. In the voice interaction method, feedback information is generated by combining the voice command and the context information of the voice command, and the reply voice corresponding to the voice command is output based on the feedback information. The context information of the voice command contains the current vehicle status information and the multi-round dialogue information of the in-vehicle voice system in the past. Therefore, the feedback information is obtained by taking into account the current vehicle status information and the multi-round dialogue information in the past, so that the output reply voice conforms to the current vehicle status information, and the multi-round dialogue information in the past is not repeated and personalized. Therefore, in the process of in-vehicle voice interaction, it is possible to flexibly provide personalized replies to the voice commands of users in the vehicle according to the actual situation of the in-vehicle voice interaction, thereby improving the intelligence of the in-vehicle voice interaction and improving the user experience.

[0030] Exemplary Implementation Environment

[0031] Please refer to Figure 1 , Figure 1 Schematic diagram of an implementation environment involved in an embodiment of the present application. The implementation environment involved in the present application includes a vehicle 10 and an in-vehicle device 20. In response to a voice command of a user (driver or passenger) in the vehicle 10, the in-vehicle device 20 may output a reply voice corresponding to the voice command based on the voice command and context information of the voice command.

[0032] Among them, the vehicle-mounted device 20 is a smart car machine, a smart speaker, etc. deployed on the vehicle 10, and a vehicle-mounted voice assistant is installed in the vehicle-mounted device 20.

[0033] It should be noted that the embodiments of the present application can also be applied to: devices that establish a communication connection with an on-board device (such as a car computer) on the vehicle 10, such as a smart speaker or a smart phone connected to the on-board device via Bluetooth, a network or an interface circuit; a server or other remote device that communicates with the on-board device (such as a car computer) on the vehicle 10 via a network.

[0034] Exemplary Methods

[0035] See also Figure 2 In an exemplary embodiment, a voice interaction method is provided, the voice interaction method comprising the following steps:

[0036] S201, obtaining a voice command.

[0037] Among them, this embodiment can be applied to electronic devices. In particular, it can be applied to vehicle-mounted devices, such as vehicle computers, or other devices connected to vehicle computers. Subsequently, the embodiment is described by taking the application of this embodiment to a vehicle-mounted device deployed with a vehicle-mounted voice system (or a vehicle-mounted voice assistant) as an example.

[0038] In this embodiment, a voice command input by a user in the vehicle is obtained. The voice command is a vehicle operation command input by the user in voice, for example, a voice command instructs to open a window, turn on the air conditioner in the vehicle, open a navigation software in the vehicle computer and navigate to a destination, etc.

[0039] In one example, the user's voice in the vehicle can be collected through a sound sensor on the vehicle, and the user's voice can be pre-processed (for example, noise removal and voice enhancement) to obtain a voice command.

[0040] In this example, the user can wake up the vehicle voice system by speaking the wake-up word. When the vehicle voice system is in the awake state, the sound sensor on the vehicle can collect the user's voice in the vehicle, pre-process the user's voice, and obtain the voice command.

[0041] S202: Acquire context information of the voice command.

[0042] The context information of the voice command includes the current vehicle status information and historical dialogue information, and the historical dialogue information includes multiple rounds of dialogue information of the vehicle voice system in the past. The current vehicle status information reflects the current vehicle actual situation, so as to timely track the current vehicle actual situation and improve the flexibility and accuracy of the voice response during the voice interaction process.

[0043] Here, "current" is understood as the current time, which may refer to the time when the voice command is obtained, or may refer to the time period from obtaining the voice command to outputting the reply voice. Past time refers to the time before the current time.

[0044] The current vehicle status information may include the current state of the vehicle itself and / or the current situation of the vehicle's surroundings, and the vehicle's surroundings may include the vehicle's internal environment and / or the vehicle's external environment. Thus, the current vehicle status information reflects the actual situation inside the vehicle, the vehicle itself, and / or the vehicle's exterior, thereby improving the comprehensiveness of the vehicle status information.

[0045] In this embodiment, after the voice command is obtained, the context information of the voice command can be obtained from the vehicle or from a remote device. Different information in the context information can be obtained from different places, such as from different databases, or requested from different devices; or can be obtained from the same place, such as from the same database.

[0046] In one example, context information of the voice command is obtained from a database, wherein the vehicle state information and the historical conversation information may be stored in different databases, the current vehicle state information may be obtained from the vehicle state database, and the historical conversation information may be obtained from the conversation database. The database may be a local database or a database stored in a remote device.

[0047] In another example, a request message may be sent to the vehicle's status management system, the request message being used to request the current vehicle status information; a response message returned by the status management system is received; and the current vehicle status information may be obtained from the response message. Thus, by requesting the vehicle status in real time, the accuracy of obtaining the current vehicle status information is improved.

[0048] In one example, the multi-round dialogue information includes N rounds of dialogue information, and obtaining historical dialogue information includes: obtaining the N rounds of dialogue information of the vehicle voice system closest to the current time, where N is a set constant and N is greater than 1. For example, obtaining the 3 rounds of dialogue information of the vehicle voice system closest to the current time.

[0049] In another example, the multi-round conversation information includes multi-round conversation information of the vehicle voice system in the past time period, and the past time period is determined according to the current time, and obtaining the historical conversation information includes: determining the past time period according to the current time and the set duration; obtaining the multi-round conversation information of the vehicle voice system in the past time period. In the process of determining the past time period according to the current time and the set duration, it can be determined that the end time of the past time period is the current time and the duration of the past time period is the set duration.

[0050] Therefore, by determining the specific number of rounds of dialogue information or the specific duration of the past time, the data volume of the historical dialogue information in each in-vehicle voice interaction is ensured to be relatively stable, avoiding too much or too little historical dialogue information, which affects the efficiency or effect of the reply voice generation.

[0051] S203: Generate feedback information according to the voice instruction and context information.

[0052] The feedback information indicates whether the vehicle operation indicated by the voice command is executed under the current vehicle status information and the reply content that is not repeated with the historical dialogue information.

[0053] In this embodiment, based on the voice command and context information, a decision is made on whether to execute the vehicle operation indicated by the voice command in the current vehicle state, and a decision result is obtained. For example, if the current window is open, the window opening operation indicated by the voice command may not be executed, or if the vehicle is in a driving state, the seat adjustment operation indicated by the voice command may not be executed; based on the context information and the above decision result, a reply content that conforms to the decision result and does not repeat the historical dialogue information can be generated. On the one hand, the reply content does not repeat the historical dialogue information, which improves the personalization of the reply content. On the other hand, by tracking the historical dialogue information, the coherence of the in-vehicle voice interaction is improved, making the voice interaction process more natural and smooth.

[0054] In one example, the voice command can be recognized by voice recognition to determine the vehicle operation indicated by the voice command; based on the current vehicle state information and the set vehicle operation rules, it is determined whether to execute the vehicle operation indicated by the voice command under the current vehicle state information. The vehicle operation rules include the state conditions that need to be met for the vehicle operation. If the current vehicle state information does not meet the state conditions that need to be met for the vehicle operation indicated by the voice command, it is determined that the vehicle operation indicated by the voice command will not be executed under the current vehicle state information. Otherwise, the vehicle operation indicated by the voice command will be executed under the current vehicle state information. Thus, through voice recognition and state condition judgment, the accuracy of determining whether to execute the vehicle operation indicated by the voice command is improved, avoiding the potential safety hazards brought by voice interaction to the vehicle and the people in the vehicle.

[0055] S204: Outputting a reply voice corresponding to the voice instruction according to the feedback information.

[0056] In this embodiment, the reply content in the feedback information can be converted from text format to voice format through relevant text-to-speech technologies, and a reply voice corresponding to the voice command can be obtained, and the reply voice can be output to the voice playback device on the vehicle, so that the voice playback device plays the reply voice, thereby realizing a reply to the voice command of the user on the vehicle.

[0057] For example, when the user instructs to open the window by voice: if the window is already open, the in-vehicle voice system will answer: "The window is already open"; if the window is not open, the in-vehicle voice system can perform the operation of opening the window and answer: "The window has been opened for you"; if the vehicle does not support voice control of the windows, the in-vehicle system will answer: "This device does not support voice control."

[0058] In the embodiment of the present application, the voice command and the context information of the voice command are integrated to generate the reply voice in the in-vehicle voice interaction scenario. The context information includes the real-time status of the vehicle and multiple rounds of conversations of the in-vehicle voice system in the past. On the one hand, it helps to improve the ability to understand the voice command during the reply voice generation process. On the other hand, it can generate a natural and personalized reply voice in combination with the scene and historical conditions. Therefore, the accuracy, flexibility, personalization and intelligence of the in-vehicle voice interaction are effectively improved, and the voice interaction experience of the vehicle user is improved.

[0059] See also Figure 3 In another exemplary embodiment, a voice interaction method is provided, the voice interaction method comprising the following steps:

[0060] S301, obtaining a voice command.

[0061] S302: Acquire context information of the voice command.

[0062] The implementation principles and technical effects of S301 and S302 refer to the aforementioned embodiments and will not be described in detail.

[0063] S303: Identify the voice interaction intention according to the voice instruction.

[0064] Among them, voice interaction intention refers to the behavioral intention expressed by the user in the vehicle through voice commands, and the behavioral intention can correspond to one or more vehicle operations. For example, if the user expresses through voice commands that the temperature inside the car is high, then it can be analyzed that the behavioral intention is to lower the temperature inside the car, and the corresponding vehicle operations can be turning on the air conditioner, lowering the air conditioner temperature, opening the window, etc.

[0065] In this embodiment, the pre-trained deep learning model can be used to perform intent recognition on the voice command to obtain the voice interaction intent corresponding to the voice command.

[0066] In an example, Figure 3 As shown, S303 may include S3031 to S30323:

[0067] S3031, recognize the voice command to obtain recognized text.

[0068] In this example, the voice command can be input into the voice recognition model, and the voice command can be recognized in the voice recognition model to obtain the recognized text. The voice recognition model can be, for example, a deep neural network, a convolutional neural network, a recurrent neural network, etc. trained for voice recognition.

[0069] S3032, input the recognized text into large language models (LLMs).

[0070] S3033, perform semantic understanding and intent recognition on the recognized text in the large language model to obtain the voice interaction intention.

[0071] Among them, a large language model can be pre-trained so that the large language model has the ability to recognize intent. Large language models such as generative pre-trained transformer (GPT) model, bidirectional encoder representation from transformers (BERT), etc.

[0072] In this example, the recognized text is input into the large language model, which contains an intent recognition network. The intent recognition network can be used to perform semantic understanding and intent recognition on the recognized text, and obtain the voice interaction intent corresponding to the recognized text. The specific process of the large language model performing semantic understanding and intent recognition on the recognized text depends on the model structure of the large language model. The model structure can refer to models such as GPT and BERT, and the model structure of the large language model is not limited here.

[0073] Therefore, by utilizing the language processing capabilities of a large language model, the accuracy of intent recognition for voice commands can be improved, thereby improving the accuracy of voice responses.

[0074] S304: Generate feedback information according to the voice interaction intention and context information.

[0075] In this embodiment, after obtaining the voice interaction intention, a decision can be made on whether to execute the vehicle operation indicated by the voice command under the current vehicle status information based on the current vehicle status information and the voice interaction intention in the context information, and a decision result can be obtained. Based on the decision result and the historical conversation information in the context information, the reply content corresponding to the voice command can be determined to obtain feedback information.

[0076] An example, such as Figure 3 As shown, S304 may include S3041 and S3042:

[0077] S3041, input context information into the large language model.

[0078] The large language model and the large language model in S3032 may be the same model.

[0079] S3042: In the large language model, dialogue generation is performed according to the voice interaction intention and context information to obtain feedback information output by the large language model.

[0080] Among them, the large language model can be pre-trained, and the learning ability of the large language model can be used to enable the large voice model to have semantic understanding, intent recognition and dialogue generation capabilities during the training process. It can generate personalized reply content based on the recognized text and context information of the voice commands.

[0081] In this example, the recognized text and context information of the voice command can be input into the large language model. In the large language model: first, the intent recognition network is used to perform semantic understanding and intent recognition on the recognized text to obtain the voice interaction intent, and then the dialogue generation network is used to generate dialogues based on the voice interaction intent and context information to obtain feedback information output by the large language model.

[0082] In one example, the context information may be filtered for information related to the voice interaction intention, including state information related to the voice interaction intention in the vehicle state information and conversation information related to the voice interaction intention in the historical conversation information; then, feedback information is generated based on the voice interaction intention and the related information. Thus, by filtering related information from the context information, redundant data in the context information is reduced, and the efficiency and accuracy of feedback information generation are improved.

[0083] In this example, the vehicle status information may include multiple pieces of status information, and the vehicle status information may be text-matched with the voice interaction intent to obtain a first matching result, the first matching result including the matching degrees of the multiple pieces of status information and the voice interaction intent; based on the first matching result, the status information related to the voice interaction intent in the vehicle status information is determined, for example, the status information related to the voice interaction intent is determined to be the status information in the first matching result whose matching degree is greater than a first matching threshold; for another example, the status information related to the voice interaction intent is determined to be the status information in the first matching result whose matching degree is arranged in the top K positions from large to small, where K is greater than or equal to 1.

[0084] In this example, the historical conversation information includes multiple rounds of conversation information, and the historical conversation information can be text-matched with the voice interaction intent to obtain a second matching result, which includes the matching degrees of the multiple rounds of conversation information and the voice interaction intent respectively; based on the second matching result, the conversation information related to the voice interaction intent in the historical conversation information is obtained, for example, the conversation information related to the voice interaction intent is determined to be the conversation information in the second matching result whose matching degree is greater than the second matching threshold; for another example, the state information related to the voice interaction intent is determined to be the state information in the second matching result whose matching degree is arranged in the first J positions from large to small, and J is greater than or equal to 1.

[0085] Furthermore, the complexity of the voice interaction intention can be determined, and if the complexity is less than the degree threshold, the associated information of the voice interaction intention is filtered from the context information. Thus, considering that too much context information is not needed when the voice interaction intention is relatively simple (or single), by filtering the associated information from the context information, the redundant data in the context information is reduced, and the efficiency of generating feedback information is improved.

[0086] Among them, the complexity of the voice interaction intention can be determined based on at least one of the following: the number or types of vehicle equipment involved in the voice interaction intention, the number of vehicle operations involved in the voice interaction intention, and the number of intentions contained in the voice interaction intention.

[0087] Furthermore, the number of vehicle operations related to the voice interaction intention may be determined. If the number of vehicle operations related to the voice interaction intention is less than a quantity threshold, the associated information of the voice interaction intention is filtered in the context information.

[0088] The number of vehicle operations related to the voice interaction intention can be determined by matching the voice interaction intention with the operation information of the vehicle operation. The operation information of the vehicle operation may include the operation object involved in the vehicle operation (devices, applications on the vehicle, etc.) and / or the effect brought about by the vehicle operation (such as cooling, improving air quality).

[0089] For example, the voice interaction intention is to open the navigation software on the vehicle and navigate to the destination. The vehicle operations related to the voice interaction intention include opening the navigation software and inputting navigation instructions into the navigation software. Status information related to the navigation software, vehicle location, etc. can be filtered out from the current vehicle status information, and conversation information related to the navigation software, vehicle location, etc. can be filtered out from the historical conversation information.

[0090] Optionally, "filtering related information of the voice interaction intention in the context information; generating feedback information according to the voice interaction intention and the related information" can be performed in the large language model.

[0091] In this optional method, the recognized text and context information of the voice command can be input into the large language model. In the large language model: first, the recognized text is semantically understood and the intent is recognized through the intent recognition network to obtain the voice interaction intent; in the context information, the associated information of the voice interaction intent is screened; the voice interaction intent and the associated information of the voice interaction intent are input into the dialogue generation network, and in the dialogue generation network, dialogue generation is performed according to the voice interaction intent and the associated information of the voice interaction intent to obtain the feedback information output by the large language model.

[0092] S305: Output a reply voice corresponding to the voice instruction according to the feedback information.

[0093] The implementation principle and technical effects of S305 refer to the aforementioned embodiments and will not be described in detail.

[0094] In the embodiment of the present application, by analyzing the voice interaction intention of the voice command and combining the voice interaction intention and the context information of the voice command, the reply voice in the in-vehicle voice interaction scenario is generated. In this process, a large language model can be used to further improve the accuracy, flexibility, personalization and intelligence of the in-vehicle voice interaction, thereby improving the voice interaction experience of vehicle users.

[0095] Based on any of the above embodiments, the following examples are further provided:

[0096] In one example, the vehicle status information is obtained by: obtaining the vehicle's driving information and / or the attribute information of the vehicle's onboard objects, the vehicle-mounted objects include vehicle-mounted devices and / or vehicle-mounted applications, and the attribute information includes identification information, currently supported vehicle functions, the current status of vehicle functions, and whether the vehicle functions currently support voice operation. Thus, the vehicle status information not only covers the equipment on the vehicle, but also covers the vehicle's own driving conditions and the applications on the vehicle, which improves the comprehensiveness of the vehicle status information and thus improves the accuracy of the generated reply voice.

[0097] The driving information of the vehicle may include at least one of the following: the model of the vehicle, the positioning information of the vehicle, the speed information of the vehicle, the number of passengers on the vehicle, or the power information of the vehicle. The power information of the vehicle may include the remaining fuel amount of the vehicle, the tire pressure of the vehicle, the remaining power of the vehicle, etc.

[0098] The identification information of the vehicle-mounted object may be the name or number of the vehicle-mounted object.

[0099] For example, when the user's voice command in the vehicle indicates navigation to the destination, feedback information can be generated based on the mileage to the destination, the vehicle's power information, the current status of the navigation software, whether the navigation software supports voice control, etc. If the remaining fuel is not enough to travel to the destination, a reply voice message of "The current fuel is not enough to travel to the destination, it is recommended to refuel in time" can be output. If the current status of the navigation software is to navigate to the destination, a prompt voice message of "Currently heading to the destination" can be output. In this way, the user's voice command can be responded to intelligently to improve the user experience.

[0100] For example, when a user opens a window in a vehicle by voice command, feedback information can be generated based on the vehicle's location information, driving speed, current state of the window, whether the window supports voice control, etc. If it is determined that it is raining locally based on the vehicle's location information, a voice prompt of "It is not advisable to open the window on a rainy day" can be output; if the vehicle is driving too fast, a voice prompt of "Considering driving safety, it is recommended not to open the window" can be output. I will not give examples one by one here.

[0101] In one example, the context information of the voice command also includes feature information of the voice command, and the feature information is obtained by extracting features of the voice command to obtain the feature information of the voice command. The feature information of the voice command includes at least one of the following: timbre feature, speech speed feature, tone feature, or dialect type.

[0102] In this example, characteristic information of the reply voice can be determined based on characteristic information of the voice command, and the feedback information includes characteristic information of the reply voice, which may include at least one of the following: timbre characteristics, speech speed characteristics, tone characteristics, or dialect type. In the process of outputting the reply voice corresponding to the voice command according to the feedback information, the reply voice can be generated based on the characteristic information of the reply voice and the reply content, and then the reply voice is output. Thus, in the in-vehicle voice interaction scenario, not only the personalization and flexibility of the content of the reply voice is achieved, but also the personalization and flexibility of one or more aspects of the timbre, speech speed, tone or dialect of the reply voice is achieved.

[0103] For example, when the timbre, speech speed, tone, etc. of the voice command reflect that the user is a child, the reply voice can adopt the timbre, speech speed, tone, etc. of a child. For another example, when the dialect type of the voice command is a certain local dialect, the reply voice can also adopt the local dialect.

[0104] Exemplary Devices

[0105] Correspondingly, an embodiment of the present application also provides a voice interaction device.

[0106] See also Figure 4 In an exemplary embodiment, a speech interaction device 400 is provided, the speech interaction device comprising: a speech acquisition unit 401, a context acquisition unit 402, a feedback generation unit 403 and a speech output unit 404, wherein:

[0107] A voice acquisition unit 401 is used to acquire voice instructions; a context acquisition unit 402 is used to acquire context information of the voice instructions, the context information includes current vehicle status information and historical dialogue information, the historical dialogue information includes multiple rounds of dialogue information of the vehicle voice system in the past; a feedback generation unit 403 is used to generate feedback information based on the voice instructions and the context information, the feedback information indicates whether the vehicle operation indicated by the voice instruction is executed under the vehicle status information and the reply content that is not repeated with the historical dialogue information; a voice output unit 404 is used to output the reply voice corresponding to the voice instruction based on the feedback information.

[0108] In some embodiments, the feedback generation unit 403 is specifically used to: identify the voice interaction intention according to the voice instruction; and generate feedback information according to the voice interaction intention and context information.

[0109] In some embodiments, the feedback generation unit 403 is specifically used to: recognize the voice command to obtain the recognized text; input the recognized text into the large language model; in the large language model, perform semantic understanding and intent recognition on the recognized text to obtain the voice interaction intention.

[0110] In some embodiments, the feedback generation unit 403 is specifically used to: input context information into a large language model; generate a dialogue in the large language model according to the voice interaction intention and the context information, and obtain feedback information output by the large language model.

[0111] In some embodiments, the feedback generation unit 403 is specifically used to: filter related information of the voice interaction intention in the context information, the related information including status information related to the voice interaction intention in the vehicle status information and conversation information related to the voice interaction intention in the historical conversation information; generate feedback information based on the voice interaction intention and the related information.

[0112] In some embodiments, vehicle status information is obtained by: obtaining the vehicle's driving information and / or attribute information of on-board objects on the vehicle, the on-board objects including on-board devices and / or on-board applications, the attribute information including identification information, currently supported vehicle functions, the current status of the vehicle functions, and whether the vehicle functions currently support voice operations; obtaining the vehicle status information based on the attribute information and / or driving information.

[0113] In some embodiments, the context information also includes feature information of the voice command, and the feature information is obtained by performing feature extraction on the voice command to obtain feature information, and the feature information includes at least one of the following: timbre feature, speaking speed feature, tone feature, and dialect type.

[0114] The voice interaction device provided in this embodiment belongs to the same application concept as the voice interaction method provided in the above embodiments of this application, and can execute the voice interaction method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the voice interaction method provided in the above embodiments of this application, and will not be repeated here.

[0115] The functions implemented by the above speech acquisition unit 401, context acquisition unit 402, feedback generation unit 403 and speech output unit 404 may be implemented by the same or different processors respectively, which is not limited in the embodiment of the present application.

[0116] It should be understood that the voice acquisition unit 401, the context acquisition unit 402, the feedback generation unit 403 and the voice output unit 404 in the above voice interaction device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the unit in the device can be implemented in the form of a hardware circuit, and the functions of some or all of the units can be realized by designing the hardware circuit. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD. Taking FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all of the above units. All units of the above device may be implemented entirely in the form of a processor calling software, or entirely in the form of a hardware circuit, or partially in the form of a processor calling software and the rest in the form of a hardware circuit.

[0117] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0118] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0119] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0120] An embodiment of the present application further provides a control device, which includes a processor, a voice collection component and a voice output component. The processor in the control device is connected to the voice collection component and the voice output component through a control circuit in the control device.

[0121] Among them, the voice collection component specifically refers to a functional component that can collect voice commands in the vehicle under the control of the processor, such as a microphone; the voice output component specifically refers to a functional component that can output reply voice under the control of the processor, such as a speaker.

[0122] The processor in the control device is also a circuit with signal processing capabilities. It generates corresponding reply voice and controls the output of the voice output component for the voice command in the vehicle by executing any of the voice interaction methods introduced in the above embodiments. The specific implementation of the processor can refer to the above processor implementation, and the embodiments of this application are not strictly limited.

[0123] The processor in the control device may refer to the processor in the aforementioned device.

[0124] Exemplary Electronic Devices

[0125] Another embodiment of the present application further provides an electronic device, in particular, the electronic device may be a vehicle-mounted device, and further, the electronic device is a vehicle-mounted voice device. Figure 5 As shown, the electronic device includes: a memory 500 and a processor 510; wherein the memory 500 is connected to the processor 510 and is used to store programs; the processor 510 is used to implement the voice interaction method disclosed in any of the above embodiments by running the program stored in the memory 500.

[0126] Specifically, the electronic device may further include: a bus, a communication interface 520 , an input device 530 and an output device 540 .

[0127] The processor 510, the memory 500, the communication interface 520, the input device 530 and the output device 540 are connected to each other via a bus.

[0128] A bus may include a pathway that transfers information between components of a computer system.

[0129] The processor 510 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0130] The processor 510 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0131] The memory 500 stores a program for executing the technical solution of the present application, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes computer operation instructions. More specifically, the memory 500 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0132] The input device 530 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0133] Output device 540 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0134] The communication interface 520 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0135] The processor 510 executes the program stored in the memory 500 and calls other devices, which can be used to implement the various steps of any voice interaction method provided in the above embodiments of the present application.

[0136] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the voice interaction method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the description of the above-mentioned embodiment of the voice interaction method.

[0137] Exemplary computer program products and storage media

[0138] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the voice interaction method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0139] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0140] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the voice interaction method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0141] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0142] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0143] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0144] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0145] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0146] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0148] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0149] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0150] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0151] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voice interaction method, characterized in that: include: Get voice commands; Acquire context information of the voice command, wherein the context information includes current vehicle state information and historical dialogue information, wherein the historical dialogue information includes multiple rounds of dialogue information of the vehicle voice system in the past; Generate feedback information according to the voice command and the context information, the feedback information indicating whether to execute the vehicle operation indicated by the voice command under the vehicle state information and the reply content that is not repeated with the historical dialogue information; According to the feedback information, a reply voice corresponding to the voice instruction is output.

2. The voice interaction method according to claim 1, characterized in that: The generating feedback information according to the voice instruction and the context information includes: According to the voice command, identifying the voice interaction intention; The feedback information is generated according to the voice interaction intention and the context information.

3. The voice interaction method according to claim 2, characterized in that: The step of identifying the voice interaction intention according to the voice instruction includes: Recognizing the voice command to obtain a recognized text; Inputting the recognized text into a large language model; In the large language model, semantic understanding and intent recognition are performed on the recognized text to obtain the voice interaction intention.

4. The voice interaction method according to claim 3, characterized in that: The generating the feedback information according to the voice interaction intention and the context information includes: Inputting the context information into the large language model; In the large language model, a dialogue is generated according to the voice interaction intention and the context information to obtain the feedback information output by the large language model.

5. The voice interaction method according to claim 3, characterized in that: The generating the feedback information according to the voice interaction intention and the context information includes: Filtering the associated information of the voice interaction intention in the context information, the associated information including state information related to the voice interaction intention in the vehicle state information and conversation information related to the voice interaction intention in the historical conversation information; The feedback information is generated according to the voice interaction intention and the associated information.

6. The voice interaction method according to any one of claims 1 to 5, characterized in that: The vehicle status information is obtained by: Acquiring driving information of the vehicle and / or attribute information of an onboard object on the vehicle, the onboard object including an onboard device and / or an onboard application, the attribute information including identification information, currently supported vehicle functions, current status of the vehicle functions, and whether the vehicle functions currently support voice operation; The vehicle status information is obtained according to the attribute information and / or the driving information.

7. The voice interaction method according to any one of claims 1 to 5, characterized in that: The context information also includes feature information of the voice instruction, and the feature information is obtained in the following manner: Feature extraction is performed on the voice instruction to obtain the feature information, where the feature information includes at least one of the following: timbre feature, speech speed feature, tone feature or dialect type.

8. A voice interaction device, characterized in that: include: A voice acquisition unit, used for acquiring voice instructions; A context acquisition unit, configured to acquire context information of the voice command, wherein the context information includes current vehicle state information and historical dialogue information, wherein the historical dialogue information includes multiple rounds of dialogue information of the vehicle voice system in the past; a feedback generating unit, configured to generate feedback information according to the voice command and the context information, the feedback information indicating whether the vehicle operation indicated by the voice command is executed under the vehicle state information and a reply content that is not repeated with the historical dialogue information; The voice output unit is used to output a reply voice corresponding to the voice instruction according to the feedback information.

9. A vehicle-mounted device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the voice interaction method as described in any one of claims 1 to 7 by running the program in the memory.

10. A chip, characterized in that: It comprises a processor and a data interface, wherein the processor reads and runs a program stored in a memory through the data interface to execute the voice interaction method as described in any one of claims 1 to 7.

11. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the voice interaction method according to any one of claims 1 to 7 is implemented.

12. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the voice interaction method as described in any one of claims 1 to 7.

Images