Voice interaction method, voice interaction device, server and readable storage medium

Abstract

The application discloses a voice interaction method, a voice interaction device, a server and a storage medium. The voice interaction method comprises the following steps: performing voice recognition on a received voice request of a current user in a vehicle cabin to obtain a current recognition text; in the case that the current recognition text comprises a salutation and is a pure salutation, obtaining a previous recognition text; wherein the previous recognition text is a recognition text corresponding to a previous voice request of a user, and the previous recognition text does not comprise a salutation; in the case that the previous recognition text is a valid instruction, issuing a wake-up instruction to the vehicle, so that the vehicle wakes up a voice assistant and a user for voice interaction according to the wake-up instruction. Thus, when a user requests to wake up a voice assistant in a vehicle, the voice assistant can be accurately woken up in time, so that the wake-up demand of the user in different types of application scenarios is met, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, and in particular to a voice interaction method, a voice interaction device, a server and a readable storage medium. BACKGROUND

[0002] Currently, a vehicle-mounted voice assistant usually needs to be woken up before an instruction can be issued, which is inconvenient to use. The wake-up-free technology is mainly to solve this problem, so that the user can directly issue an instruction to let the voice assistant execute without waking up, thereby improving the user experience. In wake-up-free, it is necessary to determine whether a voice is an instruction to the vehicle-mounted assistant. In practice, the user may say an effective instruction while chatting in the car, but it is not an instruction to the vehicle-mounted assistant, which is easy to cause false wake-up. In addition, the ASR (Automatic Speech Recognition) module may misrecognize some voices as effective instructions, which will also cause false wake-up. False wake-up will bring bad user experience to the user, and there is room for improvement. SUMMARY

[0003] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, one object of the present application is to propose a voice interaction method which can recognize voice requests in a vehicle cabin to avoid false wake-up and improve user experience.

[0004] According to the voice interaction method of the present application, the voice recognition of the received current round of user voice requests in the vehicle cabin is performed to obtain the current round of recognition text. In the case where the current round of recognition text includes an address and is a pure address, the previous round of recognition text is obtained. The previous round of recognition text is the recognition text corresponding to the previous round of user voice requests, and the previous round of recognition text does not include an address. In the case where the previous round of recognition text is an effective instruction, a wake-up instruction is issued to the vehicle so that the vehicle wakes up the voice assistant and the user for voice interaction according to the wake-up instruction.

[0005] Thus, when the user requests to wake up the voice assistant in the car, even if the user first speaks the corresponding execution action and then calls the wake-up word of the voice assistant, the voice interaction method in the present application can determine that the user has a request to wake up the voice assistant based on the voice request content, and accurately wake up the voice assistant. In other words, even if the user outputs the voice request in the unconventional order of first executing the action and then addressing the language, the voice assistant can be effectively woken up in time, thereby meeting the wake-up needs of the user in different types of application scenarios and improving the user experience.

[0006] Before the issuing of the wake-up instruction to the vehicle, the method further comprises: determining that a time interval between the current round of recognized text and the previous round of recognized text is less than a target time length. In this way, by setting the interval time length of the two rounds of recognized text, the voice assistant can also be prevented from being mistakenly woken up, and the wake-up accuracy is improved.

[0007] After the obtaining of the current round of recognized text, the method further comprises: in a case where it is determined that the current round of recognized text does not include an address, caching the current round of recognized text, time information of the current round of recognized text, and an audio area of a current round of user voice request corresponding to the current round of recognized text. In this way, the time information and the corresponding audio area of the recognized text that does not include an address are identified and cached, so that the recognized text and the corresponding time information and audio area are cached for the next round of application.

[0008] In a case where it is determined that the time interval between the current round of recognized text and the previous round of recognized text is not less than a target time length, or the previous round of recognized text is an invalid instruction, no instruction for instructing the voice assistant to wake up is issued to the vehicle. In this way, by setting the interval time length of the two rounds of recognized text and in combination with the content of the previous round of recognized text, the voice assistant can also be further prevented from being mistakenly woken up, and the wake-up accuracy is improved.

[0009] After the determining that the current round of recognized text includes an address, the method further comprises: in a case where it is determined that the current round of recognized text is not pure address and is a valid instruction, issuing a wake-up instruction to the vehicle. That is, when the recognized text includes a valid instruction, it can be determined that the user has an explicit demand for calling the voice assistant and executing a corresponding function, so as to ensure that the voice assistant is accurately woken up.

[0010] The determining that the previous round of recognized text is a valid instruction comprises: determining that an intent of the previous round of recognized text is within a valid intent list, and a length of the previous round of recognized text is greater than a target length, and determining that the previous round of recognized text points to an explicit operation object and an operation mode. In this way, it can be determined that the instruction output by the user is a valid instruction, so that the voice assistant can be woken up based on this and a specific function operation is performed.

[0011] The determining whether the wake-up word in the current round of recognized text is used as an address comprises: in a case where it is determined that the current round of recognized text includes the wake-up word, determining a matching result of the current round of recognized text and a plurality of preset rules; in a case where the current round of recognized text matches a target rule in the plurality of preset rules, determining whether the wake-up word in the current round of recognized text is used as an address according to the matching result. In this way, by preset rules, the content corresponding to the recognized text is quickly matched, and the wake-up efficiency of the voice assistant is improved.

[0012] Further, the determining whether the wake-up word is used as an address in the current round of recognized text further includes: in a case where the current round of recognized text and any of the plurality of preset rules do not match, determining whether the wake-up word is used as an address in the current round of recognized text according to a position coding feature and a part-of-speech coding feature of the wake-up word in the current round of recognized text. Thus, the position coding feature and the part-of-speech coding feature are combined to quickly and accurately identify whether the text is an address, thereby improving the wake-up efficiency of the voice assistant.

[0013] Further, after the determining whether the wake-up word is used as an address in the current round of recognized text according to the position coding feature and the part-of-speech coding feature of the wake-up word in the current round of recognized text, the method further includes: in a case where the wake-up word is determined not to be used as an address in the current round of recognized text according to the position coding feature and the part-of-speech coding feature, determining a confusion degree of the current round of recognized text; in a case where the confusion degree is greater than a target confusion degree, determining that the wake-up word is not used as an address in the current round of recognized text; in a case where the confusion degree is not greater than the target confusion degree, determining a keyword weight in the current round of recognized text; wherein the keyword weight is used to represent a proportion of a target word in word segmentation of the current round of recognized text; and in a case where the keyword weight is greater than a target weight, determining that the wake-up word is used as an address in the current round of recognized text.

[0014] Thus, the recognized text can be analyzed in detail based on multiple layers of judgment conditions, and the recognized text can be accurately determined to be an address, thereby ensuring that the voice assistant is woken up in a timely and accurate manner.

[0015] The application further provides a voice interaction device, which includes: an identification module configured to perform voice identification on a current round of user voice request received in a vehicle cabin to obtain a current round of recognized text; an acquisition module configured to, in a case where the current round of recognized text includes an address and is a pure address, acquire a previous round of recognized text corresponding to the current round of recognized text; wherein the previous round of recognized text is a recognized text corresponding to a previous round of user voice request, and the previous round of recognized text does not include an address; and a sending module configured to, in a case where the previous round of recognized text is a valid instruction, send a wake-up instruction to the vehicle, so that the vehicle wakes up a voice assistant and a user for voice interaction according to the wake-up instruction.

[0016] Therefore, by setting the voice interaction device, when a user requests to wake up a voice assistant in a vehicle, even if the user first speaks the corresponding execution action and then calls the wake-up word of the voice assistant, the voice interaction method in the application can determine that the user has a request to wake up the voice assistant based on the voice request content, and accurately wake up the voice assistant. In other words, even if the user outputs the voice request in a very regular order of first performing the action and then calling the voice, the voice assistant can be effectively woken up in time, thereby meeting the wake-up needs of the user in different types of application scenarios and improving the user experience.

[0017] The application further provides a server, which comprises a memory and a processor, and the memory stores a computer program, and the computer program is executed by the processor to realize the method. The wake-up needs of the voice assistant are met, and the voice assistant is accurately woken up.

[0018] The application further provides a nonvolatile computer readable storage medium of a computer program, which realizes the method when the computer program is executed by one or more processors. The wake-up needs of the voice assistant are met, and the voice assistant is accurately woken up.

[0019] Additional aspects and advantages of the application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and / or additional aspects and advantages of the application will become apparent and be readily appreciated from the following description, including the accompanying drawings, wherein:

[0021] Figure 1 is a main framework diagram of the voice interaction method according to the application;

[0022] Figure 2 is a schematic diagram of a classification model of the voice interaction method according to the application;

[0023] Figure 3 is a schematic diagram of the main steps of the voice interaction method according to the application;

[0024] Figure 4 is a schematic diagram of the application of the voice interaction method according to the application;

[0025] Figure 5 is one of the flow diagrams of the voice interaction method according to the application;

[0026] Figure 6 is another flow diagram of the voice interaction method according to the application;

[0027] Figure 7is a flowchart of a voice interaction method according to the present invention;

[0028] Figure 8 is a flowchart of a voice interaction method according to the present invention;

[0029] Figure 9 is a flowchart of a voice interaction method according to the present invention;

[0030] Figure 10 is an application diagram of a voice interaction method according to the present invention;

[0031] Figure 11 is a flowchart of a voice interaction method according to the present invention;

[0032] Figure 12 is a flowchart of a voice interaction method according to the present invention;

[0033] Figure 13 is a flowchart of a voice interaction method according to the present invention;

[0034] Figure 14 is a flowchart of a voice interaction method according to the present invention;

[0035] Figure 15 is a flowchart of a voice interaction method according to the present invention;

[0036] Figure 16 is an application diagram of a voice interaction method according to the present invention;

[0037] Figure 17 is a flowchart of a voice interaction method according to the present invention;

[0038] Figure 18 is a flowchart of a voice interaction method according to the present invention;

[0039] Figure 19 is a flowchart of a voice interaction method according to the present invention;

[0040] Figure 20 is a flowchart of a voice interaction method according to the present invention;

[0041] Figure 21 is a flowchart of a voice interaction method according to the present invention;

[0042] Figure 22 is a diagram of a voice interaction device according to the present invention;

[0043] Figure 23 is a diagram of a server according to the present invention. DETAILED DESCRIPTION

[0044] The voice interaction method of the present application is described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The voice interaction method described below by reference to the accompanying drawings is exemplary and is only used to explain the present application and cannot be understood as a limitation of the present application.

[0045] A voice interaction method is proposed in the present application, as shown in Figures 1-4 When the occupant in the vehicle cabin makes voice output: after the user voice request is converted into recognized text, the text and its accompanying address are received into the rule engine, and preliminary screening is performed in the rule engine, the prefix of the user query and part of the action words can be adapted in the rule engine, and the rule engine is constructed in the form of a dictionary tree to reduce the disturbance to the model, wherein the input content in the rule engine is "text query (the result of ASR output + address)", and the output content is the query result hit by the rule engine for final judgment by decision fusion; further, the text query not hit by the rule engine is input into the classification model, and judgment and analysis are performed in the classification model, and the output content is the probability of the category corresponding to a single query, and the result is used for final judgment by decision fusion; further, the input query and the classification model result are input into the decision engine, and the result is accepted according to different situations, and the output is whether the decision query is said to the user, whereby after the user outputs the voice request, the voice content is converted into recognized text, which can be accurately recognized through the rule engine, the classification model and the decision engine to ensure that the voice assistant is accurately awakened and to avoid false awakening.

[0046] Please refer to Figure 4 for a description of the execution of the voice interaction method of the present application.

[0047] A voice interaction method is proposed in the present application, as shown in Figure 5 comprising the following steps:

[0048] A100, performing voice recognition on the received user voice request in the vehicle cabin to obtain recognized text. Specifically, a microphone or other voice acquisition component is provided in the vehicle cabin to acquire the user voice request in the vehicle cabin, wherein the user voice request can come from various sound zones in the vehicle cabin, including but not limited to the sound zones of the main driver and the co-driver, the sound zone of the co-driver seat, the sound zone of the left side of the second row behind the main driver, the sound zone of the middle of the second row behind the main driver, the sound zone of the right side of the second row behind the main driver, the sound zone of the left side of the third row behind the main driver, and the sound zone of the right side of the third row behind the main driver. Further, the content of the voice is recognized after the user voice request is received to recognize the text corresponding to the voice request.

[0049] Specifically, the voice recognition mode can adopt an ASR technology (a technology of converting human voice into text) so that when a user at any position in the vehicle cabin makes voice output, the recognized text can be obtained according to the voice request of the user.

[0050] A200, in a case where it is determined that the recognized text includes the wake-up word and the wake-up word is used as a form of address in the recognized text, issuing a wake-up instruction to the vehicle so as to wake up the voice assistant and the user for voice interaction according to the wake-up instruction.

[0051] The wake-up word can be a system default noun or a manually set noun, and the wake-up word refers to the voice assistant. When the user requests the voice assistant to perform a corresponding function, the user can call the wake-up word to the voice assistant to wake up the voice assistant, for example, the wake-up word is "Xiao P", "Xiao Y" or other nouns. In this embodiment, the wake-up word is "Xiao P".

[0052] Further, when the wake-up word appears in the voice request of the user and the wake-up word is used as a form of address, the wake-up word can refer to the voice assistant, for example, the wake-up word is "Xiao P", and the voice request instruction of the user is "Xiao P, please help me open the window". In this voice request, "Xiao P" is used as a form of address and refers to the voice assistant, so that the vehicle can wake up the voice assistant according to the wake-up instruction, so that the voice assistant can communicate with the user, and then the voice assistant can perform a function or adjust according to the voice instruction of the user.

[0053] It should be noted that in actual use, the existing voice assistant can have the same word as the wake-up word in the user's communication in different contexts, that is, when the wake-up word is not used as a form of address, the wake-up word in the voice request of the user does not refer to the voice assistant, for example, the user in the vehicle cabin communicates daily, such as "Let's go to Xiao P Park" or "Open Xiao P's dressing". Although the voice request includes the wake-up word, the wake-up word is not used as a form of address, that is, the user who makes the voice request does not want to wake up the voice assistant. Therefore, by setting the voice interaction method in this embodiment, in a case where it is determined that the recognized text includes the wake-up word and the wake-up word is used as a form of address in the recognized text, the voice assistant is woken up, so that the voice assistant can be woken up by mistake.

[0054] Therefore, when the user communicates in the vehicle cabin, even if the same or similar instruction description as the wake-up word appears in the communication content, the voice interaction method in this embodiment can accurately identify whether the voice request of the user is a form of address to the voice assistant, so as to avoid false wake-up, so that the user can communicate naturally and smoothly, that is, during the communication process, there is no need to deliberately avoid the content related to the wake-up word, and the user experience is improved.

[0055] As Figure 6 shown, determining that the recognized text includes a wake-up word includes:

[0056] A210, extracting candidate words in the recognized text by sequence labeling. It should be noted that this step extracts words that may be called from the text, which is not a simple text matching, because ASR may misrecognize the call as other words with similar pronunciation, for example, misrecognize "Xiao P" as "Xiao Pi", "Xiao Pi", "Xiao Pi", etc. The sequence labeling method is used in the present application to extract candidate call words, which effectively solves the problem of ASR error.

[0057] A220, determining the wake-up word in the candidate word by pinyin comparison. That is, the candidate call words extracted in the first step in A220 may not be effective call words, so further screening is needed, which can be based on the pinyin method to screen "Xiao P" or words with the pronunciation "xiao pi".

[0058] Further, after determining the wake-up word, further disambiguation is performed on the wake-up word, that is, the call word in the recognized text may be a call or a mention or a proper noun, so it needs to be distinguished, that is, disambiguation. Specifically, disambiguation can be based on the following steps.

[0059] Further, as Figure 7 shown, determining whether the wake-up word in the recognized text is used as a call, a preliminary selection can be made based on a rule engine, and an example of executing the rule engine includes A230, A231 includes:

[0060] A230, determining the matching result of the recognized text and a plurality of preset rules. In other words, when designing voice interaction, a plurality of preset rules can be prepared, and the recognized text can be selected and distinguished in the preset rules. Among them, the preset rule can be the vehicle itself by default, or the user can set it flexibly according to his own needs, such as selecting some specific sentences as the matching content of the preset rule. Specifically, it can be set in combination with the actual functional requirements in the vehicle, such as "Xiao P, please turn on the air conditioner", "Xiao P, please turn on the main driver's window", or other types or ways of preset rules.

[0061] A231, in a case where the recognized text matches a target rule in the plurality of preset rules, determining whether the wake-up word is used as a salutation in the recognized text according to a matching result. That is, after recognizing the text, the content in the recognized text is analyzed to determine whether the wake-up word in the recognized text is a user's address to the voice assistant, that is, to determine whether the user is calling the voice assistant. Among the preset rules, the user's prefix and part of the action word can be adapted, and the preset rules are constructed in the form of a dictionary tree to reduce the disturbance to the model. Specifically, input: text query (ASR output result + salutation), output: query result hit by the preset rule for decision fusion to make a final judgment.

[0062] Therefore, in a case where the recognized text matches a target rule in the plurality of preset rules, the voice request instruction can be output, and then the voice assistant is woken up, so that the voice assistant performs a corresponding function operation based on the voice request. Specifically, as "small P's voice is big", "small P's volume is small", which conforms to the sentence pattern of "small P's XX", we determine it as a negative example. For example, "small P laboratory", "small P dressing" and business scenarios related words are not easy to distinguish, and introducing these data will greatly disturb the training model, and this part is processed by rules. If the rule is hit, the corresponding result is adopted.

[0063] Therefore, after executing the rule engine, the classification module is screened in combination with step A232.

[0064] Further, determining whether the wake-up word is used as a salutation in the recognized text also includes:

[0065] A232, in a case where the recognized text does not match any of the plurality of preset rules, determining whether the wake-up word is used as a salutation in the recognized text according to a position coding feature and a part-of-speech coding feature of the wake-up word in the recognized text.

[0066] That is, in the specific execution process, after the recognized text is identified and matched by the plurality of preset rules, the corresponding target rule in the plurality of preset rules does not match the content of the recognized text, in other words, the recognized text is not within the user's preset rules. At this time, the content of the recognized text can be further analyzed by the position coding feature and the part-of-speech coding feature of the wake-up word in the recognized text to determine whether the wake-up word is a salutation. Specifically, the position coding feature and the part-of-speech coding feature can be set and implemented based on the POS Embedding function (part-of-speech embedding) and the Salutation Embedding function (salutation embedding), respectively.

[0067] At this time, the wake-up word can be judged according to the position coding feature and the part-of-speech coding feature of the wake-up word. The position coding feature is the position of the wake-up word in the recognized text, that is, the recognized text can be matched according to the conventional sentence order to determine, for example, whether the wake-up word appears at the beginning, middle or end of a sentence, and the wake-up word at the beginning and end of the text is more likely to be an address, such as “P please turn up the volume a little” or “please turn up the volume a little, P”, in this type of recognized text, “P” is at the beginning and end of the sentence, and both are addresses, or “enter P’s studio”, in this text, “P” is in the middle of the sentence and is not an address.

[0068] In addition, the part-of-speech coding feature can identify the content of the recognized text to determine whether it is a combination of an address and an action, specifically, for example, if the user’s voice request is “P please turn up the volume a little” or “P please turn down the volume a little”, it can be identified from the text that “P” is an address and is connected with an action; while in the user’s voice request “P laboratory” or “P playground”, “P” is the name of the laboratory and playground, not an address, and the subsequent word group is not an explicit action, based on this, it can also be determined whether the content of the user’s voice instruction is a wake-up voice assistant. In addition, the wake-up word in the recognized text can also be judged based on the Masked Language Model (Masked Language Model), that is, the masked coding feature can also be selected, which can be used to determine the address.

[0069] In addition, the part-of-speech coding feature can identify the content of the recognized text to determine whether it is a combination of an address and an action, specifically, for example, if the user’s voice request is “P please turn up the volume a little” or “P please turn down the volume a little”, it can be identified from the text that “P” is an address and is connected with an action; while in the user’s voice request “P laboratory” or “P playground”, “P” is the name of the laboratory and playground, not an address, and the subsequent word group is not an explicit action, based on this, it can also be determined whether the content of the user’s voice instruction is a wake-up voice assistant. In addition, the wake-up word in the recognized text can also be judged based on the Masked Language Model (Masked Language Model), that is, the masked coding feature can also be selected, which can be used to determine the address.

[0070] Alternatively, the recognized text is input into the encoding layer of the classification model to obtain the masked coding feature of the recognized text, the position coding feature of the wake-up word and the part-of-speech coding feature of the recognized text output by the encoding layer, wherein the encoding layer includes a masked speech model, an address coding function for coding the position relationship of the wake-up word and a part-of-speech coding function for coding the verbs and prepositions in the recognized text. The masked coding feature, the position coding feature and the part-of-speech coding feature are input into the embedding layer of the classification model to obtain the embedding matrix output by the embedding layer; the embedding matrix is input into the self-attention layer of the classification model to obtain the similarity matrix output by the self-attention layer; the similarity matrix is input into the logistic regression layer of the classification model to obtain the classification recognition confidence output by the logistic regression layer; and based on the classification recognition confidence, it is determined whether the wake-up word in the recognized text is an address.

[0071] Specifically, when performing the confidence determination by the classification model, a threshold value can be set first, if the probability (score) given by the model result is greater than the set threshold value, the discrimination result is output, and if it is lower than the threshold value, the next step is entered. For example, "P opens the window" will give a high confidence positive example score, and "P you are great" will give a high confidence negative example score.

[0072] Wherein, after performing the classification model based on step A232, the decision fusion engine can be further combined with steps A233, A234, A235, A236.

[0073] Further, as shown in Figure 8 After determining whether the wake-up word in the recognized text is used as a form of address according to the position encoding feature and the part-of-speech encoding feature of the wake-up word in the recognized text, the method further comprises:

[0074] A233, in the case where the wake-up word in the recognized text is not used as a form of address according to the position encoding feature and the part-of-speech encoding feature, the confusion degree of the recognized text is determined. In other words, after the classification model determines that the wake-up word is not used as a form of address, the confusion degree of the recognized text is determined to further determine whether the wake-up word is a form of address through the confusion degree of the recognized text, so as to improve the accuracy of the output result.

[0075] Wherein, the confusion degree of the recognized text can be determined based on a language model, such as a pre-prepared 3-gram and 4-gram language model, and the recognized text is input into the language model to calculate the confusion degree.

[0076] A234, in the case where the confusion degree is greater than the target confusion degree, it is determined that the wake-up word in the recognized text is not used as a form of address. For example, "P's voice is a little louder" and "P's volume is a little lower" conform to the sentence pattern of "P's XX", and we determine it as a negative example. For example, "P's laboratory" and "P's dressing" are related to business scenarios, and it is not easy to distinguish them, and introducing these data will greatly disturb the training model, and this part is processed by rules, and the corresponding result is adopted when the rule is hit.

[0077] Further, as shown in Figure 9 After determining the confusion degree of the recognized text, the method further comprises:

[0078] A235, in the case where the confusion degree is not greater than the target confusion degree, the key word weight in the recognized text is determined; wherein the key word weight is used to represent the proportion of the target word in the word segmentation of the recognized text. Therefore, when the confusion degree cannot determine whether the wake-up word is a form of address, the key word weight of the recognized text can be determined, and the key word weight is used to further determine whether the wake-up word in the recognized text is a form of address.

[0079] A236、in the case that the keyword weight is greater than the target weight, it is determined that the wake-up word is a salutation in the recognized text. In this way, the wake-up word can be determined to be a salutation based on the keyword weight, so as to wake up the voice assistant.

[0080] In this way, in specific implementation, the confusion degree of the wake-up word in the recognized text can be calculated in advance based on the 3-gram and 4-gram language models, if the weighted confusion degree is greater than the target confusion degree, a negative example result is output, otherwise the keyword weight is combined to make a decision.

[0081] Further, the keyword weight in the recognized text is determined, including:

[0082] A2361, determining the keyword of the intersection of the subwords of the recognized text and the core word dictionary; dividing the number of keywords by the sum of the number of subwords and the number of keywords to obtain a preliminary proportion; and normalizing the preliminary proportion to obtain the keyword weight.

[0083] That is, when determining the keyword weight, a core word dictionary is prepared in advance, the proportion of the number of keywords in the sum of the number of subwords and the number of keywords is calculated to obtain a preliminary proportion of the keyword, and the preliminary proportion is normalized to obtain the weight of the keyword. For example, the core word dictionary includes AA, AB, CD, DE, …; the query subwords are AA, AB, and DD; the intersection number X of the core word dictionary and the subwords in the query is 2 (AA AB).

[0084] In this way, the weight of the keyword can be obtained, so that in the case that the keyword weight is greater than the target weight, it is determined that the wake-up word is a salutation in the recognized text, and the voice assistant is woken up. For example, in the case of "Xiaop helps me navigate to Huolushan Forest Park", because the model training data is difficult to cover all open slots, this module plays a role in improving the recall of the wake-up word of the open slot.

[0085] In this way, the rules strongly coupled with the business are combined in the decision fusion stage, so as to avoid too much introduction of difficult-to-judge data into the model and reduce the decision pressure of the model. In the case of low confidence of the model, the language model and the core word dictionary prepared in advance are combined to increase the overall recall.

[0086] After determining that the recognized text includes the wake-up word, the method further includes:

[0087] In a case where it is determined that the wake-up word is not used as an address in the recognized text, no instruction for indicating the wake-up voice assistant is issued. That is, when the wake-up word in the voice request of the user is not used as an address to wake up the voice assistant based on the above method, the voice assistant does not need to be woken up, and thus no instruction for indicating the wake-up voice assistant is issued.

[0088] The present application proposes another voice interaction method, please refer to Figure 10 , and the voice interaction method has many similarities with the above voice interaction method and the specific execution steps, and the difference lies in the wake-up free step executed when the voice instruction is pure address, and in the determination of the effective instruction in the NLU (natural language flow), further improving the accuracy of wake-up free.

[0089] As shown in Figure 11 , A110, voice recognition is performed on the received current round of user voice request in the vehicle cabin to obtain the current round of recognized text. Specifically, a microphone or other voice acquisition component is provided in the vehicle cabin to acquire the user voice request in the vehicle cabin, wherein the user voice request can come from the main driving position, the co-pilot position or the rear seat. Further, the content of the voice is recognized after receiving the user voice request to identify the corresponding current round of recognized text of the voice request.

[0090] Specifically, the voice recognition method can use ASR technology (technology for converting human voice into text) to obtain the recognized text according to the user voice request when the user at any position in the vehicle cabin outputs voice.

[0091] A240, in a case where it is determined that the current round of recognized text includes an address and is pure address, the previous round of recognized text is obtained; wherein the previous round of recognized text is the recognized text corresponding to the previous round of user voice request, and the previous round of recognized text does not include an address.

[0092] That is, after the user issues a voice instruction, the address in the recognized text is pure address, and the pure address in the recognized text only has the address without other content, such as only "Xiao P", which is identified as pure address, and after determining this condition, the previous round of recognized text can be obtained. That is, after obtaining the recognized text in each round, the recognized text can be temporarily stored, so that after further execution of the recognized text, the recognized text in the previous round can be obtained according to the identification result, and the recognized text in the previous round is judged to determine whether the recognized text in the previous round does not include an address.

[0093] A250: If the previously recognized text is determined to be a valid command, a wake-up command is issued to the vehicle, so that the vehicle, in response to the wake-up command, wakes up the voice assistant and engages in voice interaction with the user. Specifically, if it is further determined that the previously recognized text does not include an address and the recognized text is a valid command, it can be concluded that the user's voice command is intended to obtain assistance from the voice assistant.

[0094] For example, after the recognition text obtained in this round is "Xiao P", "Xiao P" is a pure address, and the recognition text of the previous round is obtained, the recognition text of the previous round is "Please open the main driving window", the previous round of recognition text does not include an address, and there is a clear execution action in the previous round of recognition text, that is, the recognizable text is a valid instruction. At this time, the voice assistant can be woken up to execute the opening of the main driving window according to the request instruction corresponding to the previous round of recognition text to meet the user's needs.

[0095] Therefore, when a user requests to wake up the voice assistant in the car, even if the user first speaks the corresponding action and then uses the voice assistant's wake-up word, the voice interaction method of the present invention can still determine the user's request to wake up the voice assistant based on the content of the voice request, thus accurately waking up the voice assistant. In other words, even if the user inputs the voice request in the unconventional order of first performing the action and then using the word, the voice assistant can still be woken up in a timely and effective manner, thus meeting the user's wake-up needs in different types of application scenarios and improving the user experience.

[0096] like Figure 12 As shown, before sending the wake-up command to the vehicle, the method further includes:

[0097] A251: Determine whether the time interval between the current round of text recognition and the previous round of text recognition is less than the target duration. That is, before issuing a wake-up command to the vehicle, the time interval between the two rounds of text recognition can be analyzed. If the time interval between the two rounds of text recognition is less than the target duration, it can be determined that the two rounds of text recognition were issued by the user based on the same requirement.

[0098] In specific implementation, the target duration can be set to 3 seconds. That is, if the current round of recognition text obtained within 3 seconds after the previous round of recognition text contains a salutation, it is determined that there is a dependency relationship between the two rounds of recognition text. In other words, when the time interval between the two rounds of recognition text is too long, the two may not necessarily be related. For example, if the user's recognition text in the previous round is "Please open the main driver's window", and the recognition text in the current round is a pure salutation "Xiao P", but the time interval between the two exceeds the target duration, such as 1 hour, there is obviously no connection between the two, and there is no need to wake up the voice assistant.

[0099] Therefore, by setting the interval between two rounds of text recognition, it is also possible to avoid false wake-ups of the voice assistant and improve wake-up accuracy.

[0100] As Figure 13 shown, after obtaining the current round of recognized text, the method further comprises:

[0101] A252, in the case of determining that the current round of recognized text does not include the address, caching the current round of recognized text, the time information of the current round of recognized text and the sound area of the current round of recognized text corresponding to the user voice request of the current round.

[0102] Wherein, the recognized text does not include the address, that is, the recognized text can be used as an effective instruction for the next round of recognized text, so the recognized text can be temporarily cached. Correspondingly, the time information and the corresponding sound area of the recognized text without the address are identified and cached to enable the recognized text to correspond to the time information and the sound area for the next round of application.

[0103] It should be noted that the sound area refers to the area of each seat in the vehicle cabin, such as the main driver sound area, the co-driver sound area, and the rear seat sound area.

[0104] Specifically, if the recognized text is "please open the co-driver window", and the corresponding time is 9:30:15, and the corresponding sound area is the co-driver.

[0105] As Figure 14 shown, the method further comprises:

[0106] A253, in the case of determining that the time interval between the current round of recognized text and the previous round of recognized text is not less than the target duration, or the previous round of recognized text is an invalid instruction, no instruction is issued to the vehicle to indicate the wake-up voice assistant.

[0107] In other words, when the time interval between the current round of recognized text and the previous round of recognized text is not less than the target duration and the previous round of recognized text is an invalid instruction, the wake-up instruction is not issued to the vehicle. Specifically, as in the specific implementation, the target duration is set to 3s, and the current round of recognized text obtained within 3s after the previous round of recognized text is issued contains the address, but the voice instruction in the previous round of recognized text is "do you listen to music?", Obviously, the recognized text is an invalid instruction, and the voice assistant does not need to be woken up. Or, as in the specific implementation, the target duration is set to 3s, and the voice instruction in the previous round of recognized text is "please open the co-driver window", obviously, the recognized text is an effective instruction, but the time interval between the two rounds of recognized text is greater than 3s, such as even 15 minutes, that is, the voice assistant does not need to be woken up.

[0108] Therefore, by setting the interval duration of the two rounds of recognized text and combining the content of the previous round of recognized text, the voice assistant can be further prevented from being woken up by mistake, and the wake-up accuracy can be improved.

[0109] It should be noted that the time interval of two rounds of recognized text refers to the end time of the previous round of recognized text and the start time of the current round of recognized text. For example, the time period of the previous round of recognized text is t1-t2, and the time period of the current round of recognized text is t3-t4, that is, the interval duration between the two rounds of recognized text is t3-t2.

[0110] After determining that the current round of recognized text includes a salutation, the method further includes:

[0111] In the case where it is determined that the current round of recognized text is not a pure salutation and is a valid instruction, an awakening instruction is issued to the vehicle. That is, when it is determined that the current round of recognized text is a pure salutation, the instruction can be analyzed based on the rule engine, classification module and decision fusion in the above-mentioned one of the voice interaction methods. The recognized text includes a salutation and also includes a valid instruction required by the customer, so as to clearly indicate that the customer has a requirement for calling the voice assistant and executing a corresponding function, thereby ensuring that the voice assistant is accurately awakened.

[0112] As shown in Figure 15 determining that the previous round of recognized text is a valid instruction includes:

[0113] A254、determining that the intent of the previous round of recognized text is within the valid intent list, and the length of the previous round of recognized text is greater than the target length, and determining that the previous round of recognized text points to a specific operation object and operation mode. That is, the conditions for determining that the previous round of recognized text is a valid instruction include that the length of the previous round of recognized text is greater than the target length, and the specific operation object and operation mode.

[0114] The length of the previous round of recognized text is greater than the target length, such as the length of the previous round of recognized text being greater than 5 characters, or other target lengths can also be set. The operation object can be a functional component in the vehicle, such as a window, air conditioner, light, audio, etc., and the operation mode can include opening, closing, raising, lowering, etc.

[0115] For example, in the recognized text "please open the air conditioner", the length of the recognized text is greater than the target length, and the operation object is the air conditioner and the operation mode is opening. Or, for example, in the recognized text "please raise the audio volume", the length of the recognized text is greater than the target length, and the operation object is the audio and the operation mode is to raise the volume.

[0116] Therefore, it can be determined that the instruction output by the user is a valid instruction, so that the voice assistant can be awakened based thereon.

[0117] In the following, some specific scenarios are combined:

[0118] As in scenario one: the previous round of audio area 1 buffer instruction is "open the main driver air conditioner", the system does not recognize the call, so the instruction is not executed, and is directly stored in the cache; Then the current round of audio area 1 receives the instruction "Xiao P", the system recognizes that the instruction is a pure call, so the previous round instruction cached in the same audio area is obtained; The time interval between the previous round instruction and the current round is 6.5-4.0=2.5s, which is less than 3s, which is the effective cache instruction, so the post-wakeup judgment process is entered; After the NLU stage, the system finally judges that the instruction is a valid instruction, so the post-wakeup is triggered, the system executes the instruction, and the cache of audio area 1 is cleared.

[0119] As in scenario two: the previous round of audio area 1 receives the instruction "open the main driver air conditioner", the system does not recognize the call, so the instruction is not executed, and is directly stored in the cache; Then the current round of audio area 1 receives the instruction "Xiao P", the system recognizes that the instruction is a pure call, so the previous round instruction cached in the same audio area is obtained; The time interval between the previous round instruction and the current round is 7.5-4.0=3.5s, which is greater than 3s, which is not the effective cache instruction, so it does not enter the post-wakeup judgment process, and the system cache of audio area 1 is cleared.

[0120] As in scenario three: the previous round of audio area 1 receives the instruction "Xiao P", the system does not recognize the call, so the instruction is not executed, and is directly stored in the cache; Then the current round of audio area 1 receives the instruction "Xiao P", the system recognizes that the instruction is a pure call, so the previous round instruction cached in the same audio area is obtained; The time interval between the previous round instruction and the current round is 6.5-4.0=3.5s, which is less than 3s, which is the effective cache instruction; After NLU, the system finally judges that the previous round instruction is an invalid instruction, so the post-wakeup is not triggered, and the system cache of audio area 1 is cleared.

[0121] As in scenario four: the previous round of audio area 1 receives the instruction "Xiao P", the system does not recognize the call, so the instruction is not executed, and is directly stored in the cache; Then the current round of audio area 2 receives the instruction "Xiao P", the system recognizes that the instruction is a pure call, so the previous round instruction cached in the same audio area is obtained, and there is no instruction cached in the current audio area 2, so the process is ended. (The system achieves independent audio area cache, which does not affect the cache of audio area 1.)

[0122] As in scenario five: the previous round of audio zone 1 receives the instruction "open the driver's window", the system does not recognize the call, so the instruction is not executed and is directly stored in the cache; then the current round of audio zone 2 receives the instruction "little P", the system recognizes that the instruction is a pure call, so it obtains the previous round of instructions from the cache of the same audio zone, and the current audio zone 2 has no cached instructions, so the process ends; the next round of audio zone 1 receives the instruction "little P", the system recognizes that the instruction is a pure call, so it obtains the previous round of instructions from the cache of the same audio zone, and the time interval between the previous round of instructions and the current round is 6.5-4.0=2.5s, which is less than 3s, so it is a valid cached instruction, and then the post-wakeup judgment process is entered; after NLU, the system finally determines that the instruction is a valid instruction, so the post-wakeup is triggered, the system executes the instruction, and the audio zone 1 cache is cleared.

[0123] As in scenario six: the previous round of audio zone 1 receives the instruction "open the driver's window", the system does not recognize the call, so the instruction is not executed and is directly stored in the cache; then the current round of audio zone 1 receives the instruction "little P", the system recognizes that the instruction is a pure call, so it obtains the previous round of instructions from the cache of the same audio zone; the time interval between the previous round of instructions and the current round is 6.5-4.0=2.5s, which is less than 3s, so it is a valid cached instruction, and then the post-wakeup judgment process is entered; after NLU, the system finally determines that the instruction is a valid instruction, so the post-wakeup is triggered, the system executes the instruction, and the audio zone 1 cache is cleared; the next round of audio zone 1 receives the instruction "little P close all windows", the system recognizes the call, and enters the call wakeup judgment process. After NLU, the system finally determines that the instruction is valid, triggers the call wakeup to execute the instruction, and clears the audio zone 1 cache.

[0124] Determining whether the wake-up word in the current round of recognized text is used as a call, comprising:

[0125] In the case where it is determined that the current round of recognized text includes the wake-up word, determining the matching result of the current round of recognized text and a plurality of preset rules; in the case where the current round of recognized text matches a target rule in the plurality of preset rules, determining whether the wake-up word in the current round of recognized text is used as a call according to the matching result.

[0126] That is, in determining whether the wake-up word in the current round of recognized text is used as a call, A230 and A231 in the above-mentioned rule engine can be combined, and specifically, A230 determines the matching result of the recognized text and a plurality of preset rules. In other words, when designing voice interaction, a plurality of preset rules can be prepared in advance, and the recognized text can be selected and screened in the preset rules. Among them, the preset rule can be the default one carried by the vehicle itself, or it can be flexibly set by the user according to his own needs, such as selecting some specific sentences as the matching content of the preset rule. Specifically, it can be set in combination with the actual functional requirements in the vehicle, such as "little P, please open the air conditioner" or "little P, please open the driver's window", or it can be other types or ways of preset rules.

[0127] In a case where the recognized text matches a target rule in the plurality of preset rules, it is determined according to a matching result whether the wake-up word is a salutation in the recognized text. That is, after the recognized text, the content in the recognized text is analyzed to determine whether the wake-up word is a salutation of the voice assistant by the user, that is, whether the user is calling the voice assistant. Among them, in the preset rules, the prefix of the user and part of the action word can be adapted, and the preset rules are constructed by the way of dictionary tree to reduce the disturbance to the model. Specifically, input: text query (ASR output result + salutation), output: query result hit by preset rules for decision fusion for final judgment.

[0128] Thus, in a case where the recognized text matches a target rule in the plurality of preset rules, the voice request instruction can be output, and then the voice assistant is woken up, so that the voice assistant performs corresponding function operation based on the voice request. Specifically, such as "small P's voice is big", "small P's volume is small", which conforms to the sentence pattern of "small P's XX", we judge it as a negative example. For example, "small P laboratory", "small P change dress" and business scene related words are not easy to distinguish, and introducing these data will greatly disturb the training model, and this part is processed by rules. If the rule is hit, the corresponding result is adopted.

[0129] In addition, determining whether the wake-up word is a salutation in the recognized text in this round also includes:

[0130] In a case where the recognized text in this round does not match any preset rule in the plurality of preset rules, it is determined according to the position coding feature and the part-of-speech coding feature of the wake-up word in the recognized text in this round whether the wake-up word is a salutation in the recognized text in this round. The execution of this step can combine the step A232 in the above-mentioned classification model, and the execution ideas of the two are the same. That is, in the specific execution process, after the recognized text is identified and matched by the plurality of preset rules, the corresponding target rule in the plurality of preset rules does not match the content of the recognized text, in other words, the recognized text is not within the user's preset rules, at this time, the content of the recognized text can be further analyzed by the position coding feature and the part-of-speech coding feature of the wake-up word in the recognized text to determine whether the wake-up word is a salutation. Specifically, the position coding feature and the part-of-speech coding feature can be set and realized based on POS Embedding (function) and Salutation Embedding (salutation embedding), respectively.

[0131] At this time, the wake-up word can be judged according to the position coding feature and the part-of-speech coding feature of the wake-up word. The position coding feature is the position of the wake-up word in the recognized text, that is, the recognized text can be matched according to the conventional sentence order to determine, for example, whether the wake-up word appears at the beginning, middle or end of a sentence, and the wake-up word at the beginning and end of the text is more likely to be an address, such as “P please turn up the volume a little” or “please turn up the volume a little, P”, in this type of recognized text, “P” is at the beginning and end of the sentence, and both are addresses, or “enter P’s studio”, in this text, “P” is in the middle of the sentence and is not an address.

[0132] In addition, the part-of-speech coding feature can identify the content of the recognized text to determine whether it is a combination of an address and an action, specifically, for example, if the user’s voice request is “P please turn up the volume a little” or “P please turn down the volume a little”, it can be identified from the text that “P” is an address and is followed by an action; while in the user’s voice request “P laboratory” or “P playground”, “P” is the name of the laboratory and playground, not an address, and the subsequent word group is not an explicit action, based on this, it can also be determined whether the content of the user’s voice instruction is a wake-up voice assistant. In addition, the wake-up word in the recognized text can also be judged based on the Masked Language Model (Masked Language Model), that is, the masked coding feature can also be selected, which can be used to determine the address.

[0133] In this way, the recognized text can be encoded to obtain the masked coding feature, the position coding feature and the part-of-speech coding feature, the masked coding feature, the position coding feature and the part-of-speech coding feature can be fused to obtain an embedding matrix, the embedding matrix can be converted by self-attention to obtain a similarity matrix, the similarity matrix can be subjected to logistic regression to obtain a classification recognition confidence, and in the case that the classification recognition confidence is greater than a target confidence, it is determined that the wake-up word in the recognized text is an address.

[0134] Alternatively, the recognized text can be input into the encoding layer of the classification model to obtain the masked coding feature of the recognized text, the position coding feature of the wake-up word and the part-of-speech coding feature of the recognized text output by the encoding layer, wherein the encoding layer includes a masked speech model, an address coding function for coding the position relationship of the wake-up word and a part-of-speech coding function for coding the verbs and prepositions in the recognized text. The masked coding feature, the position coding feature and the part-of-speech coding feature are input into the embedding layer of the classification model to obtain an embedding matrix output by the embedding layer; the embedding matrix is input into the self-attention layer of the classification model to obtain a similarity matrix output by the self-attention layer; the similarity matrix is input into the logistic regression layer of the classification model to obtain a classification recognition confidence output by the logistic regression layer; and based on the classification recognition confidence, it is determined whether the wake-up word in the recognized text is an address.

[0135] Specifically, when performing the confidence determination by the classification model, a threshold value can be set first, if the probability (score) given by the model result is greater than the set threshold value, the discrimination result is output, and if it is lower than the threshold value, the next step is entered. For example, "P opens the window" will give a high confidence positive example score, and "P you are great" will give a high confidence negative example score.

[0136] Further, after determining whether the wake-up word in the current round of recognized text is used as a form of address according to the position coding feature and the part-of-speech coding feature of the wake-up word in the current round of recognized text, the method further comprises:

[0137] Wherein, in the case of determining that the wake-up word in the current round of recognized text is not used as a form of address according to the position coding feature and the part-of-speech coding feature, the confusion degree of the recognized text is determined. Specifically, after the classification model determines that the wake-up word is not used as a form of address, the confusion degree of the recognized text is determined to further determine whether the wake-up word is a form of address through the confusion degree of the recognized text, so as to improve the accuracy of the output result. Wherein, the confusion degree of the recognized text can be determined based on a language model, such as a pre-prepared 3-gram and 4-gram language model, and then the recognized text is input into the language model to calculate the confusion degree. This step can refer to the above step A233.

[0138] In the case where the confusion degree is greater than the target confusion degree, it is determined that the wake-up word in the current round of recognized text is not used as a form of address. For example, "P's voice is a little louder" and "P's voice is a little lower" conform to the sentence pattern of "P's XX", and we determine them as negative examples. For example, "P's laboratory" and "P's dressing" are related to business scenarios, and it is not easy to distinguish them. Introducing these data will greatly disturb the training model, and this part is processed by rules. If the rule is hit, the corresponding result is adopted. This step can refer to the above step A234.

[0139] In the case where the confusion degree is not greater than the target confusion degree, the key word weight in the current round of recognized text is determined; wherein the key word weight is used to represent the proportion of the target word in the word segmentation of the current round of recognized text. Therefore, when the confusion degree cannot be used to determine whether the wake-up word is a form of address, the key word weight of the recognized text can be determined, and the key word weight can be used to further determine whether the wake-up word in the recognized text is a form of address. This step can refer to the above step A235.

[0140] In the case where the key word weight is greater than the target weight, it is determined that the wake-up word in the current round of recognized text is used as a form of address. Therefore, the key word weight can be used to determine that the wake-up word is a form of address, and the voice assistant can be awakened. This step can refer to the above step A236.

[0141] Therefore, in specific implementation, the confusion degree of the wake-up word in the recognized text can be calculated in advance based on the 3-gram and 4-gram language models, if the calculated weighted confusion degree is greater than the target confusion degree, a negative example result is output, otherwise the key word weight is combined for judgment.

[0142] Further, referring to Figure 16 The present application also proposes another voice interaction method. The voice interaction method has many similarities with the voice interaction method proposed above and the specific implementation steps, and the difference lies in the step of determining that the current recognized text matches the preset text in the white list to further improve the accuracy of the wake-up step. As shown in Figure 17 The voice interaction method comprises:

[0143] A110, performing voice recognition on the received current user voice request in the vehicle cabin to obtain the current recognized text. Specifically, a microphone or other voice acquisition component is arranged in the vehicle cabin to acquire the user voice request in the vehicle cabin, wherein the user voice request can come from the main driving position, the co-pilot position or the rear seat. Further, the content of the voice is recognized after receiving the user voice request to recognize the current recognized text corresponding to the voice request.

[0144] Specifically, the voice recognition method can adopt ASR technology (technology for converting human voice into text) so that when the user in the vehicle cabin outputs voice at any position, the recognized text can be obtained according to the user voice request.

[0145] A260, in the case that the current recognized text is determined to be reliable, the current recognized text is determined not to include the address, the current recognized text is determined to match the preset text in the white list, and the current recognized text is determined to be a valid instruction, the wake-up instruction is issued to the vehicle so as to wake up the voice assistant and the user for voice interaction according to the wake-up instruction.

[0146] The preset text in the white list is preset in advance, such as being set by default in the vehicle system or being set subjectively by the user according to his own needs, and the preset text is a text that can be used to execute the corresponding function, that is, when the recognized text corresponds to the preset text, the voice assistant can be woken up, thereby simplifying the wake-up step.

[0147] Thus, after obtaining the current round of recognized text, the credibility of the current round of recognized text is determined, and after determining the credibility, it is further determined whether the current round of recognized text includes a salutation, and after not including the salutation, the current round of recognized text is matched with the preset text in the whitelist, and after the content in the current round of recognized text and the preset text in the whitelist correspond effectively, the voice assistant is woken up and the vehicle is controlled to execute the corresponding function in the recognized text. Thus, after the recognized text is matched with the preset text, the voice assistant can be woken up, simplifying the steps and methods of waking up, and providing a more natural and more convenient voice interaction method.

[0148] As shown in Figure 18 , the preset text in the whitelist is determined by the following method;

[0149] A2611, a plurality of historical recognized texts in a target time period are obtained. That is, the preset text in the whitelist can be collected and mined from a plurality of historical recognized texts, such as in the current round of execution of whether to wake up determination, the recognized text in the previous round and the recognized text in the previous multiple rounds can be collected and used as the preset text in the current round. In this way, in the construction of the whitelist, a large amount of time is not needed to think about the input of the preset text, and at the same time, new recognized texts are constantly updated or cached, which is beneficial to guarantee the applicability of the preset text and more in line with the user's usage requirements.

[0150] A2612, from the plurality of historical recognized texts, the historical recognized text with a length greater than a target length and a frequency greater than a target frequency or a frequency ranking before a target ranking is selected as a candidate text. That is, in the construction of the whitelist, the length of the historical recognized text can be screened, and when the length of the historical recognized text is greater than the target length, the historical recognized text meets one of the conditions of being a preset text. When the occurrence frequency of the historical recognized text is greater than the target frequency or the frequency ranking is before the target ranking, the historical recognized text is also used as a preset text. Specifically, the usage frequency of the plurality of historical recognized texts is counted to form a list SL, and is sorted according to the frequency, and can be selectively retained, such as retaining the top 30% instructions, obtaining an instruction list SF, and the historical recognized text in the instruction list SF column is used as a candidate text in the whitelist.

[0151] A2613, the candidate text meeting the target sentence pattern is used as the preset text. That is, after obtaining the instruction list SF, each instruction in the instruction list SF is judged to meet the target sentence pattern, so as to ensure that the regular instructions meet the requirements, and the final instruction list, that is, the final whitelist, is obtained.

[0152] As shown in Figure 19 , the candidate text meeting the target sentence pattern is used as the preset text, including:

[0153] A2614: If the candidate text includes control action words and specific control objects arranged in sequence, the candidate text is used as the preset text. That is, the condition for the candidate text to be used as the preset text is that the control action words and specific control objects are arranged in sequence.

[0154] Control action words can include "open," "close," "raise," and "lower," and specific control objects can be functional components within the vehicle, such as windows, air conditioning, lights, and audio. Examples of such texts, arranged in this order, include "Please turn on the air conditioning," with "open" as the control action word and air conditioning as the specific control object, or "Please turn up the audio volume," with "raise" as the control action word and audio as the specific control object. These candidate texts can be used as pre-set texts and clearly align with the user's intent to wake up the voice assistant.

[0155] A2615. Or, when the candidate text includes a control action word, a degree modifier, and a specific control object arranged in sequence, the candidate text is used as a preset text.

[0156] Similarly, control action words may include open, close, increase, decrease, etc., and degree modifiers may affect the degree of operation of the control action words, such as half. The specific control objects may be functional components in the car, such as the main driving window, air conditioning, headlights, etc. The corresponding sentence pattern is <control action><degree modifier><specific control point>.

[0157] Among them, it should be noted that degree modifiers should be standardized and clear. For example, the meaning of half is clearer and more specific than that of a point or a part. For the control object, it can include multiple descriptions, such as car window, main driving window, etc., among which the main driving window is the specific control point relative to the car window.

[0158] For example, in some examples, the instruction is "open the car window", the matching result is "open-car-window", and the corresponding instruction sentence pattern is <control action><general control point>, which does not conform to regular instructions and is not used as preset text; or, the instruction is "open the car window a little", the matching result is "open-a little-car-window", and the corresponding instruction sentence pattern is <control action> a little <general control point>, which does not conform to regular instructions and is not used as preset text; or, the instruction is "open half of the main-driving window", the matching result is "open-half-main-driving window", and the corresponding instruction sentence pattern is <control action><degree modifier><specific control point>, which conforms to regular instructions and can be used as preset text.

[0159] like Figure 20 As shown, after obtaining the current round of recognition text, the method further includes:

[0160] A262、In a case where it is determined that the current round of recognized text is reliable, and it is determined that the current round of recognized text includes an address, and is not a pure address, and it is determined that the current round of recognized text is a valid instruction, an awakening instruction is issued to the vehicle.

[0161] In other words, in a case where it is determined that the recognized text is reliable text, and the current round of recognized text is a non-pure address and is a valid instruction, which meets the awakening requirement, an awakening instruction can be issued to control the voice assistant to wake up, and then control the corresponding components of the vehicle according to the awakening instruction.

[0162] wherein the reliability can be determined based on an ASR reliability determination model. Specifically, a plurality of features are constructed based on the decoding confidence of the ASR module and the TopN results of the decoding:

[0163] 1. The length of the final result: 4;

[0164] 2. The number of TopN results:

[0165] 3. The similarity between TopN results, calculate the average Jaccard similarity of TopN results, the Jaccard similarity calculation formula: sim(X, Y) = ‖X∩Y‖ / ‖X∪Y‖.

[0166] For example, the similarity between "open the window" and "open the window": 3 / 4 = 0.75; the similarity between "open the window" and "open the window": 3 / 4; the average similarity: (0.75 + 0.75) / 2 = 0.75.

[0167] and, when the ASR reliability determination model is constructed, the standard deviation of the length of Top N: std([4, 3, 3]) = 0.47; - whether the final result contains letters: no (0); - whether Top N contains an empty string: no (0); - the length of the same prefix of Top N results: 0; - the length of the same suffix of topN results: 1.

[0168] The above features are summarized: [4, 3, 0.75, 0.47, 0, 0, 0, 1]; feature expansion: add second-order features (multiply each other); final feature summary: [4, 3, 0.75, 0.47, 0, 0, 0, 1, 16, 12, 3.0, 1.88, 0, 0, 0, 4, 12, 9, 2.25, 1.41, 0, 0, 0, 3, 3.0, 2.25, 0.56, 0.35, 0.0, 0.0, 0.0, 0.75, 1.88, 1.41, 0.35, 0.22, 0.0, 0.0, 0.0, 0.47, 0, 0, 0.0, 0.0, 0, 0, 0, 0, 0, 0, 0.0, 0.0, 0, 0, 0, 0, 0, 0, 0.0, 0.0, 0, 0, 0, 0, 4, 3, 0.75, 0.47, 0, 0, 0, 1].

[0169] In the feature selection, an L1 regularization based method can be used: standardize the features, train an LR (Logistic Regression) model with L1 regularization using all the features, get the coefficients of each feature, and remove the features with coefficients close to 0. Alternatively, a random forest based method can be used: train a random forest model using all the features, the model will give an evaluation of the importance of each feature, and remove the features below a certain threshold.

[0170] Then, the model is trained using the final feature set to train an LR model.

[0171] As shown in FIG. 13, after obtaining the current round of recognized text, the method further includes: Figure 21

[0172] A2631. In a case where it is determined that the current round of recognized text is reliable, and the current round of recognized text includes an address, and is a pure address, obtaining a previous round of recognized text; wherein the previous round of recognized text is a recognized text corresponding to a previous round of user voice request, and the previous round of recognized text is unreliable, or the previous round of recognized text is reliable and does not include an address and does not match a preset text.

[0173] A2632. In a case where it is determined that a time interval between the current round of recognized text and the previous round of recognized text is less than a target time length, and the previous round of recognized text is a valid instruction, issuing a wake-up instruction to the vehicle.

[0174] In other words, in the process of determining the voice request, in a case where it is determined that the current round of recognized text is reliable and includes an address, especially a pure address, the previous round of recognized text can be determined, even if the previous round of recognized text does not include an address and does not match a preset text, if the previous round of recognized text is a valid instruction, the voice assistant can be woken up, and the voice assistant can control the components related to the vehicle with the instruction in the previous round of recognized text.

[0175] Specifically, the execution step can refer to steps A251-A254 in the voice interaction method described above.

[0176] In the execution step A263, it is determined that the previous round of recognized text is a valid instruction.

[0177] Specifically, it is determined that the intent of the previous round of recognized text is within a valid intent list, and the length of the previous round of recognized text is greater than a target length, and it is determined that the previous round of recognized text points to a clear operation object and operation method. That is, the conditions for determining that the previous round of recognized text is a valid instruction include that the length of the previous round of recognized text is greater than a target length, and a clear operation object and operation method.

[0178] ​Wherein, the length of the previous round of recognized text is greater than the target length, such as the length of the previous round of recognized text is greater than 5 characters, or can also be set to other target lengths. Wherein, the operation object can be a functional component in the vehicle, such as a window, air conditioner, light, sound, etc., and the operation mode can include opening, closing, raising, lowering, etc.

[0179] For example, in the recognized text "please open the air conditioner, little P", the length of the recognized text is greater than the target length, and the operation object is the air conditioner, and the operation mode is to open; or, for example, in the recognized text "please raise the sound volume, little P", the length of the recognized text is greater than the target length, and the operation object is the sound, and the operation mode is to raise the volume.

[0180] Thus, it can be determined that the instruction output by the user is a valid instruction, so that the voice assistant can be awakened based on this.

[0181] Determine whether the wake-up word is used as a salutation in the current round of recognized text, including:

[0182] In the case where it is determined that the current round of recognized text includes the wake-up word, determine the matching result of the current round of recognized text and a plurality of preset rules; in other words, when designing voice interaction, a plurality of preset rules can be formulated in advance, and the recognized text can be selected and screened in the preset rules. Wherein, the preset rule can be the default vehicle itself, or the user can set it flexibly according to his own needs, such as selecting some specific sentences as the matching content of the preset rule, specifically, it can be set in combination with the actual functional requirements in the vehicle, such as "little P, please open the air conditioner", "little P, please open the main driver's window", or other types or ways of preset rules. Wherein, A230 and A231 in the rule engine described above can be referred to.

[0183] In the case where the current round of recognized text matches a target rule in the plurality of preset rules, determine whether the wake-up word is used as a salutation in the current round of recognized text according to the matching result. That is, after recognizing the text, the content in the recognized text is analyzed to determine whether the wake-up word in the recognized text is the user's address to the voice assistant, that is, to determine whether the user is summoning the voice assistant. Wherein, in the preset rule, the user's prefix and part of the action word can be adapted, and the preset rule can be constructed in the form of a dictionary tree to reduce the disturbance to the model, specifically, input: text query (ASR output result + salutation), output: query result hit by the preset rule for decision fusion to make a final judgment.

[0184] Thus, in the case of identifying that the text matches a target rule in the plurality of preset rules, the voice request instruction can be output, and then the voice assistant is woken up to enable the voice assistant to perform a corresponding function operation based on the voice request. Specifically, as "P is a little louder", "P is a little quieter", and the sentence pattern of "P's XX", we determine it as a negative example. For example, "P lab", "P change", and the words related to the business scene are not easy to distinguish, and the introduction of these data will disturb the training model. This part is processed by rules. If the rule is hit, the corresponding result is adopted.

[0185] Specifically, reference can be made to A230 and A231 in the voice interaction method described above, and determination is made based on the rule engine.

[0186] Determining whether the wake-up word is used as a salutation in the current round of recognized text also includes:

[0187] In the case where the current round of recognized text does not match any of the plurality of preset rules, the position coding feature and the part-of-speech coding feature of the wake-up word in the current round of recognized text are used to determine whether the wake-up word is used as a salutation in the current round of recognized text. That is, in the specific implementation process, after the recognized text is identified and matched by the plurality of preset rules, the corresponding target rule in the plurality of preset rules does not match the content of the recognized text, in other words, the recognized text is not within the user's preset rules. At this time, the content of the recognized text can be further analyzed by the position coding feature and the part-of-speech coding feature of the wake-up word in the recognized text to determine whether the wake-up word is a salutation. Specifically, the position coding feature and the part-of-speech coding feature can be set and implemented based on POS Embedding (function) and Salutation Embedding (salutation embedding), respectively.

[0188] Specifically, reference can be made to step A232 in the voice interaction method described above, and implementation is made based on the classification model.

[0189] After determining whether the wake-up word is used as a salutation in the current round of recognized text based on the position coding feature and the part-of-speech coding feature of the wake-up word in the current round of recognized text, the method further includes:

[0190] When it is determined based on the position coding features and part-of-speech coding features that the wake-up word is not a term of address in the current round of recognition text, the confusion level of the current round of recognition text is determined; specifically, after the classification model identifies that the wake-up word is not a term of address, the confusion level of the recognized text is determined to further determine whether the wake-up word is a term of address based on the confusion level of the recognized text, thereby improving the accuracy of the output result. The confusion level of the recognized text can be determined based on a language model, such as by pre-preparing 3-gram and 4-gram language models, inputting the recognized text into the language model, and then calculating the confusion level. This step can refer to step A233 above.

[0191] If the confusion level exceeds the target confusion level, the wake-up word is determined not to be a salutation in the current recognition round. For example, "Xiao P's voice is louder" or "Xiao P's volume is lower" fit the sentence structure of "Xiao P's XX" and are therefore considered negative examples. Similarly, "Xiao P's laboratory" and "Xiao P's outfit change" are business-related terms that are difficult to distinguish, and introducing this data significantly disrupts the training model. This is handled by rules, and if a rule is matched, the corresponding result is accepted. This step can be referred to as step A234 above.

[0192] When the confusion degree is not greater than the target confusion degree, the keyword weight in the recognized text of this round is determined; wherein the keyword weight is used to characterize the proportion of the target word in the word segmentation of the recognized text of this round; thus, when it is impossible to determine whether the wake-up word is an address word based on the confusion degree, the keyword weight of the recognized text can be determined, and based on the keyword weight, it can be further determined whether the wake-up word is used as an address word in the recognized text. This step can refer to the above-mentioned step A235.

[0193] If the keyword weight is greater than the target weight, the wake-up word is determined to be a salutation in the current round of recognition text. Thus, the wake-up word can be determined as a salutation based on the keyword weight to wake up the voice assistant. This step can refer to step A236 above.

[0194] Therefore, in the specific implementation, the confusion degree of the wake-up word in the recognition text can be calculated in advance based on the 3-gram and 4-gram language models. If the calculated weighted confusion degree is greater than the target confusion degree, a negative example result is output; otherwise, the keyword weight is combined to make a judgment.

[0195] The present invention also proposes a voice interaction device 1.

[0196] like Figure 22 As shown, the voice interaction device 1 includes: a recognition module 10, an acquisition module 11 and a sending module 12.

[0197] The recognition module 10 is configured to perform voice recognition on the received current round of user voice request in the vehicle cabin to obtain a current round of recognition text.

[0198] The acquisition module 11 is configured to acquire a previous round of recognition text corresponding to the current round of recognition text in a case where it is determined that the current round of recognition text includes an address and is a pure address.

[0199] The sending module 12 is configured to send a wake-up instruction to the vehicle in a case where it is determined that the previous round of recognition text is a valid instruction, so that the vehicle wakes up the voice assistant and the user for voice interaction according to the wake-up instruction.

[0200] Therefore, by using the voice interaction device 1, when the user requests to wake up the voice assistant in the vehicle, even if the user first performs an action and then calls the wake-up word of the voice assistant, the voice interaction method in the present application can determine that the user has a request to wake up the voice assistant based on the voice request content, and accurately wake up the voice assistant. In other words, even if the user outputs the voice request in a non-conventional order of performing an action first and then calling an address, the voice assistant can be effectively woken up in time, thereby meeting the wake-up needs of the user in different types of application scenarios and improving the user experience.

[0201] The voice interaction device 1 further includes a first determination module configured to determine that a time interval between the current round of recognition text and the previous round of recognition text is less than a target time length before sending the wake-up instruction to the vehicle.

[0202] Further, the voice interaction device 1 further includes a cache module configured to:

[0203] In a case where it is determined that the current round of recognition text does not include an address, cache the current round of recognition text, time information of the current round of recognition text, and an audio region of the current round of user voice request corresponding to the current round of recognition text.

[0204] Further, the sending module 12 is further configured to not send an instruction for instructing to wake up the voice assistant to the vehicle in a case where it is determined that the time interval between the current round of recognition text and the previous round of recognition text is not less than the target time length, or the previous round of recognition text is an invalid instruction.

[0205] In a case where it is determined that the current round of recognition text includes an address, the sending module 12 is further configured to:

[0206] In a case where it is determined that the current round of recognition text is not a pure address and is a valid instruction, send a wake-up instruction to the vehicle.

[0207] In a case where the text in the previous round is determined to be a valid instruction, the first determining module is further configured to:

[0208] determine that the intent of the text in the previous round is within the list of valid intents, and that the length of the text in the previous round is greater than a target length, and that the text in the previous round points to an explicit operation object and operation manner.

[0209] In a case where the wake-up word is determined to be a form of address in the text in the current round, the first determining module is further configured to:

[0210] In a case where the text in the current round includes the wake-up word, determine a matching result of the text in the current round and a plurality of preset rules.

[0211] In a case where the text in the current round matches a target rule in the plurality of preset rules, determine whether the wake-up word is a form of address in the text in the current round according to the matching result.

[0212] Determine whether the wake-up word is a form of address in the text in the current round, and the first determining module is further configured to:

[0213] In a case where the text in the current round does not match any of the plurality of preset rules, determine whether the wake-up word is a form of address in the text in the current round according to a position coding feature and a part-of-speech coding feature of the wake-up word in the text in the current round.

[0214] After determining whether the wake-up word is a form of address in the text in the current round according to the position coding feature and the part-of-speech coding feature of the wake-up word in the text in the current round, the first determining module is further configured to:

[0215] In a case where it is determined that the wake-up word is not a form of address in the text in the current round according to the position coding feature and the part-of-speech coding feature, determine a confusion degree of the text in the current round.

[0216] In a case where the confusion degree is greater than a target confusion degree, determine that the wake-up word is not a form of address in the text in the current round.

[0217] In a case where the confusion degree is not greater than the target confusion degree, determine a keyword weight in the text in the current round; wherein the keyword weight is used to represent a proportion of a target word in a word segmentation of the text in the current round.

[0218] In a case where the keyword weight is greater than a target weight, determine that the wake-up word is a form of address in the text in the current round.

[0219] As Figure 23As shown, the present invention also proposes a server 2, including a memory 21 and a processor 22, wherein the memory 21 stores a computer program, and when the computer program is executed by the processor, each process of the above-mentioned voice interaction method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0220] The present application also provides a non-volatile computer-readable storage medium for a computer program. When the computer program is executed by one or more processors 22, the various processes of the above-mentioned voice interaction method are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0221] Those skilled in the art will appreciate that all or part of the processes in the above-described methods can be implemented by instructing related software using a computer program. The program can be stored in a non-volatile computer-readable storage medium, and when executed, the program can include the processes in the above-described methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or the like.

[0222] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with an embodiment or example is included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0223] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable actions for implementing a specific logical function or process step, and the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the present application belongs.

[0224] Although the present application has been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that, include: Perform speech recognition on the current round of user voice requests received from the vehicle cockpit to obtain the current round of recognition text; If it is determined that the current round of recognition text includes a wake-up word, and it is determined that the wake-up word is a pure address in the current round of recognition text, obtaining the previous round of recognition text; wherein the previous round of recognition text is the recognition text corresponding to the previous round of user voice request, and the previous round of recognition text does not include an address; If it is determined that the previously recognized text is a valid command, issuing a wake-up command to the vehicle so that the vehicle wakes up the voice assistant according to the wake-up command to perform voice interaction with the user; After obtaining the recognized text in this round, the method further includes: When it is determined that the recognized text in this round does not include a salutation word, the recognized text in this round, the time information of the recognized text in this round, and the sound zone of the user's voice request in this round corresponding to the recognized text in this round are cached, so that the recognized text in this round and the corresponding time information and sound zone are cached for the next round of application.

2. The voice interaction method of claim 1, wherein, Before issuing the wake-up instruction to the vehicle, the method further includes: Determine whether the time interval between the current round of recognized text and the previous round of recognized text is less than a target time length.

3. The voice interaction method of claim 1, wherein, If it is determined that the time interval between the current round of recognition text and the previous round of recognition text is not less than the target duration, or the previous round of recognition text is an invalid instruction, no instruction for waking up the voice assistant is issued to the vehicle.

4. The voice interaction method of claim 1, wherein, After determining that the current round of recognition text includes a wake-up word, and the wake-up word is a pure address in the current round of recognition text, the method further includes: When it is determined that the recognized text in this round is not a pure address but a valid instruction, a wake-up instruction is sent to the vehicle.

5. The voice interaction method according to any one of claims 1-4, characterized in that, Determining that the previous round of recognized text is a valid instruction includes: Determine that the intention of the previous round of recognition text is in the valid intention list, and the length of the previous round of recognition text is greater than the target length, and determine that the previous round of recognition text points to a clear operation object and operation method.

6. The voice interaction method of any one of claims 1-4, wherein, Determining whether the wake-up word is used as a salutation in the current round of recognition text includes: When it is determined that the current round of recognition text includes a wake-up word, determining a matching result between the current round of recognition text and a plurality of preset rules; In the case where the recognition text in this round matches the target rule in the multiple preset rules, it is determined whether the wake-up word is used as a salutation in the recognition text in this round according to the matching result.

7. The voice interaction method of claim 6, wherein, The determining whether the wake-up word is used as a salutation in the current round of recognition text further includes: When the recognition text of this round does not match any of the multiple preset rules, it is determined whether the wake-up word is used as a term of address in the recognition text of this round based on the position coding features and part-of-speech coding features of the wake-up word in the recognition text of this round.

8. The voice interaction method of claim 7, wherein, After determining whether the wake-up word is used as a salutation in the current round of recognition text based on the position coding feature and the part-of-speech coding feature of the wake-up word in the current round of recognition text, the method further includes: In a case where it is determined, according to the position coding feature and the word type coding feature, that the wake-up word is not used as an address in the current round of recognized text, determine a confusion degree of the current round of recognized text; In a case where the confusion degree is greater than a target confusion degree, determine that the wake-up word is not used as an address in the current round of recognized text; In a case where the confusion degree is not greater than the target confusion degree, determine a key word weight in the current round of recognized text; wherein the key word weight is used to represent a proportion of a target word in word segmentation of the current round of recognized text; In a case where the key word weight is greater than a target weight, determine that the wake-up word is used as an address in the current round of recognized text.

9. A voice interactive apparatus, characterized by comprising: The device for implementing the method of any one of claims 1-8 comprises: an identification module configured to perform speech recognition on a received current round of user voice request of a vehicle cabin to obtain current round of recognized text; an acquisition module configured to, in a case where it is determined that the current round of recognized text includes an address and is a pure address, acquire a previous round of recognized text corresponding to the current round of recognized text; wherein the previous round of recognized text is a recognized text corresponding to a previous round of user voice request, and the previous round of recognized text does not include an address; a sending module configured to, in a case where it is determined that the previous round of recognized text is a valid instruction, send a wake-up instruction to a vehicle so as to wake up a voice assistant and a user for voice interaction according to the wake-up instruction.

10. A server, characterized by The server comprises a memory and a processor, and the memory stores a computer program, which is executed by the processor to implement the method of any one of claims 1-8.

11. A non-transitory computer readable storage medium of a computer program, characterized in that, When the computer program is executed by one or more processors, the method of any one of claims 1-8 is implemented.

Images