Vehicle man-machine voice interaction method and system and vehicle

By combining rule-based and dictionary methods with large models, we optimize semantic understanding and multi-turn dialogue strategies, solving the problems of execution timeouts and resource waste in human-computer interaction after the integration of large models, and achieving efficient and accurate semantic understanding and user demand response.

CN118968992APending Publication Date: 2024-11-15CHONGQING CHANGAN TECH CO LTD
View PDF 0 Cites 8 Cited by

Patent Information

Application Number
CN202410959921.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing human-computer voice interaction methods, when integrated into large models, cannot adopt appropriate strategies, leading to execution timeouts and impacting user experience. Furthermore, they cannot share historical semantics at the multi-turn dialogue level, resulting in low accuracy.

Method used

By combining rule-based and dictionary-based methods with large models, we can perform intent classification, strong and weak intent judgment, and context awareness through pre-training and fine-tuning of language network models, optimize distribution strategies, and assist in multi-turn dialogue understanding through a shared history mechanism.

Benefits of technology

It improves the efficiency and accuracy of semantic understanding, ensures the proper execution of simple intents, reduces resource consumption, shortens processing time, and enhances the accuracy and user experience of multi-turn dialogues.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968992A_ABST
    Figure CN118968992A_ABST
Patent Text Reader

Abstract

The invention discloses a man-machine voice interaction method and system for a vehicle and the vehicle, and the method comprises the steps: obtaining audio data of a user, and carrying out the text conversion of the audio data, and obtaining text information; performing intention analysis on the text information to obtain an analysis result, performing scheme determination according to the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information; and performing association processing on the semantic information to obtain target semantics, performing instruction execution according to the target semantics to obtain an execution result, obtaining reply text information according to the execution result, and broadcasting target voice information obtained after voice conversion. According to the method, a rule-based result is taken as a lower limit to ensure simple intention implementation, and a result of a large model is taken as an upper limit to ensure the accuracy of semantic understanding; and for multiple rounds of dialogue scenes, user demands can be better understood by associating historical semantics, and more accurate responses can be provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of human-computer voice interaction technology, and in particular to a human-computer voice interaction method and system for a vehicle, a vehicle, and a computer-readable storage medium. Background Art

[0002] With the development of computer technology and artificial intelligence, the demand for semantic understanding is becoming more and more important. The obtained semantics can enable the vehicle to perform corresponding actions, thereby realizing intelligent and humanized human-computer interaction. With the continuous development of deep learning technology, deep learning technology has begun to be applied to the human-computer interaction experience. By integrating large models for semantic understanding and dialogue management, voice interaction between users and vehicles can be realized.

[0003] At present, existing semantic understanding methods include rule-based and dictionary-based methods, statistical-based methods, and deep learning-based methods; rule-based and dictionary-based methods use language rules and dictionaries to parse and understand the semantics of texts, define language rules in a similar form to regular expressions, and understand the meaning of texts by matching vocabulary and grammatical structures; statistical-based methods use large amounts of text data for analysis, learn language patterns and rules, and then infer the semantics of texts; and deep learning-based methods use recurrent neural network models and long short-term memory network models to train large amounts of data to capture semantic information. Existing dialogue management methods include rule-based methods and model-based methods; rule-based methods use predefined rules and patterns to guide dialogues; model-based methods use machine learning algorithms to learn and predict dialogue behaviors.

[0004] However, at the semantic understanding level, existing methods can only interact according to pre-set rules, which makes it difficult to cope with complex, changeable, and unforeseen situations, and the coverage is limited. With the integration of large models, if the two adopt a serial approach, it will cause execution timeouts and affect user experience. If they adopt a parallel approach, it will cause resource waste. Therefore, what strategies should be adopted by existing methods and large models become issues that need to be considered. At the multi-round dialogue level, the rule-based method will update the dialogue history semantics for the next round of dialogue in each round, which is difficult to handle special, complex, and diverse dialogue situations. The large model uses the text of the previous rounds as history and combines it with the current text as input. There is no dedicated independent module to process history, status and other information, which easily leads to the result of the large model not being consistent with the context. The two adopt different dialogue management schemes. For example, the previous round went to the large model to get the result, and the current dialogue went to the rule-based method. The current dialogue cannot use the semantic results obtained by the previous round of the large model, resulting in errors.

[0005] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0006] The main purpose of the present application is to provide a vehicle human-computer voice interaction method, system and vehicle, aiming to solve the problem that with the integration of large models in existing human-computer voice interactions, appropriate strategies cannot be adopted at the semantic understanding level, resulting in execution timeouts, affecting user experience, and causing waste of resources. At the multi-round dialogue level, due to the different dialogue strategies adopted, the corresponding historical semantics cannot be shared, resulting in a low accuracy rate in multi-round dialogues.

[0007] The first aspect of the present application provides a human-computer voice interaction method for a vehicle, comprising the following steps: acquiring audio data of a user, converting the audio data into text to obtain text information; performing intent analysis on the text information to obtain an analysis result, determining a semantic processing scheme for the text information based on the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information based on the target semantic processing scheme to obtain semantic information; performing association processing on the semantic information to obtain target semantics, executing instructions based on the target semantics to obtain an execution result, obtaining a reply text message based on the execution result, and broadcasting the target voice information corresponding to the reply text message.

[0008] According to the above-mentioned technical means, in order to better combine the rule-based and dictionary-based methods with the big model and give full play to the semantic understanding ability of the big model, the embodiment of the present application intelligently matches the text information obtained by processing the user's audio data to classify the content, and also optimizes the distribution strategy by analyzing the corresponding text information; then the rule-based and dictionary-based methods make up for the disadvantages of the big model in some cases, and the big model makes up for the generalization ability and multi-intention ability of the rule-based and dictionary-based methods. The two are combined through corresponding schemes, with the results obtained by the rule-based and dictionary-based methods as the lower limit to ensure the normal implementation of simple intentions, and the results obtained by the big model as the upper limit, which not only solves complex intentions, but also ensures the accuracy and room for development of semantic understanding; and for multi-round dialogue scenarios, it is assisted by a shared history mechanism to obtain more accurate semantic results, so as to better understand the needs of users and provide more accurate responses, giving full play to the multi-round dialogue capabilities of the big model.

[0009] Optionally, in one embodiment of the present application, the method of acquiring the user's audio data and converting the audio data into text to obtain text information further includes: acquiring a pre-training task, pre-training a language network training model according to the pre-training task to obtain a language representation model, wherein the pre-training task includes a masked language prediction task and a following prediction task; acquiring a natural language processing task, fine-tuning the language representation model according to the natural language processing task to obtain a content distribution model.

[0010] According to the above-mentioned technical means, in order to better combine the rule-based and dictionary-based methods with the large model and give full play to the semantic understanding ability of the large model, the embodiment of the present application pre-trains the language network model through the masked language prediction task and the following prediction task, and fine-tunes the pre-trained model through the natural language processing task to obtain a content distribution model based on the bidirectional language representation model, and uses this model to perform subsequent intent classification, judgment of strong and weak intent, first sentence judgment and context perception and other functions, classify the content through corresponding intelligent matching, and analyze the data to optimize the distribution strategy, thereby improving the efficiency and accuracy of semantic understanding.

[0011] Optionally, in one embodiment of the present application, the intent analysis of the text information to obtain the analysis result specifically includes: classifying the intent of the text information based on the content distribution model to obtain the intent type, judging the strength of the intent of the text information to obtain intent strength information, judging the first sentence of the text information to obtain core sentence information, and performing context perception on the text information to obtain context perception information; obtaining the analysis result of the text information according to the intent type, the intent strength information, the core sentence information and the context perception information.

[0012] According to the above-mentioned technical means, the embodiment of the present application uses the obtained content distribution model to perform functions such as intent classification, strong and weak intent judgment, first sentence judgment and context perception on the text information converted from the user's audio data, so as to realize the classification of content through intelligent matching, and then obtain analysis results based on the corresponding intent type, intent strength information, core sentence information and context perception information, so as to optimize the distribution strategy by analyzing the data, so as to improve the efficiency and accuracy of semantic understanding.

[0013] Optionally, in one embodiment of the present application, the judging the strength of the intent of the text information to obtain the intent strength information specifically includes: establishing an intent strength corpus, setting a strength threshold based on the strong intent corpus and the weak intent corpus of the intent strength corpus, and judging the strength of the intent of the text information based on the strength threshold to obtain a judgment result; if the judgment result is that the intent strength value of the text information is greater than or equal to the strength threshold, then the intent strength information is a strong intent; if the judgment result is that the intent strength value of the text information is less than the strength threshold, then the intent strength information is a weak intent.

[0014] According to the above-mentioned technical means, in the embodiment of the present application, when the content distribution model needs to judge the strength of the intent of text information, the strength threshold is set by using the strong intent corpus and the weak intent corpus in the established intent strength corpus, and the strength threshold is compared with the intent strength value of the text information. If the intent strength value of the text information is greater than or equal to the strength threshold, the text information is judged to have a strong intent; if the intent strength value of the text information is less than the strength threshold, the text information is judged to have a weak intent, thereby achieving more accurate semantic understanding and decision making.

[0015] Optionally, in one embodiment of the present application, the semantic processing scheme is determined for the text information according to the analysis result to obtain a target semantic processing scheme, and the text information is semantically understood according to the target semantic processing scheme to obtain semantic information, specifically including: if the analysis result is that the text information is a simple intent, then the target semantic processing scheme is determined to be a rule- and dictionary-based method processing scheme according to the simple intent; the text information is processed in a single-word chain manner according to the rule- and dictionary-based method processing scheme to obtain a single-word chain network, and the single-word chain network is processed in an arc manner to obtain a business grammar network; the single-word chain network and the business grammar network are merged to obtain a target network, the path weight is solved for the target network to obtain a target path, and semantic information is obtained according to the target path.

[0016] According to the above-mentioned technical means, after analyzing that the intention corresponding to the text information is a simple intention, the embodiment of the present application parses the simple intention through a rule-based and dictionary-based method, uses the rule model in the rule-based and dictionary-based method to calculate the similarity of the text information, and has confirmed the corresponding standard text. Then, the standard text is processed using the grammar in the rule-based and dictionary-based method. The grammar represents the structure and constraints of the language by defining various symbols and rules, that is, the corresponding standard text is processed into a single-word chain network, and the single-word chain network is arc-fitted to obtain a business grammar network, and the single-word chain network and the business grammar network are merged, and then the merged network is solved for the path with the largest weight, that is, the target semantics, which can not only solve most of the rule statements, but also save the required resources, thereby efficiently and quickly completing the semantic understanding of the text information.

[0017] Optionally, in one embodiment of the present application, a semantic processing scheme is determined for the text information based on the analysis result to obtain a target semantic processing scheme, and semantic understanding processing is performed on the text information based on the target semantic processing scheme to obtain semantic information, specifically including: if the analysis result is that the text information has a question and answer intention, then determining the target semantic processing scheme to be a knowledge question and answer model processing scheme based on the question and answer intention; and performing question determination on the text information based on the knowledge question and answer model processing scheme to obtain corresponding semantic information.

[0018] According to the above-mentioned technical means, after analyzing that the intent corresponding to the text information is a question-and-answer intent, the embodiment of the present application solves question-and-answer intentions such as encyclopedias, document questions and answers, and standard questions and answers through a knowledge question-and-answer model, thereby efficiently and quickly completing the semantic understanding of the text information.

[0019] Optionally, in one embodiment of the present application, a semantic processing scheme is determined for the text information based on the analysis result to obtain a target semantic processing scheme, and semantic understanding processing is performed on the text information according to the target semantic processing scheme to obtain semantic information, specifically including: if the analysis result is that the text information is an arrangement intention, then according to the arrangement intention, the target semantic processing scheme is determined to be a speech arrangement model processing scheme; the text information is organized and arranged according to the speech arrangement model processing scheme to obtain corresponding semantic information.

[0020] According to the above-mentioned technical means, after analyzing that the intention corresponding to the text information is the arrangement intention, the embodiment of the present application solves the arrangement intention through a voice arrangement model, which not only shortens the execution time but also reduces resource waste, thereby efficiently and quickly completing the semantic understanding of the text information to improve the user experience.

[0021] Optionally, in one embodiment of the present application, a semantic processing scheme is determined for the text information based on the analysis result to obtain a target semantic processing scheme, and semantic understanding processing is performed on the text information according to the target semantic processing scheme to obtain semantic information, specifically including: if the analysis result is that the text information has multiple intentions, then according to the multiple intentions, the target semantic processing scheme is determined to be a multi-intention model processing scheme; the text information is subjected to intent analysis according to the multi-intention model processing scheme to obtain multiple text intents, and all the text intents are combined to obtain corresponding semantic information.

[0022] According to the above-mentioned technical means, after analyzing that the intent corresponding to the text information is multiple intents, the embodiment of the present application solves the multiple intents through a multi-intent model, thereby achieving full-scene multi-intent controllable; by combining a large model to improve the coverage of complex voice scenes, the semantic understanding of the text information can be completed efficiently and quickly.

[0023] Optionally, in one embodiment of the present application, the semantic information is associated with the target semantics, instructions are executed according to the target semantics to obtain execution results, reply text information is obtained according to the execution results, and target voice information corresponding to the reply text information is broadcast, specifically including: obtaining historical conversation semantic results and historical system behaviors, semantically associating the semantic information according to the historical conversation semantic results and the historical system behaviors to obtain target semantics, and obtaining corresponding semantic execution instructions according to the target semantics; executing corresponding operations according to the semantic execution instructions to obtain execution results, generating corresponding reply text information according to the execution results, and confirming a response method according to the reply text information; converting the reply text information into speech to obtain target voice information, and broadcasting the target voice information according to the response method.

[0024] According to the above-mentioned technical means, the embodiment of the present application assists in understanding semantics for multi-round dialogue scenarios through a shared history mechanism, because the final semantic result of the current dialogue not only depends on the text information of the user of the current dialogue, but also depends on the semantic result of the previous round of dialogue and the system behavior after the previous round of execution. Then, the user's semantic information is semantically associated according to the semantic result of the previous round of dialogue and the system behavior after the previous round of execution to obtain the target semantics, and the corresponding operation is performed. The reply text information corresponding to the obtained execution result is converted into target voice information, and the target voice information is broadcast according to the determined response method, so as to better understand the user's needs and provide more accurate responses, giving full play to the capabilities of large-model multi-round dialogues.

[0025] Optionally, in one embodiment of the present application, the corresponding operation is executed according to the semantic execution instruction to obtain an execution result, and then further includes: if an exception occurs in the execution result, corresponding exception information is obtained, an exception analysis is performed on the exception information, a corresponding processing plan is obtained, and the processing plan is called; the exception information is processed according to the processing plan to obtain a processing result, a re-execution instruction is generated according to the processing result, and the corresponding operation is executed again according to the re-execution instruction.

[0026] According to the above-mentioned technical means, when an exception occurs in the execution result, the embodiment of the present application obtains the corresponding exception information in a timely manner, and calls the corresponding processing scheme to process the exception information, and executes the corresponding operation again after the processing is completed, thereby ensuring the stability of the system and improving the reliability of the dialogue.

[0027] The second aspect of the present application provides a human-computer voice interaction system for a vehicle, and the human-computer voice interaction system of the target vehicle includes: a text conversion module, which is used to obtain the user's audio data, convert the audio data into text, and obtain text information; a semantic understanding module, which is used to perform intent analysis on the text information to obtain an analysis result, determine a semantic processing scheme for the text information based on the analysis result to obtain a target semantic processing scheme, and perform semantic understanding processing on the text information based on the target semantic processing scheme to obtain semantic information; a voice reply module, which is used to perform association processing on the semantic information to obtain target semantics, execute instructions according to the target semantics to obtain execution results, obtain reply text information according to the execution result, and broadcast the target voice information corresponding to the reply text information.

[0028] Optionally, in one embodiment of the present application, the human-computer voice interaction system of the target vehicle further includes: a model training unit, used to obtain a pre-training task, and pre-train the language network training model according to the pre-training task to obtain a language representation model, wherein the pre-training task includes a masked language prediction task and a context prediction task; a model fine-tuning unit, used to obtain a natural language processing task, and fine-tune the language representation model according to the natural language processing task to obtain a content distribution model.

[0029] Optionally, in one embodiment of the present application, the semantic understanding module includes: a text information processing unit, which is used to classify the intent of the text information based on the content distribution model to obtain the intent type, judge the strength of the intent of the text information to obtain the intent strength information, judge the first sentence of the text information to obtain the core sentence information, and perform context perception on the text information to obtain context perception information; an information analysis unit, which is used to obtain the analysis result of the text information according to the intent type, the intent strength information, the core sentence information and the context perception information; a first scheme determination unit, which is used to determine the target semantic processing scheme as a rule-based and dictionary-based method processing scheme according to the simple intent if the analysis result shows that the text information is a simple intent; a first scheme processing unit, which is used to perform single-word chain processing on the text information according to the rule-based and dictionary-based method processing scheme to obtain a single-word chain network, and perform arc processing on the single-word chain network to obtain a business grammar network; a path weight solving unit, which is used to merge the single-word chain network and the business grammar network to obtain a target network, and to solve the target semantic processing scheme for the business grammar network. The target network solves the path weight to obtain the target path, and obtains semantic information according to the target path; the second scheme determination unit is used to determine the target semantic processing scheme as a knowledge question and answer model processing scheme according to the question and answer intent if the analysis result is that the text information is a question and answer intent; the second scheme processing unit is used to perform question determination on the text information according to the knowledge question and answer model processing scheme to obtain corresponding semantic information; the third scheme determination unit is used to determine the target semantic processing scheme as a voice arrangement model processing scheme according to the arrangement intent if the analysis result is that the text information is an arrangement intent; the third scheme processing unit is used to organize and arrange the text information according to the voice arrangement model processing scheme to obtain corresponding semantic information; the fourth scheme determination unit is used to determine the target semantic processing scheme as a multi-intention model processing scheme according to the multi-intention if the analysis result is that the text information is multi-intention; the fourth scheme processing unit is used to perform intent analysis on the text information according to the multi-intention model processing scheme to obtain multiple text intents, and combine all the text intents to obtain corresponding semantic information.

[0030] Optionally, in one embodiment of the present application, the text information processing unit also includes: an intention strength judgment subunit, which is used to establish an intention strength corpus, set a strength threshold according to the strong intention corpus and the weak intention corpus of the intention strength corpus, and judge the intention strength of the text information according to the strength threshold to obtain a judgment result; a first judgment subunit, which is used to determine that if the judgment result is that the intention strength value of the text information is greater than or equal to the strength threshold, then the intention strength information is a strong intention; and a second judgment subunit, which is used to determine that if the judgment result is that the intention strength value of the text information is less than the strength threshold, then the intention strength information is a weak intention.

[0031] Optionally, in one embodiment of the present application, the voice reply module includes: a historical semantic association unit, which is used to obtain historical conversation semantic results and historical system behaviors, semantically associate the semantic information according to the historical conversation semantic results and the historical system behaviors, obtain target semantics, and obtain corresponding semantic execution instructions according to the target semantics; a reply text determination unit, which is used to execute corresponding operations according to the semantic execution instructions, obtain execution results, generate corresponding reply text information according to the execution results, and confirm the response method according to the reply text information; a reply text broadcasting unit, which is used to convert the reply text information into voice, obtain target voice information, and broadcast the target voice information according to the response method.

[0032] Optionally, in one embodiment of the present application, the voice reply module also includes: an exception handling unit, which is used to obtain corresponding exception information if an exception occurs in the execution result, perform exception analysis on the exception information, obtain a corresponding processing plan, and call the processing plan; a re-execution unit, which is used to process the exception information according to the processing plan, obtain a processing result, generate a re-execution instruction according to the processing result, and execute the corresponding operation again according to the re-execution instruction.

[0033] A third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a human-machine voice interaction program for the vehicle stored in the memory and executable on the processor, wherein the human-machine voice interaction program for the vehicle, when executed by the processor, implements the steps of the human-machine voice interaction method for the vehicle as described in the above embodiment.

[0034] The fourth aspect of the present application provides a computer-readable storage medium, which stores a human-computer voice interaction program for a vehicle. When the human-computer voice interaction program for the vehicle is executed by a processor, the steps of the human-computer voice interaction method for the vehicle as described in the above embodiment are implemented.

[0035] Beneficial effects of this application:

[0036] (1) In order to better combine the rule-based and dictionary-based methods with the big model and give full play to the semantic understanding ability of the big model, the embodiments of the present application intelligently match the text information obtained by processing the user's audio data to classify the content, and also optimize the distribution strategy by analyzing the corresponding text information; then the rule-based and dictionary-based methods make up for the disadvantages of the big model in some cases, while the big model makes up for the generalization ability and multi-intention ability of the rule-based and dictionary-based methods. The two are combined through corresponding solutions, with the results obtained by the rule-based and dictionary-based methods as the lower limit to ensure the normal implementation of simple intentions, and the results obtained by the big model as the upper limit, which not only solves complex intentions, but also ensures the accuracy and room for semantic understanding; and for multi-round dialogue scenarios, it is assisted by a shared history mechanism to obtain more accurate semantic results, so as to better understand the user's needs and provide more accurate responses, giving full play to the multi-round dialogue capabilities of the big model.

[0037] (2) In order to better combine the rule-based and dictionary-based methods with the large model and give full play to the semantic understanding ability of the large model, the embodiment of the present application pre-trains the language network model through the masked language prediction task and the context prediction task, and fine-tunes the pre-trained model through the natural language processing task to obtain a content distribution model based on the bidirectional language representation model. This model is used to perform subsequent functions such as intent classification, judgment of strong and weak intent, first sentence judgment, and context perception. The content is classified through corresponding intelligent matching, and the data can be analyzed to optimize the distribution strategy, thereby improving the efficiency and accuracy of semantic understanding.

[0038] (3) After analyzing that the intent corresponding to the text information is a simple intent, the embodiment of the present application parses the simple intent through a rule-based and dictionary-based method, and uses the rule model in the rule-based and dictionary-based method to calculate the similarity of the text information, and has confirmed the corresponding standard text. Then, the standard text is processed using the grammar in the rule-based and dictionary-based method. The grammar represents the structure and constraints of the language by defining various symbols and rules, that is, the corresponding standard text is processed into a single-word chain network, and the single-word chain network is arc-fitted to obtain a business grammar network, and the single-word chain network and the business grammar network are merged, and then the path with the largest weight is solved for the merged network, that is, the target semantics. This can not only solve most of the rule statements, but also save the required resources, thereby completing the semantic understanding of the text information efficiently and quickly.

[0039] (4) After analyzing that the intent corresponding to the text information is a question-and-answer intent, the embodiment of the present application uses a knowledge question-and-answer model to solve question-and-answer intents such as encyclopedia, document question-and-answer, and standard question-and-answer. The arrangement intent is solved by a voice arrangement model. For multiple intents, the multiple intents are solved by a multi-intent model, thereby achieving full-scenario multi-intent control. By combining a large model to improve the coverage of complex voice scenarios, not only the execution time is shortened, but also the waste of resources is reduced, thereby efficiently and quickly completing the semantic understanding of the text information to improve the user experience.

[0040] (5) In the embodiment of the present application, for multi-round dialogue scenarios, a shared history mechanism is used to assist in understanding semantics, because the final semantic result of the current dialogue depends not only on the text information of the users of the current dialogue, but also on the semantic result of the previous dialogue and the system behavior after the previous dialogue is completed. Then, the semantic information of the users is semantically associated according to the semantic result of the previous dialogue and the system behavior after the previous dialogue is completed to obtain the target semantics; and corresponding operations are performed to convert the reply text information corresponding to the obtained execution result into target voice information, and the target voice information is broadcast according to the determined response method, so as to better understand the needs of the users and provide more accurate responses, thereby giving full play to the multi-round dialogue capabilities of the large model.

[0041] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 It is a flow chart of a preferred embodiment of the human-machine voice interaction method of the vehicle of the present application;

[0044] Figure 2 This is a schematic diagram of a single-word chain network in a preferred embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of the business grammar network in a preferred embodiment of the present application;

[0046] Figure 4 This is a schematic diagram of the target network in a preferred embodiment of the present application;

[0047] Figure 5It is a schematic diagram of weight transfer of a single-word chain network in a preferred embodiment of the present application;

[0048] Figure 6 It is a schematic diagram of the weight transfer of the business grammar network in a preferred embodiment of the present application;

[0049] Figure 7 Schematic diagram of weight transfer of target network in a preferred embodiment of the present application;

[0050] Figure 8 It is an overall flow chart of a preferred embodiment of the human-machine voice interaction method for the vehicle of the present application;

[0051] Fig. 9 It is a structural schematic diagram of a preferred embodiment of the human-machine voice interaction system of the vehicle of the present application;

[0052] Fig.10 This is a schematic structural diagram of a preferred embodiment of the vehicle of the present application.

[0053] Among them, 10-the human-computer voice interaction system of the vehicle; 100-the text conversion module, 200-the semantic understanding module and 300-the voice response module; 501-the memory, 502-the processor and 503-the communication interface. DETAILED DESCRIPTION

[0054] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0055] The human-machine voice interaction method, system, vehicle and storage medium of the vehicle of the embodiment of the present application are described below with reference to the accompanying drawings. With regard to the existing human-machine voice interaction mentioned in the above background technology, with the integration of large models, it is impossible to adopt a suitable solution at the semantic understanding level, resulting in execution timeout, affecting user experience, and causing waste of resources. At the multi-round dialogue level, due to the different dialogue strategies adopted, the corresponding historical semantics cannot be shared, resulting in a low accuracy rate in multi-round dialogues. The present application provides a human-machine voice interaction method for a vehicle. In this method, in order to better combine the rule-based and dictionary-based methods with the large model and give full play to the semantic understanding ability of the large model, the embodiment of the present application intelligently matches the text information obtained by processing the user's audio data to classify the content, and also analyzes the corresponding text. This information is used to optimize the distribution strategy; then the rule-based and dictionary-based methods make up for the disadvantages of the large model in some cases, while the large model makes up for the lack of generalization and multi-intention capabilities of the rule-based and dictionary-based methods. The two are combined through corresponding solutions, with the results obtained by the rule-based and dictionary-based methods as the lower limit to ensure the normal implementation of simple intentions, and the results obtained by the large model as the upper limit, which not only solves complex intentions, but also ensures the accuracy and space for semantic understanding; and for multi-round dialogue scenarios, it is assisted by a shared history mechanism to obtain more accurate semantic results, so as to better understand user needs and provide more accurate responses, giving full play to the multi-round dialogue capabilities of the large model. In this way, the existing human-computer voice interaction is solved. With the integration of large models, it is unable to adopt appropriate solutions at the semantic understanding level, resulting in execution timeouts, affecting user experience, and causing resource waste. At the multi-round dialogue level, due to different dialogue strategies adopted, the corresponding historical semantics cannot be shared, resulting in a low accuracy rate in multi-round dialogues.

[0056] Specifically, Figure 1 A flowchart of a human-machine voice interaction method for a vehicle provided in an embodiment of the present application.

[0057] like Figure 1 As shown, the human-machine voice interaction method of the vehicle includes the following steps:

[0058] In step S101, the audio data of the user is obtained, and the audio data is converted into text to obtain text information.

[0059] It can be understood that in the embodiment of the present application, when it is detected that the user starts the voice interaction function of the vehicle and sends a piece of audio data with content (for example, opening the main driving window halfway, etc.), the user's audio data is obtained, and the audio data is optimized to obtain target audio data, wherein the optimization processing includes optimization operations such as active noise reduction and echo cancellation to eliminate interference from other invalid audio, thereby improving the purity of the audio data; then, the target audio data is converted into text through ASR (Automatic Speech Recognitio, speech recognition technology) to obtain text information.

[0060] That is to say, after the user starts the voice interaction function of the vehicle, the present application obtains the audio data with content sent by the user in real time, and performs a series of optimization processes such as active noise reduction and echo cancellation on the audio data to eliminate the interference of other invalid audio, thereby improving the purity of the audio data and the accuracy of text conversion.

[0061] In step S102, the text information is subjected to intent analysis to obtain an analysis result, a semantic processing scheme is determined for the text information based on the analysis result to obtain a target semantic processing scheme, and semantic understanding processing is performed on the text information based on the target semantic processing scheme to obtain semantic information.

[0062] It can be understood that in the embodiment of the present application, after obtaining the user's text information, it is necessary to perform intent analysis on the text information. The present application combines the rule-based and dictionary-based methods with the large model, that is, the identified text information is subjected to the content distribution model based on the BERT (Bidirectional Encoder Representation from Transformers) model to perform intent classification, strong and weak intent judgment, first sentence judgment and context perception and other functions, and accurately and quickly judge the context, category, demand and keywords and other details of the text information. The corresponding training process of the content distribution model based on the BERT model goes through two stages, namely the pre-training stage and the fine-tuning stage. For the pre-training stage, the deep representation of the language is learned through the pre-training task, that is, the pre-training task is obtained, and the language network model is pre-trained according to the pre-training task to obtain a language representation model, wherein the pre-training task includes the MLM (Masked Language Model) task and the NSP (Next SentencePrediction, following prediction) task; in the masked language prediction task, some tokens in the input sequence are randomly masked, and the masked tokens are predicted by the corresponding model. During the training process, it is necessary to predict the masked tokens based on the contextual information to learn the semantic representation of the language; and in the following prediction task, it is predicted whether two sentences are continuous in the original text, and the relationship between sentences is learned by training the corresponding model, so as to better understand the coherence of the text in subsequent tasks. For the fine-tuning stage, after the pre-training is completed, fine-tuning is performed by adding task-specific output layers to adapt to different NLP (Natural Language Processing) tasks, that is, a natural language processing task is obtained, and the language representation model is fine-tuned according to the natural language processing task to obtain a content distribution model. The language representation learned in the pre-training stage is used in the fine-tuning process, so that the model can quickly adapt to new tasks and achieve excellent performance.

[0063] That is to say, in order to better combine the rule-based and dictionary-based methods with the large model and give full play to the semantic understanding ability of the large model, the embodiment of the present application pre-trains the language network model through the masked language prediction task and the context prediction task, and fine-tunes the pre-trained model through the natural language processing task to obtain a content distribution model based on the bidirectional language representation model, and uses this model to perform subsequent intent classification, judgment of strong and weak intent, first sentence judgment, and context perception. The content is classified through corresponding intelligent matching, and the data can be analyzed to optimize the distribution strategy, thereby improving the efficiency and accuracy of semantic understanding.

[0064] It can be understood that in the embodiment of the present application, the obtained content distribution model is used to perform intent classification, strong and weak intent judgment, first sentence judgment and context perception on the text information, so as to accurately and quickly judge the content context, category, needs and keywords and other details of the text information, that is, the text information is classified according to the content distribution model to obtain intent types, which include simple intent types, multiple intent types, question and answer intent types and voice arrangement intent types. The purpose of intent classification is to determine whether the content distribution model will use a rule-based and dictionary-based method or a large model to formulate a semantic processing solution. For simple intents, a rule-based and dictionary-based method is used, and for complex intents, a large model is used. Through this semantic processing solution, for simple intents, compared with the parallel method, it can reduce resource consumption, and compared with the serial method, it can reduce the time consumption by about 700ms; for complex intents, compared with the parallel method, it can reduce resource consumption, and compared with the serial method, it can reduce the time consumption by about 200ms. Intent classification can also avoid using rule-based and dictionary-based methods to handle multiple intents; the text information is judged for its strength of intent to obtain intent strength information, and an intent strength corpus is established. A strength threshold is set based on the strong intent corpus and the weak intent corpus of the intent strength corpus, and the text information is judged for its strength of intent based on the strength threshold to obtain a judgment result; if the judgment result is that the intent strength value of the text information is greater than or equal to the strength threshold, then the intent strength information corresponding to the text information is judged to be a strong intent; if the judgment result is that the intent strength value of the text information is less than the strength threshold, then the intent strength information corresponding to the text information is judged to be a weak intent; thereby achieving more accurate semantic understanding and decision making; the text information is judged for the first sentence, generally It is through context information, syntactic structure, vocabulary, and grammar that the logical relationship and semantic coherence between sentences are understood, so as to determine the core sentences in the text information, that is, the core sentence information. For example, opening the car window is an intention of opening, which is mainly used to determine the core action. The first sentence can generally have a certain guiding role in the theme and content of the text; context perception is performed on the text information to obtain context perception information, which is determined by using deep learning to determine the structure of the sentence (subject, predicate, object, etc.), so as to determine the recipient of the action. For example, when opening the car window, the recipient is the car window; and local relevant information is recommended according to the user's location; then the analysis result of the text information is obtained according to the intention type, the intention strength information, the core sentence information and the context perception information.

[0065] That is to say, the embodiment of the present application uses the obtained content distribution model to perform intent classification, strong and weak intent judgment, first sentence judgment, and context perception on the text information converted from the user's audio data, so as to classify the content through intelligent matching, and then obtain analysis results based on the corresponding intent type, intent strength information, core sentence information, and context perception information, thereby optimizing the distribution strategy by analyzing the data, so as to improve the efficiency and accuracy of semantic understanding.

[0066] It can be understood that in the embodiment of the present application, after obtaining the analysis result corresponding to the text information, if the analysis result is that the text information is a simple intent, then the target semantic processing scheme is determined to be a rule-based and dictionary-based method processing scheme according to the simple intent; the text information is semantically understood according to the rule-based and dictionary-based method in the rule-based and dictionary-based method processing scheme to obtain semantic information, and the rule-based and dictionary-based method includes grammar, customized platform and rule model, and grammar refers to the description of natural language with certain rules, similar to regular expressions; the customized platform can provide personalized solutions; the rule model is obtained by training a large amount of data and existing network data generated by the Chatette tool to obtain a corresponding similarity model, and the similarity model is used to select an optimal standard data for natural language understanding, wherein the existing network data refers to the data of user tests in the database, so as to train by using correct text and semantic results; the grammar is based on the ABNF (Augmented Backus-Naur Form) specification, and the structure and constraints of the language are represented by defining various symbols and rules. The grammar matching algorithm is WFST (Weighted Finite-State Transducer, weighted finite state transition machine), the corresponding processing flow is to process the input text information (for example, open the main driving window halfway) by single-word chain processing to obtain S network (Semantic network), that is, single-word chain network, such as Figure 2 As shown; then, the basic grammar (i.e., defining numbers) and business grammar are used to process the S network with arcs to obtain the G network (Grammar network), i.e., the business grammar network, as shown Figure 3 As shown; then, merge the S network with the G network. If a new network is formed, namely the target network, such as Figure 4 As shown; then the target network is weighted and the maximum weight path is obtained, which is the semantic information, specifically, as Figure 5 The weight transfer and Figure 6 The weight transfer of the G network shown, Figure 5 and Figure 6The thick circle in the middle represents the initial state, that is, the circle where the number 0 is located. The double-layer circle represents the terminal state. The edges connecting the circles represent the state jump. a represents the input label, b and c both represent the output label, and the numerical value represents the weight of the transfer. Figure 5 0-a:b / 0.1->1 means that when the initial state is 0 and the input label a is received, it will be converted to state 1 with a probability of 0.1, that is, a weight of 0.1 is transferred, and the output label b is obtained. Figure 5 and Figure 6 The expressions of other weight paths in are similar, and we will not elaborate on them one by one here. After that, the weight transfer diagram obtained after merging is as follows Figure 7 As shown, the maximum weight path of the target network will be solved. In the rule model, cosine similarity is used to measure the similarity of two vectors, that is, the text information and the corresponding standard text are similarly calculated by the cosine similarity calculation formula to obtain a similarity value, and the similarity value is compared with a preset threshold. If the similarity value is greater than the preset threshold, the standard text is determined to be the target text information, that is, the standard data with the closest similarity is taken as the optimal result. For example, the optimal result of opening the window on the main driver's side is to open the main driver's window, where the cosine similarity calculation formula is:

[0067]

[0068] Among them, A·B is the dot product of the vectors corresponding to the text information and the standard text, ‖A‖ and ‖B‖ are the norms of the vectors corresponding to the text information and the standard text. If the value of cosine similarity is closer to 1, it means that the two vectors are more similar. In this application, grammar is preferentially used to process simple intents. The use of grammar can not only solve most regular statements, but also save resources.

[0069] That is to say, after analyzing that the intent corresponding to the text information is a simple intent, the embodiment of the present application parses the simple intent through a rule-based and dictionary method, uses the rule model in the rule-based and dictionary method to calculate the similarity of the text information, and has confirmed the corresponding standard text. Then, the standard text is processed using the grammar in the rule-based and dictionary method. The grammar represents the structure and constraints of the language by defining various symbols and rules, that is, the corresponding standard text is processed into a single-word chain network, and the single-word chain network is arc-fitted to obtain a business grammar network, and the single-word chain network and the business grammar network are merged, and then the merged network is solved for the path with the largest weight, that is, the target semantics, which can not only solve most of the rule statements, but also save the required resources, thereby efficiently and quickly completing the semantic understanding of the text information.

[0070] If the analysis result is that the text information has a question and answer intention, then the target semantic processing scheme is determined to be a knowledge question and answer model processing scheme based on the question and answer intention, and the knowledge question and answer model is used to perform question determination on the text information based on the knowledge question and answer model processing scheme to obtain corresponding semantic information, wherein the knowledge question and answer model is used to solve problems such as encyclopedias, document question and answer, and standard question and answer, for example, who is someone.

[0071] That is to say, after analyzing that the intent corresponding to the text information is a question-and-answer intent, the embodiment of the present application solves question-and-answer intentions such as encyclopedias, document questions and answers, and standard questions and answers through a knowledge question-and-answer model, thereby efficiently and quickly completing the semantic understanding of the text information.

[0072] If the analysis result is that the text information is an arrangement intention, the target semantic processing scheme is determined to be a speech arrangement model processing scheme according to the arrangement intention, and the text information is organized and arranged according to the speech arrangement model of the speech arrangement model processing scheme to obtain corresponding semantic information, for example, turning on the air conditioner when closing the car window;

[0073] That is to say, after analyzing that the intention corresponding to the text information is the editing intention, the embodiment of the present application solves the editing intention through a voice editing model, which not only shortens the execution time but also reduces resource waste, thereby efficiently and quickly completing the semantic understanding of the text information to improve the user experience.

[0074] If the analysis result is that the text information has multiple intentions, the target semantic processing scheme is determined to be a multi-intention model processing scheme based on the multiple intentions, and the text information is analyzed for intent based on the multi-intention model in the multi-intention model processing scheme to obtain multiple text intentions, and all the text intentions are combined to obtain corresponding semantic information, for example, closing the car windows and turning on the air conditioner.

[0075] That is to say, after analyzing that the intent corresponding to the text information is multiple intents, the embodiment of the present application solves the multiple intents through a multi-intent model, thereby achieving full-scene multi-intent controllable; by combining a large model to improve the coverage of complex voice scenes, the semantic understanding of the text information can be completed efficiently and quickly.

[0076] In step S103, the semantic information is associated and processed to obtain target semantics, instructions are executed according to the target semantics to obtain execution results, reply text information is obtained according to the execution results, and target voice information corresponding to the reply text information is broadcast.

[0077] It can be understood that in the embodiments of the present application, after obtaining the semantic information corresponding to the text information, in view of the introduction of a dialogue management system in a multi-round dialogue scenario, through a shared history mechanism, no matter how the content distribution model is classified, each round of dialogue will update the historical semantics, and the historical semantics will be shared between the large model and the rule-based and dictionary-based methods, thereby assisting in obtaining more accurate semantic results; for example, the knowledge question-and-answer model uses a slot extraction method to store the name, place, song name, etc. of historical data in Redis, and associates historical interactions to assist in understanding. For example, if the historical text is who someone is, the name of someone will be stored in redis. If the current text is I want to listen to his song, the person he refers to in the text will be obtained from redis. Among them, Redis is a memory-based non-relational database with fast read and write performance, excellent performance and scalability; the main features are support for multiple types of data such as strings, hashes, lists, sets, etc., support for rich data structures, and can provide developers with extremely powerful data processing and storage services. In general, it realizes the functions of dialogue process control, intent understanding, knowledge base and information management, response strategy optimization and exception handling, better understands the needs of users and provides more accurate responses, solves the corpus problems of cross-domain, single action, single entity and chat, and gives full play to the multi-round dialogue capabilities of large models. The specific processing flow of the dialogue management system is as follows: Assume that the dialogue state s at the current time t+1 is t+1 , the state s at the previous time t t , the system behavior a at the previous time t t (e.g., the state of the vehicle computer), and the user behavior o corresponding to the current time t+1 t+1 , which can be expressed as: t+1 ←s t +a t +o t+1 , that is, to update s using semantic standard specifications t , and consider a t To assist t+1, the target semantics that conform to the context information can be obtained, and the misabsorption and fantasy of chat-style corpus can be avoided; for example, if the historical text is who is someone, then this historical text will go to the knowledge question-answering model, and the semantics of who is someone will be updated to the historical semantics. The current text is I want to listen to his song, then in the dialogue management system, he will be referred to as the artist in the historical semantics: someone, and the target semantics of I want to listen to someone's song will be obtained; then, the corresponding semantic instruction is obtained according to the target semantics, for example, play someone's song; the corresponding operation is performed according to the semantic instruction to obtain the execution result, and the corresponding reply text information is generated according to the execution result, for example, someone's song has been played, and the response method is confirmed according to the reply text information; the reply text information is converted into speech through TTS (Text to Speech) to obtain the target speech, and the target speech is broadcast according to the response method.

[0078] That is to say, for multi-round dialogue scenarios, the embodiments of the present application assist in understanding semantics through a shared history mechanism, because the final semantic result of the current dialogue depends not only on the text information of the users of the current dialogue, but also on the semantic result of the previous round of dialogue and the system behavior after the previous round of execution. Then, the user's semantic information is semantically associated according to the semantic result of the previous round of dialogue and the system behavior after the previous round of execution to obtain the target semantics; and the corresponding operations are performed to convert the reply text information corresponding to the obtained execution result into target voice information, and the target voice information is broadcast according to the determined response method, so as to better understand the user's needs and provide more accurate responses, giving full play to the capabilities of large-model multi-round dialogues.

[0079] It can be understood that in the embodiments of the present application, the introduced dialogue management system has expanded multiple functions. In the post-processing stage, it is similar to rule-based dialogue management, involving inheritance and reference rules, state rejection rules, history acquisition and update, state acquisition and update, source search, etc.; and a visualization platform is used to provide dialogue memory function, and dialogue process control is used to determine the start, end and transition of the dialogue, as well as the direction of the dialogue at different stages; a knowledge base is used to manage the information and context involved in the dialogue, so as to ensure that historical information can be accurately and efficiently retrieved; dialogue plan optimization refers to continuous optimization based on user feedback to improve dialogue quality and efficiency.

[0080] It can be understood that in the embodiments of the present application, the corresponding execution results will be monitored in real time to see if there are any abnormalities. If there are any abnormalities in the execution results, the corresponding abnormal information will be obtained, the abnormal information will be analyzed, and the corresponding processing plan will be obtained. Thereafter, the processing plan will be called up, and the abnormal information will be processed according to the processing plan to obtain the processing result; a re-execution instruction will be generated according to the processing result, and the operation will be performed according to the re-execution instruction.

[0081] That is to say, when an exception occurs in the execution result, the embodiment of the present application obtains the corresponding exception information in a timely manner, and calls the corresponding processing solution to process the exception information, and executes the corresponding operation again after the processing is completed, thereby ensuring the stability of the system and improving the reliability of the dialogue.

[0082] The following further describes the entire implementation process of the human-machine voice interaction method for a vehicle according to the present application. Figure 8As shown in the figure: the audio data of the user is obtained, the audio data is optimized to obtain the target audio data, and the target audio data is converted into text through ASR to obtain text information; the text information is classified into intent based on the content distribution model to obtain the intent type, the intent strength of the text information is judged to obtain the intent strength information, the first sentence of the text information is judged to obtain the core sentence information, and the text information is context-aware to obtain the context-aware information; the analysis result of the text information is obtained according to the intent type, the intent strength information, the core sentence information and the context-aware information; if the analysis result is that the text information is a simple intent, the target semantic processing scheme is determined to be a rule-based and dictionary-based method processing scheme, and the text information is semantically understood according to the rule-based and dictionary-based method processing scheme to obtain semantic information; if the analysis result is that the text information is a question-and-answer intent, the target semantic processing scheme is determined to be a knowledge question-and-answer model processing scheme, and the text information is question-determined according to the knowledge question-and-answer model processing scheme to obtain the question. corresponding semantic information; if the analysis result is that the text information is an arrangement intention, the target semantic processing scheme is determined to be a voice arrangement model processing scheme, and the text information is organized and arranged according to the voice arrangement model processing scheme to obtain corresponding semantic information; if the analysis result is that the text information is multi-intent, the target semantic processing scheme is determined to be a multi-intent model processing scheme, and the text information is analyzed for intent through the multi-intent model processing scheme to obtain multiple text intentions, and all the text intentions are combined to obtain corresponding semantic information; historical conversation semantic results and historical system behaviors are obtained, and the semantic information is semantically associated according to the historical conversation semantic results and the historical system behaviors to obtain target semantics, and corresponding semantic execution instructions are obtained according to the target semantics; corresponding operations are executed according to the semantic execution instructions to obtain execution results, corresponding reply text information is generated according to the execution results, and a response method is confirmed according to the reply text information; the reply text information is converted into speech through TTS to obtain a target speech, and the target speech is broadcast according to the response method.

[0083] Secondly, a human-machine voice interaction system for a vehicle proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0084] Fig. 9 It is a block diagram of a human-machine voice interaction system for a vehicle according to an embodiment of the present application.

[0085] like Fig. 9 As shown, the human-machine voice interaction system 10 of the vehicle includes: a text conversion module 100, a semantic understanding module 200 and a voice response module 300.

[0086] Specifically, the text conversion module 100 is used to obtain the user's audio data, and perform text conversion on the audio data to obtain text information.

[0087] The semantic understanding module 200 is used to perform intent analysis on the text information to obtain analysis results, determine a semantic processing scheme for the text information based on the analysis results to obtain a target semantic processing scheme, and perform semantic understanding processing on the text information based on the target semantic processing scheme to obtain semantic information.

[0088] The voice reply module 300 is used to perform association processing on the semantic information to obtain the target semantics, execute instructions according to the target semantics to obtain execution results, obtain reply text information according to the execution results, and broadcast the target voice information corresponding to the reply text information.

[0089] Optionally, in one embodiment of the present application, the human-machine voice interaction system 10 of the target vehicle further includes: a model training unit and a model fine-tuning unit.

[0090] Among them, the model training unit is used to obtain a pre-training task, and pre-train the language network training model according to the pre-training task to obtain a language representation model, wherein the pre-training task includes a masked language prediction task and a context prediction task.

[0091] The model fine-tuning unit is used to obtain a natural language processing task, and fine-tune the language representation model according to the natural language processing task to obtain a content distribution model.

[0092] Optionally, in one embodiment of the present application, the semantic understanding module 200 includes: a text information processing unit, an information analysis unit, a first solution determination unit, a first solution processing unit, a path weight solving unit, a second solution determination unit, a second solution processing unit, a third solution determination unit, a third solution processing unit, a fourth solution determination unit and a fourth solution processing unit.

[0093] Among them, the text information processing unit is used to classify the intent of the text information based on the content distribution model to obtain the intent type, judge the strength of the intent of the text information to obtain the intent strength information, judge the first sentence of the text information to obtain the core sentence information, and perform context perception on the text information to obtain context perception information.

[0094] An information analysis unit is used to obtain an analysis result of the text information according to the intention type, the intention strength information, the core sentence information and the context perception information.

[0095] An intent type determination unit is used to determine the target intent type of the text information based on the analysis result, wherein the target intent type includes simple intent, question-and-answer intent, editing intent and multiple intents.

[0096] The first solution determination unit is used to determine the target semantic processing solution as a rule- and dictionary-based method processing solution according to the simple intent if the analysis result is that the text information is a simple intent.

[0097] The first scheme processing unit is used to perform single-word chain processing on the text information according to the rule-based and dictionary-based method processing scheme to obtain a single-word chain network, and perform arc pasting processing on the single-word chain network to obtain a business grammar network.

[0098] The path weight solving unit is used to merge the single-word chain network and the business grammar network to obtain a target network, solve the path weight of the target network to obtain a target path, and obtain semantic information according to the target path.

[0099] The second solution determination unit is used to determine that the target semantic processing solution is a knowledge question and answer model processing solution according to the question and answer intention if the analysis result is that the text information is a question and answer intention.

[0100] The second solution processing unit is used to determine the question of the text information according to the knowledge question and answer model processing solution to obtain corresponding semantic information.

[0101] The third scheme determination unit is used to determine that the target semantic processing scheme is a speech arrangement model processing scheme according to the arrangement intention if the analysis result is that the text information is an arrangement intention.

[0102] The third scheme processing unit is used to organize and arrange the text information according to the speech arrangement model processing scheme to obtain corresponding semantic information.

[0103] The fourth scheme determination unit is used to determine the target semantic processing scheme as a multi-intent model processing scheme according to the multi-intent if the analysis result is that the text information has multiple intents.

[0104] The fourth scheme processing unit is used to perform intent analysis on the text information according to the multi-intention model processing scheme to obtain multiple text intents, and combine all the text intents to obtain corresponding semantic information.

[0105] Optionally, in one embodiment of the present application, the text information processing unit further includes: an intention strength judgment subunit, a first judgment subunit and a second judgment subunit.

[0106] Among them, the intention strength judgment subunit is used to establish an intention strength corpus, set a strength threshold according to the strong intention corpus and the weak intention corpus of the intention strength corpus, and judge the intention strength of the text information according to the strength threshold to obtain a judgment result.

[0107] The first determination subunit is used to determine that if the determination result is that the intention strength value of the text information is greater than or equal to the strength threshold, then the intention strength information is a strong intention.

[0108] The second determination subunit is used for determining that if the determination result is that the intention strength value of the text information is less than the strength threshold, then the intention strength information is a weak intention.

[0109] Optionally, in one embodiment of the present application, the voice reply module 300 includes: a historical semantic association unit, a reply text determination unit and a reply text broadcasting unit.

[0110] Among them, the historical semantic association unit is used to obtain historical dialogue semantic results and historical system behaviors, semantically associate the semantic information according to the historical dialogue semantic results and the historical system behaviors, obtain target semantics, and obtain corresponding semantic execution instructions according to the target semantics.

[0111] The reply text determination unit is used to execute the corresponding operation according to the semantic execution instruction, obtain the execution result, generate the corresponding reply text information according to the execution result, and confirm the response method according to the reply text information.

[0112] The reply text broadcasting unit is used to convert the reply text information into voice to obtain target voice information, and broadcast the target voice information according to the response method.

[0113] Optionally, in one embodiment of the present application, the voice reply module 300 further includes: an exception handling unit and a re-execution unit.

[0114] Among them, the exception handling unit is used to obtain corresponding exception information if an exception occurs in the execution result, perform exception analysis on the exception information, obtain a corresponding processing plan, and call the processing plan.

[0115] The re-execution unit is used to process the abnormal information according to the processing scheme to obtain a processing result, generate a re-execution instruction according to the processing result, and re-execute the corresponding operation according to the re-execution instruction.

[0116] It should be noted that the above explanation of the embodiment of the human-computer voice interaction method for a vehicle is also applicable to the human-computer voice interaction system for the vehicle of this embodiment, and will not be repeated here.

[0117] In order to better combine the rule-based and dictionary-based methods with the big model and give full play to the semantic understanding ability of the big model, the embodiment of the present application intelligently matches the text information obtained by processing the user's audio data to classify the content, and also optimizes the distribution strategy by analyzing the corresponding text information; then the rule-based and dictionary-based methods make up for the disadvantages of the big model in some cases, and the big model makes up for the generalization ability and multi-intention ability of the rule-based and dictionary-based methods. The two are combined through corresponding schemes, with the results obtained by the rule-based and dictionary-based methods as the lower limit to ensure the normal implementation of simple intentions, and the results obtained by the big model as the upper limit, which not only solves complex intentions, but also ensures the accuracy and room for development of semantic understanding; and for multi-round dialogue scenarios, it is assisted by a shared history mechanism to obtain more accurate semantic results, thereby better understanding user needs and providing more accurate responses, giving full play to the multi-round dialogue capabilities of the big model.

[0118] Fig.10 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:

[0119] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0120] When the processor 502 executes the program, the human-machine voice interaction method for the vehicle provided in the above embodiment is implemented.

[0121] Furthermore, the vehicle also includes:

[0122] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0123] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0124] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0125] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0126] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0127] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0128] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the human-computer voice interaction method for a vehicle as described above is implemented.

[0129] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0130] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0131] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0132] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory.

[0133] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0134] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0135] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0136] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0137] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

Claims

1. A human-machine voice interaction method for a vehicle, characterized in that: The human-machine voice interaction method of the vehicle comprises: Acquire audio data of the user, and convert the audio data into text to obtain text information; Performing intent analysis on the text information to obtain an analysis result, determining a semantic processing scheme for the text information according to the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information; The semantic information is associated and processed to obtain target semantics, instructions are executed according to the target semantics to obtain execution results, reply text information is obtained according to the execution results, and target voice information corresponding to the reply text information is broadcast.

2. The human-machine voice interaction method for a vehicle according to claim 1, characterized in that: The step of obtaining audio data of the user and converting the audio data into text to obtain text information further includes: Acquire a pre-training task, and perform pre-training processing on the language network training model according to the pre-training task to obtain a language representation model, wherein the pre-training task includes a masked language prediction task and a context prediction task; A natural language processing task is obtained, and the language representation model is fine-tuned according to the natural language processing task to obtain a content distribution model.

3. The human-machine voice interaction method for a vehicle according to claim 2, characterized in that: The performing intention analysis on the text information to obtain the analysis result specifically includes: Based on the content distribution model, the text information is classified into intent types, the strength of the intent of the text information is judged, the strength of the intent is obtained, the first sentence of the text information is judged, the core sentence information is obtained, and the context of the text information is perceived to obtain context perception information; The analysis result of the text information is obtained according to the intention type, the intention strength information, the core sentence information and the context perception information.

4. The human-machine voice interaction method for a vehicle according to claim 3, characterized in that: The determining the strength of the intention of the text information to obtain the intention strength information specifically includes: Establishing an intention strength corpus, setting a strength threshold according to the strong intention corpus and the weak intention corpus in the intention strength corpus, and judging the intention strength of the text information according to the strength threshold to obtain a judgment result; If the judgment result is that the intention strength value of the text information is greater than or equal to the strength threshold, then the intention strength information is a strong intention; If the judgment result is that the intention strength value of the text information is less than the strength threshold, then the intention strength information is a weak intention.

5. The human-machine voice interaction method for a vehicle according to claim 1, characterized in that: Determining a semantic processing scheme for the text information according to the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information specifically includes: If the analysis result is that the text information is a simple intention, determining the target semantic processing scheme as a rule-based and dictionary-based method processing scheme according to the simple intention; Performing single-word chain processing on the text information according to the rule-based and dictionary-based method processing scheme to obtain a single-word chain network, and performing arc pasting processing on the single-word chain network to obtain a business grammar network; The single-word chain network and the business grammar network are merged to obtain a target network, path weights are solved for the target network to obtain a target path, and semantic information is obtained according to the target path.

6. The human-machine voice interaction method for a vehicle according to claim 1, characterized in that: Determining a semantic processing scheme for the text information according to the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information specifically includes: If the analysis result is that the text information is a question-answering intention, determining the target semantic processing scheme as a knowledge question-answering model processing scheme according to the question-answering intention; The text information is subjected to question determination according to the knowledge question answering model processing scheme to obtain corresponding semantic information.

7. The human-machine voice interaction method for a vehicle according to claim 1, characterized in that: Determining a semantic processing scheme for the text information according to the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information specifically includes: If the analysis result is that the text information is an arrangement intention, determining the target semantic processing scheme as a speech arrangement model processing scheme according to the arrangement intention; The text information is organized and arranged according to the speech arrangement model processing solution to obtain corresponding semantic information.

8. The human-machine voice interaction method for a vehicle according to claim 1, characterized in that: Determining a semantic processing scheme for the text information according to the analysis result to obtain a target semantic processing scheme, and performing semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information specifically includes: If the analysis result is that the text information has multiple intents, determining the target semantic processing scheme as a multi-intent model processing scheme according to the multiple intents; The text information is analyzed for intent according to a multi-intent model processing solution to obtain multiple text intents, and all the text intents are combined to obtain corresponding semantic information.

9. The human-machine voice interaction method for a vehicle according to claim 1, characterized in that: The associating processing of the semantic information to obtain target semantics, executing instructions according to the target semantics to obtain execution results, obtaining reply text information according to the execution results, and broadcasting target voice information corresponding to the reply text information specifically includes: Acquire historical conversation semantic results and historical system behaviors, semantically associate the semantic information according to the historical conversation semantic results and the historical system behaviors, obtain target semantics, and obtain corresponding semantic execution instructions according to the target semantics; Execute a corresponding operation according to the semantic execution instruction to obtain an execution result, generate a corresponding reply text message according to the execution result, and confirm a response method according to the reply text message; The reply text message is voice-converted to obtain target voice information, and the target voice information is broadcast according to the response method.

10. The human-machine voice interaction method for a vehicle according to claim 9, characterized in that: The corresponding operation is executed according to the semantic execution instruction to obtain the execution result, and then the following is further included: If the execution result is abnormal, obtain corresponding abnormal information, perform abnormal analysis on the abnormal information, obtain a corresponding processing plan, and call the processing plan; The abnormal information is processed according to the processing scheme to obtain a processing result, a re-execution instruction is generated according to the processing result, and a corresponding operation is performed again according to the re-execution instruction.

11. A human-machine voice interaction system for a vehicle, characterized in that: The human-machine voice interaction system of the vehicle comprises: A text conversion module is used to obtain the user's audio data and convert the audio data into text to obtain text information; A semantic understanding module is used to perform intent analysis on the text information to obtain an analysis result, determine a semantic processing scheme for the text information according to the analysis result to obtain a target semantic processing scheme, and perform semantic understanding processing on the text information according to the target semantic processing scheme to obtain semantic information; The voice reply module is used to perform association processing on the semantic information to obtain the target semantics, execute instructions according to the target semantics to obtain the execution result, obtain the reply text information according to the execution result, and broadcast the target voice information corresponding to the reply text information.

12. A vehicle, characterized in that: The vehicle includes: a memory, a processor, and a human-machine voice interaction program for the vehicle stored in the memory and executable on the processor. When the human-machine voice interaction program for the vehicle is executed by the processor, the steps of the human-machine voice interaction method for the vehicle as described in any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a human-machine voice interaction program for a vehicle, and when the human-machine voice interaction program for a vehicle is executed by a processor, the steps of the human-machine voice interaction method for a vehicle as described in any one of claims 1 to 10 are implemented.

Citation Information

Cited By

  • Multi-modal voice interaction method based on large model, electronic equipment and storage medium

    CN119559946A

  • Multimodal Speech Interaction Method, Electronic Device, and Storage Medium Based on Large Language Model

    CN119559946B

  • Vehicle-mounted voice interaction method and device, vehicle, medium and program

    CN119864034A

  • Vehicle-mounted voice interaction test method, device and equipment, storage medium and vehicle

    CN119889288A

  • Voice interaction method, system and device based on artificial intelligence and medium

    CN119964570A