A voice control method, device, electronic device and storage medium

By performing visible-and-speakable recognition and natural language understanding processing on the user's control voice, the recognition and processing results are generated, and the target control command is determined. This solves the problem that the vehicle cannot execute commands of non-current interface controls in the visible-and-speakable mode, and realizes comprehensive voice control.

CN115457951BActive Publication Date: 2024-11-19BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210505936.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-10
Publication Date
2024-11-19
Estimated Expiration
2042-05-10

AI Technical Summary

Technical Problem

In existing technology, when a vehicle is in the "see-and-talk" mode, if the user's spoken command is not a trigger command for a control displayed on the current vehicle screen, the vehicle will discard the command and not execute it.

Method used

By performing visible and speakable recognition and natural language understanding on the user's control voice, recognition results and processing results are generated. Based on these results, target control commands are determined and corresponding control operations are executed.

Benefits of technology

Regardless of whether the command spoken by the user triggers a control on the current in-vehicle infotainment screen, the command can be executed, achieving comprehensive control in the "what you see is what you speak" mode.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115457951B_ABST
    Figure CN115457951B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a voice control method, device, electronic device and storage medium, the method comprising: obtaining user control voice; performing visible or spoken recognition on the user control voice to form a recognition result; performing natural language understanding processing on the user control voice to form a processing result; determining a target control instruction based on the recognition result and the processing result; and executing a control operation corresponding to the target control instruction. If the user control voice is recognized as a trigger instruction of a certain control in the interface displayed on the current vehicle screen, the trigger instruction is executed. If the user control voice is not a trigger instruction of a certain control in the interface displayed on the current vehicle screen, the user intention can be determined based on the processing result of the natural language understanding processing method, and the corresponding control operation is executed. In this way, regardless of whether the instruction spoken by the user is a trigger instruction of a certain control in the interface displayed on the current vehicle screen, the instruction can be executed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of voice control technology, and in particular to a voice control method, device, electronic device and storage medium. Background Art

[0002] With the continuous advancement of technology, electronic devices with voice control functions are introduced in more and more scenarios, such as the introduction of in-vehicle terminals with voice control functions in vehicles.

[0003] At present, some vehicles have the "see and speak" function. When the vehicle is in the "see and speak" mode, there is no need to touch the car screen manually. As long as the function options that can be displayed on the screen can be controlled through voice interaction, the "mouth only, no hands" is realized. For example, there is a play control on the interface of a multimedia application. The user's operation intention is to play music. Just say "play music" to complete the automatic playback of music.

[0004] In practice, when the vehicle is in the see-and-speak mode, if the command spoken by the user is not a trigger command for a control in the interface currently displayed on the vehicle screen, the vehicle will directly discard the command and not execute it. In this case, how to implement the execution of the command regardless of whether the command spoken by the user is a trigger command for a control in the interface currently displayed on the vehicle screen is a problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a voice control method, device, electronic device and storage medium.

[0006] In a first aspect, the present disclosure provides a voice control method, comprising:

[0007] Get user control voice;

[0008] Performing visible or audible recognition on the user control voice to form a recognition result;

[0009] Performing natural language understanding processing on the user control voice to form a processing result;

[0010] Determining a target control instruction based on the recognition result and the processing result;

[0011] Execute the control operation corresponding to the target control instruction.

[0012] In a second aspect, the present disclosure further provides a voice control device, comprising:

[0013] An acquisition module, used to acquire user control voice;

[0014] A recognition module, used for visually recognizing the user-controlled voice and forming a recognition result;

[0015] A processing module, used for performing natural language understanding processing on the user control voice to form a processing result;

[0016] A determination module, configured to determine a target control instruction based on the recognition result and the processing result;

[0017] An execution module is used to execute the control operation corresponding to the target control instruction.

[0018] In a third aspect, the present disclosure further provides an electronic device, including: a processor and a memory;

[0019] The processor is used to execute the steps of any of the above methods by calling the program or instruction stored in the memory.

[0020] In a fourth aspect, the present disclosure further provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction enables a computer to execute the steps of any of the above methods.

[0021] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0022] The technical solution provided by the embodiment of the present disclosure forms a recognition result by performing visible or spoken recognition on the user control voice; forms a processing result by performing natural language understanding processing on the user control voice; determines the target control instruction based on the recognition result and the processing result; and executes the control operation corresponding to the target control instruction. In essence, if the user control voice is recognized as a trigger instruction of a certain control in the interface displayed on the current vehicle screen, the trigger instruction is executed. If the user control voice is not a trigger instruction of a certain control in the interface displayed on the current vehicle screen, the user intention can be determined based on the processing result of the natural language understanding processing method, and the corresponding control operation is executed. In this way, regardless of whether the instruction spoken by the user is a trigger instruction of a certain control in the interface displayed on the current vehicle screen, the instruction can be executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0025] Figure 1 A flow chart of a voice control method provided by an embodiment of the present disclosure;

[0026] Figure 2 A flowchart of another voice control method provided by an embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram of a vehicle interior layout provided by an embodiment of the present disclosure;

[0028] Figure 4 for Figure 3 Schematic diagram of the connection relationship between various components of the vehicle;

[0029] Figure 5 A schematic diagram of the structure of a voice control device proposed in an embodiment of the present disclosure;

[0030] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0033] Figure 1 A flow chart of a voice control method provided by an embodiment of the present disclosure. Figure 1 , the method comprising:

[0034] S110: Acquire user control voice.

[0035] S120: Perform visible or spoken recognition on the user control voice to generate a recognition result.

[0036] Seeing is saying means that no manual touch is required. As long as the function options that can be displayed on the car computer can be controlled through voice interaction, the "only mouth, no hands" is realized. For example, for a multimedia application, there is a play button on the interface, and the operation intention is to play music. You only need to say "play music" to complete the automatic playback of music, realizing the function of seeing is saying.

[0037] There are many methods for implementing this step, which are not limited in this application. Exemplarily, the method for implementing this step includes: loading a control instruction set corresponding to the application currently displayed by the vehicle computer; the control instruction set includes standard control instructions generated according to the interface control data of the application interface; and using the standard control instructions in the control instruction set that match the user control voice as the recognition result of the user control voice.

[0038] Among them, each standard control instruction in the control instruction set is generated according to the interface control data of the application program interface. The interface control data may include control data corresponding to all interface controls, that is, all interface controls in the interactive interface have corresponding standard control instructions.

[0039] It should be noted that, in practice, an application often has multiple interactive interfaces, and the interactive interfaces of the application can be switched in response to user operations. In practice, one application can be set to correspond to multiple control instruction sets, and each control instruction set includes standard control instructions corresponding to each interface control in an interactive interface. Optionally, in practice, one application can also be set to correspond to one control instruction set, and the control instruction set includes standard control instructions corresponding to the interface controls in each interface of the application.

[0040] “Using the standard control instruction in the control instruction set that matches the user control voice as the recognition result of the user control voice” may specifically include: converting the user control voice into voice text; and searching the control instruction set for the standard control instruction that matches the voice text.

[0041] Specifically, the electronic device may input the user-controlled voice into an automatic speech recognition (ASR) engine set up offline to obtain the voice text output by the ASR engine.

[0042] The user control voice and the standard control instruction "match" when the voice text of the user control voice contains any verb and any control text segmentation in the standard control instruction, or the verb in the voice text is the same as any verb in the standard control instruction and the similarity between the noun in the voice text and any control text segmentation in the standard control instruction is greater than or equal to a preset similarity threshold.

[0043] Therefore, taking the standard control instruction in the control instruction set that matches the user control voice as the recognition result of the user control voice is essentially to determine that the user's voice control intention is the control intention represented by the standard control instruction.

[0044] S130: Perform natural language understanding processing on the user control voice to generate a processing result.

[0045] There are many ways to implement this step. Exemplarily, the text in the user-controlled voice is preprocessed and normalized, and then the sentence trunk is extracted; based on the sentence trunk, the intention of the user-controlled voice is understood, and the semantic understanding result (i.e., the processing result) is output.

[0046] The text in the user's control voice is preprocessed, including Chinese word segmentation and removal of modal words (such as "hmm" and "bar"). Text normalization includes normalization of numbers and entities. For example, "1.5 seconds" becomes "1.5 seconds" after normalization; "large screen brightness" becomes "central control brightness" after normalization. Extracting the sentence stem is to extract entity words, action words and numerical values ​​in the sentence. The extracted sentence stem is mainly used for subsequent retrieval. The action words in the extracted sentence stem can be used to understand the user's intention.

[0047] Performing natural language understanding and processing on user control voice is a conventional technical means in this field and will not be elaborated in detail here.

[0048] S140: Determine a target control instruction based on the recognition result and the processing result.

[0049] There are multiple methods for implementing this step. Exemplarily, the method for implementing this step includes: setting the priority of the recognition result and the priority of the processing result, and determining the target control instruction in the recognition result and the processing result based on their respective priorities.

[0050] Alternatively, the implementation method of this step includes: determining the target control instruction from the recognition result and the processing result, that is, selecting one of the recognition result and the processing result as the target control instruction.

[0051] Further, the confidence of the recognition result can be determined; based on the confidence of the recognition result, one of the recognition result and the processing result is selected to be determined as the target control instruction. The confidence is a parameter reflecting whether the recognition result is accurate or not. Exemplarily, the similarity between the voice text of the user control voice and the standard control instruction is used as the confidence of the recognition result. Alternatively, the similarity between the recognition result and the processing result is used as the confidence of the recognition result.

[0052] In some embodiments, a threshold value may be pre-set, and the threshold value is used to judge the confidence level. If the confidence level of the recognition result is greater than the threshold value, the recognition result is determined as the target control instruction. If the confidence level of the recognition result is less than or equal to the threshold value, the processing result is determined as the target control instruction.

[0053] In other embodiments, if an application corresponds to a control instruction set, and the control instruction set includes standard control instructions corresponding to interface controls in each interface of the application, it is possible that the control corresponding to the control data in the recognition result does not appear in the picture displayed on the current vehicle screen. For this situation, optionally, if the confidence of the recognition result is greater than a set threshold, and the interface currently displayed by the vehicle includes the control corresponding to the control data in the recognition result; the recognition result is determined as a target control instruction. If the confidence of the recognition result is greater than the set threshold, but the interface currently displayed by the vehicle does not include the control corresponding to the control data in the recognition result; the processing result is determined as a target control instruction. If the confidence of the recognition result is less than or equal to the set threshold, the processing result is determined as a target control instruction.

[0054] S150: Execute a control operation corresponding to the target control instruction.

[0055] If the target control instruction includes an identification result, since each standard control instruction (i.e., the identification result) is generated based on the control data of the corresponding interface control, each standard control instruction can be used to trigger the execution of a control operation on the interface control involved in the standard control instruction, i.e., each standard control instruction can be used to trigger the execution of a control operation on the interface control to which the control data that generated the standard control instruction belongs.

[0056] If the target control instruction includes a processing result, since the processing result is the user's intention, the control operation can be performed according to the user's intention.

[0057] The above technical solution forms a recognition result by performing visible or spoken recognition on the user control voice; performs natural language understanding processing on the user control voice to form a processing result; determines the target control instruction based on the recognition result and the processing result; and executes the control operation corresponding to the target control instruction. In essence, if the user control voice is recognized as a trigger instruction of a control in the interface displayed on the current vehicle screen, the trigger instruction is executed. If the user control voice is not a trigger instruction of a control in the interface displayed on the current vehicle screen, the user intention can be determined based on the processing result of the natural language understanding processing method, and the corresponding control operation is executed. In this way, regardless of whether the instruction spoken by the user is a trigger instruction of a control in the interface displayed on the current vehicle screen, the instruction can be executed.

[0058] On the basis of the above technical solutions, it is further considered that there may be multiple rounds of dialogue in practice. Multiple rounds of dialogue is a way for the car computer to obtain necessary information to finally obtain clear user instructions after initially clarifying the user's intention in the dialogue between people and the car computer. Multiple rounds of dialogue correspond to the processing of one thing. The mode of multiple rounds of dialogue is usually manifested as a question-and-answer mode, that is, in the process of dialogue, the car computer can not only respond to questions raised by the user, but also ask questions to the user. In the multiple rounds of dialogue, the user control voice is not a trigger instruction of a certain control in the interface displayed on the current car computer screen. Therefore, if the user control voice is in a multiple round dialogue, the processing result is determined as the target control instruction. Furthermore, it can also be set to determine whether the user control voice is in a multiple round dialogue after obtaining the user control voice. If so, the user control voice is subjected to natural language understanding processing; the user control voice is no longer subjected to visible and audible recognition.

[0059] There are many ways to "determine whether the user control voice is in a multi-round dialogue", and this application does not limit this. Exemplarily, the vehicle monitors the user control voice, and after collecting the user control voice, the user control voice is processed to determine whether each slot in the slot group of the user intention corresponding to the user control voice lacks slot information. If there is at least one slot that lacks slot information, the marking state parameter is currently entering the multi-round dialogue state. Repeat the supplementary step until each slot in the slot group after supplementation does not lack slot information, and the marking state parameter is currently exiting the multi-round dialogue state. Among them, the supplementary step includes: continue to monitor the user control voice, and when the user control voice is collected again, the user control voice is processed to obtain supplementary slot information; fill the supplementary slot information into the slot to be supplemented, and determine whether each slot in the slot group after supplementation lacks slot information. The method of "determining whether the user control voice is in a multi-round dialogue" includes: while obtaining the user control voice, obtaining the state parameter; if the state parameter is currently entering the multi-round dialogue state, the user control voice is in a multi-round dialogue. If the state parameter is the current exit multi-round dialogue state, the user-controlled voice is not in a multi-round dialogue.

[0060] On the basis of the above technical solutions, optionally, the standard control instruction in the control instruction set includes the first control instruction and / or the second control instruction.

[0061] The first control instruction refers to an instruction generated according to the static control data in the interface control data, that is, a control instruction of the static control.

[0062] The static control data may be control data corresponding to a static control in the interactive interface. The static control may be an interface control that is always displayed in a fixed manner, that is, the static control will not change with user preferences or settings.

[0063] For example, the main interface of the setting application has three fixed controls, namely "My Device", "Control Center" and "More Settings", and the control instruction generated based on the control data corresponding to these three controls is the first control instruction of the setting application.

[0064] Optionally, if the control instruction set includes the first control instruction. "Loading the control instruction set corresponding to the application currently displayed by the vehicle" includes: the vehicle receives the control instruction set sent by the server, or reads the control instruction set pre-stored in the vehicle. In some embodiments, for each application, the server can receive the control instructions of all static controls corresponding to each interactive interface of the application input by the developer and the control method corresponding to each control instruction. Among them, the control instruction of each static control includes a verb set and a control text segmentation set corresponding to the static control. The control text segmentation set corresponding to the static control is a segmentation set extracted by the developer from the control data of the static control, that is, the static control text in the static control data. The static control text can be the control name of the static control that can be seen by the user, and the verb set in the control instruction contains multiple verbs with similar semantics.

[0065] In other embodiments, for each static control in each interactive interface of each application, the server can extract a control text word segmentation set from the control data of the static control, that is, the static control text in the static control data, and then combine the control text word segmentation set with the pre-set different verb sets to obtain multiple control instructions of the static control, and the verb set in each control instruction contains multiple verbs with similar semantics. For each control instruction of each static control, the server can also use the control instruction corresponding to the verb set in the control instruction and the control function of the static control corresponding to the control text word segmentation set to determine the control mode corresponding to the control instruction.

[0066] Among them, each word in the control text word set can be connected by "|", and each verb in the verb set can also be connected by "|".

[0067] Therefore, in the embodiment of the present disclosure, the word segmentation set content that conforms to the Extended Backus-Naur Form (EBNF) grammatical paradigm can be obtained, so that the first control instruction can be loaded into the language model of the grammar engine.

[0068] Taking the control instruction corresponding to the "Music Application" icon as an example, if a control instruction is used to open the music application, the verb set can be "open|open|enter|enter|click|click", and the control text word segmentation set can be "music|music application|music's|music icon".

[0069] The second control instruction refers to an instruction generated according to the dynamic control data in the interface control data. The dynamic control is a control formed by filling the control data in the dynamic content reserved field. The second control instruction is the control instruction of the dynamic control.

[0070] The dynamic control data may be control data corresponding to a dynamic control in an interactive interface. The dynamic control may be an interface control that can be dynamically updated or changed according to user preferences or settings.

[0071] For example, in a music application, the playlist names displayed in the tab of the "Playlist" option can be dynamically updated. Therefore, the playlist links corresponding to these playlist names belong to the dynamic controls of the main interface of the music application, and the control instructions generated based on the control data corresponding to each playlist link are the second control instructions corresponding to the main interface of the music application.

[0072] If the control instruction set includes the second control instruction, "loading the control instruction set corresponding to the application currently displayed by the vehicle computer" includes: processing the dynamic control data to generate the second control instruction.

[0073] There are many methods for “processing dynamic control data to generate a second control instruction”. For example, a method for processing dynamic control data to generate a second control instruction includes:

[0074] First, extract the dynamic control text from the dynamic control data.

[0075] In the disclosed embodiment, the dynamic control data includes the control text, control display parameters, etc. of the control.

[0076] Specifically, the electronic device may extract the dynamic control text of the dynamic control to which the dynamic control data belongs from the dynamic control data corresponding to the interactive interface. The dynamic control text may be the control name of the dynamic control that can be seen by the user.

[0077] For example, the playlist link is a dynamic control of the main interface of the application. Taking the playlist link named "Ambient Piano Music When the Soul Dialogues with the Endless Void" as an example, its dynamic control text is "Ambient Piano Music When the Soul Dialogues with the Endless Void".

[0078] Secondly, the dynamic control text is segmented to obtain the segmentation set corresponding to the dynamic control text.

[0079] In the disclosed embodiment, after extracting the dynamic control text of any dynamic control, the electronic device may perform word segmentation processing on the dynamic control text to obtain a word segmentation set corresponding to the dynamic control text, namely, a control text word segmentation set of the dynamic control.

[0080] Specifically, the electronic device can use any word segmentation processing algorithm to split the dynamic control text into multiple control text word segments, and then combine any multiple adjacent control text word segments to obtain multiple word segmentation combinations, and finally obtain a word segmentation set corresponding to the dynamic control text containing multiple control text word segments and multiple word segmentation combinations.

[0081] Optionally, the method of combining multiple control text segmentations and multiple segmentation combinations to obtain a segmentation set may include connecting the multiple control text segmentations and the multiple segmentation combinations using “|” to obtain the segmentation set.

[0082] Therefore, in the embodiment of the present disclosure, the word segmentation set content that conforms to the EBNF grammar paradigm can be obtained, so that the generated second control instruction can be loaded into the language model of the Grammar engine.

[0083] Finally, a second control instruction is generated according to the word segmentation set.

[0084] In the disclosed embodiment, after obtaining the word segmentation set corresponding to the dynamic control text, the electronic device may generate a second control instruction according to the word segmentation set based on a preset control instruction generation method.

[0085] Optionally, the second control instruction is generated according to a preset verb set and participle set.

[0086] The electronic device can combine different pre-set verb sets and word segmentation sets to obtain multiple control instructions of the dynamic control, and the verb set in each control instruction contains multiple verbs with similar semantics. For each control instruction of each dynamic control, the electronic device can also use the control instruction corresponding to the verb set in the control instruction and the control function of the dynamic control corresponding to the word segmentation set to determine the control mode corresponding to the control instruction.

[0087] In some embodiments of the present disclosure, before "performing word segmentation processing on the dynamic control text to obtain a word segmentation set corresponding to the dynamic control text", the voice control method may further include: preprocessing the dynamic control text.

[0088] In the disclosed embodiment, after extracting the dynamic control text, the electronic device may pre-process the dynamic control text before performing word segmentation processing on the dynamic control text to obtain the dynamic control text that can be used for text processing.

[0089] Optionally, the preprocessing may include symbol removal processing and digital conversion processing.

[0090] Symbol removal can be used to remove symbols in dynamic control text, such as punctuation marks, special symbols, mathematical symbols, and any other symbols that have no semantic meaning.

[0091] The number conversion process can be used to convert Arabic numerals in the dynamic control text into Chinese numerals. If the Arabic numeral has more than two digits, the entire Arabic numeral can be converted into a Chinese numeral, or each digit can be converted into a numeral.

[0092] Specifically, after extracting the dynamic control text, the electronic device can first remove the symbols in the dynamic control text to obtain the dynamic control text after removing the symbols. Then, the electronic device can convert the Arabic numerals in the dynamic control text after removing the symbols into Chinese numerals. Taking the dynamic control text "Chinese Class Representative's Private Collection of 200 Idiom Songs" as an example, the Arabic numeral "200" can be converted into the Chinese numeral "two hundred" or the Chinese numeral "two zero zero" to obtain the dynamic control text after digital conversion "Chinese Class Representative's Private Collection of 200 Idiom Songs|Chinese Class Representative's Private Collection of 200 Idiom Songs", and the electronic device can perform word segmentation and word segmentation combination on the converted dynamic control text to obtain the word segmentation set "Chinese Class Representative's Private Collection of 200 Idiom Songs|Chinese Class Representative's Private Collection of 200 Idiom Songs|Two Hundred Idiom Songs|Two Hundred Idiom Songs|Idiom Songs|Chinese Class Representative|Private Collection of Cheat Sheets".

[0093] Therefore, in the embodiment of the present disclosure, after displaying the interactive interface, the electronic device can generate control instructions for all dynamic controls in the interactive interface, namely, second control instructions, based on the dynamic control data in the interactive interface, and then use these control instructions as dynamic loading content to achieve reliable and efficient loading of the second control instructions of the interactive interface.

[0094] Figure 2 This is a flow chart of another voice control method provided by an embodiment of the present disclosure. Figure 2 , the method comprising:

[0095] S201: Acquire user control voice.

[0096] S202: Determine whether the user control voice is in a multi-round dialogue. If yes, execute S203; if no, execute S205.

[0097] S203: Perform natural language understanding processing on the user control voice to generate a processing result.

[0098] S204: Determine the processing result as the target control instruction.

[0099] S205: Perform visible or spoken recognition on the user control voice to form a recognition result.

[0100] S206: Perform language understanding processing on the user control voice to form a processing result.

[0101] S207: Determine whether the similarity between the voice text of the user control voice and the recognition result is greater than a first set threshold. If yes, execute S208; if not, execute S210.

[0102] S208: Determine whether the interface currently displayed by the vehicle computer includes the control corresponding to the control data in the recognition result. If yes, execute S209; if not, execute S210.

[0103] S209: discard the processing result and determine the recognition result as the target control instruction.

[0104] S210: Determine whether the similarity between the recognition result and the processing result is greater than a second set threshold. If yes, execute S208; if not, execute S211.

[0105] S211, discard the recognition result, and determine the processing result as the target control instruction.

[0106] In the above technical solution, the function of S207 is to initially determine whether the recognition result is accurate. If the similarity between the voice text of the user control voice and the recognition result is greater than the first set threshold, the recognition result is determined to be accurate, and the recognition result is used as the user's intention to control the vehicle, that is, the visible and speakable function is used. If the similarity between the user control voice and the recognition result is less than or equal to the first set threshold, it is determined that the recognition result may not be accurate. In this case, S210 is executed to further determine whether the recognition result is accurate. If the similarity between the recognition result and the processing result is greater than the second set threshold, that is, the results obtained by the two different processing methods tend to be consistent, in this case, the recognition result is considered accurate. The recognition result is used as the user's intention to control the vehicle, that is, the visible and speakable function is used. If the similarity between the recognition result and the processing result is less than or equal to the second set threshold, that is, the results obtained by the two different processing methods are far apart, in this case, the recognition result is considered inaccurate. The processing result is used as the user's intention to control the vehicle, that is, the visible and speakable function is not used.

[0107] The essence of the above technical solution is to determine whether the recognition result is accurate through two judgments. If the recognition result is accurate, the recognition result is used as the user's intention to control the vehicle, that is, the see-and-speak function is used; if the recognition result is inaccurate, the processing result is used as the user's intention to control the vehicle, that is, the see-and-speak function is not used. This setting ensures that when the vehicle is in the see-and-speak mode, the command spoken by the user can be executed regardless of whether it is a trigger command of a control in the interface displayed on the current vehicle screen.

[0108] Figure 3 A schematic diagram of the interior layout of a vehicle provided in accordance with an embodiment of the present disclosure. Figure 4for Figure 3 Schematic diagram of the connection relationship between the various components of the vehicle. Figure 3 and Figure 4 , the interior space of the vehicle is divided into multiple sound zones, each of which corresponds to a seat. A microphone is provided in each sound zone, and the microphone is used to collect voice information of passengers in the sound zone. The vehicle is also provided with a camera, and the camera is used to collect images of passengers in the vehicle. The vehicle is also provided with multiple audio and video devices (including a car screen), and each audio and video device can interact with passengers in the corresponding sound zone to meet the usage needs of passengers in different sound zones. Each audio and video device is directly or indirectly connected to the interactive controller through the system-on-chip (SoC) associated with it. The microphone, camera and system-on-chip are connected through a bus. The interactive controller can determine which seat's passenger issued the instruction corresponding to the voice signal and which screen performs the display based on the position of the microphone of the collected voice signal and the image data collected by the camera. The system-on-chip SOC of the audio and video device that performs the display executes the control instruction corresponding to the voice data.

[0109] The above technical solution of this application can be Figure 3 and Figure 4 The system-on-chip (SoC) of any audio-visual device is executed.

[0110] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0111] Figure 5 This is a schematic diagram of the structure of the voice control device proposed in the embodiment of the present disclosure. Figure 5 As shown, the device comprises:

[0112] An acquisition module 310 is used to acquire user control voice;

[0113] A recognition module 320, configured to perform visible or spoken recognition on the user control voice to form a recognition result;

[0114] The processing module 330 is used to perform natural language understanding processing on the user control voice to form a processing result;

[0115] A determination module 340, configured to determine a target control instruction based on the recognition result and the processing result;

[0116] The execution module 350 is used to execute the control operation corresponding to the target control instruction.

[0117] Further, the determination module 340 is used to:

[0118] One of the recognition result and the processing result is selected and determined as the target control instruction.

[0119] Further, the determination module 340 is used to:

[0120] Determining the confidence level of the recognition result;

[0121] Based on the confidence of the recognition result, one of the recognition result and the processing result is selected to be determined as the target control instruction.

[0122] Further, the determination module 340 is used to:

[0123] If the confidence of the recognition result is greater than a set threshold, the recognition result is determined as a target control instruction;

[0124] If the confidence level of the recognition result is less than or equal to the set threshold, the processing result is determined as the target control instruction.

[0125] Further, the determination module 340 is used to:

[0126] If the confidence level of the recognition result is greater than a set threshold, and the interface currently displayed on the vehicle computer screen includes the control corresponding to the control data in the recognition result; the recognition result is determined as a target control instruction;

[0127] If the confidence of the recognition result is greater than a set threshold, and the interface currently displayed on the vehicle computer screen does not include the control corresponding to the control data in the recognition result; determine the processing result as a target control instruction;

[0128] If the confidence level of the recognition result is less than or equal to the set threshold, the processing result is determined as the target control instruction.

[0129] Furthermore, the identification module 320 is used to:

[0130] Loading a control instruction set corresponding to the application currently displayed on the vehicle computer; the control instruction set includes standard control instructions generated according to the interface control data of the application interface;

[0131] The standard control instruction in the control instruction set that matches the user control voice is used as the recognition result of the user control voice.

[0132] Further, the determination module 340 is used to:

[0133] If the user control voice is in a multi-round dialogue, the processing result is determined as a target control instruction.

[0134] The device disclosed in the above embodiments can implement the process of the method disclosed in the above method embodiments, and has the same or corresponding beneficial effects. To avoid repetition, it will not be described here.

[0135] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure is shown in FIG. Figure 6 As shown, the electronic device may include a mobile phone, a PAD, a car audio and video device and other intelligent terminals, and the electronic device includes:

[0136] One or more processors 301, Figure 6 A processor 301 is taken as an example;

[0137] Memory 302;

[0138] The electronic device may further include: an input device 303 and an output device 304 .

[0139] The processor 301, memory 302, input device 303 and output device 304 in the electronic device may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0140] The memory 302 is a non-transitory computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the voice control method in the embodiment of the present disclosure. The processor 301 executes various functional applications and data processing of the server by running the software programs, instructions and modules stored in the memory 302, that is, the voice control method of the above method embodiment is implemented.

[0141] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 302 may optionally include a memory remotely arranged relative to the processor 301, and these remote memories may be connected to the terminal device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0142] The input device 303 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 304 may include a display device such as a display screen.

[0143] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute a voice control method when executed, the method comprising:

[0144] Get user control voice;

[0145] Performing visible or audible recognition on the user control voice to form a recognition result;

[0146] Performing natural language understanding processing on the user control voice to form a processing result;

[0147] Determining a target control instruction based on the recognition result and the processing result;

[0148] Execute the control operation corresponding to the target control instruction.

[0149] Optionally, the computer executable instructions, when executed by a computer processor, can also be used to execute the technical solution of the voice control method provided in any embodiment of the present disclosure.

[0150] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present disclosure can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0151] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0152] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voice control method, It is characterized in that include: Get user control voice; Performing visible or audible recognition on the user control voice to form a recognition result; Performing natural language understanding processing on the user control voice to form a processing result; Determining a target control instruction based on the recognition result and the processing result; Execute the control operation corresponding to the target control instruction; The determining of the target control instruction based on the recognition result and the processing result includes: Determining the confidence level of the recognition result; Based on the confidence of the recognition result, one of the recognition result and the processing result is selected to be determined as the target control instruction.

2. The method according to claim 1, It is characterized in that The step of selecting one of the recognition result and the processing result as the target control instruction based on the confidence of the recognition result further includes: If the confidence of the recognition result is greater than a set threshold, the recognition result is determined as a target control instruction; If the confidence level of the recognition result is less than or equal to the set threshold, the processing result is determined as the target control instruction.

3. The method according to claim 1, It is characterized in that The step of selecting one of the recognition result and the processing result as the target control instruction based on the confidence of the recognition result further includes: If the confidence level of the recognition result is greater than a set threshold, and the interface currently displayed on the vehicle computer screen includes the control corresponding to the control data in the recognition result; the recognition result is determined as a target control instruction; If the confidence of the recognition result is greater than a set threshold, and the interface currently displayed on the vehicle computer screen does not include the control corresponding to the control data in the recognition result; determine the processing result as a target control instruction; If the confidence level of the recognition result is less than or equal to the set threshold, the processing result is determined as the target control instruction.

4. The method according to claim 1, It is characterized in that The visible or spoken recognition of the user control voice to form a recognition result includes: Loading a control instruction set corresponding to the application currently displayed on the vehicle computer; the control instruction set includes standard control instructions generated according to the interface control data of the application interface; The standard control instruction in the control instruction set that matches the user control voice is used as the recognition result of the user control voice.

5. The method according to claim 1, It is characterized in that Also includes: If the user control voice is in a multi-round dialogue, the processing result is determined as a target control instruction.

6. A voice control device, It is characterized in that include: An acquisition module, used to acquire user control voice; A recognition module, used for visually recognizing the user-controlled voice and forming a recognition result; A processing module, used for performing natural language understanding processing on the user control voice to form a processing result; A determination module, configured to determine a target control instruction based on the recognition result and the processing result; An execution module, used for executing the control operation corresponding to the target control instruction; A determination module, used to determine the confidence of the recognition result; Based on the confidence of the recognition result, one of the recognition result and the processing result is selected to be determined as the target control instruction.

7. An electronic device, It is characterized in that include: Processor and memory; The processor is used to execute the steps of the method according to any one of claims 1 to 5 by calling the program or instruction stored in the memory.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the method according to any one of claims 1 to 5.