Human-computer interaction method and device, electronic equipment, storage medium and vehicle
By identifying the vehicle cockpit interaction scene and sound area voiceprint, filtering voice data, generating and executing instructions, the problem of wake-up word dependence in the existing vehicle voice interaction system is solved, and natural and efficient human-computer interaction is achieved.
Patent Information
- Application Number
- CN202411207677.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-26
AI Technical Summary
The existing vehicle voice interaction system needs to use wake-up words to wake up the vehicle assistant first, resulting in unnatural, inefficient voice interaction, and may trigger wake-up by mistake, making it impossible to distinguish between user voice scenes and other interactions.
By obtaining vehicle cockpit interaction scene information, dividing voice zones, marking voice patterns, identifying voice data sources, filtering suspected human-computer interaction scenes, filtering voice data, generating instructions and requesting authorization before executing.
It realizes a natural entry into human-computer interaction, improves interaction efficiency, reduces the probability of false triggering, and distinguishes the interaction between user voice scenes and car-computer assistants.
Smart Images

Figure CN119025069A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of interaction, and in particular to a human-computer interaction method, a human-computer interaction device, an electronic device, a storage medium and a vehicle. Background Art
[0002] In the prior art, for the voice interaction system (such as the car assistant) on the current vehicle, if the user wants to interact with it by voice and give instructions, he must first use the wake-up word to wake up the voice. After the car assistant detects that the user has issued the wake-up word, it will feedback the information that the user can interact with, and then the user can give instructions to the car through voice sound, thereby meeting the user's needs.
[0003] This interaction method has defects. That is, when the user needs to use voice commands, he must first use the wake-up word to wake up. The voice interaction is not natural enough and the interaction efficiency is low. In addition, accidentally speaking the wake-up word may mistakenly trigger the wake-up of the voice interaction.
[0004] Therefore, a human-computer interaction solution is needed to distinguish whether the user's voice scenario is an interaction with the car assistant, an interaction with other people, or a phone call, so that the human-computer interaction can be naturally entered. Summary of the invention
[0005] The purpose of the present invention is to provide a human-computer interaction method, a human-computer interaction device, an electronic device, a storage medium and a vehicle, which at least solve the problem of distinguishing interaction scenes, the problem of entering human-computer interaction, and the problem of interpreting user instructions.
[0006] The present invention provides the following scheme:
[0007] According to one aspect of the present invention, a human-computer interaction method is provided, the human-computer interaction method comprising:
[0008] Obtain vehicle cockpit interaction scene information;
[0009] Classifying non-human-computer interaction and human-computer interaction scenarios according to the vehicle cockpit interaction scenario information;
[0010] Among them, according to the human-computer interaction scenario, the semantics of the user's voice translated into instructions are filtered;
[0011] According to the semantics of the instruction, an authorization request is sent before execution;
[0012] Execute instructions based on authorization.
[0013] Furthermore, the obtaining of vehicle cabin interaction scene information includes:
[0014] Divide multiple sound zones according to the seats in the vehicle cabin;
[0015] Mark multiple voiceprints between different voice zones to determine whether the voice data within a preset period comes from the same voice zone;
[0016] If,no, then the speech data is classified as speech data of non-human-computer interaction scenario;
[0017] If,yes, then the speech data is classified as speech data suspected of human-computer interaction,scenario.
[0018] Furthermore, the voice data classified as the voice data of the suspected human-computer interaction scene includes:
[0019] Get the status information of the interactive devices in the car;
[0020] The state information of the in-vehicle interactive device includes the working state of the device starting and receiving the voice data for processing the current audio zone;
[0021] According to the state information of the in-vehicle interactive device, identifying whether the device in the device working state of starting and receiving voice data for processing the current sound zone is a preset human-computer interactive device;
[0022] If not, the current voice data is classified as voice data for non-human-computer interaction scenarios.
[0023] Furthermore, it also includes:
[0024] According to the classification of voice data as voice data suspected of human-computer interaction scenarios, the module of voice recognition semantics is awakened;
[0025] The module of speech recognition semantics translates text information according to the semantics of speech recognition;
[0026] According to the text information, filtering texts that meet preset characteristics;
[0027] The filtering of texts meeting preset characteristics according to the text information includes setting text slot information;
[0028] The translated text information is filtered according to the text slot information.
[0029] Furthermore, filtering texts that meet preset characteristics includes:
[0030] Determining whether the text data translated based on the current voice data meets the preset characteristics of the imperative sentence;
[0031] If yes, then determining whether the text data translated based on the current voice data contains the preset features of the vehicle function title;
[0032] If yes, then determine whether the text data translated based on the current voice data contains preset features that meet the user's needs;
[0033] If yes, then generate command information for predicting and controlling the vehicle function according to the vehicle function title and the user's needs and wishes;
[0034] Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization;
[0035] Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
[0036] Furthermore, filtering texts that meet preset characteristics includes:
[0037] Determine whether the text data translated based on the current voice data contains the preset features of the vehicle assistant title;
[0038] If yes, then determining whether the text data translated based on the current voice data contains the preset features of the vehicle function title;
[0039] If yes, then determine whether the text data translated based on the current voice data contains preset features that meet the user's needs;
[0040] If yes, then generate command information for predicting and controlling the vehicle function according to the vehicle function title and the user's needs and wishes;
[0041] Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization;
[0042] Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
[0043] Furthermore, filtering texts that meet preset characteristics includes:
[0044] Determine whether the text data translated based on the current voice data includes one or more preset features that meet the user's needs;
[0045] If yes, then it is determined that the text data including the preset features that meet the user's needs and wishes in the text data translated based on the current voice data also includes the preset features that meet the imperative sentence;
[0046] If, yes, then the text data expressing the user's needs and intentions in an imperative sentence is used to generate command information for predicting the control of vehicle functions;
[0047] Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization;
[0048] Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
[0049] According to two aspects of the present invention, there is provided a human-computer interaction device, the human-computer interaction device comprising:
[0050] An information acquisition module, used to acquire vehicle cockpit interaction scene information;
[0051] A scene classification module, used to classify non-human-computer interaction and human-computer interaction scenes according to the vehicle cockpit interaction scene information;
[0052] The speech translation module is used to filter the semantics of user speech translated into instructions according to the human-computer interaction scenario;
[0053] An authorization request module, used to send an authorization request before execution according to the semantics of the instruction;
[0054] The instruction execution module is used to execute instructions according to the authorization obtained.
[0055] According to three aspects of the present invention, there is provided an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0056] A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the human-computer interaction method.
[0057] According to four aspects of the present invention, a computer-readable storage medium is provided, comprising: a computer program executable by an electronic device is stored therein, and when the computer program runs on the electronic device, the electronic device executes the steps of the human-computer interaction method.
[0058] According to five aspects of the present invention, there is provided a vehicle, comprising:
[0059] An electronic device, used to implement the steps of the human-computer interaction method;
[0060] A processor, the processor runs a program, and when the program runs, the steps of the human-computer interaction method are executed from the data output by the electronic device;
[0061] The storage medium is used to store a program, and when the program is running, the steps of the human-computer interaction method are executed for the data output from the electronic device.
[0062] Through the above scheme, the following beneficial technical effects are obtained:
[0063] The present application marks voiceprints in multiple audio intervals, divides audio zones, identifies human-computer interaction scenarios and non-human-computer interaction scenarios, screens voice data suspected of human-computer interaction scenarios, attempts human-computer interaction, and then naturally enters into interaction with the car assistant.
[0064] This application guides the process of voice interaction by setting the recognition features of voice interaction, and then naturally enters into subsequent multiple rounds of voice interaction.
[0065] This application distinguishes the voice interaction of a person using a communication device or other independently used voice interaction port from the voice interaction of the car assistant, thereby reducing the probability of erroneous access to human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 It is a flow chart of a human-computer interaction method provided by one or more embodiments of the present invention.
[0067] Figure 2 It is a structural diagram of a human-computer interaction device provided by one or more embodiments of the present invention.
[0068] Figure 3 A structural block diagram of an electronic device of a human-computer interaction method provided by one or more embodiments of the present invention. DETAILED DESCRIPTION
[0069] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0070] Figure 1 It is a flow chart of a human-computer interaction method provided by one or more embodiments of the present invention.
[0071] like Figure 1 The human-computer interaction methods shown include:
[0072] Step S1, obtaining vehicle cabin interaction scene information;
[0073] Step S2, classifying non-human-computer interaction and human-computer interaction scenarios according to the vehicle cockpit interaction scenario information;
[0074] Step S3, filtering the semantics of the user's voice translated into instructions according to the human-computer interaction scenario;
[0075] Step S4, sending an authorization request before execution according to the semantics of the instruction;
[0076] Step S5, executing the instruction according to the authorization obtained.
[0077] Specifically, in this embodiment, the vehicle assistant has the function of voice interaction, and human-computer interaction is realized by taking the vehicle assistant as the human-computer interaction object.
[0078] Through the vehicle cockpit interaction scene, human-computer interaction and non-human-computer interaction (the human-computer interaction between people and the car assistant is the human-computer interaction to be described). For example, if there are multiple voiceprint data in the cockpit, it is considered to be an interaction between people and classified as non-human-computer interaction. During human-computer interaction, the only two parties in the interaction should be the person and the car assistant to interact and promote instructions and responses.
[0079] When a human-computer interaction scenario occurs, the voice data is received and processed, such as recognizing semantic instructions.
[0080] Since there is no wake-up word, the semantics identified as commands need to be screened through semantic features to further determine whether the interpreted semantic command is the user's true intention. Before execution, the command can be executed after obtaining authorization by sending an authorization request.
[0081] As the number of voice interactions increases, machine learning is used to enhance the recognition of users' true intentions, thereby simplifying the form of sending authorization requests or even canceling them.
[0082] The authorization request can be sent in the form of a voice query, such as, "Are you sure you want to XXX?", which can be gradually simplified to "Are you ready?", and further simplified to "OK", and it is no longer necessary to request authorization for each instruction.
[0083] In this embodiment, obtaining vehicle cabin interaction scene information includes:
[0084] Divide multiple sound zones according to the seats in the vehicle cabin;
[0085] Mark multiple voiceprints between different voice zones to determine whether the voice data within a preset period comes from the same voice zone;
[0086] If,no, then the speech data is classified as speech data of non-human-computer interaction scenario;
[0087] If,yes, then the speech data is classified as speech data suspected of human-computer interaction,scenario.
[0088] Specifically, in this embodiment, the vehicle cabin is divided into multiple seats, and the voice of the passenger in each seat can form a unique voiceprint signal corresponding to each seat. According to the seats in the cabin, the sound zones are divided, and the voice of each sound zone is recognized through voiceprint recognition technology.
[0089] When the sounds are alternated between different sound zones, it can be considered as human-to-human interaction, not human-computer interaction. When the voiceprint signal is single and no other voiceprint signals appear within a preset time interval, the voice data can be classified as suspected human-computer interaction scene voice data. The data is further processed to confirm whether it is voice data of human-computer interaction scene.
[0090] In this embodiment, the voice data classified as the voice data suspected of human-computer interaction scene includes:
[0091] Get the status information of the interactive devices in the car;
[0092] The status information of the in-vehicle interactive device includes the working status of the device that starts and receives the voice data for processing the current audio zone;
[0093] According to the status information of the in-vehicle interactive device, identifying whether the device in the device working state of starting and receiving the voice data for processing the current audio zone is a preset human-computer interactive device;
[0094] If not, the current voice data is classified as voice data for non-human-computer interaction scenarios.
[0095] Specifically, a possibility cannot be ruled out that when a user uses a mobile phone to make a call, although there is only one voiceprint signal, at the same time, relevant information of the voiceprint information or status information of the mobile phone device can be detected on the mobile phone device.
[0096] For example, by detecting whether the mobile phone has activated the call line, or further checking whether the voice data converted on the mobile phone device is the same as the voice data of the current speaker, it is confirmed that the person is making a call, not a human-computer interaction with the car assistant. This scenario can be identified as a non-human-computer interaction scenario.
[0097] In this embodiment, it also includes:
[0098] According to the classification of voice data as voice data suspected of human-computer interaction scenarios, the module of voice recognition semantics is awakened;
[0099] The module of speech recognition semantics translates text information according to the semantics of speech recognition;
[0100] According to the text information, filter the text that meets the preset characteristics;
[0101] Filtering texts that meet preset characteristics according to text information includes setting text slot information;
[0102] Filter the translated text information according to the text slot information.
[0103] Specifically, in this embodiment, the voice data of the suspected human-computer interaction scene is further processed. Before a round of human-computer interaction occurs, the voice data of the suspected human-computer interaction scene cannot be determined as the voice data of the human-computer interaction scene.
[0104] For at least one human-computer interaction, the module for wake-up tone recognition semantics (the voice command recognition function of the car assistant starts to take effect and enters the semantic recognition of the voice representation). The module for voice recognition semantics translates text information according to the semantics of voice recognition, and filters text that meets the preset features to confirm that the voice data of the suspected human-computer interaction scene is the voice data of the human-computer interaction scene. For example, the text slot information is set according to the preset features, and the semantic text is filtered and translated.
[0105] In this embodiment, filtering text that meets the preset characteristics includes:
[0106] Determining whether the text data translated based on the current voice data meets the preset characteristics of the imperative sentence;
[0107] If yes, then determining whether the text data translated based on the current voice data contains the preset features of the vehicle function title;
[0108] If yes, then determine whether the text data translated based on the current voice data contains preset features that meet the user's needs;
[0109] If yes, then generate command information for predicting and controlling the vehicle function according to the vehicle function title and the user's needs and wishes;
[0110] Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization;
[0111] Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
[0112] Specifically, the text of preset features includes the text of preset features of imperative sentences. After the voice data is translated into text data, the text data is identified, such as determining whether the text data meets the preset features of the imperative sentence. If it is an imperative sentence, it can be understood as a "command" or "request". The corresponding imperative sentence has more directionality in triggering the corresponding function, but for the "machine", it is necessary to use machine learning methods and large model technology to optimize and strengthen it in order to improve the accuracy and reliability of the directionality, and further filter whether the text data contains the preset features of the vehicle function title, whether it contains the preset features that meet the user's needs, etc., to further clarify the user's instruction intentions and the objects and functions pointed to.
[0113] In this embodiment, the first level of judgment and filtering starts from the preset features of the imperative sentence, and then passes through the second level of filtering of the preset features of the vehicle function title and the third level of filtering of the preset features that meet the user's needs and intentions, and finally generates instruction information for predicting and controlling the vehicle function based on the vehicle function title and the user's needs and intentions.
[0114] In another embodiment, it is not necessary to go through three levels of screening, nor is it necessary to screen the text data using the three preset features mentioned above, and the order of screening does not necessarily have to start from judging the filtering based on the preset features of the imperative sentence.
[0115] In this embodiment, filtering text that meets the preset characteristics includes:
[0116] Determine whether the text data translated based on the current voice data contains the preset features of the vehicle assistant title;
[0117] If yes, then determining whether the text data translated based on the current voice data contains the preset features of the vehicle function title;
[0118] If yes, then determine whether the text data translated based on the current voice data contains preset features that meet the user's needs;
[0119] If yes, then generate command information for predicting and controlling the vehicle function according to the vehicle function title and the user's needs and wishes;
[0120] Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization;
[0121] Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
[0122] Specifically, due to the richness of voice data and the directionality of the commands of the car assistant in the cockpit environment, it is highly likely that the commands are related to vehicle functions. In this embodiment, the first layer of judgment and filtering starts from the preset features of the vehicle function titles, and then passes through the second layer of filtering of the preset features that meet the user's needs and wishes, and finally generates command information for predicting and controlling vehicle functions based on the vehicle function titles and user needs and wishes.
[0123] In this embodiment, filtering text that meets the preset characteristics includes:
[0124] Determine whether the text data translated based on the current voice data includes one or more preset features that meet the user's needs;
[0125] If yes, then it is determined that the text data including the preset features that meet the user's needs and wishes in the text data translated based on the current voice data also includes the preset features that meet the imperative sentence;
[0126] If, yes, then the text data expressing the user's needs and intentions in an imperative sentence is used to generate command information for predicting the control of vehicle functions;
[0127] Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization;
[0128] Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
[0129] Specifically, under the premise that the personnel do not have a deep understanding of the vehicle functions, or are not very concerned about the functions of the vehicle, the recognition of the expression of intention is the first priority. In this embodiment, the first layer of judgment and filtering starts from whether the text data includes one or more preset features that meet the user's needs and intentions, and then passes through the second layer of filtering including preset features that meet the imperative sentence, and finally outputs the command information in the form of an imperative sentence based on the user's needs and intentions, generating a pre-judgment command to control the vehicle function.
[0130] In the above embodiments, with the optimization and enhancement using machine learning methods and large model technologies, the accuracy and reliability of the human-computer interaction recognition and analysis of personnel's instruction intentions are continuously enhanced, and some scenarios requiring authorization can be further simplified, or the analysis conclusions under some statements and the resulting vehicle function control can bypass the part requiring authorization.
[0131] For example, setting the authorization number threshold for human-computer interaction commands;
[0132] According to the human-computer interaction statements with the same or similar preset characteristics, authorization is requested and obtained. There is no record of not obtaining authorization within the preset time interval, and the number of authorizations obtained exceeds the authorization number threshold of the human-computer interaction instruction. According to the preset simplified authorization request strategy, the steps of requesting authorization are simplified.
[0133] For example, based on a human-computer interaction statement with the same or similar preset features, authorization is requested and obtained, there is no record of not obtaining authorization within a preset time interval, and the number of authorizations obtained exceeds the authorization number threshold of the human-computer interaction instruction, according to the preset strategy of skipping the request for authorization, the step of skipping the request for authorization is skipped, and the self-judgment instruction is directly executed.
[0134] After executing the self-judgment instruction, the next round of human-computer interaction speech is judged and interpreted to see whether it is a correction to the previous self-judgment instruction;
[0135] If yes, then exit the preset policy of bypassing the request authorization, and reset the data that has no unauthorized record within the preset time interval and the number of authorizations exceeds the authorization number threshold of the human-computer interaction instruction to zero.
[0136] It also includes sending data for judging and interpreting whether the voice of the next round of human-computer interaction is a correction to the previous self-judgment instruction after executing the self-judgment instruction to the cloud data model for training, so as to enhance the recognition of user intentions.
[0137] Since everyone's language style and accent are different, human-computer interaction has certain personalized elements. With the help of machine learning methods and large model technology for training, the accuracy and reliability of identifying human-computer interaction scenarios and non-human-computer interaction scenarios are further enhanced, as well as the ability to analyze the intention of personnel in human-computer interaction under different language styles and accents.
[0138] In this embodiment, after obtaining authorization to execute the pre-judged instructions for controlling the vehicle function, the voice data suspected of being a human-computer interaction scene can be confirmed as the voice data of the human-computer interaction scene. Before obtaining authorization, doubts about the voice data of the human-computer interaction scene cannot be eliminated. Although the data processing process is carried out in accordance with the voice data of the human-computer interaction scene, the result of not obtaining authorization may be that the car assistant misunderstands the user's intention, and the voice data may not be the voice data of the human-computer interaction scene at all.
[0139] If it can be confirmed that the voice data is from a human-computer interaction scenario, but the lack of authorization can be regarded as the car assistant misunderstanding the user's intention, and the interaction process can be increased to enhance the positioning of the user's intention. However, if the voice data is not from a human-computer interaction scenario, increasing the interaction process and enhancing the positioning of the user's intention will have a counterproductive effect. Therefore, for reasons of caution, as a suspected human-computer interaction scenario, the voice data processing attempts to obtain authorization. If authorization is not obtained, it will not affect the nature of the voice data in the suspected human-computer interaction scenario, and other strategies can be triggered. For example, after the first round of voice interaction, if authorization is not obtained, the collection, filtering and interpretation of environmental information is triggered to determine whether there is an interference source. If so, the command information for predicting the control of vehicle functions is output on the human-computer terminal again based on the preset features, and authorization is requested; if not, the data is sent to the large model of the cloud server as a sample to train the large model.
[0140] Figure 2 It is a structural diagram of a human-computer interaction device provided by one or more embodiments of the present invention.
[0141] like Figure 2 The human-computer interaction device includes: an information acquisition module, a scene classification module, a speech translation module, an authorization request module, and an instruction execution module;
[0142] An information acquisition module, used to acquire vehicle cockpit interaction scene information;
[0143] A scene classification module is used to classify non-human-computer interaction and human-computer interaction scenes according to the vehicle cockpit interaction scene information;
[0144] The speech translation module is used to filter the semantics of user speech translated into instructions according to the human-computer interaction scenario;
[0145] An authorization request module, used to send an authorization request before execution according to the semantics of the instruction;
[0146] The instruction execution module is used to execute instructions according to the authorization obtained.
[0147] It is worth noting that although the present system only discloses an information acquisition module, a scene classification module, a speech translation module, an authorization request module, and an instruction execution module, it does not mean that the present device is limited to the above-mentioned basic functional modules. Rather, what the present invention wants to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. In other words, the present system is open rather than closed. Just because the present embodiment only discloses individual basic functional modules, it cannot be considered that the scope of protection of the claims of the present invention is limited to the above-mentioned basic functional modules.
[0148] Through the above scheme, the following beneficial technical effects are obtained:
[0149] The present application marks voiceprints in multiple audio intervals, divides audio zones, identifies human-computer interaction scenarios and non-human-computer interaction scenarios, screens voice data suspected of human-computer interaction scenarios, attempts human-computer interaction, and then naturally enters into interaction with the car assistant.
[0150] This application guides the process of voice interaction by setting the recognition features of voice interaction, and then naturally enters into subsequent multiple rounds of voice interaction.
[0151] This application distinguishes the voice interaction of a person using a communication device or other independently used voice interaction port from the voice interaction of the car assistant, thereby reducing the probability of erroneous access to human-computer interaction.
[0152] In a specific embodiment, the client and the cloud are combined to determine whether the user wants to interact with the vehicle. If the decision result is "non-human-computer interaction", the received audio data is ignored and the vehicle (vehicle assistant) does not respond. If the decision result is "human-computer interaction", the user's instructions are parsed, the voice image is evoked and displayed to the user, and the action required by the user is completed.
[0153] For the decision-making process, the voice client on the vehicle will first perform a "preliminary screening". If the "preliminary screening" passes, the audio data will be sent to the cloud, allowing the cloud to make further judgments and analyze the user's instructions. If the cloud passes the second confirmation, the cloud will reply the parsed instructions to the client, and the client will complete the action specified by the user after receiving the instruction.
[0154] The "initial screening" action of the voice client on the car computer is a very critical skill. Without this initial screening function, all the audio on the car computer will be sent to the cloud, which will consume a huge amount of network traffic and cause serious waste of car computer resources.
[0155] The execution process in this embodiment is as follows:
[0156] Step 1: The voice client opens Android's AudioRecord to receive the audio data streams of the vehicle's four audio zones.
[0157] When the voice client is started, it reads the audio data streams from the microphones in the vehicle's four sound zones (driver's seat, co-driver's seat, and the left and right rear seats), and sends the data streams directly to the voice client's "wake-up-free decision" module for further processing.
[0158] Step 2: The "wake-up-free judgment" module performs preliminary screening based on the audio data of the current four locations to roughly determine whether the user wants to perform human-computer interaction.
[0159] 1. First, filter the audio data streams of the four sound zones, extract the human voice, and determine the current main sound zone based on the strength of the human voice in different sound zones. If the human voice cannot be recognized, it is impossible to be human-computer interaction.
[0160] 2. According to the extracted vocal and silent segments, the vocal data is segmented and sentence-divided.
[0161] 3. For each segment of human voice, identify its voiceprint, and through the voiceprint, you can identify different speakers. For each segment of human voice, mark the "speaker".
[0162] 4. Based on the speakers of the previous and subsequent voices, determine whether the current user is "speaking independently" (perhaps wanting human-computer interaction) or "multiple speakers talking" (in this case, the people in the car are chatting, not human-computer interaction).
[0163] 5. If the user is "speaking independently" at this time, it is preliminarily determined that the user wants to conduct "human-computer interaction".
[0164] Step 3: If step 2 determines that "human-computer interaction" is to be performed, then the audio of the user's command is sent to the cloud for secondary confirmation to determine whether the user really wants to perform human-computer interaction.
[0165] 1. After receiving the audio data sent by the voice client, the cloud will parse the audio to find out the user's "intention" and "voice text" expressed by the current audio data.
[0166] 2. Based on the "intention" and "voice text", determine whether the user currently wants to perform "human-computer interaction" (the judgment here is a scoring result, and the "items" considered in the scoring generally include but are not limited to:
[0167] Is it an imperative sentence?
[0168] Did you mention the “name of the car computer”?
[0169] Is it a commonly used function of the vehicle?
[0170] How strong is the user's intent? (For example, "Give me directions to Zhongguancun" is a much stronger intent than "I want to go to Zhongguancun")
[0171] Does it match the user's instruction set? (Use the big model to train the user's vehicle instruction set)
[0172] Step 4: If the cloud determines that the user wants "human-computer interaction" at this time, it will let the voice client on the car computer execute the user's instructions and complete the function the user wants.
[0173] After receiving the command from the cloud, the voice client on the car starts to execute the user's command if it is "human-computer interaction". At this point, the traditional wake-up process has been "skipped" and the user's command is directly executed and completed.
[0174] In another specific embodiment, scenario 1, the driver and the co-driver are chatting. At this time, the voice client on the vehicle computer will execute to the fourth point of step 2 above, and judge that multiple voiceprints and multiple users are speaking. At this time, it is determined to be non-human-computer interaction and the user's instructions will not be executed. In addition, the vehicle computer network traffic will not be consumed.
[0175] In another specific embodiment, scenario 2, the driver is alone in the car and makes a phone call, saying "I plan to go to Zhongguancun tomorrow, xxx". At this time, the voice client on the car computer will execute the second point of step 3 above. The cloud determines that although the current user is speaking, there is no strong intention. Therefore, it is judged as non-human-computer interaction and the user instruction will not be executed.
[0176] Figure 3 A structural block diagram of an electronic device of a human-computer interaction method provided by one or more embodiments of the present invention.
[0177] like Figure 3 As shown, the present application provides an electronic device, including: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;
[0178] A computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the steps of a human-computer interaction method.
[0179] The present application also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of a human-computer interaction method.
[0180] The present application also provides a vehicle, comprising:
[0181] Electronic device, used in the steps of human-computer interaction method;
[0182] A processor, wherein the processor runs a program, and when the program runs, the steps of the human-computer interaction method are executed from the data output by the electronic device;
[0183] The storage medium is used to store a program, and when the program is running, it executes the steps of the human-computer interaction method for the data output from the electronic device.
[0184] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0185] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through a process, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. In addition, in an embodiment of the present invention, the electronic device can be a handheld device such as a smart phone or a tablet computer, or can be an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiment of the present invention.
[0186] The execution subject of the electronic device control in the embodiment of the present invention may be an electronic device, or a functional module in the electronic device that can call and execute a program. The electronic device may obtain the firmware corresponding to the storage medium. The firmware corresponding to the storage medium is provided by the supplier. The firmware corresponding to different storage media may be the same or different, which is not limited here. After the electronic device obtains the firmware corresponding to the storage medium, the firmware corresponding to the storage medium may be written into the storage medium, specifically, the firmware corresponding to the storage medium may be burned into the storage medium. The process of burning the firmware into the storage medium may be implemented using existing technology, which will not be described in detail in the embodiment of the present invention.
[0187] The electronic device may also obtain a reset command corresponding to the storage medium. The reset command corresponding to the storage medium is provided by the supplier. The reset commands corresponding to different storage media may be the same or different, and are not limited here.
[0188] At this time, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented by the existing technology and will not be described in detail in the embodiments of the present invention.
[0189] For the convenience of description, the above devices are described in terms of functions and are divided into various units and modules. Of course, when implementing the present application, the functions of each unit and module can be implemented in the same or multiple software and / or hardware.
[0190] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined.
[0191] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0192] It can be known from the description of the above implementation modes that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute the methods described in the various implementation modes of the present application or certain parts of the implementation modes.
[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A human-computer interaction method, It is characterized in that The human-computer interaction method comprises: Obtain vehicle cockpit interaction scene information; Classifying non-human-computer interaction and human-computer interaction scenarios according to the vehicle cockpit interaction scenario information; Among them, according to the human-computer interaction scenario, the semantics of the user's voice translated into instructions are filtered; According to the semantics of the instruction, an authorization request is sent before execution; Execute instructions based on authorization.
2. The human-computer interaction method according to claim 1, It is characterized in that The obtaining of vehicle cabin interaction scene information includes: Divide multiple sound zones according to the seats in the vehicle cabin; Mark multiple voiceprints between different voice zones to determine whether the voice data within a preset period comes from the same voice zone; If,no, then the speech data is classified as speech data of non-human-computer interaction scenario; If,yes, then the speech data is classified as speech data suspected of human-computer interaction,scenario.
3. The human-computer interaction method according to claim 2, It is characterized in that The voice data classified as the voice data suspected of human-computer interaction scene includes: Get the status information of the interactive devices in the car; The state information of the in-vehicle interactive device includes the working state of the device starting and receiving the voice data for processing the current audio zone; According to the state information of the in-vehicle interactive device, identifying whether the device in the device working state of starting and receiving voice data for processing the current sound zone is a preset human-computer interactive device; If not, the current voice data is classified as voice data for non-human-computer interaction scenarios.
4. The human-computer interaction method according to claim 3, It is characterized in that Also includes: According to the classification of voice data as voice data suspected of human-computer interaction scenarios, the module of voice recognition semantics is awakened; The module of speech recognition semantics translates text information according to the semantics of speech recognition; According to the text information, filtering texts that meet preset characteristics; The filtering of texts meeting preset characteristics according to the text information includes setting text slot information; The translated text information is filtered according to the text slot information.
5. The human-computer interaction method according to claim 4, It is characterized in that The filtering of texts meeting the preset characteristics includes: Determining whether the text data translated based on the current voice data meets the preset characteristics of the imperative sentence; If yes, then determining whether the text data translated based on the current voice data contains the preset features of the vehicle function title; If yes, then determine whether the text data translated based on the current voice data contains preset features that meet the user's needs; If yes, then generate command information for predicting and controlling the vehicle function according to the vehicle function title and the user's needs and wishes; Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization; Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
6. The human-computer interaction method according to claim 4, It is characterized in that The filtering of texts meeting the preset characteristics includes: Determine whether the text data translated based on the current voice data contains the preset features of the vehicle assistant title; If yes, then determining whether the text data translated based on the current voice data contains the preset features of the vehicle function title; If yes, then determine whether the text data translated based on the current voice data contains preset features that meet the user's needs; If yes, then generate command information for predicting and controlling the vehicle function according to the vehicle function title and the user's needs and wishes; Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization; Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
7. The human-computer interaction method according to claim 4, It is characterized in that The filtering of texts meeting the preset characteristics includes: Determine whether the text data translated based on the current voice data includes one or more preset features that meet the user's needs; If yes, then it is determined that the text data including the preset features that meet the user's needs and wishes in the text data translated based on the current voice data also includes the preset features that meet the imperative sentence; If, yes, then the text data expressing the user's needs and intentions in an imperative sentence is used to generate command information for predicting the control of vehicle functions; Output the command information for predicting and controlling the vehicle function on the human-machine terminal and request authorization; Based on the authorization obtained, the predicted instructions for controlling vehicle functions are executed.
8. A human-computer interaction device, It is characterized in that The human-computer interaction device comprises: An information acquisition module, used to acquire vehicle cockpit interaction scene information; A scene classification module, used to classify non-human-computer interaction and human-computer interaction scenes according to the vehicle cockpit interaction scene information; The speech translation module is used to filter the semantics of user speech translated into instructions according to the human-computer interaction scenario; An authorization request module, used to send an authorization request before execution according to the semantics of the instruction; The instruction execution module is used to execute instructions according to the authorization obtained.
9. An electronic device, It is characterized in that include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the human-computer interaction method according to any one of claims 1 to 7.
10. A computer-readable storage medium, It is characterized in that include: It stores a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of the human-computer interaction method described in any one of claims 1 to 7.
Citation Information
Cited By
Speech interaction method and apparatus, and vehicle
WO2026170398A1
