Vehicle function execution method based on voice instruction, vehicle-mounted terminal and storage medium

Abstract

The invention provides a vehicle function execution method based on a voice instruction, a vehicle-mounted terminal and a storage medium, and relates to the technical field of vehicles. When the similarity between the voice wake-up scene feature and the historical voice wake-up scene feature is higher than the set similarity threshold, the possibility that the user initiates the same voice instruction as the historical voice wake-up scene feature is high. Therefore, the voice instructions which are input by the user in history and correspond to the historical voice wake-up scene features with the similarity to the voice wake-up scene features higher than the set similarity threshold value can be searched from the preset database, and the multiple searched voice instructions are displayed on a screen of the vehicle-mounted terminal in a text mode in sequence; and in response to the selection operation of the target voice instruction in the multiple displayed voice instructions input by the user, the vehicle is controlled to execute the vehicle function associated with the target voice instruction, so that the steps of the user are saved, and the accuracy of executing the corresponding vehicle function by the vehicle according to the content of the voice instruction is improved.

Description

Technical Field

[0001] The present application relates to the technical field of vehicles, and in particular, to a method for executing vehicle functions based on voice commands, an in-vehicle terminal, and a storage medium. Background Art

[0002] With the development of technology, the intelligent cockpit system is making great strides towards increasing intelligence. Among them, the in-vehicle voice assistant is a convenient intelligent software, enabling users to control the vehicle to execute corresponding vehicle functions (such as navigation functions, music playback functions, and adjusting the temperature of the in-vehicle air conditioner, etc.) through voice.

[0003] Currently, the way for users to control the execution of corresponding vehicle functions through voice is that after the user wakes up the voice assistant, the user needs to give a voice command, and the in-vehicle terminal recognizes the voice command. After successful recognition, the vehicle function corresponding to the voice command is executed. However, due to the recognition of voice commands being affected by the user's tone and pitch as well as the sound environment where the user is located, if the content of the voice command is relatively long, such as "Please navigate to the intersection of West Zhongshan Road and North Nanjing Road", it is easy for the vehicle terminal to misrecognize the voice command, resulting in the failure of the vehicle to execute the corresponding vehicle function according to the misrecognized voice command content, and the user experience is poor. Summary of the Invention

[0004] The present application provides a method for executing vehicle functions based on voice commands, an in-vehicle terminal, and a storage medium, which are used to solve the problem in the prior art that the vehicle misrecognizes the content of the voice command, resulting in the failure of executing the corresponding vehicle function and poor user experience.

[0005] In a first aspect, a method for executing vehicle functions based on voice commands provided by the present application is applied to an in-vehicle terminal. The method provided by the present application includes:

[0006] In response to the user's wake-up operation on the voice assistant, wake up the preset voice assistant;

[0007] Obtain the user's identity feature, the vehicle's positioning feature, and the time feature of waking up the voice assistant;

[0008] Fuse the user's identity feature, the vehicle's positioning feature, and the time feature of waking up the voice assistant to obtain a voice wake-up scene feature;

[0009] Search in the preset database for the voice commands input by the user corresponding to the historical voice wake-up scene features whose similarity to the voice wake-up scene feature is higher than the set similarity threshold;

[0010] Sequentially display the multiple found voice commands on the screen of the in-vehicle terminal in text form;

[0011] In response to a selection operation of a target voice command among multiple displayed voice commands input by a user, control the vehicle to execute a vehicle function associated with the target voice command.

[0012] In some embodiments, multiple found voice commands are displayed in text form on the screen of the in-vehicle terminal in sequence, including:

[0013] Obtain the number of times each of the found voice commands has been used historically.

[0014] For each voice command, determine the user's interest level in the voice command based on the number of times the voice command has been used and the similarity between the corresponding historical voice wake-up scenario features and the voice wake-up scenario features, where both the number of times used and the similarity are positively correlated with the interest level.

[0015] Display the multiple found voice commands in text form on the screen of the in-vehicle terminal in descending order according to the interest level of each voice command.

[0016] In some embodiments, determining the interest level of a voice command based on the number of times the voice command has been used and the similarity between the corresponding historical voice wake-up scenario features and the voice wake-up scenario features includes:

[0017] According to the number of times the voice command has been used, look up the corresponding first sub-interest level in a preset first relationship table, where the number of times used is positively correlated with the first sub-interest level.

[0018] According to the similarity between the historical voice wake-up scenario features of the voice command and the voice wake-up scenario features, look up the corresponding second sub-interest level in a preset second relationship table, and the similarity is positively correlated with the second sub-interest level.

[0019] Perform a weighted average on the first sub-interest level and the second sub-interest level to obtain the interest level of the voice command.

[0020] In some embodiments, the method provided in this application further includes: after responding to a selection operation of a target voice command among multiple displayed voice commands input by a user, increment the number of times the target voice command has been used by one.

[0021] In some embodiments, obtaining the user's identity characteristics includes:

[0022] Determine the user's orientation in the vehicle according to the way the user wakes up the voice assistant.

[0023] Control the in-vehicle camera to take a picture in the direction of the user to obtain an image containing the user's face as the user's identity characteristic.

[0024] In some embodiments, determining the orientation of a user inside a vehicle according to the way the user wakes up the voice assistant includes:

[0025] When it is determined that the wake-up operation of the user on the voice assistant is the case where the user triggers the wake-up button on the vehicle's steering wheel, determine that the orientation of the user inside the vehicle is the driver's seat.

[0026] In some embodiments, determining the orientation of a user inside a vehicle according to the way the user wakes up the voice assistant includes:

[0027] When it is determined that the wake-up operation of the user on the voice assistant is the case where the user inputs wake-up voice data to a plurality of microphones arranged in an array in the vehicle, according to the preset geometric layout of the microphones, the phase difference and time difference of the wake-up voice data reaching each microphone, and the preset sound source localization algorithm, identify the orientation of the user who inputs the wake-up voice data.

[0028] In a second aspect, the present application further provides an in-vehicle terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the in-vehicle terminal executes the method provided in the first aspect of the present application.

[0029] In a third aspect, the present application further provides a storage medium storing a computer program, and when the computer program is executed by a processor, the computer executes the method provided in the first aspect of the present application.

[0030] In a fourth aspect, the present application further provides a computer program product including a computer program, and when the computer program is run, the in-vehicle terminal executes the method provided in the first aspect of the present application.

[0031] The present application provides a method for executing vehicle functions based on voice commands, an in-vehicle terminal, and a storage medium. The identity characteristics of the user, the positioning characteristics of the vehicle, and the time characteristics of waking up the voice assistant are fused to obtain voice wake-up scenario characteristics. It can be understood that when the similarity between the voice wake-up scenario characteristics and the historical voice wake-up scenario characteristics is higher than the set similarity threshold, the possibility that the user is the same as the one who initiated the same voice command in history is high. Therefore, it is possible to search in a preset database for the voice commands input by the user corresponding to the historical voice wake-up scenario characteristics whose similarity to the voice wake-up scenario characteristics is higher than the set similarity threshold, and sequentially display the multiple found voice commands in text form on the screen of the in-vehicle terminal; in response to a selection operation input by the user for a target voice command among the multiple displayed voice commands, control the vehicle to execute the vehicle function associated with the target voice command. In this way, without the user issuing a voice command and without the in-vehicle terminal having to recognize the content of the voice command, it saves the user's steps while improving the accuracy of the vehicle executing the corresponding vehicle function according to the content of the voice command. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0033] Figure 1 It is a flowchart of the method for executing vehicle functions based on voice commands provided by an embodiment of the present application;

[0034] Figure 2 is Figure 1 a detailed flowchart of S105 in

[0035] Figure 3 It is a functional module block diagram of the device for executing vehicle functions based on voice commands provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0037] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, where for the purpose of clear expression, some details are enlarged and some details may be omitted. The shapes of various regions and layers shown in the figures, as well as their relative sizes and positional relationships, are merely exemplary. In practice, there may be deviations due to manufacturing tolerances or technical limitations, and those skilled in the art can design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0038] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element, or there may be an intermediate layer / element between them. Additionally, if a layer / element is "on" another layer / element in one orientation, then when the orientation is reversed, the layer / element may be "under" the other layer / element.

[0039] Next, the technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0040] Please refer to Figure 1 , a method for executing vehicle functions based on voice commands provided by an embodiment of the present application is applied to an in-vehicle terminal. Among them, the in-vehicle terminal may be, but is not limited to, an in-vehicle computer ECU (electronic control unit, ECU). Application programs such as a voice assistant, a music application, a navigation application, and a shopping application are installed on the in-vehicle terminal. As Figure 1 shown, the method provided by an embodiment of the present application includes:

[0041] S101: In response to a user's wake-up operation on the voice assistant, wake up the preset voice assistant.

[0042] Exemplarily, the ways for the user to wake up the voice assistant include, but are not limited to, the following two:

[0043] The first one: The user triggers the wake-up button on the vehicle's steering wheel as the wake-up operation on the voice assistant.

[0044] The second one: The user inputs wake-up voice data (such as "Hello, my car") to the multiple microphones arranged in an array in the vehicle as the wake-up operation on the voice assistant.

[0045] S102: Obtain the user's identity characteristics, the vehicle's positioning characteristics, and the time characteristics of waking up the voice assistant.

[0046] In some embodiments, the method for obtaining the user's identity features may specifically be to determine the user's position in the vehicle according to the way the user wakes up the voice assistant; control the in-vehicle camera to capture in the direction of the user to obtain an image including the user's face as the user's identity features.

[0047] For example, when it is determined that the user's wake-up operation for the voice assistant is to trigger the wake-up button on the vehicle's steering wheel, it is determined that the user's position in the vehicle is the driver's seat.

[0048] For another example, when it is determined that the user's wake-up operation for the voice assistant is to input wake-up voice data to multiple microphones arranged in an array in the vehicle, according to the preset geometric layout of the microphones, the phase difference and time difference of the wake-up voice data reaching each microphone, and a preset sound source localization algorithm (such as a beamforming algorithm or a minimum mean square error algorithm, etc.), the position of the user who inputs the wake-up voice data is identified.

[0049] In addition, the positioning features of the vehicle can be obtained through the in-vehicle GPS, and the time features of waking up the voice assistant can be recorded through the in-vehicle timing device.

[0050] S103: Fuse the user's identity features, the vehicle's positioning features, and the time features of waking up the voice assistant to obtain voice wake-up scene features.

[0051] For example, convert the user's identity features into a body feature space vector, convert the vehicle's positioning features into a positioning feature space vector, and convert the time features of waking up the voice assistant into a time feature space vector, and splice the body feature space vector, the positioning feature space vector, and the time feature space vector to obtain multi-dimensional voice wake-up scene features.

[0052] S104: From a preset database, search for the voice commands input by the user corresponding to the historical voice wake-up scene features whose similarity to the voice wake-up scene features is higher than a set similarity threshold.

[0053] It can be understood that under the same voice wake-up scene features, the probability that the user inputs the same voice command is very high.

[0054] S105: Display the multiple found voice commands in text form on the screen of the in-vehicle terminal in sequence.

[0055] Exemplarily, S105 can be specifically implemented as:

[0056] S201: Obtain the number of times each of the multiple found voice commands has been used historically.

[0057] S202: For each voice command, determine the user's interest level in the voice command based on the number of times the voice command has been used and the similarity between the corresponding historical voice wake-up scene features and the voice wake-up scene features.

[0058] Understandably, the lower the number of times the voice command has been used and the higher the similarity, the greater the likelihood that the voice command will be used under the current voice wake-up scene features. Therefore, both the number of times used and the similarity are positively correlated with the interest level.

[0059] S203: According to the interest levels of each voice command, display the multiple found voice commands in text form on the screen of the in-vehicle terminal in descending order.

[0060] For example, the 1st voice command is "Please navigate to the company", the 2nd voice command is "Please play music", the 3rd voice command is "Please navigate to People's Park", and the 4th voice command is "Please open the driver's window". Exemplarily, S203 can be specifically implemented as follows:

[0061] Step 1: According to the number of times the voice command has been used, find the first sub-interest level corresponding to the number of times used from a preset first relationship table, where the number of times used is positively correlated with the first sub-interest level.

[0062] Step 2: According to the similarity between the historical voice wake-up scene features of the voice command and the voice wake-up scene features, find the second sub-interest level corresponding to the similarity from a preset second relationship table, and the similarity is positively correlated with the second sub-interest level.

[0063] Step 3: Perform a weighted average on the first sub-interest level and the second sub-interest level to obtain the interest level of the voice command.

[0064] For example, according to the formula Q = K 1 Q 1 +K 2 Q 2 , determine the interest level of the voice command, where Q is the interest level of the voice command, Q 1 is the first sub-interest level, Q 2 is the second sub-interest level, K 1 is the first weighting coefficient, K 2 is the second weighting coefficient. For example, K 1 is 0.6, and K 2 is 0.4.

[0065] In some embodiments, the method provided by the embodiments of the present application further includes: after responding to the selection operation of the target voice command among the multiple displayed voice commands input by the user, incrementing the number of times the target voice command has been used, further enhancing the user's interest in the selected voice command.

[0066] S106: In response to the selection operation of the target voice command among the multiple displayed voice commands input by the user, control the vehicle to execute the vehicle function associated with the target voice command.

[0067] For example, execute the function of navigating to the company, or the function of playing music, etc., which is not limited herein.

[0068] The present application provides a method for executing a vehicle function based on a voice command, which fuses the identity characteristics of the user, the positioning characteristics of the vehicle, and the time characteristics of waking up the voice assistant to obtain the voice wake-up scene characteristics. It can be understood that when the similarity between the voice wake-up scene characteristics and the historical voice wake-up scene characteristics is higher than the set similarity threshold, the possibility that the user issues the same voice command as in history is high. Therefore, it is possible to search in the preset database for the voice commands input by the user corresponding to the historical voice wake-up scene characteristics whose similarity to the voice wake-up scene characteristics is higher than the set similarity threshold, and display the multiple found voice commands in text form on the screen of the in-vehicle terminal in sequence; in response to the selection operation of the target voice command among the multiple displayed voice commands input by the user, control the vehicle to execute the vehicle function associated with the target voice command. In this way, without the user issuing a voice command and without the in-vehicle terminal having to recognize the content of the voice command, it saves the user's steps and improves the accuracy of the vehicle executing the corresponding vehicle function according to the content of the voice command.

[0069] Please refer to Figure 3 , the embodiments of the present application also provide a device for executing a vehicle function based on a voice command, which is applied to an in-vehicle terminal. It should be noted that the basic principle and the technical effects generated by the device for executing a vehicle function based on a voice command provided by the embodiments of the present application are the same as those of the above embodiments. For a brief description, for the parts not mentioned in the embodiments of the present application, reference can be made to the corresponding content in the above embodiments. The device provided by the embodiments of the present application includes a voice assistant wake-up unit, a feature acquisition unit, a feature fusion unit, a command search unit, a command display unit, and a function execution unit, wherein,

[0070] The voice assistant wake-up unit is used to wake up the preset voice assistant in response to the user's wake-up operation on the voice assistant.

[0071] The feature acquisition unit is used to obtain the identity characteristics of the user, the positioning characteristics of the vehicle, and the time characteristics of waking up the voice assistant.

[0072] A feature fusion unit, configured to fuse the identity features of the user, the positioning features of the vehicle, and the time features of waking up the voice assistant to obtain voice wake-up scene features.

[0073] An instruction search unit, configured to search, from a preset database, for the voice instructions input by the user in history corresponding to the historical voice wake-up scene features whose similarity to the voice wake-up scene features is higher than a set similarity threshold.

[0074] An instruction display unit, configured to sequentially display the multiple searched voice instructions on the screen of the in-vehicle terminal in text form.

[0075] A function execution unit, configured to, in response to a selection operation of the user on a target voice instruction among the multiple displayed voice instructions, control the vehicle to execute a vehicle function associated with the target voice instruction.

[0076] In some embodiments, the instruction display unit is specifically configured to obtain the number of times the multiple searched voice instructions have been used in history respectively; for each voice instruction, determine the user's interest in the voice instruction according to the number of times the voice instruction has been used and the similarity between the corresponding historical voice wake-up scene features and the voice wake-up scene features, where both the number of times used and the similarity are positively correlated with the interest; and display the multiple searched voice instructions on the screen of the in-vehicle terminal in text form in a descending order according to the interest of each voice instruction.

[0077] In some embodiments, the instruction display unit is specifically configured to, according to the number of times the voice instruction has been used, search from a preset first relationship table for a first sub-interest corresponding to the number of times used, where the number of times used is positively correlated with the first sub-interest; according to the similarity between the historical voice wake-up scene features of the voice instruction and the voice wake-up scene features, search from a preset second relationship table for a second sub-interest corresponding to the similarity, and the similarity is positively correlated with the second sub-interest; and perform weighted averaging on the first sub-interest and the second sub-interest to obtain the interest of the voice instruction.

[0078] In some embodiments, the device provided in this application further includes: a usage times statistics unit, configured to, after responding to a selection operation of the user on a target voice instruction among the multiple displayed voice instructions, increment the number of times the target voice instruction has been used by one.

[0079] In some embodiments, the feature acquisition unit is specifically configured to determine the orientation of the user in the vehicle according to the way the user wakes up the voice assistant; control the in-vehicle camera to take a picture in the direction of the user to obtain an image including the user's face as the identity feature of the user.

[0080] In some embodiments, the feature acquisition unit is specifically configured to determine that the user's position in the vehicle is the driver's seat when it is determined that the user's wake-up operation on the voice assistant is to trigger the wake-up button on the vehicle's steering wheel.

[0081] In other embodiments, the feature acquisition unit is specifically configured to, when it is determined that the user's wake-up operation on the voice assistant is to input wake-up voice data to a plurality of microphones arranged in an array in the vehicle, identify the position of the user who inputs the wake-up voice data according to the preset geometric layout of the microphones, the phase difference and time difference of the wake-up voice data reaching each microphone, and the preset sound source localization algorithm.

[0082] In addition, an embodiment of the present application further provides an in-vehicle terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the in-vehicle terminal is caused to execute the method provided in the above embodiments of the present application.

[0083] In addition, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the computer is caused to execute the method provided in the above embodiments of the present application.

[0084] In addition, the present application further provides a computer program product including a computer program. When the computer program is run, the in-vehicle terminal is caused to execute the method provided in the above embodiments of the present application.

[0085] In the above description, no detailed description is made of the technical details such as the composition of each layer. However, those skilled in the art should understand that various technical means can be used to form layers, regions, etc. of the required shape. In addition, in order to form the same structure, those skilled in the art can also design methods that are not exactly the same as the methods described above. In addition, although the above embodiments are described separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination.

[0086] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0087] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A method for executing vehicle functions based on voice commands, characterized in that: Applied to a vehicle-mounted terminal, the method comprises: In response to a user's wake-up operation on the voice assistant, waking up the preset voice assistant; Acquire the identity characteristics of the user, the positioning characteristics of the vehicle, and the time characteristics for waking up the voice assistant; The identity feature of the user, the positioning feature of the vehicle, and the time feature of the voice assistant are integrated to obtain a voice wake-up scene feature; Searching, from a preset database, for a voice command inputted historically by a user corresponding to a historical voice wake-up scene feature whose similarity to the voice wake-up scene feature is higher than a set similarity threshold; Displaying the multiple voice commands found in the form of text on the screen of the vehicle terminal in sequence; In response to a user input selecting a target voice command among the displayed plurality of voice commands, the vehicle is controlled to execute a vehicle function associated with the target voice command.

2. The method according to claim 1, characterized in that The method of sequentially displaying the multiple voice commands found in the form of text on the screen of the vehicle-mounted terminal includes: Get the number of times each of the multiple voice commands found has been used in history; For each of the voice commands, determining the user's interest in the voice command according to the number of times the voice command has been used and the similarity between the corresponding historical voice wake-up scene feature and the voice wake-up scene feature, wherein the number of times the voice command has been used and the similarity are both positively correlated with the interest; According to the interest level of each of the voice commands, the multiple voice commands found are displayed in text form on the screen of the vehicle terminal in descending order.

3. The method according to claim 2, characterized in that Determining the interest of the voice command according to the number of times the voice command has been used and the similarity between the corresponding historical voice wake-up scene feature and the voice wake-up scene feature includes: According to the number of times the voice command has been used, searching a preset first relationship table for a first sub-interest level corresponding to the number of times the voice command has been used, wherein the number of times the voice command has been used is positively correlated with the first sub-interest level; According to the similarity between the historical voice wake-up scene feature of the voice command and the voice wake-up scene feature, searching the second sub-interest corresponding to the similarity from a preset second relationship table, wherein the similarity is positively correlated with the second sub-interest; The first sub-interest level and the second sub-interest level are weighted averaged to obtain the interest level of the voice command.

4. The method according to claim 2, characterized in that: The method further includes: in response to a user input of a selection operation of a target voice instruction among the displayed plurality of voice instructions, increasing the number of times the target voice instruction has been used by one.

5. The method according to claim 1, characterized in that The obtaining of the identity feature of the user includes: Determining the user's location in the vehicle based on how the user wakes up the voice assistant; The vehicle-mounted camera is controlled to shoot in the direction of the user to obtain an image including the user's face as the user's identity feature.

6. The method according to claim 5, characterized in that Determining the location of the user in the vehicle according to the method in which the user wakes up the voice assistant, including: When it is determined that the user's wake-up operation on the voice assistant is that the user triggers a wake-up button on the steering wheel of the vehicle, the user's position in the vehicle is determined to be the main driving seat.

7. The method according to claim 5, characterized in that Determining the location of the user in the vehicle according to the method in which the user wakes up the voice assistant, including: When it is determined that the user has performed a wake-up operation on the voice assistant, that is, the user inputs wake-up voice data into multiple microphones arranged in an array in the vehicle, the position of the user who inputs the wake-up voice data is identified based on the preset geometric layout of the microphones, the phase difference and time difference of the wake-up voice data reaching each microphone, and the preset sound source localization algorithm.

8. A vehicle-mounted terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the vehicle-mounted terminal executes the method according to any one of claims 1 to 7.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer is caused to perform the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, the vehicle-mounted terminal executes the method as claimed in any one of claims 1 to 7.

Images