Voice wake-up method and device, electronic equipment, storage medium and vehicle

Abstract

The invention relates to a voice wake-up method and device, electronic equipment, a storage medium and a vehicle. The method comprises the following steps: collecting a target voice signal; when it is detected that the target voice signal comprises the wake-up word and the generation position of the target voice signal is recognized to be in the cabin of the vehicle based on a position model, the current gear of the vehicle is determined; and awakening a voice assistant in the cabin under the condition that the target voice signal is determined to be the voice signal in the cabin based on the current gear. According to the method, the personnel outside the cabin of the vehicle can be prevented from waking up the voice assistant in the cabin, and the safety of the vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a voice wake-up method, device, electronic device, storage medium and vehicle. Background Art

[0002] With the development of the Internet of Vehicles, voice interaction technology has been introduced in many vehicles. Users can use voice commands to wake up the voice assistant in the cabin to control the vehicle to perform corresponding operations. However, there are certain safety risks in controlling vehicles through voice commands. How to make the interaction between people and cars safer has become an important research topic in the field of automotive technology.

[0003] The current wake-up method for the voice assistant in the vehicle cabin is: when the voice signal is detected to include a wake-up word, and the location where the voice signal is generated is identified as inside the vehicle cabin based on the position model, the voice assistant in the vehicle cabin is woken up.

[0004] However, the current wake-up method has certain problems: the position model may misjudge the location where the voice signal is generated, and may identify the location where the voice signal from outside the cabin is generated as inside the cabin. In this way, when the voice assistant in the cabin receives the wake-up word from outside the cabin, it can also start the voice assistant in the cabin and then control the vehicle, increasing the safety risks of the car. Summary of the invention

[0005] The present application provides a voice wake-up method, device, electronic device, storage medium and vehicle, which can prevent people outside the cabin from waking up the voice assistant in the cabin, thereby improving the safety of the car.

[0006] In a first aspect, an embodiment of the present application provides a voice wake-up method, including: collecting a target voice signal; determining a current gear position of the vehicle when it is detected that the target voice signal includes a wake-up word and identifying the target voice signal as being generated inside a cabin of the vehicle based on a position model; and waking up a voice assistant in the cabin when it is determined that the target voice signal is a voice signal inside the cabin based on the current gear position.

[0007] In the second aspect, an embodiment of the present application provides a voice wake-up device, including: a collection unit, for collecting a target voice signal; a determination unit, for determining a current gear position of a vehicle when it is detected that the target voice signal includes a wake-up word and the location where the target voice signal is generated is identified as being inside the cabin of the vehicle based on a position model; a wake-up unit, for waking up a voice assistant in the cabin when it is determined that the target voice signal is a voice signal inside the cabin based on the current gear position.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, wherein the processor is used to execute a computer program stored in a memory, and when the computer program is executed by the processor, the steps of any one of the voice wake-up methods provided in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the voice wake-up methods provided in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising: a voice wake-up device as described in the second aspect, or an electronic device as described in the third aspect, or a computer-readable storage medium as described in the fourth aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the voice wake-up method as described in the first aspect.

[0012] In the seventh aspect, an embodiment of the present application provides a chip, which includes a processor, a memory and a communication interface, the communication interface is coupled to the processor, the memory is used to store programs or instructions that can be run on the processor, and the processor is used to execute the program or instructions to implement the steps of the voice wake-up method as described in the first aspect.

[0013] The technical solution provided by the embodiment of the present application has the following advantages over the prior art: In the embodiment of the present application, the target voice signal is collected; when it is detected that the target voice signal includes a wake-up word, and the target voice signal is identified as being generated in the cabin of the vehicle based on the position model, the current gear of the vehicle is determined; when the target voice signal is determined to be a voice signal in the cabin based on the current gear, the voice assistant in the cabin is woken up. In this solution, when it is detected that the target voice signal includes a wake-up word, and the target voice signal is identified as being generated in the cabin based on the position model, the source of the target voice signal is also determined based on the current gear, and when it is determined that the target signal is a voice signal in the cabin based on the current gear, the voice assistant in the cabin is woken up. In this way, it can be avoided that the position model misjudges the generation location of the voice signal from outside the cabin as being inside the cabin when identifying the generation location of the voice signal, resulting in the voice signal from outside the cabin also waking up the voice assistant, so that it can further prevent people outside the vehicle from controlling the vehicle by waking up the voice assistant, reducing the safety risks of the car. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0016] Figure 1 A flowchart of a voice wake-up method provided in this application;

[0017] Figure 2 A flowchart of a method for training a position model provided in this application;

[0018] Figure 3 A flowchart of a method for obtaining in-cabin training samples provided in this application;

[0019] Figure 4 A flowchart of a method for obtaining extravehicular training samples provided in this application;

[0020] Figure 5 A schematic diagram of the network structure of a multi-scale dilated deep temporal convolutional network MDTC provided in this application;

[0021] Figure 6 A flowchart of another voice wake-up method provided by this application;

[0022] Figure 7 A schematic diagram of the structure of a voice wake-up device provided in this application;

[0023] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0026] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0027] The embodiments of the present application provide a voice wake-up method, device, electronic device, storage medium and vehicle. The method includes: collecting a target voice signal; determining the current gear of the vehicle when it is detected that the target voice signal includes a wake-up word and the target voice signal is identified as being generated inside the cabin of the vehicle based on the position model; waking up the voice assistant in the cabin when the target voice signal is determined to be a voice signal inside the cabin based on the current gear. In this way, it can be avoided that the position model misjudges the generation location of the voice signal from outside the cabin as being inside the cabin when identifying the generation location of the voice signal, causing the voice signal from outside the cabin to also wake up the voice assistant. In this way, it can further prevent people outside the vehicle from controlling the vehicle by waking up the voice assistant, reducing the potential safety hazards of the vehicle.

[0028] The present application is applied to the scenario of determining whether to wake up the voice assistant in the vehicle cabin. The electronic device in the embodiment of the present application can be a vehicle-mounted terminal, or a mobile phone, notebook, computer, etc. that is connected to the vehicle for communication. The specific details can be determined based on actual conditions and are not limited here.

[0029] The technical solution of the present application is explained in detail below through several specific embodiments.

[0030] Figure 1 A flowchart of a voice wake-up method provided in this application is shown in FIG. Figure 1 As shown, the voice wake-up method may include the following steps S101 to S103:

[0031] S101: Collect target voice signals.

[0032] Specifically, the vehicle's in-cabin voice assistant collects the target voice signal through the voice assistant's microphone.

[0033] S102: When it is detected that the target voice signal includes a wake-up word and the target voice signal is identified as being generated inside a vehicle cabin based on a position model, determine a current gear position of the vehicle.

[0034] In some embodiments of the present application, S102 includes the following steps S102A1 to S102A3:

[0035] S102A1, detecting whether the target voice signal includes a wake-up word;

[0036] S102A2: when it is detected that the target voice signal includes a wake-up word, identifying a generation position of the target voice signal based on a position model;

[0037] S102A3: When it is identified that the target voice signal is generated inside the vehicle cabin, determine the current gear position of the vehicle.

[0038] In an embodiment of the present application, it is detected whether the target voice signal includes a wake-up word; when it is detected that the target voice signal includes the wake-up word, the generation location of the target voice signal is identified based on the position model; when it is identified that the generation location of the target voice signal is inside the cabin of the vehicle, the current gear position of the vehicle is determined. It should be understood that the vehicle has a lot of ambient sounds, some of which are generated inside the cabin of the vehicle, and some of which are generated outside the cabin of the vehicle; and the proportion of ambient sounds including the wake-up word in the ambient sounds of the vehicle is relatively small. Therefore, it is first detected whether the target voice signal includes the wake-up word, and when it is detected that the target voice signal includes the wake-up word, the generation location of the target voice signal is identified based on the position model; when it is identified that the generation location of the target voice signal is inside the cabin of the vehicle, the current gear position of the vehicle is determined. In this way, the number of times the position model is enabled to identify the generation location of the target voice signal in the process of waking up the voice assistant in the vehicle cabin can be greatly reduced, thereby reducing the computing power consumption on the vehicle side.

[0039] In some embodiments of the present application, S102 includes the following steps S102B1 to S102B2:

[0040] S102B1, detecting whether the target voice signal includes a wake-up word, and identifying a generation position of the target voice signal based on a position model;

[0041] S102B2: When it is detected that the target voice signal includes a wake-up word and when it is identified that the target voice signal is generated inside a vehicle cabin, determine the current gear position of the vehicle.

[0042] In some embodiments of the present application, S102 includes the following steps S102C1 to S102C3:

[0043] S102C1. Identify the generation location of the target speech signal based on the location model.

[0044] S102C2: When it is identified that the target voice signal is generated inside the vehicle cabin, detect whether the target voice signal includes a wake-up word.

[0045] S102C3. When it is detected that the target voice signal includes a wake-up word, determine the current gear position of the vehicle.

[0046] In some embodiments of the present application, before identifying the location where the target voice signal is generated based on the location model, the voice wake-up method further includes training the location model. Figure 2 A flow chart of a method for training a position model provided in an embodiment of the present application. Figure 2 As shown, the training method of the position model includes the following steps S102D1 to S102D2:

[0047] S102D1. Obtain training sample data, which includes multiple in-cabin training samples and multiple out-cabin training samples, each in-cabin training sample includes an in-cabin sample voice signal and an in-cabin generation position corresponding to the in-cabin sample voice signal, and each out-cabin training sample includes an out-cabin sample voice signal and an out-cabin generation position corresponding to the out-cabin sample voice signal.

[0048] In some embodiments of the present application, the above-mentioned voice wake-up method also includes obtaining multiple in-cabin training samples. Figure 3 A method for obtaining in-cabin training samples is provided in an embodiment of the present application. Figure 3 The method for obtaining in-cabin training samples includes the following steps S102D1-11 to S102D1-12.

[0049] S102D1-11. Collect multiple in-cabin audio signals, including: at least one command audio signal responded to by the in-cabin voice assistant when it is in the awake state, at least one in-cabin human voice audio signal collected by the in-cabin microphone, and at least one recorded audio signal played by the in-cabin equipment.

[0050] Specifically, the wake-up word data played by the high-fidelity speaker is collected through the microphone in the cabin to obtain at least one recorded audio signal; the wake-up word data issued by a real person is collected through the microphone in the cabin to obtain at least one human voice audio signal in the cabin; the wake-up data corresponding to the voice assistant in the cabin when it is in the wake-up state is collected to obtain at least one command audio signal.

[0051] S102D1-12, determining as a plurality of in-cabin training samples at least one in-cabin human voice audio signal and at least one recorded audio signal that meets the target condition and the in-cabin generation position corresponding to the audio signal, as well as at least one command audio signal and the in-cabin generation position corresponding to each command audio signal;

[0052] The target conditions include: an audio signal whose decibel value is within a preset range in the cabin.

[0053] In the embodiment of the present application, the above-mentioned preset range in the cabin can be determined according to actual conditions, and the present application does not make any specific limitation.

[0054] Exemplarily, at least one in-cabin human voice audio signal and at least one recorded audio signal, an audio signal with a decibel value between 55 and 65 decibels and the in-cabin generation position corresponding to the audio signal, as well as at least one command audio signal and the in-cabin generation position corresponding to each command audio signal, are determined as multiple in-cabin training samples.

[0055] In the embodiment of the present application, the above-mentioned decibel value may be the decibel value when it reaches the microphone, or may be the decibel value when the audio signal is generated, and the embodiment of the present application does not impose any specific limitation on this.

[0056] In the embodiment of the present application, multiple in-cabin audio signals are collected, and the multiple in-cabin audio signals include: at least one command audio signal that the voice assistant in the cabin responds to when it is in the wake-up state, and the audio signal that meets the target condition in at least one in-cabin human voice audio signal and at least one recorded audio signal and the in-cabin generation position corresponding to the audio signal, and at least one command audio signal, and the in-cabin generation position corresponding to each command audio signal are determined as multiple in-cabin training samples; wherein the target condition includes: an audio signal with a decibel value within a preset range in the cabin. It should be understood that the owner may wake up the voice assistant in the vehicle cabin in any scenario. Therefore, the corresponding scenario when the voice assistant in the vehicle cabin is in the wake-up state is relatively diverse, that is, the scenario corresponding to the above-mentioned at least one command audio signal is relatively diverse. The embodiment of the present application also collects the wake-up word data issued by a real person to obtain at least one in-cabin human voice audio signal, and the wake-up word data played by a high-fidelity speaker to obtain at least one recorded audio signal; in this way, the diversity of the in-cabin training samples is increased, so that the accuracy of the position model in identifying the position where the voice signal is generated is improved, which can prevent people outside the vehicle cabin from waking up the voice assistant in the cabin to a greater extent, and improve the safety of the car. At the same time, the robustness of the position model is improved, so that the position model can identify the location where the voice signal is generated in a variety of scenarios. In addition, the embodiment of the present application determines the audio signal with a decibel value within a preset range in the cabin among at least one human voice audio signal in the cabin and at least one recorded audio signal as multiple cabin training samples. In this way, the volume of the collected cabin audio signal is more consistent with the volume of the cabin audio signal in the real scene, thus improving the accuracy of the position model in identifying the location where the voice signal is generated.

[0057] In some embodiments of the present application, the above-mentioned voice wake-up method also includes obtaining multiple out-of-cabin training samples. Figure 4 A method for obtaining an extravehicular training sample is provided in an embodiment of the present application. Figure 4 The method for obtaining extravehicular training samples includes the following steps S102D1-21 to S102D1-22.

[0058] S102D1-21. Collect multiple out-of-cabin audio signals through a microphone in the cabin. The multiple out-of-cabin audio signals include: at least one out-of-cabin human voice audio signal and at least one recorded audio signal played by an out-of-cabin device; the distance between the out-of-cabin generation position of each out-of-cabin audio signal and the position of the vehicle is less than or equal to a distance threshold.

[0059] Exemplarily, when the windows are closed, the wake-up word data played by the high-fidelity speaker at the main driver's seat of the vehicle 10-20 cm away from the vehicle body is collected through the microphone in the cabin to obtain at least one recorded audio signal; when the windows are closed, the wake-up word data played by the high-fidelity speaker at the co-driver's seat of the vehicle 10-20 cm away from the vehicle body is collected through the microphone in the cabin to obtain at least one recorded audio signal; when the windows are closed, the wake-up word data played by the high-fidelity speaker at the second row of the vehicle 10-20 cm away from the vehicle body is collected through the microphone in the cabin to obtain at least one recorded audio signal; the wake-up word data played by the high-fidelity speaker at the tailgate of the vehicle 10-20 cm away from the vehicle body is collected through the microphone in the cabin to obtain at least one recorded audio signal.

[0060] Exemplarily, when the car window is closed, a real person shouts the wake-up word close to the car window glass to obtain at least one human voice audio signal outside the cabin.

[0061] S102D1-22. Determine, among multiple extra-cabin audio signals, audio signals whose decibel values ​​are within a preset range outside the cabin, and the corresponding extra-cabin generation locations of the audio signals as multiple extra-cabin training samples, wherein the starting value of the preset range outside the cabin is greater than the starting value of the preset range inside the cabin.

[0062] In the embodiment of the present application, the above-mentioned preset range outside the cabin can be determined according to actual conditions, and the present application does not make any specific restrictions.

[0063] Exemplarily, among a plurality of extra-cabin audio signals, audio signals with decibel values ​​within a range of 95 to 105 decibels and the extra-cabin generation locations of the corresponding audio signals are determined as a plurality of extra-cabin training samples.

[0064] In an embodiment of the present application, a plurality of out-of-cabin audio signals are collected by a microphone in the cabin, and the plurality of out-of-cabin audio signals include: at least one out-of-cabin human voice audio signal, and at least one recorded audio signal played by an out-of-cabin device; the distance between the out-of-cabin generation position of each out-of-cabin audio signal and the position of the vehicle is less than or equal to a distance threshold; among the plurality of out-of-cabin audio signals, the audio signals whose decibel values ​​are within the out-of-cabin preset range, and the out-of-cabin generation positions of the corresponding audio signals are determined as a plurality of out-of-cabin training samples, wherein the starting value of the out-of-cabin preset range is greater than the starting value of the in-cabin preset range. In this way, the recorded audio signals played by the out-of-cabin device and the audio signals emitted by real people are collected separately, which increases the diversity of out-of-cabin training samples. In this way, the accuracy of the position model in identifying the generation position of the voice signal is improved, which can prevent the personnel outside the vehicle from waking up the voice assistant in the cabin to a greater extent, and improve the safety of the car. At the same time, the robustness of the position model is improved, so that the position model can identify the generation position of the voice signal in a variety of scenarios. In addition, in an embodiment of the present application, among multiple out-of-cabin audio signals, audio signals whose decibel values ​​are within a preset range outside the cabin, and the corresponding out-of-cabin generation positions of the audio signals, are determined as multiple out-of-cabin training samples, wherein the starting value of the preset range outside the cabin is greater than the starting value of the preset range inside the cabin. It should be understood that when collecting out-of-cabin audio signals, it should be taken into account that in the process of transmitting the out-of-cabin audio signals to the cabin, due to the obstruction of obstacles such as windows and seats in the vehicle, a certain attenuation will occur. Therefore, the audio signal outside the cabin must be able to wake up the voice assistant inside the cabin, and its volume is usually higher than the audio signal inside the cabin. In this way, in an embodiment of the present application, the starting value of the preset range outside the cabin is greater than the starting value of the preset range inside the cabin, which is closer to the volume of the audio signal outside the cabin in the real scene, thereby improving the accuracy of the position model in identifying the position where the voice signal is generated.

[0065] S102D2. Train the basic model based on the neural network by training sample data to obtain a position model.

[0066] In the embodiment of the present application, training sample data is first obtained, and the basic model based on the neural network is trained by the training sample data to obtain a position model. In this way, the position model used to identify the position where the audio signal is generated in the present solution can be obtained.

[0067] In some embodiments of the present application, the above-mentioned neural network is a multi-scale dilation deep temporal convolution network MDTC (Multi-scale Dilation Deep Temporal Convolution), and the above-mentioned training sample data is used to train the basic model based on the neural network to obtain the position model, including: training the basic model based on the multi-scale dilation deep temporal convolution network MDTC through training sample data to obtain the position model.

[0068] In an embodiment of the present application, by training sample data, a basic model based on a multi-scale dilated deep temporal convolutional network MDTC is trained to obtain a position model for identifying the location where an audio signal is generated in the present solution.

[0069] In some embodiments of the present application, a basic model based on a multi-scale dilated deep temporal convolutional network MDTC is trained by training sample data to obtain a position model, including the following steps S102D2-11 to S102D2-15:

[0070] S102D2-11. Perform feature extraction on the sample speech signal in each training sample in the training sample data to obtain speech features corresponding to each training sample, where the speech features are used to indicate the spectrum information of the corresponding training sample. When the training sample is an in-cabin training sample, the sample speech signal is an in-cabin sample speech signal. When the training sample is an out-of-cabin training sample, the sample speech signal is an out-of-cabin sample speech signal.

[0071] In some embodiments of the present application, the above-mentioned speech features are Fank (Filter Bank) speech features.

[0072] S102D2-12. Classify the speech features corresponding to each training sample through the multi-scale dilated deep temporal convolutional network MDTC to obtain a prediction vector when the predicted generation position of each training sample is the actual generation position.

[0073] Figure 5 A schematic diagram of the network structure of a multi-scale dilated deep temporal convolutional network MDTC provided in an embodiment of the present application.

[0074] like Figure 5 As shown in C, the network structure of MDTC includes multiple dilated-depth temporal convolution stacks (DTC-Stack), feature fusion layer Add, fully connected layer FC and activation function sigmoid. Figure 5 As shown in Figure B, each dilated-depth temporal convolution stack includes four dilated-depth temporal convolution blocks (DTC-Blocks) with different dilation rates. The dilation rates of the four DTC blocks are set to 1, 2, 4, and 8, respectively. Figure 5As shown in Figure A, each dilated-depth temporal convolution block includes a dilated-depth temporal convolution network layer (Dilated-Depth Temporal Convolution, Dilated-Depth TCN), a batch normalization layer (BN, Batch Normalization), a point-by-point convolution layer (Point-Conv, Point-Convolution), a batch normalization and activation function layer ReLU (Rectified Linear Unit), a point-by-point convolution layer, a BN layer, a SE layer (Squeeze-and-Extraction, squeezing and excitation), a normalization layer Scale, a feature fusion layer Add, and an activation function layer ReLU.

[0075] The speech features corresponding to each training sample in the training sample data are input into the multi-scale dilated deep temporal convolutional network MDTC, such as Figure 5 As shown in Figure C, MDTC learns the speech features corresponding to each training sample through multiple dilated-depth temporal convolution stacks (DTC-Stack, Dilated-Depth Temporal Convolution--Stack) (wherein, each dilated-depth temporal convolution block in each dilated-depth temporal convolution stack uses a dilated-depth temporal convolution network (Dilated-Depth TCN) to obtain the temporal context of the speech features. After the dilated-depth temporal convolution network, two layers of point-by-point convolution (Point Conv) are used to integrate features from different channels, and batch normalization (BN) and RELU activation functions are inserted between different convolution layers. In addition, a squeeze and activation (SE) module is added after the last point-by-point convolution layer to learn the attention information between different channels. In order to prevent gradient disappearance and gradient explosion, a residual connection between the input and the last REU activation function is also used). The feature fusion layer Add sums the speech features corresponding to each training sample learned by multiple dilated-depth temporal convolution stacks respectively; the features output by the feature fusion layer Add are processed based on the fully connected layer and the activation function sigmoid to obtain a prediction vector when the predicted generation position of each training sample is the actual generation position.

[0076] S102D2-13. Normalize the prediction vector corresponding to each training sample to obtain the probability value corresponding to each training sample.

[0077] In the embodiment of the present application, the prediction vector corresponding to each training sample can be normalized by the normalized exponential function softmax to obtain the probability value corresponding to each training sample, or the prediction vector corresponding to each training sample can be normalized by other methods to obtain the probability value corresponding to each training sample, for example, the prediction vector corresponding to each training sample can be normalized by the sigmoid function to obtain the probability value corresponding to each training sample. This application is not specifically limited by comparison.

[0078] S102D2-14. Based on the loss function and the probability value corresponding to each training sample, determine the loss value corresponding to the multi-scale dilated deep temporal convolutional network MDTC.

[0079] In an embodiment of the present application, the loss function may be a cross entropy loss function. The above-mentioned loss value corresponding to the multi-scale expansion deep time convolution network MDTC is determined based on the loss function and the probability value corresponding to each training sample. Specifically, it may be: based on the cross entropy loss function and the probability value corresponding to each training sample, the loss value corresponding to the multi-scale expansion deep time convolution network MDTC is determined;

[0080] The cross entropy loss function is:

[0081]

[0082] Among them, N is the number of training samples in the training sample data, p i It represents the probability value corresponding to the actual generation position when the generation position predicted by the multi-scale dilation deep temporal convolution network MDTC is the i-th training sample among N training samples, y i Represents the true label value of the i-th training sample. When the generation position predicted by the multi-scale dilated deep temporal convolutional network MDTC is the actual generation position, the true label value of the i-th training sample is equal to 1.

[0083] S102D2-15. Based on the loss value, the model parameters of the multi-scale dilated deep temporal convolutional network MDTC are optimized through an optimization algorithm to obtain a position model.

[0084] In some embodiments of the present application, the above-mentioned optimization algorithm is a gradient descent algorithm.

[0085] In an embodiment of the present application, training sample data is obtained, and the training sample data includes multiple in-cabin training samples and multiple out-cabin training samples, each in-cabin training sample includes an in-cabin sample voice signal and an in-cabin generation position corresponding to the in-cabin sample voice signal, and each out-cabin training sample includes an out-cabin sample voice signal and an out-cabin generation position corresponding to the out-cabin sample voice signal; through the training sample data, the basic model based on the multi-scale dilation deep temporal convolution network MDTC (Multi-scale Dilation Deep Temporal Convolution) is trained to obtain a position model. In this scheme, the training sample data is obtained, and the basic model based on the multi-scale dilation deep temporal convolution network MDTC is trained through the training sample data to obtain a position model. In this way, by collecting the corresponding training sample data under the application scenario of the present application, the basic model based on the multi-scale dilation deep temporal convolution network MDTC is trained to obtain a position model. A position model that better meets the application scenario requirements of the present application can be obtained, thereby improving the accuracy of the position model in identifying the position where the voice signal is generated, and can prevent people outside the vehicle cabin from waking up the voice assistant in the cabin to a greater extent, thereby improving the safety of the car.

[0086] S103: When it is determined based on the current gear position that the target voice signal is a voice signal in the cabin, wake up the voice assistant in the cabin.

[0087] It should be understood that there may be a situation where the frequency spectrum of the outside voice signal and the inside voice signal is not high. In this way, the position model may produce some misjudgments and identify the location where the outside voice signal is generated as inside the cabin, which may cause the outside personnel to wake up the voice assistant in the vehicle cabin, posing a safety hazard.

[0088] In an embodiment of the present application, a target voice signal is collected; when it is detected that the target voice signal includes a wake-up word, and the target voice signal is identified as being generated in the cabin of the vehicle based on the position model, the current gear position of the vehicle is determined; when the target voice signal is determined to be a voice signal in the cabin based on the current gear position, the voice assistant in the cabin is woken up. In this solution, when it is detected that the target voice signal includes a wake-up word, and the target voice signal is identified as being generated in the cabin based on the position model, the source of the target voice signal is also determined based on the current gear position, and when it is determined that the target signal is a voice signal in the cabin based on the current gear position, the voice assistant in the cabin is woken up. In this way, it can be avoided that the position model misjudges the generation location of the voice signal from outside the cabin as being inside the cabin when identifying the generation location of the voice signal, resulting in the voice signal from outside the cabin also waking up the voice assistant. In this way, it can further prevent people outside the vehicle from controlling the vehicle by waking up the voice assistant, reducing the potential safety hazards of the vehicle.

[0089] Another embodiment of the present invention is a further optimization based on the above embodiment. Figure 6 A flowchart of a voice wake-up method provided by another embodiment of the present invention. In one embodiment, Figure 6 As shown, the voice wake-up method of this embodiment includes:

[0090] S601: Collect target voice signals.

[0091] S602: When it is detected that the target voice signal includes a wake-up word and the target voice signal is identified as being generated inside a vehicle cabin based on a position model, determine a current gear position of the vehicle.

[0092] S603. When the current gear position is not P, wake up the voice assistant in the cabin; or, when the current gear position is P, determine whether there is a human body in the cabin; if there is a human body in the cabin, wake up the voice assistant in the cabin.

[0093] In some embodiments of the present application, S603 determines whether there is a human body in the cabin, including the following steps:

[0094] S603A1. Identify whether the image in the cabin includes a human body.

[0095] The embodiment of the present application does not impose any specific limitation on the method of acquiring images inside the cabin.

[0096] In some embodiments of the present application, images inside the cabin are acquired through an image acquisition device.

[0097] In some other embodiments of the present application, an infrared image of the interior of the cabin is acquired by an infrared temperature measuring camera, and the infrared image is determined as an image of the interior of the cabin.

[0098] S603A2, when the image in the cabin includes a human body, determining that the cabin includes a human body;

[0099] S603A3: When the image in the cabin does not include a human body, determine that there is no human body in the cabin.

[0100] In the embodiment of the present application, it is identified whether the image in the cabin includes a human body. If the image in the cabin includes a human body, it is determined that the cabin includes a human body; if the image in the cabin does not include a human body, it is determined that the cabin does not include a human body. In this way, by judging whether the cabin of the vehicle includes a human body through the image in the cabin, misjudgment can be avoided to a greater extent, and further, it can further prevent people outside the cabin from waking up the voice assistant in the cabin, thereby improving the safety of the vehicle.

[0101] It is understandable that when the current gear is not P, it means that the vehicle is in the process of driving. When the vehicle is in the process of driving, the target voice signal is less likely to be sent by a person outside the cabin. Therefore, when the current gear is not P, the voice assistant in the cabin can be directly awakened. When the current gear is P, it means that the vehicle is parked. When the vehicle is parked, the target voice signal may be sent by a person outside the cabin. Therefore, it is also necessary to identify whether the image in the cabin includes a human body. If a human body is included in the cabin, the voice assistant in the cabin is awakened. In this way, based on whether the voice signal includes a wake-up word, the location where the voice signal is generated, the current gear of the vehicle, and whether a human body is recognized in the vehicle cabin, it is determined whether to wake up the voice assistant in the cabin. Compared with the existing solution that determines whether to wake up the voice assistant in the cabin based only on whether the voice signal includes a wake-up word and whether the location where the voice signal is generated is in the cabin, it can avoid people outside the vehicle from waking up the voice assistant in the cabin, thereby improving the safety of the car.

[0102] In summary, the embodiments of the present application provide a voice wake-up method, device, electronic device, storage medium and vehicle. The method includes: collecting a target voice signal; when it is detected that the target voice signal includes a wake-up word, and the target voice signal is identified as a voice signal in the cabin based on the position model, determining the current gear position of the vehicle to determine that the target voice signal is a voice signal in the cabin; when it is determined that the target voice signal is a voice signal in the cabin based on the current gear position, waking up the voice assistant in the cabin. In this way, it can be avoided that the position model misjudges the generation position of the voice signal from outside the cabin as being inside the cabin when identifying the generation position of the voice signal, resulting in the voice signal from outside the cabin also waking up the voice assistant. In this way, it can further prevent people outside the vehicle from controlling the vehicle by waking up the voice assistant, reducing the potential safety hazards of the vehicle.

[0103] The present application also provides a voice wake-up device, Figure 7 A schematic diagram of the structure of a voice wake-up device provided in this application, such as Figure 7 As shown, the voice wake-up device 70 includes:

[0104] A collection unit 71, used for collecting a target speech signal;

[0105] A determination unit 72, configured to determine a current gear position of the vehicle when it is detected that the target voice signal includes a wake-up word and the target voice signal is identified as being generated inside a vehicle cabin based on the position model;

[0106] The wake-up unit 73 is used to wake up the voice assistant in the cabin when it is determined that the target voice signal is a voice signal in the cabin based on the current gear position.

[0107] In some embodiments of the present application, the wake-up unit 73 is specifically used to wake up the voice assistant in the cabin when the current gear position is not P gear; or, when the current gear position is P gear, determine whether there is a human body in the cabin; if there is a human body in the cabin, wake up the voice assistant in the cabin.

[0108] In some embodiments of the present application, the wake-up unit 73 is specifically used to identify whether the image in the cabin includes a human body; if the image in the cabin includes a human body, determine that the cabin includes a human body; if the image in the cabin does not include a human body, determine that the cabin does not include a human body.

[0109] In some embodiments of the present application, the voice wake-up device also includes an acquisition unit 74, which is used to acquire training sample data, and the training sample data includes multiple in-cabin training samples and multiple out-cabin training samples, each in-cabin training sample includes an in-cabin sample voice signal and an in-cabin generation position corresponding to the in-cabin sample voice signal, and each out-cabin training sample includes an out-cabin sample voice signal and an out-cabin generation position corresponding to the out-cabin sample voice signal; the training unit 75 is used to train the basic model based on the neural network through the training sample data to obtain a position model.

[0110] In some embodiments of the present application, the training unit 75 is specifically used to train a basic model based on a multi-scale dilated deep temporal convolutional network MDTC through training sample data to obtain a position model.

[0111] In some embodiments of the present application, the training unit 75 is specifically used to extract features from the sample speech signal in each training sample in the training sample data to obtain speech features corresponding to each training sample, where the speech features are used to indicate the spectrum information of the corresponding training sample. When the training sample is an in-cabin training sample, the sample speech signal is an in-cabin sample speech signal; when the training sample is an out-of-cabin training sample, the sample speech signal is an out-of-cabin sample speech signal; the speech features corresponding to each training sample are classified through the multi-scale dilation deep time convolution network MDTC to obtain a prediction vector when the predicted generation position of each training sample is the actual generation position; the prediction vector corresponding to each training sample is normalized to obtain a probability value corresponding to each training sample; based on the loss function and the probability value corresponding to each training sample, the loss value corresponding to the multi-scale dilation deep time convolution network MDTC is determined; based on the loss value, the model parameters of the multi-scale dilation deep time convolution network MDTC are optimized through an optimization algorithm to obtain a position model.

[0112] In some embodiments of the present application, the collection unit 71 is also used to collect multiple in-cabin audio signals, and the multiple in-cabin audio signals include: at least one command audio signal responded to by the in-cabin voice assistant when it is in an awake state, at least one in-cabin human voice audio signal collected by the microphone in the cabin, and at least one recorded audio signal played by the in-cabin device collected by the microphone in the cabin; the determination unit 72 is also used to determine the audio signals and the in-cabin generation positions corresponding to the audio signals that meet the target conditions in at least one in-cabin human voice audio signal and at least one recorded audio signal, as well as at least one command audio signal and the in-cabin generation position corresponding to each command audio signal as multiple in-cabin training samples; wherein the target conditions include: audio signals whose decibel values ​​are within a preset range in the cabin.

[0113] In some embodiments of the present application, the collection unit 71 is further used to collect multiple out-of-cabin audio signals through a microphone in the cabin, and the multiple out-of-cabin audio signals include: at least one out-of-cabin human voice audio signal, and at least one recorded audio signal played by an out-of-cabin device; the distance between the out-of-cabin generation position of each out-of-cabin audio signal and the position of the vehicle is less than or equal to a distance threshold; the determination unit 72 is further used to determine, among the multiple out-of-cabin audio signals, audio signals whose decibel values ​​are within a preset range outside the cabin, and the corresponding out-of-cabin generation positions of the audio signals as multiple out-of-cabin training samples, wherein the starting value of the preset range outside the cabin is greater than the starting value of the preset range inside the cabin.

[0114] It should be noted that the above-mentioned voice wake-up device can be the electronic device in the above-mentioned method embodiment of the present application, or it can be a functional module and / or functional entity in the electronic device that can implement the functions of the device embodiment, and the embodiment of the present application is not limited.

[0115] In the embodiment of the present application, each module can implement the voice wake-up method provided in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0116] Figure 8 The structural diagram of an electronic device provided in an embodiment of the present application is used to exemplify an electronic device that implements any voice wake-up method in an embodiment of the present application, and should not be understood as a specific limitation on the embodiment of the present application.

[0117] like Figure 8As shown, the electronic device 800 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0118] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although an electronic device 800 having various devices is shown, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0119] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, the computer program containing a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processor 801, the functions defined in any voice wake-up method provided in the embodiment of the present application can be executed.

[0120] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0121] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0122] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0123] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device implements any voice wake-up method.

[0124] In an embodiment of the present application, a computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include but are not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0125] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] The units involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

[0127] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0128] In the context of the present application, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] An embodiment of the present application also provides a vehicle, comprising: the above-mentioned voice wake-up device, or the above-mentioned electronic device, or the above-mentioned computer-readable storage medium.

[0130] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0131] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0132] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A voice wake-up method, characterized in that: The method comprises: Collecting target voice signals; When it is detected that the target voice signal includes a wake-up word and the target voice signal is generated in a vehicle cabin based on the position model, determining a current gear position of the vehicle; When it is determined based on the current gear that the target voice signal is a voice signal in the cabin, the voice assistant in the cabin is woken up.

2. The method according to claim 1, characterized in that The step of determining, based on the current gear position, that the target voice signal is a voice signal in the cabin includes: When the current gear is not the P gear, determining that the target voice signal is a voice signal in the cabin; or, When the current gear position is the P gear, determining whether a human body is included in the cabin; In the case where a human body is included in the cabin, the target voice signal is determined to be a voice signal in the cabin.

3. The method according to claim 2, characterized in that The determining whether the cabin includes a human body comprises: identifying whether the image in the cabin includes a human body; In a case where the image in the cabin includes a human body, determining that the cabin includes a human body; In a case where the image in the cabin does not include a human body, it is determined that the cabin does not include a human body.

4. The method according to any one of claims 1 to 3, characterized in that The model training process of the position model includes: Acquire training sample data, the training sample data comprising a plurality of in-cabin training samples and a plurality of out-cabin training samples, each in-cabin training sample comprising an in-cabin sample voice signal and an in-cabin generation position corresponding to the in-cabin sample voice signal, and each out-cabin training sample comprising an out-cabin sample voice signal and an out-cabin generation position corresponding to the out-cabin sample voice signal; The basic model based on the neural network is trained using the training sample data to obtain the position model.

5. The method according to claim 4, characterized in that The step of training a basic model based on a neural network by using the training sample data to obtain the position model includes: The basic model based on the multi-scale dilated deep temporal convolutional network MDTC is trained through the training sample data to obtain the position model.

6. The method according to claim 5, characterized in that The method of training a basic model based on a multi-scale dilated deep temporal convolutional network (MDTC) by using the training sample data to obtain the position model includes: Performing feature extraction on a sample speech signal in each training sample in the training sample data to obtain a speech feature corresponding to each training sample, wherein the speech feature is used to indicate frequency spectrum information of the corresponding training sample, and when the training sample is an in-cabin training sample, the sample speech signal is an in-cabin sample speech signal, and when the training sample is an out-of-cabin training sample, the sample speech signal is an out-of-cabin sample speech signal; Classifying the speech features corresponding to each training sample through the multi-scale dilated deep temporal convolutional network MDTC to obtain a prediction vector when the predicted generation position of each training sample is the actual generation position; Normalizing the prediction vector corresponding to each training sample to obtain a probability value corresponding to each training sample; Based on the loss function and the probability value corresponding to each training sample, determining the loss value corresponding to the multi-scale dilated deep temporal convolutional network MDTC; Based on the loss value, the model parameters of the multi-scale dilated deep temporal convolutional network MDTC are optimized by an optimization algorithm to obtain the position model.

7. The method according to claim 4, characterized in that The obtaining of training sample data comprises: Collecting multiple in-cabin audio signals, the multiple in-cabin audio signals include: at least one command audio signal responded to by the voice assistant in the cabin when it is in an awake state, at least one in-cabin human voice audio signal collected by a microphone in the cabin, and at least one recorded audio signal played by an in-cabin device collected by a microphone in the cabin; Determine the audio signal that meets the target condition in the at least one in-cabin human voice audio signal and the at least one recorded audio signal and the in-cabin generation position corresponding to the audio signal, as well as the at least one command audio signal and the in-cabin generation position corresponding to each of the command audio signals as the multiple in-cabin training samples; Wherein, the target condition includes: an audio signal whose decibel value is within a preset range in the cabin.

8. The method according to claim 4, characterized in that The obtaining of training sample data comprises: A plurality of out-of-cabin audio signals are collected by a microphone in the cabin, wherein the plurality of out-of-cabin audio signals include: at least one out-of-cabin human voice audio signal and at least one recorded audio signal played by an out-of-cabin device; and the distance between the out-of-cabin generation position of each out-of-cabin audio signal and the position of the vehicle is less than or equal to a distance threshold; The audio signals whose decibel values ​​are within a preset range outside the cabin, and the corresponding locations of the audio signals generated outside the cabin, among the multiple outside-cabin audio signals, are determined as the multiple outside-cabin training samples, wherein the starting value of the preset range outside the cabin is greater than the starting value of the preset range inside the cabin.

9. A voice wake-up device, characterized in that: include: A collection unit, used for collecting target voice signals; a determination unit, configured to determine a current gear position of the vehicle when it is detected that the target voice signal includes a wake-up word and when it is identified based on a position model that a generation location of the target voice signal is inside a cabin of the vehicle; A wake-up unit is used to wake up the voice assistant in the cabin when it is determined that the target voice signal is a voice signal in the cabin based on the current gear position.

10. An electronic device, characterized in that: include: A processor, wherein the processor is used to execute a computer program stored in a memory, and when the computer program is executed by the processor, the steps of the voice wake-up method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the voice wake-up method according to any one of claims 1 to 8 are implemented.

12. A vehicle, characterized in that: include: The voice wake-up device as claimed in claim 9, or the electronic device as claimed in claim 10, or comprising the computer-readable storage medium as claimed in claim 11.

Images