In-vehicle voice assistant control method and device, electronic equipment and vehicle

Abstract

The invention relates to an in-vehicle voice assistant control method and device, electronic equipment and a vehicle, and relates to the technical field of vehicle control, and the method comprises the steps: obtaining voice information collected by microphone equipment, and carrying out the voice recognition of the voice information; if it is recognized that the voice information contains a keyword for waking up an in-vehicle voice assistant, an audio signal corresponding to the voice information is analyzed, and sound source directivity information is obtained; based on the sound source directivity information, whether the audio signal is transmitted in a multi-path mode or not is judged; and if the audio signal is transmitted in a multi-path manner, determining that the wake-up behavior corresponding to the voice information is a behavior of waking up the in-vehicle voice assistant outside the vehicle, and controlling not to wake up the in-vehicle voice assistant. According to the technical scheme, whether awakening of the voice assistant in the vehicle comes from the outside of the vehicle or not can be accurately judged, then corresponding suppression control is accurately carried out, sound insulation materials do not need to be additionally arranged on the vehicle body, and cost can be saved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and specifically to a control method, device, electronic equipment and vehicle for an in-vehicle voice assistant. Background Art

[0002] With the rapid development of vehicle technology, in-car voice assistants receive user voices and control some functions of the vehicle to help users solve problems, providing a lot of convenience for users to drive vehicles. Since the in-car voice assistant wakes up by receiving voices, the noise outside the vehicle may cause the in-car voice assistant to wake up incorrectly, and then the in-car voice assistant needs to be suppressed from being woken up.

[0003] At present, more sound insulation materials can be added to the car body to isolate the sound from outside the car as much as possible and reduce the probability of the voice assistant being awakened in the car. However, this method will not only increase the cost, but also these sound insulation materials will increase the weight of the vehicle, resulting in a reduction in the vehicle's endurance. Summary of the invention

[0004] In view of this, the present application provides a control method, device, electronic device and vehicle for an in-car voice assistant. The main purpose is to improve the current method of adding sound insulation materials to the car body to isolate the sound from outside the car from interfering with the in-car voice assistant, which will increase costs. In addition, these sound insulation materials will increase the vehicle's own weight, leading to technical problems such as reducing the vehicle's endurance.

[0005] In a first aspect, the present application provides a method for controlling an in-vehicle voice assistant, comprising:

[0006] Acquire voice information collected by the microphone device, and perform voice recognition on the voice information;

[0007] If it is recognized that the voice information contains a keyword for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain sound source directionality information;

[0008] Based on the sound source directivity information, determining whether the audio signal is multipath propagated;

[0009] If the audio signal is multi-path propagated, it is determined that the wake-up behavior corresponding to the voice information is a behavior of waking up the in-car voice assistant outside the car, and the in-car voice assistant is controlled not to be woken up.

[0010] Optionally, parsing the audio signal corresponding to the voice information to obtain sound source directivity information includes:

[0011] Calculating a data covariance matrix according to the audio signal;

[0012] Performing noise removal on the data covariance matrix to obtain an estimated signal covariance matrix;

[0013] Performing eigendecomposition and calculation on the estimated signal covariance matrix to obtain a main eigenvalue concentration, and using the main eigenvalue concentration as the sound source directivity information.

[0014] Optionally, judging whether the audio signal is multipath propagated based on the sound source directivity information includes:

[0015] Comparing the main eigenvalue concentration with a preset threshold value;

[0016] If the main eigenvalue concentration is less than the preset threshold value, it is determined that the audio signal is multipath propagated.

[0017] Optionally, calculating a data covariance matrix according to the audio signal includes:

[0018] Extracting signal data of a time period in which the keyword appears in the audio signal;

[0019] The signal data is calculated to obtain a data covariance matrix.

[0020] Optionally, performing noise removal processing on the data covariance matrix to obtain an estimated signal covariance matrix includes:

[0021] A noise covariance matrix corresponding to the estimated noise energy of the audio signal is subtracted from the data covariance matrix to obtain an estimated signal covariance matrix.

[0022] Optionally, the estimated signal covariance matrix is ​​subjected to eigendecomposition and calculated to obtain a main eigenvalue concentration, including:

[0023] Calculating the audio signal to obtain signal-to-noise ratios at different frequencies;

[0024] Selecting a target frequency point whose signal-to-noise ratio is greater than a preset threshold value among the different frequency points;

[0025] Performing eigendecomposition on a signal covariance matrix estimated corresponding to the target frequency point;

[0026] Based on the matrix eigenvalues ​​obtained by eigendecomposition, the main eigenvalue concentration is calculated.

[0027] Optionally, after determining whether the audio signal is multipath propagated based on the sound source directivity information, the method further includes:

[0028] If the audio signal is propagated in a single direct path, it is determined that the wake-up behavior corresponding to the voice information is an behavior of waking up the in-car voice assistant in the car, and control is performed to wake up the in-car voice assistant.

[0029] In a second aspect, the present application provides a control device for an in-vehicle voice assistant, comprising:

[0030] An acquisition module is configured to acquire voice information collected by a microphone device and perform voice recognition on the voice information;

[0031] The parsing module is configured to parse the audio signal corresponding to the voice information to obtain the sound source directionality information if it is recognized that the voice information contains a keyword for waking up the in-vehicle voice assistant;

[0032] A determination module, configured to determine whether the audio signal is multipath propagated based on the sound source directivity information;

[0033] The control module is configured to determine that the wake-up behavior corresponding to the voice information is an act of waking up the in-car voice assistant outside the car if the audio signal is multi-path propagation, and control not to wake up the in-car voice assistant.

[0034] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0035] In a fourth aspect, the present application provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0036] In a fifth aspect, the present application provides a vehicle, comprising: the device as described in the second aspect or the electronic device as described in the fourth aspect.

[0037] By means of the above technical scheme, a control method, device, electronic device and vehicle for an in-car voice assistant are provided. Compared with the current existing technology, the present application can determine whether the wake-up of the in-car voice assistant comes from outside the car only through the in-car microphone device when the vehicle stops, the doors and windows are closed and the in-car system is powered on, and if so, suppress it. Specifically, firstly, the voice information collected by the microphone device is obtained, and the voice information is voice recognized; if the voice information is identified to contain keywords for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain the sound source directivity information; then based on the sound source directivity information, it is determined whether the audio signal is multi-path propagation; if the audio signal is multi-path propagation, it is determined that the wake-up behavior corresponding to the voice information is the behavior of waking up the in-car voice assistant from outside the car, and the in-car voice assistant is controlled not to be awakened. By applying the technical scheme of the present application, it is possible to accurately determine whether the wake-up of the in-car voice assistant comes from outside the car, and then accurately perform the corresponding suppression control, and the present technical scheme does not need to add additional sound insulation materials to the vehicle body, which can save costs, and will not increase the vehicle's own weight, avoiding affecting the vehicle's endurance.

[0038] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A schematic diagram of a flow chart of a method for controlling an in-vehicle voice assistant provided in an embodiment of the present application is shown;

[0042] Figure 2 A schematic diagram showing an application scenario provided by an embodiment of the present application is shown;

[0043] Figure 3 A schematic diagram showing another application scenario provided by an embodiment of the present application is shown;

[0044] Figure 4 A schematic diagram of a flow chart of another method for controlling an in-vehicle voice assistant provided in an embodiment of the present application is shown;

[0045] Figure 5 A schematic diagram of a process provided by an embodiment of the present application is shown;

[0046] Figure 6 A schematic structural diagram of a control device for an in-vehicle voice assistant provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0048] This embodiment provides a method for controlling an in-vehicle voice assistant. Figure 1 As shown, the method includes:

[0049] Step 101: Acquire voice information collected by a microphone device and perform voice recognition on the voice information.

[0050] Among them, the microphone device may include a distributed microphone array, and these microphones are dispersedly arranged in the space where the sound source is located. The distance between the sound source and the microphones and the distance between the microphones are at a similar order of magnitude. It should be noted that this embodiment has no limitation on the arrangement of the microphone device.

[0051] The microphone device has the function of collecting voice information and outputting voice information; the voice information may include voice information inside the vehicle and voice information outside the vehicle; voice recognition is a high-tech technology that allows machines to convert voice signals into corresponding text or commands through recognition and understanding. It is widely used in various fields such as industry, home appliances, communications, automotive electronics, medical care, home services, consumer electronics, etc.

[0052] In a specific application scenario, when acquiring voice information collected by a microphone device and performing voice recognition on the voice information, the method of this embodiment may include: using a communication device to acquire the voice information collected by the microphone device, and using a voice recognition tool to perform voice recognition on the voice information. This embodiment does not limit this.

[0053] Step 102: If it is recognized that the voice information contains keywords for waking up the in-car voice assistant, the audio signal corresponding to the voice information is analyzed to obtain sound source directionality information.

[0054] Among them, the in-car voice assistant may refer to an intelligent voice system that is combined with a voice assistant service to execute simple commands and greatly improve driving safety in order to avoid being distracted by reading screen content for a long time while driving; the keyword may be a pre-set word that triggers the in-car voice assistant to start working, and the keyword may specifically include a single word or multiple words. In a specific application scenario, if it is recognized that the voice information contains a keyword for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain the sound source directionality information, and the method of this embodiment may include: if the voice assistant or the vehicle host recognizes that the voice information contains a keyword for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain the sound source directionality information, and this embodiment does not limit this.

[0055] The directivity of a sound source can be the response of the transducer in a specified plane through the center of the sound as a function of the direction of the emitted or incident sound wave when the frequency is fixed. When measuring the sound pressure level radiated by a sound source at a fixed distance from the sound source in a given frequency band, it is often found that the sound pressure level is different in different directions from the sound source. The low-frequency radiation of many noise sources is almost non-directional, and its directivity increases with increasing frequency. This is because the sound waves radiated from different parts of the vibration source arrive at different points in space at different times, so there is interference and uneven directional radiation.

[0056] like Figure 2 and Figure 3 As shown, for the case of in-car wake-up, due to the small reflection in the car, the microphone device mainly receives the audio signal of a single direct path; for the case of outside-car wake-up, due to the obstruction of the car body and windows, no direct path can be formed, and the sound can only be transmitted to the car through the vibration of the solid, thereby changing the point sound source into a surface sound source, resulting in a multi-path propagation effect. Based on the two different propagation situations, this embodiment utilizes the characteristics of voice signal propagation to avoid the disadvantage that the distributed array in the microphone device cannot directly utilize the high-frequency directional characteristics, so that the out-of-window wake-up judgment can be performed only by relying on the distributed array, that is, based on the audio signal, the audio signal is analyzed, and then the directionality of the sound source is determined according to the sound frequency and direction. The sound directionality is used to determine whether the audio signal is multi-path propagation, and then distinguish whether the voice information comes from inside or outside the car, which is used to determine whether the voice information is a user instruction inside the car. The voice assistant can determine whether to execute the operation corresponding to the voice information based on the above analysis.

[0057] Step 103: Based on the sound source directivity information, determine whether the audio signal is multi-path propagated.

[0058] In some examples, the audio signal may be transmitted through a sound transmission medium inside the vehicle and through a sound transmission medium outside the vehicle, the sound transmission medium inside the vehicle may be air transmission, and the sound transmission medium outside the vehicle may be vehicle glass or vehicle shell and air transmission, etc.; multi-path transmission may be sound transmission through multiple paths formed by wake-up outside the vehicle, wherein wake-up outside the vehicle may be when the vehicle is stopped, the windows and doors are closed, and the system inside the vehicle is powered on, and a person outside the vehicle wakes up the voice assistant inside the vehicle. This embodiment does not limit this.

[0059] For an embodiment of the present application, when judging whether an audio signal is multi-path propagated based on the sound source directionality information, the method of this embodiment may include: judging whether the audio signal is multi-path propagated based on the sound source directionality information through a voice assistant or a vehicle host, and this embodiment does not limit this.

[0060] Step 104: If the audio signal is multi-path propagated, determine that the wake-up behavior corresponding to the voice information is a behavior of waking up the voice assistant inside the car from outside the car, and control not to wake up the voice assistant inside the car.

[0061] For the embodiment of the present application, if the audio signal is determined to be multi-path propagation through the above-mentioned identification and analysis, then it can be determined that the current wake-up behavior is the behavior of waking up the voice assistant inside the car from outside the car. For example, at this time, the voice assistant inside the car can be controlled not to be woken up.

[0062] The embodiment of the present application provides a control method for an in-car voice assistant, which can save costs, does not increase the vehicle's own weight, and avoids affecting the vehicle's endurance. Specifically, first obtain the voice information collected by the microphone device, and perform voice recognition on the voice information; if the voice information is identified to contain keywords for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain the sound source directivity information; then based on the sound source directivity information, determine whether the audio signal is multi-path propagation; if the audio signal is multi-path propagation, determine that the wake-up behavior corresponding to the voice information is the behavior of waking up the in-car voice assistant from outside the vehicle, and control not to wake up the in-car voice assistant. By applying the technical solution of the embodiment of the present application, it is possible to accurately determine whether the wake-up of the in-car voice assistant comes from outside the vehicle, and then accurately perform corresponding suppression control, and the technical solution does not need to add additional sound insulation materials to the vehicle body, which can save costs, and does not increase the vehicle's own weight, avoiding affecting the vehicle's endurance.

[0063] Further, in order to facilitate understanding of the implementation process of this embodiment, the principle content involved in this embodiment is specifically introduced below. It should be noted that the following example content is only one optional way of the method of this embodiment, and this embodiment does not limit the specific implementation method of the method.

[0064] Regarding the wake-up methods of the in-car voice assistant, it can be divided into in-car wake-up and outside-car wake-up.

[0065] For in-car wake-up, since the reflection inside the car is small, the microphone mainly receives the audio signal of the direct path. The data model can be shown as the following formula 1:

[0066] x(t)=d θ *s(t)+n(t) (Formula 1)

[0067] In formula 1, x(t) represents the microphone received signal vector, d θ represents the steering vector of the sound source position, s(t) represents the sound source signal, and n(t) represents the noise signal vector.

[0068] Furthermore, the data covariance matrix obtained for the in-vehicle wake-up data can be expressed as the following formula 2:

[0069]

[0070] In Formula 2, R represents the covariance matrix in the single-path propagation scenario, that is, the data covariance matrix obtained by the in-vehicle wake-up data, E represents the mean operation, x(t) represents the microphone received signal vector, and d θ The steering vector representing the sound source position, σ 2 represents the noise power, H represents the conjugate transpose operation, and I represents the identity matrix.

[0071] In specific application scenarios, when waking up outside the car, due to the obstruction of the car body and windows, no direct path can be formed. The sound can only be transmitted into the car through the vibration of solids, thus changing from a point sound source to a surface sound source, resulting in a multipath propagation effect. Assuming that there are P propagation paths, the data model can be shown as follows in Formula 3:

[0072]

[0073] In formula 3, x c (t) represents the microphone receiving signal vector in the multipath propagation scenario, that is, the data covariance matrix obtained by the wake-up data outside the vehicle, d θ represents the steering vector of the sound source position, s(t) represents the sound source signal, and n(t) represents the noise signal vector.

[0074] Furthermore, the data covariance matrix obtained for the external wake-up data can be expressed as the following formula 4:

[0075]

[0076] In formula 4, R c represents the covariance matrix in the multipath propagation scenario, E represents the mean operation, x c(t) represents the microphone received signal vector in the multipath propagation scenario, The steering vector representing the sound source position, σ 2 represents the noise power, H represents the conjugate transpose operation, and I represents the identity matrix.

[0077] In the above formula 4 and formula 5, there may be noise data. In this embodiment, the noise data may be removed to obtain an estimated signal covariance matrix.

[0078] For example, the estimated signal covariance matrix of in-car wake-up can be expressed as follows:

[0079]

[0080] In formula 5, R s represents the estimated signal covariance matrix of the in-car wake-up, R represents the covariance matrix, σ 2 represents the noise power, I represents the unit matrix, d θ represents the steering vector of the sound source position, and H represents the conjugate transpose operation.

[0081] The estimated signal covariance matrix of the vehicle external wake-up can be expressed as follows:

[0082]

[0083] In formula six, represents the estimated signal covariance matrix of the external wake-up, R c represents the covariance matrix, σ 2 represents the noise power, I represents the identity matrix, represents the steering vector of the sound source position, and H represents the conjugate transpose operation.

[0084] Based on Formula 5 and Formula 6, it can be analyzed that since the in-car wake-up is a direct path propagation, the obtained signal covariance matrix is ​​a rank-one matrix, while the signal covariance matrix rank of the out-car wake-up is greater than 1. Therefore, the in-car wake-up and out-car wake-up can be identified based on this feature.

[0085] According to the above exemplary principle content, this embodiment provides the following Figure 4 The specific method shown comprises the following steps:

[0086] Step 201: Acquire voice information collected by a microphone device and perform voice recognition on the voice information.

[0087] For the specific implementation process of the embodiment of the present application, please refer to the relevant description in step 101 of the embodiment, which will not be repeated here.

[0088] Step 202: Calculate and obtain a data covariance matrix based on the audio signal.

[0089] For the embodiment of the present application, the data covariance matrix may be a covariance matrix obtained from audio signal data.

[0090] Optionally, step 202 of the embodiment may specifically include: extracting signal data of the time period in which the keyword appears in the audio signal; calculating the signal data to obtain a data covariance matrix. For this embodiment, whether it is the case of waking up inside the car or waking up outside the car, the data covariance matrix will be calculated based on the audio information of the voice information, such as the form of formula 2 or the form of formula 4 above. Whether it is the case of waking up inside the car or waking up outside the car can be determined based on further analysis results, such as executing the process shown in steps 203 to 205.

[0091] Step 203: Perform noise removal on the data covariance matrix to obtain an estimated signal covariance matrix.

[0092] Optionally, step 203 may specifically include: subtracting a noise covariance matrix corresponding to the noise energy estimated by the audio signal from the data covariance matrix to obtain an estimated signal covariance matrix.

[0093] This embodiment can estimate the noise energy, and then obtain the noise data in the audio signal, such as the noise covariance matrix corresponding to the noise energy estimated by the audio signal. The data covariance matrix is ​​removed by using the estimated noise energy, such as subtracting the noise covariance matrix corresponding to the noise energy estimated by the audio signal from the data covariance matrix to obtain an estimated signal covariance matrix, which can be in the form shown in the above formula 5 or formula 6.

[0094] Step 204: perform eigendecomposition and calculation on the estimated signal covariance matrix to obtain the main eigenvalue concentration, and use the main eigenvalue concentration as the sound source directivity information.

[0095] Optionally, step 204 may specifically include: first calculating the audio signal to obtain the signal-to-noise ratio of different frequency points; then selecting a target frequency point whose signal-to-noise ratio is greater than a preset threshold among the different frequency points; then performing eigendecomposition on the estimated signal covariance matrix corresponding to the target frequency point, and then calculating the main eigenvalue concentration based on the matrix eigenvalues ​​obtained by the eigendecomposition.

[0096] In specific application scenarios, there will be certain deviations in noise estimation. If a frequency point with a relatively small signal-to-noise ratio is selected to obtain the corresponding estimated signal covariance matrix, when the matrix is ​​subjected to eigendecomposition, the corresponding main eigenvalue is not very stable, which affects the accuracy of subsequent evaluation. Therefore, this embodiment can select a frequency point with a signal-to-noise ratio greater than a certain threshold to obtain the corresponding estimated signal covariance matrix. When the matrix is ​​subjected to eigendecomposition, the corresponding main eigenvalue is relatively stable, and then the in-vehicle wake-up and out-of-vehicle wake-up can be accurately judged based on the concentration of the main eigenvalue.

[0097] When the estimated signal covariance matrix is ​​subjected to eigendecomposition, multiple eigenvalues ​​λ may be obtained, and the largest eigenvalue among them is selected as the main eigenvalue.

[0098] For example, the main eigenvalue concentration is obtained by using the following formula 7:

[0099]

[0100] In Formula 7, ε is the main eigenvalue concentration, λ1 is the main eigenvalue, M is the number of matrix eigenvalues, and λi is the i-th eigenvalue.

[0101] Step 205: compare the main eigenvalue concentration with a preset threshold value. If the main eigenvalue concentration is less than the preset threshold value, it is determined that the audio signal is multipath propagated.

[0102] For the embodiment of the present application, the absolute value obtained by subtracting the main eigenvalue concentration from 1 is compared with a preset threshold value to determine a reference value, and then the reference value is used to determine whether the audio signal is single-path propagation or multi-path propagation.

[0103] Optionally, through step 204 of the above embodiment, it can be obtained that the main eigenvalue concentration is close to 1 when waking up in the car, and is significantly less than 1 when waking up outside the car, so that it can be distinguished whether it is waking up outside the car. A threshold value is preset, the main eigenvalue concentration is subtracted from 1, and an absolute value analysis is performed to obtain the concentration difference. If the concentration difference is less than the preset threshold value, the voice wake-up signal is an in-car wake-up; if the concentration difference is greater than the preset threshold value, the voice wake-up signal is an outside-car wake-up. Among them, the threshold value is based on the difference between the in-car wake-up and 1, and then absolute value processing is performed; and the difference between the outside-car wake-up and 1 is subtracted, and then absolute value processing is performed, and then the absolute value corresponding to the above-mentioned in-car wake-up and the absolute value corresponding to the outside-car wake-up are comprehensively counted to determine a threshold value.

[0104] For example, the absolute value A corresponding to the in-car wake-up minus 1 and the absolute value B corresponding to the out-car wake-up minus 1 are counted respectively, wherein the absolute value A includes 0.3, 0.5, 0.2, 0.4, 0.3, and the absolute value B includes: 0.6, 0.7, 0.5, 0.8, 0.9. After statistics, the average value of the absolute value A is 0.34, and the average value of the absolute value B is 0.7. And from the above embodiment, it is known that the concentration of the main characteristic values ​​of the in-car wake-up is close to 1, while the concentration of the main characteristic values ​​of the out-car wake-up is significantly less than 1, and the average value of the absolute value A of 0.34 is less than the average value of the absolute value B of 0.7, then the above values ​​are valid. Then, the average value of the absolute value A of 0.34 is added to the average value of the absolute value B of 0.7, and divided by 2, and the quotient of 0.52 is used as the threshold value. The current main eigenvalue concentration TZ is extracted as 0.45 and subtracted from 1. The absolute value of the difference 0.55 is the concentration difference. If the concentration difference 0.55 is greater than the threshold value 0.52, it is determined that the audio signal is multipath propagation.

[0105] Through the specific implementation of the above embodiment, it can be determined that the voice information is single-path propagation or multi-path propagation. The single-path propagation corresponds to in-car wake-up; the multi-path propagation corresponds to outside-car wake-up, and then the voice information can be determined as in-car wake-up information or outside-car wake-up information, so as to process the voice information relatively, avoiding the impact of wrong operation on user experience and potential safety hazards. At the same time, there will be no wake-up failure caused by the low voice of the person in the car waking up the voice assistant.

[0106] Step 206: If the audio signal is multi-path propagated, determine that the wake-up behavior corresponding to the voice information is a behavior of waking up the voice assistant inside the car from outside the car, and control not to wake up the voice assistant inside the car.

[0107] For the embodiments of the present application, in some application scenarios, it may also include: if the wake-up behavior is an behavior of waking up an in-car voice assistant in the car, then controlling the waking up of the in-car voice assistant.

[0108] Specifically, Figure 2 and Figure 3 The schematic diagram of an application scenario shown in the figure shows that for in-car wake-up, due to the small reflection inside the car, the microphone mainly receives the audio signal of the direct path; when waking up outside the car, due to the obstruction of the car body and windows, no direct path can be formed, and the sound can only be transmitted into the car through the vibration of solids, thus changing the point sound source into a surface sound source, resulting in a multipath propagation effect.

[0109] The embodiment of the present application provides a control method for an in-car voice assistant, which can save costs, does not increase the vehicle's own weight, and avoids affecting the vehicle's endurance. By utilizing the characteristics of signal propagation, the disadvantage that the distributed array cannot directly utilize the high-frequency directional characteristics is avoided, so that the out-of-window wake-up judgment can be performed only by relying on the distributed array, wherein the distributed microphone array can be a microphone dispersedly arranged in the space where the sound source is located, and the distance between the sound source and the microphone and the microphone spacing are at a similar order of magnitude; at the same time, it does not rely on other sensors such as images, and only uses the microphone signal to determine whether it is a specific wake-up outside the car. First, the voice information collected by the microphone device is obtained, and the voice information is recognized; if the voice information is recognized to contain keywords for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain the sound source directional information; then based on the sound source directional information, it is determined whether the audio signal is multi-path propagation; if the audio signal is multi-path propagation, it is determined that the wake-up behavior corresponding to the voice information is the behavior of waking up the in-car voice assistant outside the car. By applying the technical solution of the embodiments of the present application, it is not necessary to rely on other sensors such as images, the cost is low, and distributed arrays are supported. There are no strict requirements for the array type and layout, and the processing is simple and the accuracy is high.

[0110] Figure 5 A schematic diagram of a process provided in an embodiment of the present application is shown.

[0111] The wake-up module is used to identify the audio signal within the time period when the keyword appears in the audio signal. The wake-up module inputs the voice signal in the time period when the wake-up word appears into the outside vehicle wake-up judgment module; the judgment module calculates the data covariance matrix through the obtained time period information, specifically by calculating the frequency point signal-to-noise ratio, and selects the target frequency point with a higher signal-to-noise ratio. For the target frequency point, the noise data is estimated by the noise energy estimation module, and the estimated signal covariance matrix corresponding to the target frequency point is obtained after removing the noise data; then the estimated signal covariance matrix is ​​eigen-decomposed and the main eigenvalue concentration is calculated, and it is compared with the threshold to determine whether it is an outside vehicle wake-up. If not, the wake-up result is accepted and the voice assistant responds.

[0112] By applying the technical solution of this embodiment, it is possible to accurately determine whether the wake-up of the in-car voice assistant comes from outside the car, and then accurately perform corresponding suppression control. In addition, this technical solution does not require additional sound insulation materials to the car body, which can save costs and will not increase the vehicle's own weight, thereby avoiding affecting the vehicle's endurance.

[0113] Further, as Figure 1 and Figure 4 The specific implementation of the method shown in the embodiment provides a control device for an in-vehicle voice assistant, such as Figure 6As shown, the device includes: an acquisition module 31, an analysis module 32, a judgment module 33, and a control module 34.

[0114] The acquisition module 31 is configured to acquire the voice information collected by the microphone device and perform voice recognition on the voice information;

[0115] The parsing module 32 is configured to parse the audio signal corresponding to the voice information to obtain the sound source directivity information if it is recognized that the voice information contains a keyword for waking up the in-vehicle voice assistant;

[0116] A judging module 33 is configured to judge whether the audio signal is multipath propagated based on the sound source directivity information;

[0117] The control module 34 is configured to determine that the wake-up behavior corresponding to the voice information is a behavior of waking up the in-car voice assistant outside the car if the audio signal is multi-path propagation, and control not to wake up the in-car voice assistant.

[0118] In a specific application scenario, the analysis module 32 is specifically configured to calculate a data covariance matrix based on the audio signal; perform noise removal on the data covariance matrix to obtain an estimated signal covariance matrix; perform eigendecomposition and calculation on the estimated signal covariance matrix to obtain a main eigenvalue concentration, and use the main eigenvalue concentration as the sound source directivity information.

[0119] In a specific application scenario, the judgment module 33 is specifically configured to compare the main eigenvalue concentration with a preset threshold value; if the main eigenvalue concentration is less than the preset threshold value, it is determined that the audio signal is multipath propagated.

[0120] In a specific application scenario, the analysis module 32 is further configured to extract signal data of the time period in which the keyword appears in the audio signal; calculate the signal data to obtain a data covariance matrix; and convert the data covariance matrix to obtain a noisy data covariance matrix.

[0121] In a specific application scenario, the analysis module 32 is further configured to subtract a noise covariance matrix corresponding to the estimated noise energy of the audio signal from the data covariance matrix to obtain an estimated signal covariance matrix.

[0122] In a specific application scenario, the analysis module 32 is further configured to calculate the audio signal to obtain the signal-to-noise ratio of different frequency points; select a target frequency point whose signal-to-noise ratio is greater than a preset threshold among the different frequency points; perform eigendecomposition on the estimated signal covariance matrix corresponding to the target frequency point; and calculate the main eigenvalue concentration based on the matrix eigenvalues ​​obtained by the eigendecomposition.

[0123] In a specific application scenario, the judgment module 33 is further configured to determine that the wake-up behavior corresponding to the voice information is an act of waking up the in-car voice assistant in the car if the audio signal is propagated in a single direct path, and control the waking up of the in-car voice assistant.

[0124] It should be noted that for other corresponding descriptions of the functional units involved in the control device of the in-vehicle voice assistant provided in this embodiment, please refer to Figure 1 and Figure 4 The corresponding description in will not be repeated here.

[0125] Based on the above Figure 1 and Figure 4 The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 4 The method shown.

[0126] Based on this understanding, the technical solution of this embodiment can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each implementation scenario of this embodiment.

[0127] Based on the above Figure 1 and Figure 4 The method shown, and Figure 6 In order to achieve the above-mentioned purpose, the embodiment of the present application also provides an electronic device that can be configured on the vehicle (such as a new energy vehicle or a traditional vehicle) side, and the device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 4 The method shown.

[0128] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0129] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0130] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0131] Based on the above electronic device, the embodiment of the present application further provides a vehicle, which may specifically include: Figure 6 The control device of the in-vehicle voice assistant shown, or the above-mentioned electronic device. The vehicle can be a new energy vehicle or a traditional vehicle.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the solution of this embodiment, compared with the current existing control method of the in-car voice assistant, the present application provides a control method, device, electronic device and vehicle for the in-car voice assistant, which can be used to determine whether the wake-up comes from outside the car only through the in-car microphone array when the vehicle is stopped, the doors and windows are closed and the in-car system is powered on. If so, it is suppressed. It can avoid the limited sound insulation effect of the car body, and increasing the sound insulation will further increase the hardware cost and reduce the battery life; avoid the poor wake-up suppression effect for the loud sound source close to the window position; and reduce the existing method of locating the sound source through the small-pitch microphone array in the car to determine whether it is outside the car. The restrictions on the layout of the microphone in the car are reduced. Specifically, first obtain the voice information collected by the microphone device, and perform voice recognition on the voice information; if the voice information is identified to contain keywords for waking up the in-car voice assistant, parse the audio signal corresponding to the voice information to obtain the sound source directionality information; then, based on the sound source directionality information, determine whether the audio signal is multi-path propagation; if the audio signal is multi-path propagation, determine that the wake-up behavior corresponding to the voice information is the behavior of waking up the in-car voice assistant from outside the car. By applying the technical solution of this application, it is not necessary to rely on other sensors such as images, the cost is low, and distributed arrays are supported. There are no strict requirements for array type and layout, and the processing is simple and the accuracy is high.

[0133] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0134] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

Claims

1. A method for controlling an in-vehicle voice assistant, characterized in that: The method comprises: Acquire voice information collected by the microphone device, and perform voice recognition on the voice information; If it is recognized that the voice information contains a keyword for waking up the in-car voice assistant, the audio signal corresponding to the voice information is parsed to obtain sound source directionality information; Based on the sound source directivity information, determining whether the audio signal is multipath propagated; If the audio signal is multi-path propagated, it is determined that the wake-up behavior corresponding to the voice information is a behavior of waking up the in-car voice assistant outside the car, and the in-car voice assistant is controlled not to be woken up.

2. The method according to claim 1, characterized in that The step of parsing the audio signal corresponding to the voice information to obtain the sound source directivity information includes: Calculating a data covariance matrix according to the audio signal; Performing noise removal on the data covariance matrix to obtain an estimated signal covariance matrix; Performing eigendecomposition and calculation on the estimated signal covariance matrix to obtain a main eigenvalue concentration, and using the main eigenvalue concentration as the sound source directivity information.

3. The method according to claim 2, characterized in that The determining, based on the sound source directivity information, whether the audio signal is multipath propagated includes: Comparing the main eigenvalue concentration with a preset threshold value; If the main eigenvalue concentration is less than the preset threshold value, it is determined that the audio signal is multipath propagated.

4. The method according to claim 2, characterized in that: The step of calculating a data covariance matrix based on the audio signal includes: Extracting signal data of a time period in which the keyword appears in the audio signal; The signal data is calculated to obtain a data covariance matrix.

5. The method according to claim 2, characterized in that: The step of performing noise removal processing on the data covariance matrix to obtain an estimated signal covariance matrix includes: A noise covariance matrix corresponding to the estimated noise energy of the audio signal is subtracted from the data covariance matrix to obtain an estimated signal covariance matrix.

6. The method according to claim 2, characterized in that The step of performing eigendecomposition and calculating the estimated signal covariance matrix to obtain the main eigenvalue concentration includes: Calculating the audio signal to obtain signal-to-noise ratios at different frequencies; Selecting a target frequency point whose signal-to-noise ratio is greater than a preset threshold value among the different frequency points; Performing eigendecomposition on a signal covariance matrix estimated corresponding to the target frequency point; Based on the matrix eigenvalues ​​obtained by eigendecomposition, the main eigenvalue concentration is calculated.

7. The method according to any one of claims 1 to 6, characterized in that After determining whether the audio signal is multipath propagated based on the sound source directivity information, the method further includes: If the audio signal is propagated in a single direct path, it is determined that the wake-up behavior corresponding to the voice information is an behavior of waking up the in-car voice assistant in the car, and control is performed to wake up the in-car voice assistant.

8. A control device for an in-vehicle voice assistant, characterized in that: include: An acquisition module is configured to acquire voice information collected by a microphone device and perform voice recognition on the voice information; The parsing module is configured to parse the audio signal corresponding to the voice information to obtain the sound source directionality information if it is recognized that the voice information contains a keyword for waking up the in-vehicle voice assistant; A determination module, configured to determine whether the audio signal is multipath propagated based on the sound source directivity information; The control module is configured to determine that the wake-up behavior corresponding to the voice information is a behavior of waking up the in-car voice assistant outside the car if the audio signal is multi-path propagation, and control not to wake up the in-car voice assistant.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: include: The device as claimed in claim 8 or the electronic device as claimed in claim 10.

Images