Voice control method and device, vehicle, storage medium and program product

Abstract

The present disclosure relates to the technical field of vehicle control, and particularly relates to a voice control method and device, a vehicle, a storage medium and a program product. The method comprises the following steps: in response to detecting a voice control signal, determining acoustic characteristic information of the voice control signal, the voice control signal being used for controlling a voice control system in a vehicle; determining a sound source position of the voice control signal according to at least the acoustic characteristic information; and if it is determined that the sound source position is an extravehicular position, controlling the voice control system not to be controlled by the voice control signal. In this way, the voice control system in the vehicle is prevented from being controlled by an extravehicular voice signal, driving safety is improved, unnecessary energy consumption is avoided, and the user driving experience is improved.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle control technology, and in particular to a voice control method, device, vehicle, storage medium, and program product. Background Technology

[0002] With the continuous maturation of vehicle-to-everything (V2X) architecture, automobiles have entered the era of intelligence. As one of the most common and simplest forms of human interaction, voice interaction has become the primary control method in smart cockpits. Due to limited interior space and the inconvenience of manual operation while driving, using voice commands for navigation, turning lights on / off, and playing music has become the most common in-vehicle interaction method to improve driving efficiency and experience.

[0003] However, because the voice acquisition device inside the vehicle can collect voice signals from both inside and outside the vehicle, these external voice signals can control the in-vehicle voice control system, posing a safety risk or causing unnecessary energy consumption. Therefore, to prevent the in-vehicle voice control system from being controlled by external voice signals, it is necessary to suppress external voice signals. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a voice control method, device, vehicle, storage medium, and program product.

[0005] According to a first aspect of the present disclosure, a voice control method is provided, the method comprising: In response to the detection of a voice control signal, acoustic feature information of the voice control signal is determined, the voice control signal being used to control the voice control system inside the vehicle; Based at least on the acoustic feature information, determine the sound source location of the voice control signal; If the location of the sound source is determined to be outside the vehicle, then the voice control system is controlled not to be controlled by the voice control signal.

[0006] Optionally, the acoustic feature information includes spectral flatness and / or speech loudness; the step of determining the acoustic feature information of the speech control signal in response to detecting the speech control signal includes: In response to the detection of a voice control signal, it determines whether the vehicle's doors and windows are closed; When the car door and the car window are in the closed state, the acoustic feature information of the voice control signal is determined.

[0007] Optionally, determining the acoustic feature information of the voice control signal when the vehicle door and the vehicle window are in the closed state includes: When the car door and the car window are closed, the voice control signal is converted into a voice control signal in the frequency domain; Based on the voice control signal in the frequency domain, determine the acoustic feature information of the voice control signal.

[0008] Optionally, determining the acoustic feature information of the voice control signal based on the voice control signal in the frequency domain includes: Based on the voice control signal in the frequency domain, determine the spectral flatness of the target frequency band in the voice control signal, wherein the target frequency band includes at least a high frequency band; The spectral flatness of the target frequency band is determined as the acoustic feature information of the voice control signal; Determining the sound source location of the voice control signal based at least on the acoustic feature information includes: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold, the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold, the sound source location of the voice control signal is determined to be outside the vehicle.

[0009] Optionally, determining the acoustic feature information of the voice control signal based on the voice control signal in the frequency domain includes: The voice control signal in the frequency domain is mapped to multiple Bark domain bands in the Bark domain, and the loudness value corresponding to each Bark domain band is determined. The Bark domain band with a center frequency less than or equal to the center frequency threshold is defined as the first Bark domain band, and the Bark domain band with a center frequency greater than the center frequency threshold is defined as the second Bark domain band. The sum of loudness values ​​corresponding to the first Bark domain band is determined as the low-frequency loudness, and the sum of loudness values ​​corresponding to the second Bark domain band is determined as the high-frequency loudness. The low-frequency loudness and the high-frequency loudness are determined as the acoustic feature information of the voice control signal; Determining the sound source location of the voice control signal based at least on the acoustic feature information includes: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the ratio of the high-frequency loudness to the low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the ratio of the high-frequency loudness to the low-frequency loudness is greater than or equal to a preset loudness threshold, the sound source location of the voice control signal is determined to be outside the vehicle.

[0010] Optionally, when the acoustic feature information includes spectral flatness and speech loudness, and the speech loudness includes high-frequency loudness and low-frequency loudness, determining the sound source location of the speech control signal at least based on the acoustic feature information includes: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold and the ratio of the high-frequency loudness to the low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold, and the ratio of the high-frequency loudness to the low-frequency loudness is greater than or equal to a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle.

[0011] Optionally, the method further includes: The area inside and outside the vehicle is divided into multiple points, and for each point, the spatial spectrum of the point is determined based on the guiding vector of the point relative to the voice acquisition device and the voice control signal acquired by the voice acquisition device. The point with the largest spatial spectrum is determined as the target point; Determining the sound source location of the voice control signal based at least on the acoustic feature information includes: Based on the target location and the acoustic feature information, the sound source location of the voice control signal is determined.

[0012] Optionally, the method further includes: If the location of the sound source is determined to be inside the vehicle, then the voice control system is controlled to respond to the voice control signal.

[0013] Optionally, the voice acquisition device for detecting the voice control signal is located inside the vehicle.

[0014] According to a second aspect of the present disclosure, a voice control device is provided, comprising: The first determining module is configured to determine the acoustic feature information of the voice control signal in response to the detection of the voice control signal, the voice control signal being used to control the voice control system in the vehicle. The second determining module is configured to determine the sound source location of the voice control signal based at least on the acoustic feature information. The first control module is configured to, if it is determined that the location of the sound source is outside the vehicle, control the voice control system to not be controlled by the voice control signal.

[0015] According to a third aspect of the present disclosure, a vehicle is provided, including: a processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to enable the vehicle to implement the voice control method described in the first aspect of the present disclosure.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the voice control method described in the first aspect of the present disclosure.

[0017] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the voice control method described in the first aspect of the present disclosure.

[0018] By adopting the above technical solution, when a voice control signal is detected, the acoustic feature information of the voice control signal is determined, and at least based on the acoustic feature information, it is determined whether the sound source location of the voice control signal is outside the vehicle. When the sound source location is determined to be outside the vehicle, the voice control system is controlled to prevent it from being controlled by the voice control signal, that is, the voice control signal emitted by the sound source outside the vehicle is suppressed. In this way, the voice control system inside the vehicle is prevented from being controlled by the voice signal outside the vehicle, improving driving safety, avoiding unnecessary energy consumption, and enhancing the user's driving experience.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] Figure 1 This is a schematic diagram illustrating a voice interaction scenario according to an exemplary embodiment.

[0022] Figure 2 This is a flowchart illustrating a voice control method according to an exemplary embodiment.

[0023] Figure 3 This is a schematic diagram illustrating an interior and exterior space of a vehicle according to an exemplary embodiment.

[0024] Figure 4This is a schematic diagram illustrating a voice control method according to an exemplary embodiment.

[0025] Figure 5 This is a block diagram illustrating a voice control device according to an exemplary embodiment.

[0026] Figure 6 This is a block diagram illustrating a vehicle according to an exemplary embodiment. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0028] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0029] As mentioned in the background section, with the increasing popularity of in-vehicle voice interaction systems, the issue of unauthorized external voice control of the in-vehicle voice control system has gradually attracted attention. When a vehicle is parked in a public place, it may receive voice interaction requests from people outside the vehicle. In the in-vehicle voice interaction environment, interference from external voice signals may cause the vehicle's infotainment system to be woken up, thereby reducing the accuracy and reliability of the voice control system.

[0030] Figure 1 This is a schematic diagram illustrating a voice interaction scenario according to an exemplary embodiment. For example... Figure 1 As shown, when a vehicle is parked in a public place such as a parking lot or roadside, or while driving on the road, the voice signals of any pedestrians approaching the vehicle can be captured by the microphone inside the car. These voice signals can then trigger the in-vehicle voice control system, allowing the user to access vehicle information or control vehicle functions, such as navigation, turning lights on / off, or playing music. This could lead to privacy breaches or unauthorized vehicle actions, causing security risks or unnecessary energy consumption. Therefore, it is necessary to suppress external voice signals and ensure that the in-vehicle voice control system only responds to in-vehicle voice signals to improve driver safety and convenience, and protect the safety and privacy of the vehicle owner.

[0031] In view of this, the present disclosure provides a voice control method, device, vehicle, storage medium, and program product. When a voice control signal is detected, the acoustic feature information of the voice control signal is determined, and at least based on the acoustic feature information, it is determined whether the sound source location of the voice control signal is outside the vehicle. When the sound source location is determined to be outside the vehicle, the voice control system is controlled to prevent it from being controlled by the voice control signal. That is, the voice control signal emitted from the sound source outside the vehicle is suppressed, and the voice control system inside the vehicle is prevented from being controlled by the voice signal outside the vehicle. This improves driving safety, avoids unnecessary energy consumption, and enhances the user's driving experience.

[0032] Figure 2 This is a flowchart illustrating a voice control method according to an exemplary embodiment. Figure 2 As shown, the voice control method may include the following steps.

[0033] In step S21, in response to the detection of a voice control signal, the acoustic feature information of the voice control signal is determined, and the voice control signal is used to control the voice control system in the vehicle.

[0034] In this disclosure, the voice control signal can be referred to as the voice interaction signal, used to interact with the in-vehicle voice control system. For example, the voice control information can be a voice signal to control the vehicle to play voice messages, a voice signal to control the vehicle to navigate, or a voice signal to control the vehicle's autonomous driving, etc. This disclosure does not specifically limit it in this regard.

[0035] It should be understood that the vehicle stores text that can control the voice control system, and then determines whether the voice signal is a voice control signal for controlling the voice control system based on the detected voice signal and the pre-stored text.

[0036] Furthermore, the voice control signal refers to the voice signal collected by a voice acquisition device installed inside the vehicle. This voice acquisition device can be a microphone or similar device installed within the vehicle. The voice control signal can originate from an in-vehicle sound source, such as a passenger or vehicle owner inside the vehicle, or from an external sound source, such as a pedestrian outside the vehicle. For example, the in-vehicle voice control system can be the vehicle's infotainment system.

[0037] The specific methods for determining the acoustic features of the voice control signal will be described below, and will not be repeated here.

[0038] In step S22, the location of the sound source of the voice control signal is determined based at least on the acoustic feature information.

[0039] In a smart cockpit, signals emitted from both inside and outside the vehicle are captured by a voice acquisition device installed within the vehicle. However, the voice signals captured by this device differ somewhat in acoustic characteristics from those outside the vehicle. For example, voice signals from outside the vehicle undergo complex acoustic processes such as reflection and refraction by the vehicle body before reaching the interior, or they may be attenuated by the vehicle body. In contrast, voice signals from inside the vehicle are easily captured by the voice acquisition device due to their airborne propagation. Therefore, the acoustic characteristics of voice signals inside and outside the vehicle differ, allowing the location of the voice control signal's source to be determined based on these acoustic features.

[0040] In addition, voice control signals can also refer to voice signals collected by a voice acquisition device installed outside the vehicle. When the voice acquisition device is installed outside the vehicle, the signal emitted from inside the vehicle will undergo complex acoustic processes such as reflection and refraction by the vehicle body before reaching the outside, or it may be attenuated by the vehicle body before reaching the outside. However, the voice signal outside the vehicle can be better acquired by the voice acquisition device through air propagation. Therefore, there are certain differences in acoustic characteristics between the voice signals inside and outside the vehicle. Thus, the location of the sound source of the voice control signal can be determined based on the acoustic characteristics of the voice control signal.

[0041] In step S23, if the location of the sound source is determined to be outside the vehicle, the voice control system is not controlled by the voice control signal.

[0042] In this disclosure, if the source location of the voice control signal is determined to be outside the vehicle, the voice control signal is suppressed to prevent the in-vehicle voice control system from being controlled by the external voice control signal. For example, when the source location of the control voice signal is outside the vehicle, the vehicle control unit is not woken up by the voice control signal.

[0043] By adopting the above technical solution, when a voice control signal is detected, the acoustic feature information of the voice control signal is determined, and at least based on the acoustic feature information, it is determined whether the sound source location of the voice control signal is outside the vehicle. When the sound source location is determined to be outside the vehicle, the voice control system is controlled to prevent it from being controlled by the voice control signal, that is, the voice control signal emitted by the sound source outside the vehicle is suppressed. In this way, the voice control system inside the vehicle is prevented from being controlled by the voice signal outside the vehicle, improving driving safety, avoiding unnecessary energy consumption, and enhancing the user's driving experience.

[0044] Furthermore, in this disclosure, if the location of the sound source is determined to be inside the vehicle, the voice control system is controlled to respond to the voice control signal to meet the voice interaction needs of the user inside the vehicle, thereby improving the flexibility of voice control.

[0045] In this disclosure, acoustic feature information includes spectral flatness and / or speech loudness.

[0046] As mentioned above, if the voice acquisition device used to detect voice control signals is located inside the vehicle, the voice control signals from outside the vehicle will be blocked by the vehicle body, doors, and windows when they reach the device, reducing the sound intensity. Therefore, the sound perceived by the human ear will be weaker, and the location of the sound source can be distinguished based on the loudness. Furthermore, the voice control signals from outside the vehicle undergo complex acoustic processes such as reflection and refraction at the vehicle body, doors, and windows. High-frequency components of the sound are often absorbed, while low-frequency components are more likely to propagate inside the vehicle, resulting in a muffled sound. Therefore, the location of the sound source can be distinguished based on the spectral flatness. Similarly, if the voice acquisition device used to detect voice control signals is located outside the vehicle, the signal strength inside the vehicle is reduced, and the high-frequency components of the signal inside the vehicle are often absorbed, while low-frequency components are more likely to propagate inside the vehicle, resulting in a muffled sound.

[0047] Considering that when the car doors and windows are closed, the external voice control signal attenuates more before reaching the car interior, or the internal voice control signal attenuates more before reaching the outside of the car, and the degree of reflection and refraction is greater, the difference in acoustic characteristics between the external and internal voice control signals acquired by the voice acquisition device will be greater, making it easier to distinguish between the external and internal voice control signals.

[0048] Therefore, in this disclosure, step S21, in response to detecting a voice control signal, determines the acoustic feature information of the voice control signal, which may include: in response to detecting a voice control signal, determining whether the vehicle doors and windows are in a closed state; and when the doors and windows are in a closed state, determining the acoustic feature information of the voice control signal.

[0049] By employing the above technical solution, once it is determined that the car doors and windows are closed, i.e., the interior of the vehicle is sealed, the acoustic characteristics of the voice control signal are then determined, and the location of the sound source is determined based on these acoustic characteristics. Thus, when the interior of the vehicle is sealed, the acoustic characteristics of the external and internal voice control signals differ significantly. This makes it easier to distinguish between the external and internal voice control signals, thereby improving the accuracy and reliability of determining the sound source location based on acoustic characteristics.

[0050] Furthermore, the determined acoustic feature information typically refers to acoustic features in the frequency domain. Therefore, it is necessary to convert the acquired voice control signal into a frequency domain voice control signal, and then determine the acoustic feature information based on the frequency domain voice control signal. For example, a specific implementation method for determining the acoustic feature information of the voice control signal when the doors and windows are closed can be: when the doors and windows are closed, convert the voice control signal into a frequency domain voice control signal; and determine the acoustic feature information of the voice control signal based on the frequency domain voice control signal.

[0051] Assuming the in-vehicle microphone array acquires the voice control signal x(t), an N-point Short Time Fourier Transform (STFT) can be used to convert the voice control signal x(t) into a frequency domain voice control signal X(t, f), where the voice signal frames can be labeled as follows. The frequency point can be marked as .

[0052] It should be understood that before employing the N-point short-time Fourier transform, the time-domain speech control signal can be windowed to reduce edge effects. For example, a window function can be multiplied on the time-domain speech control signal to reduce the impact of discontinuities and incompleteness at both ends of the speech control signal on the subsequent determination of acoustic feature information.

[0053] In one embodiment, the acoustic feature information includes spectral flatness. Determining the acoustic feature information of the voice control signal based on the voice control signal in the frequency domain may include: determining the spectral flatness of a target frequency band in the voice control signal, wherein the target frequency band includes at least a high-frequency band; and determining the spectral flatness of the target frequency band as the acoustic feature information of the voice control signal.

[0054] Spectral flatness of a speech signal is a parameter that measures the shape of the signal's spectrum and describes whether the energy distribution of the signal is uniform in the frequency domain. Higher spectral flatness indicates that the signal's spectrum is closer to white noise. Lower spectral flatness indicates that certain frequency components in the signal have a larger proportion of energy. Compared to in-vehicle voice control signals, external voice control signals are often absorbed by the vehicle body, windows, and doors in the mid-to-high frequency range, resulting in lower spectral flatness in this range. Therefore, the spectral flatness of the target frequency band can be used to distinguish between external and in-vehicle voice control signals. The target frequency band can be either mid-to-high frequency or high frequency.

[0055] For example, the spectral flatness (SF) of a speech signal can be calculated by taking the ratio of the geometric mean to the arithmetic mean of the signal's spectral amplitude. For instance, the spectral flatness of a target frequency band can be calculated using the following formula (1): (1) in, Characterizes the spectral flatness of the target frequency band. Characterizing the lower limit frequency point of the target frequency band. The upper frequency point that characterizes the target frequency band. Characterizing the amplitude of the signal spectrum, The number of frequency points between the lower and upper limits of the target frequency band.

[0056] After determining the spectral flatness of the target frequency band in the manner described above, the spectral flatness of the target frequency band is determined as the acoustic feature information of the voice control signal. Then, the sound source location of the voice control signal is determined based on the acoustic feature information.

[0057] For example, determining the sound source location of a voice control signal based at least on acoustic feature information may include: when the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold, then determining the sound source location of the voice control signal to be outside the vehicle; or, when the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold, then determining the sound source location of the voice control signal to be outside the vehicle.

[0058] For example, when the voice acquisition device used to detect the voice control signal is located inside the vehicle, if If the location of the voice control signal is determined to be outside the vehicle, then the location of the voice control signal source is determined to be inside the vehicle. Otherwise, the location of the voice control signal source is determined to be inside the vehicle. When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if... If the location is outside the vehicle, the sound source of the voice control signal is determined to be outside the vehicle. Otherwise, the sound source of the voice control signal is determined to be inside the vehicle.

[0059] In another embodiment, the acoustic feature information includes speech loudness. Speech loudness is a subjective perception, describing how the human ear perceives sound intensity. In this embodiment, determining the acoustic feature information of the speech control signal based on the speech control signal in the frequency domain may include: mapping the speech control signal in the frequency domain to multiple Bark bands in the Bark domain, and determining the loudness value corresponding to each Bark band; determining Bark bands with center frequencies less than or equal to a center frequency threshold as first Bark bands, and Bark bands with center frequencies greater than the center frequency threshold as second Bark bands; determining the sum of the loudness values ​​corresponding to the first Bark bands as low-frequency loudness, and the sum of the loudness values ​​corresponding to the second Bark bands as high-frequency loudness; and determining the low-frequency loudness and high-frequency loudness as the acoustic feature information of the speech control signal.

[0060] First, a specific implementation method for mapping the speech control signal in the frequency domain to multiple Bark domain bands in the Bark domain and determining the loudness value corresponding to each Bark domain band is described.

[0061] For example, firstly, the frequency domain speech control signal is converted into a frequency domain representation that matches the characteristics of human hearing, for example, converted to multiple Bark domain bands. Table 1 shows the mapping relationship between the Bark domain and the frequency domain. In Table 1, the frequency domain speech control signal is mapped to 1-24 Bark domain bands. Then, for each Bark domain band, the following steps are performed: determine the energy distribution of the Bark domain band in each frame of the speech control signal based on the upper limit frequency point, lower limit frequency point, and the power spectrum of each frequency point between the upper limit frequency point and the lower limit frequency point; determine the average energy of the Bark domain band in the speech control signal based on the energy distribution of the Bark domain band in each frame of the speech control signal; determine the sound pressure level corresponding to the Bark domain band based on the average energy of the Bark domain band in the speech control signal and a preset reference sound pressure level; determine the loudness value corresponding to the Bark domain band based on the sound pressure level corresponding to the Bark domain band and the sound pressure level corresponding to the center frequency of the Bark domain band.

[0062] Table 1

[0063] For example, the loudness value corresponding to the Bark domain band can be determined by the following steps: Step (a): Calculate the energy distribution of each Bark domain band in each frame signal using the following formula (2): (2) in, Characterizes the energy distribution of the i-th Bark domain band in each frame of signal. The lower limit frequency of the i-th Bark domain band is defined as the frequency point corresponding to the difference between the cutoff frequency and the bandwidth of the i-th Bark domain band. The frequency point that represents the upper limit of the i-th Bark domain band, that is, the frequency point corresponding to the cutoff frequency of the i-th Bark domain band. Characterize the power spectrum corresponding to frequency point f, and .

[0064] Step (b): Obtain the average energy of each Bark domain band in the entire speech control signal using either direct averaging or weighted averaging. .

[0065] Step (c) can be used to calculate the sound pressure level corresponding to each Bark domain using formula (3): (3) in, Characterizing the sound pressure level corresponding to the i-th Bark domain band, This represents the preset reference sound pressure. The reference sound pressure is typically set to the lowest sound pressure that the human ear can hear, approximately 20 μPa.

[0066] Step (d): Determine the loudness value corresponding to each Bark band. For example, the loudness value corresponding to each Bark band can be evaluated based on a loudness model. For instance, the loudness model proposed by Zwicker can be used for evaluation. Taking the evaluation using a simplified model as an example, the loudness value corresponding to each Bark band can be determined using the following formula (4): (4) in, Characterizes the loudness value corresponding to the i-th Bark domain band. It represents the sound pressure level corresponding to the center frequency of the i-th Bark domain band in a quiet state.

[0067] At this point, the loudness values ​​corresponding to the 24 Bark domain bands can be obtained.

[0068] Next, the Bark bands with center frequencies less than or equal to the center frequency threshold are designated as the first Bark band, and the Bark bands with center frequencies greater than the center frequency threshold are designated as the second Bark band. For example, assuming the center frequency threshold is 4000Hz, Bark bands 1 to 18 in Table 1 can be designated as the first Bark band, and Bark bands 19 to 24 can be designated as the second Bark band.

[0069] Then, the sum of the loudness values ​​corresponding to the first domain band is determined as the low-frequency loudness, and the sum of the loudness values ​​corresponding to the second Bark domain band is determined as the high-frequency loudness. For example, the low-frequency loudness can be determined using formula (5). High-frequency loudness is determined by formula (6). .

[0070] (5) (6) Finally, high-frequency loudness and low-frequency loudness were determined as the acoustic feature information of the voice control signal.

[0071] In this embodiment, determining the sound source location of the voice control signal based at least on acoustic feature information may include: when the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the ratio of high-frequency loudness to low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or, when the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the ratio of high-frequency loudness to low-frequency loudness is greater than or equal to a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle.

[0072] For example, the ratio of high to low frequency loudness As a feature, the location of the sound source is determined. When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if... If the high-frequency feature loss is significant, the sound source location of the voice control signal is determined to be outside the vehicle; otherwise, the sound source location is determined to be inside the vehicle. When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if... If the high-frequency feature loss is small, the sound source location of the voice control signal is determined to be outside the vehicle; otherwise, the sound source location is determined to be inside the vehicle. Characterizes the preset loudness threshold.

[0073] In another embodiment, the acoustic feature information includes spectral flatness and speech loudness, and the speech loudness includes high-frequency loudness and low-frequency loudness. Accordingly, determining the sound source location of the voice control signal based at least on the acoustic feature information may include: when the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold and the ratio of high-frequency loudness to low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or, when the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold and the ratio of high-frequency loudness to low-frequency loudness is greater than or equal to a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle.

[0074] In this embodiment, the voice control signal can be analyzed from multiple dimensions to determine the sound source location of the voice control signal, which can accurately determine the sound source location and thus improve the accuracy of suppressing external voice signals.

[0075] The above describes a specific implementation scheme for determining the sound source location of a voice control signal based solely on acoustic feature information. However, in this disclosure, the sound source location of the voice control signal can also be determined by combining location information and acoustic feature information.

[0076] Therefore, in one embodiment, the method may further include: The area inside and outside the vehicle is divided into multiple points, and for each point, the spatial spectrum of the point is determined based on the guide vector of the point relative to the voice acquisition device and the voice control signal acquired by the voice acquisition device. The point with the largest spatial spectrum is determined as the target point.

[0077] Figure 3 This is a schematic diagram illustrating the interior and exterior spaces of a vehicle according to an exemplary embodiment. For example... Figure 3 As shown, the area within box 1 is the vehicle interior area, and the area outside box 1 but within box 2 is the vehicle exterior area. The vehicle interior and exterior areas are divided into G points (G grids) with a certain precision. Then, the sound source is located using the MUSIC (Multiple Signal Classification) algorithm. The specific calculation steps are as follows: Step (e): Based on the coordinates of G points inside and outside the vehicle And the locations of the M voice acquisition devices Calculate the steering vector of the j-th point relative to each voice acquisition device. Each element in the guide vector can be determined using formula (7): (7) in, The phase difference between the j-th point and the k-th voice acquisition device is represented. f Characterizing frequency points, Where is the sampling rate, and c is the speed of sound, typically taken as 340 m / s.

[0078] Step (f): Determine the cross-correlation matrix of the voice control signal using the following formula (8), where, Characterization of conjugate devices: (8) Step (g): Perform eigenvalue decomposition on the cross-correlation matrix to obtain the eigenvalue matrix. and the matrix composed of eigenvectors The eigenvectors correspond to the directions in the signal space and noise space, while the eigenvalues ​​reflect the energy magnitude in each direction. Therefore, based on the magnitude of the eigenvalues, the eigenvectors are divided into signal subspaces. and noise subspace The signal subspace is composed of the values ​​corresponding to the largest eigenvalues. The eigenvectors constitute the noise subspace, while the eigenvectors of the remaining eigenspaces constitute the noise subspace. The cross-correlation matrix is ​​composed of the following: (9) Step (h): Based on the noise subspace And the guide vector of each point relative to each voice acquisition device The spatial spectrum of each point is determined by the following formula (10): (10) in, The spatial spectrum representing the j-th point.

[0079] At this point, the spatial spectrum of each point can be obtained, and the point with the largest spatial spectrum can be determined as the target point.

[0080] In this embodiment, determining the sound source location of the voice control signal based at least on acoustic feature information may include: determining the sound source location of the voice control signal based on the target location and acoustic feature information.

[0081] For example, when the target location is outside the vehicle area, and the spectral flatness is less than a preset flatness threshold, and the ratio of high-frequency loudness to low-frequency loudness is less than a preset loudness threshold, the sound source location of the voice control signal is determined to be outside the vehicle.

[0082] Figure 4 This is a schematic diagram illustrating a voice control method according to an exemplary embodiment. For example... Figure 4 As shown, assuming the voice acquisition device is located inside the vehicle, and the acoustic feature information includes spectral flatness and voice loudness, and the voice loudness includes high-frequency loudness and low-frequency loudness, and if the target point is determined to be an external point, then the flag is set to 1, otherwise it is set to 0.

[0083] First, the spectral flatness SF and the ratio of high-frequency loudness to low-frequency loudness RL of the target frequency band are calculated. Next, it is determined whether SF < ths and RL < thl are satisfied. If both are satisfied, it is checked whether flag is 1. If flag = 1, the sound source location is determined to be outside the vehicle, and interaction is denied, so that the voice control system is not controlled by the voice control signal. If SF < ths and RL < thl are not satisfied simultaneously, or if flag is not equal to 1, interaction is allowed, that is, the voice control system responds to the voice control signal.

[0084] By adopting the above technical solution, the voice control signal can be analyzed from multiple dimensions to determine the sound source location of the voice control signal. This can accurately determine the sound source location and thus improve the accuracy of suppressing external voice signals.

[0085] Based on the same inventive concept, this disclosure also provides a voice control device. Figure 5 This is a block diagram illustrating a voice control device according to an exemplary embodiment. Figure 5 As shown, the voice control device 500 may include: The first determining module 501 is configured to determine the acoustic feature information of the voice control signal in response to the detection of the voice control signal, the voice control signal being used to control the voice control system in the vehicle. The second determining module 502 is configured to determine the sound source location of the voice control signal based at least on the acoustic feature information. The first control module 503 is configured to control the voice control system to not be controlled by the voice control signal if it is determined that the sound source location is outside the vehicle.

[0086] Optionally, the acoustic feature information includes spectral flatness and / or speech loudness; the first determining module 501 may include: The first determining submodule is configured to determine whether the vehicle doors and windows are closed in response to the detection of a voice control signal. The second determining submodule is configured to determine the acoustic feature information of the voice control signal when the vehicle door and the vehicle window are in the closed state.

[0087] Optionally, the second determining submodule may include: The conversion submodule is configured to convert the voice control signal into a frequency domain voice control signal when the door and the window are closed. The third determining submodule is configured to determine the acoustic feature information of the voice control signal based on the voice control signal in the frequency domain.

[0088] Optionally, the third determining submodule is configured as follows: Based on the voice control signal in the frequency domain, determine the spectral flatness of the target frequency band in the voice control signal, wherein the target frequency band includes at least a high frequency band; The spectral flatness of the target frequency band is determined as the acoustic feature information of the voice control signal; The second determining module 502 is configured to: when the voice acquisition device for detecting the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold, determine that the sound source location of the voice control signal is outside the vehicle; or, when the voice acquisition device for detecting the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold, determine that the sound source location of the voice control signal is outside the vehicle.

[0089] Optionally, the third determining submodule is configured as follows: The voice control signal in the frequency domain is mapped to multiple Bark domain bands in the Bark domain, and the loudness value corresponding to each Bark domain band is determined. The Bark domain bands with center frequencies less than or equal to the center frequency threshold are defined as the first Bark domain bands, and the Bark domain bands with center frequencies greater than the center frequency threshold are defined as the second Bark domain bands. The sum of loudness values ​​corresponding to the first Bark domain band is determined as the low-frequency loudness, and the sum of loudness values ​​corresponding to the second Bark domain band is determined as the high-frequency loudness. The low-frequency loudness and the high-frequency loudness are determined as the acoustic feature information of the voice control signal; The second determining module 502 is configured as follows: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the ratio of the high-frequency loudness to the low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the ratio of the high-frequency loudness to the low-frequency loudness is greater than or equal to a preset loudness threshold, the sound source location of the voice control signal is determined to be outside the vehicle.

[0090] Optionally, when the acoustic feature information includes spectral flatness and speech loudness, and the speech loudness includes high-frequency loudness and low-frequency loudness, the second determining module 502 is configured to: when the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold and the ratio of the high-frequency loudness to the low-frequency loudness is less than a preset loudness threshold, then determine that the sound source location of the voice control signal is outside the vehicle; or, when the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold and the ratio of the high-frequency loudness to the low-frequency loudness is greater than or equal to a preset loudness threshold, then determine that the sound source location of the voice control signal is outside the vehicle.

[0091] Optionally, the voice control device 500 may further include: The third determining module is configured to divide the vehicle interior and exterior areas into multiple points, and for each point, determine the spatial spectrum of the point based on the guiding vector of the point relative to the voice acquisition device and the voice control signal acquired by the voice acquisition device. The fourth determination module is configured to determine the point with the largest spatial spectrum as the target point; The second determining module 502 is configured to: determine the sound source location of the voice control signal based on the target location and the acoustic feature information.

[0092] Optionally, the voice control device 500 may further include: The second control module is configured to control the voice control system to respond to the voice control signal if the location of the sound source is determined to be inside the vehicle.

[0093] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0094] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the voice control method provided in this disclosure.

[0095] Figure 6 This is a block diagram illustrating a vehicle according to an exemplary embodiment. For example, vehicle 600 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other type of vehicle. Vehicle 600 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0096] Reference Figure 6The vehicle 600 may include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. The vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of the vehicle 600 can be interconnected via wired or wireless means.

[0097] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, and a navigation system, etc.

[0098] The perception system 620 may include several sensors for sensing information about the environment surrounding the vehicle 600. For example, the perception system 620 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0099] The decision control system 630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0100] The drive system 640 may include components that provide powered motion to the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0101] Some or all of the functions of vehicle 600 are controlled by computing platform 650. Computing platform 650 may include at least one processor 651 and memory 652, processor 651 can execute instructions 653 stored in memory 652.

[0102] Processor 651 can be any conventional processor, such as a commercially available CPU. Processors may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems-on-chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0103] The memory 652 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0104] In addition to instruction 653, memory 652 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 652 can be used by computing platform 650.

[0105] In this embodiment of the disclosure, the processor 651 may execute instructions 653 to complete all or part of the steps of the above-described voice control method.

[0106] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described voice control method when executed by the programmable device.

[0107] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0108] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0109] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0110] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A voice control method, characterized in that, The method includes: In response to the detection of a voice control signal, acoustic feature information of the voice control signal is determined, the voice control signal being used to control the voice control system inside the vehicle; Based at least on the acoustic feature information, determine the sound source location of the voice control signal; If the location of the sound source is determined to be outside the vehicle, then the voice control system is controlled not to be controlled by the voice control signal.

2. The method according to claim 1, characterized in that, The acoustic feature information includes spectral flatness and / or speech loudness; the step of determining the acoustic feature information of the speech control signal in response to the detection of the speech control signal includes: In response to the detection of a voice control signal, it determines whether the vehicle's doors and windows are closed; When the car door and the car window are in the closed state, the acoustic feature information of the voice control signal is determined.

3. The method according to claim 2, characterized in that, When the vehicle door and the vehicle window are in the closed state, determining the acoustic feature information of the voice control signal includes: When the car door and the car window are closed, the voice control signal is converted into a voice control signal in the frequency domain; Based on the voice control signal in the frequency domain, determine the acoustic feature information of the voice control signal.

4. The method according to claim 3, characterized in that, Determining the acoustic feature information of the voice control signal based on the voice control signal in the frequency domain includes: Based on the voice control signal in the frequency domain, determine the spectral flatness of the target frequency band in the voice control signal, wherein the target frequency band includes at least a high frequency band; The spectral flatness of the target frequency band is determined as the acoustic feature information of the voice control signal; Determining the sound source location of the voice control signal based at least on the acoustic feature information includes: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold, the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold, the sound source location of the voice control signal is determined to be outside the vehicle.

5. The method according to claim 3, characterized in that, Determining the acoustic feature information of the voice control signal based on the voice control signal in the frequency domain includes: The voice control signal in the frequency domain is mapped to multiple Bark domain bands in the Bark domain, and the loudness value corresponding to each Bark domain band is determined. The Bark domain band with a center frequency less than or equal to the center frequency threshold is defined as the first Bark domain band, and the Bark domain band with a center frequency greater than the center frequency threshold is defined as the second Bark domain band. The sum of loudness values ​​corresponding to the first Bark domain band is determined as the low-frequency loudness, and the sum of loudness values ​​corresponding to the second Bark domain band is determined as the high-frequency loudness. The low-frequency loudness and the high-frequency loudness are determined as the acoustic feature information of the voice control signal; Determining the sound source location of the voice control signal based at least on the acoustic feature information includes: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the ratio of the high-frequency loudness to the low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the ratio of the high-frequency loudness to the low-frequency loudness is greater than or equal to a preset loudness threshold, the sound source location of the voice control signal is determined to be outside the vehicle.

6. The method according to claim 2, characterized in that, When the acoustic feature information includes spectral flatness and speech loudness, and the speech loudness includes high-frequency loudness and low-frequency loudness, determining the sound source location of the speech control signal based at least on the acoustic feature information includes: When the voice acquisition device used to detect the voice control signal is located inside the vehicle, if the spectral flatness is less than a preset flatness threshold and the ratio of the high-frequency loudness to the low-frequency loudness is less than a preset loudness threshold, then the sound source location of the voice control signal is determined to be outside the vehicle; or When the voice acquisition device used to detect the voice control signal is located outside the vehicle, if the spectral flatness is greater than or equal to a preset flatness threshold, and the ratio of the high-frequency loudness to the low-frequency loudness is greater than or equal to a preset loudness threshold, then the sound source location of the voice control signal is determined to be inside the vehicle.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: The area inside and outside the vehicle is divided into multiple points, and for each point, the spatial spectrum of the point is determined based on the guiding vector of the point relative to the voice acquisition device and the voice control signal acquired by the voice acquisition device. The point with the largest spatial spectrum is determined as the target point; Determining the sound source location of the voice control signal based at least on the acoustic feature information includes: Based on the target location and the acoustic feature information, the sound source location of the voice control signal is determined.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: If the location of the sound source is determined to be inside the vehicle, then the voice control system is controlled to respond to the voice control signal.

9. A voice control device, characterized in that, include: The first determining module is configured to determine the acoustic feature information of the voice control signal in response to the detection of the voice control signal, the voice control signal being used to control the voice control system in the vehicle. The second determining module is configured to determine the sound source location of the voice control signal based at least on the acoustic feature information. The first control module is configured to, if it is determined that the location of the sound source is outside the vehicle, control the voice control system to not be controlled by the voice control signal.

10. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to enable the vehicle to implement the voice control method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the voice control method as described in any one of claims 1-8.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the voice control method according to any one of claims 1-8.

Images