Masking sound generation method and device, equipment and medium

Abstract

The invention discloses a masking sound generation method and device, equipment and a medium. The method comprises the steps that a to-be-masked sound signal and an initial masking sound signal of a vehicle are acquired; determining voice intelligibility corresponding to the initial masking sound signal based on the to-be-masked sound signal and the initial masking sound signal; determining target voice energy of the initial masking sound signal; determining a target gain corresponding to an initial masking sound signal based on voice intelligibility and the target voice energy; and adjusting the initial masking sound signal based on the target gain to obtain a target masking sound signal. According to the scheme, the playing gain can be calculated based on the voice intelligibility and the voice energy of the initial masking sound signal used for masking the sound signal to be masked, and gain adjustment is carried out on the initial masking sound signal based on the playing gain, so that under the condition that the playing volume of the loudspeaker is not changed, the masking sound signal after gain adjustment is played through the loudspeaker, and the sound signal to be masked can be masked. According to the invention, a good masking effect can be achieved on the back-row speech, and the driving experience is not affected.

Description

Technical Field

[0001] This disclosure relates to speech processing technology, and more particularly to a method, apparatus, device, and medium for generating masking sounds. Background Technology

[0002] Currently, ride-sharing is becoming increasingly common in daily life, making it especially important to protect the privacy of users in the same vehicle environment.

[0003] In related technologies, when users use ride-sharing services, to prevent conversations among rear passengers from being heard by the driver in the front, thus compromising passenger privacy, masking sounds such as music can be played to prevent the driver from hearing the rear passengers' voices. However, in actual use, the front speakers of the vehicle often need to be at a relatively high volume to ensure an ideal masking effect for different rear passengers, which causes discomfort for the driver and affects the driving experience.

[0004] Therefore, how to ensure that different passengers in the back row can achieve a relatively ideal concealment effect has become an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this disclosure provides a masking sound generation method, apparatus, device, and medium that enables front-row passengers to achieve a good masking effect on the sound of rear-row passengers while maintaining a constant speaker volume.

[0006] One aspect provides a method for generating masking sounds, including:

[0007] Acquire the vehicle's target sound signal and initial masking sound signal;

[0008] Based on the sound signal to be masked and the initial masking sound signal, determine the speech intelligibility corresponding to the initial masking sound signal;

[0009] Determine the target speech energy of the initial masking acoustic signal;

[0010] Based on the speech intelligibility and the target speech energy, determine the target gain corresponding to the initial masking sound signal;

[0011] Based on the target gain, the initial masking sound signal is adjusted to obtain the target masking sound signal.

[0012] In another aspect, a masking sound generating device is provided, comprising:

[0013] The first acquisition module is used to acquire the sound signal to be masked and the initial masking sound signal of the vehicle;

[0014] The first determining module is used to determine the speech intelligibility corresponding to the initial masking sound signal based on the sound signal to be masked and the initial masking sound signal;

[0015] The second determining module is used to determine the target speech energy of the initial masking sound signal;

[0016] The third determining module is used to determine the target gain corresponding to the initial masking sound signal based on the speech intelligibility and the target speech energy.

[0017] The sound processing module is used to adjust the initial masking sound signal based on the target gain to obtain the target masking sound signal.

[0018] In another aspect, the embodiments provide a computer program product that, when the instruction processor in the computer program product is executed, performs the masking sound generation method proposed in the first aspect of the present disclosure.

[0019] In another aspect, an electronic device is proposed, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the masking sound generation method described in the first aspect above.

[0020] The masking sound generation method provided in this disclosure can acquire the vehicle's sound signal to be masked and the initial masking sound signal, and determine the speech intelligibility corresponding to the initial masking sound signal based on the sound signal to be masked and the initial masking sound signal. Since the target gain of the initial masking sound signal can be determined based on the target speech energy and the corresponding speech intelligibility of the initial masking sound signal, the initial masking sound signal can be adjusted based on the target gain to obtain the target masking sound signal. The solution provided in this disclosure, after acquiring the sound signal to be masked and the initial masking sound signal, can calculate the speech intelligibility of the initial masking sound signal relative to the sound signal to be masked before playing the initial masking sound signal through the front speakers of the vehicle. Based on the speech intelligibility and speech energy, the playback gain allocated to the initial masking sound signal can be calculated. Therefore, the gain of the initial masking sound signal can be adjusted based on the playback gain to obtain the target masking sound signal. Thus, while keeping the speaker playback volume unchanged, playing the target masking sound signal through the speakers can ensure that the front passengers receive a low speech intelligibility of the sound transmitted from the rear passengers to the front, that is, the front passengers cannot understand the content of the rear passengers' speech. In this way, a good masking effect on the speech of the rear passengers is achieved without affecting the driving experience. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a sound masking system provided in an exemplary embodiment of this disclosure.

[0022] Figure 2 This is a schematic flowchart of a masking sound generation method provided in an exemplary embodiment of this disclosure.

[0023] Figure 3 This is a flowchart illustrating a masking sound generation method provided in another exemplary embodiment of this disclosure.

[0024] Figure 4 This is a schematic flowchart of a masking sound generation method provided in another exemplary embodiment of this disclosure.

[0025] Figure 5 This is a flowchart illustrating a masking sound generation method provided in yet another exemplary embodiment of this disclosure.

[0026] Figure 6 This is a flowchart illustrating a masking sound generation method provided in yet another exemplary embodiment of this disclosure.

[0027] Figure 7 This is a flowchart illustrating a masking sound generation method provided in yet another exemplary embodiment of this disclosure.

[0028] Figure 8 This is a flowchart illustrating a masking sound generation method provided in yet another exemplary embodiment of this disclosure.

[0029] Figure 9 This is a schematic diagram of the structure of a masking sound generation device provided in another exemplary embodiment of this disclosure.

[0030] Figure 10 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0031] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0032] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0033] Application Overview

[0034] In related technologies, vehicles play masking sounds, such as music, to reduce the driver's ability to hear conversations among rear passengers. However, to ensure a satisfactory masking effect for all rear passengers, the front speakers need to be at a relatively high volume, which can cause significant discomfort to the driver and negatively impact the driving experience.

[0035] The masking sound generation method provided in this disclosure can acquire the vehicle's sound signal to be masked and the initial masking sound signal, and determine the speech intelligibility corresponding to the initial masking sound signal based on the sound signal to be masked and the initial masking sound signal. Since the target gain of the initial masking sound signal can be determined based on the target speech energy and the corresponding speech intelligibility of the initial masking sound signal, the initial masking sound signal can be adjusted based on the target gain to obtain the target masking sound signal. The solution provided in this disclosure, after acquiring the sound signal to be masked and the initial masking sound signal, can calculate the speech intelligibility of the initial masking sound signal relative to the sound signal to be masked before playing the initial masking sound signal through the front speakers of the vehicle. Based on the speech intelligibility and speech energy, the playback gain allocated to the initial masking sound signal can be calculated. Therefore, the gain of the initial masking sound signal can be adjusted based on the playback gain to obtain the target masking sound signal. Thus, while keeping the speaker playback volume unchanged, playing the target masking sound signal through the speakers can ensure that the front passengers receive a low speech intelligibility of the sound transmitted from the rear passengers to the front, that is, the front passengers cannot understand the content of the rear passengers' speech. In this way, a good masking effect on the speech of the rear passengers is achieved without affecting the driving experience.

[0036] Exemplary System

[0037] Figure 1 This is a schematic diagram of a sound masking system provided in an exemplary embodiment of this disclosure. The sound masking system can execute the masking sound generation method provided in the embodiments of this disclosure to achieve sound masking functionality.

[0038] For example, such as Figure 1 As shown, the aforementioned sound masking system may include speakers A1 and A2 located at the driver's headrest, front microphone B1 and rear microphone B2, and front sound source C1 and rear sound source C2. Furthermore, the sound masking system also includes: an acoustic echo cancellation (AEC) module, a source separation (SS) and noise suppression (NS) module, a speech intelligence index (SII) module, a signal mixing module (MIX), and a speech enhancement (SE) module.

[0039] In some examples, the process of implementing sound masking using the masking sound generation method provided in this disclosure is as follows:

[0040] (1) Voice signal y1 is collected through the front microphone B1 and voice signal y2 is collected through the rear microphone B2.

[0041] The aforementioned voice signal y1 includes: passenger voice signal x1 transmitted to the front microphone B1, noise signal, and excitation signal output through speakers A1 and A2 and transmitted to the front microphone B1; the aforementioned voice signal y2 includes: passenger voice signal x2 transmitted to the rear microphone B2, noise signal, and excitation signal output through speakers A1 and A2 and transmitted to the rear microphone B2.

[0042] (2) The AEC module is used to use the Acoustic Echo Cancellation (AEC) algorithm to eliminate the echo signals included in the speech signals y1 and y2 respectively. It can obtain the passenger speech signal x1 transmitted to the front microphone B1 and the passenger speech signal x2 transmitted to the rear microphone B2, as well as the transfer function h1 from speakers A1 and A2 to the front microphone B1. The transfer function h1 can be regarded as the result of merging the transfer functions from speakers A1 and A2 to the front microphone B1 respectively.

[0043] (3) SS&NS module, used to process x1 and x2 using sound source separation algorithm and noise suppression algorithm to obtain the speech signal v2 emitted by sound source C2, and the speech signal d collected by front microphone B1 after the speech signal v2 emitted by sound source C2 is transmitted to front microphone B1. That is, the speech signal d collected by front microphone B1 after the speech signal v2 is transmitted to front microphone B1 is the sound signal d to be masked.

[0044] (4) The MIX module is used to reverse the speech signal v2 emitted by sound source C2 and then mix it with other audio signals (such as music, running water sound, etc.) to obtain the initial masking sound signal s. mix .

[0045] (5) SII module, used to process the transfer function h1 and the initial masking acoustic signal s mix By performing convolution operations, the signals played by speakers A1 and A2 collected by the front microphone B1 can be obtained, which can be called the masked sound source signal e. Based on the sound signal d to be masked and the masked sound source signal e, the SII of each frame of speech signal in the masked sound source signal e can be calculated in accordance with the standard ANSI S3.5-1997.

[0046] (6) SE module, used to determine the initial masking acoustic signal s mixThe speech energy of each frame of speech signal is calculated as the speech energy of multiple sub-band signals and the sum of the speech energy of multiple sub-band signals. Based on the speech energy of any one sub-band signal, the sum of the speech energy of multiple sub-band signals, and the SII of the speech signal frame, the sub-band gain of each sub-band signal is determined. Then, based on the sub-band gain of each sub-band signal, the gain of each sub-band signal can be adjusted to obtain multiple target sub-band signals with adjusted gain. Finally, these multiple target sub-band signals are combined to obtain the target masking sound signal s. max .

[0047] In some embodiments, after obtaining the target masking sound signal through the above embodiments, the target masking sound signal can be played through speakers A1 and A2 to achieve masking of the sound signal to be masked.

[0048] It should be noted that the above sound masking system can repeatedly execute the above steps to mask the sound in the vehicle in real time.

[0049] The technical solution provided in this disclosure, after acquiring the sound signal to be masked and the initial masking sound signal, can calculate the speech intelligibility of the initial masking sound signal relative to the sound signal to be masked before playing the initial masking sound signal through the front speakers of the vehicle. Based on the speech intelligibility and speech energy, the playback gain allocated to the initial masking sound signal can be calculated. Therefore, the gain of the initial masking sound signal can be adjusted based on the playback gain to obtain the target masking sound signal. Thus, while keeping the speaker playback volume unchanged, playing the target masking sound signal through the speakers can ensure that the front passengers receive a low speech intelligibility of the sound transmitted from the rear passengers to the front, that is, the front passengers cannot understand the content of the rear passengers' speech. In this way, a good masking effect on the speech of the rear passengers is achieved without affecting the driving experience.

[0050] Moreover, since the gain of different sub-bands can be adjusted, the target frame masking sound signal obtained by synthesizing the target sub-band signal after gain adjustment can mask the sound of different frequency bands frame by frame, thus realizing fine two-dimensional time-frequency control and greatly improving the masking effect.

[0051] Exemplary methods

[0052] Figure 2 This is a schematic flowchart of a masking sound generation method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 2 As shown, the above method may include the following steps:

[0053] Step 201: Obtain the vehicle's sound signal to be masked and the initial masking sound signal.

[0054] Because conversations between rear-seat passengers can be transmitted to the front seats, potentially compromising their privacy, it is necessary to mask their conversations. The aforementioned sound signals to be masked are the audio signals transmitted from the rear-seat passengers to the front seats; these are the signals that the vehicle's sound masking system needs to block. The initial masking signal is used to mask the sound signals to be masked.

[0055] In some examples, the presence of an audio signal within the vehicle can trigger the vehicle's audio masking function. After activation, the audio signal within the vehicle can be acquired in real time, and then processed using audio signal processing technology to obtain the audio signal to be masked. The initial masking signal is then determined based on this masking signal. Detailed descriptions are provided in the following embodiments, which will not be repeated here.

[0056] For example, the initial masking sound signal can be the signal obtained by reversing the voices of the rear passengers in the vehicle, or it can be the signal obtained by reversing the voices of the rear passengers in the vehicle and then mixing in signals such as music and flowing water.

[0057] Step 202: Based on the sound signal to be masked and the initial masking sound signal, determine the speech intelligibility corresponding to the initial masking sound signal.

[0058] In some embodiments, speech intelligibility is a standardized metric used to quantify how well a speech signal can be understood in a noisy environment; it is also known as the Speech Intelligence Index (SII). A higher speech intelligibility indicates that the speech signal is easier to understand. The speech intelligibility corresponding to the initial masking sound signal mentioned above refers to the intelligibility of the sound signal collected when the masked sound signal is played through a microphone in the front of the vehicle and played through a speaker, with the masked sound signal treated as noise.

[0059] In some examples, according to acoustic speech intelligibility quantification evaluation standards, such as ANSIS 3.5-1997, a series of processes, including framing, sub-band decomposition, and speech energy calculation, can be performed on the masked sound signal and the initial masked sound signal to calculate the speech intelligibility of each frame of the masked sound source signal corresponding to the initial masked sound signal. For details, please refer to the detailed description in the following embodiments; the embodiments disclosed herein will not be repeated here.

[0060] Step 203: Determine the target speech energy of the initial masking sound signal.

[0061] In some embodiments, the target speech energy can be used to indicate the energy distribution characteristics of a speech signal in any frequency band. The target speech energy includes the speech energy of each sub-band signal included in each frame of the initial masking sound signal, and the sum of the speech energies of each sub-band signal. When calculating the target speech energy of the initial masking sound signal in the time domain, the initial masking sound signal can be framed and sub-band decomposed to obtain multiple sub-band signals, and then the speech energy of each sub-band signal and the sum of the speech energies of each sub-band signal can be calculated. For details, please refer to the detailed description in the following embodiments; the embodiments disclosed herein will not be repeated here.

[0062] In other embodiments, the target speech energy of the initial masking sound signal can also be calculated in the frequency domain. Therefore, after the initial masking sound signal is processed by framing, each frame of speech signal can be converted from the time domain to the frequency domain and divided into sub-bands to obtain multiple sub-band signals. Then, the target speech energy is calculated based on the amplitude spectrum of each sub-band signal. For details, please refer to the detailed description in the related technology. This disclosure will not repeat the details in the embodiments.

[0063] Step 204: Determine the target gain corresponding to the initial masking sound signal based on speech intelligibility and target speech energy.

[0064] In some embodiments, the target gain is an adjustment parameter for the signal value of a sub-band signal (i.e., the instantaneous value of the signal at any specific moment) in audio signal processing, used to control the intensity of the speech signal in a specific frequency band, for example, to amplify or attenuate the intensity of the speech signal in a certain frequency band.

[0065] In some embodiments, since speech intelligibility is obtained based on the speech energy of each sub-band signal and its corresponding intelligibility contribution weight, and the target speech energy can be used to indicate the energy intensity of a speech signal in any frequency band, and the energy intensity is related to the square of the signal value of the speech signal in that frequency band, the adjustment parameter of the signal value of the sub-band signal of the initial speech signal in a certain frequency band, i.e., the target gain, can be determined based on speech intelligibility and target speech energy, so as to adjust the signal value of the sub-band signal through the target gain.

[0066] In some examples, the target speech energy mentioned above includes a first speech energy and a second speech energy; wherein, the first speech energy is the energy of any one of the multiple sub-band signals included in each frame of the initial masking sound signal, and the second speech energy is the sum of the first speech energies of the multiple sub-band signals.

[0067] Step 205: Adjust the initial masking acoustic signal based on the target gain to obtain the target masking acoustic signal.

[0068] In some embodiments, since the target gain is a parameter for adjusting the signal values ​​of sub-band signals in audio signal processing, it is actually performed by dividing the initial masking sound signal into frames and decomposing each frame of audio signal into multiple sub-band signals. Then, the target gain is used to adjust the signal values ​​of the corresponding sub-band signals. Afterwards, the adjusted sub-band signals are synthesized in the time domain to obtain a single-frame masking sound signal, which is any frame of the target masking sound signal. For the time-domain synthesis of the adjusted sub-band signals, please refer to the relevant technical descriptions; the embodiments disclosed herein will not be elaborated upon here.

[0069] The technical solution provided in this disclosure, after acquiring the sound signal to be masked and the initial masking sound signal, can calculate the speech intelligibility of the initial masking sound signal relative to the sound signal to be masked before playing the initial masking sound signal through the front speakers of the vehicle. Based on the speech intelligibility and speech energy, the playback gain allocated to the initial masking sound signal can be calculated. Therefore, the gain of the initial masking sound signal can be adjusted based on the playback gain to obtain the target masking sound signal. Thus, while keeping the speaker playback volume unchanged, playing the target masking sound signal through the speakers can ensure that the front passengers receive a low speech intelligibility of the sound transmitted from the rear passengers to the front, that is, the front passengers cannot understand the content of the rear passengers' speech. In this way, a good masking effect on the speech of the rear passengers is achieved without affecting the driving experience.

[0070] In some embodiments, such as Figure 3 As shown, step S201 above can specifically include the following steps:

[0071] Step S2011: Determine the fourth voice signal emitted by the rear passenger in the vehicle.

[0072] The interior of a vehicle is a complex acoustic environment with various sound sources of interference (such as the voices of the driver, front passenger, and rear passenger) and environmental noise. If it is necessary to obtain the voice signal of a specific sound source, the voice signal must first be acquired and then processed in a series of signal processing steps.

[0073] In some examples, the speech signal in the vehicle can be collected first through the microphone in the vehicle, and then some signal processing such as sound source separation and noise reduction can be performed on the speech signal to obtain the fourth language signal emitted by the rear passenger.

[0074] In some embodiments, step S2011 may specifically include: acquiring a fifth voice signal collected by multiple audio sensors of the vehicle; wherein the multiple audio sensors include a front audio sensor and a rear audio sensor; and performing target processing on the fifth voice signal to obtain a fourth voice signal.

[0075] In some examples, the aforementioned multiple audio sensors can be an array of in-vehicle microphones deployed in different locations within the vehicle. For example, a microphone array can be set in the front row of the vehicle and a microphone array can be set in the rear row of the vehicle. That is, the multiple audio sensors can include front microphones and rear microphones.

[0076] In some examples, a fifth speech signal can be obtained by real-time acquisition of speech signals (including sounds and noise from various sources) inside the vehicle through multiple in-vehicle microphone arrays; and the fifth speech signal can be subjected to target processing to obtain a fourth speech signal; wherein, the target processing may include echo cancellation (AEC), sound source separation (SS), and noise suppression (NS).

[0077] In some examples, the fifth voice signal may include: the voice signal of the occupants inside the vehicle, noise signals in the environment, and an excitation signal played through a speaker; wherein the excitation signal may be music or a target masking sound signal used in the previous loop to mask the voice of the rear passengers, and the excitation signal is known.

[0078] For example, assuming two microphones are installed in the front and rear rows of a vehicle, the fifth voice signal y[k] collected by the two microphones can be represented as follows:

[0079] y[k]=[y1[k]y2[k]] T

[0080] in,[·] T Let y1[k] represent the transpose, k represent the time index, y1[k] represent the speech signal captured by the first microphone, and y2[k] represent the speech signal captured by the second microphone.

[0081] y i [k] = h i [j]*s mask [k]+x i [j]

[0082] Where * denotes convolution, y i [k] represents the sound signal collected by the i-th microphone, h i [k] is the transfer function from the speaker to the i-th microphone, s mask [k] is the excitation signal played by the speaker, x i [k] represents the voice signal of the driver and passengers inside the vehicle collected by the i-th microphone plus the noise signal, where i = 1, 2.

[0083] In the acoustic environment inside a vehicle, when an excitation signal is played through a speaker, it is reflected by space and collected by a microphone to form an echo signal, denoted as m. i [k], m i[k] is the result of convolving the excitation signal played by the speaker with the transfer function from the speaker to the microphone, m i [k] can be represented as follows:

[0084] m i [k] = h i [k]*s mask [k]

[0085] Where i = 1, 2, and * denote convolution.

[0086] Eliminating the echo signal in the fifth speech signal using an echo cancellation algorithm can specifically include: estimating the transfer function using an adaptive filtering algorithm. The estimated echo signal m is obtained by performing a convolution operation on the estimated transfer function and the excitation signal. i [k], and eliminate the echo signal m from the fifth speech signal collected by the microphone. i [k] gives the in-vehicle voice signal plus noise signal x collected by the i-th microphone. i [k]; where, This represents the transfer function from the speaker to the first microphone (i.e., the front microphone). This represents the transfer function from the loudspeaker to the second microphone (i.e., the rear microphone).

[0087] The voice signal plus noise signal x[k] of the occupants inside the vehicle, collected by the microphone above, includes two voice signals, which can be represented as follows:

[0088] x[k]=[x1[k]x2[k]]T

[0089] Where x1[k] represents the voice signal and noise signal of the driver and passengers in the vehicle collected by the first microphone (i.e., i=1), and x2[k] represents the voice signal and noise signal of the driver and passengers in the vehicle collected by the second microphone (i.e., i=2).

[0090] Assuming the voice signals of the occupants in the vehicle include voice signals from the front and rear sound sources, then x1[k] and x2[k] above represent the following respectively:

[0091] x1[k]=g 11 [k]*v1[j]+g 12 [k]*v2[k]+n1[k]

[0092] x2[k]=g 21 [k]*v1[k]+g 22 [k]*v²[k]+n²[k]

[0093] Among them, v i[k] represents the speech signal emitted by the i-th sound source, g ij [k] represents the transfer function from the j-th sound source to the i-th microphone, n i [k] represents the noise signal collected by the i-th microphone, where i,j = 1, 2. When i = 1, it represents the first sound source, which is the front sound source; when i = 2, it represents the second sound source, which is the back sound source.

[0094] By processing x[k] using a sound source separation algorithm, the speech signal of the rear passenger mixed with noise can be separated. Then, by using a noise suppression algorithm to denoise the speech signal of the rear passenger mixed with noise, the pure speech signal v2[k] of the rear passenger, i.e., the fourth speech signal, and the speech signal d[k] = g of the rear passenger's voice transmitted to the front microphone can be obtained. 12 [k]*v2[k], where d[k] is the signal that needs to be masked in the sound masking system provided by the vehicle. The calculation of d[k] can be found in the detailed description in the following embodiments.

[0095] Step S2012: The fourth speech signal is reversed to obtain the fifth speech signal.

[0096] In some examples, the aforementioned reversal processing of the fourth speech signal refers to reversing the order of the time-domain sampling points of the fourth speech signal to obtain a reversed signal. That is, the last sampling point of the fourth speech signal is taken as the first of the reversed signal, and so on, to obtain the fifth speech signal. For example, the fourth speech signal can be sampled first to obtain multiple sampling arrays, and then the order of these multiple sampling arrays can be reversed to obtain the fifth speech signal. Another example is that the fourth speech signal can be first segmented into frames, and then the sampling points of each frame can be reversed individually to obtain the fifth speech signal; alternatively, the order of all frames can be reversed to obtain the fifth speech signal. The specific method can be determined according to the actual situation, and this disclosure does not limit this aspect.

[0097] Step S2013: Based on the fifth speech signal and the preset audio signal, the initial masking sound signal is obtained.

[0098] In some examples, the preset audio signal can be music or the sound of flowing water, etc. For example, when the preset audio signal is music, a piano or violin piece can be selected.

[0099] In some embodiments, a preset audio signal can be mixed with the fifth speech signal to obtain an initial masking sound signal. During the mixing process of the fifth speech signal and the preset audio signal, it is necessary to ensure that the fifth speech signal and the preset audio signal are time-synchronized. When the spectrum of the preset audio signal does not match the spectrum of the fifth speech signal, the preset audio signal is spectrally shaped to obtain an adjusted preset audio signal. The adjusted preset audio signal matches the spectrum of the fifth speech signal and is time-synchronized. After performing Fourier transforms on each frame of the fifth speech signal and the adjusted preset audio signal respectively, each frame signal is converted from the time domain to the frequency domain. The corresponding frequency domain signals are added together to obtain a mixed spectrum. Then, an inverse Fourier transform is performed on the mixed spectrum to obtain a single-frame time domain signal. These single-frame time domain signals are then merged to obtain a mixed signal, which can be used as the initial masking sound signal.

[0100] In other embodiments, after merging the time-domain signals of each single frame to obtain a mixed signal, the mixed signal can be further processed by smoothing filtering to obtain the initial masking sound signal.

[0101] The above embodiment is illustrated by taking the fifth voice signal mixed with a preset audio signal to obtain the initial masking sound signal as an example. Of course, the fifth voice signal (i.e., the reversed signal of the voice signal issued by the rear passenger) can also be used directly as the initial masking sound signal. The specific method can be determined according to the actual situation, and this embodiment does not limit it.

[0102] The technical solution provided in this embodiment can determine the fourth voice signal emitted by the rear passenger in the vehicle, and reverse the fourth voice signal to obtain the fifth voice signal. Based on the fifth voice signal and the preset audio signal, an initial masking sound signal is obtained. Since the voice of the rear passenger is reversed to mask the voice of the rear passenger, the initial masking sound signal has the same frequency and rhythm as the voice to be masked, and has low speech intelligibility. Therefore, the masking effect of the initial masking sound signal can be improved.

[0103] In some embodiments, such as Figure 4 As shown, step S201 above can specifically include the following steps:

[0104] Step S2014: Determine the fourth voice signal emitted by the rear passenger in the vehicle.

[0105] The determination of the fourth voice signal can be referred to the detailed description in the above embodiments, and will not be repeated here in the embodiments of this disclosure.

[0106] Step S2015: Determine the second transfer function for the fourth voice signal to be transmitted to the front audio sensor.

[0107] In some examples, the aforementioned front audio sensor is a sensor used to collect voice signals inside the vehicle; for example, the front audio sensor is a front microphone.

[0108] In some embodiments, the second transfer function described above can be used to describe the frequency domain characteristics of the signal received at the front microphone after the speech signal propagates from the sound source (e.g., the rear seat of the vehicle) through the in-vehicle acoustic environment. Therefore, the transfer function can reflect the transmission path of the sound source signal. Different transfer functions correspond to different transmission paths of the sound source signal; that is, the second transfer function reflects the transmission path from the rear seat of the vehicle to the front microphone.

[0109] Because speech signals experience amplitude attenuation and phase delay during transmission, the transfer function is related to the frequency and transmission distance of the speech signal. After establishing an acoustic propagation model related to frequency and transmission distance in advance, the frequency of the fourth speech signal and the transmission distance between the rear and front microphones of the vehicle can be obtained. Substituting the obtained frequency and transmission distance into the acoustic propagation model yields the second transfer function. For detailed descriptions of the acoustic propagation model in related technologies, please refer to the embodiments disclosed herein.

[0110] Step S2016: Determine the sound signal to be masked based on the fourth speech signal and the second transfer function.

[0111] In some examples, after obtaining the second transfer function, the fourth speech signal can be convolved with the second transfer function to obtain the sound signal to be masked.

[0112] For example, let the fourth speech signal be denoted as v2[k], and the second transfer function be denoted as g. 12 [k], the sound signal to be masked is denoted as d[k]; d[k] can be calculated using the following formula:

[0113] d[k]=g 12 [k]*v2[k]

[0114] Where k represents the time index.

[0115] The technical solution provided in this disclosure can determine the fourth voice signal emitted by the rear passenger in the vehicle and the second transfer function of the fourth voice signal transmitted to the front audio sensor. Therefore, it can obtain an accurate sound signal to be masked based on the voice signal and the second transfer function. Subsequently, based on the accurate sound signal to be masked, an initial masking sound signal that can be used to mask the sound signal to be masked can be accurately determined. This solves the problem of sound signal distortion caused by the complex acoustic environment inside the vehicle.

[0116] In some embodiments, such as Figure 5As shown, step 202 above may include the following steps:

[0117] Step 2021: Determine the masking sound source signal based on the initial masking sound signal.

[0118] In some embodiments, the masking sound source signal refers to the sound signal collected by the front microphone when the initial masking sound signal is played through the vehicle's speakers. In other words, the masking sound source signal is the signal obtained after the initial masking sound signal is transmitted from the speakers to the front microphones.

[0119] Because the amplitude of a sound signal (i.e., a speech signal) attenuates during transmission, the energy of the sound signal changes (i.e., the signal strength attenuates). Since the speech intelligibility of a sound signal is related to the speech energy, in order to accurately calculate the speech intelligibility of a sound signal, it is necessary to consider the attenuation of the initial masking sound signal along the transmission path from the speaker to the front microphone. Therefore, when the initial masking sound signal is played by the speaker and transmitted from the speaker to the front microphone, the sound signal after the initial masking sound signal has been transmitted through the above transmission path is collected by the front microphone, which is the masking sound source signal.

[0120] In some embodiments, step 2021 may specifically include: determining a first transfer function between the vehicle's speaker and the front audio sensor; and determining a masking sound source signal based on the initial masking sound signal and the first transfer function.

[0121] In some examples, the first transfer function described above is used to describe the frequency domain characteristics of the signal received at the front audio sensor (e.g., the front microphone) after the sound source signal (i.e., the initial masking sound signal) output by the speaker has propagated through the in-vehicle acoustic environment.

[0122] In some embodiments, when determining the first transfer function, the transfer function from the speaker to the microphone can be estimated by means of actual speech signal acquisition and processing and an adaptive filtering algorithm, as described in the above embodiments. Alternatively, the first transfer function can be calculated using an acoustic propagation model, as described in the above embodiments, by means of calculating the second transfer function. The specific method can be determined according to the actual usage, and the embodiments disclosed herein do not limit this.

[0123] In some examples, after determining the first transfer function corresponding to the transmission path formed between the speaker and the front audio sensor (e.g., the front microphone), the masking sound source signal can be obtained by calculating the convolution of the initial masking sound signal and the first transfer function.

[0124] For example, let the initial masking acoustic signal be denoted as s mix [k], the first transfer function is denoted as h1[k], and the masking signal of the sound source is denoted as e[k]; e[k] can be calculated using the following formula:

[0125] e[k]=h1[k]*s mix [k]

[0126] Where * denotes convolution.

[0127] Step 2022: Perform frame segmentation on the masked sound source signal to obtain multiple frames of the first speech signal.

[0128] Both the masked sound source signal and the sound signal to be masked are continuous long sequences. In order to achieve a better masking effect in subsequent signal processing, the solution of this embodiment is to mask each frame of the sound signal to be masked based on the masked sound source signal. Therefore, the masked sound source signal and the sound signal to be masked can be processed in frames separately.

[0129] In some examples, when processing the masked sound source signal in frames, the frame length can be preset to M, and then the masked sound source signal can be divided according to the frame length M to obtain multiple frames of first speech signal, each frame of first speech signal having a frame length of M; where M can be a positive integer; the empirical value of M is usually between 10ms and 40ms, for example, M is 32ms.

[0130] The above method uses a fixed frame length for frame division processing. Of course, a non-fixed frame length method can also be used for frame division processing. The specific method can be determined according to the actual use situation. This disclosure does not limit this method.

[0131] Step 2023: Perform frame segmentation on the sound signal to be masked to obtain multiple frames of second speech signal.

[0132] The explanation of framing the masked sound signal can be found in the above embodiment of framing the masked sound source signal. This disclosure will not repeat the details, but it should be noted that the frame length of the first speech signal and the corresponding second speech signal in each frame must be the same.

[0133] Step 2024: Process the multiple frames of first speech signals and the corresponding multiple frames of second speech signals according to a preset method to obtain the speech intelligibility of each frame of speech signal in the masked sound source signal.

[0134] In some examples, the speech intelligibility of the masked sound source signal can be calculated frame by frame based on the signal-to-noise ratio according to the ANSI S3.5-1997 standard; where the masked sound source signal is the useful speech signal and the sound signal to be masked is equivalent to the noise signal.

[0135] Specifically, according to the ANSI S3.5-1997 standard, for any frame of first speech signal and the corresponding frame of second speech signal, sub-band decomposition is performed on both, resulting in multiple sub-band signals corresponding to each frame. Based on these sub-band signals, a sub-band audibility function (also called a signal-to-noise ratio mapping function) and a sub-band importance function are determined for each sub-band signal. The speech intelligibility of the frame of first speech signal is then obtained by summing these functions. The sub-band audibility function indicates the contribution of a sub-band signal to the overall intelligibility, while the sub-band importance function indicates the importance weight of each sub-band signal. The calculation of the sub-band audibility function and sub-band importance function can be found in ANSI S3.5-1997 and other relevant technical documents; details are omitted here.

[0136] In some examples, the aforementioned ANSI S3.5-1997 standard may include a mapping table representing the subband importance function, which indicates the relationship between different frequencies and the importance weights of each subband signal. Therefore, after obtaining multiple subband signals, the importance weight corresponding to each frequency can be obtained by querying the mapping table based on the frequency corresponding to each subband signal.

[0137] In some examples, after obtaining multiple sub-band signals corresponding to the first speech signal of any frame and the second speech signal of the corresponding frame, the signal-to-noise ratio (SNR) of each sub-band signal is calculated based on the multiple sub-band signals corresponding to the first speech signal of the frame and the second speech signal of the corresponding frame. Then, the SNR of each sub-band signal is converted into a decibel value through logarithmic transformation, and the converted decibel value is clipped between -15 and 15 dB and normalized to obtain the sub-band audibility function of each sub-band signal, which ranges between 0 and 1.

[0138] For example, assuming any frame of the masked sound source signal is decomposed into N sub-band signals, the speech intelligibility SII of each frame of the speech signal in the masked sound source signal can be calculated based on the following formula:

[0139]

[0140] Where n is the sub-band index, N is the total number of sub-bands, and A n It is the subband audibility function, i n It is the subband importance function, and i n The following constraints must be met:

[0141]

[0142] The technical solution provided in this disclosure can determine the masked sound source signal based on the initial masking sound signal, and perform frame-by-frame processing on both the masked sound source signal and the sound signal to be masked to obtain multiple frames of first speech signals corresponding to the masked sound source signal and multiple frames of second speech signals corresponding to the sound signal to be masked. Then, based on a preset method, the multiple frames of first speech signals and the corresponding multiple frames of second speech signals are processed to obtain the speech intelligibility of each frame of the masked sound source signal. Because this solution can process the masked sound source signal and the sound signal to be masked frame by frame and calculate the speech intelligibility frame by frame, subsequent frame-by-frame dynamic sound masking can be performed based on the frame-by-frame speech intelligibility to improve the accuracy of sound masking.

[0143] In some embodiments, when the target speech energy includes a first speech energy and a second speech energy, such as Figure 6 As shown, step 203 above may include the following steps:

[0144] Step 2031: Perform frame segmentation on the initial masking sound signal to obtain multiple frames of third speech signal.

[0145] In some examples, when processing the initial masking sound signal in frames, the frame length can be preset to M, and then the initial masking sound signal can be divided according to the frame length M to obtain multiple frames of third speech signal, each frame of third speech signal having a frame length of M; where M can be a positive integer.

[0146] Step 2032: Subband decomposition is performed on the target frame speech signal to obtain multiple subband signals.

[0147] The target frame speech signal is any one of the multiple frames of third speech signals.

[0148] In some examples, the target frame speech signal is sub-banded according to different frequency ranges using filters of different frequency bands, resulting in multiple sub-band signals, each corresponding to a specific frequency range. For example, the multiple sub-band signals may include a first sub-band signal, a second sub-band signal, and a third sub-band signal, where the first sub-band signal corresponds to the low-frequency range, the second sub-band signal corresponds to the mid-frequency range, and the third sub-band signal corresponds to the high-frequency range.

[0149] Step 2033: Determine the first speech energy of the first sub-band signal.

[0150] The first sub-band signal is any one of the multiple sub-band signals.

[0151] In some embodiments, the first speech energy of the first sub-band signal is the sum of the energy densities of all sampling points within the first sub-band signal, which is also the sum of the squares of the signal values ​​of all sampling points within the first sub-band signal. Therefore, when determining the first speech energy of the first sub-band signal, it is necessary to calculate the squares of the signal values ​​of all sampling points within the first sub-band signal and then sum the squares of the signal values ​​of all sampling points. Furthermore, normalization processing can be performed, that is, the sum of the squares of the signal values ​​of all sampling points is divided by the frame length, which can ensure that the energy of each sub-band signal is independent of the frame length.

[0152] For example, after processing the initial masking sound signal into frames based on frame length M to obtain multiple frames of third speech signals, and then performing sub-band decomposition on the target frame speech signal in the multiple frames of third speech signals to obtain N sub-band signals, the first speech energy of the nth sub-band signal of the l-th frame can be calculated based on the following formula, which can be denoted as σ. s,n [l]:

[0153]

[0154] Where l represents the frame index, n represents the sub-band index, E represents the number of all sampling points in the nth sub-band signal of the l-th frame, and k represents the sampling point index, (s n [l,k]) 2 It represents the square of the signal value at the k-th sampling point within the n-th sub-band signal of the l-th frame.

[0155] Step 2034: Sum the first speech energy of each sub-band signal to obtain the second speech energy of the target frame speech signal.

[0156] In some examples, assuming that after subband decomposition of the target frame speech signal, N subband signals are obtained, the second speech energy of the target frame speech signal can be calculated based on the following formula:

[0157]

[0158] Where, σ s,n [l] represents the first speech energy of the nth sub-band signal in the l-th frame, N represents the number of sub-band signals corresponding to the speech signal in the target frame, and n represents the sub-band index.

[0159] The technical solution provided in this disclosure can perform frame-by-frame processing on the initial masking sound signal to obtain multiple frames of third speech signals. Then, it performs sub-band decomposition on the target frame speech signal within these multiple frames of third speech signals to obtain multiple sub-band signals. This allows for the determination of the first speech energy of the first sub-band signal among the multiple sub-band signals, and the summation of the first speech energy of each sub-band signal to obtain the second speech energy of the target frame speech signal. Because this disclosure can divide each frame of the initial masking sound signal into multiple sub-band signals through sub-band decomposition and calculate the energy of different sub-band signals and the sum of the sub-band energies of each frame of speech signal, it can subsequently mask each sub-band signal (i.e., sound signals in different frequency bands) frame by frame based on the energy of different sub-band signals and the sum of the sub-band energies of each frame of speech signal. This significantly improves the granularity of sound masking, thus achieving fine-grained control and dynamic adjustment of the masking strategy.

[0160] In some embodiments, such as Figure 7 As shown, step 204 above may include the following steps:

[0161] Step 2041: Determine the first correlation parameter based on the first speech energy and the corresponding speech intelligibility of the first sub-band signal.

[0162] In some examples, for any frame of speech signal in the initial masking sound signal, after determining the speech intelligibility of any frame of speech signal and the first speech energy of the first sub-band signal among the multiple sub-band signals corresponding to that frame of speech signal, the speech intelligibility of that frame of speech signal is used as the index of the first speech energy to obtain the first correlation parameter.

[0163] Step 2042: Determine the second correlation parameter based on the first speech energy and corresponding speech intelligibility of multiple sub-band signals.

[0164] In some examples, each frame of speech signal in the initial masking sound signal can include multiple corresponding sub-band signals. After calculating the first speech energy of each sub-band signal, multiple first speech energies can be obtained. The speech intelligibility of each frame of speech signal is used as the index of each sub-band signal, that is, the first correlation parameter corresponding to each sub-band signal is obtained. The summation of multiple first correlation parameters can obtain the second correlation parameter.

[0165] Step 2043: Determine the target gain of the first sub-band signal based on the first correlation parameter, the second correlation parameter, the first speech energy, and the second speech energy.

[0166] In some examples, the target gain of the first subband signal can be calculated according to a preset formula based on the first correlation parameter, the second correlation parameter, the first speech energy, and the second speech energy.

[0167] For example, suppose the initial masking acoustic signal (denoted as s) mix After [k]) Frame processing, the frame index is l. Any frame of speech signal can be decomposed into N sub-band signals after sub-band decomposition. The target gain of the nth sub-band signal of the lth frame can be calculated by the following formula, where n = 1, ... N.

[0168]

[0169] Among them, the above (σ) s,n [l]) SII[l] This represents the first correlation parameter. σ represents the first correlation parameter. s,n [l] represents the first speech energy, σ s [l] indicates the second speech energy.

[0170] Based on the above formula, it can be seen that when SII[l] = 0, At this point, the speech energy of the nth subband signal in the l-th frame can be expressed as: That is, the speech energy of each sub-band signal is equal. At this time, the masking effect of the initial masking sound signal is the worst. Therefore, it is necessary to adjust the gain of each sub-band signal after the initial masking sound signal is framed; when SII[l]=1, W n [l] = 1. At this point, the masking effect of the initial masking sound signal has reached its best effect. Therefore, no processing is required for each sub-band signal after the initial masking sound signal is framed, that is, no gain adjustment is required for each sub-band signal.

[0171] Thus, the smaller SII[l] is, the worse the masking effect of the initial masking sound signal is; the larger SII[l] is, the better the masking effect of the initial masking sound signal is. When SII[l] is close to 1, it means that the initial masking sound signal can achieve good masking of the sound signal to be masked without gain adjustment. At this time, the gain adjustment of the initial masking sound signal is not required.

[0172] It should be noted that the above embodiments did not consider the influence of the voice signal of the front passenger and the noise inside the vehicle when calculating the subband gain of the initial masking sound signal. However, if the initial masking sound signal is sufficient to achieve good masking of the sound signal to be masked by adjusting the gain, the addition of the voice signal of the front passenger and the noise inside the vehicle will make the masking effect of the sound signal to be masked even better.

[0173] The technical solution provided in this disclosure can determine a first correlation parameter based on the first speech energy and corresponding speech intelligibility of a first sub-band signal, and determine a second correlation parameter based on multiple first speech energies and corresponding speech intelligibility of multiple sub-band signals. Then, based on the first correlation parameter, the second correlation parameter, the first speech energy, and the second speech energy, the target gain of the first sub-band signal can be determined. Because this solution can calculate the sub-band gain of each sub-band signal frame-by-frame based on the energy and speech intelligibility of each frame, the sub-band gain of each sub-band signal can be adjusted frame-by-frame to achieve fine-grained control.

[0174] In some embodiments, the target gain may include multiple sub-band gains, and each frame of speech signal in the initial masking sound signal may include corresponding multiple sub-band signals, with each sub-band gain corresponding to a different sub-band signal; such as Figure 8 As shown, step S205 above can specifically include the following steps:

[0175] Step S2051: Adjust the gain of the corresponding sub-band signals based on the multiple sub-band gains to obtain multiple target sub-band signals.

[0176] In some embodiments, since the gain is used to adjust the signal value of the subband signal, a subband gain is applied to each subband signal to obtain a target subband signal with adjusted gain, denoted as . Target subband signal It can be expressed as follows:

[0177]

[0178] Among them, W n [l] represents the subband gain of the nth subband signal in the lth frame. s represents the signal after gain adjustment for the nth sub-band signal in the l-th frame. n [k] represents the k-th sampling point within the n-th sub-band signal.

[0179] After adjusting the gain of each sub-band signal based on multiple sub-gains to obtain multiple target sub-band signals, the sum of the speech energy of each target sub-band signal can be calculated, as shown in the following formula:

[0180]

[0181] The above formula shows that the total energy of each sub-band signal remains unchanged before and after gain adjustment. Therefore, adding a sub-band gain W to each sub-band signal included in each frame of the initial masking sound signal is sufficient. n [l] will not affect the total energy of the output signal.

[0182] Step S2052: Combine multiple target sub-band signals to obtain the target frame masking sound signal.

[0183] Among them, the aforementioned target masking acoustic signal includes the target frame masking acoustic signal.

[0184] In some embodiments, since the above-mentioned multiple target sub-band signals are signals corresponding to different frequency bands in the same frame of speech signal, the target sub-band signals can be aligned according to the timestamp in the time domain and synthesized into a target frame masking sound signal by overlapping and adding them together.

[0185] Furthermore, since the initial masking sound signal includes multiple frames of speech signals, after adjusting the gain of multiple sub-band signals corresponding to each frame of speech signal according to the above steps S2051 and S2052, multiple target frame masking sound signals can be obtained; after merging the multiple target frame masking sound signals in the time domain according to the time sequence, a continuous speech signal can be obtained, that is, the continuous speech signal is the target masking sound signal.

[0186] In some embodiments, after obtaining the target masking sound signal, the target masking sound signal is played through a speaker without changing the speaker volume to mask the sound signal to be masked. At this time, the speech intelligibility of the target masking sound signal obtained based on the target masking sound signal and the sound signal to be masked has changed.

[0187] The technical solution provided in this disclosure allows for gain adjustment of multiple sub-band signals based on the gains of each sub-band, resulting in multiple target sub-band signals. These target sub-band signals are then synthesized to obtain a target frame masking sound signal. The target frame masking signal is a single frame of audio signal with gain adjustment used to mask the sound signal to be masked. Because this solution allows for gain adjustment of different sub-band signals, the target frame masking sound signal obtained by synthesizing the gain-adjusted target sub-band signals can mask sounds in different frequency bands frame by frame. This achieves fine-grained time-frequency two-dimensional control, thus greatly improving the masking effect.

[0188] Exemplary device

[0189] Figure 9 This is a schematic diagram of a masking sound generation device provided as an exemplary embodiment of the present disclosure. The device can be installed in electronic devices such as terminal devices and servers, or on objects such as vehicles, to execute the masking sound generation method of any of the embodiments described above.

[0190] like Figure 9 As shown, the above-mentioned device 300 includes:

[0191] The first acquisition module 301 can be used to acquire the vehicle's sound signal to be masked and the initial masking sound signal;

[0192] The first determining module 302 can be used to determine the speech intelligibility corresponding to the initial masking sound signal based on the sound signal to be masked and the initial masking sound signal;

[0193] The second determining module 303 can be used to determine the target speech energy of the initial masking sound signal;

[0194] The third determining module 304 can be used to determine the target gain corresponding to the initial masking sound signal based on the speech intelligibility and the target speech energy.

[0195] The sound processing module 305 can be used to adjust the initial masking sound signal based on the target gain to obtain the target masking sound signal.

[0196] In one possible implementation, the first determining module 302 may be specifically used to: determine the masking sound source signal based on the initial masking sound signal;

[0197] The masked sound source signal is processed by framing to obtain multiple frames of first speech signal;

[0198] The sound signal to be masked is processed into frames to obtain multiple frames of second speech signal;

[0199] The speech intelligibility of each frame of the masked sound source signal is obtained by processing the multi-frame first speech signal and the corresponding multi-frame second speech signal according to a preset method.

[0200] In one possible implementation, the second determining module 303 may be specifically used to: perform frame-segmentation processing on the initial masking sound signal to obtain multiple frames of third speech signal;

[0201] The target frame speech signal is sub-band decomposed to obtain multiple sub-band signals; the target frame speech signal is any one of the multiple frames of third speech signals.

[0202] Determine the first speech energy of the first sub-band signal; the first sub-band signal is any one of the plurality of sub-band signals.

[0203] The first speech energy of the target frame speech signal is obtained by summing the first speech energy of each sub-band signal;

[0204] The target speech energy includes the first speech energy and the second speech energy.

[0205] In one possible implementation, the third determining module 304 may be specifically used to: determine a first correlation parameter based on the first speech energy and the corresponding speech intelligibility of the first sub-band signal;

[0206] A second correlation parameter is determined based on the multiple first speech energies of the multiple sub-band signals and the corresponding speech intelligibility;

[0207] The target gain of the first subband signal is determined based on the first correlation parameter, the second correlation parameter, the first speech energy, and the second speech energy.

[0208] In one possible implementation, the first determining module 302 may be specifically used to: determine a first transfer function between the vehicle's speaker and the front audio sensor;

[0209] The masking sound source signal is determined based on the initial masking sound signal and the first transfer function.

[0210] In one possible implementation, the first acquisition module 301 may be specifically used to: determine the fourth voice signal emitted by the rear passenger in the vehicle;

[0211] The fourth speech signal is reversed to obtain the fifth speech signal;

[0212] The initial masking sound signal is obtained based on the fifth speech signal and the preset audio signal.

[0213] In one possible implementation, the first acquisition module 301 may be specifically used to: determine a fourth voice signal emitted by a rear passenger in the vehicle; and determine a second transfer function for the fourth voice signal to be transmitted to the front audio sensor.

[0214] The sound signal to be masked is determined based on the fourth speech signal and the second transfer function.

[0215] In one possible implementation, the first acquisition module 301 may be specifically used to: acquire multiple fifth voice signals collected by multiple audio sensors of the vehicle; wherein, the multiple audio sensors include the front audio sensor and the rear audio sensor;

[0216] The fifth speech signal is subjected to target processing to obtain the fourth speech signal.

[0217] In one possible implementation, the target gain includes multiple sub-band gains, each frame of the initial masking sound signal includes multiple corresponding sub-band signals, and each sub-band gain corresponds to a different sub-band signal; the sound processing module 305 can be specifically used to: adjust the gain of the corresponding multiple sub-band signals based on the multiple sub-band gains to obtain multiple target sub-band signals;

[0218] The multiple target sub-band signals are synthesized to obtain the target frame masking sound signal; the target masking sound signal includes the target frame masking sound signal.

[0219] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0220] Exemplary electronic devices

[0221] Figure 10 A structural diagram of an electronic device provided in an embodiment of this disclosure includes at least one processor 111 and a memory 112.

[0222] The processor 111 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.

[0223] The memory 112 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 111 may execute one or more computer program instructions to implement the masking sound generation methods and / or other desired functions of the various embodiments of this disclosure described above.

[0224] In one example, the electronic device 11 may also include an input device 113 and an output device 114, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0225] The input device 113 may include various sensors, including but not limited to: a distance sensor for detecting the distance between a target object and the vehicle; an image sensor for acquiring information about the vehicle's surrounding environment. In some examples, the input device may also include a pressure sensor for detecting seat pressure to determine the presence and location of passengers; a temperature sensor for monitoring the temperature inside the cabin; a humidity sensor for monitoring the humidity inside the cabin to assist in regulating the in-vehicle environment; an air quality sensor for monitoring in-vehicle air quality, such as carbon dioxide and volatile organic compounds (VOCs); a light sensor for detecting the intensity of light inside and outside the vehicle; an acceleration sensor for detecting changes in the vehicle's acceleration; a distance sensor for detecting the distance between the vehicle and other objects; a touchscreen sensor for interaction with the vehicle's infotainment system; biometric sensors, such as fingerprint recognition and facial recognition; a heart rate monitor for monitoring the driver's heart rate; a sound sensor for voice recognition and interaction to enable voice control; a seat sensor for monitoring seat usage, such as whether the seat is occupied and the passenger's body size; and wireless communication sensors, such as Bluetooth and Wi-Fi, for connecting to smart devices to achieve data transmission and remote control. In addition to the examples given above, the input device may include more or fewer sensors, which will not be elaborated here.

[0226] The output device 114 can output various information or signals to other hardware or devices, which may include displays, car audio systems, seats, windows, steering wheels, communication networks, and their connected remote output devices. The displays may include multiple different displays such as a driver's side display, a passenger side display, and a rear-seat display. The car audio system may include multiple speakers located in different positions within the vehicle cabin, and each display or speaker can operate independently.

[0227] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device 11 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 11 may include any other suitable components depending on the specific application.

[0228] Exemplary computer program products and computer-readable storage media

[0229] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the masking sound generation methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0230] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0231] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the masking sound generation methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0232] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0233] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0234] Various modifications and variations can be made to this disclosure without departing from its spirit and scope. Therefore, this disclosure is also intended to include such modifications and variations if they fall within the scope of the claims of this disclosure and their equivalents.

Claims

1. A method for generating masking sound, comprising: Acquire the vehicle's target sound signal and initial masking sound signal; Based on the sound signal to be masked and the initial masking sound signal, determine the speech intelligibility corresponding to the initial masking sound signal; Determine the target speech energy of the initial masking acoustic signal; Based on the speech intelligibility and the target speech energy, determine the target gain corresponding to the initial masking sound signal; Based on the target gain, the initial masking sound signal is adjusted to obtain the target masking sound signal.

2. The method according to claim 1, wherein determining the speech intelligibility corresponding to the initial masking sound signal based on the masked sound signal and the initial masking sound signal comprises: The masking sound source signal is determined based on the initial masking sound signal; The masked sound source signal is processed by framing to obtain multiple frames of first speech signal; The sound signal to be masked is processed into frames to obtain multiple frames of second speech signal; The speech intelligibility of each frame of the masked sound source signal is obtained by processing the multi-frame first speech signal and the corresponding multi-frame second speech signal according to a preset method.

3. The method according to claim 1, wherein, Determining the target speech energy corresponding to the initial masking sound signal includes: The initial masking sound signal is processed into frames to obtain multiple frames of third speech signal; The target frame speech signal is sub-band decomposed to obtain multiple sub-band signals; the target frame speech signal is any one of the multiple frames of third speech signals. Determine the first speech energy of the first sub-band signal; the first sub-band signal is any one of the plurality of sub-band signals. The first speech energy of the target frame speech signal is obtained by summing the first speech energy of each sub-band signal; The target speech energy includes the first speech energy and the second speech energy.

4. The method according to claim 3, wherein, Determining the target gain corresponding to the initial masking sound signal based on the speech intelligibility and the target speech energy includes: Based on the first speech energy and the corresponding speech intelligibility of the first sub-band signal, a first correlation parameter is determined; A second correlation parameter is determined based on the multiple first speech energies of the multiple sub-band signals and the corresponding speech intelligibility; The target gain of the first subband signal is determined based on the first correlation parameter, the second correlation parameter, the first speech energy, and the second speech energy.

5. The method according to claim 2, wherein, The step of determining the masking sound source signal based on the initial masking sound signal includes: Determine the first transfer function between the vehicle's speakers and the front audio sensors; The masking sound source signal is determined based on the initial masking sound signal and the first transfer function.

6. The method according to claim 1, wherein, The acquisition of the vehicle's target sound signal and initial masking sound signal includes: Identify the fourth voice signal emitted by a rear passenger in the vehicle; The fourth speech signal is reversed to obtain the fifth speech signal; The initial masking sound signal is obtained based on the fifth speech signal and the preset audio signal.

7. The method according to claim 1, wherein, The acquisition of the vehicle's sound signal to be masked includes: Identify the fourth voice signal emitted by a rear passenger in the vehicle; Determine the second transfer function for the fourth voice signal to be transmitted to the front audio sensor; The sound signal to be masked is determined based on the fourth speech signal and the second transfer function.

8. The method according to claim 6 or 7, wherein, The determination of the fourth voice signal emitted by the rear passenger in the vehicle includes: The fifth voice signal is acquired from multiple audio sensors of the vehicle; wherein the multiple audio sensors include the front audio sensor and the rear audio sensor. The fifth speech signal is subjected to target processing to obtain the fourth speech signal.

9. The method according to claim 1, wherein, The target gain includes multiple sub-band gains, and each frame of speech signal in the initial masking sound signal includes multiple corresponding sub-band signals. Each sub-band gain in the multiple sub-band gains corresponds to a different sub-band signal. The step of adjusting the initial masking acoustic signal based on the target gain to obtain the target masking acoustic signal includes: Based on the multiple sub-band gains, the corresponding multiple sub-band signals are adjusted to obtain multiple target sub-band signals; The multiple target sub-band signals are synthesized to obtain the target frame masking sound signal; the target masking sound signal includes the target frame masking sound signal.

10. A masking sound generating device, comprising: The first acquisition module is used to acquire the vehicle's sound signal to be masked and the initial masking sound signal; The first determining module is used to determine the speech intelligibility corresponding to the initial masking sound signal based on the sound signal to be masked and the initial masking sound signal; The second determining module is used to determine the target speech energy of the initial masking sound signal; The third determining module is used to determine the target gain corresponding to the initial masking sound signal based on the speech intelligibility and the target speech energy. The sound processing module is used to adjust the initial masking sound signal based on the target gain to obtain the target masking sound signal.

11. A computer-readable storage medium storing a computer program for performing the masking sound generation method according to any one of claims 1-9.

12. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the masking sound generation method according to any one of claims 1-9.

Images