Microphone control method and device, vehicle and storage medium

Abstract

The embodiment of the invention discloses a microphone control method and device, a vehicle and a storage medium, and relates to the technical field of vehicle control. The method comprises the steps that audio signals collected by a plurality of microphones can be acquired, one microphone corresponds to one audio signal, and the audio signals comprise sound intensity and sound duration; positioning the audio signal, and determining a user sound source position corresponding to the audio signal; determining a control strategy according to the user sound source position and the sound intensity and sound duration of each audio signal; and controlling the plurality of microphones based on the control strategy. According to the technical scheme, the automatic control function of the microphone is achieved, a driver does not need to manually adjust the state of the microphone, the situations that attention is distracted and the sight line leaves the front road due to manual operation of the driver are avoided, the risk of traffic accidents caused by distraction of the driver is reduced, and the driving safety is improved. Therefore, the driving safety of the vehicle is improved, and more comfortable and convenient communication experience is provided for a user.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of vehicle control technology, and in particular, to a microphone control method, device, vehicle and storage medium. Background Art

[0002] At present, when controlling the on-board microphone, the driver manually touches the mute button on the on-board display screen to control the on-board microphone. However, the driver's manual control of the on-board microphone will cause the driver's line of sight to leave the road ahead, greatly increasing the risk of traffic accidents and thus reducing the driving safety of the vehicle. Summary of the invention

[0003] The embodiments of the present application provide a microphone control method, device, vehicle and storage medium, which realize the automatic control function of the microphone without manual control of the microphone switch, so as to solve the problem in the prior art that the driving safety of the vehicle is reduced due to the driver manually controlling the microphone switch.

[0004] In a first aspect, an embodiment of the present application provides a microphone control method, which is applied to a vehicle, wherein the vehicle includes a plurality of microphones, and the method includes:

[0005] Acquire audio signals collected by multiple microphones, wherein one microphone corresponds to one audio signal, and the audio signal includes sound intensity and sound duration;

[0006] Perform positioning processing on the audio signal to determine the user sound source position corresponding to the audio signal;

[0007] Determine the control strategy based on the user's sound source location, the sound intensity and sound duration of each audio signal;

[0008] Control multiple microphones based on a control strategy.

[0009] In an embodiment of the present application, the vehicle can obtain audio signals collected by multiple microphones, and perform positioning processing on the audio signals to determine the user sound source position corresponding to the audio signal, and then determine the control strategy according to the user sound source position, the sound intensity and sound duration of each audio signal, such as controlling multiple microphones to be in an on state or controlling multiple microphones to be in an off state, and then controlling multiple microphones based on the control strategy to adjust the current working state of the microphone, thereby realizing the automatic control function of the microphone, without the driver having to manually adjust the microphone state, avoiding the situation where the driver is distracted and the eyes leave the road ahead due to manual operation, thereby reducing the risk of traffic accidents caused by driver distraction, thereby improving the driving safety of the vehicle; by determining the control strategy through the user sound source position, the sound intensity and sound duration of each audio signal, and automatically controlling multiple microphones based on the control strategy, the microphone state can be intelligently adjusted according to the actual sound environment in the vehicle, providing users with a more comfortable and convenient communication experience, and can reduce unnecessary power consumption, thereby improving the user experience.

[0010] In a second aspect, an embodiment of the present application provides a microphone control device, which is applied to a vehicle, wherein the vehicle includes a plurality of microphones, and the device includes:

[0011] A signal acquisition module is used to acquire audio signals collected by multiple microphones, wherein one microphone corresponds to one audio signal, and the audio signal includes sound intensity and sound duration;

[0012] A positioning module is used to perform positioning processing on the audio signal and determine the user sound source position corresponding to the audio signal;

[0013] A strategy determination module, used to determine a control strategy based on the user's sound source location, the sound intensity and sound duration of each audio signal;

[0014] The control module is used to control the multiple microphones based on the control strategy.

[0015] In a third aspect, an embodiment of the present application provides a vehicle, the vehicle comprising:

[0016] at least one processor; and a memory communicatively coupled to the at least one processor;

[0017] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the microphone control method of any embodiment of the present application.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for controlling a microphone as in any embodiment of the present application.

[0019] The description of the second, third and fourth aspects in this application can refer to the detailed description of the first aspect; and the beneficial effects described in the second, third and fourth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0020] In this application, the name of the microphone control device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalent technologies.

[0021] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flowchart of a microphone control method provided in an embodiment of the present application;

[0024] Figure 2 is another flowchart of the microphone control method provided in an embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of a microphone control device provided in an embodiment of the present application;

[0026] Figure 4 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0028] It should be noted that the terms "first", "second", "target", and "original" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including", "having", and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0029] Figure 1 1 is a flow chart of a method for controlling a microphone provided in an embodiment of the present application. This embodiment can be applied to a scenario where the working state of a microphone installed inside a vehicle needs to be automatically controlled. A method for controlling a microphone provided in this embodiment can be executed by a control device for a microphone provided in an embodiment of the present application, and the device can be implemented in software and / or hardware. In a specific embodiment, the control device for the microphone can be integrated in a vehicle.

[0030] In a specific embodiment, the vehicle may include a vehicle controller and a microphone; a plurality of microphones may be installed inside the vehicle to capture audio signals from different directions inside the vehicle.

[0031] The execution subject of this method may be a vehicle controller of the vehicle, see Figure 1 The microphone control method of this embodiment includes but is not limited to the following steps:

[0032] S110: Acquire audio signals collected by multiple microphones.

[0033] Among them, the audio signal is the representation of sound on the time axis. It usually converts sound vibration into electrical signals or digital signals through audio collection devices such as microphones. It has the characteristics of continuity, time-varying and non-stationary. One microphone corresponds to one audio signal. The audio signal can include sound intensity and sound duration. Sound intensity is a physical quantity that measures the energy of sound, usually expressed in sound pressure level (SPL), in decibels (dB). Sound duration is the length of time that the sound lasts, usually measured in seconds (s).

[0034] Specifically, when it is necessary to automatically control the working status of the microphone installed inside the vehicle, audio signals collected by multiple microphones can be obtained. For example, in one implementation, when the user inside the vehicle makes a sound, the microphone can collect the surrounding sound and convert it into an electrical signal, i.e., an audio signal, and then send the collected audio signal to the vehicle controller. At this time, one microphone corresponds to one audio signal.

[0035] Optionally, in another implementation, initial audio signals collected by multiple microphones may be acquired, and the initial audio signals may be preprocessed to obtain audio signals.

[0036] The initial audio signal is a sound signal captured in real time by a microphone; the preprocessing includes at least one of denoising, filtering, and adjusting the dynamic range.

[0037] Specifically, the audio signals collected by the multiple microphones can be obtained respectively, and these audio signals can be determined as the initial audio signals. In this case, one microphone corresponds to one initial audio signal, and then each of the multiple initial audio signals is preprocessed to obtain the audio signal. That is, for each of the multiple initial audio signals, the current initial audio signal can be denoised, for example, by analyzing the spectral characteristics of the noise, removing the noise spectrum from the current initial audio signal to achieve noise reduction; or, a high-pass filter can be used to remove low-frequency noise below a first preset cutoff frequency from the current initial audio signal, a low-pass filter can be used to remove high-frequency noise above a second preset cutoff frequency from the current initial audio signal, and a band-pass filter can be used to retain the audio signal in the current initial audio signal that is within a preset frequency range (for example, between 300 Hz and 3400 Hz); or, the dynamic range of the current initial audio signal can be adjusted, for example, the difference between the larger amplitude and the smaller amplitude in the current initial audio signal can be reduced to make the overall volume more balanced. The first preset cutoff frequency is a preset value used to determine whether the signal is a low-frequency signal; the second preset cutoff frequency is a preset value used to determine whether the signal is a high-frequency signal. Through pre-processing steps such as denoising, filtering, and adjusting the dynamic range, the quality of the audio signal can be improved, providing data support for subsequent sound source localization of the audio signal, thereby improving the efficiency and accuracy of sound source localization.

[0038] Optionally, after obtaining multiple audio signals, the sound intensity of each audio signal in the multiple audio signals can be automatically detected by using a sound pressure sensor, and the duration of each audio signal can be recorded to obtain the sound duration of each audio signal.

[0039] S120: Perform positioning processing on the audio signal to determine the user sound source position corresponding to the audio signal.

[0040] The user sound source position is the specific position of the user who makes the sound in the vehicle.

[0041] Specifically, after obtaining multiple audio signals, the multiple audio signals can be positioned. For example, the multiple audio signals can be time-synchronized and synchronized to a unified time axis to ensure that each microphone starts collecting audio signals at the same time point; then, feature extraction is performed on the multiple audio signals to extract features for sound source positioning, such as time difference, frequency difference or phase difference, etc., which are used to reflect the difference when the sound reaches different microphones; then, according to the application scenario and positioning accuracy requirements, a suitable sound source positioning algorithm is selected, such as a beamforming algorithm and a least squares estimation algorithm, etc., and the extracted features are calculated and processed using the sound source positioning algorithm to obtain the location information of the sound source, that is, the user sound source location.

[0042] Optionally, when there are multiple sound sources, multi-source separation technology can be used to process the audio signal to separate the audio signals of different sound sources, and then the audio signals of different sound sources are positioned to obtain the position corresponding to each sound source, thereby obtaining the sound source positions of multiple users.

[0043] S130: Determine a control strategy according to the user's sound source position, the sound intensity and the sound duration of each audio signal.

[0044] The control strategy is a series of instructions or decision-making processes for dynamically adjusting the working state of the microphone; optionally, the control strategy may include controlling multiple microphones to be in an on state and controlling multiple microphones to be in an off state.

[0045] Specifically, after obtaining the user sound source position corresponding to the audio signal, the control strategy can be determined according to the user sound source position, the sound intensity and sound duration of each audio signal. For example, it can be determined whether the user sound source position, the sound intensity and sound duration of each audio signal meet the preset conditions, so as to determine the specific control strategy. For example, when the user sound source position, the sound intensity and sound duration of each audio signal meet the preset conditions, the control strategy can be determined to control multiple microphones to be in the on state; when the user sound source position, the sound intensity and sound duration of each audio signal do not meet the preset conditions, the control strategy can be determined to control multiple microphones to be in the off state.

[0046] S140: Control multiple microphones based on the control strategy.

[0047] Specifically, after obtaining the control strategy, multiple microphones can be controlled based on the control strategy, that is, the current working state of each of the multiple microphones, that is, the open state or the closed state, can be obtained respectively, and then the control strategy is analyzed to determine the expected working state of the multiple microphones, and when the current working state is different from the expected working state, the current working state of the corresponding microphone is switched to the expected working state, so as to control the multiple microphones. For example, when the control strategy is to control multiple microphones to be in an open state, and the current working state of the microphone is a closed state, the microphone can be turned on to switch the microphone from a closed state to an open state; when the control strategy is to control multiple microphones to be in an closed state, and the current working state of the microphone is an open state, the microphone can be turned off to switch the microphone from an open state to a closed state; when the control strategy is to control multiple microphones to be in an open state, and the current working state of the microphone is an open state, the open state of the microphone can be kept unchanged.

[0048] According to the technical solution of the embodiment of the present application, the vehicle can obtain audio signals collected by multiple microphones, and locate the audio signals to determine the user sound source position corresponding to the audio signal, and then determine the control strategy according to the user sound source position, the sound intensity and sound duration of each audio signal, such as controlling multiple microphones to be in an on state or controlling multiple microphones to be in an off state, and then controlling multiple microphones based on the control strategy to adjust the current working state of the microphone, thereby realizing the automatic control function of the microphone, without the driver having to manually adjust the microphone state, avoiding the situation where the driver is distracted and the eyes leave the road ahead due to manual operation, thereby reducing the risk of traffic accidents caused by driver distraction, thereby improving the driving safety of the vehicle; by determining the control strategy through the user sound source position, the sound intensity and sound duration of each audio signal, and automatically controlling multiple microphones based on the control strategy, the microphone state can be intelligently adjusted according to the actual sound environment in the car, providing users with a more comfortable and convenient communication experience, and can reduce unnecessary power consumption, thereby improving the user experience.

[0049] The following further describes a microphone control method provided by an embodiment of the present application. Figure 2 1 is another flow chart of the microphone control method provided in the embodiment of the present application. The embodiment of the present application is optimized based on the above embodiments. Figure 2 The method of this embodiment includes but is not limited to the following steps:

[0050] S201: Acquire audio signals collected by multiple microphones.

[0051] S202: Determine the arrival time difference of each pair of microphones among the multiple microphones.

[0052] Among them, the arrival time difference is the time delay difference caused by the difference in distance between the sound source and different microphones when the same sound propagates from the sound source to different microphones. That is, when the user makes a sound, the sound will propagate in the air in the form of sound waves and be received by each microphone in turn. Since the distance between each microphone and the user is different, the time it takes for the sound to reach each microphone will also be different. This time difference is called the arrival time difference.

[0053] Specifically, after acquiring audio signals collected by multiple microphones, the arrival time difference of each pair of microphones in the multiple microphones can be determined, including Sa1-Sa2:

[0054] Sa1. For any pair of microphones, calculate the similarity of the audio signals collected by the pair of microphones at different time offsets, and use the time offset with the highest similarity as the target time offset.

[0055] The target time offset is the time offset when the similarity between the audio signals collected by the two microphones reaches the highest level, reflecting the time difference required for the sound wave to propagate from the sound source to the two microphones.

[0056] Specifically, for any pair of microphones, the similarity of the audio signals collected by the pair of microphones at different time offsets is calculated, including Sb1-Sb2:

[0057] Sb1. Sample the audio signals collected by the pair of microphones based on a sampling time interval to obtain a first discrete audio signal and a second discrete audio signal.

[0058] The sampling time interval is the time difference between two adjacent sampling points in the process of sampling the audio signal.

[0059] The first discrete audio signal and the second discrete audio signal are two discrete audio signals obtained by sampling audio signals collected by two different microphones, and each sample point corresponds to an audio signal at a time point. It should be noted that the first discrete audio signal and the second discrete audio signal may be aligned or offset in time.

[0060] Specifically, a pre-set sampling time interval can be obtained, and the audio signals collected by two microphones in the pair of microphones can be sampled based on the sampling time interval respectively to obtain two discrete audio signals, namely a first discrete audio signal and a second discrete audio signal, and the first discrete audio signal and the second discrete audio signal are discrete audio signals corresponding to audio signals collected by different microphones.

[0061] Exemplarily, it is known that the pair of microphones includes microphone 1 and microphone 2, and the sampling time interval is 0.001s. Then, the audio signal collected by microphone 1 can be sampled based on the sampling time interval to obtain a first discrete audio signal {sampling point 11, sampling point 12, sampling point 13, sampling point 14}, and the time interval between each sampling point is 0.001s; then the audio signal collected by microphone 2 is sampled based on the sampling time interval to obtain a second discrete audio signal {sampling point 21, sampling point 22, sampling point 23, sampling point 24}, and the time interval between each sampling point is 0.001s.

[0062] Sb2. Calculate similarities between the first discrete audio signal and the second discrete audio signal at different time offsets based on a discrete cross-correlation function.

[0063] The discrete cross-correlation function is used to measure the similarity between two discrete audio signals at different time offsets, and the similarity between the two discrete audio signals is evaluated by calculating the product sum of the two discrete audio signals at different time offsets.

[0064] Specifically, after obtaining the first discrete audio signal and the second discrete audio signal, a reasonable time offset range can be determined according to actual conditions, and the time offset range should cover all possible time offset situations; then a discrete cross-correlation function is applied to the first discrete audio signal and the second discrete audio signal to calculate their similarities at different time offsets, that is, for each time offset value in the time offset range, any time offset value is selected as the current time offset value, and then the second discrete audio signal is shifted relative to the first discrete audio signal by the current time offset value (which may include positive shift and negative shift), and the product sum of the corresponding sampling points of the two discrete audio signals is calculated to obtain the similarity between the first discrete audio signal and the second discrete audio signal at the current time offset value, and then other time offset values ​​can be selected as the current time offset value, and the above process is repeated to obtain the similarity between the first discrete audio signal and the second discrete audio signal at different time offsets.

[0065] Optionally, the discrete cross-correlation function is calculated as:

[0066]

[0067] Wherein, A[t] represents the audio signal of the first discrete audio signal at the tth sampling point; B[t+τ] represents the audio signal of the second discrete audio signal at the t+τth sampling point; N represents the length of the shorter discrete audio signal of the first discrete audio signal and the second discrete audio signal; τ represents the time offset value, that is, the number of samples of the second discrete audio signal shifted along the time axis relative to the first discrete audio signal. When τ is a positive number, it indicates that the second discrete audio signal is shifted to the right (future), and when τ is a negative number, it indicates that the second discrete audio signal is shifted to the left (past); R AB(τ) represents the similarity between the first discrete audio signal and the second discrete audio signal when the time is offset by τ.

[0068] In an embodiment of the present application, by calculating the similarity of two discrete audio signals at different time offsets through a discrete cross-correlation function, the time delay between the two discrete audio signals can be accurately estimated, and the influence of noise on the similarity calculation can be suppressed to a certain extent, thereby providing data support for the subsequent determination of the target time offset, thereby improving the accuracy and efficiency of determining the target time offset.

[0069] After calculating the similarities of the audio signals collected by the pair of microphones at different time offsets, the similarities at different time offsets may be sorted, and the time offset with the highest similarity may be selected, and then the time offset may be determined as the target time offset.

[0070] Sa2. Determine the arrival time difference of the pair of microphones according to the target time offset.

[0071] Specifically, after obtaining the target time offset, the arrival time difference of the pair of microphones can be determined based on the target time offset and the sampling time interval, that is, the product between the target time offset and the sampling time interval can be calculated, and the product can be determined as the arrival time difference of the pair of microphones, which can improve the accuracy and efficiency of determining the arrival time difference.

[0072] For example, it is known that the sampling time interval is 0.001 s, and the target time offset corresponding to microphone 1 and microphone 2 is 10 (ie, 10 sampling points), then the arrival time difference between microphone 1 and microphone 2 is 0.01 s (ie, 0.001×10).

[0073] In an embodiment of the present application, by finding the time offset with the highest similarity, the time difference between the audio signal arriving at the two microphones from the sound source can be estimated more accurately, thereby improving the accuracy of determining the arrival time difference, providing data support for the subsequent determination of the user's sound source position, thereby improving the sound source localization accuracy, and thus improving the accuracy of determining the user's sound source position.

[0074] S203: Determine the user sound source position according to the arrival time difference and the position data of multiple microphones.

[0075] The position data of the microphone is the specific installation position of the microphone in the car, which is usually expressed in the form of two-dimensional coordinates.

[0076] Specifically, after obtaining the arrival time difference of each pair of microphones among the multiple microphones, the user sound source position can be determined according to the arrival time difference and the position data of the multiple microphones. That is, three microphones can be arbitrarily selected from the multiple microphones, namely, microphone M1, microphone M2 and microphone M3, and the position data of microphone M1 is known to be (x1, y1), the position data of microphone M2 is (x2, y2), the position data of microphone M3 is (x3, y3), and the arrival time difference between microphone M1 and microphone M2 is T 12 , the arrival time difference between microphone M1 and microphone M3 is T 13 , the propagation speed of sound in the air is c. At this time, assuming that the user's sound source position is (x, y), the user's sound source position (x, y) can be determined by the following formula:

[0077]

[0078] S204: Determine whether the user sound source position is within the set area, whether the sound intensity of each audio signal is greater than the set intensity, and whether the sound duration of each audio signal is greater than the set duration.

[0079] The set area is a pre-set specific area in the car, and the audio signal in the area will be regarded as a valid signal. The user can adjust and set the set area according to actual use needs, and the embodiment of the present application does not specifically limit this. For example, the set area can be the main driving area.

[0080] The set intensity is a preset sound intensity threshold value, which is used to determine whether the audio signal is a valid signal. The user can adjust and set the set intensity according to actual use requirements, and the embodiments of the present application do not make specific limitations on this. The set duration is a preset sound duration threshold value, which is used to determine whether the audio signal is a valid signal. The user can adjust and set the set duration according to actual use requirements, and the embodiments of the present application do not make specific limitations on this. The set area, set intensity, and set duration can ensure that the sound of a certain intensity and a certain duration emitted by a user in a specific area is a valid sound.

[0081] Specifically, after obtaining the user's sound source position, it can be determined whether the user's sound source position is within the set area, whether the sound intensity of each audio signal is greater than the set intensity, and whether the sound duration of each audio signal is greater than the set duration; if the user's sound source position is within the set area, the sound intensity of each audio signal is greater than the set intensity, and the sound duration of each audio signal is greater than the set duration, it indicates that the audio signal is a valid signal, and S205 can be executed at this time; if the user's sound source position is outside the set area, or the sound intensity of any audio signal is not greater than the set intensity, or the sound duration of any audio signal is not greater than the set duration, it indicates that the audio signal is a short noise signal, that is, the audio signal is not a valid signal, and S206 can be executed at this time.

[0082] S205: When the user sound source position is within the set area, the sound intensity of each audio signal is greater than the set intensity, and the sound duration of each audio signal is greater than the set duration, determine the control strategy to control multiple microphones to be in an open state.

[0083] Specifically, when the user's sound source position is within the set area, the sound intensity of each audio signal is greater than the set intensity, and the sound duration of each audio signal is greater than the set duration, it indicates that the audio signal is a valid signal. At this time, the control strategy can be determined to control multiple microphones to be in the on state, and then S207 can be executed.

[0084] S206: When the user sound source position is outside the set area, or the sound intensity of any audio signal is not greater than the set intensity, or the sound duration of any audio signal is not greater than the set duration, determine the control strategy to control multiple microphones to be in a closed state.

[0085] Specifically, when the user's sound source position is outside the set area, or the sound intensity of any audio signal is not greater than the set intensity, or the sound duration of any audio signal is not greater than the set duration, it indicates that the audio signal is a noise signal. At this time, it can be determined that the control strategy is to control multiple microphones to be in the off state, and then S207 can be executed.

[0086] Exemplarily, in a car conference scenario, the set area can be set as the main driving area, the set intensity is 50dB, and the set duration is 5s; when the user's sound source position is in the main driving area, the sound intensity of each audio signal is greater than 50dB, and the sound duration of each audio signal is greater than 5s, it indicates that the driver is speaking, and the microphone needs to be turned on at this time, that is, the control strategy is determined to control multiple microphones to be in the on state; when the user's sound source position is outside the main driving area, or the sound intensity of any audio signal is not greater than 50dB, or the sound duration of any audio signal is not greater than 5s, it indicates that the driver is not speaking or the audio signal is a noise signal, and the microphone needs to be turned off at this time, that is, the control strategy is determined to control multiple microphones to be in the off state.

[0087] S207: Control multiple microphones based on the control strategy.

[0088] The implementation process and technical principle of S207 are the same as those of S140, and can be included in the description of S140 above, so they will not be repeated here.

[0089] S208: When the multiple microphones are switched from the off state to the on state, an audio signal to be padded with a preset padded time length is obtained.

[0090] Among them, the preset filling time is a preset value, which is an artificially set time in order to maintain the continuity of the audio signal or meet specific needs; when the microphone state is switched from the off state to the on state, the preset filling time is used to read the audio signal from the cache space to fill the audio gap during the period when the microphone is turned off.

[0091] The audio signal to be complemented is an audio signal used to fill in the missing or insufficient part of the audio signal collected by the microphone, that is, the audio signal read from the cache space and cached during the period when the microphone is turned off.

[0092] Specifically, when the microphone is turned off, the vehicle controller can collect audio signals in real time through other audio collection devices, and store the collected audio signals in the cache space for subsequent completion of the audio signals. The cache space can be a ring buffer, which is a first-in-first-out data structure used to store audio signals in the recent period of time.

[0093] When controlling multiple microphones to switch from an off state to an on state, the moment when the microphone switches from an off state to an on state can be determined, and then the audio signal within a preset filling time before the moment is obtained from the cache space, and the audio signal is determined as the audio signal to be filled.

[0094] Exemplarily, it is known that the preset filling duration is 0.5s, and the moment when multiple microphones are switched from the off state to the on state is 9:18:20:500. When the multiple microphones are switched from the off state to the on state, the audio signals within 0.5s before 9:18:20:500 can be read from the ring buffer, that is, the audio signals between 9:18:20:000 and 9:18:20:500 are read, and the audio signals between 9:18:20:000 and 9:18:20:500 are determined as the audio signals to be filled.

[0095] S209: Merge the audio signal to be completed and the audio signal to obtain a merged audio signal.

[0096] The combined audio signal is a complete audio signal obtained by combining the audio signal to be completed and the audio signal.

[0097] Specifically, after obtaining the audio signal to be completed, the audio signal to be completed and the audio signal can be merged using voice splicing technology to obtain a merged audio signal, and the merged audio signal is processed to ensure a smooth transition of the merged audio signal at the merge point without obvious discontinuity or jump.

[0098] S210: Send the combined audio signal to a target terminal.

[0099] The target terminal is the final device that receives and processes the combined audio signal, and is used to convert the audio signal into audible sound; the target terminal usually has the ability to decode, play and / or further process the audio signal. For example, in a vehicle conference scenario, the target terminal can be other vehicles, remote servers, or personal devices of other conference participants (such as smartphones, tablets, laptops, and desktop computers, etc.).

[0100] For example, in a vehicle conferencing scenario, after obtaining the merged audio signal, the vehicle controller can send the merged audio signal to the target terminals of other conference participants via the Internet, so that the target terminal converts the merged audio signal into a sound signal audible to the user and plays it to the corresponding conference participants.

[0101] Optionally, S201 to S210 can be integrated into one module. When the user is in a scenario where automatic control of the microphone status is required (such as an in-car meeting scenario and an in-car call scenario, etc.), the corresponding controls on the in-car display screen can be triggered to execute the control process of S201 to S210, thereby avoiding unnecessary calculations and waste of power.

[0102] According to the technical solution of the embodiment of the present application, the vehicle can obtain audio signals collected by multiple microphones and determine the arrival time difference of each pair of microphones in the multiple microphones, and then determine the user sound source position according to the arrival time difference and the position data of the multiple microphones, and use the audio signals captured by the multiple microphones and combined with the arrival time difference between each pair of microphones to more accurately locate the sound source, and reduce the problem of inaccurate sound source positioning caused by environmental interference or error accumulation of a single microphone, thereby reducing the positioning error, thereby improving the accuracy of determining the user sound source position, and providing data support for the subsequent determination of the control strategy; then, when the user sound source position is within the set area, the sound intensity of each audio signal is greater than the set intensity, and the sound duration of each audio signal is greater than the set duration, the control strategy can be determined to control the multiple microphones to be in the on state; at the user sound source position When the vehicle is outside the set area, or the sound intensity of any audio signal is not greater than the set intensity, or the sound duration of any audio signal is not greater than the set duration, the control strategy is determined to control multiple microphones to be in an off state. An accurate control strategy can be determined according to the actual sound environment in the vehicle, thereby improving the accuracy and efficiency of determining the control strategy, thereby improving the control accuracy and efficiency of the microphone; thereafter, multiple microphones are automatically controlled based on the control strategy, and the microphone state can be automatically adjusted according to the actual sound environment in the vehicle, without the driver having to manually control the microphone state, thereby improving the driving safety of the vehicle and providing users with a more comfortable and convenient communication experience. At the same time, turning on the microphone when needed and turning off the microphone when not needed can reduce unnecessary energy consumption and extend the service life of the microphone, thereby improving the user experience.

[0103] In addition, when multiple microphones are switched from an off state to an on state, an audio signal to be completed with a preset completion time is obtained, and then the audio signal to be completed and the audio signal are merged to obtain a merged audio signal, and the merged audio signal is sent to the target terminal, thereby realizing the audio completion function, ensuring the continuity of the merged audio signal, and filling the sound gaps during the period when the microphone is turned off, thereby avoiding the problem of incomplete audio signal due to the microphone being turned off, ensuring the integrity of the audio signal, and providing users with a more reliable communication experience.

[0104] Figure 3 is a schematic diagram of a structure of a microphone control device provided in an embodiment of the present application, referring to Figure 3 , the microphone control device may include:

[0105] The signal acquisition module 310 is used to acquire audio signals collected by multiple microphones, wherein one microphone corresponds to one audio signal, and the audio signal includes sound intensity and sound duration;

[0106] The positioning module 320 is used to perform positioning processing on the audio signal to determine the user sound source position corresponding to the audio signal;

[0107] A strategy determination module 330, for determining a control strategy according to the user sound source position, the sound intensity and the sound duration of each audio signal;

[0108] The control module 340 is used to control the multiple microphones based on the control strategy.

[0109] In one embodiment, the positioning module 320 is specifically used for:

[0110] determining a time difference of arrival for each pair of microphones in the plurality of microphones;

[0111] The user's sound source position is determined based on the arrival time difference and the position data of multiple microphones.

[0112] In one embodiment, the positioning module 320 determines the arrival time difference of each pair of microphones in the plurality of microphones, including:

[0113] For any pair of microphones, calculate the similarity of the audio signals collected by the pair of microphones at different time offsets, and take the time offset with the highest similarity as the target time offset;

[0114] The arrival time difference for the pair of microphones is determined based on the target time offset.

[0115] In one embodiment, the positioning module 320 calculates the similarity of the audio signals collected by the pair of microphones at different time offsets, including:

[0116] Sampling the audio signal collected by the pair of microphones based on a sampling time interval to obtain a first discrete audio signal and a second discrete audio signal;

[0117] Based on the discrete cross-correlation function, calculating the similarity between the first discrete audio signal and the second discrete audio signal at different time offsets;

[0118] The positioning module 320 determines the arrival time difference of the pair of microphones according to the target time offset, including:

[0119] The arrival time difference of the pair of microphones is determined based on the target time offset and the sampling time interval.

[0120] In one embodiment, the policy determination module 330 is specifically used to:

[0121] When the user's sound source position is within the set area, the sound intensity of each audio signal is greater than the set intensity, and the sound duration of each audio signal is greater than the set duration, determining the control strategy to control multiple microphones to be in an open state;

[0122] When the user's sound source position is outside the set area, or the sound intensity of any audio signal is not greater than the set intensity, or the sound duration of any audio signal is not greater than the set duration, the control strategy is determined to control multiple microphones to be in a closed state.

[0123] In one embodiment, the control device of the microphone further includes a padding module, and the padding module is specifically used for:

[0124] When the multiple microphones are switched from an off state to an on state, an audio signal to be padded with a preset padded time length is obtained;

[0125] Merging the audio signal to be padded and the audio signal to obtain a merged audio signal;

[0126] The combined audio signal is sent to the target terminal.

[0127] In one embodiment, the signal acquisition module 310 is specifically used for:

[0128] Obtaining initial audio signals collected by multiple microphones;

[0129] The initial audio signal is preprocessed to obtain an audio signal, wherein the preprocessing includes at least one of denoising, filtering, and adjusting a dynamic range.

[0130] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0131] The microphone control device provided in this embodiment can be applied to the microphone control method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0132] Figure 4 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. Figure 4 A block diagram of an exemplary vehicle 11 suitable for implementing embodiments of the present application is shown. Figure 4 The vehicle 11 shown is only an example and should not bring any limitation to the function and scope of use of the present embodiment.

[0133] like Figure 4As shown, the vehicle 11 is embodied in the form of a general purpose computing electronic device. The components of the vehicle 11 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting various system components (including the system memory 28 and the processing unit 16).

[0134] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0135] The vehicle 11 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the vehicle 11, including volatile and non-volatile media, removable and non-removable media.

[0136] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The vehicle 11 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0137] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0138] The vehicle 11 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the vehicle 11, and / or any device that enables the vehicle 11 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the vehicle 11 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20.

[0139] like Figure 4 As shown, the network adapter 20 communicates with other modules of the vehicle 11 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the vehicle 11, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0140] The processing unit 16 executes various functional applications and page displays by running the programs stored in the system memory 28, such as implementing a microphone control method provided in any embodiment of the present application.

[0141] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a microphone control method provided by any embodiment of the present application is implemented, for example.

[0142] The computer storage medium of this embodiment can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing programs, which can be used by instruction execution systems, devices or devices or used in combination with them.

[0143] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0144] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0145] Computer program code for performing the operation of the present application can be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0146] It should be understood by those skilled in the art that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0147] In addition, the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0148] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the inventive concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for controlling a microphone, characterized in that: Applied to a vehicle, the vehicle includes a plurality of microphones, and the method includes: Acquire audio signals collected by the multiple microphones, wherein one microphone corresponds to one audio signal, and the audio signal includes sound intensity and sound duration; Performing positioning processing on the audio signal to determine a user sound source position corresponding to the audio signal; Determining a control strategy according to the user sound source position, the sound intensity and the sound duration of each of the audio signals; The plurality of microphones are controlled based on the control strategy.

2. The microphone control method according to claim 1, characterized in that: The performing positioning processing on the audio signal to determine the user sound source position corresponding to the audio signal includes: determining a time difference of arrival for each pair of microphones in the plurality of microphones; The user sound source position is determined according to the arrival time difference and the position data of the multiple microphones.

3. The microphone control method according to claim 2, characterized in that: Determining the arrival time difference of each pair of microphones in the plurality of microphones comprises: For any pair of microphones, calculate the similarity of the audio signals collected by the pair of microphones at different time offsets, and take the time offset with the highest similarity as the target time offset; The arrival time difference of the pair of microphones is determined according to the target time offset.

4. The microphone control method according to claim 3, characterized in that: The calculating the similarity of the audio signals collected by the pair of microphones at different time offsets includes: Sampling the audio signal collected by the pair of microphones based on a sampling time interval to obtain a first discrete audio signal and a second discrete audio signal; calculating, based on a discrete cross-correlation function, similarities between the first discrete audio signal and the second discrete audio signal at different time offsets; Determining the arrival time difference of the pair of microphones according to the target time offset comprises: The arrival time difference of the pair of microphones is determined according to the target time offset and the sampling time interval.

5. The microphone control method according to claim 1, characterized in that: The determining of the control strategy according to the user sound source position, the sound intensity and the sound duration of each of the audio signals comprises: When the user sound source position is within the set area, the sound intensity of each of the audio signals is greater than the set intensity, and the sound duration of each of the audio signals is greater than the set duration, determining that the control strategy is to control the multiple microphones to be in an on state; When the user sound source position is outside the set area, or the sound intensity of any one of the audio signals is not greater than the set intensity, or the sound duration of any one of the audio signals is not greater than the set duration, it is determined that the control strategy is to control the multiple microphones to be in a closed state.

6. The microphone control method according to claim 1, characterized in that: The method further comprises: When the multiple microphones are switched from an off state to an on state, an audio signal to be padded with a preset padded time length is obtained; Merging the audio signal to be padded and the audio signal to obtain a merged audio signal; The combined audio signal is sent to a target terminal.

7. The microphone control method according to claim 1, characterized in that: The obtaining of audio signals collected by the multiple microphones includes: Acquire initial audio signals collected by the multiple microphones; The initial audio signal is preprocessed to obtain the audio signal, wherein the preprocessing includes at least one of denoising, filtering, and adjusting a dynamic range.

8. A microphone control device, characterized in that: Applied to a vehicle, the vehicle includes a plurality of microphones, and the device includes: A signal acquisition module, used to acquire audio signals collected by the multiple microphones, wherein one microphone corresponds to one audio signal, and the audio signal includes sound intensity and sound duration; A positioning module, used to perform positioning processing on the audio signal to determine the user sound source position corresponding to the audio signal; A strategy determination module, used to determine a control strategy according to the user sound source position, the sound intensity and the sound duration of each of the audio signals; A control module is used to control the multiple microphones based on the control strategy.

9. A vehicle, characterized in that: The vehicle comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the microphone control method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the microphone control method according to any one of claims 1 to 7 is implemented.

Images