In-vehicle communication assistance system

Abstract

An "in-vehicle communication assistance system" is provided to relay spoken voice without canceling unwanted tones. A first variable filter (31) generates a cancel tone at the first seat by taking the tone of the second seat audio source (1) as input. The transfer functions of the first variable filter (31) and the second variable filter (36) are updated to minimize the level of the signal (Eu) obtained by subtracting the output of the auxiliary filter (35) that generates a correction signal to correct the position difference between the first seat microphone (9) and the first seat from the output of the first seat microphone (9) and the output of the second variable filter (36) that takes the tone of the first seat audio source (2) as input. During the period when the level of the signal (Eu) exceeds the threshold, the signal (Eu) is relayed to the second seat as spoken voice.

Description

Technical Field

[0001] This invention relates to technology for assisting communication based on in-vehicle speech. Background Technology

[0002] As a technology to assist communication based on speech within the vehicle, it is known to pick up the speech of a user sitting in the first seat of the car through a microphone for the first seat and output it from a speaker for the second seat of the car (e.g., Patent Document 1).

[0003] Furthermore, in such technology, it is also known to output from a speaker for a second seat the sound picked up by a microphone for the first seat, after canceling the sound such as music output from a speaker for the first seat (e.g., Patent Document 2).

[0004] Furthermore, as a technology related to the present invention, it is known that a microphone is configured near the noise cancellation position, an adaptive filter generates a cancellation tone at the noise cancellation position based on the output signal of the noise source, and a speaker outputs the cancellation tone. In the adaptive filter, the transfer function is adaptively set as an error signal using a signal after the output of the microphone is corrected by an auxiliary filter (e.g., Patent Documents 3 and 4).

[0005] In this technique, an auxiliary filter is configured with a pre-learned transfer function that corrects the signal actually output by the microphone to the signal output from the microphone when the microphone is positioned at the noise cancellation location. By using such an auxiliary filter, noise is cancelled at noise cancellation locations different from the microphone's position.

[0006] Existing technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2002-51392

[0009] Patent Document 2: Japanese Patent Application Publication No. 2010-163054

[0010] Patent Document 3: Japanese Patent Application Publication No. 2020-12917

[0011] Patent Document 4: Japanese Patent Application Publication No. 2018-72770 Summary of the Invention

[0012] The problem that the invention aims to solve

[0013] In order for users in the first and second seats to enjoy different music, in the case where the sound of the first sound source is output from the speaker of the first seat and the sound of the second sound source is output from the speaker of the second seat, and the sound of the two sound sources is propagated to the microphone of the first seat, simply canceling the sound of the first sound source by the sound picked up by the microphone of the first seat cannot accurately and clearly output the speech of the user in the first seat to the user in the second seat from the speaker of the second seat.

[0014] Furthermore, if the user in the first seat and the user in the second seat are listening to different audio sources, it is desirable that each user cannot hear the audio source being listened to by other users.

[0015] Therefore, the objective of this invention is to provide an in-vehicle communication assistance system that, in an environment where users in different seats of a car are listening to different audio sources, suppresses the audio sources that other users are listening to, and enables good speech-based communication between users.

[0016] Methods for solving problems

[0017] To achieve the aforementioned objective, the present invention provides an in-vehicle communication assistance system installed in a car having a first seat and a second seat that are distinct from each other. The system includes: a microphone for picking up sound from the first seat; a first sound source device; a first speaker for a user in the first seat that outputs the output sound of the first sound source device; a second sound source device; a second speaker for a user in the second seat that outputs the output sound of the second sound source device; a noise control unit that outputs a cancellation tone from the first speaker to the user in the first seat, canceling the output sound of the second sound source device output from the second speaker; and a speech output unit. The noise control unit includes: a first adaptive filter that generates the cancellation tone based on the output sound of the second sound source device as input; a second adaptive filter that takes the output sound of the first sound source device as input; and a first signal generation unit that generates a first signal, which is a signal obtained by subtracting at least the output of the second adaptive filter from the output of the microphone. The first and second adaptive filters perform adaptive operations to update their transfer functions, thereby reducing the level of the first signal. Furthermore, the speech output unit outputs the first signal as the speech of the user in the first seat from the second speaker.

[0018] Here, such an in-vehicle communication assistance system can also be configured such that, in the noise control unit, an auxiliary filter is provided that takes the output sound of the second sound source device as input, and in the first signal generation unit, a signal obtained by subtracting the output of the second adaptive filter and the output of the auxiliary filter from the output of the microphone is generated as the first signal. A pre-determined transfer function is set to the auxiliary filter, and this pre-determined transfer function serves as the transfer function for the sound picked up at the listening position of the user in the first seat, by subtracting the output of the auxiliary filter from the output of the microphone.

[0019] Furthermore, the above-mentioned in-vehicle communication assistance system can also be configured such that, in the voice output unit, when the first signal is greater than a predetermined level, the first signal is output from the second speaker.

[0020] Furthermore, in this case, the adaptive operation of the first adaptive filter and the second adaptive filter can also be stopped in the speech output unit when the first signal is greater than a predetermined level.

[0021] According to the in-vehicle communication assistance system described above, the sound of the second audio source device heard by the second-seat user is suppressed for the first-seat user, and the SN can better output the first-seat user's speech from the second speaker for the second-seat user, thus assisting the first-seat user in hearing the speech.

[0022] Invention Effects

[0023] As described above, according to the present invention, an in-vehicle communication assistance system is provided that, in an environment where users in each seat of a car are listening to different audio sources, the user can suppress the audio from other users' audio sources and can conduct good speech-based communication between users. Attached Figure Description

[0024] Figure 1 This is a block diagram illustrating the structure of the in-vehicle communication assistance system according to an embodiment of the present invention.

[0025] Figure 2 This diagram illustrates the configuration of the speaker and the first seat microphone in an in-vehicle communication assistance system according to an embodiment of the present invention.

[0026] Figure 3 This is a block diagram illustrating the structure of an active noise control device according to an embodiment of the present invention.

[0027] Figure 4 This is a flowchart illustrating the speech relay processing involved in the embodiments of the present invention.

[0028] Figure 5 This is a block diagram illustrating the structure for learning the transfer function of the auxiliary filter according to an embodiment of the present invention.

[0029] Label Explanation

[0030] 1…Second seat audio source, 2…First seat audio source, 3…Active noise control device, 4…ICC processing unit, 5…Second seat speaker, 6…First seat speaker, 7…Adder for second seat speaker, 8…Adder for first seat speaker, 9…First seat microphone, 31…First variable filter, 32…First adaptive algorithm execution unit, 33…Estimation filter, 34…First subtractor, 35…Auxiliary filter, 36…Second variable filter, 37…Second adaptive algorithm execution unit, 38…Second subtractor, 50…First stage learning processing unit, 51…Estimation filter for learning, 52…Variable filter for first stage learning, 53…Adaptive algorithm execution unit for first stage learning, 60…Microphone for learning, 70…Second stage learning processing unit, 71…Fixed filter, 72…Variable filter for second stage learning, 73…Adaptive algorithm execution unit for second stage learning, 74…Subtractor for second stage learning. Detailed Implementation

[0031] The embodiments of the present invention will be described below.

[0032] exist Figure 1 The structure of the in-vehicle communication assistance system according to this embodiment is shown in the figure.

[0033] The in-vehicle communication assistance system is a system installed in a car, as shown in the figure, which includes a second seat audio source 1, a first seat audio source 2, an active noise control device 3, an ICC processing unit 4 (In-Car Communication processing unit 4), a second seat speaker 5, a first seat speaker 6, a second seat speaker adder 7, a first seat speaker adder 8, and a first seat microphone 9.

[0034] like Figure 2 As shown, one seat in the car is designated as the first seat, and the other seat is designated as the second seat. The first seat speaker 6 is a speaker for the user of the first seat, located near the first seat. The second seat speaker 5 is a speaker for the user of the second seat, located near the second seat. Furthermore, Figure 2As an example, the configuration or number of the first-seat speaker 6 or the second-seat speaker 5 can be arbitrary, as long as the first-seat speaker 6 mainly emits sound to the user in the first seat and the second-seat speaker 5 mainly emits sound to the user in the second seat.

[0035] In addition, the first seat microphone 9 is, for example, Figure 2 The microphone was positioned near the first seat.

[0036] return Figure 1 The first seat audio source 2 is an audio source that outputs music and other sounds heard by the user in the first seat. The sound output by the first seat audio source 2 is output from the first seat speaker 6 via the first seat speaker using the addition unit 8.

[0037] In addition, the second seat audio source 1 is an audio source that outputs music and other sounds to the user in the second seat. The sound output by the second seat audio source 1 is output from the second seat speaker 5 via the addition unit 7 of the second seat speaker.

[0038] The sound from the second seat audio source 1, the sound from the first seat audio source 2, and the sound picked up by the first seat microphone 9 are input to the active noise control device 3. The active noise control device 3 generates a canceling sound obtained by canceling the sound from the second seat audio source 1 heard by the user in the first seat from these input sounds, and outputs it from the first seat speaker 6 via the first seat speaker using the addition unit 8.

[0039] In addition, the active noise control device 3 uses the input sound to extract the speech Eu of the user in the first seat contained in the sound picked up by the microphone 9 in the first seat, and outputs it to the ICC processing unit 4.

[0040] Furthermore, the ICC processing unit 4 outputs the spoken voice Eu input from the active noise control device 3, or the sound after applying prescribed audio processing to the spoken voice Eu, from the second seat speaker 5 via the second seat speaker addition unit 7, thereby assisting the user in the second seat to hear the spoken voice of the user in the first seat.

[0041] Next, in Figure 3 The structure of the active noise control device 3 is shown in the figure.

[0042] As shown in the figure, the active noise control device 3 includes a first variable filter 31, a first adaptive algorithm execution unit 32, an estimation filter 33 with a pre-set transfer function S^(z), a first subtractor 34, an auxiliary filter 35 with a pre-set transfer function H(z), a second variable filter 36, a second adaptive algorithm execution unit 37, and a second subtractor 38.

[0043] The sound input from the second seat audio source 1 is processed by the first variable filter 31 as a canceled sound, and is output from the first seat speaker 6 via the first seat speaker using the adder 8.

[0044] In addition, the sound input from the second seat audio source 1 is transmitted to the first subtractor 34 after passing through the auxiliary filter 35.

[0045] The first subtractor 34 subtracts the output of the auxiliary filter 35 from the tone picked up by the first seat microphone 9 and sends it to the second subtractor 38.

[0046] In the auxiliary filter 35, the transfer function H(z) that corrects the signal actually output by the first seat microphone 9 is preset so that the output of the first subtractor 34 is the signal output from the first seat microphone 9 when the first seat microphone 9 is located at the ear position of the user in the first seat. The method for setting the transfer function H(z) of this auxiliary filter 35 will be described later.

[0047] On the other hand, the sound input from the first seat audio source 2 is transmitted to the second subtractor 38 after passing through the second variable filter 36.

[0048] The second subtractor 38 outputs the signal Eu, which is the result of subtracting the output of the second variable filter 36 from the output of the first subtractor 34, to the first adaptive algorithm execution unit 32 and the second adaptive algorithm execution unit 37, and outputs it as spoken voice Eu to the ICC processing unit 4.

[0049] Next, in the estimation filter 33, there is a preset estimated transfer characteristic S^(z) of the transfer function S(z) from the active noise control device 3 to the first seat microphone 9 estimated by actual measurement, etc. The estimation filter 33 convolves the transfer characteristic S^(z) with the input sound from the first seat audio source 2 and outputs it to the first adaptive algorithm execution unit 32.

[0050] The first variable filter 31, the first adaptive algorithm execution unit 32, and the estimation filter 33 constitute the Filtered-X adaptive filter.

[0051] The first adaptive algorithm execution unit 32 uses the signal after the transfer function S^(z) is convolved by the estimation filter 33 and the signal Eu output from the second subtractor 38. It performs an adaptive operation, using the signal output from the second subtractor 38 as an error, and executes adaptive algorithms such as NLMS (Normalized Least Mean Square Adaptive Filtering) or LMS (Least Mean Square Adaptive Filtering) to update the transfer function W(z) of the first variable filter 31, minimizing the level of the signal Eu output from the second subtractor 38. Furthermore, through this adaptive operation, the transfer function W(z) of the first variable filter 31 is updated to minimize the component of the sound output by the second seat audio source 1 contained in the signal Eu. As a result, the first variable filter 31 is adapted to output a cancellation tone that cancels the sound output by the second seat audio source 1 at the ear position of the user in the first seat.

[0052] In addition, the second variable filter 36 and the second adaptive algorithm execution unit 37 also constitute an adaptive filter.

[0053] Furthermore, the second adaptive algorithm execution unit 37 uses the tone output from the first seat audio source 2 and the signal Eu output from the second subtractor 38 to perform an adaptive operation, such as NLMS or LMS, by using the signal output from the second subtractor 38 as an error to update the transfer function U(z) of the second variable filter 36, so that the level of the signal Eu output from the second subtractor 38 is minimized. Through this adaptive operation, the transfer function U(z) of the second variable filter 36 is updated so that the component of the tone output from the first seat audio source 2 contained in the signal Eu is minimized.

[0054] In this way, through the operation of such an active noise control device 3, the components of the sound output by the second seat audio source 1 and the components of the sound output by the first seat audio source 2 contained in the signal Eu transmitted to the ICC processing unit 4 are minimized.

[0055] Therefore, when the voice of the user in the first seat is included in the sound picked up by the microphone 9 of the first seat, the voice Eu transmitted from the active noise control device 3 to the ICC processing unit 4 represents the voice of the user in the first seat.

[0056] Next, the speech relay processing performed by the ICC processing unit 4 to output the speech Eu of the user in the first seat from the second seat speaker 5 via the addition unit 7 will be explained.

[0057] exist Figure 4 The text in the middle indicates the process of relaying the voice message.

[0058] As shown in the figure, in the speech relay processing, the ICC processing unit 4 repeatedly measures the level of the speech Eu input from the active noise control device 3 (step 402) until the measured level exceeds the specified threshold (step 404).

[0059] Furthermore, if the measured level of the spoken voice Eu exceeds the specified threshold (step 404), the spoken voice output action is initiated (step 406).

[0060] Here, in the speech output operation, the speech Eu input from the active noise control device 3 is output to the addition unit 7 for the second seat speaker, thereby performing the output operation from the second seat speaker 5. Alternatively, in the speech output operation, a predetermined audio processing may be applied to the speech Eu input from the active noise control device 3 before outputting it to the addition unit 7 for the second seat speaker, thereby performing the output operation from the second seat speaker 5.

[0061] Furthermore, by stopping the updates of the transfer function W(z) of the first variable filter 31 and the transfer function U(z) of the second variable filter 36, the adaptive operation of the active noise control device 3 is stopped (step 408). More specifically, by setting the step size of the gain used to determine the update of the transfer function W(z) performed by the first adaptive algorithm execution unit 32 and the step size of the gain used to determine the update of the transfer function U(z) performed by the second adaptive algorithm execution unit 37 to 0, the updates of the transfer function W(z) of the first variable filter 31 and the transfer function U(z) of the second variable filter 36 are stopped.

[0062] Next, the measurement of the level of the spoken voice Eu input from the active noise control device 3 is repeated (step 410) until the measured level is below the threshold (step 412).

[0063] Furthermore, if the measured level of the spoken voice Eu falls below the threshold (step 412), the spoken voice output action is stopped (step 414).

[0064] Furthermore, the adaptive operation of the active noise control device 3 is restarted (step 416). More specifically, in step 414, the step size of the gain update of the transfer function W(z) performed by the first adaptive algorithm execution unit 32 and the step size of the gain update of the transfer function U(z) performed by the second adaptive algorithm execution unit 37 are restored to their original values ​​before the adaptive operation was stopped in step 408.

[0065] The above explains the speech relay processing performed by ICC Processing Unit 4.

[0066] Based on this speech relay processing, the speech of the user in the first seat is detected with high accuracy, and the speech output action is performed when the user in the first seat is speaking, outputting a speech output action from the speaker 5 in the second seat that consists of sound consisting of only the speech component, which can assist the user in the second seat in listening to the speech of the user in the first seat.

[0067] Furthermore, in the above speech relay processing, the adaptive action of the active noise control device 3 is stopped when the speech output is in progress because the suppression of the speech of the user in the first seat acts as a disturbance, causing the adaptive action to malfunction.

[0068] Next, the method for setting the transfer function H(z) of the auxiliary filter 35 of the active noise control device 3 described above will be explained.

[0069] The transfer function H(z) of the auxiliary filter 35 is set, for example, by performing a first-stage learning process and a second-stage learning process as follows while outputting the sound output from the second seat audio source 1 from the second seat speaker 5.

[0070] The first stage of learning processing and use Figure 5 The first-stage learning processing unit 50 shown in Figure a and the learning microphone 60 positioned at the ear of the user in the first seat are used for this process.

[0071] The first-stage learning processing unit 50 includes a learning estimation filter 51 with an estimated value Sv^(z) of the transfer function Sv(z) from the first-stage learning processing unit 50 to the learning microphone 60, a first-stage learning variable filter 52, and a first-stage learning adaptive algorithm execution unit 53.

[0072] Furthermore, in this structure, the sound output from the second-seat audio source 1 is sent to the first-stage learning variable filter 52, and the output of the first-stage learning variable filter 52 is sent to the first-seat speaker 6. Additionally, the sound output from the second-seat audio source 1 is input to the first-stage learning adaptive algorithm execution unit 53 via the learning estimation filter 51. The first-stage learning adaptive algorithm execution unit 53 uses the output of the learning microphone 60 as an error and executes an adaptive algorithm such as NLMS or LMS while using the output of the learning estimation filter 51 to update the transfer function W(z) of the first-stage learning variable filter 52.

[0073] Furthermore, through the execution of the adaptive algorithm, the transfer function W(z) of the variable filter 52 is used as the result of the first stage of learning after convergence and stabilization.

[0074] Next, the second phase of learning will use... Figure 5 The second stage of learning processing is carried out by the learning processing unit 70 shown in b.

[0075] The second-stage learning processing unit 70 includes a fixed filter 71 that sets the transfer function W(z) obtained as the result of the first-stage learning processing as the transfer function, a variable filter 72 for second-stage learning, an adaptive algorithm execution unit 73 for second-stage learning, and a subtractor 74 for second-stage learning.

[0076] Furthermore, in this structure, the sound output from the second seat audio source 1 is passed through a fixed filter 71 and then output to the first seat speaker 6.

[0077] Furthermore, the sound output from the second seat audio source 1 is transmitted to the second-stage learning subtractor 74 after passing through the second-stage learning variable filter 72. The second-stage learning subtractor 74 subtracts the output of the second-stage learning variable filter 72 from the signal picked up by the first seat microphone 9 and outputs the result.

[0078] The second-stage adaptive algorithm execution unit 73 uses the output of the second-stage learning subtractor 74 as error, and executes adaptive algorithms such as NLMS or LMS while using the tone output by the second seat audio source 1 to update the transfer function H(z) of the second-stage learning variable filter 72.

[0079] Furthermore, through the execution of the adaptive algorithm, the converged and stabilized transfer function H(z) is set as the transfer function H(z) of the auxiliary filter 35 of the active noise control device 3.

[0080] Here, the learned transfer function H(z), as described above, in the active noise control device 3, becomes the transfer function of the signal output from the first subtractor 34 when the first seat microphone 9 is located at the position of the user's ear in the first seat.

[0081] The embodiments of the present invention have been described above.

[0082] Thus, according to this embodiment, the sound of the second seat audio source 1 heard by the user in the second seat is suppressed for the user in the first seat, and the user in the second seat can output the speech of the user in the first seat more effectively from the second seat speaker 5.

[0083] Furthermore, the above description shows a case where one of each of the first seat speaker 6, second seat speaker 5, and first seat microphone 9 is provided; however, multiple of these can also be provided. Additionally, when multiple first seat microphones 9 are provided, the spoken voice Eu is generated for each of the first seat microphones. Figure 4 In step 404 of the speech relay processing, when all the speech Eu generated for each first seat microphone 9 exceeds the threshold, the process proceeds to step 406 and begins the speech output action.

[0084] Furthermore, the above description addresses the case where the audio source 1 of the second seat is canceled for the user in the first seat, and the user's speech is output to the speaker 5 of the second seat. However, this embodiment can also be applied to the case where the audio source of other seats is canceled for each user in multiple seats, and the speech of each user is output to the speakers of other seats.

[0085] Furthermore, this embodiment can also be extended to situations where the audio source for each seat has multiple channels, or to cancel the audio source for other seats for each of the user's left and right ears.

[0086] Furthermore, the auxiliary filter 35 of the active noise control device 3 in this embodiment can also be omitted, and in this case, the same effect can be obtained to a certain extent. In addition, in this case, the signal Eu is used as the signal obtained by subtracting the output of the second variable filter 36 from the sound picked up by the first seat microphone 9.

Claims

1. An in-vehicle communication assistance system, installed in a car having a first seat and a second seat that are distinct from each other, characterized in that, have: The microphone picks up the sound from the first seat; First sound source device; The first loudspeaker is a loudspeaker used by the user in the first seat that outputs the output sound of the first sound source device; Second sound source device; The second loudspeaker is a loudspeaker used by the user in the second seat to output the output sound of the second sound source device; The noise control unit outputs a cancellation tone from the first speaker to the user in the first seat, which cancels the output tone of the second sound source device output from the second speaker. as well as Voice output unit The noise control unit has: The first adaptive filter generates the cancel tone by taking the output tone of the second sound source device as input. The second adaptive filter takes the output sound of the first sound source device as input; An auxiliary filter takes the output sound of the second sound source device as its input. as well as A first signal generation unit generates a first signal, which is the signal obtained by subtracting the output of the second adaptive filter and the output of the auxiliary filter from the output of the microphone. The first adaptive filter and the second adaptive filter perform an adaptive operation to update their own transfer functions, so as to reduce the level of the first signal. The speech output unit outputs the first signal as the speech of the user in the first seat from the second speaker.

2. The in-vehicle communication assistance system as described in claim 1, characterized in that, A pre-determined transfer function is set to the auxiliary filter. This pre-determined transfer function is the transfer function of the sound picked up at the listening position of the user in the first seat, by subtracting the output of the auxiliary filter from the output of the microphone.

3. The in-vehicle communication assistance system as described in claim 2, characterized in that, When the first signal is greater than a specified level, the speech output unit outputs the first signal from the second speaker.

4. The in-vehicle communication assistance system as described in claim 3, characterized in that, When the first signal is greater than a specified level, the speech output unit stops the adaptive operation of the first adaptive filter and the second adaptive filter.

5. The in-vehicle communication assistance system as described in claim 1, characterized in that, When the first signal is greater than a specified level, the speech output unit outputs the first signal from the second speaker.

6. The in-vehicle communication assistance system as described in claim 5, characterized in that, When the first signal is greater than a specified level, the speech output unit stops the adaptive operation of the first adaptive filter and the second adaptive filter.

Images