Active control system of a vehicle dual microphone and a control method therefor

Abstract

An active control system of a vehicle dual microphone formed by a first microphone (510) and a second microphone (520), the control system comprising: a mouth shape change detection unit (100) installed in a vehicle and configured to capture an image of a driver sitting in a driver's seat of the vehicle and / or a passenger sitting in a passenger's seat and detect a change in the mouth shape of the driver and / or the passenger; a user and seat position detection unit (200) configured to detect driving state information of the driver and / or the passenger; an infotainment system (300) configured to determine a microphone beamforming control mode according to the captured image or images and the driving state information output by the mouth shape change detection unit (100) and the user and seat position detection unit (200), to control a microphone beamforming of the first microphone (510) installed adjacent to the driver's seat according to the determined microphone beamforming control mode, and to selectively adjust a specific microphone gain according to the determined microphone beamforming control mode to separately control the microphone beamforming of the first microphone (510) and the second microphone (520) installed adjacent to the passenger seat; and a microphone beamforming controller (400) configured to adjust an installation angle of the first microphone (510) installed adjacent to the driver's seat to be directed toward the driver, or installation angles of the first microphone (510) and the second microphone (520) installed adjacent to the driver's seat and the passenger's seat to be directed toward the driver and the passenger, respectively, according to the control of the infotainment system (300), wherein the user and seat position detection unit (200) includes: a first database (210) configured to store and manage the driving state information including selective seat height information of the driver or the front passenger, seat position information of the driver's seat or the front passenger's seat of the vehicle using a database method according to the control of the infotainment system (300); a first controller (220) configured to store or read the driving state information stored in the first database (210) according to the control of the infotainment system (300); and a second network communication unit (230) configured to transmit the driving state information to the infotainment system (300) according to the control of the first controller (220).

Description

BACKGROUNDArea of ​​Revelation

[0001] The present disclosure relates to an active control system of a vehicle dual microphone and a control method thereof. More specifically, the present disclosure relates to an active control system of a vehicle dual microphone and a control method thereof for selectively controlling a beamforming region of a microphone installed around a driver's seat or a passenger's seat according to a user's body information, vehicle seat position information, and the presence of a passenger. Discussion of the related issue

[0002] Generally, a microphone installed in a vehicle is used for making phone calls while driving or for recognizing a specific command using a voice recognition program.

[0003] However, there is often unnecessary background noise or sounds, such as mechanical or engine noise from the vehicle or noise coming from outside the vehicle, as well as noise made by the driver inside the vehicle.

[0004] To minimize the influence of unnecessary noise, such as ambient mechanical or engine noise or noise from the vehicle's perimeter, two or more microphones are mounted on the vehicle. Microphone beamforming is controlled using a differential comparison algorithm, which differentially compares the audio signals input from different microphones and separates speech from other noises.

[0005] For example, Korean patent application KR 10 2015 047 981 A discloses an apparatus and method for controlling a beamforming microphone taking into account a position of a driver's seat in order to improve the call quality of a vehicle hands-free device by controlling the directional characteristics of a beamforming microphone depending on the position of the driver's seat in order to capture the driver's voice as best as possible.

[0006] Furthermore, KR 10 1 750 338 B1 discloses a beamforming method and apparatus for recognizing a speaker's voice, searching for a stored image of the speaker whose voice is recognized, searching for the speaker by a camera based on the retrieved image, recognizing a position of the retrieved speaker, and performing beamforming of a microphone depending on the position of the speaker.

[0007] However, since microphone beamforming in conventional technologies is controlled only by considering the current position information of a speaker, there is a limitation in controlling microphone beamforming by considering a driver's seat height, a change in seat position, a change in the posture of the driver or passenger, and whether a passenger is riding along. Therefore, there is a limitation in actively adjusting a microphone beamforming area directed toward the speaker's mouth according to a variable situation of the driver or passenger.

[0008] Therefore, according to conventional technologies, the ability to decompose sounds of a driver or passenger is impaired, so there are limitations in improving the speech recognition rate.

[0009] Furthermore, from JP 2007-302 155 A, an active control system of a vehicle dual microphone is known, the system comprising: a mouth detection unit installed in a vehicle and configured to capture an image of a driver sitting in a driver's seat of the vehicle and / or a passenger sitting in a passenger seat; a user and seat position detection unit configured to capture driving state information of the driver and / or the passenger;an infotainment system configured to determine a microphone beamforming control mode according to image data and user and seat position information output from the mouth shape change detection unit and the user and seat position detection unit, to control microphone beamforming of the driver's seat according to the determined microphone beamforming control mode, and to selectively adjust a specific microphone gain according to the determined microphone beamforming control mode to separately control the microphone beamforming of the driver's seat and the front passenger seat;and a microphone beamforming controller configured to adjust an installation angle of a first microphone installed adjacent to the driver's seat to be directed toward the driver, or installation angles of the first microphone and a second microphone installed adjacent to the driver's seat and the front passenger's seat to be directed toward the driver and the front passenger, respectively, according to the control of the infotainment system;

[0010] US 9,747,917 B2 discloses methods and systems for receiving desired sounds. The system includes a position sensor configured to determine the position of a speaking occupant within a defined space and transmit the position of the speaking occupant. A plurality of microphones are configured to receive sounds from the defined space and transmit audio signals corresponding to the received sound. A processor, in communication with the position sensor and the microphones, is configured to receive the position of the speaking occupant and the audio signals, apply a beamformer to the audio signals to direct a microphone beam toward the occupant's position, and generate a beamformer output signal.

[0011] US 2002 / 0 035 475 A1 describes a speech recognition device that can prevent erroneous manipulation due to erroneous speech recognition, even in a noisy environment. As long as a speech duration detected based on the level of a speech signal uttered by an operator (user) approximately matches a speech duration detected based on viseme data acquired by detecting the operator's mouth, the speech recognition device outputs speech manipulation phrase data as a result of the speech recognition.

[0012] US 9 571 922 B2 further discloses a device for controlling a beamforming microphone taking into account the position of a driver's seat, which device may comprise: a driver's seat sensor configured to detect the position of the driver's seat and output the detected position as a driver's seat signal; a direction control unit configured to calculate a beamforming angle based on the driver's seat signal; and a signal processing unit configured to process speech signals output by a first and a second microphone and to extract and output a speech signal generated from a direction that matches the beamforming angle. OVERVIEW OF REVELATION

[0013] Accordingly, the present disclosure is directed to an active control system of a vehicle dual microphone and a control method thereof that substantially obviates one or more problems due to limitations and disadvantages of the related art.

[0014] An object of the present disclosure is to provide an active control system of a vehicle dual microphone and a control method thereof for controlling a beamforming portion of a microphone installed adjacent to each of a driver's seat and a front passenger's seat to be directed toward a person (a speaker) uttering a voice command, using seat height information and seat position information of the driver or the front passenger, and front passenger's seat driving information.

[0015] Another object of the present disclosure is to provide an active control system for a vehicle dual microphone and a control method thereof for controlling the beamforming of two microphones installed respectively on the driver's seat and the passenger's seat to be directed toward the driver when only the driver is traveling in the vehicle.

[0016] Another object of the present disclosure is to provide an active control system for a vehicle dual microphone and a control method therefor to control beamforming to maintain a gain of a microphone installed adjacent to a speaker at an initial baseline setting and to relatively reduce a gain of a microphone positioned away from the speaker when both the driver and the passenger are traveling in the vehicle.

[0017] The objects are achieved by an active control system for a vehicle dual microphone having the features of claim 1 and an active control method having the features of claim 7. Advantageous further developments can be found in the subclaims.

[0018] Additional advantages, objects, and features of the disclosure will be set forth in part in the description which follows, and in part will become apparent to those of ordinary skill in the art upon examination of the following points, or may be learned from practice of the disclosure. The objects and other advantages of the disclosure may be realized and attained by the structure particularly pointed out in the written description and claims hereto and in the accompanying drawings.

[0019] To achieve these objects and other advantages, an active control system of a vehicle dual microphone formed by a first microphone and a second microphone, as embodied and generally described herein, includes a mouth shape change detection unit installed in a vehicle and configured to capture an image of a driver sitting in a driver's seat of the vehicle and / or a passenger sitting in a passenger seat and detect a change in the mouth shape of the driver and / or the passenger. The control system further includes a user and seat position detection unit configured to detect driving state information of the driver and / or the passenger.The control system also includes an infotainment system configured to determine a microphone beamforming control mode according to the captured image(s) and the driving state information output from the mouth change detection unit and the user and seat position detection unit. The infotainment system is further configured to control the microphone beamforming of the first microphone installed adjacent to the driver's seat according to the determined microphone beamforming control mode. The infotainment system is also configured to selectively adjust a specific microphone gain in the specific microphone beamforming control mode to separately control the microphone beamforming of the first microphone and the second microphone installed adjacent to the passenger seat.The control system also includes a microphone beamforming controller configured to adjust an installation angle of a first microphone installed adjacent to the driver seat to be directed toward the driver, or adjust installation angles of first and second microphones installed adjacent to the driver seat and the front passenger seat, respectively, to be directed toward the driver and the front passenger, according to the control of the infotainment system.The user and seat position detection unit comprises a first database configured to store and manage driving state information, which selectively includes seat height information of the driver or the front passenger, seat position information of the driver's seat or the front passenger seat of the vehicle, and front passenger driving information of the front passenger seat of the vehicle, using a database method according to the control of the infotainment system; a first controller configured to store or read the user and seat information stored in the first database according to the control of the infotainment system; and a second network communication unit configured to transmit the driving state information to the infotainment system according to the control of the first controller.

[0020] In another aspect of the present disclosure, an active control method of a first and second microphone forming a vehicle dual microphone includes capturing an image of a driver sitting in a driver's seat of a vehicle and / or capturing an image of a passenger sitting in a passenger seat, and detecting a change in the mouth shape of the driver and / or the passenger. The method further includes capturing driving state information of the driver and / or the passenger.The method also includes determining a microphone beamforming control mode according to the captured image or images obtained by detecting the change in the mouth shape of the driver and / or the passenger and the driving state information, controlling the microphone beamforming of the first microphone installed adjacent to the first driver seat and the second microphone installed adjacent to the second driver seat according to the determined microphone beamforming control mode, and selectively adjusting a specific microphone gain according to the determined microphone beamforming control mode to separately control the microphone beamforming of the first and second microphones.The method further includes adjusting an installation angle of a first microphone installed adjacent to the driver's seat to be directed toward the driver, or adjusting an installation angle of first and second microphones installed adjacent to the driver's seat and the front passenger's seat to be directed toward the driver and the front passenger, respectively.The driving state information is obtained by: a first database configured to store and manage user and seat information, which selectively includes seat height information of the driver or the front passenger seat, seat position information of the driver's seat or the front passenger seat of the vehicle, and passenger information of the front passenger seat of the vehicle using a database method according to the control of the infotainment system; a first controller configured to store or read the user and seat information stored in the first database according to the control of the infotainment system; and a second network communication unit configured to transmit the driving state information to the infotainment system according to the control of the first controller. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are included to facilitate understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate the embodiment(s) of the disclosure and, together with the description, serve to explain the principles of the disclosure. In the drawings, the following is shown: Fig. 1 is a block diagram of an active control system of a vehicle dual microphone according to an embodiment of the present disclosure; and Fig. 2 and Fig. 3 are flowcharts of an active control method of a vehicle dual microphone according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF REVELATION

[0022] Embodiments of the present disclosure will be described in detail for those of ordinary skill in the art to easily implement the disclosure with reference to the accompanying drawings. However, the present disclosure may be implemented in various forms and is not limited to these embodiments. To clearly describe the present disclosure, parts unrelated to the description are omitted from the drawings, and like reference numerals throughout the description indicate similar elements.

[0023] Throughout the description, one of ordinary skill in the art would understand that the terms "include," "comprising," and "having" are to be interpreted by default as inclusive or open, rather than exclusive or closed, unless explicitly defined otherwise. Furthermore, terms disclosed in the description, such as "unit," "module," etc., refer to units for processing at least one function or operation, which may be implemented by hardware, software, or a combination thereof.

[0024] Throughout the specification, when a particular part "includes" a particular component, this means that the part may further include another component, rather than excluding another component, unless otherwise disclosed. The same reference numerals are used throughout the drawings to refer to the same parts.

[0025] Hereinafter, the configuration and control method of an active control system of a vehicle dual microphone according to an embodiment of the present disclosure will be described with reference to Fig. 1-3 described in detail.

[0026] Fig. 1 is a block diagram of an active control system of a vehicle dual microphone according to an embodiment of the present disclosure.

[0027] As in Fig. As shown in Figure 1, the vehicle dual microphone active control system according to an embodiment of the present disclosure may include a mouth shape change detection unit 100 installed in a vehicle and capturing an image of a driver in a driver's seat of the vehicle and / or a passenger in a passenger seat positioned to one side of the driver's seat to detect a change in the mouth shape of the driver and / or the passenger. The control system may further include a user and seat position detection unit 200 for detecting driving state information (e.g., information about the seat height of the driver and / or the passenger, seat position information of the driver's seat and / or the passenger seat of the vehicle, and information acquired by selectively combining at least one of the pieces of the driving state information of the passenger seat of the vehicle) of the driver and / or the passenger.The control system may also include an infotainment system 300 that determines a microphone beamforming control mode according to the image data and the user and seat position information output by the mouth shape change detection unit 100 and the user and seat position detection unit 200. The infotainment system 300 may further control the microphone beamforming of the driver's seat according to the determined microphone beamforming control mode. The infotainment system 300 may also selectively adjust a specific microphone gain according to the determined microphone beamforming control mode to separately control the microphone beamforming of the driver's seat and the passenger seat.The control system may further include a microphone beamforming controller 400 that adjusts an installation angle of a first microphone 510 installed adjacent to the driver's seat to face the driver, or the installation angles of the first and second microphones 510 and 520 installed adjacent to the driver's seat and the front passenger's seat, respectively, to face the driver and the front passenger, according to the control of the infotainment system 300. The control system may also include a microphone unit 500 that includes the first and second microphones 510 and 520 installed adjacent to the driver's seat of the vehicle and the front passenger's seat, each disposed on a side surface of the driver's seat, to receive a user's voice command.The installation angles of the first and second microphones 510 and 520 are adjusted to control the microphone beamforming range according to the control of the microphone beamforming controller 400.

[0028] In this embodiment, the mouth change detection unit 100 may include at least one camera 110 installed in the vehicle and capturing an image of a driver in the driver's seat of the vehicle or a passenger in the passenger seat positioned to one side of the driver's seat according to the control of the infotainment system 300. The detection unit 100 may further include a first network communication unit 120 for transmitting data of the image captured by the camera 110 to the outside.

[0029] The user and seat position detection unit 200 may include a first database 210 for storing and managing user and seat information using a database mode under the control of the infotainment system 300. The user and seat information selectively includes seat height information of the driver or the front passenger, seat position information of the driver's seat or the front passenger's seat of the vehicle, and passenger travel information of the front passenger's seat of the vehicle. The detection unit 200 may further include a first controller 220 for storing and reading the user and seat information stored in the first database 210 under the control of the infotainment system 300. The detection unit 210 may also include a second network communication unit 230 for transmitting the user and seat information to the outside under the control of the first controller 220.

[0030] The infotainment system 300 may include a third network communication unit 310 for data communication with the mouth shape change detection unit 100, the user and seat position detection unit 200, or the microphone beamforming controller 400. The infotainment system 300 may further include a second controller 320 that determines a microphone beamforming control mode based on the user's driving information and seat information when a voice command for voice recognition is uttered in the vehicle, and outputs a specific microphone beamforming control signal. The infotainment system 300 may also include a display unit 330 for displaying the image data and / or driving information according to the control of the second controller 320.The infotainment system 300 may further include a second database 340 that stores and manages the beamforming control signal using a database method based on a predicted driver mouth position value obtained by considering the user and seat information. The second database 340 also reads the signal according to the control of the second controller 320.

[0031] In this embodiment, the second controller may include a beamforming processor 322 that determines a microphone beamforming control mode based on the driving information included in the user and seat information. The beamforming processor 322 also outputs the beamforming control signal to align an installation angle of the first microphone 510, installed adjacent to the driver's seat, toward the driver or to align the installation angles of the first and second microphones 510 and 520, installed adjacent to the driver's seat and the front passenger's seat, toward the driver and the front passenger, according to the determined microphone beamforming control mode.The second controller 320 may further include an image data processor 324 that processes image data transmitted from the mouth shape change detection unit 100 and estimates current coordinates of the mouth using image data obtained by detecting a change in the mouth position and mouth shape of the driver or the front passenger riding in the driver's seat or the front passenger's seat. The second controller 320 may also include a seat position information processor 326 that receives the user and seat position information transmitted from the user and the seat position detection unit 200. The seat position information processor 326 also separately detects the seat height information of the driver or the front passenger, the seat position information of the driver's seat or the front passenger's seat of the vehicle, and the passenger travel information of the front passenger's seat of the vehicle.

[0032] In this embodiment, in response to the input of the user voice command, the infotainment system 300 may search and acquire search information corresponding to a user voice command and output the corresponding information as text information or audio information including the text information through the display unit 330 or a speaker (not shown).

[0033] The microphone beamforming controller 400 may include a fourth network communication unit 410 that receives the microphone beamforming control signal output from the third network communication unit 310 of the infotainment system 300. The microphone beamforming controller 400 may further include a first microphone angle adjustment unit 420 and a second microphone angle adjustment unit 430 that outputs a microphone installation angle adjustment signal for adjusting a microphone beamforming area of ​​the driver's seat or the passenger's seat according to the microphone beamforming control signal output from the fourth network communication unit 410.The microphone beamforming controller 400 may also include a first servo motor 440 and a second servo motor 450 installed on the driver's seat of the vehicle and the passenger's seat on the side surface of the driver's seat, respectively. These servo motors adjust the installation angle of the microphone unit 500, which includes at least the first or second microphone 510 or 520, in one or more arbitrary directions, according to the microphone installation angle adjustment signal, by inputting the user's voice command thereto.

[0034] According to one embodiment of the present disclosure, the first and second microphones 510 and 520 may be installed in front of the driver's seat and the passenger seat, or on a side surface or a top surface thereof. The first and second microphones 510 and 520 may be installed at a distance between them in the range of 6.8-13 cm (2.67-5.11 inches).

[0035] Hereinafter, an active control method of a vehicle dual microphone according to an embodiment of the present disclosure will be described with reference to Fig. 2 and Fig. 3 described.

[0036] First, as in Fig. 2, when the vehicle is turned on, the infotainment system 300 can be automatically started, and the user can manually press a PTT (Push to Track) button or touch a voice recognition icon displayed on the display unit 330 of the infotainment system 300 to set a voice recognition mode (S11).

[0037] Then, when the infotainment system 300 makes a request to the mouth shape change detection unit 100 for the driver's mouth position change information to control the microphone beamforming of the driver's seat (S13), the mouth shape change detection unit 100 may transmit image data of change information of the driver's mouth shape to the infotainment system 300 with the camera 110 (S15).

[0038] When the infotainment system 300 makes a request to the user and seat position detection unit 200 for the driving state information (e.g., information about the seat height of the driver and / or the front passenger, seat position information of the driver's seat and / or the front passenger seat of the vehicle, and information acquired by selectively combining at least part of the passenger information of the front passenger seat of the vehicle) of the driver and / or the front passenger inputted by the driver or the front passenger in advance (S17), the user and seat position detection unit 200 may read the driver's or the front passenger's seat height inputted by the driver or the front passenger, which is stored in the first database 210, and seat information including current seat position information of the driver's seat or the front passenger seat and front passenger seat driving information.The detection unit 200 can further transmit the read information to the infotainment system 300 (S19).

[0039] The infotainment system 300 may then receive the user and seat information. The infotainment system 300 may further determine whether only the driver is driving the vehicle or both the driver and the front passenger are driving the vehicle, using the front passenger seat driving information included in the user and seat information. The infotainment system 300 may further determine a microphone beamforming control mode according to the determination result (S21).

[0040] In this case, as a determination result of operation S21, when it is determined that only the driver is driving in the vehicle, an active single microphone control mode can be set (S23).

[0041] Subsequently, the infotainment system 300 may estimate predicted driver mouth position coordinates based on image data included in the current driver seat height information and driver seat position information (S25).

[0042] Subsequently, the infotainment system 300 may access the second database 340, which previously sets and stores a microphone beamforming position setting value corresponding to the estimated mouth position of the driver, may read a corresponding position setting value for controlling the microphone beamforming, and transmits the corresponding position setting value to the microphone beamforming controller 400 (S27).

[0043] Accordingly, the microphone beamforming controller 400 may control the beamforming range using the microphone beamforming position setting value corresponding to the driver's mouth position of the first microphone 510 installed adjacent to the driver's seat (S29).

[0044] In other words, the microphone beamforming controller 400 can separately control the drive of the first and second servo motors 440 and 450 of the microphone beamforming controller 400 to control a beamforming area of ​​each microphone based on the initial microphone beamforming position adjustment value of the first and second microphones 510 and 520 transmitted from the infotainment system 300. The controller 400 can adjust the corresponding microphone beamforming area directed toward the driver in any one or more directions, including an upward direction, a downward direction, a left direction, and a right direction.

[0045] In this case, after the microphone beamforming range of the first microphone 510 is completely adjusted, the microphone beamforming controller 400 may generate a microphone beamforming adjustment completion message and send the message to the infotainment system 300 (S31).

[0046] The infotainment system 300 may determine whether a current state is a state in which in-vehicle voice recognition is being performed (S33). As a determination result of operation S33, if voice recognition is currently being performed, the infotainment system 300 may generate a passenger image transmission request message requesting real-time transmission of image data obtained by photographing the driver and the passenger in their respective seats (driver's seat and front passenger's seat) to the in-vehicle camera 110. The infotainment system 300 may also transmit the message to the mouth shape change detection unit 100.

[0047] Accordingly, the mouth shape change detection unit 100, which receives the corresponding passenger image transmission request message, can control the camera 110 to generate the image data formed by photographing the driver and the passenger sitting on their respective seats, and can transmit the image data to the infotainment system 300 in real time (S37).

[0048] Accordingly, the infotainment system 300 may estimate a position of the driver's or passenger's mouth from the image data received in real time (S39) and may determine whether the position coordinates are changed compared to an estimated initial driver's mouth position (S41).

[0049] As a result of operation S41, if it is determined that coordinates of the estimated current mouth position have changed, the second controller 320 of the infotainment system 300 may calculate a variation value (upper, lower, left, and right delta values) based on an estimated previous mouth position (S43). The second controller 320 may then store the corresponding variation value in the second database 340 and also transmit the variation value to the microphone beamforming controller 400 (S45).

[0050] Accordingly, the microphone beamforming controller 400 may receive the corresponding variation value and adjust the microphone beamforming ranges of the first microphone 510 installed adjacent to the driver's seat and the second microphone 520 installed adjacent to the passenger seat to be directed toward the current mouth position of the driver (S47).

[0051] In other words, the microphone beamforming controller 400 can control the driving of the first and second servo motors 440 and 450 to align the microphone beamforming areas of the first and second microphones 510 and 520, through which information is transmitted from the infotainment system 300, with the current mouth position of the driver. The microphone beamforming controller 400 can adjust the corresponding microphone beamforming area in one or more arbitrary directions, including up, down, left, and right.

[0052] In this case, when the microphone beamforming ranges of the corresponding first and second microphones 510 and 520 are completely adjusted, the microphone beamforming controller 400 may generate a microphone beamforming adjustment completion message and transmit the message to the infotainment system 300 (S49).

[0053] Then, the infotainment system 300 may remove noise from a voice input to the first and second microphones 510 and 520 and process a driver's voice command (S51). Subsequently, the active control method of the vehicle dual microphone according to the present disclosure may be completed.

[0054] However, as a determination result of operation S17, after determining that both the driver and the passenger are traveling in the vehicle, the infotainment system 300 may set the microphone beamforming control mode to an active dual microphone control mode (S22) as shown in Fig. 3 shown.

[0055] Subsequently, the infotainment system 300 may make a request to the driver's mouth position change detection unit 100 to detect driver's mouth shape change information to control the microphone beamforming of the driver's seat (S24).

[0056] Accordingly, the mouth change detection unit 100 may transmit image data including change information of the driver's mouth shape to the infotainment system 300 using the camera 110 (S26).

[0057] In this case, the infotainment system 300 may estimate the predicted driver's mouth position coordinates based on image data included in the current driver's seat height information and the current driver's seat position information (S28).

[0058] The infotainment system 300 can then access the second database 340, which previously sets and stores the microphone beamforming position setting value corresponding to the estimated current mouth position of the driver. The infotainment system 300 can also read the microphone beamforming position setting value corresponding to the current mouth position to control the microphone beamforming. The infotainment system 300 can further transmit the microphone beamforming position setting value to the microphone beamforming controller 400 (S30).

[0059] Accordingly, the microphone beamforming controller 400 may control the beamforming range of the first microphone 510 installed adjacent to the driver's seat with an initial positioning value (S32).

[0060] In other words, the microphone beamforming controller 400 can control the drive of the first or second servo motor 440 or 450 of the microphone beamforming controller 400. The microphone beamforming controller 400 can also adjust the corresponding microphone beamforming range in any one or more directions, including an upward direction, a downward direction, a left direction, and a right direction, to be directed toward the driver and the front passenger, to control a beamforming range of each microphone based on the adjustment value information of the microphone beamforming position of the first or second microphone 510 or 520 transmitted from the infotainment system 300.

[0061] In this case, when the microphone beamforming range of the corresponding first or second microphone 510 or 520 is completely adjusted, the microphone beamforming controller 400 may generate a microphone beamforming adjustment completion message and transmit the message to the infotainment system 300 (S34).

[0062] The infotainment system 300 may determine whether a current state is a state in which voice recognition is being performed in the vehicle (S36). As a determination result of operation S36, if voice recognition is currently being performed, the infotainment system 300 may generate a passenger image transmission request message requesting real-time transmission of image data obtained by photographing the driver and passenger sitting in their respective seats (driver's seat and front passenger's seat) to the in-vehicle camera 110. The infotainment system 300 may transmit the message to the mouth change detection unit 100 (S38).

[0063] Accordingly, the mouth shape change detection unit 100, which receives the corresponding passenger image transmission request message, can control the camera 110 to generate the image data formed by photographing the driver and the passenger on their respective seats, and can transmit the image data to the infotainment system 300 in real time (S40).

[0064] Accordingly, the infotainment system 300 may estimate a mouth shape of the driver or the front passenger from the image data received in real time and determine whether a first speaker is the driver or the front passenger (S42).

[0065] In this case, after determining that the driver is the first speaker, the infotainment system 300 may decrease a gain value of the second microphone 520 installed adjacent to the passenger seat by 6 dB or more based on a default setting value, while maintaining a gain value of the first microphone 510 installed adjacent to the driver seat at a default setting value (S44). The infotainment system 300 may increase the discrimination of a sound source introduced into the first microphone 510 installed adjacent to the driver seat.

[0066] In this case, the infotainment system 300 may reduce the gain of the second microphone 520, which is installed adjacent to the front passenger seat, to a level equal to or less than 25% of the default setting. The infotainment system 300 may attenuate a sound source signal introduced into the second microphone 520 to an ambient noise level to prevent a sound source from being introduced into the second microphone 520, which is installed next to the front passenger seat, if possible.

[0067] However, as a determination result of operation S42, after determining that the front passenger is the first speaker, the infotainment system 300 may reduce a gain value of the first microphone 510 installed adjacent to the driver's seat by 6 dB or more based on a default setting value, while maintaining a gain value of the second microphone 520 installed adjacent to the front passenger seat at a default setting value (S46). The infotainment system 300 may increase the discrimination of a sound source introduced into the second microphone 520 installed adjacent to the front passenger seat.

[0068] In this case, the infotainment system 300 may reduce the gain of the first microphone 510 installed adjacent to the driver's seat to a level equal to or less than 25% of the default setting. The infotainment system 300 may attenuate a sound source signal introduced into the first microphone 510 to an ambient noise level to prevent a sound source from being introduced into the first microphone 510 installed adjacent to the driver's seat, if possible.

[0069] Then, the infotainment system 300 may detect a sound pressure level (voltage level) of a voiced, i.e., spoken, sound source introduced into the first or second microphone 510 or 520, which is maintained at a base microphone gain setting for a preset period of time. The infotainment system 300 may determine whether a voiced, i.e., spoken, sound source is present (S48).

[0070] In this case, as a determination result, if there is no change in the sound pressure level, the infotainment system 300 may terminate a voice recognition processing process (S50). However, if a sound source input to a specific microphone is present (when a change in the sound pressure level is detected), the infotainment system 300 may process a voice command input to the corresponding microphone and then terminate the active control process of the in-vehicle dual microphone.

[0071] In the above-mentioned active control system of the in-vehicle dual microphone and a control method thereof according to the present disclosure, a beamforming region of a microphone installed adjacent to each of the driver's seat and the front passenger's seat can be controlled to be directed toward a person (a speaker) transmitting a voice command using seat height information and seat position information of a driver or a front passenger, and driving information of the front passenger's seat. This can advantageously increase a driver's voice recognition rate.

[0072] When only one driver is in a vehicle, the beamforming of two microphones, installed at the driver's seat and the passenger's seat, can be controlled to point toward the driver. When both the driver and passenger are in the vehicle, the beamforming can also be controlled to maintain the gain of a microphone installed adjacent to a speaker at an initial setting and relatively decrease the gain of a microphone located away from the speaker. This can advantageously improve the ability to separate the driver's or passenger's voice and noise.

[0073] In the above-configured active control system of the in-vehicle dual microphone and a control method thereof according to the present disclosure, a beamforming area of ​​a microphone installed adjacent to each of the driver's seat and the front passenger's seat can be controlled to be directed toward a person (speaker) transmitting a voice command using seat height information and seat position information of a driver or a front passenger, and driving information of the front passenger's seat. This can increase a driver's voice recognition rate.

[0074] When only one driver is in a vehicle, the beamforming of two microphones, one installed in the driver's seat and the other in the front passenger seat, can be controlled to point toward the driver. When both the driver and the front passenger are in the vehicle, the beamforming can also be controlled to maintain the gain of a microphone installed adjacent to a speaker at a baseline setting and relatively decrease the gain of a microphone located away from the speaker. This can improve the ability to separate the driver's or passenger's voice and noise.

[0075] Those of ordinary skill in the art will recognize that the effects that may be achieved with the present disclosure are not limited to what is specifically described above, and other advantages of the present disclosure will be better understood from the detailed description.

[0076] The above disclosure may also be implemented as computer-readable code on a computer-readable recording medium. A computer-readable recording medium is any data storage device capable of storing data that can subsequently be read by a computer. Examples of computer-readable recording media include a hard disk drive (HDD), a solid-state drive (SSD), a silicon disk drive (SDD), a read-only memory (ROM), a random access memory (RAM), CD-ROM, magnetic tapes, floppy disks, optical data storage devices, etc.

Claims

[1] Active control system of a vehicle dual microphone formed by a first microphone (510) and a second microphone (520), the control system comprising: a mouth shape change detection unit (100) installed in a vehicle and configured to capture an image of a driver sitting in a driver's seat of the vehicle and / or a passenger sitting in a passenger's seat and detect a change in the mouth shape of the driver and / or the passenger; a user and seat position detection unit (200) configured to detect driving state information of the driver and / or the passenger; an infotainment system (300) configured to determine a microphone beamforming control mode according to the captured image or images and the driving state information output by the mouth shape change detection unit (100) and the user and seat position detection unit (200), to control a microphone beamforming of the first microphone (510) installed adjacent to the driver's seat according to the determined microphone beamforming control mode, and to selectively adjust a specific microphone gain according to the determined microphone beamforming control mode to separately control the microphone beamforming of the first microphone (510) and the second microphone (520) installed adjacent to the passenger seat; and a microphone beamforming controller (400) configured to adjust an installation angle of the first microphone (510) installed adjacent to the driver's seat to be directed toward the driver, or installation angles of the first microphone (510) and the second microphone (520) installed adjacent to the driver's seat and the passenger's seat to be directed toward the driver and the passenger, respectively, according to the control of the infotainment system (300), wherein the user and seat position detection unit (200) includes: a first database (210) configured to store and manage the driving state information including selective seat height information of the driver or the front passenger, seat position information of the driver's seat or the front passenger's seat of the vehicle using a database method according to the control of the infotainment system (300); a first controller (220) configured to store or read the driving state information stored in the first database (210) according to the control of the infotainment system (300); and a second network communication unit (230) configured to transmit the driving state information to the infotainment system (300) according to the control of the first controller (220). [2] The control system according to claim 1, wherein the mouth shape change detecting unit (100) includes: at least one camera (110) installed in the vehicle and configured to capture the image of a driver sitting in the driver's seat of the vehicle or the image of the passenger sitting in the passenger seat positioned on one side of the driver's seat, according to the control of the infotainment system (300); and a first network communication unit (120) configured to transmit data of the image captured by the camera (110) to the infotainment system (300). [3] The control system according to claim 1, wherein the driving state information includes information about a seat height of the driver and / or the front passenger, seat position information of the driver's seat and / or the front passenger's seat. [4] The control system of claim 1, wherein the infotainment system (300) includes: a third network communication unit (310) configured to communicate data-wise with the mouth shape detection unit (100), the user and seat position detection unit (200), or the microphone beamforming controller (400); a second controller (320) configured to determine a microphone beamforming control mode based on the driving condition information and the captured image or images when a voice command for voice recognition is pronounced in the vehicle, and to output a specific beamforming control signal; a display unit (330) configured to display the image data and / or driving information according to the control of the second controller (320); and a second database (340) configured to store and manage the beamforming control signal using a database method based on a predicted driver mouth position value obtained by taking into account the driving state information and the captured image or images, and to read the signal according to the control of the second controller (320). [5] The control system of claim 4, wherein the second controller (320) includes: a beamforming processor (322) configured to determine a microphone beamforming control mode based on the driving state information and the captured image or images, and output the beamforming control signal corresponding to the determined microphone beamforming control mode to align the installation angle of the first microphone (510) installed adjacent to the driver's seat toward the driver, or output the beamforming control signal to align the installation angles of the first (510) and second microphones (520) installed adjacent to the driver's seat and the front passenger's seat toward the driver and the front passenger; an image data processor (324) configured to process the captured image or images transmitted from the mouth shape change detection unit (100) and to detect a change in a mouth position and a mouth shape of the driver or the passenger riding in the driver's seat or the passenger's seat; and a seat position information processor (326) configured to receive the driving state information transmitted from the user and seat position detection unit (200) and to separately detect seat height information of the driver and / or the front passenger, seat position information of the driver's seat and / or the front passenger seat of the vehicle. [6] The control system of claim 1, wherein the microphone beamforming controller (400) includes: a fourth network communication unit (410) configured to receive the beamforming control signal output from the infotainment system (300); first (420) and second microphone angle adjustment units (430) configured to output a microphone installation angle adjustment signal for adjusting a microphone beamforming area of ​​the driver's seat and / or the passenger's seat according to the beamforming control signal output by the fourth network communication unit (410); and first (440) and second servo motors (450) installed on the driver's seat of the vehicle and on the passenger's seat of the vehicle and configured to adjust the installation angle of the first (510) or second microphone (520) in any one or more of an upward direction, a downward direction, a left direction, and a right direction according to the microphone installation angle adjustment signal. [7] An active control method for a first (510) and second microphone (520) forming a vehicle dual microphone, the control method comprising: Capturing an image of a driver sitting in a driver's seat of a vehicle and / or capturing an image of a passenger sitting in a passenger seat and detecting a change in the mouth shape of the driver and / or the passenger; Recording driving status information of the driver and / or the passenger; Determining a microphone beamforming control mode according to the captured image or images obtained by detecting the change in the mouth shape of the driver and / or the passenger and the driving state information, controlling the microphone beamforming of the first microphone (510) installed adjacent to the first driver's seat and the second microphone (520) installed adjacent to the second driver's seat according to the determined microphone beamforming control mode, and selectively setting a specific microphone gain according to the determined microphone beamforming control mode for separately controlling the microphone beamforming of the first (510) and the second microphone (520); and Adjusting an installation angle of the first microphone (510) installed adjacent to the driver's seat to be directed toward the driver, and / or adjusting an installation angle of the first (510) and second microphones (520) installed adjacent to the driver's seat and the passenger's seat to be directed toward the driver and the passenger, respectively, where the driving condition information is obtained by: a first database (210) configured to store and manage user and seat information, selectively including seat height information of the driver or the front passenger seat, seat position information of the driver's seat or the front passenger seat of the vehicle, and passenger information of the front passenger seat of the vehicle using a database method according to the control of the infotainment system (300); a first controller (220) configured to store or read the user and seat information stored in the first database (210) according to the control of the infotainment system (300); and a second network communication unit (230) configured to transmit the driving state information to the infotainment system (300) according to the control of the first controller (220). [8] A control method according to claim 7, further comprising: Determining the microphone beamforming control mode based on the driving condition information and the captured image or images; and when only the driver is driving in the vehicle, outputting a beamforming control signal for directing an installation angle of the first microphone (510) installed adjacent to the driver's seat toward the driver; or when both the driver and the front passenger are traveling in the vehicle, outputting a beamforming control signal for aligning the installation angles of the first (510) and second microphones (520) installed adjacent to the driver's seat and the front passenger's seat, respectively, toward the driver and the front passenger. [9] A control method according to claim 8, further comprising: after determining that the driver is a first speaker, reducing a gain value of the second microphone (520) by a preset gain value or higher based on a default value while maintaining a gain value of the first microphone (510) at a default value; and after determining that the passenger is the first speaker, reducing a gain value of the first microphone (510) by a preset gain value or higher based on a default value while maintaining a gain value of the second microphone (520) at a default value.

Images