Hands-free control device for vehicle and hands-free control method for vehicle

Abstract

To improve convenience for an occupant carrying a mobile terminal in a vehicle.SOLUTION: A communication control unit 116 detects whether communication between a hands-free control device for a vehicle and a driver's mobile terminal is possible. An occupant detection unit 113 identifies an occupant boarding the vehicle and specifies a boarding position of the occupant based on an image captured by a camera 20 that images an interior of the vehicle. A determination unit 114 determines whether the mobile terminal 2 of the occupant identified by the occupant detection unit 113 is in a state connectable by short-range wireless communication. When a plurality of mobile terminals 2 are determined to be in a state connectable by short-range wireless communication by the determination unit 114, a decision unit 115 decides, based on boarding positions of a plurality of occupants corresponding to the plurality of mobile terminals 2, the mobile terminal 2 to be connected by short-range wireless communication. The communication control unit 116 switches to connect, by short-range wireless communication, the mobile terminal 2 decided by the decision unit 115 with the hands-free control device for a vehicle.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a hands-free control device for a vehicle that receives an image captured by a camera, and a hands-free control method for a vehicle. [Background technology]

[0002] While driving a car, the driver uses a mobile device (mainly a smartphone or feature phone) Holding a mobile phone in your hand and using it for calls, etc. is prohibited by law and ordinance. The mobile terminal and the hands-free device in the vehicle are connected via short-range wireless communication, allowing hands-free communication. Stories and manipulation methods are used.

[0003] Usually, hands-free devices in vehicles automatically connect to the most recently connected mobile device. Therefore, if a different driver than the previous driver is driving, However, automatic connection was not possible, and it was necessary to select the mobile device of the new driver. If a driver is driving without realizing it, the hands-free function cannot be used, which is inconvenient. .

[0004] In response to this, a table that associates the user's face information with the identification information of the mobile device is referred to. By identifying the driver's face captured by the driver's seat camera, the system establishes a connection with the mobile device. A method for automatically establishing this has been proposed (see, for example, Patent Document 1). In this case, it is not possible to capture the entire interior of the vehicle, including passengers other than the driver. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-042577 Summary of the Invention [Problem to be solved by the invention]

[0006] For example, if the driver temporarily gets off the vehicle, the connection destination of the vehicle's hands-free device will be It would be convenient to switch from the driver's mobile device to the passenger's mobile device. .

[0007] The present embodiment has been made in view of the above circumstances, and its purpose is to The present invention aims to provide a technology that improves convenience for passengers who carry portable terminals. [Means for solving the problem]

[0008] In order to solve the above problem, a certain aspect of the vehicle hands-free control device of this embodiment includes a communication control unit that detects whether communication is possible between the vehicle hands-free control device and a mobile terminal of a vehicle driver; a passenger detection unit that identifies passengers in the vehicle from video captured by a camera that captures the interior of the vehicle and specifies the passenger's seating position; a judgment unit that determines whether the passenger's mobile terminal identified by the passenger detection unit is in a state where it can be connected via short-range wireless communication; and, if there are multiple mobile terminals that the judgment unit has determined to be in a state where it can be connected via short-range wireless communication, a decision unit that determines which mobile terminal to connect via short-range wireless communication based on the seating positions of multiple passengers specified by multiple pieces of biometric information corresponding to the multiple mobile terminals.If communication with the vehicle driver's mobile terminal is not possible, the communication control unit switches the connection between the mobile terminal determined by the decision unit and this vehicle hands-free control device so that they can communicate via short-range wireless communication.

[0009] Another aspect of the present embodiment is a method executed by a vehicular hands-free control device. This method is executed by the vehicular hands-free control device, and includes the steps of: detecting whether communication between the vehicular hands-free control device and a mobile terminal of a vehicle driver is possible; identifying a vehicle occupant from an image captured by a camera that captures an interior of the vehicle and specifying the seating position of the occupant; determining whether the mobile terminal of the identified occupant is in a state where it can be connected via short-range wireless communication; if there are multiple mobile terminals determined to be in a state where it can be connected via short-range wireless communication, determining a mobile terminal to connect via short-range wireless communication based on the seating positions of multiple occupants specified by multiple pieces of biometric information corresponding to the multiple mobile terminals; and, if communication with the vehicle driver's mobile terminal is not possible, switching the connection between the determined mobile terminal and the vehicular hands-free control device so that the short-range wireless communication is performed between the determined mobile terminal and the vehicular hands-free control device.

[0010] Any combination of the above components and the expression of the present embodiment may be used as an apparatus, a method, a system, a recording medium, or the like. The conversion between the recording medium, computer program, etc. is also included as an aspect of this embodiment. It is valid. [Effects of the Invention]

[0011] According to this embodiment, the convenience of passengers carrying mobile terminals in a vehicle is improved. It can be done. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of camera installation in a vehicle. [Figure 2] 1 is a diagram for explaining a configuration example of a hands-free control system for a vehicle according to an embodiment; [Figure 3] FIG. 2 is a diagram illustrating an example of a user list held in a user list holding unit. [Figure 4] FIG. 10 is a diagram illustrating an example of detection of passengers and their positions in a vehicle. [Figure 5] 4 is a flowchart showing a flow of a face authentication process of the hands-free control device for a vehicle according to the first embodiment. [Figure 6] 10 is a flowchart showing a flow of a face authentication process of the hands-free control device for a vehicle according to the second embodiment. [Figure 7] 10 is a flowchart showing a flow of a face authentication process of a hands-free control device for a vehicle according to a modified example of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] This embodiment is a system for realizing a hands-free function of a mobile terminal carried by a vehicle passenger. The present invention relates to a hands-free control system for a vehicle. The system uses a camera that captures a wide range of images inside the vehicle, including not only the driver's seat but also the passenger seat and rear seats. Use Mela.

[0014] FIG. 1 is a diagram showing an example of installation of a camera 20 in a vehicle C1. The camera 20 is attached to the rearview mirror. Repurpose the camera used in the In-Cabin Monitoring System Alternatively, a camera dedicated to the hands-free control system may be provided.

[0015] FIG. 2 is a diagram illustrating an example of the configuration of a hands-free control system 1 for a vehicle according to an embodiment. The vehicle hands-free control system 1 includes a vehicle hands-free control device 10, It is equipped with a camera 20, a microphone 30, a speaker 40, a monitor 50 and an operation unit 60.

[0016] The camera 20 captures images of the interior of the vehicle and outputs the captured images to the vehicle hands-free control device 10. The camera 20 may be a visible light camera or an infrared camera. Alternatively, the camera may be capable of capturing images in both the visible light range and the infrared range.

[0017] The camera 20 includes a lens, an image sensor, and a signal processing circuit. OS (Complementary Metal Oxide Semiconductor) image sensor or CCD (Charge e) Coupled Device (ICD) image sensors can be used. The image sensor is a The infrared light is converted into an electrical video signal and output to the signal processing circuit. When taking a photograph, an image sensor having sensitivity in the infrared region is used. The video signal input from the The signal is output to the hands-free control device 10.

[0018] The vehicle hands-free control device 10 includes a processing unit 11, a storage unit 12, a wireless communication unit 13, and a vehicle control unit. The vehicle hands-free control device 10 may be mounted in a dedicated housing. Car navigation systems, display audio, drive recorders, may be incorporated as a function of the in-vehicle monitoring system.

[0019] The processing unit 11 includes an image recognition unit 111, a registration unit 112, a passenger detection unit 113, and a determination unit 114. , a determination unit 115, a communication control unit 116, and a function control unit 117. The functions of the processing unit 11 are as follows: It can be realized by the cooperation of hardware and software resources, or by hardware resources alone. The hardware resources are CPU, ROM, RAM, and GPU (Graphics Processing Unit). Unit), DSP (Digital Signal Processor), ISP (Image Signal Processor), AS IC (Application Specific Integrated Circuit), FPGA (Field-Programmable Ga te Array) and other LSIs can be used. The program is available.

[0020] The storage unit 12 is a nonvolatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The user list holding unit 121 is configured to include a recording medium having a user list. 121 transmits facial recognition data of the vehicle occupant and information about the mobile terminal 2 carried by the occupant. The user list is limited to people who have a driver's license. It is possible for people without a driver's license to register.

[0021] FIG. 3 is a diagram showing an example of a user list held in the user list holding unit 121. As shown in FIG. The user list shown in Figure 3 links face authentication data and information about mobile device 2 for each registered user. The information about the mobile terminal 2 is registered as the name of the mobile terminal 2 and the connection status. The connection status and AV source are registered. Whether music is automatically played, whether video is automatically played, and whether data communication is used or not are registered. The use / non-use of data communication is determined by the vehicular hands-free control device 10 controlling the mobile terminal 2. It specifies whether or not data communication will be performed using a mobile phone network (4G / 5G) through the service. For example, data communication can be used to stream music and video.

[0022] The AV source is the music data or video data when music is played or video is played. For example, the AV source may be a music file stored in the mobile terminal 2. You can also use the car navigation system and the Use the radio and TV broadcasts received by the tuner built into the Spray Audio. It can also be inserted into car navigation systems and display audio systems. You can use audio and video files recorded on a CD / DVD.

[0023] In addition, each user can register an in-car environment other than the AV environment. For example, Or you can register the seat position when sitting in the passenger seat, the air conditioning status, etc. The air conditioner status can be registered as cooling, heating, fan switching, set temperature, etc. Furthermore, the position of the side mirrors can also be registered as part of the in-car environment.

[0024] Returning to FIG. 2, the image recognition unit 111 recognizes the image in the frame image of the video input from the camera 20. The face detection classifier is used to search for the face of a person. It uses dictionary data generated by learning from a large number of images containing faces. For example, Haar-like features, HOG (Histogram of Gradients) features, LBP (Local Binary Pattern Recognition) features, ry Patterns) features, etc. can be used.

[0025] The registration unit 112 registers the passenger's face authentication data and information about the portable terminal 2 in a user list. Passengers who wish to register on the user list should, while sitting in the front seat, press the operation panel 6. 0 to activate the registration mode. When the registration mode is activated, the registration unit 112 performs image recognition. The recognition unit 111 extracts the feature amount of the face area of the passenger detected in the frame image and identifies the passenger. The system then instructs the system to generate facial recognition data to identify the user.

[0026] When the registration unit 112 acquires the face authentication data of the passenger from the image recognition unit 111, The control unit 116 is caused to start a pairing process with the mobile terminal 2 carried by the passenger. The dual-use hands-free control device 10 and the mobile terminal 2 can be connected by short-range wireless communication. For short-range wireless communication, Bluetooth (registered trademark) and Wi-Fi (registered trademark) can be used. In the following, in this embodiment, an example in which Bluetooth is used is assumed.

[0027] The registration unit 112 receives the profile of the portable terminal 2 from at least one of the speaker 40 and the monitor 50. First, the registration unit 112 guides the user through the registration procedure. The registration unit 112 instructs the communication control unit 116 to perform the following procedure. , and transmits a signal (e.g., an advertisement packet) to notify the surroundings of its presence. When the mobile terminal 2 scans the signal, the vehicular hands-free control The control device 10 is displayed as a connectable device. When the vehicle hands-free control device 10 is selected, the mobile terminal 2 and the vehicle hands-free control device 1 In the pairing process, the encryption of the communication data between the two is performed. The encryption key used for encryption is exchanged.

[0028] When pairing with the portable terminal 2 is completed, the registration unit 112 performs the following via the communication control unit 116: The name and identification information of the mobile terminal 2 are acquired. The name of the mobile terminal 2 is not clear to the user. For example, it is preferable that the passenger has a name that is easy to understand when using various services in the mobile terminal 2. The user account name may be used. If the name entered is difficult to understand, the passenger may edit the name on the mobile terminal 2 using the operation unit 60. It can be collected.

[0029] The passenger inputs the above-mentioned connection status setting items from the operation unit 60. The user list storage unit associates the face authentication data with the name of the mobile device and the connection status. The passenger is registered in the user list in 121. In addition, the passenger can set the in-car environment settings as described above. You can also enter the information. In that case, each item in the car's environment will also be linked and registered.

[0030] The passenger detection unit 113 detects passengers in the vehicle C1 from the image input from the camera 20. Specifically, the passenger detection unit 113 recognizes the face of the passenger by the image recognition unit 111. The facial feature data of the passenger detected in the frame image and the multiple data registered in the user list are The system matches multiple facial recognition data sets to identify the passenger's face.

[0031] The passenger detection unit 113 detects the riding position of at least one passenger riding in the vehicle C1. Specifically, the passenger detection unit 113 identifies the seating position of each passenger, such as the driver's seat, the passenger seat, Identify whether it is the right rear seat, left rear seat, or rear center seat.

[0032] 4 is a diagram showing an example of detection of passengers and their positions in the vehicle C1. In the image F1, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat. When the image recognition unit 111 detects a face of an object, it recognizes that a passenger is present in the driver's seat. If a person's face is detected within the passenger seat detection area A2, a passenger is present in the passenger seat. Similarly, the image recognition unit 111 recognizes a person in the detection area A3 for passengers in the right rear seat. If the image recognition unit 111 detects the face of the passenger, it recognizes that there is a passenger in the right rear seat. If a person's face is detected within the detection area A4 for passengers in the left rear seat, it is determined that a passenger is present in the left rear seat. Although it does not exist in the vehicle shown in Figure 4, it is possible to recognize a vehicle with a rear center seat. In this case, a detection area for the rear center seat passenger is also set.

[0033] Returning to FIG. 2, the determination unit 114 determines whether or not the face of the passenger identified by the passenger detection unit 113 is associated with the face of the passenger. It is determined whether the mobile terminal 2 connected to the communication system is in a state where it can be connected by short-range wireless communication. If the control unit 116 recognizes the mobile terminal 2 as a connectable device, the determination unit 114 The system determines that the portable terminal 2 is in a connectable state. If the mobile device 2 is not connected, the mobile device 2 is turned off, or the Bluetooth function is disabled, If it is turned off, the determination unit 114 determines that the portable terminal 2 is not in a connectable state. .

[0034] If there are multiple portable terminals 2 that the determining unit 114 has determined to be in a connectable state (i.e. In the case where there are multiple passengers, the determination unit 115 determines whether the mobile terminals 2 are linked to the multiple mobile terminals 2. The mobile terminal that connects is selected based on the boarding locations of multiple passengers identified by multiple facial recognition data. Specifically, the determination unit 115 determines the seat number of the passenger in the seat in the order of priority. A mobile device 2 linked to the facial recognition data of the passenger sitting in the highest seat, Mobile terminal 2 is selected as the terminal to be connected.

[0035] For example, the seat priority is set as follows: driver's seat: 1, passenger seat: 2, right rear seat: 3, left rear seat: 4. In this example, when the driver gets off the vehicle, the mobile terminal 2 of the passenger sitting in the passenger seat is set. The priority of the seats is set by the user. For example, if a person with a high position sits in the right rear seat, the priority can be changed. :You can also set it to 1.

[0036] The wireless communication unit 13 includes an antenna and transmits and receives signals to and from the mobile terminal 2 using a short-distance wireless communication method. The in-vehicle communication unit 14 executes communication processing for connecting to the in-vehicle network 3. Network 3 is a CAN (Controller Area Network), a LIN (Local Interconnect Network), Ethernet (registered trademark), etc. can be used. Work 3 contains various ECUs (Electronic Control Units) for controlling the engine and auxiliary equipment. )4 is connected. For example, the ECU for controlling the seat position is also connected. The vehicle network 3 also includes a control unit for an air conditioning system in the vehicle C1. can also be connected.

[0037] In the example shown in FIG. 2, a receiver 5b of the wireless key system is connected to the in-vehicle network 3. The receiver 5b receives a wireless key (smart key, intelligent key) carried by the driver. a receiver for receiving radio signals from the remote control key (also called a remote control key, etc.) 5a It is a machine.

[0038] The communication control unit 116 controls the wireless communication unit 13 to perform pairing processing with the mobile terminal 2, pairing It also handles connection / disconnection processing with the paired mobile terminal 2 and manages paired devices. The communication control unit 116 controls the in-vehicle communication unit 14 to communicate with various devices connected to the in-vehicle network 3. Sends and receives control signals between ECU 4 and devices.

[0039] The function control unit 117 controls various functions of the vehicular hands-free control system 1 (for example, The function control unit 117 controls the functions of the mobile terminal (free call function, audio playback function, video playback function, etc.). It is also possible to control functions within the mobile terminal 2 connected via long-distance wireless communication.

[0040] The microphone 30 collects sounds inside the vehicle C1 and outputs the collected sounds to the function control unit 117. The microphone 30 can be used for hands-free calling. For example, one may be installed in each of the front and rear seats, or At least one microphone 30 may be installed in the car navigation system. systems, display audio, drive recorders, or in-vehicle monitoring systems It may be a repurposed microphone.

[0041] The speaker 40 reproduces and outputs the audio data supplied from the function control unit 117. In this embodiment, the speaker 40 can be used for hands-free calling. For example, one may be installed in the front seat and one in the rear seat. Alternatively, the speaker 40 may be installed for each seat. Gaming systems, display audio, car audio systems, drive records The speaker may be a speaker adapted from an in-vehicle monitoring system.

[0042] The monitor 50 reproduces and displays the video data supplied from the function control unit 117. The 50 uses LCD displays, OLED displays, mini LED displays, etc. A plurality of monitors 50 may be installed in the vehicle C1. For example, One may be installed in each seat and rear seat, or one may be installed in each seat. At least one monitor 50 is used for a car navigation system, a display audio It is a device that has been repurposed from a monitor of a driving recorder or an in-vehicle monitoring system. It is also possible.

[0043] Example 1 FIG. 5 is a flowchart showing a flow of a face authentication process of the hands-free control device 10 for a vehicle according to the first embodiment. The first embodiment is an example in which only the driver is in the vehicle C1. The processing unit 11 acquires an image of the interior of the vehicle captured by the camera 20 (S10). The recognition unit 111 recognizes the passenger who is in the driver's seat position from the frame image of the acquired video. Detect a face (S11).

[0044] The passenger detection unit 113 compares the facial feature data of the detected passenger with the facial feature data registered in the user list. The face recognition data stored in the user list is compared (S12). If the data does not exist (S13: No), the face recognition process ends. If there is face authentication data that matches the user list (S13: Yes), the determination unit 114 refers to the user list. The mobile device 2 associated with the matching face authentication data is identified, and the mobile device 2 The communication control unit 116 is requested to connect to the

[0045] When the communication control unit 116 receives the request, it checks whether there is a mobile terminal 2 currently connected. If there is a currently connected mobile terminal 2 (S14: Yes), the communication control unit 11 The mobile terminal 6 disconnects communication with the currently connected mobile terminal 2 (S15). If not (S14: No), the process of step S15 is skipped. The mobile terminal 2 is connected to the mobile terminal 2 requested by the determination unit 114 (S16).

[0046] The determination unit 114 refers to the user list and, if necessary, issues an AV The function control unit 117 receives a request to switch the AV source. The function control unit 117 switches the AV source in response to the request. Acquires audio data or video data from a source and outputs the audio from the speaker 40. Or, the image is displayed on the monitor 50.

[0047] In the example shown in Figure 3 above, the registration information for registrant No. 1 is mobile device name: AAA, handsets: Mobile phone calls: in use, music playback: yes, video playback: no, data communication: in use, AV source: mobile The registration information for passenger No. 2 is mobile device name: BBB, hands-free calling : In use, Music playback: None, Video playback: Yes, Data communication: Not in use, AV source: TV broadcast It is delivery.

[0048] When the driver is changed from registered user No. 1 to registered user No. 2, the passenger detection unit 113 ,Facial feature data of the passenger in the driver's seat position in the image and facial recognition data: The determination unit 114 then sends the information to the communication control unit 116 to confirm that the information matches the mobile terminal AAA0002. After connecting with mobile terminal BBB, the function control The function control unit 117 switches the AV source from the mobile terminal 2 to the television broadcast. Switches the driver's seat position to the position for registered user No. 2. Function control The unit 117 switches the settings of the air conditioner to the settings for registrant No. 2.

[0049] The function control unit 117 controls the function of the mobile terminal 2 to which a new connection has been established via the communication control unit 116. For example, the function control unit 117 can control the sound mode of the mobile terminal 2. You can switch the computer to silent mode.

[0050] The process of FIG. 5 may be started at any timing. For example, It starts when power is supplied to the vehicle hands-free control device 10 by starting the engine or the like. In addition, during the period when power is supplied to the vehicular hands-free control device 10, The process from step S10 to step S12 is periodically executed, and the passenger displayed at the driver's seat position changes. Detects changes to the

[0051] Example 2 FIG. 6 is a flowchart showing a flow of a face authentication process of the hands-free control device 10 for a vehicle according to the second embodiment. In the following explanation, a vehicle C1 is parked in the parking lot of a store, and the driver Assume that the driver leaves vehicle C1 to enter the store.

[0052] The processing unit 11 acquires an image of the interior of the vehicle captured by the camera 20 (S20). When the unit 111 detects the absence of the driver from the frame image of the acquired video (S21: Yes), and it is determined whether there are any faces of other passengers in the image that have not been matched (S23 If there are no other passenger faces that have not been matched (S23: No), the facial recognition process is terminated. That is, the state in which communication with the portable terminal 2 is possible while driving is maintained.

[0053] If there is a face of another passenger that has not been matched (S23: Yes), the image recognition unit 111 identifies the face of the passenger in the seat with the next highest priority from the image (S24 The passenger detection unit 113 compares the facial feature data of the identified passenger with the facial feature data registered in the user list. The plurality of face authentication data stored are compared (S25).

[0054] If there is no matching face authentication data in the user list (S26: No), The process proceeds to S23, and the image recognition unit 111 recognizes the seat that is shown in the image at the position of the seat with the next highest priority. For example, the priority of the seats is 1 for the driver's seat, 2 for the passenger seat, and 3 for the passenger seat. 2, right rear seat: 3, left rear seat: 4 When the driver is absent, the passenger seat →The faces of the passengers are matched in the order of right rear seat → left rear seat.

[0055] In step S26, if matching face authentication data exists in the user list (S 26: Yes), the determination unit 114 refers to the user list and associates it with the matching face authentication data. The mobile terminal 2 connected to the communication control unit 116 is identified, and the communication control unit 116 is instructed to connect to the mobile terminal 2. When the communication control unit 116 receives the request, it communicates with the mobile terminal 2 of the driver who is currently connected. (S27), and connect to the requested mobile terminal 2 (S28). The process of 7 is performed when the driver's mobile terminal 2 is out of the communication range of the short-range wireless communication and the communication is cut off. If the user list is disabled, the process is skipped. Switch to the settings of the authenticated passenger. For example, the passenger sitting in the front or rear seat A passenger answers a call from the mobile terminal 2 of the driver outside the vehicle C1 in a hands-free manner. You can answer.

[0056] When a plurality of microphones 30 are installed in the vehicle C1, the function control unit 117 When a call is made, the seat of the passenger holding the newly connected mobile terminal 2 is the first to be connected. The microphone 30 installed at the closest position picks up the voice of the speaker inside the vehicle C1. When a plurality of the car parkers 40 are installed in the vehicle C1, the function control unit 117 When speaking, the voice of the other person is output from the speaker 40 installed at the position closest to the seat. do.

[0057] The function control unit 117 controls the function of the mobile terminal 2 to which a new connection has been established via the communication control unit 116. For example, the function control unit 117 can control the sound mode of the mobile terminal 2. You can switch the computer to silent mode.

[0058] (Modification of Example 2) FIG. 7 is a flowchart showing a face authentication process of the hands-free control device 10 for a vehicle according to a modified example of the second embodiment. In the flowchart shown in FIG. 6, the driver In this situation, the driver who got off the bus immediately started driving. For example, the driver may get out of the car to remove luggage from the trunk. There may be cases where the customer returns immediately.

[0059] In the flowchart according to the modification shown in FIG. 7, the following is performed in comparison with the flowchart shown in FIG. Step S22 is added. When the image recognition unit 111 detects that the driver is absent (S21: Yes), the determination unit 114 determines whether the receiver 5b receives the signal from the wireless key 5a via the in-vehicle communication unit 14. The determination unit 114 determines whether the receiver 5b is a wireless device or not based on the obtained radio wave intensity. The distance between the line key 5a is estimated.

[0060] The determination unit 114 compares the estimated distance to the wireless key 5a with a set value (S22). If the distance to the line key 5a exceeds the set value (S22: Yes), the process proceeds to step S23. If the distance to the wireless key 5a exceeds the set value, the receiver This also includes cases where the remote control 5b cannot receive radio waves from the wireless key 5a. If the value is equal to or less than the predetermined value (S22: No), the mobile terminal 2 to be connected is not switched, and the driver's mobile terminal The connection with the mobile terminal 2 is maintained.

[0061] In the example shown in FIG. 7, after the absence of the driver is detected, the process of switching the mobile terminal 2 to be connected is performed. The condition for the activation is that the distance to the wireless key 5a exceeds a set value. In this regard, the condition for triggering the switching process is that a set time has elapsed since the absence of the driver was detected. Also, the AND of both can be used as the trigger condition for the switching process. In addition, the condition for initiating the switching process may be a condition for a vehicle hands-free The distance from the control device 10 to the mobile terminal 2 carried by the driver exceeds a set value. In addition, as a condition for initiating the switching process, the mobile terminal 2 carried by the driver may be The distance exceeds the set value and the set time has elapsed since the absence of the driver was detected. The AND condition may be used.

[0062] If the driver stops driving due to a dead battery, malfunction, or operational error in the driver's mobile terminal 2, Even when the driver is sitting in a seat, communication between the mobile terminal 2 and the vehicle hands-free control device 10 is possible. In this case, the image recognition unit 111 determines whether the driver is present. If the communication control unit 116 detects that communication with the driver's mobile terminal 2 is not possible, the process proceeds to step S23. The following switching process of the mobile terminal 2 to be connected may be initiated.

[0063] As described above, according to this embodiment, a person carrying the portable terminal 2 in the vehicle C1 This will improve the convenience of passengers who have previously driven the vehicle. This reduces the hassle of communication between family members or colleagues. When the driver is replaced due to reasons such as a change of driver, the mobile terminal 2 of the new driver can be automatically connected. Even if the passenger leaves vehicle C1, the mobile phone of the passenger sitting in the next highest priority seat will be It can automatically connect to the terminal 2, and the hands-free function of the vehicle can be effectively used. When registering, the user account name used in the mobile terminal 2 can be automatically entered.

[0064] The present invention has been described above based on the embodiments. These embodiments are merely examples, and The fact that various variations are possible in the combination of each component and each treatment process, and that such variations It will be understood by those skilled in the art that the embodiments are within the scope of the present invention.

[0065] In the above-described embodiment, the function control unit 117 newly connects via the communication control unit 116. In the above example, the sound mode of the established mobile terminal 2 is switched to silent mode. In this regard, the function control unit 117 determines the functions of the portable terminal 2 to be controlled based on the information detected by the passenger detection unit 113. For example, the function control unit 117 may change the seating position of the passenger in the driver's seat. The sound mode of the seated passenger's mobile device 2 is switched to silent mode, and the driver's seat The sound mode of the portable terminal 2 of the passenger sitting elsewhere may be kept set.

[0066] In the above-described embodiment, the image recognition unit 111 recognizes the frame of the image input from the camera 20. In this regard, the face of a person was detected in the frame image. In addition, biometric information other than the face may be used as the authentication target. For example, The iris may be used. Alternatively, the passenger may hold the palm of their hand over the camera 20 to capture an image. The fingerprint or veins reflected in the image may be used as biometric information.

[0067] In the above-described embodiment, the mobile terminal 2 may be a smartphone or a flash drive having a calling function. In this regard, the mobile device 2 is a tablet device that does not have a calling function. This also includes portable music players, portable game consoles, etc. Hands-free function cannot be used. Even in this case, by connecting with these mobile terminals 2 by short-range wireless communication, The last two can be used as AV sources. [Explanation of symbols]

[0068] C1 Vehicle, 1 Vehicle hands-free control system, 2 Mobile terminal, 3 In-vehicle network Network, 4 ECU, 5a Radio key, 5b Receiver, 10 Vehicle hands Free control device, 11 processing unit, 111 image recognition unit, 112 registration unit, 113 Passenger detection unit, 114 judgment unit, 115 decision unit, 116 communication control unit, 11 7 function control unit, 12 memory unit, 121 user list holding unit, 13 wireless communication unit , 14 In-vehicle communication unit, 20 Camera, 30 Microphone, 40 Speaker, 50 Monitor, 60 control panel.

Claims

1. A hands-free control device for a vehicle, a communication control unit that detects whether communication is possible between the vehicle hands-free control device and a mobile terminal of a vehicle driver; a passenger detection unit that identifies a passenger in the vehicle from an image captured by a camera that captures an image of the interior of the vehicle and outputs the captured image to the vehicle hands-free control device, and identifies the passenger's seating position; a determination unit that determines whether the portable terminal of the passenger identified by the passenger detection unit is in a state that can be connected via short-range wireless communication; a determination unit that, when there are multiple portable terminals that the determination unit has determined to be in a state where they can be connected via short-range wireless communication, determines a portable terminal to connect via short-range wireless communication based on the boarding positions of multiple passengers corresponding to the multiple portable terminals; the communication control unit, when communication with the mobile terminal of the driver of the vehicle is not possible, switches the connection between the mobile terminal determined by the determination unit and the vehicle hands-free control device so that the short-range wireless communication is performed; Hands-free vehicle control device.

2. 2. The hands-free control device for a vehicle according to claim 1, wherein the determination unit determines the mobile terminal of the passenger sitting in the seat with the highest priority among the seats occupied by passengers in the vehicle as the mobile terminal to be connected via the short-range wireless communication.

3. a plurality of microphones disposed within the vehicle; a function control unit that controls a hands-free calling function of the vehicle hands-free device, 3. The hands-free control device for a vehicle according to claim 1, wherein the function control unit controls to collect voices inside the vehicle from a microphone arranged near a seat in which a passenger having a mobile terminal determined by the determination unit is seated.

4. A vehicular hands-free control method executed by a vehicular hands-free control device, comprising: detecting whether communication between the vehicle hands-free control device and a mobile terminal of a vehicle driver is possible; a step of identifying a passenger in the vehicle and specifying the passenger's seating position from an image captured by a camera that captures an image of the interior of the vehicle and outputs the captured image to the vehicle hands-free control device; determining whether the mobile terminal of the identified passenger is in a state where it can be connected via short-range wireless communication; When there are a plurality of portable terminals determined to be in a state connectable via the short-range wireless communication, determining a portable terminal to be connected via the short-range wireless communication based on the boarding positions of a plurality of passengers corresponding to the plurality of portable terminals; If communication with the mobile terminal of the driver of the vehicle is not possible, switching the connection between the determined mobile terminal and the vehicle hands-free control device so that the short-range wireless communication is performed; A hands-free control method for a vehicle, comprising:

Images