Vehicle device and method executed by the vehicle device

Abstract

To improve convenience for a passenger carrying a portable terminal in a vehicle.SOLUTION: A passenger detection unit 113 identifies a passenger on board a vehicle from a video captured by a camera 20 that captures the inside of the cabin of the vehicle, and determines a boarding position of the passenger. A determination unit 114 determines whether or not a mobile terminal 2 of the passenger identified by the passenger detection unit 113 is in a state of connectable by near field communication. In the case where there are multiple mobile terminals 2 that the determination unit 114 determines to be in a state of being connectable by near field communication, a decision unit 115 decides the mobile terminal 2 to be connected by near field communication on the basis of the boarding positions of multiple passengers corresponding to the multiple mobile terminals 2. A communication control unit 116 connects the mobile terminal 2 decided by the decision unit 115 to the vehicle device by near field communication, and continues the connection with a driver's mobile terminal until a predetermined time elapses even in the case where the passenger detection unit 113 becomes unable to identify the driver of the vehicle.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a vehicle device into which video captured by a camera is input, and a method executed by the vehicle device.

Background Art

[0002] During driving of an automobile, it is prohibited by laws and regulations for a driver to hold a mobile terminal (mainly, a smartphone or a feature phone) in hand and use it for a call or the like. As a countermeasure, a method of connecting a mobile terminal and a hands-free compatible device on the vehicle side by short-range wireless communication and performing hands-free calls and operations is used. Normally, the hands-free device on the vehicle side is set to automatically connect to the mobile terminal that was last connected most recently. Therefore, when a driver different from the previous driver drives, it is not automatically connected, and an operation of selecting a new driver's mobile terminal is required. Also, if a new driver drives without noticing, there is an inconvenience that the hands-free function cannot be used.

[0003]

[0004] On the other hand, a method has been proposed in which, by referring to a table associating the face information of a user and the identification information of a mobile terminal and identifying the face of the driver captured by a driver's seat camera, a connection with the mobile terminal is automatically established (see, for example, Patent Document 1). Note that this driver's seat camera cannot capture the entire interior of the vehicle including passengers other than the driver.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0006] For example, when the driver gets off the vehicle temporarily, it is convenient if the connection destination of the hands-free device on the vehicle side switches from the driver's mobile terminal to the mobile terminal of the passenger sitting in the passenger seat. .

[0007] This embodiment has been made in view of such a situation, and its object is to provide a technology for improving the convenience of passengers who carry mobile terminals in a vehicle.

MEANS FOR SOLVING THE PROBLEMS

[0008] In order to solve the above problems, a hands-free control device for a vehicle according to an aspect of the present embodiment includes a passenger detection unit that identifies a passenger boarding the vehicle and specifies the boarding position of the passenger from an image captured by a camera that captures the interior of the vehicle cabin, a determination unit that determines whether or not the mobile terminal of the passenger identified by the passenger detection unit is in a state where it can be connected by short-range wireless communication, a determination unit that determines whether or not the mobile terminal of the passenger identified by the passenger detection unit is in a state where it can be connected by short-range wireless communication, and when there are a plurality of mobile terminals determined to be in a state where they can be connected by the short-range wireless communication, a determination unit that determines a mobile terminal to be connected by the short-range wireless communication based on the boarding positions of a plurality of passengers specified by a plurality of biometric information corresponding to the plurality of mobile terminals, a communication control unit that connects the mobile terminal determined by the determination unit and the hands-free control device for the vehicle by the short-range wireless communication, and continues the connection with the mobile terminal of the driver until a predetermined time elapses even when the passenger detection unit cannot identify the driver of the vehicle while the mobile terminal of the passenger boarding the driver's seat of the vehicle is connected by the short-range wireless communication.

[0009] ​Another aspect of the present embodiment is a method executed by a hands-free control device for a vehicle. This method includes: identifying a passenger boarding the vehicle and specifying the boarding position of the passenger from an image captured by a camera that captures the interior of the vehicle; determining whether a mobile terminal of the identified passenger is in a state where it can be connected by short-range wireless communication; when there are a plurality of mobile terminals determined to be in a state where they can be connected by short-range wireless communication, determining a mobile terminal to be connected by the short-range wireless communication based on the boarding positions of a plurality of passengers specified by a plurality of biometric information corresponding to the plurality of mobile terminals; connecting the determined mobile terminal and the hands-free control device for the vehicle by the short-range wireless communication, and continuing the connection with the mobile terminal of the driver of the vehicle for a predetermined time even when the driver of the vehicle cannot be identified while the mobile terminal of the passenger boarding the driver's seat of the vehicle is connected by the short-range wireless communication. The hands-free control device for a vehicle executes these steps.

[0010] In addition, any combination of the above components, and those obtained by converting the expressions of the present embodiment among devices, methods, systems, recording media, computer programs, etc. are also effective as aspects of the present embodiment. In addition, any combination of the above components, and those obtained by converting the expressions of the present embodiment among devices, methods, systems, recording media, computer programs, etc. are also effective as aspects of the present embodiment. effective.

Advantages of the Invention

[0011] According to the present embodiment, the convenience of passengers carrying mobile terminals in the vehicle can be improved. can be improved.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0013] This embodiment relates to a hands-free control system for a vehicle for realizing the hands-free function of a mobile terminal possessed by a passenger of the vehicle. In the hands-free control system for a vehicle according to the embodiment, a camera is used to widely photograph the interior of the vehicle including not only the driver's seat but also the passenger seat and the rear seats. is used.

[0014] FIG. 1 is a diagram showing an installation example of the camera 20 in the vehicle C1. In the example shown in FIG. 1, the camera 20 is attached to the rearview mirror. The camera 20 may be diverted from a camera used in a drive recorder or an in-cabin monitoring system, or a dedicated camera for the hands-free control system may be prepared.

[0015] FIG. 2 is a diagram for explaining a configuration example of the hands-free control system 1 for a vehicle according to the embodiment. The hands-free control system 1 for a vehicle includes a hands-free control device 10 for a vehicle, a camera 20, a microphone 30, a speaker 40, a monitor 50, and an operation unit 60.

[0016] The camera 20 photographs the interior of the vehicle and outputs the photographed video to the hands-free control device 10 for the vehicle. It can be done. The camera 20 may be a visible light camera or an infrared camera. Also it may be a camera capable of photographing both the visible light range and the infrared range.

[0017] The camera 20 includes a lens, an imaging element, and a signal processing circuit. As the imaging element, for example, a CM OS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charg e Coupled Device) image sensor can be used. The imaging element converts the light incident through the lens into an electrical video signal and outputs it to the signal processing circuit. When photographing an infrared video an imaging element having sensitivity in the infrared range is used. The signal processing circuit performs signal processing such as A / D conversion and noise removal on the video signal input from the imaging element and outputs it to the vehicle hands-free control device 10.

[0018] The vehicle hands-free control device 10 includes a processing unit 11, a storage unit 12, a wireless communication unit 13, and a vehicle interior communication unit 14. The vehicle hands-free control device 10 may be mounted in a dedicated housing or may be incorporated as a function of a car navigation system, a display audio, a drive recorder, or a vehicle interior monitoring system.

[0019] The processing unit 11 includes an image recognition unit 111, a registration unit 112, a passenger detection unit 113, a determination unit 114 , a decision unit 115, a communication control unit 116, and a function control unit 117. The functions of the processing unit 11 can be realized by the cooperation of hardware resources and software resources, or only by hardware resources . As hardware resources, a CPU, a ROM, a RAM, a GPU (Graphics Processing Unit), DSP (Digital Signal Processor), ISP (Image Signal Processor), AS IC (Application Specific Integrated Circuit), FPGA (Field-Programmable Ga te Array), and other LSIs can be used. As software resources, programs such as firmware can be used.

[0020] The storage unit 12 is configured to include a non-volatile recording medium such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and includes a user list holding unit 121. The user list holding unit 121 is a list in which face authentication data of vehicle passengers and information regarding the portable terminal 2 held by the passengers are linked and registered. In the user list, not only those who hold a driver's license but also those who do not hold a driver's license can be registered.

[0021] FIG. 3 is a diagram showing an example of the user list held in the user list holding unit 121. In the user list shown in FIG. 3, for each registrant, face authentication data and information regarding the portable terminal 2 are linked and registered. As information regarding the portable terminal 2, the name of the portable terminal 2, the connection status, and the AV source are registered. As the connection status, the use / non-use of hands-free calling, the presence / absence of automatic music playback, the presence / absence of automatic video playback, and the use / non-use of data communication are registered. The use / non-use of data communication defines whether the vehicle hands-free control device 10 performs data communication using the portable telephone network (4G / 5G) through the portable terminal 2. For example, it is possible to perform streaming playback of music and video using data communication.

[0022] ​​​​​​​​ The AV source indicates a source of music data or video data when there is music playback or video playback. For example, as the AV source, music files or video files stored in the mobile terminal 2 can be used. Also, radio broadcasts or TV broadcasts received by a tuner installed in a car navigation system or a display audio can be used. Further, audio files or video files recorded on a CD / DVD inserted into a car navigation system or a display audio can be used. Also, for each registrant, an in-vehicle environment other than the AV environment can be registered. For example, the seat position, air conditioner state, etc. when sitting in the driver's seat or the passenger seat can be registered. As the air conditioner state, switching between cooling, heating, and blowing, set temperature, etc. can be registered. Further, as the in-vehicle environment, the position of the side mirror, etc. can also be registered. Returning to FIG. 2. The image recognition unit 111 searches for a person's face in the frame image of the video input from the camera 20 using an identifier for detecting a person's face. For the identifier for face detection, dictionary data generated by learning a large number of images in which a person's face is reflected is used. For face recognition, for example, Haar-like features, HOG (Histogram of Gradients) features, LBP (Local Binary Patterns) features, etc. can be used. The registration unit 112 records the face authentication data of the passenger and information regarding the mobile terminal 2 in the user list. A passenger who wishes to register in the user list sits in the front seat and operates the operation unit 6.

[0023]

[0024]

[0025] Operate 0 to start the registration mode. When the registration mode is started, the registration unit 112 performs image recognition. The identification unit 111 extracts a feature amount of the face area of ​​the passenger detected in the frame image and identifies the passenger. The system then prompts the user to generate facial recognition data to identify the user.

[0026] When the registration unit 112 acquires the face authentication data of the passenger from the image recognition unit 111, The control unit 116 starts a pairing process with the mobile terminal 2 carried by the passenger. The dual-use hands-free control device 10 and the mobile terminal 2 can be connected via short-range wireless communication. For short-distance wireless communication, Bluetooth (registered trademark) and Wi-Fi (registered trademark) can be used. In the following, in this embodiment, an example in which Bluetooth is used is assumed.

[0027] The registration unit 112 receives the profile of the portable terminal 2 from at least one of the speaker 40 and the monitor 50. First, the registration unit 112 instructs the user to turn on the Bluetooth function. The registration unit 112 instructs the communication control unit 116 to transmit the , and transmits a signal (e.g., an advertising packet) to notify the surroundings of its presence. When the mobile terminal 2 scans the signal, the vehicular hands-free system is displayed on the screen of the mobile terminal 2. The control device 10 is displayed as a connectable device on the screen. When the user selects the vehicle hands-free control device 10, the mobile terminal 2 and the vehicle hands-free control device 1 In the pairing process, the encryption of the communication data between the two is performed. The encryption key used for encryption is exchanged.

[0028] When pairing with the portable terminal 2 is completed, the registration unit 112 performs , obtain the name and identification information of the mobile terminal 2. The name of the mobile terminal 2 is preferably a name that is easy for the user to understand. For example, the user account name used by the passenger in various services within the mobile terminal 2 may be used. If the name obtained from the mobile terminal 2 is difficult to understand, the passenger can edit the name of the mobile terminal 2 from the operation unit 60. The passenger inputs the above-described connection state setting items from the operation unit 60. The registration unit 112 associates the face authentication data, the name of the mobile terminal, and each item of the connection state, and registers them in the user list in the user list holding unit 121. The passenger can also input the above-described in-vehicle environment setting items. In that case, each item of the in-vehicle environment is also registered in association. The passenger detection unit 113 identifies the faces of the passengers boarding the vehicle C1 from the video input from the camera 20. Specifically, the passenger detection unit 113 collates the feature data of the faces of the passengers detected in the frame image by the image recognition unit 111 with a plurality of face authentication data registered in the user list to identify the faces of the passengers. The passenger detection unit 113 identifies the boarding positions of at least one passenger boarding the vehicle C1. Specifically, for each passenger, the passenger detection unit 113 identifies whether the boarding position is the driver's seat, the front passenger seat, the right rear seat, the left rear seat, or the rear center seat. FIG. 4 is a diagram showing a detection example of the passengers and boarding positions in the vehicle C1. In the frame image F1 shown in FIG. 4, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat passenger.

[0029] The passenger inputs the above-described connection state setting items from the operation unit 60. The registration unit 112 associates the face authentication data, the name of the mobile terminal, and each item of the connection state, and registers them in the user list in the user list holding unit 121. The passenger can also input the above-described in-vehicle environment setting items. In that case, each item of the in-vehicle environment is also registered in association. The passenger detection unit 113 identifies the faces of the passengers boarding the vehicle C1 from the video input from the camera 20. Specifically, the passenger detection unit 113 collates the feature data of the faces of the passengers detected in the frame image by the image recognition unit 111 with a plurality of face authentication data registered in the user list to identify the faces of the passengers. The passenger detection unit 113 identifies the boarding positions of at least one passenger boarding the vehicle C1. Specifically, for each passenger, the passenger detection unit 113 identifies whether the boarding position is the driver's seat, the front passenger seat, the right rear seat, the left rear seat, or the rear center seat. FIG. 4 is a diagram showing a detection example of the passengers and boarding positions in the vehicle C1. In the frame image F1 shown in FIG. 4, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat passenger.

[0030] The passenger detection unit 113 identifies the faces of the passengers boarding the vehicle C1 from the video input from the camera 20. Specifically, the passenger detection unit 113 collates the feature data of the faces of the passengers detected in the frame image by the image recognition unit 111 with a plurality of face authentication data registered in the user list to identify the faces of the passengers. The passenger detection unit 113 identifies the boarding positions of at least one passenger boarding the vehicle C1. Specifically, for each passenger, the passenger detection unit 113 identifies whether the boarding position is the driver's seat, the front passenger seat, the right rear seat, the left rear seat, or the rear center seat. FIG. 4 is a diagram showing a detection example of the passengers and boarding positions in the vehicle C1. In the frame image F1 shown in FIG. 4, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat passenger. The passenger detection unit 113 identifies the boarding positions of at least one passenger boarding the vehicle C1. Specifically, for each passenger, the passenger detection unit 113 identifies whether the boarding position is the driver's seat, the front passenger seat, the right rear seat, the left rear seat, or the rear center seat.

[0031] The passenger detection unit 113 identifies the boarding positions of at least one passenger boarding the vehicle C1. Specifically, for each passenger, the passenger detection unit 113 identifies whether the boarding position is the driver's seat, the front passenger seat, the right rear seat, the left rear seat, or the rear center seat. FIG. 4 is a diagram showing a detection example of the passengers and boarding positions in the vehicle C1. In the frame image F1 shown in FIG. 4, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat passenger. The passenger detection unit 113 identifies the boarding positions of at least one passenger boarding the vehicle C1. Specifically, for each passenger, the passenger detection unit 113 identifies whether the boarding position is the driver's seat, the front passenger seat, the right rear seat, the left rear seat, or the rear center seat.

[0032] FIG. 4 is a diagram showing a detection example of the passengers and boarding positions in the vehicle C1. In the frame image F1 shown in FIG. 4, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat passenger. In the frame image F1 shown in FIG. 4, the image recognition unit 111 detects a person in the detection area A1 for the driver's seat passenger. When the face of an object is detected, it is recognized that there is a passenger in the driver's seat. Similarly, the image recognition unit 111 When detecting the face of a person within the detection area A2 for boarding the passenger seat, it is recognized that there is a passenger in the passenger seat Similarly, the image recognition unit 111, when detecting the face of a person within the detection area A3 for boarding the right rear seat it is recognized that there is a passenger in the right rear seat. Similarly, the image recognition unit 111 When detecting the face of a person within the detection area A4 for boarding the left rear seat, it is recognized that there is a passenger in the left rear seat it is recognized that there is a passenger. Although not present in the vehicle interior shown in FIG. 4, in the case of a vehicle having a rear center seat, a detection area for boarding the rear center seat is also set.

[0033] Returning to FIG. 2. The determination unit 114 determines whether the mobile terminal 2 associated with the face of the passenger identified by the passenger detection unit 113 is in a state where it can be connected by short-range wireless communication. When the communication control unit 116 recognizes the mobile terminal 2 as a connectable device, the determination unit 114 determines that the mobile terminal 2 is in a connectable state. If the identified passenger does not carry the mobile terminal 2, if the power of the mobile terminal 2 is off, or if the Bluetooth function is off, the determination unit 114 determines that the mobile terminal 2 is not in a connectable state .

[0034] When there are a plurality of mobile terminals 2 determined by the determination unit 114 to be in a connectable state (i.e., , when there are a plurality of passengers), the decision unit 115, based on the boarding positions of the plurality of passengers specified by the plurality of face authentication data associated with the plurality of mobile terminals 2, determines the mobile terminal 2 to be connected. Specifically, the decision unit 115, among the seats where the passengers are seated, selects the mobile terminal 2 associated with the face authentication data of the passenger sitting in the seat with the highest priority and connects it.​ It is determined for the mobile terminal 2 to connect to.

[0035] For example, the seat priorities are set as follows: driver's seat: 1, passenger seat: 2, rear right seat: 3, rear left seat: 4. In this example, when the driver gets off the vehicle, the mobile terminal 2 of the passenger sitting in the passenger seat will be connected to the vehicle hands-free control device 10. Note that the seat priorities can be set and changed. For example, when a person with a higher position sits in the rear right seat, the priority can be set to the rear right seat: 1.

[0036] The wireless communication unit 13 includes an antenna and transmits and receives signals with the mobile terminal 2 using a short-range wireless communication method. The in-vehicle communication unit 14 executes communication processing for connecting to the in-vehicle network 3. As the in-vehicle network 3, CAN (Controller Area Network), LIN (Local Interconnect Network), Ethernet (registered trademark), etc. can be used. Various ECUs (Electronic Control Unit) 4 for controlling the engine and auxiliary machines are connected to the in-vehicle network 3. For example, an ECU for controlling the seat position is also connected. In addition, the control unit of the air conditioner system in the vehicle C1 can also be connected to the in-vehicle network 3.

[0037] In the example shown in FIG. 2, a receiver 5b of the wireless key system is connected to the in-vehicle network 3. The receiver 5b is a receiver for receiving a wireless signal from the wireless key (also called a smart key, intelligent key, remote control key, etc.) 5a held by the driver.

[0038] ​​​​​​​​​The communication control unit 116 controls the wireless communication unit 13 to perform pairing processing with the mobile terminal 2, connection / disconnection processing with the paired mobile terminal 2, management of paired devices, etc. Also, the communication control unit 116 controls the in-vehicle communication unit 14 to transmit and receive control signals to and from various ECUs 4 and devices connected to the in-vehicle network 3. The function control unit 117 controls various functions of the vehicle hands-free control system 1 (for example, hands-free call function, audio playback function, video playback function, etc.). The function control unit 117 can also control the functions in the mobile terminal 2 connected by short-range wireless communication.

[0039] The microphone 30 collects the sound inside the vehicle C1 and outputs it to the function control unit 117. In this embodiment, it can be used for hands-free calls. A plurality of microphones 30 may be installed inside the vehicle C1. For example, one may be installed in each of the front seats and the rear seats, or one may be installed for each seat. At least one microphone 30 may be a diverted microphone of the car navigation system, display audio, drive recorder, or in-vehicle monitoring system. The speaker 40 plays and outputs the audio data supplied from the function control unit 117. In this embodiment,

[0040] it can be used for hands-free calls. A plurality of speakers 40 may be installed inside the vehicle C1. For example, one may be installed in each of the front seats and the rear seats, or one may be installed for each seat. At least one speaker 40 may be a diverted speaker of the car navigation system, display audio, car audio system, drive recorder, etc.

[0041] ​​​​​​​​​​It may also be a diversion of the speaker of the rear speaker or the in-vehicle monitoring system.

[0042] The monitor 50 reproduces and displays the video data supplied from the function control unit 117. Monitor For the 50, a liquid crystal display, an organic EL display, a mini LED display, etc. can be used. A plurality of monitors 50 may be installed in the vehicle C1. For example, one may be installed in each of the front seat and the rear seat, or one may be installed for each seat. Note that at least one monitor 50 may be a diversion of the monitor of the car navigation system, the display audio system, the drive recorder, or the in-vehicle monitoring system.

[0043] (Example 1) FIG. 5 is a flowchart showing the flow of the face authentication process of the hands-free control device 10 for a vehicle according to Example 1. Example 1 is an example in the case where only the driver is on board in the vehicle C1. The processing unit 11 acquires the video of the interior of the vehicle taken by the camera 20 (S10). The image recognition unit 111 detects the face of the passenger reflected in the position of the driver's seat from the frame image of the acquired video (S11). The passenger detection unit 113 collates the feature data of the face of the detected passenger with a plurality of face authentication data registered in the user list (S12). If there is no matching face authentication data in the user list (S13: No), the face authentication process ends. If there is matching face authentication data in the user list (S13: Yes), the determination unit 114 refers to the user list, identifies the mobile terminal 2 associated with the matching face authentication data, and the mobile terminal 2

[0044] ​​​​​​​requests the communication control unit 116 to connect to it.

[0045] Upon receiving the request, the communication control unit 116 checks whether there is a currently connected mobile terminal 2 (S14). If there is a currently connected mobile terminal 2 (S14: Yes), the communication control unit 11 6 disconnects the communication with the currently connected mobile terminal 2 (S15). If there is no currently connected mobile terminal 2 (S14: No), the process of step S15 is skipped. The communication control unit 116 connects to the mobile terminal 2 requested by the determination unit 114 (S16).

[0046] The determination unit 114 refers to the user list and, if necessary, requests the function control unit 117 to switch the AV source. Upon receiving the AV source switching request, the function control unit 117 switches the AV source according to the request. The function control unit 117 obtains audio data or video data from the switched AV source and outputs audio from the speaker 40, or displays video on the monitor 50.

[0047] In the example shown in FIG. 3 above, the registration information of registrant No. 1 is mobile terminal name: AAA, hands-free call: used, music playback: yes, video playback: no, data communication: used, AV source: mobile terminal 2. The registration information of passenger No. 2 is mobile terminal name: BBB, hands-free call : used, music playback: no, video playback: yes, data communication: not used, AV source: TV broadcast is.

[0048] When the driver switches from registrant No. 1 to registrant No. 2, the passenger detection unit 113 obtains the feature data of the face of the passenger reflected in the position of the driver's seat in the image and the face authentication data: Verify the match with FR0002. The determination unit 114 requests the communication control unit 116 to disconnect from the mobile terminal AAA and connect to the mobile terminal BBB. After connecting to the mobile terminal BBB, the function control unit 117 switches the AV source from the mobile terminal 2 to the television broadcast. The function control unit 117 switches the seat position of the driver's seat to the position for registrant No. 2. The function control unit 117 switches the air conditioner settings to the settings for registrant No. 2.

[0049] The function control unit 117 can control the functions of the newly connected mobile terminal 2 via the communication control unit 116. For example, the function control unit 117 can switch the sound mode of the mobile terminal 2 to the silent mode. For example, the function control unit 117 can switch the sound mode of the mobile terminal 2 to the silent mode. The processing of FIG. 5 may be started at an arbitrary timing. For example, it is started when power is supplied to the vehicle hands-free control device 10 due to the engine start of vehicle C1, etc.

[0050] The processing of FIG. 5 may be started at an arbitrary timing. For example, it is started when power is supplied to the vehicle hands-free control device 10 due to the engine start of vehicle C1, etc. When power is supplied to the vehicle hands-free control device 10, the processing from step S10 to step S12 is periodically executed to detect whether the passenger reflected in the driver's seat position has changed. When power is supplied to the vehicle hands-free control device 10, the processing from step S10 to step S12 is periodically executed to detect whether the passenger reflected in the driver's seat position has changed. When power is supplied to the vehicle hands-free control device 10, the processing from step S10 to step S12 is periodically executed to detect whether the passenger reflected in the driver's seat position has changed. When power is supplied to the vehicle hands-free control device 10, the processing from step S10 to step S12 is periodically executed to detect whether the passenger reflected in the driver's seat position has changed.

[0051] (Embodiment 2) FIG. 6 is a flowchart showing the face authentication process flow of the vehicle hands-free control device 10 according to Embodiment 2. In the following description, it is assumed that vehicle C1 is parked in the parking lot of a certain store and the driver leaves vehicle C1 to enter the store. In the following description, it is assumed that vehicle C1 is parked in the parking lot of a certain store and the driver leaves vehicle C1 to enter the store. In the following description, it is assumed that vehicle C1 is parked in the parking lot of a certain store and the driver leaves vehicle C1 to enter the store.

[0052] The processing unit 11 acquires the image of the vehicle interior captured by the camera 20 (S20). When the image recognition unit 111 detects the absence of the driver from the frame image of the acquired video (S21: Yes), determine whether there is a face of another passenger who has not been verified in the image (S23 ). If there is no face of another passenger who has not been verified (S23: No), the face authentication process ends . That is, the communicable state with the mobile terminal 2 during driving is maintained.

[0053] If there is a face of another passenger who has not been verified (S23: Yes), the image recognition unit 111 identifies the face of the passenger shown at the position of the next highest-priority seat in the image (S24 ). The passenger detection unit 113 collates the feature data of the identified passenger's face with a plurality of face authentication data registered in the user list (S25).

[0054] If there is no matching face authentication data in the user list (S26: No), go to step S23, and the image recognition unit 111 identifies the face of the passenger shown at the position of the next highest-priority seat in the image (S24). For example, if the seat priorities are set as driver's seat: 1, passenger seat: 2, right rear seat: 3, left rear seat: 4, when the driver is absent, the faces of the passengers are collated in the order of passenger seat → right rear seat → left rear seat .

[0055] In step S26, if there is matching face authentication data in the user list (S 26: Yes), the determination unit 114 refers to the user list, identifies the mobile terminal 2 associated with the matching face authentication data, and requests the communication control unit 116 to connect to the mobile terminal 2 . When receiving the request, the communication control unit 116 disconnects the communication with the current connected driver's mobile terminal 2 (S27) and connects to the requested mobile terminal 2 (S28). The process of step S2 7 is that the driver's mobile terminal 2 is outside the communicable range by short-range wireless communication and the communication is interrupted . If it is disconnected, it is skipped. The function control unit 117 refers to the user list and switches to the setting environment of the authenticated passenger. For example, a passenger sitting in the passenger seat or the rear seat can answer a call from the driver's mobile terminal 2 outside the vehicle C1 hands - free. When the authenticated passenger is recognized, the system switches to the corresponding setting environment. For example, a passenger sitting in the passenger seat or the rear seat can answer a call from the driver's mobile terminal 2 outside the vehicle C1 hands - free. When a passenger sitting in the passenger seat or the rear seat answers a call from the driver's mobile terminal 2 outside the vehicle C1, they can answer it hands - free. It can be answered hands - free.

[0056] When multiple microphones 30 are installed inside the vehicle C1, during a hands - free call, the function control unit 117 acquires the speaker's voice inside the vehicle C1 from the microphone 30 installed closest to the seat where the passenger holding the newly connected mobile terminal 2 is sitting. Similarly, when multiple speakers 40 are installed inside the vehicle C1, during a hands - free call, the function control unit 117 outputs the speaker's voice of the other party from the speaker 40 installed closest to that seat. When a hands - free call is made, if the newly connected mobile terminal 2 is held by a passenger sitting in a seat, the function control unit 117 acquires the speaker's voice inside the vehicle C1 from the microphone 30 installed closest to the seat where the passenger is sitting. When a hands - free call is made, if the newly connected mobile terminal 2 is held by a passenger sitting in a seat, the function control unit 117 acquires the speaker's voice inside the vehicle C1 from the microphone 30 installed closest to the seat where the passenger is sitting. When multiple speakers 40 are installed inside the vehicle C1, during a hands - free call, the function control unit 117 outputs the speaker's voice of the other party from the speaker 40 installed closest to that seat. When multiple speakers 40 are installed inside the vehicle C1, during a hands - free call, the function control unit 117 outputs the speaker's voice of the other party from the speaker 40 installed closest to that seat. It outputs the speaker's voice of the other party.

[0057] The function control unit 117 can control the functions of the newly connected mobile terminal 2 via the communication control unit 116. For example, the function control unit 117 can switch the sound mode of the mobile terminal 2 to the silent mode. The function control unit 117 can control the functions of the newly connected mobile terminal 2 via the communication control unit 116. For example, the function control unit 117 can switch the sound mode of the mobile terminal 2 to the silent mode. The function control unit 117 can switch the sound mode of the mobile terminal 2 to the silent mode.

[0058] (Modification of Example 2) FIG. 7 is a flowchart showing the face authentication process flow of the vehicle hands - free control device 10 according to the modification of Example 2. In the flowchart shown in FIG. 6, an example of a situation where the driver gets out of the vehicle C1 and the driver is absent was explained. From this situation, there may be a case where the driver who got out immediately returns to the driver's seat. For example, a case where the driver gets out once, unloads luggage from the trunk, and immediately returns can be considered. In the flowchart shown in FIG. 6, an example of a situation where the driver gets out of the vehicle C1 and the driver is absent was explained. From this situation, there may be a case where the driver who got out immediately returns to the driver's seat. For example, a case where the driver gets out once, unloads luggage from the trunk, and immediately returns can be considered. When the driver gets out of the vehicle C1 and the driver is absent, there may be a case where the driver who got out immediately returns to the driver's seat. For example, a case where the driver gets out once, unloads luggage from the trunk, and immediately returns can be considered. When the driver gets out of the vehicle C1 and the driver is absent, there may be a case where the driver who got out immediately returns to the driver's seat. For example, a case where the driver gets out once, unloads luggage from the trunk, and immediately returns can be considered. For example, a case where the driver gets out once, unloads luggage from the trunk, and immediately returns can be considered.

[0059] In the flowchart according to the modified example shown in FIG. 7, with respect to the flowchart shown in FIG. 6, Step S22 is added. When the image recognition unit 111 detects the absence of the driver (S21: Yes), the determination unit 114 acquires, via the in-vehicle communication unit 14, the radio wave intensity of the radio wave received by the receiver 5b from the wireless key 5a. The determination unit 114 estimates the distance between the receiver 5b and the wireless key 5a from the acquired radio wave intensity.

[0060] The determination unit 114 compares the estimated distance to the wireless key 5a with a set value (S22). If the distance to the wireless key 5a exceeds the set value (S22: Yes), the process proceeds to step S23 and the same processing as in FIG. 6 is performed. If the distance to the wireless key 5a exceeds the set value, it also includes the case where the receiver 5b cannot receive the radio wave from the wireless key 5a. If the distance to the wireless key 5a is less than or equal to the set value (S22: No), the connection to the driver's mobile terminal 2 is maintained without switching the connected mobile terminal 2.

[0061] In the example shown in FIG. 7, after detecting the absence of the driver, the condition that the distance to the wireless key 5a exceeds the set value is used as the condition for activating the switching process of the connected mobile terminal 2. In this regard, the fact that a set time has elapsed since the absence of the driver was detected may be used as the activation condition for the switching process. Also, the AND condition of both may be used as the activation condition for the switching process. Further, the fact that the distance from the vehicle hands-free control device 10 to the mobile terminal 2 held by the driver exceeds the set value may be used as the activation condition for the switching process. Also, the fact that the distance to the mobile terminal 2 held by the driver exceeds the set value and the fact that a set time has elapsed since the absence of the driver was detected may be used as the activation condition for the switching process. ​​​​​​​​​​​​ The above AND conditions may be used.

[0062] If the driver is unable to drive due to a dead battery, some malfunction, or an operation error in the driver's mobile terminal 2, Even when the driver is seated in a seat, communication between the mobile terminal 2 and the vehicle hands-free control device 10 is possible. In this case, the image recognition unit 111 may not recognize the absence of the driver. If the communication control unit 116 detects that communication with the driver's mobile terminal 2 is not possible, the process proceeds to step S23. The following switching process of the connected mobile terminal 2 may be initiated.

[0063] As described above, according to the present embodiment, a person carrying the portable terminal 2 in the vehicle C1 Specifically, this will improve the convenience of passengers who have previously driven the vehicle, regardless of who was the previous driver. The driver's mobile terminal 2 can be automatically connected without any trouble, reducing the inconvenience. When the driver is replaced, the mobile terminal 2 of the new driver can be automatically connected. Even if the passenger leaves vehicle C1, the mobile phone of the passenger sitting in the next highest priority seat will be It can automatically connect to the terminal 2, and the hands-free function of the vehicle can be effectively used. In addition, face recognition data When registering, the user account name used in the mobile terminal 2 can be automatically entered.

[0064] The present invention has been described above based on the embodiments. These embodiments are merely examples. The fact that various modifications are possible in the combination of each component and each treatment process, and that such modifications It will be appreciated by those skilled in the art that the embodiments are within the scope of the present invention.

[0065] In the embodiment described above, the function control unit 117 newly connects via the communication control unit 116. An example of switching the sound mode of the established mobile terminal 2 to the silent mode has been described. In this regard, the function control unit 117 may change the function of the mobile terminal 2 to be controlled according to the boarding position of the passenger detected by the passenger detection unit 113. For example, the function control unit 117 may switch the sound mode of the mobile terminal 2 of the passenger sitting in the driver's seat to the silent mode, and the sound mode of the mobile terminal 2 of the passengers sitting outside the driver's seat may maintain the setting.

[0066] In the above-described embodiment, the image recognition unit 111 detected the face of a person in the frame image of the video input from the camera 20. In this regard, in order to identify the individual reflected in the image, biometric information other than the face may be used as an authentication target. For example, the iris reflected in the image may be used as the biometric information. Also, when the passenger holds the palm in front of the camera 20, fingerprints or veins reflected in the image may be used as biometric information.

[0067] In the above-described embodiment, a smartphone and a feature phone having a call function are assumed as the mobile terminal 2. In this regard, the mobile terminal 2 includes a tablet without a call function, a portable music player, a portable game machine, and the like. Even when the hands-free function cannot be used, by connecting these mobile terminals 2 by short-range wireless communication, these mobile terminals 2 can be used as an AV source.

Explanation of Signs

[0068] C1 Vehicle, 1 Hands-Free Control System for Vehicle, 2 Mobile Terminal, 3 In-Vehicle Network, 4 ECU, 5a Wireless Key, 5b Receiver, 10 Hands-Free for Vehicle ​​​​​​Free control device, 11 processing unit, 111 image recognition unit, 112 registration unit, 113 Passenger detection unit, 114 judgment unit, 115 decision unit, 116 communication control unit, 11 7 function control unit, 12 memory unit, 121 user list holding unit, 13 wireless communication unit , 14 in-vehicle communication unit, 20 camera, 30 microphone, 40 speaker, 50 Monitor, 60 operation unit.

Claims

1. A hands-free control device for a vehicle, a passenger detection unit that identifies a passenger who is riding in the vehicle from an image captured by a camera that captures an image of the interior of the vehicle and outputs the captured image to the vehicle hands-free control device, and that identifies the riding position of the passenger; a determination unit that determines whether or not a portable terminal of the passenger identified by the passenger detection unit is in a state that allows connection via short-range wireless communication; a determination unit that, when there are multiple portable terminals that the determination unit has determined to be in a state where they can be connected via short-range wireless communication, determines a portable terminal to be connected via the short-range wireless communication based on boarding positions of multiple passengers corresponding to the multiple portable terminals; a communication control unit that connects the mobile terminal determined by the determination unit and the vehicle hands-free control device via the short-range wireless communication, and in a state where the mobile terminal is connected to a mobile terminal of a passenger sitting in the driver's seat of the vehicle via the short-range wireless communication, continues the connection with the driver's mobile terminal until a predetermined time has elapsed even if the passenger detection unit is no longer able to identify the driver of the vehicle; A hands-free control device for a vehicle.

2. 2. The hands-free control device for a vehicle according to claim 1, wherein the determination unit determines the portable terminal of a passenger sitting in a seat with the highest priority among the seats in which the passenger is seated in the vehicle as the portable terminal to be connected via the short-range wireless communication.

3. A vehicular hands-free control method executed by a vehicular hands-free control device, comprising: A step of identifying a passenger who is riding in the vehicle and specifying a riding position of the passenger from an image captured by a camera that captures an image of the interior of the vehicle and outputs the captured image to the vehicle hands-free control device; determining whether the mobile terminal of the identified passenger is in a state where it can be connected via short-range wireless communication; determining a mobile terminal to be connected via the short-range wireless communication based on boarding positions of a plurality of passengers corresponding to the plurality of mobile terminals when a plurality of mobile terminals are determined to be in a state connectable via the short-range wireless communication; a step of connecting the determined portable terminal and the vehicle hands-free control device via the short-range wireless communication, and in a state where the determined portable terminal and the portable terminal of a passenger sitting in the driver's seat of the vehicle are connected via the short-range wireless communication, continuing the connection with the portable terminal of the driver until a predetermined time has elapsed even if the driver of the vehicle cannot be identified; is executed.

Images