Vehicle mounted device

Abstract

To simply and reliably determine a position of a user in a vehicle interior and provide control according to the position of the user.SOLUTION: An onboard device mounted on a vehicle includes: a mobile terminal communication unit that communicates with a mobile terminal located in a vehicle interior; and a control unit that provides the mobile terminal with an interface related to operation of the on-board device. The control unit makes an interface to be provided to the mobile terminal different according to a seating position of a user determined by comparing a vehicle interior image obtained by photographing the vehicle interior with a face image of the user of the mobile terminal. In addition, the onboard device further includes a display unit. The control unit causes the mobile terminal to make a display that expands display contents of the display unit if the seating position is a front seat, and causes the mobile terminal to make the same display as the display contents of the display unit if the seating position is other than the front seat.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an in-vehicle device and a server.

Background Art

[0002] As background art, there is Japanese Patent Application Laid-Open No. 2019-22012 (hereinafter referred to as Patent Document 1). In Patent Document 1, it is described that "the portable information terminal 2 has a function of transmitting ultrasonic waves. Three or more ultrasonic sensors are arranged in the vehicle interior of the vehicle V, and an ultrasonic receiving device for receiving ultrasonic waves using these ultrasonic sensors is provided in the vehicle V. The position detection unit 12 acquires information indicating the propagation time of the ultrasonic wave from the portable information terminal 2 to each ultrasonic sensor, or information indicating the reception intensity of the ultrasonic wave by each ultrasonic sensor, when the ultrasonic wave transmitted by the portable information terminal 2 is received by each ultrasonic sensor, from the ultrasonic receiving device. The position detection unit 12 converts the propagation time or the reception intensity or the like into a distance, and detects the position of the portable information terminal 2 by so-called "triangulation".

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 describes a method for detecting the position of a mobile device using tripoint positioning, which requires the placement of multiple ultrasonic sensors or the like within the vehicle cabin. Installing multiple sensors solely for the purpose of detecting the position of a mobile device increases costs and makes it difficult to retrofit the function of detecting the position of a mobile information device. For example, when linking a mobile device with an in-vehicle device and receiving operations on the in-vehicle device via the mobile device, it is desirable to have different interfaces for the mobile device in the front seats (driver's seat, passenger seat, etc.) where direct operation of the in-vehicle device is possible, and for other seats. However, such improvements in convenience require significant costs, and therefore such technologies have not become widespread in the market. Thus, the present invention aims to easily and reliably determine the user's position within the vehicle cabin and provide control according to the user's position. [Means for solving the problem]

[0005] The in-vehicle device according to the present invention is an in-vehicle device mounted in a vehicle and comprises a mobile terminal communication unit that communicates with a mobile terminal located in the vehicle's interior, and a control unit that provides the mobile terminal with an interface for operating the in-vehicle device. The control unit differentiates the interface provided to the mobile terminal according to the seating position of the user, which is determined by comparing an image of the vehicle's interior taken from the vehicle's interior with a facial image of the user of the mobile terminal. [Effects of the Invention]

[0006] According to the present invention, the user's position inside the vehicle can be easily and reliably determined, and control can be provided according to the user's position. [Brief explanation of the drawing]

[0007] [Figure 1] This is an explanatory diagram of an in-vehicle control system according to an embodiment of the present invention. [Figure 2] This is a configuration diagram showing the configuration of a mobile device. [Figure 3] This is a configuration diagram showing the server configuration. [Figure 4] This is a diagram showing the configuration of the in-vehicle device. [Figure 5]This is an explanatory diagram showing a specific example of an interior image of a vehicle. [Figure 6] This is an explanatory diagram of control based on seating position. [Figure 7] This is a flowchart showing the processing procedure for mobile devices. [Figure 8] This is a flowchart showing the server's processing steps. [Figure 9] This is a flowchart showing the processing procedure for in-vehicle devices. [Figure 10] This is an explanatory diagram for the first modified example. [Figure 11] This is an explanatory diagram for variation 2. [Figure 12] This is an explanatory diagram for the third modified example. [Figure 13] This is an explanatory diagram regarding seating position and the display on a mobile device. [Figure 14] This flowchart shows the processing procedure for interface integration. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings.

[0009] Figure 1 is an explanatory diagram of an in-vehicle control system according to an embodiment of the present invention. The in-vehicle control system according to this embodiment includes a portable terminal 30 mounted on a vehicle 20, an in-vehicle device 40, and a server 10.

[0010] The mobile terminal 30 is, for example, a smartphone, and communicates with the in-vehicle unit 40 via short-range wireless communication, and can obtain the in-vehicle unit ID, which is the identification information of the in-vehicle unit 40. When the mobile terminal 30 launches a predetermined application inside the vehicle, it captures an image of the user's face and registers the face image with the server 10 (1). At this time, the mobile terminal 30 transmits the mobile terminal ID and the in-vehicle unit ID, which are the identification information of the mobile terminal 30, along with the face image to the server 10, and the server 10 stores the face image in association with the mobile terminal ID and the in-vehicle unit ID.

[0011] The in-vehicle device 40 acquires an in-vehicle image and transmits it to the server 10 together with the in-vehicle device ID (2). For example, if an image captured for recording by a drive recorder includes an image of the interior of the vehicle, that image can be used as the in-vehicle image.

[0012] If the server 10 receives the in-vehicle image and the in-vehicle device ID from the in-vehicle device 40, it extracts the face image registered in association with the received in-vehicle device ID (3). The server 10 performs face recognition using the extracted face image and the in-vehicle image. If it identifies a face image that matches the face of a person included in the in-vehicle image, the server 10 determines the seating position from the position in the in-vehicle image and sets it as the seating position of the user of the mobile terminal 30 associated with the face image (4). The server 10 notifies the mobile terminal 30 of this seating position (5).

[0013] The mobile terminal 30 receives the notification of the seating position from the server 10 and stores it as the seating position of the user of its own terminal. Thereafter, when the mobile terminal 30 transmits a control request to the in-vehicle device 40, it can specify the stored seating position (6).

[0014] If the in-vehicle device 40 receives a control request specifying the seating position from the mobile terminal 30, it executes the control related to the specified seating position (7).

[0015] Specifically, the mobile terminal 30 can specify the seating position of the user and transmit a request for volume control to the in-vehicle device 40. In this case, the in-vehicle device 40 controls the speaker so that the volume at the specified seating position satisfies the request.

[0016] The control request specifying the seating position can be used for controls such as air conditioning control, seat angle control, locking / unlocking, and interior lighting control in addition to volume control. Note that when transmitting a control request from the mobile terminal 30 to the in-vehicle device 40, it is not essential to specify the seating position, and it does not prevent the transmission of a control request for which the specification of the seating position is not required.

[0017] As described above, in the in-vehicle device control system shown in Figure 1, the server 10 stores the user's face image from the mobile terminal 30 in association with the mobile terminal ID, and performs face recognition using the in-vehicle interior image and the face image to determine the user's seating position. Then, when the in-vehicle device 40 receives a predetermined control request from the mobile terminal 30, it executes control related to the seating position of the user of the mobile terminal 30. Therefore, the user's position in the vehicle can be determined simply and reliably, and control can be provided according to the user's position.

[0018] Figure 2 is a configuration diagram showing the configuration of a mobile terminal 30. The mobile terminal 30 is, for example, a smartphone and includes a touch panel display 31, a mobile terminal camera 32, an in-vehicle communication unit 33, a server communication unit 34, a storage unit 35, and a control unit 36.

[0019] The touch panel display 31 is an input / output interface that receives input from the user and outputs it. In this embodiment, the mobile terminal camera 32 is used to capture an image of the user's face. The in-vehicle unit communication unit 33 is a communication interface that connects to the in-vehicle unit 40 via short-range wireless communication. For short-range wireless communication, for example, Bluetooth® may be used. The server communication unit 34 is a communication interface that connects to the server 10.

[0020] The memory unit 35 is a storage device such as flash memory. The memory unit 35 stores application programs 35a, mobile terminal ID 35b, in-vehicle device ID 35c, and seating position data 35d, etc.

[0021] The application program 35a is executed by the control unit 36 ​​to realize various functions. In this embodiment, the application program 35a functions as a remote controller that requests control from the in-vehicle device 40, and for that purpose, it can capture an image of the user's face and communicate with the server 10.

[0022] The mobile terminal ID 35b is identification information used to uniquely identify the mobile terminal 30. The in-vehicle unit ID 35c is obtained from the in-vehicle unit 40 and stored when short-range wireless communication with the in-vehicle unit 40 is established. Since the in-vehicle unit 40 is fixed to the vehicle 20, the in-vehicle unit ID indicates which vehicle the mobile terminal 30 is located in.

[0023] The seating position data 35d is data indicating the seating position received from the server 10. This seating position data 35d allows the system to identify where the user of the mobile terminal 30 is seated within the vehicle. Specifically, the driver's seat, passenger seat, right rear seat, left rear seat, etc., are used as seating positions.

[0024] Furthermore, if the user exits the vehicle and the short-range wireless communication between the mobile terminal 30 and the in-vehicle unit 40 is disconnected, it is desirable to delete the in-vehicle unit ID 35c and seating position data 35d from the storage unit 35.

[0025] The control unit 36 ​​is, for example, a CPU (Central Processing Unit), and it realizes the remote control function of the in-vehicle device 40 by reading and executing the application program 35a. Specifically, by executing the application program 35a, the control unit 36 ​​operates as the in-vehicle device connection unit 36a, the face image transmission unit 36b, and the command transmission unit 36c.

[0026] The in-vehicle device connection unit 36a establishes communication with the in-vehicle device 40 when the application program 35a is started, obtains the in-vehicle device ID from the in-vehicle device 40, and stores it in the storage unit 35 as the in-vehicle device ID 35c.

[0027] When the application program 35a is started, the face image transmission unit 36b acquires the in-vehicle device ID, then captures the user's face image with the mobile terminal camera 32 and sends it to the server 10 along with the mobile terminal ID 35b and the in-vehicle device ID 35c to request registration of the face image. Also, if the face image transmission unit 36b receives the seating position from the server 10, it stores the seating position data 35d indicating the received seating position in the storage unit 35.

[0028] The command transmission unit 36c is a processing unit that receives operations from the user, generates control requests, i.e., commands, to be sent to the in-vehicle unit 40, and transmits them to the in-vehicle unit 40. At this time, the command transmission unit 36c can transmit seating position data 35d together with the command.

[0029] Figure 3 is a configuration diagram showing the configuration of server 10. As shown in Figure 3, server 10 has a communication unit 11, a storage unit 12, and a control unit 13. The communication unit 11 is a communication interface that communicates with the mobile terminal 30 and the in-vehicle device 40.

[0030] The storage unit 12 is a storage device such as a hard disk drive or flash memory, and stores the face image management data 12a. The face image management data 12a is registered in association with the face image received from the mobile terminal 30, the mobile terminal ID, and the in-vehicle device ID.

[0031] The control unit 13 is, for example, a CPU, and by executing a predetermined program, it operates as a face image management unit 13a, a face recognition processing unit 13b, and a seating position determination unit 13c. When the face image management unit 13a receives a face image, a mobile terminal ID, and an in-vehicle device ID from the mobile terminal 30, it associates the mobile terminal ID and the in-vehicle device ID with the face image and registers them in the face image management data 12a.

[0032] When the face recognition processing unit 13b receives an in-vehicle interior image and an in-vehicle unit ID from the in-vehicle unit 40, it extracts a face image registered in association with the received in-vehicle unit ID from the face image management data 12a, and performs face recognition using the extracted face image and the in-vehicle interior image.

[0033] The seating position determination unit 13c determines the seating position from its position in the vehicle interior image when the face recognition processing unit 13b identifies a face image that matches the face of a person included in the vehicle interior image, and sets this as the seating position of the user of the mobile terminal 30 associated with the face image. The seating position determination unit 13c then notifies the mobile terminal 30 of the seating position.

[0034] Figure 4 is a diagram showing the configuration of the in-vehicle unit 40. The in-vehicle unit 40 includes a touch panel display 41, an in-vehicle camera 42, a mobile terminal communication unit 43, a server communication unit 44, a storage unit 45, and a control unit 46.

[0035] The touch panel display 41 is an input / output interface that receives input from the user and outputs a display. The in-vehicle camera 42 is a camera capable of capturing images of the inside of the vehicle. It is preferable to use the camera of a drive recorder. That is, the in-vehicle unit 40 may have the function of a drive recorder, or the drive recorder and the in-vehicle camera 42 may be shared. The mobile terminal communication unit 43 is a communication interface that communicates with the mobile terminal 30 via short-range wireless communication. The server communication unit 44 is a communication interface that communicates with the server 10.

[0036] The memory unit 45 is a storage device such as flash memory. The memory unit 45 stores the in-vehicle device ID 45a, etc. The in-vehicle device ID 45a is identification information for uniquely identifying the in-vehicle device 40.

[0037] The control unit 46 is, for example, a CPU, and by reading and executing a predetermined program, it has a mobile terminal connection unit 46a, an in-vehicle image transmission unit 46b, and a command processing unit 46c.

[0038] The mobile terminal connection unit 46a establishes communication with the mobile terminal 30 and performs the process of transmitting the in-vehicle unit ID 45a to the mobile terminal 30.

[0039] The in-vehicle image transmission unit 46b is a processing unit that transmits the in-vehicle image captured by the in-vehicle camera 42 and the in-vehicle ID 45a to the server. The transmission of the in-vehicle image can be performed periodically, for example, at regular intervals.

[0040] The command processing unit 46c is a processing unit that executes commands received from the mobile terminal 30. Specifically, the command processing unit 46c can execute commands related to volume control, air conditioning control, seat angle control, locking / unlocking, and interior lighting control. Command execution is performed by controlling the operation of speakers, air conditioners, seat angle adjustment mechanisms, locking mechanisms, interior lighting fixtures, etc., installed in the vehicle 20. When a seating position is received along with a command, the command processing unit 46c controls the operation of various devices to satisfy the command specifications at the received seating position.

[0041] Figure 5 is an explanatory diagram illustrating a specific example of an interior car image. In Figure 5, the interior car image F1 contains images of multiple people's faces. Partial image F2 of interior car image F1 contains an image of the face of a person seated in the driver's seat, and partial image F3 contains an image of the face of a person seated in the passenger seat. Similarly, partial image F4 contains an image of the face of a person seated on the right side of the rear seat, and partial image F5 contains an image of the face of a person seated on the left side of the rear seat.

[0042] Since the in-vehicle camera 42 is fixed to the vehicle 20, the positions of partial images F2 to F5 in the interior image F1 correspond to the seating positions. Therefore, by using face recognition to identify a person included in the interior image F1, the seating position of that person can be determined based on the position of their face in the interior image F1.

[0043] Figure 6 is an explanatory diagram of control based on seating position. In Figure 6, the user of the mobile terminal 30 is seated on the right side of the rear seat of the vehicle 20. In this state, if the mobile terminal 30 sends a command requesting volume control along with the seating position, the vehicle's speakers will be controlled so that the volume indicated by the command is set on the right side of the rear seat. Note that for convenience, Figure 6 shows the case where the position of the mobile terminal 30 and the user's seating position are the same, but since the mobile terminal 30 transmits the user's seating position, control can be performed based on the user's seating position even if the position of the mobile terminal 30 and the user's seating position are different.

[0044] Figure 7 is a flowchart showing the processing procedure of the mobile terminal 30. First, when the control unit 36 ​​starts the application program 35a, the in-vehicle device connection unit 36a establishes communication with the in-vehicle device 40 (step S101). Having established communication, the in-vehicle device connection unit 36a obtains the in-vehicle device ID from the in-vehicle device 40 and stores it in the storage unit 35 as the in-vehicle device ID 35c (step S102).

[0045] After acquiring and storing the in-vehicle device ID, the face image transmission unit 36b captures the user's face image using the mobile terminal camera 32 (step S103), and requests registration of the face image by transmitting the face image to the server 10 along with the mobile terminal ID 35b and the in-vehicle device ID 35c (step S104). If the face image transmission unit 36b receives the seating position from the server 10 (step S105), it stores the seating position data 35d indicating the received seating position in the storage unit 35 (step S106).

[0046] After step S106, the command transmission unit 36c monitors for the occurrence of a command (step S107). A command is generated, for example, in response to an operation from the user. If no command has been generated (step S107; No), the command transmission unit 36c repeats step S107.

[0047] If a command is generated (step S107; Yes), the command transmission unit 36c determines whether or not it is a command dependent on the seating position (step S108). If it is not a command dependent on the seating position (step S108; No), the command transmission unit 36c transmits the generated command to the in-vehicle unit 40 (step S111) and returns to step S107.

[0048] If the command depends on the seating position (step S108; Yes), the command transmission unit 36c reads the seating position data 35d from the storage unit 35 (step S109), transmits the command to the in-vehicle device 40 along with the seating position data (step S110), and returns to step 107.

[0049] Figure 8 is a flowchart showing the processing procedure of the server 10. The face image management unit 13a of the server 10 determines whether or not it has received a face image from the mobile terminal 30 (step S201). If it has received a face image from the mobile terminal 30 (step S201; Yes), the face image management unit 13a registers the mobile terminal ID and in-vehicle device ID received along with the face image in the face image management data 12a (step S202).

[0050] After step S202, or if no face image has been received (step S201; No), the face recognition processing unit 13b determines whether or not an interior image has been received from the in-vehicle device 40 (step S203). If no interior image has been received (step S203; No), the process returns to step S201.

[0051] If an in-vehicle image is received (Step S203; Yes), the face recognition processing unit 13b extracts a face image registered in association with the in-vehicle device ID received along with the in-vehicle image from the face image management data 12a (Step S204), and performs face recognition using the extracted face image and the in-vehicle image (Step S205).

[0052] After step S205, the seating position determination unit 13c determines the seating position from the position of the person identified by facial recognition in the in-vehicle image (step S206). The seating position determination unit 13c notifies the seating position to the mobile terminal 30 of the person identified by facial recognition (step S207), and the process returns to step S201.

[0053] Figure 9 is a flowchart showing the processing procedure of the in-vehicle unit 40. The mobile terminal connection unit 46a of the in-vehicle unit 40 first establishes communication with the mobile terminal 30 and transmits the in-vehicle unit ID 45a to the mobile terminal 30 (step S301).

[0054] After step S301, the in-vehicle image transmission unit 46b captures an image of the interior of the vehicle with the in-vehicle camera 42 (step S302), and transmits the in-vehicle image and the in-vehicle ID 45a to the server (step S303).

[0055] The command processing unit 46c monitors whether or not a command has been received from the mobile terminal 30 (step S304). If no command has been received (step S304; No), the process returns to step S302.

[0056] If a command is received (step S304; Yes), the command processing unit 46c determines whether or not seating position data has been received along with the command (step S305). If seating position data has been received along with the command (step S305; Yes), the command processing unit 46c executes the command for the seating position (step S306) and returns to step S302. On the other hand, if seating position data has not been received (step S305; No), the command processing unit 46c executes the command (step S307) and returns to step S302.

[0057] In the above description, an example configuration was given in which the server determines the seating position and the mobile terminal 30 stores the seating position. However, the present invention is not limited to the above configuration and can be implemented in various modified forms.

[0058] (Variation 1) Figure 10 is an explanatory diagram of Modification 1. In the in-vehicle device control system of Modification 1, the server 10 transmits the determined seating position to the in-vehicle device 40, and the in-vehicle device 40 stores the seating position. The operation of the in-vehicle device control system of Modification 1 will be described below.

[0059] First, the process from (1) facial image registration to (4) seating position determination is the same as in Figure 1. After determining the seating position, the server 10 notifies the in-vehicle unit 40 of the mobile terminal ID associated with the facial image and the seating position (5). Upon receiving the notification, the in-vehicle unit 40 stores the seating position and the mobile terminal ID in association.

[0060] Subsequently, the mobile terminal 30 sends a control request to the in-vehicle unit 40 (6). When sending this control request, the seating position cannot be directly specified, but it is possible to specify that "the seating position of the user of the mobile terminal 30 should be the target of control."

[0061] When the in-vehicle unit 40 receives a control request from the mobile terminal 30, it identifies the seating position of the mobile terminal 30 based on the mobile terminal ID and executes the specified control (7). Therefore, the in-vehicle unit 40 can execute control targeting the seating position of the user of the mobile terminal 30.

[0062] (Modification 2) Figure 11 is an explanatory diagram of Modified Example 2. In the in-vehicle control system of Modified Example 2, the in-vehicle unit 40 determines the seating position, and the mobile terminal 30 stores the seating position. The operation of the in-vehicle control system of Modified Example 2 will be described below.

[0063] When a designated application is launched inside the vehicle, the mobile terminal 30 captures an image of the user's face and registers the face image by transmitting it to the in-vehicle unit 40 along with the mobile terminal ID (1). The in-vehicle unit 40 stores the face image in association with the mobile terminal ID.

[0064] The in-vehicle unit 40 periodically captures images of the vehicle interior (2) and performs face recognition using the stored face images and the vehicle interior images. If a face image matching a person's face included in the vehicle interior images is identified, the in-vehicle unit 40 determines the seating position from the position in the vehicle interior images and sets it as the seating position of the user of the mobile terminal 30 associated with the face image (3). The in-vehicle unit 40 notifies the mobile terminal 30 of the seating position (4).

[0065] The mobile terminal 30 receives a seating position notification from the in-vehicle unit 40 and stores it as the seating position of the terminal's user. Subsequently, when the mobile terminal 30 sends a control request to the in-vehicle unit 40, it can specify the stored seating position (5).

[0066] If the in-vehicle unit 40 receives a control request from the mobile terminal 30 specifying a seating position, it will execute control related to the specified seating position (6).

[0067] (Variation 3) Figure 12 is an explanatory diagram of Modified Example 3. In the in-vehicle control system of Modified Example 3, the in-vehicle unit 40 determines and stores the seating position. The operation of the in-vehicle control system of Modified Example 3 will be described below.

[0068] When a designated application is launched inside the vehicle, the mobile terminal 30 captures an image of the user's face and registers the face image by transmitting it to the in-vehicle unit 40 along with the mobile terminal ID (1). The in-vehicle unit 40 stores the face image in association with the mobile terminal ID.

[0069] The in-vehicle unit 40 periodically captures images of the vehicle interior (2) and performs face recognition using the stored face images and the vehicle interior images. If a face image matching a person's face included in the vehicle interior images is identified, the in-vehicle unit 40 determines the seating position from the position in the vehicle interior images and sets it as the seating position of the user of the mobile terminal 30 associated with the face image (3). The in-vehicle unit 40 stores the seating position associated with the mobile terminal ID of the mobile terminal 30.

[0070] Subsequently, the mobile terminal 30 transmits a control request to the in-vehicle unit 40 (4). When transmitting this control request, the seating position cannot be directly specified, but it is possible to specify that "the seating position of the user of the mobile terminal 30 should be the target of control."

[0071] When the in-vehicle unit 40 receives a control request from the mobile terminal 30, it identifies the seating position of the mobile terminal 30 based on the mobile terminal ID and executes the specified control (5). Therefore, the in-vehicle unit 40 can execute control targeting the seating position of the user of the mobile terminal 30.

[0072] (Specific examples of interface integration) Next, we will explain how to apply the identified seating position to the interface communication between the mobile device and the in-vehicle device. Figure 13 is an explanatory diagram regarding seating position and the display on the mobile terminal. The in-vehicle unit 40 works in conjunction with the mobile terminal 30, and the mobile terminal 30 can receive operations on the in-vehicle unit 40. In this case, it is desirable to have different mobile terminal interfaces for the front seats (driver's seat, passenger seat, etc.) where the in-vehicle unit can be operated directly, and for the other seats.

[0073] In Figure 13, a map is displayed on the touch panel display 41 of the in-vehicle unit 40. When the mobile terminals 30c and 30d, located in the rear seats, are linked with the in-vehicle unit 40, they display the same content as the touch panel display 41. Therefore, even rear-seat passengers who cannot directly operate the touch panel display 41 can use the display on the mobile terminals 30 as a substitute for the touch panel display 41 to input commands to the in-vehicle unit 40. For convenience, this process of displaying the same content as the touch panel display 41 is referred to as "duplicating the display content."

[0074] On the other hand, when the mobile terminal 30a located in the driver's seat and the mobile terminal 30b located in the passenger seat are linked with the in-vehicle unit 40, they do not duplicate the content displayed on the touch panel display 41, but rather display additional related content that is separate from it. This is because front-seat occupants can directly operate the touch panel display 41, allowing them to differentiate between operating the touch panel display 41, which displays different content, and operating the mobile terminals 30a, etc. In this way, operability for front-seat occupants can be improved. For convenience, this process of displaying additional content related to the content displayed on the touch panel display 41 is referred to as "extension of display content."

[0075] Here, the mobile terminal 30a located in the driver's seat displays the "User A Location List" as an additional display related to the content displayed on the touch panel display 41. Similarly, the mobile terminal 30b located in the passenger seat displays the "User B Location List" as an additional display related to the content displayed on the touch panel display 41. These are displays for setting navigation destinations, for example, but different candidate location lists are displayed for each user. In this way, when extending the display, the content can be made different by using information associated with the user. Information associated with the user can include, for example, past driving history or locations registered and edited by the user. Therefore, even when displaying the same map and listing destination candidates, it is possible to display a list prioritizing ramen restaurants on User A's mobile terminal 30a, and a list prioritizing cafes on User B's mobile terminal 30b.

[0076] Figure 14 is a flowchart showing the interface linkage processing procedure. First, the control unit 46 of the in-vehicle unit 40 receives an interface linkage command from the mobile terminal 30 (step S401). The control unit 46 identifies the seating position (step S402). The seating position may be identified by receiving seating position data from the mobile terminal 30, or by referring to data received from the server 10. Alternatively, the in-vehicle unit 40 may identify the seating position by comparing a face image with an interior image of the vehicle.

[0077] The control unit 46 determines whether the seating position is in the front seat (step S403). If the seating position is in the front seat (step S403; Yes), the control unit 46 expands the display content of the touch panel display 41 (step S404). If the seating position is not in the front seat (step S403; No), the control unit 46 duplicates the display content of the touch panel display 41 (step S405). The control unit 46 provides the expanded or duplicated display to the mobile terminal 30, starts accepting operations from the mobile terminal 30 (step S406), and terminates the process. Operation acceptance from the mobile terminal 30 ends at predetermined timings, such as when an operation to terminate interface cooperation is received, when communication between the mobile terminal 30 and the in-vehicle unit 40 is lost, or when the power of the in-vehicle unit 40 is turned off.

[0078] As described above, according to this embodiment, the in-vehicle unit 40 mounted on the vehicle 20 includes a mobile terminal communication unit 43 that communicates with a mobile terminal 30 located inside the vehicle, and a control unit 46 that provides the mobile terminal 30 with an interface for operating the in-vehicle unit 40. The control unit 46 then differentiates the interface provided to the mobile terminal 30 according to the seating position of the user, which is determined by comparing an image of the vehicle interior taken inside the vehicle with a facial image of the user of the mobile terminal 30. This configuration and operation allows for the simple and reliable determination of the user's position within the vehicle and provides control tailored to that position. As a result, the user can automatically use an interface appropriate to their seating position.

[0079] Furthermore, the in-vehicle unit 40 is further equipped with a touch panel display 41 as a display unit, and the control unit 46 causes the mobile terminal 30 to display an expanded version of the display unit's content if the seating position is in the front seat, and causes the mobile terminal 30 to display the same content as the display unit's content if the seating position is in a position other than the front seat. Therefore, passengers in the front seats can use the touch panel display 41 and the mobile terminal 30 in combination, while passengers in seats other than the front seats can use the mobile terminal 30 instead of the touch panel display 41.

[0080] Furthermore, the in-vehicle unit 40 further includes a server communication unit 44 that transmits the in-vehicle images to the server 10, and the control unit 46 can be configured to provide different interfaces to the mobile terminal depending on the seating position determined by the server 10 comparing the in-vehicle images with the facial image of the mobile terminal user. In this configuration, the server 10 is used to reduce the processing load on the in-vehicle unit 40 and lower the cost of the in-vehicle unit 40.

[0081] Furthermore, the control unit 46 can also determine the seating position by comparing the user's facial image received from the mobile terminal 30 with the in-vehicle image. This configuration allows for the stable provision of a function to determine seating position without relying on communication with server 10.

[0082] Alternatively, images captured by the dashcam camera may be used as the in-vehicle images. This configuration makes effective use of existing equipment and provides user-position-dependent control at low cost without requiring additional equipment.

[0083] Furthermore, the control unit 46 can provide different interfaces to the mobile terminal 30 depending on the user's seating position and the user information associated with the user. Therefore, by utilizing user history, characteristics, registration information, etc., convenience can be further improved.

[0084] Furthermore, the disclosed server 10 includes a communication unit 11 that communicates with the in-vehicle unit 40 and the mobile terminal 30; a face image management unit 13a that manages the user's face image received from the mobile terminal 30 by associating it with the identification information of the mobile terminal and the identification information of the in-vehicle unit; a face recognition processing unit 13b that, when it receives an in-vehicle interior image and the identification information of the in-vehicle unit from the in-vehicle unit 40, performs face recognition using the face image managed in association with the identification information of the in-vehicle unit 40 and the in-vehicle interior image; and a seating position determination unit 13c that, when the face recognition processing unit 13b identifies a face image that matches the face of a person included in the in-vehicle interior image, determines the seating position from the position in the in-vehicle interior image and sets it as the seating position of the user of the mobile terminal associated with the face image. With this configuration and operation, the user's position within the vehicle can be easily and reliably determined, and control can be provided according to the user's position.

[0085] It should be noted that the present invention is not limited to the embodiments described above, and can be implemented by appropriately modifying the configuration and operation. For example, in this embodiment, a configuration that transmits and stores a facial image was illustrated for the sake of simplicity, but a configuration that transmits and stores feature quantities extracted from the facial image is also possible. Furthermore, instead of transmitting the entire interior image of the vehicle, a configuration that extracts and transmits a partial image corresponding to the seating position is also possible. Furthermore, the center of the rear seat can also be designated as one of the seating positions, and the rear seats may be arranged in two rows. Alternatively, data showing the correspondence between in-vehicle images and seating positions may be generated according to the vehicle and model, and stored by the in-vehicle device 40 and the server 10. Furthermore, the mobile terminal 30 may also have a pre-existing photograph of the user's face stored on it. Furthermore, the in-vehicle unit 40 may be configured to communicate with the server 10 using the communication function of the drive recorder. Furthermore, this can also be applied to interface integration with displays for the rear seats. In this case, the display would be duplicated for the front seats and extended for the rear seats. [Explanation of symbols]

[0086] 10…Server, 11…Communication Unit, 12…Storage Unit, 12a…Face Image Management Data, 13…Control Unit, 13a…Face Image Management Unit, 13b…Face Recognition Processing Unit, 13c…Seating Position Determination Unit, 20…Vehicle, 30…Mobile Terminal, 31…Touch Panel Display, 32…Mobile Terminal Camera, 33…In-Vehicle Device Communication Unit, 34…Server Communication Unit, 35…Storage Unit, 35a…Application Program, 35b…Mobile Terminal ID, 35c…In-Vehicle Device ID, 35d…Seating Position Data, 36…Control Unit, 36a…In-Vehicle Device Connection Unit, 36b…Face Image Transmission Unit, 36c…Command Transmission Unit, 40…In-Vehicle Device, 41…Touch Panel Display, 42…In-Vehicle Device Camera, 43…Mobile Terminal Communication Unit, 44…Server Communication Unit, 45…Storage Unit, 45a…In-Vehicle Device ID

Claims

1. An on-board device mounted on a vehicle, a mobile terminal communication unit that communicates with a mobile terminal located within the vehicle interior; a control unit that provides an interface for operating the vehicle-mounted device to the mobile terminal; Equipped with The control unit is characterized in that it provides a different interface to the mobile terminal depending on the seating position of the user determined by comparing an image of the interior of the vehicle taken inside the vehicle with a facial image of the user of the mobile terminal.

2. Further comprising a display unit, The control unit is If the seating position is a front seat, the mobile terminal is caused to display an expanded display content of the display unit; 2. The vehicle-mounted device according to claim 1, wherein if the seating position is other than a front seat, the portable terminal displays the same content as that displayed on the display unit.

3. A server communication unit that transmits the vehicle interior image to a server, The vehicle-mounted device according to claim 1, characterized in that the control unit provides a different interface to the mobile terminal depending on the seating position determined by the server by comparing the vehicle interior image with a facial image of the user of the mobile terminal.

4. 2. The vehicle-mounted device according to claim 1, wherein the control unit determines the seating position by comparing the facial image of the user received from the mobile terminal with the vehicle interior image.

5. 2. The vehicle-mounted device according to claim 1, wherein an image captured by a camera of a drive recorder is used as the vehicle interior image.

6. 2. The vehicle-mounted device according to claim 1, wherein the control unit provides different interfaces to the mobile terminal depending on a seating position of the user and user information associated with the user.

Images