Remote control device, operation method for remote control device, and program

Abstract

To increase convenience of users who share a vehicle.SOLUTION: A remote control device which remotely controls a vehicle includes: a storage unit that registers biological information of a user and identification information which is set in a terminal device of the user, in association with each other; a communication unit that obtains biological information of a first user who presents inside the vehicle; and an arithmetic unit that gives an authority for remote control of the vehicle, to identification information associated with the biological information of the first user among the biological information registered in the storage unit.SELECTED DRAWING: Figure 7A

Description

[Technical Field]

[0001] The present invention relates to a remote control device, a method for operating a remote control device, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology that identifies the last parked location of a vehicle to be remotely controlled, determines whether an operation requested by a user is an acceptable operation based on the last parked location, and rejects the operation requested by the user if it is not acceptable. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-231964 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not anticipate multiple people sharing a vehicle. For example, if multiple people share a vehicle using the technology of Patent Document 1, a situation may arise where one user's remote control of the vehicle becomes inaccessible, impairing the convenience of the other users. In sharing services and the like that have become increasingly popular in recent years, there is a demand for improved convenience for users who share a vehicle with multiple people.

[0005] In view of the above-mentioned problems, the present invention aims to provide technology that can improve the convenience of users who share a vehicle with multiple people. Furthermore, the present invention provides access to a safe, affordable, and easily accessible sustainable transportation system for all people by improving traffic safety through the expansion of public transportation, with particular consideration given to the needs of vulnerable people, women, children, persons with disabilities, and the elderly. One aspect of the present invention contributes to the development of a sustainable transportation system. [Means for solving the problem]

[0006] A remote control device according to one aspect of the present invention is a remote control device for remotely controlling a vehicle, the remote control device comprising: a storage unit for registering biometric information of a user in association with identification information set in a terminal device of the user; a communication means for acquiring biometric information of a first user present inside the vehicle; and a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means. 、 The storage means has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group, When the communication means acquires the biometric information of the second user after the biometric information of the first user is acquired, The computing means maintains the remote control authority assigned to the identification information associated with the terminal device of the first user and the remote control authority assigned to the identification information associated with the terminal device of the second user. do. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the convenience of users who share a vehicle. For example, since it can be inferred that the ability to board a vehicle means that the user has the authority to operate the vehicle, it is possible to improve the convenience of users by granting the authority to remotely operate the vehicle to a user authenticated by biometric authentication and allowing remote operation from outside the vehicle. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a vehicle remote control system. [Figure 2] FIG. 2 is a diagram showing the hardware configuration of a remote control device, a vehicle, and a terminal device. [Figure 3] FIG. 2 is a diagram illustrating an example of the operation of the vehicle remote control system. [Figure 4] FIG. 4 is a diagram illustrating an example of a screen displayed on a terminal device. [Figure 5] FIG. 3 is a diagram illustrating authority management information stored in a storage device of a remote control device. [Figure 6] 10 is a flowchart showing an example of the operation of the terminal device. [Figure 7A]A diagram that schematically explains the authorization of remote control (authorization). [Figure 7B] FIG. 7B is a diagram illustrating the flow of a modified example of the process of S705 in FIG. 7A. [Figure 8] 10 is a flowchart showing an example of the operation of the remote control device (registering fellow passengers as the same group). [Figure 9] 10 is a flowchart showing an example of the operation of the remote control device (output of operation instructions). [Figure 10] 10 is a flowchart showing an example of the operation of the remote control device (deletion of authority). [Figure 11] 10 is a flowchart showing an example of vehicle operation (display of usage code). [Figure 12] 10 is a flowchart showing an example of vehicle operation (transmission of identification information). [Figure 13] 1 is a flowchart showing an example of vehicle operation (operation based on instructions). DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0010] [System Overview] An example configuration of a vehicle remote control system 150 including a remote control device 100 according to an embodiment will be described with reference to Fig. 1. The remote control device 100 provides a user with a service for remotely controlling a vehicle 110 using a terminal device 120. Hereinafter, a service for remotely controlling a vehicle 110 using the terminal device 120 will be referred to as a remote control service. Remote control may be performed not only from a location remote from the vehicle 110, but also near the vehicle 110 or inside the vehicle 110.

[0011] In the remote control service, the remote control device 100 remotely controls the vehicle 110 in accordance with remote control instructions transmitted from the terminal device 120. The remote control device 100 may be capable of providing the remote control service to multiple users, each having their own terminal device 120. Although only two terminal devices 120 are shown in FIG. 1, the remote control device 100 may provide the remote control service to three or more users. Furthermore, the remote control device 100 may be capable of remotely controlling multiple vehicles 110. Although only two vehicles 110 are shown in FIG. 1, the remote control device 100 may be capable of remotely controlling three or more vehicles 110.

[0012] The remote control device 100, the vehicle 110, and the terminal device 120 are each connected to a wide area network 130 such as the Internet. The remote control device 100 may be connected to the wide area network 130 via a local area network (LAN) (not shown). The vehicle 110 may be connected to the wide area network 130 via a cellular network (not shown). The terminal device 120 may be connected to the wide area network 130 via a cellular network (not shown) or via a LAN (not shown). The remote control device 100 and the vehicle 110 can communicate with each other via the wide area network 130. The remote control device 100 and the terminal device 120 can communicate with each other via the wide area network 130. The vehicle 110 and the terminal device 120 may also be able to communicate with each other via the wide area network 130. Alternatively or additionally, the vehicle 110 and the terminal device 120 may be able to communicate with each other directly via short-range communication such as Bluetooth (registered trademark).

[0013] FIG. 2 is a diagram showing the hardware configuration of the remote control device 100, the vehicle 110, and the terminal device 120 in the vehicle remote control system 150 according to the embodiment.

[0014] (Remote control device 100) The remote control device 100 (server) performs processes related to the registration of user information, the granting of authorization for remote control, the determination of the presence or absence of authorization for remote control, and remote control instructions received from the terminal device 120. The remote control device 100 includes a processor 51, a storage device 52, and a communication device 53.

[0015] The processor 51 (arithmetic unit) is a general-purpose integrated circuit for executing operations by the remote control device 100. The processor 51 may be configured by, for example, a central processing unit (CPU). The processor 51 performs various processes related to the vehicle remote control service by reading and executing programs stored in the storage device 52. When the processor 51 receives a registration request for an account for using the remote control service from the user's terminal device 120 via the communication device 53, the processor 51 registers user information in the storage device 52. In response to the registration request from the user's terminal device 120, the processor 51 transmits (downloads) a predetermined remote control program 37 to the terminal device 120 via the communication device 53.

[0016] The storage device 52 is a device for storing information and software used in the operation of the remote control device 100. The storage device 52 may be configured with a hard disk drive (HDD) or a solid state drive (SSD). The storage device 52 stores user information registration and various information, for example, authority management information 57. The authority management information 57 is information for managing the authority to execute remote control of the vehicle 110. The authority to execute remote control of the vehicle 110 will be referred to as operation authority hereinafter. Specific examples of the authority management information 57 will be described later. The storage device 52 stores programs to be read and executed by the processor 51 and remote control programs to be provided to the user's terminal device 120. The storage device 52 can also store information obtained from the terminal device 120 and the vehicle 110 via the communication device 53.

[0017] The communication device 53 transmits and receives various information wirelessly. The communication device 53 is a device that enables the remote control device 100 to communicate with external devices (e.g., a router or a base station). The communication device 53 may be configured, for example, as a network card. The communication device 53 may perform wireless communication or wired communication. The communication device 53 may support short-range wireless communication such as Bluetooth (registered trademark). Furthermore, the communication device 53 may support cellular communication, WiFi (registered trademark) communication, etc. The communication device 53 mediates wireless communication between the control device 15 of the vehicle 110 and the communication device 36 of the terminal device 120.

[0018] (Configuration of vehicle 110) The vehicle 110 has a propulsion device 4, a braking device 5, a steering device 6, an external sensor 7, a vehicle sensor 8, a communication device 9, a navigation device 10, a driving operation device 11, a biometric information acquisition device 12, an interface device (HMI device) 13, and a control device 15.

[0019] The components of the vehicle 110 are connected to each other via an in-vehicle communication network such as a Controller Area Network (CAN) so that signals can be transmitted.

[0020] The propulsion device 4 is a device that applies driving force to the vehicle and includes, for example, a power source and a transmission. The power source has at least one of an internal combustion engine such as a gasoline engine and an electric motor.

[0021] The brake device 5 is a device that applies braking force to the vehicle, and includes, for example, a brake caliper that presses a pad against a brake rotor, and an electric cylinder that supplies hydraulic pressure to the brake caliper.

[0022] The steering device 6 is a device for changing the steering angle of the wheels, and includes, for example, a rack-and-pinion mechanism for steering the wheels and an electric motor for driving the rack-and-pinion mechanism. The steering device 6 includes an electric power steering device. The electric power steering device assists steering operations or includes a motor that generates driving force for automatically steering the front wheels, a sensor that detects the steering angle, and the like. The propulsion device 4, the brake device 5, and the steering device 6 are controlled by a control device 15.

[0023] The external sensor 7 is a sensor that detects objects around the vehicle, etc. The external sensor 7 includes a radar 16, a lidar (Light Detection and Ranging: LIDAR) 17, and a camera 18, and outputs the detection results to the control device 15.

[0024] The radar 16 is, for example, a millimeter wave radar, and is capable of detecting objects around the vehicle 110 and measuring the distance to the objects using radio waves. A plurality of radars 16 are provided around the vehicle 110, for example, one radar 16 is provided in the front center of the vehicle, one at each front corner, and one at each rear corner.

[0025] The LIDAR 17 is capable of detecting objects around the vehicle 110 by light and measuring the distance to the objects. A plurality of LIDARs 17 are provided around the vehicle, for example, one LIDAR 17 is provided at each corner of the front of the vehicle, one LIDAR 17 is provided in the center of the rear, and one LIDAR 17 is provided on each side of the rear.

[0026] Camera 18 is a device that captures images of the surroundings of vehicle 110, and is, for example, a digital camera that uses a solid-state image sensor such as a CCD or CMOS. Camera 18 includes a front camera that captures images in front of vehicle 110 and a rear camera that captures images behind vehicle 110.

[0027] The vehicle sensor 8 includes a vehicle speed sensor that detects the speed of the vehicle 110, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and a direction sensor that detects the orientation of the vehicle 110. The yaw rate sensor is, for example, a gyro sensor. The vehicle sensor 8 may also include a sensor that detects the remaining fuel or battery charge for driving the propulsion device 4, and a sensor that detects the operating state of the brake device 5, etc. The vehicle sensor 8 may also include a sensor that detects the lock state (locked or unlocked) of the doors and a sensor that detects the open / closed state of the windows. The detection results of the vehicle sensor 8 are transmitted to the terminal device 120 and the remote control device 100 via the communication device 9. In the terminal device 120, the detection results of the vehicle sensor 8 may be displayed on a UI screen of the remote control program 37 in the input / output device 30. In the remote control device 100, the processor 51 may perform various processes, such as authorization, using the detection results of the vehicle sensor 8.

[0028] The communication device 9 provides a function for the vehicle 110 to communicate with the outside. The communication device 9 may support short-range wireless communication such as Bluetooth (registered trademark). Furthermore, the communication device 9 may support cellular communication, WiFi (registered trademark) communication, etc. The communication device 9 mediates wireless communication between the control device 15 of the vehicle 110 and the terminal device 120 and the remote control device 100. That is, the control device 15 can communicate with the terminal device 120 and the remote control device 100 carried by the user via the communication device 9 using a communication method such as infrared communication or Bluetooth (registered trademark).

[0029] The navigation device 10 acquires the current position of the vehicle and provides route guidance to the destination, and includes a GNSS receiver 20 and a map storage unit 21. The GNSS receiver 20 identifies the vehicle's position (latitude and longitude) based on signals received from artificial satellites (positioning satellites). The map storage unit 21 is configured with a storage device such as a flash memory or a hard disk, and stores map information.

[0030] The driving operation device 11 is installed inside the vehicle cabin and accepts input operations performed by the user to control the vehicle. The driving operation device 11 includes, as driving operation units, for example, a steering wheel, an accelerator pedal, a brake pedal, a parking brake device, a shift lever, and a push start switch (engine start button). The push start switch accepts input operations to start the vehicle by driving operations from the user. The driving operation device 11 includes a sensor that detects the amount of operation and outputs a signal indicating the amount of operation to the control device 15.

[0031] The biometric information acquisition device 12 acquires biometric information of a user present inside the vehicle 110. The biometric information acquisition device 12 is, for example, a camera (indoor camera) provided inside the cabin of the vehicle 110, and acquires an image including the face of a user present inside the vehicle 110 (in-vehicle user). The biometric information acquisition device 12 is capable of acquiring images including the faces of users seated in the front seats (driver's seat and passenger seat) of the vehicle 110 and users seated in the rear seats located behind the front seats. The biometric information acquisition device 12 outputs the acquired biometric information to the control device 15. Note that the biometric information acquisition device 12 may be any of various detection devices that detect one or more of a user's fingerprint, retina, and iris as biometric information.

[0032] The interface device 13 (HMI device) provides an interface (HMI: Human Machine Interface) between the control device 15 and the user, notifies the user of various information by displaying and outputting audio, and accepts input operations by the user. The interface device 13 has a display device 23 made of liquid crystal, organic EL, or the like, and functions as a touch panel that can accept input operations from the user, and an input device 24 such as buttons and switches.

[0033] The control device 15 is an electronic control unit (ECU) including a CPU, a non-volatile memory (ROM), a volatile memory (RAM), etc. The control device 15 can perform various vehicle controls by executing arithmetic processing based on a program in the CPU. At least some of the functional units of the control device 15 may be realized by hardware such as an LSI, an ASIC, or an FPGA, or may be realized by a combination of software and hardware.

[0034] The control device 15 includes an external environment recognition unit 41, a position identification unit 42, a user recognition unit 43, a vehicle control unit 44, and a storage device 45.

[0035] The external environment recognition unit 41 acquires information about other vehicles and obstacles present around the vehicle 110 based on the detection results of the external environment sensor 7. For example, the external environment recognition unit 41 acquires information about the positions and sizes of other vehicles and obstacles, as well as information about traffic light displays, lane markings on roads, and markings (white lines, etc.) painted on the road surface of parking lots, etc. The external environment recognition unit 41 also analyzes images acquired by the camera 18 based on an image analysis method such as pattern matching to acquire the presence and size of other vehicles and obstacles. Furthermore, the external environment recognition unit 41 can calculate the distance to an obstacle using signals from the radar 16 and the lidar 17 and acquire the position of the obstacle.

[0036] The position determining unit 42 is capable of detecting the position of the vehicle based on a signal from the GNSS receiving unit 20 of the navigation device 10. In addition to the signal from the GNSS receiving unit 20, the position determining unit 42 is also capable of acquiring the vehicle speed and yaw rate from the vehicle sensor 8 and determining the position and attitude of the vehicle using so-called inertial navigation.

[0037] The user recognition unit 43 analyzes the biometric information (e.g., a facial image) acquired by the biometric information acquisition device 12 based on an image analysis method such as feature extraction processing, and acquires facial information by extracting the facial features of the user from the facial image acquired by the analysis. When the biometric information acquisition device 12 acquires facial images including the faces of multiple users present inside the vehicle 110, the user recognition unit 43 acquires multiple pieces of facial information by extracting the facial features of each of the multiple users from the facial images as biometric information.

[0038] The processor 51 of the remote control device 100 refers to the storage device 52 and determines whether or not the authorization for remote control is given to the identification information associated with the terminal device 120. If the authorization for remote control is given to the identification information, the processor 51 transmits a remote control signal from the terminal device 120 to the vehicle 110.

[0039] The vehicle control unit 44 controls the vehicle 110 in accordance with remote operation instructions. For example, it controls locking (door locking) and unlocking (door unlocking) the doors of the vehicle 110, starting up a power system that supplies power to each part of the vehicle 110, starting up the propulsion device 4, etc. For example, upon receiving a remote operation instruction to unlock the doors, the vehicle control unit 44 unlocks the doors. Furthermore, upon receiving a remote operation instruction to start up the propulsion device 4, the vehicle control unit 44 starts up the propulsion device 4. If the propulsion device 4 is an internal combustion engine, the vehicle control unit 44 turns on an ignition device. Furthermore, the vehicle control unit 44 controls predetermined operations of the vehicle 110 based on the remote operation instructions of the driving operation device 11.

[0040] The storage device 45 is configured with a flash memory, a hard disk, etc., and stores various programs executed by the control device 15 and various parameters required for executing the programs. The storage device 45 also registers various information obtained from the remote operation device 100 via the communication device 9. The various information includes user information registered in the authority management information 57 obtained from the remote operation device 100.

[0041] (Terminal device 120) The terminal device 120 is a device that can be carried by a user and is capable of communicating with the remote control device 100 and the control device 15 of the vehicle 110. The remote control program 37 provides various functions for remotely controlling the vehicle. The terminal device 120 may be a mobile device such as a smartphone. As shown in FIG. 2 , the terminal device 120 has an input / output device 30, an imaging device 31, a processor 33, a storage device 35, a communication device 36, and a remote control program 37. The remote control program 37 is a program obtained from the remote control device 100. When the remote control program 37 is executed in the terminal device 120, The input / output device 30 presents information to a user operating the terminal device 120 and accepts input from the user operating the terminal device 120. The input / output device 30 functions as, for example, a touch panel, and upon accepting input from the user, outputs a signal corresponding to the input to the processor 33. The input / output device 30 functions as a UI (user interface) for various functions related to vehicle remote control provided by the remote control program 37. Operation input to the remote control program 37 and display of information processed by the remote control program 37 can be performed via the input / output device 30.

[0042] The imaging device 31 can capture images (still images and moving images) in an imaging mode set by the input / output device 30, and the imaging device 31 is, for example, a digital camera configured with a CMOS or the like. The processor 33 can acquire facial image features as biometric information of the user by performing predetermined image processing on an image captured of the user operating the terminal device 120. When the user registers an account for using a remote control service with the remote control device 100, the user may transmit biometric information together with a registration request transmitted from the terminal device 120 to the remote control device 100, and the remote control device 100 may associate the identification information (user ID or terminal ID) with the biometric information and register the biometric information in the storage device 52.

[0043] The processor 33 controls various processes in the terminal device 120. The processor 33 executes programs stored in the storage device 35 to realize various functions related to vehicle remote control using the remote control program 37. The processor 33 controls the display screen of the terminal device 120, and controls, for example, the display of a UI corresponding to various functions related to vehicle remote control provided by the remote control program 37.

[0044] The storage device 35 is a device for storing information and software used in the operation of the terminal device 120. The storage device 52 may be configured with a hard disk drive (HDD) or a solid state drive (SSD). The storage device 52 stores, for example, a remote operation program 37.

[0045] The communication device 36 provides a function for the terminal device 120 to communicate with the outside. The communication device 36 may support short-range wireless communication such as Bluetooth (registered trademark). Furthermore, the communication device 36 may support cellular communication, Wi-Fi (registered trademark) communication, etc.

[0046] (Example of system operation) An example of the operation of the vehicle remote control system 150 including the remote control device 100 according to the embodiment will be described with reference to Fig. 3. Before executing this operation, it is assumed that the user has registered an account for using the remote control service with the remote control device 100 and has installed the remote control program 37 in the user's own terminal device 120. It is also assumed that the user has registered in the remote control program 37 the user ID of the account for using the remote control service.

[0047] In S300, the user pairs the communication device 9 of the vehicle 110 with the communication device 36 of the terminal device 120. This enables the communication device 9 of the vehicle 110 and the communication device 36 of the terminal device 120 to communicate with each other. The terminal device 120 may acquire the vehicle ID of the vehicle 110 during pairing, and may transmit this vehicle ID to the remote control device 100 together with its own terminal ID or a user ID registered in the remote control program 37. The remote control device 100 may store this vehicle ID as the vehicle ID of the vehicle to be remotely controlled by the terminal device 120 (in column 503 of the authority management information 57, which will be described later).

[0048] In S301, the biometric information acquisition device 12 acquires biometric information of a user present inside the vehicle 110. The biometric information acquisition device 12 is, for example, a camera (indoor camera) provided inside the vehicle 110, and acquires an image including the face of a user (for example, a first user) present inside the vehicle 110.

[0049] The communication device 9 of the vehicle 110 transmits the biometric information of the user (first user) acquired by the biometric information acquisition device 12 to the remote control device 100.

[0050] In S306, the communication device 53 of the remote operation device 100 communicates with the vehicle 110 to acquire biometric information of the first user present inside the vehicle 110. The remote operation device 100 determines whether the biometric information of the first user matches the biometric information registered in the authority management information 57. If the biometric information of the first user matches the biometric information registered in the authority management information 57, the remote operation device 100 grants authority for remote operation, on the prerequisite that identification information (user ID, terminal ID) associated with the terminal device 120 of the first user is registered in the authority management information 57. In this case, the user (first user) can omit the operations of S302 and S305 using the terminal device 120.

[0051] If the biometric information of the first user does not match the biometric information registered in the authority management information 57, or if the biometric information is not registered, the user acquires operation authority for performing remote operation by processing using the usage code (S302 to S306). When the usage code presented by the interface device 13 inside the vehicle 110 matches the usage code transmitted from the user's terminal device 120, the remote operation device 100 grants remote operation authority to the identification information associated with the first user's terminal device, which is registered in the storage device 52.

[0052] In steps S302 to S306, the user acquires operation authority to execute remote control. Specifically, in step S302, the user uses the remote control program 37 to request the remote control device 100 to issue a usage code. The usage code is information that must be transmitted to the remote control device 100 to acquire operation authority to execute remote control.

[0053] In S303, the remote control device 100 generates a usage code in response to the user's request and transmits it to the vehicle 110. The remote control device 100 may, for example, randomly generate a four-digit number as the usage code. The remote control device 100 associates the generated usage code with identification information associated with the terminal device 120 that requested the issuance of the usage code and stores them in the storage device 52 for use in a later step.

[0054] In S304, the vehicle 110 presents the usage code using the display device 23 in accordance with the user's operation using the input device 24. The presentation of the usage code may be by displaying the usage code on the display device 23, by playing a sound representing the usage code from the speaker of the interface device 13, or by a combination of these. The presentation of the usage code requires operation of the input device 24. Therefore, the user needs to unlock the vehicle 110 using the key of the vehicle 110 to obtain the usage code.

[0055] In S305, the user uses the remote operation program 37 to transmit the usage code displayed on the display device 23 to the remote operation device 100. In S306, the remote operation device 100 grants the terminal device 120 remote operation authority to perform remote operation, on the condition that the usage code transmitted in S305 matches the usage code created in S303 and that the usage code is received in S305 within a predetermined time (for example, within five minutes) after the usage code is created in S303. A validity period may be set for the operation authority.

[0056] In S306, it is also possible to grant authority to remotely control the vehicle 110 using the user's biometric information transmitted in S301. For example, the storage device 52 registers the user's biometric information in association with identification information set in the user's terminal device 120. The communication device 53 may acquire biometric information of a first user present inside the vehicle 110 from the vehicle 110, and the remote control device 100 may grant authority to remotely control the vehicle 110 to identification information associated with the first user's biometric information among the biometric information registered in the storage device 52.

[0057] The remote operation device 100 may grant operation authority to each individual terminal device 120, or may grant operation authority to the terminal device 120 on a user-by-user basis. Assume that a user has multiple terminal devices 120. In this case, the remote operation device 100 may grant operation authority to the terminal device 120 used to transmit the usage code among the multiple terminal devices 120, but not to the other terminal devices 120. The remote operation device 100 may grant operation authority not only to the terminal device 120 used to transmit the usage code among the user's multiple terminal devices 120, but also to the other terminal devices 120. In this case, it may be considered that the remote operation device 100 grants operation authority to the user. In the following description, a terminal device 120 that has been granted operation authority may be referred to as a terminal device 120 having operation authority. Furthermore, a user of a terminal device 120 that has been granted operation authority may be referred to as a user having operation authority. In the following description, a case will be described in which operation authority is granted to each individual terminal device 120, but instead, operation authority may be granted to the user of each individual terminal device 120. As in granting operation authority, the remote operation device 100 may revoke the operation authority of each individual terminal device 120, or may revoke the operation authority of the terminal device 120 on a user-by-user basis.

[0058] In S307, the user uses the remote operation program 37 of the terminal device 120 for which the user has the operation authority to transmit a remote operation instruction for the vehicle 110 to the remote operation device 100. A specific example of the remote operation instruction will be described later. In S308, the remote operation device 100 confirms that the terminal device 120 that transmitted the remote operation instruction has the operation authority.

[0059] In S309, the remote control device 100 transmits a remote control instruction to the vehicle 110 in order to remotely operate the vehicle 110 in accordance with the remote control instruction from the user. The remote control device 100 outputs a signal for operating the vehicle 110 based on the remote control request received from the terminal device 120 having identification information that has been granted authority to perform remote control.

[0060] In S310, the vehicle 110 executes the operation instructed by the remote control device 100. By repeating the operations of S307 to S310, the user can repeatedly instruct remote operation as long as the operation authority is valid.

[0061] In S311, the remote operation device 100 cancels the operation authority in accordance with a remote operation instruction from the user. In S311, the remote operation device 100 may cancel the operation authority granted to the terminal device 120 in response to the expiration of the validity period of the operation authority. By periodically canceling the operation authority in this manner, it is possible to prevent unauthorized use of the remote operation service. The remote operation device 100 may update the validity period of the operation authority when S302 to S306 are executed within the validity period.

[0062] (Screen display of terminal device 120) 4, an example of a screen displayed on the input / output device 30 of the terminal device 120 by the remote operation program 37 will be described. In addition to the screen 400 and the screen 410, the input / output device 30 may display, for example, a menu screen for transitioning to these screens.

[0063] 4 shows an example of a screen 400 used to issue and transmit a usage code. The screen 400 includes an issue request button 401 for requesting the issuance of a usage code in S302 described above. When the user selects (e.g., presses) the issue request button 401 using the input / output device 30, a request to issue a usage code is transmitted to the remote operation device 100. The address of the remote operation device 100 may be set in advance in the remote operation program 37.

[0064] The screen 400 further includes a field 402 for inputting a usage code, and a usage code send button 403 for transmitting the usage code inputted in the field 402 to the remote operation device 100. When the user inputs the usage code in the field 402 using the input / output device 30 and selects (e.g., presses) the usage code send button 403, the usage code inputted in the field 402 is transmitted to the remote operation device 100. In the example of ST41 in Fig. 4, the issue request button 401, the field 402, and the usage code send button 403 are included in the same screen 400, but these objects may be distributed and arranged on multiple screens.

[0065] 4 shows an example of a screen 410 used for remotely controlling the vehicle 110. The screen 410 includes a button 411 for unlocking the doors of the vehicle 110. When the user selects (e.g., presses) the button 411 using the input / output device 30, the doors of the vehicle 110 are unlocked.

[0066] The screen 410 further includes a button 412 for activating the power supply or propulsion device 4 of the vehicle 110. When the user selects (e.g., presses) the button 412 using the input / output device 30, the power supply or propulsion device 4 of the vehicle 110 is turned on (e.g., power on state, ignition on state). This allows the user to start the vehicle 110. To further improve security, the remote control program 37 may require the user to enter an activation code presented by the display device 23 of the vehicle 110 after the button 412 is pressed.

[0067] The screen 410 further includes a button 413 for starting the air conditioner of the vehicle 110. The air conditioner of the vehicle 110 is started when the user selects (e.g., presses) the button 413 using the input / output device 30. The remote operation program 37 may provide a remote operation for changing the setting (e.g., temperature) of the air conditioner of the vehicle 110. In addition, the remote operation program 37 may provide a remote operation for operating a defroster.

[0068] The screen 410 further includes a button 414 for changing the state of the vehicle 110 to make it easier to find the vehicle 110. When the user selects (e.g., presses) the button 414 using the input / output device 30, for example, the lights of the vehicle 110 flash for a predetermined period of time.

[0069] As described above, when the remote operation program 37 includes an unlock operation and a power-on operation as remote operations, the user can start the vehicle 110 using the terminal device 120 instead of a key for the vehicle 110. Therefore, the terminal device 120 can function as a digital key. The remote operation program 37 may include operations other than those described in ST42 of FIG. 4 as remote operations, or may not include some or all of the operations described in ST42 of FIG. 4. In the example of ST42 of FIG. 4, buttons 411 to 414 are included on the same screen 410, but these objects may be distributed and arranged on multiple screens.

[0070] (Permissions Management Information) 5 is a diagram illustrating an example of authority management information 57 stored in the storage device 52 of the remote operation device 100. The storage device 52 registers user identification information, information on the group to which the user belongs, biometric information, etc. in the authority management information 57. In FIG. 5, the operation authority of each user is managed in a table format. Alternatively, the operation authority of each user may be managed in another format. In the authority management information 57, a record is generated for each user who has registered an account for using the remote operation service in the remote operation device 100.

[0071] Column 500 (user ID) represents identification information (i.e., user ID) for uniquely identifying a user. The identification information (user ID) may be the account name of an account for the user to use the remote operation service. The account name may be a character string specified by the user (e.g., an email address), or a unique character string generated by the remote operation device 100. When a user registers an account, the remote operation device 100 adds a record to the authority management information 57 and registers the user ID in column 500 of that record.

[0072] Column 501 (terminal ID) indicates identification information (i.e., terminal ID) for uniquely identifying the terminal device 120 used by the user. The terminal ID may be, for example, the address of the communication device 36 (BD address in the case of Bluetooth). The terminal ID may be the serial number of the terminal device 120. The terminal ID may be a character string uniquely assigned to the terminal device 120 by the remote operation device 100. When the user inputs an account name, the remote operation program 37 of the terminal device 120 may transmit the user ID of the terminal device 120 to the remote operation device 100. The remote operation device 100 registers the received terminal ID in column 501. When the remote operation device 100 assigns a terminal ID to the terminal device 120, the remote operation device 100 registers the terminal ID in column 501 and transmits the terminal ID to the terminal device 120. The terminal device 120 stores the received terminal ID for use in subsequent processing.

[0073] A user may be able to register only one terminal device 120 to be used for remote control, or may be able to register multiple terminal devices 120. When a user registers multiple terminal devices 120, multiple terminal IDs are registered in column 501.

[0074] Column 502 (Vehicle ID) shows identification information (i.e., vehicle ID) for uniquely identifying the vehicle that the user remotely controls. The vehicle ID may be, for example, an IP address assigned to the communication device 9 of the vehicle 110. The vehicle ID may be stored in the vehicle 110. The terminal device 120 may acquire the vehicle ID from the vehicle 110 when performing pairing in S300 of FIG. 3, and transmit this vehicle ID to the remote control device 100 together with a request for issuing a usage code in S302. The remote control device 100 registers the received vehicle ID in column 502.

[0075] A user may be able to register only one vehicle 110 to be remotely controlled, or may be able to register multiple vehicles 110. When a user registers multiple vehicles 110, multiple vehicle IDs are registered in column 502. When multiple vehicle IDs are registered, remote control device 100 may acquire the vehicle ID of the vehicle to be remotely controlled along with a remote control instruction.

[0076] Column 503 (operation authority) indicates whether or not the user has operation authority to perform remote operation. When remote operation authority is granted, "operation authority granted" is set in column 503. Operation authority may include multiple operation authorities with different scopes of authority. Furthermore, the scope of operation authority may change depending on whether or not the vehicle 110 is in a traveling state.

[0077] Column 504 (group ID) indicates identification information (i.e., group ID) for identifying the group to which each user belongs. When a user registers an account for using the remote operation service, the remote operation device 100 can add a record to the authority management information 57 and register the group ID to which the user belongs.

[0078] Column 505 shows biometric information of a user. The biometric information is image information including the user's face, and for example, when registering identification information (user ID or terminal ID), image information including the user's face may be registered as biometric information.

[0079] The image information including the user's face may be image information captured by the imaging device 31 of the terminal device 120. Also, image information including the user's face acquired by the biometric information acquisition device 12 inside the vehicle 110 may be registered as biometric information.

[0080] (Example of operation of terminal device 120) Fig. 6 is a flowchart showing an example of the operation of the terminal device 120. An example of the operation performed by the terminal device 120 will be described with reference to Fig. 6. Each step in Fig. 6 may be performed by the processor 33 executing a computer program stored in the storage device 35. Alternatively, some or all of the multiple steps in Fig. 6 may be performed by a dedicated integrated circuit such as an ASIC. In such a case, the dedicated integrated circuit may operate as a functional unit for performing each step.

[0081] In S601, the terminal device 120 determines whether the user has started using the remote operation program 37. If it is determined that the user has started using the remote operation program 37 ("YES" in S601), the terminal device 120 transitions the process to S602, and otherwise ("NO" in S601), repeats S601. The terminal device 120 may determine that the user has started using the remote operation program 37 when the remote operation program 37 is started or when the remote operation program 37 is changed from background to active.

[0082] In S602, the terminal device 120 transmits an inquiry to the remote operation device 100 as to whether the terminal device 120 has operation authority. In S603, the terminal device 120 determines whether the terminal device 120 has operation authority based on the result of the inquiry. If the terminal device 120 determines that the terminal device 120 has operation authority ("YES" in S603), the terminal device 120 transitions the process to S604, and otherwise ("NO" in S603), the terminal device 120 transitions the process to S607.

[0083] For example, if the biometric information of the user in the vehicle 110 matches the biometric information registered in the authority management information 57 and identification information (user ID or terminal ID) associated with the user's terminal device 120 is registered, the process proceeds to S604 (YES in S603). On the other hand, if the user's biometric information does not match the biometric information registered in the authority management information 57 (NO in S603), the process proceeds to S607, and the user obtains operation authority to perform remote operation through processing using the usage code (S607 to S611).

[0084] If the terminal device 120 has the operation authority, the terminal device 120 performs remote operation of the vehicle 110 through S604 to S606. In S604, the terminal device 120 uses the output device 404 to display a screen for performing remote operation (for example, screen 410 of ST42 in FIG. 4).

[0085] In S605, the terminal device 120 determines whether a remote operation instruction has been received from the user using the input / output device 30. If it is determined that a remote operation instruction has been received from the user (YES in S605), the terminal device 120 transitions the process to S605, and otherwise (NO in S605) repeats S605. The remote operation instruction may be, for example, pressing any of the buttons 411 to 414 in ST42 of FIG. 4. In S606, the terminal device 120 transmits the remote operation instruction received from the user to the remote operation device 100.

[0086] If the terminal device 120 does not have the operation authority, the terminal device 120 acquires the operation authority for the vehicle 110 through steps S607 to S611. In step S607, the terminal device 120 uses the input / output device 30 to display a screen (for example, the screen 400 in ST41 in FIG. 4) for requesting the issuance of a usage code.

[0087] In S608, the terminal device 120 determines whether an issuance instruction has been received from the user using the input / output device 30. An issuance instruction is an instruction to request the issuance of a usage code. If it is determined that an issuance instruction has been received from the user ("YES" in S608), the terminal device 120 transitions the process to S609, and otherwise ("NO" in S608), the terminal device 120 repeats S608. The issuance instruction may be, for example, pressing the issuance request button 401 in ST41 of FIG. 4. In S609, the terminal device 120 transmits a request for issuance of a usage code to the remote operation device 100.

[0088] In S610, the terminal device 120 determines whether or not a code transmission instruction has been received from the user using the input / output device 30. A code transmission instruction is an instruction to transmit a usage code. If it is determined that a code transmission instruction has been received from the user ("YES" in S610), the terminal device 120 transitions the process to S611, and otherwise ("NO" in S610), the terminal device 120 repeats S610. The usage code instruction may be, for example, pressing the issuance request button 401 in ST41 of FIG. 4. In S611, the terminal device 120 transmits the usage code entered in the field 402 to the remote operation device 100.

[0089] The data transmitted from the terminal device 120 to the remote operation device 100 may include identification information associated with the terminal device 120. The identification information associated with the terminal device 120 may be a user ID or a terminal ID. The user ID may be the same type as the user ID described in column 500 of FIG. 5. The terminal ID may be the same type as the terminal ID described in column 501 of FIG. 5.

[0090] (Example of operation of the remote control device 100) 7A to 10 are flowcharts showing an example of the operation of the remote control device 100. Each step in FIGS. 7A to 10 may be performed by the processor 51 executing a computer program stored in the storage device 52. Alternatively, some or all of the steps in FIGS. 7A to 10 may be performed by a dedicated integrated circuit such as an ASIC. In such a case, the dedicated integrated circuit may operate as a functional unit for executing each step. Note that the example of the operation of the remote control device 100 described in FIGS. 7A to 10 may be executed by the control device 15 of the vehicle 110.

[0091] (Remote control authorization) Fig. 7A is a flowchart showing an example of the operation of the remote operation device 100. An example of the operation of the remote operation device 100 regarding authorization for remote operation will be described with reference to Fig. 7A.

[0092] In S700, the remote operation device 100 determines whether the biometric information of the user (first user) transmitted from the vehicle 110 in S301 matches the biometric information registered in the authority management information 57. If the biometric information of the user (first user) matches the biometric information registered in the authority management information 57 ("YES" in S700), the remote operation device 100 proceeds to S702. If the biometric information does not match ("NO" in S700), the remote operation device 100 proceeds to S701.

[0093] In S701, the remote control device 100 determines whether the usage code created in S303 matches the usage code sent in S305, and if they match ("YES" in S701), it transitions the processing to S702; otherwise ("NO" in S701), it terminates the processing related to authorization.

[0094] In S702, the remote operation device 100 determines whether the identification information (user ID or terminal ID) included in the usage code transmitted in S305 is registered in the authority management information 57. If it is registered ("YES" in S702), the process proceeds to S703, and otherwise ("NO" in S702), the process related to authorization is terminated. Note that the identification information (user ID or terminal ID) may be obtained inside the vehicle 110 by the communication device 9 from the terminal device 120 of the user inside the vehicle and transmitted to the remote operation device 100.

[0095] In S703, the remote operation device 100 determines whether a group is registered for the identification information (user ID or terminal ID) to which authorization is to be granted, by referring to the authority management information 57. Specifically, the remote operation device 100 determines whether a group is registered by referring to the column 504 of the authority management information 57.

[0096] If group registration has been performed ("YES" in S703), the remote operation device 100 proceeds to S704, and the remote operation device 100 grants remote operation authority to the identification information (user ID or terminal ID) registered in the group. For example, in ST51 of Fig. 5, the remote operation device 100 grants remote operation authority to the identification information of user A (user ID=UA) and user B (user ID=UA) who are registered in the same group 1 (ST51 of Fig. 5).

[0097] When the remote control device 100 grants remote control authority to the identification information associated with the terminal device of the user (first user) by comparing the usage codes, the storage device 52 registers the biometric information of the user (first user) acquired by the communication device 53 in association with the identification information associated with the terminal device of the user (first user).

[0098] The target of the authorization may be, for example, the current operator (user) or the last operator (user) at the time of starting up the vehicle 110, and the authorization for remote operation may be granted to the identification information of a user belonging to the same group as the current operator or the last operator. For example, when the propulsion device 4 of the vehicle 110 starts or stops, the communication device 9 of the vehicle 110 may acquire identification information associated with the terminal device 120 of the first user inside the vehicle 110 and transmit the acquired information to the communication device 53. The communication device 53 may acquire, via the communication device 9 of the vehicle 110, identification information associated with the terminal device 120 of the first user present inside the vehicle 110 when the vehicle 110 starts or stops. The processor 51 of the remote operation device 100 may register the identification information acquired by the communication device 53 in the storage device 52 and grant the authorization for remote operation to the identification information registered in the storage device 52.

[0099] The storage device 52 has a function of registering multiple users as a group, and can register, for example, a first user (e.g., user ID=UA in ST51 in FIG. 5) and a second user (e.g., user ID=UB in ST51 in FIG. 5) as group 1. When the communication device 53 acquires the biometric information of the second user after the first user, the remote operation device 100 maintains the remote operation authority granted to the identification information associated with the terminal device of the first user and the remote operation authority granted to the identification information associated with the terminal device of the second user.

[0100] On the other hand, if it is determined in S703 that group registration has not been performed ("NO" in S703), the process proceeds to S705, where the remote operation device 100 grants remote operation authority to the identification information determined to be registered in S702, and ends the process. In this case, the remote operation device 100 deletes the remote operation authority of the first user, and grants remote operation authority to the identification information associated with the terminal device 120 of a new user, a third user (for example, user ID=UC in ST52 of FIG. 5).

[0101] When a first user who has never driven a specific vehicle 110 before drives the vehicle 110, such as when the vehicle 110 is resold, the previous remote operation authority may be deleted. For example, when the communication device 53 acquires biometric information of a third user after the first user, the remote operation device 100 deletes the remote operation authority granted before the new user, the third user. The remote operation device 100 deletes the remote operation authority granted to the identification information associated with the terminal device of the first user. If the first user is registered in a group, information about the group ID to which the first user belongs may be deleted. Furthermore, the biometric information of the first user may be deleted together with the deletion of the remote operation authority. Then, the remote operation device 100 grants remote operation authority to the identification information associated with the terminal device of the new user, the third user (for example, user ID=UC in FIG. 3) (ST52 in FIG. 5).

[0102] (Variation) Fig. 7B is a diagram illustrating the flow of a modification of the process of S705 in Fig. 7A. In the modification of Fig. 7B, the remote control device 100 may switch between maintaining the granted authority and granting the authority to a new user depending on whether the location where the biometric information was acquired is a parking area where the vehicle 110 is normally parked (a normal parking area).

[0103] The position identifying unit 42 of the vehicle 110 identifies the position of the vehicle 110 based on signals from the GNSS receiving unit 20 of the navigation device 10 and signals from the vehicle sensor 8. If the vehicle remains stopped for a predetermined period of time, the position identifying unit 42 determines that the vehicle is parked and identifies the position of the vehicle 110 as a parking position. The parking position identified by the position identifying unit 42 is transmitted to the remote control device 100 by the communication device 9. Date and time information identifying the date and time of parking may be transmitted together with the parking position. Furthermore, when biometric information is acquired by the biometric information acquisition device 12, the communication device 9 may associate the biometric information with the parking position identified by the position identifying unit 42 and the date and time information and transmit them to the remote control device 100.

[0104] The communication device 53 acquires, from the vehicle 110, the biometric information, the parking position of the vehicle 110, and date and time information that specifies the parking date and time.

[0105] The remote control device 100 obtains the parking frequency for each parking position of the vehicle 110 during a specified period based on information on the parking position of the vehicle 110 and the parking date and time, and identifies the parking position with the highest parking frequency as the parking area where the vehicle 110 is normally parked (normal parking area).

[0106] Here, the parking frequency refers to the number of times the vehicle is parked at each parking position in a predetermined period (for example, one month), such as 4 times at parking position A, 20 times at parking position B, and 6 times at parking position C. The parking frequency may also be the cumulative (total) amount of time the vehicle has been parked at each parking position.

[0107] In S711, the remote control device 100 determines whether the location where the biometric information was acquired is within the normal parking area. If the location where the biometric information was acquired is not within the normal parking area (NO in S711), the process proceeds to S713, where the remote control device 100 deletes the remote control authority granted to the identification information associated with the biometric information of the first user. Then, the remote control device 100 grants the remote control authority to the identification information associated with the biometric information acquired at a location outside the normal parking area.

[0108] For example, the remote control device 100 deletes the remote control authority of the first user who was previously granted the remote control authority, and grants the remote control authority to the identification information associated with the terminal device 120 of the new user, the third user (for example, user ID=UC in ST52 of Figure 5).

[0109] On the other hand, if it is determined in S711 that the location where the biometric information was acquired is within the normal parking area (YES in S711), the process proceeds to S712, where the remote operation device 100 maintains the remote operation authority assigned to the identification information associated with the biometric information of the first user. For example, the remote operation device 100 assigns (maintains) the remote operation authority to the identification information associated with the terminal device 120 of the first user (user ID=UA in ST51 of FIG. 5), who is the user to whom the remote operation authority was first assigned.

[0110] The use in the case where the location where the biometric information was acquired is within the normal parking area may be, for example, use in the case where the vehicle 110 is shared by a family. In such a case, when new biometric information is acquired, the previously granted remote control authority is not deleted, and the authority of the user who was previously granted remote control authority is maintained. This improves convenience when the vehicle 110 is shared by multiple users. Furthermore, by maintaining the authority of the user who was granted remote control authority, it becomes possible to remotely restrict the operation of the vehicle 110 even if the vehicle 110 is stolen, for example, thereby improving the security of the vehicle 110.

[0111] (Register passengers as the same group) Fig. 8 is a flowchart showing an example of the operation of the remote control device 100. An example of the operation of the process executed by the remote control device 100 to register fellow passengers of the vehicle 110 as part of the same group will be described with reference to Fig. 8.

[0112] In S800, the communication device 53 receives biometric information from the vehicle 110. The communication device 53 acquires a plurality of face images including the faces of a plurality of users present inside the vehicle 110.

[0113] In S801, the remote control device 100 determines whether the facial information of each of the multiple users is registered as biometric information in the storage device 52. If the facial information is not registered as the user's biometric information in the storage device 52 ("NO" in S801), the process ends. In this case, the user acquires operation authority to execute remote control by processing using the usage code (S302 to S306).

[0114] On the other hand, if it is determined in S801 that face information is registered in the storage device 52 as biometric information of the user (YES in S801), the process proceeds to S802.

[0115] In S802, the remote operation device 100 registers a plurality of users in the storage device 52 as the same group.

[0116] In S803, the remote control device 100 grants remote control authority to the identification information (user ID or terminal ID) registered in the group. When there are multiple passengers inside the vehicle 110, the remote control device 100 grants remote control authority to the identification information (user ID or terminal ID) of the multiple passengers.

[0117] (Remote control instruction output) Fig. 9 is a flowchart showing an example of the operation of the remote control device 100. An example of the operation relating to the output of a remote operation instruction executed by the remote control device 100 will be described with reference to Fig. 9. The output of a remote operation instruction refers to a process of transmitting a remote operation instruction received from the terminal device 120 to the vehicle 110 to be operated.

[0118] In S901, the remote operation device 100 receives a remote operation instruction from the terminal device 120.

[0119] In S902, the remote operation device 100 determines whether or not the remote operation device 100 has authority to perform remote operation based on the identification information (user ID or terminal ID) included in the remote operation instruction, with reference to the authority management information 57. If it is determined that the remote operation device 100 has authority to perform remote operation (YES in S902), the process proceeds to S903; otherwise (NO in S902), S902 is repeated.

[0120] In S903, the remote control device 100 identifies the vehicle 110 to be operated (for example, vehicle ID=V1 in FIG. 5) based on the identification information included in the remote control instruction, with reference to the authority management information 57. Thereafter, the remote control device 100 transmits a remote control instruction (remote control signal) to the identified vehicle 110 in accordance with the remote control instruction.

[0121] (Remote control permission removed) 10 is a flowchart showing an example of the operation of the remote operation device 100. An example of the operation of the remote operation device 100 regarding the deletion of the authority of remote operation will be described with reference to FIG.

[0122] In S1001, the remote operation device 100 receives an instruction to delete a remote operation authority from the terminal device 120. The remote operation device 100 refers to the authority management information 57 (for example, column 503 in FIG. 5 ) and identifies the remote operation authority to be deleted based on the identification information (user ID or terminal ID) included in the instruction to delete the authority.

[0123] In S1002, the remote operation device 100 refers to the authority management information 57 (for example, column 504 in FIG. 5) and determines whether or not the identification information of the remote operation authority to be deleted is registered in a group. If it is determined that the identification information is registered in a group ("YES" in S1002), the process proceeds to S1003.

[0124] In S1003, the remote operation device 100 deletes the remote operation authority of the entire group. For example, in ST51 of Fig. 5, when an instruction to delete the remote operation authority is received from the terminal device 120 having the identification information of user A (user ID = UA) or user B (user ID = UA), the remote operation device 100 deletes the remote operation authority associated with the terminal devices 120 of all users (A, B) in group 1 to which user A and user B belong.

[0125] On the other hand, if group registration has not been performed ("NO" in S1002), the process proceeds to S1004. In S1004, the remote operation device 100 deletes the remote operation authority identified in S1001. For example, in ST52 of FIG. 5, when an instruction to delete the remote operation authority is received from the terminal device 120 having the identification information of user C (user ID=UC), the remote operation device 100 deletes the remote operation authority associated with the terminal device 120 of user C.

[0126] (Example of operation of vehicle 110) 11 to 13 are flowcharts showing an example of the operation of vehicle 110. Each step in FIGS. 11 to 13 may be performed by processor 51 executing a computer program stored in storage device 45. Alternatively, some or all of the steps in FIGS. 11 to 13 may be performed by a dedicated integrated circuit such as an ASIC. In such a case, the dedicated integrated circuit may operate as a functional unit for executing each step.

[0127] (Display usage code) Fig. 11 is a flowchart showing an example of the operation of the vehicle 110. An example of the operation of the vehicle 110 regarding the display of a usage code will be described with reference to Fig. 11. In S1101, the vehicle 110 determines whether or not it has received the usage code transmitted from the remote control device 100 in S303 (Fig. 3). If it is determined that the usage code has been received ("YES" in S1101), the vehicle 110 transitions the process to S1102, and otherwise ("NO" in S1101) repeats S1101.

[0128] In S1102, the vehicle 110 displays the usage code on the display device 23 in response to an operation of the input device 24 by the user.

[0129] (Transmission of identification information) Fig. 12 is a flowchart showing an example of the operation of the vehicle 110. An example of the operation related to the transmission of identification information executed by the vehicle 110 will be described with reference to Fig. 12. In the operation shown in Fig. 12, the vehicle 110 transmits to the remote operation device 100 identification information associated with the terminal device 120 of the current operator (user) or the last operator (user) at the time of startup.

[0130] In S1201, the vehicle control unit 44 of the vehicle 110 determines whether to start or stop the propulsion device 4 by controlling the ON or OFF of the ignition device. If it is determined that the propulsion device 4 has started or stopped ("YES" in S1201), the vehicle 110 transitions the process to S1202, and otherwise ("NO" in S1201) repeats S1201.

[0131] In S1202, the vehicle 110 acquires identification information (user ID or terminal ID) associated with the terminal device 120 in the vehicle 110. The identification information (user ID or terminal ID) may be acquired using the communication device 9 of the vehicle 110. The identification information associated with the terminal device 120 may be the terminal ID of the terminal device 120 or the user ID of the user of the terminal device 120. When multiple terminal devices 120 are present in the vehicle 110, the vehicle 110 may acquire identification information (user ID or terminal ID) from each of the multiple terminal devices 120.

[0132] In S1203, the vehicle 110 transmits the identification information (user ID or terminal ID) acquired in S1202 to the remote control device 100. This identification information is used for registration in the authority management information 57 (e.g., FIG. 5) and for determining whether to grant authority for remote control (e.g., S702 in FIG. 7A). Note that in the above example, an example has been described in which identification information associated with the terminal device 120 in the vehicle 110 is acquired when the vehicle 110 is started or stopped. However, the vehicle 110 may also transmit to the remote control device 100 identification information associated with a terminal device 120 that has traveled together with the vehicle 110 for a predetermined period of time.

[0133] The communication device 53 of the remote control device 100 acquires, from the vehicle 110, identification information (user ID or terminal ID) associated with the terminal device 120 of the first user present inside the vehicle 110 when the vehicle 110 starts or stops. The processor 51 of the remote control device 100 registers the identification information (user ID or terminal ID) acquired by the communication device 53 in the storage device 52, and grants remote control authority to the identification information (user ID or terminal ID) registered in the storage device 52.

[0134] (Operation based on remote control instructions) 13 is a flowchart showing an example of the operation of the vehicle 110. With reference to FIG. 13, an example of the operation relating to vehicle control based on a remote operation instruction executed by the vehicle 110 will be described.

[0135] 13, the vehicle 110 operates in accordance with a remote control instruction. In S1301, the vehicle 110 determines whether or not it has received the remote control instruction transmitted in S309 from the remote control device 100 via the communication device 9. If it is determined that the remote control instruction has been received ("YES" in S1301), the vehicle 110 transitions the process to S1302, and otherwise ("NO" in S1301) repeats S1301.

[0136] In S1302, the vehicle 110 executes vehicle control in accordance with the remote operation instruction. The vehicle control unit 44 controls the vehicle 110 in accordance with the remote operation instruction. For example, the vehicle control unit 44 controls locking and unlocking of the doors of the vehicle 110, activation of the power system that supplies power to each part of the vehicle 110, activation of the propulsion device 4, etc.

[0137] [Other embodiments] Furthermore, a remote control program that realizes one or more functions described in each embodiment can be supplied to a terminal device via a network or a storage medium, and one or more processors in a computer of the system or device can read and execute the remote control program. The present invention can also be realized in such an embodiment.

[0138] <Summary of the embodiment> (Item 1) The remote control device of the above embodiment is a remote control device (100) for remotely operating a vehicle, a storage means (52) for registering biometric information of a user in association with identification information set in the terminal device of the user; a communication means (53) for acquiring biometric information of a first user present inside the vehicle; and a calculation means (51) that grants the authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means (52).

[0139] The remote control device of item 1 can improve convenience for users who share a vehicle with multiple people. For example, since it can be inferred that the ability to board a vehicle means that the user has the authority to operate the vehicle, remote control authority can be granted to a user authenticated by biometric authentication, and remote control from outside the vehicle can be permitted, thereby improving user convenience.

[0140] (Item 2) When the biometric information of the first user is not registered in the storage means (52), the calculation means (51) When the usage code presented by the interface device inside the vehicle matches the usage code transmitted from the user's terminal device, The remote control authority is granted to identification information associated with the terminal device of the first user, which is registered in the storage means (52).

[0141] According to the remote control device of item 2, if biometric information has not been registered, the user can enter the vehicle using a mechanical unlocking method and authenticate the usage code by operating inside the vehicle, thereby activating the remote control function provided by the vehicle remote control system.

[0142] (Item 3) When the calculation means (51) grants the authorization for remote operation to the identification information associated with the terminal device of the first user by comparing the usage code, The storage means (52) registers the biometric information of the first user acquired by the communication means (53) in association with identification information associated with the terminal device of the user.

[0143] According to the remote control device of item 3, when activating the remote control function provided by the vehicle remote control system, it is possible to register the biometric information of the user in the vehicle in association with the identification information associated with the user's terminal device.

[0144] (Item 4) When the communication means (53) acquires biometric information of a second user after the first user, the calculation means (51) deletes the remote control authority that was granted to the second user before the second user.

[0145] According to the remote control device of item 4, when the user of a vehicle is changed, the authority of the previous user can be deleted by acquiring the biometric information of the subsequent user. This improves the security of the vehicle when remotely controlling it. For example, when a vehicle is resold, the previous user's authority to remotely control the vehicle can be deleted by having a user (e.g., a staff member) enter the vehicle at the resale destination and acquire the biometric information, without leaving the authority to the previous user.

[0146] (Item 5) The communication means (53) acquires an image including a face of the first user as the biometric information, The calculation means (51) acquires facial information by extracting facial features of the first user from the image, The storage means (52) registers the face information and the identification information as biometric information of the first user in association with each other.

[0147] According to the remote control device of item 5, it is possible to associate face information extracted as biometric information of a user present inside a vehicle with identification information.

[0148] (Item 6) The storage means (52) has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group, When the communication means (53) acquires the biometric information of the second user after the biometric information of the first user, The calculation means (51) maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user.

[0149] According to the remote control device of Item 6, by registering users in a group, it is possible to ensure security in remote control while improving convenience for multiple users when multiple users share a single vehicle. By adding or removing users from the group, it is possible to set the range of users who can use the remote control functions provided by the vehicle remote control system.

[0150] (Item 7) The communication means (53) acquires images including faces of a plurality of users present inside the vehicle, When the facial information of the plurality of users extracted from the image is registered in the storage means (52) as biometric information of the plurality of users, the calculation means (51) registers the plurality of users as the same group.

[0151] According to the remote control device of item 7, passengers can be registered as the same group and given remote control authority, thereby improving convenience for multiple users.

[0152] (Item 8) The calculation means (51) outputs a signal for operating the vehicle based on a request for remote operation received from a terminal device having identification information that has been granted the authority to perform remote operation.

[0153] According to the remote control device of item 8, a vehicle can be remotely controlled based on a remote control request transmitted from a terminal device having identification information that has been granted authority to perform remote control.

[0154] (Item 9) The communication means (53) acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle, The calculation means (51) Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked; When the location where the biometric information is acquired is not within the normal parking area, the calculation means (51) deletes the authority of remote operation assigned to the identification information associated with the biometric information of the first user, The identification information associated with the biometric information acquired at the location is given authority to remotely operate the vehicle.

[0155] (Item 10) The communication means (53) acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle, The calculation means (51) Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked; When the location where the biometric information was acquired is within the normal parking area, the calculation means (51) maintains the remote operation authority granted to the identification information associated with the biometric information of the first user.

[0156] According to the remote control device of items 9 and 10, the remote control authority can be maintained or updated based on the location where the biometric information was acquired. When the location where the biometric information was acquired is within a normal parking area, for example, use when a family shares a vehicle can be envisioned. In such a case, when new biometric information is acquired, the authority of the user who was previously granted remote control authority is maintained without deleting the previously granted remote control authority. This improves convenience when a vehicle is shared by multiple users. Furthermore, by maintaining the authority of the user who was granted remote control authority, it becomes possible to remotely restrict the operation of the vehicle even if the vehicle is stolen, for example, thereby improving the security of the vehicle 110.

[0157] (Item 11) The control method for the remote control device of the above embodiment is a control method for a remote control device that remotely operates a vehicle, a step of registering the user's biometric information and the identification information set in the user's terminal device in association with each other in a storage means (52); A step in which a communication means (53) acquires biometric information of a first user present inside the vehicle; The method includes a step in which a calculation means (51) grants the authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user from among the biometric information registered in the storage means (52).

[0158] The control method for a remote control device according to Item 11 can improve convenience for users who share a vehicle with multiple people. For example, since being able to board a vehicle can be inferred to mean having the authority to operate the vehicle, remote control authority can be granted to a user authenticated by biometric authentication, and remote control from outside the vehicle can be permitted, thereby improving user convenience.

[0159] (Item 12) A program that causes a computer to execute the control method described in Item 11.

[0160] According to the program of item 12, it is possible to provide a program that causes a computer to execute a control method for a remote control device.

[0161] The present invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0162] 100: remote control device, 110: vehicle, 120: terminal device

Claims

1. A remote control device for remotely operating a vehicle, a storage means for registering biometric information of a user in association with identification information set in a terminal device of the user; a communication means for acquiring biometric information of a first user present inside the vehicle; a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means; and Equipped with The storage means has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group, When the communication unit acquires biometric information of a second user after the first user, A remote control device characterized in that the calculation means maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user.

2. When the biometric information of the first user is not registered in the storage means, the calculation means When the usage code presented by the interface device inside the vehicle matches the usage code transmitted from the user's terminal device, 2. The remote control device according to claim 1, wherein the authority to perform the remote control is given to identification information associated with the terminal device of the first user, which is registered in the storage means.

3. When the computing means grants the authorization for remote operation to the identification information associated with the terminal device of the first user by comparing the usage code, 3. The remote control device according to claim 2, wherein the storage means registers the biometric information of the first user acquired by the communication means in association with identification information associated with the terminal device of the user.

4. The remote control device according to claim 1, characterized in that, when the communication means acquires biometric information of a second user after the first user, the calculation means deletes the remote control authority granted to the second user before the second user.

5. the communication means acquires an image including a face of the first user as the biometric information; the computing means acquires facial information by extracting facial features of the first user from the image; 2. The remote control device according to claim 1, wherein the storage unit registers the face information and the identification information as biometric information of the first user in association with each other.

6. the communication means acquires images including faces of a plurality of users present inside the vehicle; The remote control device according to claim 1, characterized in that, when the facial information of the plurality of users extracted from the image is registered in the storage means as biometric information of the plurality of users, the calculation means registers the plurality of users as the same group.

7. 2. The remote control device according to claim 1, wherein the calculation means outputs a signal for operating the vehicle based on a request for remote control received from a terminal device having identification information to which the authority for remote control has been granted.

8. A remote control device for remotely operating a vehicle, a storage means for registering biometric information of a user in association with identification information set in a terminal device of the user; a communication means for acquiring biometric information of a first user present inside the vehicle; a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means, The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle; The calculation means Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked; When the location where the biometric information was acquired is not within the normal parking area, the calculation means deletes the authority of remote operation assigned to the identification information associated with the biometric information of the first user, and A remote control device characterized in that authority to remotely control the vehicle is granted to the identification information associated with the biometric information acquired at the location.

9. A remote control device for remotely operating a vehicle, a storage means for registering biometric information of a user in association with identification information set in a terminal device of the user; a communication means for acquiring biometric information of a first user present inside the vehicle; a computing means for granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means, The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle; The calculation means Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked; A remote control device characterized in that the calculation means maintains the remote control authority granted to the identification information corresponding to the biometric information of the first user when the location where the biometric information was acquired is within the normal parking area.

10. A control method for a remote control device that remotely controls a vehicle, comprising: a step of registering the biometric information of the user and the identification information set in the terminal device of the user in association with each other in a storage means; a step of acquiring biometric information of a first user present inside the vehicle by a communication means; a step of granting, by a computing means, authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means; and The storage means has a function of registering a plurality of users as a group, and when the first user and the second user are registered as the same group, When the communication unit acquires biometric information of a second user after the first user, A control method characterized in that the calculation means maintains the remote control authority granted to the identification information associated with the terminal device of the first user and the remote control authority granted to the identification information associated with the terminal device of the second user.

11. A control method for a remote control device that remotely controls a vehicle, comprising: a step of registering the biometric information of the user and the identification information set in the terminal device of the user in association with each other in a storage means; a step of acquiring biometric information of a first user present inside the vehicle by a communication means; and a step by a computing means of granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means, The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle; The calculation means Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked; When the location where the biometric information was acquired is not within the normal parking area, the calculation means deletes the authority of remote operation assigned to the identification information associated with the biometric information of the first user, and A control method comprising: granting authority to remotely control the vehicle to the identification information associated with the biometric information acquired at the location.

12. A control method for a remote control device that remotely controls a vehicle, comprising: a step of registering the biometric information of the user and the identification information set in the terminal device of the user in association with each other in a storage means; a step of acquiring biometric information of a first user present inside the vehicle by a communication means; and a step by a computing means of granting authority to remotely operate the vehicle to the identification information associated with the biometric information of the first user among the biometric information registered in the storage means, The communication means acquires the biometric information, the parking position of the vehicle, and date and time information specifying the parking date and time from the vehicle; The calculation means Based on the information on the parking position of the vehicle and the parking date and time, a parking frequency for each parking position of the vehicle during a predetermined period is acquired, and the parking position with the highest parking frequency is identified as a normal parking area in which the vehicle is normally parked; A control method characterized in that the calculation means maintains the remote operation authority granted to the identification information corresponding to the biometric information of the first user when the location where the biometric information was acquired is within the normal parking area.

13. A program for causing a computer to execute the control method according to any one of claims 10 to 12.

Images