Vehicle remote monitoring system, vehicle and remote monitoring device

Abstract

This invention provides a vehicle remote monitoring system, vehicle, and remote monitoring device that allows the administrator of the remote monitoring device to transmit voice information from the remote monitoring device to the vehicle when they wish to communicate with the vehicle, and that minimizes confusion for the administrator when multiple vehicles send call requests to the remote monitoring device. [Solution] The system comprises a plurality of vehicles 40A, 40B, and 40C each having a camera and a first communication device, and a remote monitoring device 20 having a second communication device that can wirelessly communicate with the first communication device and a display 25 that displays camera images acquired by the camera. When the remote monitoring device sends a first call request to a selected vehicle, it becomes possible to transmit voice information from the second communication device to the first communication device of the selected vehicle. When a vehicle sends a second call request to the remote monitoring device, the remote monitoring device decides whether or not to establish a call between the second communication device and the first communication device of the vehicle that sent the second call request.

Description

Technical Field

[0001] The present invention relates to a vehicle remote monitoring system, a vehicle, and a remote monitoring device.

Background Art

[0002] The following Patent Document 1 discloses an invention in which communication is automatically established between a vehicle and a management device due to a request from the vehicle when an abnormality occurs in the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the vehicle remote monitoring system of the above Patent Document 1, communication is not smoothly established when an administrator operating the management device wants to communicate with the vehicle, and there is a risk that the administrator may be confused by requests from a plurality of vehicles.

[0005] In consideration of the above facts, an object of the present invention is to provide a vehicle remote monitoring system, a vehicle, and a remote monitoring device that enable transmission of voice information from the remote monitoring device to the vehicle when an administrator of the remote monitoring device wants to talk to the vehicle, and that make it difficult for the administrator to be confused when call requests are sent from a plurality of vehicles to the remote monitoring device.

Means for Solving the Problems

[0006] A vehicle remote monitoring system according to a first aspect of the present invention comprises a plurality of vehicles each having a camera that photographs at least one of the interior and exterior of the vehicle and a first communication device, and a remote monitoring device having a second communication device that can wirelessly communicate with the first communication device and a display that displays camera images acquired by the camera, wherein when the remote monitoring device issues a first call request to a selected vehicle which is at least one of the vehicles, the second communication device becomes capable of transmitting voice information to the first communication device of the selected vehicle, and when the vehicle issues a second call request to the remote monitoring device, the remote monitoring device has a call permission determination unit that determines whether or not to establish a call between the second communication device and the first communication device of the vehicle which issued the second call request.

[0007] A second aspect of the present invention is a vehicle remote monitoring system in the first aspect of the present invention, wherein the camera includes an in-vehicle camera capable of photographing the inside of the vehicle and an out-of-vehicle camera capable of photographing the outside of the vehicle.

[0008] A third aspect of the vehicle remote monitoring system of the present invention, in the vehicle remote monitoring system of the first or second aspect of the present invention, enables the transmission of voice information from the second communication device to the first communication devices of all the vehicles when the remote monitoring device issues the first call request to all the vehicles.

[0009] A fourth aspect of the vehicle remote monitoring system of the present invention is a vehicle remote monitoring system of any of the first to third aspects of the present invention, wherein when a plurality of vehicles make the second call request to the remote monitoring device, the call permission determination unit is able to transmit call hold information to at least one of the vehicles.

[0010] A fifth aspect of the vehicle remote monitoring system of the present invention is a vehicle remote monitoring system of the fourth aspect of the present invention, wherein when a plurality of the vehicles issue the second call request to the remote monitoring device, the call permission determination unit is able to establish a call between the second call device and the first call device of at least one of the vehicles.

[0011] A vehicle according to a sixth aspect of the present invention is a vehicle having a camera that photographs at least one of the interior and exterior of the vehicle and a first communication device, wherein a second communication device provided on a remote monitoring device having a display that displays camera images acquired by the camera is able to communicate with the first communication device, and when the remote monitoring device issues a first call request to a selected vehicle which is at least one of the vehicles, the second communication device becomes capable of transmitting voice information to the first communication device of the selected vehicle, and when the vehicle issues a second call request to the remote monitoring device, the call permission determination unit of the remote monitoring device makes a decision on whether or not to establish a call between the second communication device and the first communication device of the vehicle which issued the second call request.

[0012] A remote monitoring device according to a seventh aspect of the present invention is a remote monitoring device having a second communication device that can communicate with a first communication device provided in a plurality of vehicles having cameras that photograph at least one of the interior and exterior of the vehicle, and a display that displays camera images acquired by the cameras, wherein when the remote monitoring device issues a first call request to a selected vehicle which is at least one of the vehicles, the second communication device becomes capable of transmitting voice information to the first communication device of the selected vehicle, and when the vehicle issues a second call request to the remote monitoring device, the device has a call permission determination unit that determines whether or not to establish a call between the second communication device and the first communication device of the vehicle which issued the second call request. [Effects of the Invention]

[0013] In the vehicle remote monitoring system of the first aspect of the present invention, the vehicle of the sixth aspect, and the remote monitoring device of the seventh aspect, when the remote monitoring device issues a first call request to a selected vehicle, which is at least one vehicle, it becomes possible to transmit voice information from the second call device to the first call device of the selected vehicle. Therefore, for example, an administrator of the remote monitoring device who determines that it is necessary to convey predetermined information to the selected vehicle based on a camera image displayed on a display will be able to convey the predetermined information to the selected vehicle.

[0014] In the vehicle remote monitoring system of the first aspect of the present invention, the vehicle of the sixth aspect, and the remote monitoring device of the seventh aspect, when a vehicle sends a second call request to the remote monitoring device, the call permission determination unit of the remote monitoring device decides whether or not to establish a call between the second call device and the first call device of the vehicle that sent the second call request. Therefore, when a second call request is sent to the remote monitoring device from multiple vehicles, the administrator is less likely to become confused.

[0015] A second aspect of the present invention, a vehicle remote monitoring system, can display camera images from an in-vehicle camera and camera images from an external vehicle camera on a display.

[0016] In a third aspect of the present invention, when an administrator wants to transmit predetermined information to all vehicles, the remote monitoring device can transmit the predetermined information to all vehicles.

[0017] A vehicle remote monitoring system according to a fourth aspect of the present invention can notify the occupant of a vehicle that a second call request has been made to a remote monitoring device but a call has not been established with the remote monitoring device, that the call is on hold.

[0018] A fifth aspect of the present invention, a vehicle remote monitoring system, is capable of establishing a communication between a second communication device and a first communication device of at least one vehicle when multiple vehicles send a second communication request to a remote monitoring device. [Brief explanation of the drawing]

[0019] [Figure 1] This diagram shows the overall configuration of the vehicle remote monitoring system according to the embodiment. [Figure 2] This is a control block diagram of the control device for a remote monitoring system. [Figure 3] This is a functional block diagram of the control unit. [Figure 4] This diagram shows the first display of the remote monitoring device. [Figure 5] This diagram shows the second display of the remote monitoring device. [Figure 6] This is a control block diagram of the bus's ECU. [Figure 7] It is a functional block diagram of an ECU. [Figure 8] It is a flowchart showing the processing executed by the control device of the remote monitoring device. [Figure 9] It is a flowchart showing the processing executed by the ECU of the bus.

Embodiments for Carrying Out the Invention

[0020] Hereinafter, embodiments of a vehicle remote monitoring system, a vehicle, and a remote monitoring device according to the present invention will be described with reference to the drawings.

[0021] As shown in FIG. 1, a vehicle remote monitoring system 10 (hereinafter referred to as system 10) of the embodiment has a remote monitoring device 20 and a plurality of buses (vehicles) 40A, 40B, and 40C.

[0022] The remote monitoring device 20 shown in FIGS. 1 and 2 is installed in a building (not shown) owned by the administrator of the system 10. The remote monitoring device 20 includes a control device 21, a first display 24, a second display 25, an input device 27, and an audio input / output device (second communication device) 28.

[0023] As shown in FIG. 2, as a hardware configuration, the control device 21 includes a CPU (Central Processing Unit) 21A, a ROM (Read Only Memory) 21B, a RAM (Random Access Memory) 21C, a storage 21D, a wireless communication I / F (Interface) 21E, and an input / output I / F 21G. The CPU 21A, the ROM 21B, the RAM 21C, the storage 21D, the wireless communication I / F 21E, and the input / output I / F 21G are communicably connected to each other via an internal bus 21Z. The CPU 21A and the CPU 41A described later can acquire information regarding time from a timer.

[0024] The CPU 21A is the central processing unit, which executes various programs and controls various components. The CPU 21A reads programs from ROM 21B or storage 21D and executes them using RAM 21C as the working area. The CPU 21A controls each component and performs various calculations according to the programs recorded in ROM 21B or storage 21D.

[0025] ROM21B stores various programs and data. For example, multiple programs are installed in ROM21B. ROM21B also records map data, call hold information, and vehicle ID information, which will be described later. RAM21C temporarily stores programs or data as a working area. Storage21D consists of a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs and data.

[0026] The map data includes information about the areas in which buses 40A, 40B, and 40C, which are registered vehicles in system 10, operate. Although only the three buses 40A, 40B, and 40C are depicted in Figure 1, many more buses are registered in system 10.

[0027] The wireless communication interface (I / F21E) is an interface for wireless communication with various devices. For example, the wireless communication interface (I / F21E) can communicate wirelessly with the wireless communication interface (I / F41E) on buses 40A, 40B, and 40C via a network (e.g., the Internet). The wireless communication interface (I / F21E) can send and receive audio data, camera image data, and sensor group detection values, as described later.

[0028] The input / output interface 21G is an interface for communicating with various devices. For example, the input / output interface 21G is connected to a first display 24, a second display 25, an input device 27, and an audio input / output device 28. The first display 24 and the second display 25 can display various images. For example, the first display 24 can display a map image 30, which will be described later. The second display 25 can display, for example, a camera image received wirelessly by the wireless communication interface 21E from the wireless communication interface 41E on buses 40A, 40B, and 40C. The input device 27 can input various information to the control device 21. The input device 27 is, for example, a keyboard. The audio input / output device 28 is equipped with a microphone and a speaker.

[0029] Figure 3 shows a block diagram illustrating an example of the functional configuration of the control device 21. The control device 21 has a functional configuration consisting of a transmit / receive control unit 211, a SIP control unit (call permission determination unit) 212, and a display control unit 213. These functions are realized by the CPU 21A reading and executing a program stored in the ROM 21B.

[0030] The transmit / receive control unit 211 controls the wireless communication interface 21E. The wireless communication interface 21E controlled by the transmit / receive control unit 211 can receive camera image data (described later) and record data received from an external source in the ROM 21B and storage 21D. Furthermore, when a call using SIP (described later) is being performed, the wireless communication interface 21E can wirelessly transmit audio data input to the audio input / output device 28 and receive audio data received from the wireless communication interfaces 41E on buses 40A, 40B, and 40C.

[0031] The SIP control unit 212 controls communication between the voice input / output device 28, which uses SIP (Session Initiation Protocol), and buses 40A, 40B, and 40C (speaker 44, microphone 45). In other words, the control unit 21 functions as a SIP server.

[0032] When a first call request is sent from the remote monitoring device 20 to a selected vehicle (wireless communication I / F 41E) by a touch operation on the first call request operation units 34A2, 34B2, and 34C2 (described later), the SIP control unit 212 enables the transmission of voice information from the control device 21 to the selected vehicle. That is, the voice input to the voice input / output device 28 is output from the speaker (first call device) 44 of the selected vehicle. The selected vehicle is the bus corresponding to the first call request operation unit 34A2, 34B2, and 34C2 that was touched. Also, for example, when a first call request is sent from the remote monitoring device 20 to the wireless communication I / F 41E of all buses 40A, 40B, and 40C by a touch operation on the simultaneous request operation unit 36 ​​(described later), the SIP control unit 212 enables the transmission of voice information from the control device 21 to all buses 40A, 40B, and 40C.

[0033] When the SIP control unit 212 receives a second call request (described later) from buses 40A, 40B, or 40C, and a touch operation is performed on the call authorization units 34A3, 34B3, or 34C3 (described later), the SIP control unit 212 establishes a call between the buses 40A, 40B, or 40C corresponding to the operated call authorization unit 34A3, 34B3, or 34C3 and the control device 21. That is, the voice information input to the microphones 45 on buses 40A, 40B, or 40C is output from the voice input / output device 28, and the voice information input to the voice input / output device 28 is output from the speaker 44. On the other hand, the SIP control unit 212 does not authorize a call between the buses 40A, 40B, or 40C corresponding to the unoperated call authorization unit 34A3, 34B3, or 34C3 and the control device 21 (voice input / output device 28).

[0034] When a second call request is issued from buses 40A, 40B, or 40C and a touch operation is performed on one of the call hold notification units 34A4, 34B4, or 34C4 (described later), the SIP control unit 212 transmits voice data based on call hold information to the wireless communication interface 41E of buses 40A, 40B, or 40C corresponding to the operated call hold notification unit 34A4, 34B4, or 34C4. This voice data may contain a message such as, "We are currently on a call with another vehicle, please wait a moment."

[0035] The display control unit 213 controls the first display 24 and the second display 25, both of which are touch panel type.

[0036] Figure 4 shows the first display 24 controlled by the display control unit 213. The first display 24 displays a map image 30 based on the map data described above. The map image 30 includes a road image 31, a bus image 32A representing bus 40A, a bus image 32B representing bus 40B, and a bus image 32C representing bus 40C. The display control unit 213 changes the positions of the bus images 32A, 32B, and 32C on the map image 30 based on the position information (GNSS information) transmitted from buses 40A, 40B, and 40C.

[0037] Figure 5 shows a second display 25 controlled by a display control unit 213. The second display 25 displays camera image display areas 33A, 33B, and 33C. Furthermore, the camera image display areas 33A, 33B, and 33C have a first display area 33A1, 33B1, 33C1, a second display area 33A2, 33B2, 33C2, and a third display area 33A3, 33B3, 33C3. The first display areas 33A1, 33B1, and 33C1 display camera images acquired by the in-vehicle cameras (cameras) 46 of the buses 40A, 40B, and 40C in real time. The second display areas 33A2, 33B2, and 33C2 display camera images acquired by the out-of-vehicle front cameras (cameras) (out-of-vehicle cameras) 47 of the buses 40A, 40B, and 40C in real time. The third display areas 33A3, 33B3, and 33C3 display camera images acquired in real time by the external rear cameras (cameras) 48 of buses 40A, 40B, and 40C.

[0038] Furthermore, the second display 25 displays the second call request reception display units 34A1, 34B1, 34C1, the first call request operation units 34A2, 34B2, 34C2, the call permission units 34A3, 34B3, 34C3, the call hold notification units 34A4, 34B4, 34C4, and the call termination operation units 34A5, 34B5, 34C5 to the right of the camera image display areas 33A, 33B, and 33C, respectively. The second call request reception display unit 34A1, the first call request operation unit 34A2, the call permission unit 34A3, the call hold notification unit 34A4, and the call termination operation unit 34A5 correspond to bus 40A. The second call request reception display unit 34B1, the first call request operation unit 34B2, the call permission unit 34B3, the call hold notification unit 34B4, and the call termination operation unit 34B5 correspond to bus 40B. The second call request reception display unit 34C1, the first call request operation unit 34C2, the call permission unit 34C3, the call hold notification unit 34C4, and the call termination operation unit 34C5 are compatible with bus 40C.

[0039] Furthermore, a group request operation unit 36 ​​is displayed at the lower end of the second display 25.

[0040] The second call request reception indicators 34A1, 34B1, and 34C1 are equipped with illumination fixtures and blink when the wireless communication I / F21E receives a second call request from buses 40A, 40B, and 40C.

[0041] As described above, when a touch operation is performed on the first call request operation unit 34A2, 34B2, or 34C2, the first call request is wirelessly transmitted to the selected vehicle. Also, as described above, when a second call request is issued from buses 40A, 40B, or 40C and the call permission unit 34A3, 34B3, or 34C3 is operated, a call is established between the corresponding buses 40A, 40B, or 40C and the control device 21 (voice input / output device 28). Furthermore, when a second call request is issued from buses 40A, 40B, or 40C and a touch operation is performed on the call hold notification unit 34A4, 34B4, or 34C4, voice data based on call hold information is transmitted to buses 40A, 40B, or 40C. Additionally, when a touch operation is performed on the simultaneous request operation unit 36, the first call request is wirelessly transmitted to all buses as described above.

[0042] When a touch operation is performed on the call termination operation units 34A5, 34B5, and 34C5, the SIP control unit 212 controls the communication between the remote monitoring device 20 and buses 40A, 40B, and 40C based on the second call request, rendering it impossible.

[0043] As shown in Figure 6, buses 40A, 40B, and 40C are equipped with an ECU 41, a GNSS (Global Navigation Satellite System) receiver 43, a speaker 44, a microphone 45, an in-vehicle camera 46, an external front camera 47, an external rear camera 48, a second call request operation unit 49S, a call termination operation unit 49F, a sensor group 51, and a driving assistance operation unit 62. As shown in Figure 6, the ECU 41 has the same hardware configuration as the control unit 21. That is, the ECU 41 has a hardware configuration of a CPU 41A, a ROM 41B, a RAM 41C, a storage 41D, a wireless communication I / F 41E, an input / output I / F 41G, and an internal bus 41Z. For example, multiple programs are installed in the ROM 41B.

[0044] The input / output interface 41G of the ECU 41 is connected to a GNSS receiver 43, a speaker 44, a microphone 45, an in-vehicle camera 46, an out-of-vehicle front camera 47, an out-of-vehicle rear camera 48, a second call request operation unit 49S, a call termination operation unit 49F, a sensor group 51, and a driver assistance operation unit 62.

[0045] The GNSS receiver 43 receives GNSS signals transmitted from GNSS satellites and acquires position information for buses 40A, 40B, and 40C equipped with an ECU 41. The acquired position information is wirelessly transmitted to the remote monitoring device 20 via a wireless communication interface 41E.

[0046] Speaker 44 can output various sounds. For example, speaker 44 can output audio based on call hold information. Furthermore, speaker 44 can output audio input to the audio input / output device 28 when it receives a first call request or when a call is established between the remote monitoring device 20 and buses 40A, 40B, and 40C.

[0047] Various types of audio are input to microphone 45. For example, when audio is input to microphone 45 when communication is established between the remote monitoring device 20 and buses 40A, 40B, and 40C, this audio is output from the audio input / output device 28 of the remote monitoring device 20.

[0048] The in-vehicle camera 46 is installed inside the buses 40A, 40B, and 40C. For example, the in-vehicle camera 46 is installed on the ceiling of the buses 40A, 40B, and 40C. The in-vehicle camera 46 repeatedly acquires image data representing subjects inside the buses 40A, 40B, and 40C at predetermined intervals.

[0049] The external front camera 47 is installed at the front end of buses 40A, 40B, and 40C. The external front camera 47 repeatedly acquires image data representing an external subject located in front of the front end of buses 40A, 40B, and 40C at predetermined intervals.

[0050] The external rear camera 48 is installed at the rear end of buses 40A, 40B, and 40C. The external rear camera 48 repeatedly acquires image data representing an external subject located behind the rear end of buses 40A, 40B, and 40C at predetermined intervals.

[0051] The second call request operation unit 49S and the call termination operation unit 49F are, for example, mechanical buttons installed inside the buses 40A, 40B, and 40C. However, the second call request operation unit 49S and the call termination operation unit 49F may also be images (operation images) displayed on a touch panel display installed inside the buses 40A, 40B, and 40C.

[0052] Sensor group 51 is a collective term for the sensors installed on buses 40A, 40B, and 40C. These sensors include, for example, a vehicle speed sensor, an accelerator pedal sensor, a brake pedal sensor, a turn signal sensor, a LiDAR sensor, a radar sensor, air pressure sensors installed on the tires of each wheel, a steering angle sensor that detects the steering angle of the steering wheel, and an acceleration sensor that detects acceleration in various directions generated on the vehicle body. Each of these sensors repeatedly performs a detection operation after a predetermined time has elapsed. In this specification, the detection value acquired by each sensor included in sensor group 51 is referred to as the sensor group detection value.

[0053] The driver assistance control unit 62 is a switch located on the instrument panel of buses 40A, 40B, and 40C.

[0054] Figure 7 shows an example of the functional configuration of the ECU 41 in a block diagram. The ECU 41 has a transmission / reception control unit 411, a call control unit 412, and a driver assistance control unit 413. These functions are realized when the CPU 41A reads and executes a program stored in the ROM 41B.

[0055] The transmit / receive control unit 411 controls the wireless communication interface 41E. The wireless communication interface 41E controlled by the transmit / receive control unit 411 can communicate wirelessly with the wireless communication interface 21E of the remote monitoring device 20 via the network. For example, by controlling the wireless communication interface 41E, the transmit / receive control unit 411 repeatedly wirelessly transmits image data, location information, and sensor group detection values ​​from the in-vehicle camera 46, the exterior front camera 47, and the exterior rear camera 48 to the remote monitoring device 20 at predetermined intervals.

[0056] Furthermore, the transmission / reception control unit 411 wirelessly transmits an automatic driving execution signal to the remote monitoring device 20 when the driving support operation unit 62 is in the ON position and buses 40A, 40B, and 40C are performing Level 4 and Level 5 driving support control, which will be described later.

[0057] The call control unit 412 controls the speaker 44 and the microphone 45. For example, when the wireless communication interface 41E receives a first call request from the remote monitoring device 20, the call control unit 412 enables the function of the speaker 44.

[0058] Furthermore, when the second call request operation unit 49S is operated by a crew member, the call control unit 412 causes the wireless communication I / F 41E to wirelessly transmit the second call request to the remote monitoring device 20.

[0059] Furthermore, when the call termination operation unit 49F is operated, the call control unit 412 terminates the call between the remote monitoring device 20 and buses 40A, 40B, and 40C based on the second call request.

[0060] Furthermore, when a second call request is wirelessly transmitted to the remote monitoring device 20 and a touch operation is performed on the call permission units 34A3, 34B3, and 34C3, the call control unit 412 enables the functions of the speaker 44 and microphone 45.

[0061] Furthermore, when the wireless communication interface 41E receives voice data based on call hold information from the remote monitoring device 20, the call control unit 412 causes the speaker 44 to output voice based on this voice data. As a result, the speaker 44 outputs voice, for example, "We are currently on a call with another vehicle, please wait a moment."

[0062] When the driver assistance control unit 62, provided on buses 40A, 40B, and 40C, is turned ON, the driver assistance control unit 413 performs driver assistance control using data acquired by the GNSS receiver 43, the external front camera 47, the external rear camera 48, and the sensor group 51. This driver assistance control is performed by operating various devices on buses 40A, 40B, and 40C, including the brake system, engine, electric motor, steering system (not shown), and turn signals. In this specification, "driver assistance control" includes driver assistance control levels 1 to 5 as defined by the SAE (Society of Automotive Engineers). When a predetermined operation is performed on the driver assistance control unit 62, the driver assistance control unit 413 determines (selects) the level of driver assistance control to be performed from levels 1 to 5.

[0063] (Mechanism of action and effect) Next, the operation and effects of this embodiment will be described.

[0064] First, the processing flow of the remote monitoring device 20 in this embodiment will be explained using the flowchart in Figure 8. The CPU 21A of the remote monitoring device 20 repeatedly executes the process shown in the flowchart in Figure 8 after a predetermined amount of time has elapsed.

[0065] First, in step S10 (the word "step" will be omitted hereafter), the CPU 21A determines whether or not it has received camera image data from any of the buses 40A, 40B, or 40C.

[0066] If CPU 21A determines "Yes" in S10, it proceeds to S11 and displays the received camera images in the camera image display areas 33A, 33B, and 33C of the second display 25.

[0067] When S10 determines No, or when CPU 21A has finished processing S11, it proceeds to S12 to determine whether or not it has received location information from any of the buses 40A, 40B, or 40C.

[0068] If CPU 21A determines Yes in S12, it proceeds to S13 and displays bus images 32A, 32B, and 32C at the positions corresponding to the received location information on the map image 30 of the first display 24.

[0069] When the CPU 21A determines No in S12 or after completing the processing in S13, it proceeds to S14 and determines whether a touch operation has been performed on any of the first call request operation units 34A2, 34B2, 34C2 or the simultaneous request operation unit 36.

[0070] If CPU 21A determines "Yes" in S14, it proceeds to S15 and sends the first call request to buses 40A, 40B, and 40C.

[0071] When the processing in S15 is completed or when No is determined in S14, the CPU 21A proceeds to S16 and determines whether or not a second call request has been received from any of the buses 40A, 40B, or 40C.

[0072] If CPU 21A determines "Yes" in S16, it proceeds to S17. At this time, CPU 21A blinks the second call request reception indicators 34A1, 34B1, and 34C1 corresponding to the buses 40A, 40B, and 40C that issued the second call request for a predetermined period of time.

[0073] In S17, the CPU 21A determines whether a touch operation has been performed on the call permission units 34A3, 34B3, and 34C3 corresponding to the blinking second call request reception display units 34A1, 34B1, and 34C1.

[0074] If CPU 21A determines Yes in S17, it proceeds to S18 and enables communication between the audio input / output device 28 of the remote monitoring device 20 and the speakers 44 and microphones 45 of buses 40A, 40B, and 40C corresponding to the touch-operated call permission units 34A3, 34B3, and 34C3.

[0075] After completing the processing in S18, CPU 21A proceeds to S19 and determines whether or not a call termination operation has been performed. That is, CPU 21A determines whether or not a touch operation has been performed on the call termination operation units 34A5, 34B5, and 34C5.

[0076] If CPU21A determines "Yes" in S19, it proceeds to S20 and disables communication between buses 40A, 40B, and 40C.

[0077] On the other hand, CPU 21A, which determined No in S17, proceeds to S21 to determine whether a touch operation was performed on any of the call hold notification units 34A4, 34B4, or 34C4.

[0078] If CPU 21A determines "Yes" in S21, it proceeds to S22 and transmits voice data based on the call hold information to the wireless communication I / F 41E of buses 40A, 40B, and 40C, which correspond to the call hold notification units 34A4, 34B4, and 34C4 that were touched.

[0079] When the processes in S20 and S22 are completed, or when the result in S19 and S21 is determined to be No, the CPU 21A temporarily terminates the process shown in the flowchart in Figure 8.

[0080] Next, we will explain the processes performed by CPU 41A on buses 40A, 40B, and 40C using the flowchart in Figure 9. CPU 41A repeatedly executes the processes shown in the flowchart in Figure 9 after a predetermined amount of time has elapsed.

[0081] First, in S30, the CPU 41A controls the wireless communication interface 41E to wirelessly transmit camera image data acquired by the in-vehicle camera 46, the exterior front camera 47, and the exterior rear camera 48 to the remote monitoring device 20.

[0082] After completing the processing in S30, CPU41A proceeds to S31 and determines whether or not it has received the first call request.

[0083] If the response in S31 is "Yes", the CPU 41A proceeds to S32 and enables the function of the speaker 44.

[0084] After completing the processing in S32, CPU 41A proceeds to S33 and determines whether or not the second call request operation unit 49S has been operated.

[0085] If the CPU 41A determines "Yes" in S33, it proceeds to S34 and wirelessly transmits the second call request to the remote monitoring device 20.

[0086] When the result in S33 is determined to be No, or when the processing in S34 is completed, the CPU 41A proceeds to S35 and determines whether or not a call has been established between itself and the remote monitoring device 20.

[0087] If the system determines "Yes" in S35, the CPU 41A proceeds to S36 and enables the functions of the speaker 44 and microphone 45. As a result, the voice spoken by the administrator of the remote monitoring device 20 and input to the audio input / output device 28 is output from the speaker 44. Therefore, the occupants of buses 40A, 40B, and 40C can hear this voice. Furthermore, when voices spoken by the occupants are input to the microphone 45, this voice is output from the audio input / output device 28 of the remote monitoring device 20.

[0088] After completing the processing in S36, CPU 21A proceeds to S37 and determines whether or not a call termination operation has been performed. That is, CPU 41A determines whether or not the call termination operation unit 49F has been operated.

[0089] If CPU 21A determines "Yes" in S37, it proceeds to S38 and disables communication with the remote monitoring device 20.

[0090] On the other hand, if the result in S35 is No, the CPU 41A proceeds to S39 to determine whether the wireless communication I / F 41E has received voice data based on the call hold information. If the result is Yes, the CPU 41A proceeds to S40 to enable the function of the speaker 44 and has the speaker 44 output voice data based on the call hold information. For example, the speaker 44 will output a message such as, "We are currently on a call with another vehicle, please wait a moment."

[0091] When processing S38 and S40 is completed, or when No is determined in S37 and S39, CPU 41A temporarily terminates the processing of the flowchart in Figure 9.

[0092] As described above, in the system 10 of the embodiment, when the remote monitoring device 20 sends a first call request to a selected vehicle, which is at least one bus 40A, 40B, or 40C, the audio input / output device 28 of the remote monitoring device 20 can transmit audio information to the speaker 44 of the selected vehicle. Therefore, for example, if the administrator of the remote monitoring device 20 feels the need to communicate with the selected vehicle based on the camera images inside and outside the bus 40A, 40B, or 40C displayed on the second display 25, they can notify the selected vehicle of the necessary information through the speaker 44.

[0093] Even if buses 40A, 40B, and 40C receive a first call request, if the speaker 44 does not output audio information unless a predetermined operation button on buses 40A, 40B, and 40C is operated, the necessary information will not be notified to the occupants of buses 40A, 40B, and 40C unless the occupants operate this operation button. In contrast, as in this embodiment, if the remote monitoring device 20 sends a first call request to the selected vehicle, and the speaker 44 outputs audio information even if the selected vehicle does not perform a predetermined operation, this problem will not occur.

[0094] For example, when an automated driving execution signal is wirelessly transmitted from buses 40A, 40B, and 40C, which are performing Level 4 or 5 driver assistance control, to the remote monitoring device 20, information is displayed on the first display 24 or the second display 25 to notify the administrator of which bus 40A, 40B, or 40C transmitted the automated driving execution signal. This allows the administrator to recognize that there is no driver on these buses 40A, 40B, or 40C. In such cases, the administrator can use the voice input / output device 28 to notify buses 40A, 40B, and 40C, which are performing Level 4 or 5 driver assistance control, of information regarding traffic conditions around buses 40A, 40B, and 40C, information regarding weather conditions in the surrounding area, and information regarding delays in the schedule, etc.

[0095] Furthermore, the administrator can use the audio input / output device 28 to issue safety warnings to passengers on any of the buses 40A, 40B, or 40C, while taking into account the camera images displayed on the second display 25.

[0096] Furthermore, when a touch operation is performed on any of the first call request operation units 34A2, 34B2, 34C2 or the simultaneous request operation unit 36 ​​of the remote monitoring device 20, the remote monitoring device 20 can transmit voice information to all buses 40A, 40B, and 40C. Therefore, if an administrator wants to notify all buses 40A, 40B, and 40C of a specific piece of information, the remote monitoring device 20 can transmit that information to all buses 40A, 40B, and 40C as voice information. For example, if a linear precipitation band occurs in the sky above the area where all buses 40A, 40B, and 40C are located, the remote monitoring device 20 can simultaneously notify all buses 40A, 40B, and 40C of this information.

[0097] Furthermore, in system 10, when any of the buses 40A, 40B, or 40C sends a second call request to the remote monitoring device 20, the CPU 21A decides whether or not to establish a call between the audio input / output device 28 and the speaker 44 and microphone 45 of the bus 40A, 40B, or 40C that sent the second call request, based on whether or not a touch operation has been performed on the call authorization units 34A3, 34B3, or 34C3. In other words, the administrator can decide which bus 40A, 40B, or 40C to establish a call with by selecting which call authorization unit 34A3, 34B3, or 34C3 to touch. Therefore, the administrator is less likely to become confused when a second call request is sent to the remote monitoring device 20 from multiple buses 40A, 40B, or 40C.

[0098] Furthermore, when multiple buses 40A, 40B, and 40C send a second call request to the remote monitoring device 20, the control device 21 (SIP control unit 212) can establish a call between the audio input / output device 28 and the speaker 44 and microphone 45 of at least one of the buses 40A, 40B, or 40C. For example, the administrator can select buses 40A, 40B, or 40C with a high need (priority) for establishing a call, while considering the camera images displayed on the second display 25, and establish a call with these buses. It is also possible for a selected vehicle that has received a first call request from the remote monitoring device 20 to send a second call request to the remote monitoring device 20. Once a call is established between this selected vehicle and the remote monitoring device 20, the occupants of the selected vehicle can ask the administrator any necessary questions regarding the message notified by the administrator.

[0099] Furthermore, the second display 25 of the remote monitoring device 20 displays camera images from both inside and outside the buses 40A, 40B, and 40C. As a result, administrators can obtain more information about buses 40A, 40B, and 40C compared to when they can only view camera images from inside or outside the buses.

[0100] Furthermore, when multiple buses 40A, 40B, and 40C send a second call request to the remote monitoring device 20, the control device 21 (SIP control unit 212) can transmit voice data based on call hold information to at least one of the buses 40A, 40B, and 40C. Therefore, the crew members of buses 40A, 40B, and 40C that have sent a second call request to the remote monitoring device 20 but have not yet established a call with the remote monitoring device 20 can be notified that their call is on hold. As a result, the crew members can wait for the call with the remote monitoring device 20 to begin in a calmer state compared to when they are not notified that their call is on hold.

[0101] The system 10, remote monitoring device 20, and buses 40A, 40B, and 40C according to the embodiment have been described above, but these can be modified as appropriate without departing from the spirit of the present invention.

[0102] For example, buses 40A, 40B, and 40C do not necessarily need to be able to perform driver assistance control. In other words, the driver may operate buses 40A, 40B, and 40C.

[0103] Call hold information may also be in the form of text or images. In this case, for example, text or images representing the call hold information will be displayed on a display (not shown) provided in buses 40A, 40B, and 40C. This text could be, for example, "We are currently on a call with another vehicle, please wait a moment," or text representing the estimated time until the call is established.

[0104] The remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the first call request operation units 34A2, 34B2, and 34C2. Similarly, the remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the call permission units 34A3, 34B3, and 34C3. Similarly, the remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the call hold notification units 34A4, 34B4, and 34C4. Similarly, the remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the mass request operation unit 36. Furthermore, when a first predetermined operation is performed using an input device (e.g., a mouse) connected to the control device 21, the remote monitoring device 20 may perform the same operation as when the first call request operation units 34A2, 34B2, and 34C2 are operated. Similarly, when a second predetermined operation is performed using an input device, the remote monitoring device 20 may perform the same operation as when the call permission units 34A3, 34B3, and 34C3 are operated. Similarly, when a third predetermined operation is performed using the input device, the remote monitoring device 20 may perform the same operation as when the call hold notification units 34A4, 34B4, and 34C4 are operated. Similarly, when a fourth predetermined operation is performed using the input device, the remote monitoring device 20 may perform the same operation as when the mass request operation unit 36 ​​is operated.

[0105] A text input section may be provided on the touch-panel displays installed on buses 40A, 40B, and 40C, and the text information entered using this text input section may be sent to the remote monitoring device 20 along with the second call request. This message is displayed on the second display 25 of the remote monitoring device 20. This message is, for example, the reason why the occupants of buses 40A, 40B, and 40C who entered the text information wish to speak with the administrator. Furthermore, multiple messages (text information) expressing this reason may be pre-recorded in ROM 41B or storage 41D, and at least one message selected from the multiple messages displayed on the display may be sent to the remote monitoring device 20 along with the second call request. For example, the occupants of buses 40A, 40B, and 40C select any message from the multiple messages by touching the display.

[0106] Buses 40A, 40B, and 40C may be equipped with at least one of the following: an interior camera 46, an exterior front camera 47, and an exterior rear camera 48. Furthermore, buses 40A, 40B, and 40C may also be equipped with a camera that photographs the outside of the vehicle. The interior camera 46, exterior front camera 47, exterior rear camera 48, and camera that photographs the outside of the vehicle may include multiple cameras. That is, for example, buses 40A, 40B, and 40C may be equipped with two or more interior cameras 46, exterior front cameras 47, and exterior rear cameras 48. Also, there may be two or more interior cameras 46, exterior front cameras 47, or exterior rear cameras 48.

[0107] The number of displays provided on the remote monitoring device 20 does not have to be two; it may be one or three or more. If the remote monitoring device 20 has only one display, all images displayed on the first display 24 and the second display 25 will be displayed on this display.

[0108] Communication control between the remote monitoring device 20 and buses 40A, 40B, and 40C may be performed using a control method other than SIP. For example, communication control may be performed using Web Real-Time Communication. In this case, SIP may be combined with a control method other than SIP.

[0109] Multiple speakers 44 and microphones 45 may be provided on buses 40A, 40B, and 40C.

[0110] Furthermore, buses 40A, 40B, and 40C may be equipped with at least one of an in-vehicle microphone and an external microphone that constantly acquire ambient sounds. The audio data acquired by these in-vehicle and external microphones is wirelessly transmitted in real time to the remote monitoring device 20 by the wireless communication I / F 41E. Furthermore, the administrator of the remote monitoring device 20 can listen to the received audio via the audio input / output device 28 by operating a predetermined operating device provided on the remote monitoring device 20. The audio acquired by the in-vehicle microphone may include, for example, audio related to disputes between occupants that occur inside the vehicle. The audio acquired by the external microphone may include, for example, siren sounds emitted by emergency vehicles traveling around buses 40A, 40B, and 40C. In this way, the administrator can select buses 40A, 40B, and 40C with a high need (priority) for establishing a call, or determine which buses 40A, 40B, and 40C should receive the first call request, while considering the audio inside and outside the buses 40A, 40B, and 40C.

[0111] Furthermore, a bus may have multiple vehicles. For example, if a bus is composed of multiple vehicles connected to each other, each vehicle may be equipped with a speaker 44, a microphone 45, an in-vehicle camera 46, an exterior camera, and a display. In this case, the remote monitoring device 20 may send a first communication request to all vehicles, or it may send a first communication request to only some of the vehicles. That is, in this case, each vehicle can be a selected vehicle. Also, when a second call request is sent from each vehicle to the remote monitoring device 20 using the second call request operation unit 49S provided in each vehicle that makes up a single bus, the remote monitoring device 20 may establish a call with the speaker 44 and microphone 45 of some vehicles while not establishing a call with the remaining vehicles, taking into consideration the camera images acquired by the in-vehicle cameras 46 of each vehicle displayed on the second display 25. Furthermore, in this case, the remote monitoring device 20 may send call hold information to the remaining vehicles. Furthermore, when a second call request is sent to the remote monitoring device 20 from each vehicle constituting a single bus, the remote monitoring device 20 may establish communication between the speakers 44 and microphones 45 of all vehicles.

[0112] The present invention may also be applied to vehicles of a different type than buses 40A, 40B, and 40C (for example, passenger cars).

[0113] The data such as map data stored in ROM21B may be stored on an external server (not shown), and the control device 21 may retrieve this data by accessing the external server via the Internet.

[0114] When the wireless communication interface 21E receives a second call request from buses 40A, 40B, or 40C, the speaker of the audio input / output device 28 may generate a predetermined sound. Also, when the wireless communication interface 21E receives a second call request from buses 40A, 40B, or 40C, at least one of the first display 24 and the second display 25 may display a message indicating the receipt of the second call request. [Explanation of Symbols]

[0115] 10. Vehicle Remote Monitoring System (System) 20 Remote monitoring device 212 SIP Control Unit (Call Permission Determination Unit) 24. First display (display) 25. Second display (display) 28. Voice Input / Output Device (Second Communication Device) 40A 40B 40C Bus (vehicle) 44. Speaker (First communication device) 45. Microphone (First communication device) 46. ​​In-car camera (camera) 47. Front-facing camera (external camera) 48. External rear camera (camera) (external camera)

Claims

1. Multiple vehicles each having a camera that captures at least one of the interior and exterior of the vehicle and a first communication device, A remote monitoring device having a first communication device and a second communication device capable of wireless communication with the first communication device, and a display for displaying camera images acquired by the camera, Equipped with, When the remote monitoring device issues a first call request to at least one of the selected vehicles, the second call device becomes capable of transmitting voice information from the selected vehicle to the first call device. A vehicle remote monitoring system having a call permission determination unit that determines whether or not to establish a call between the second call device and the first call device of the vehicle that made the second call request when the vehicle makes a second call request to the remote monitoring device.

2. The vehicle remote monitoring system according to claim 1, wherein the camera includes an in-vehicle camera capable of photographing the inside of the vehicle and an out-of-vehicle camera capable of photographing the outside of the vehicle.

3. The vehicle remote monitoring system according to claim 1 or claim 2, wherein when the remote monitoring device issues the first call request to all of the vehicles, the second call device enables the transmission of voice information to the first call devices of all the vehicles.

4. The vehicle remote monitoring system according to claim 1 or claim 2, wherein when multiple vehicles issue the second call request to the remote monitoring device, the call permission determination unit is capable of transmitting call hold information to at least one of the vehicles.

5. The vehicle remote monitoring system according to claim 4, wherein when multiple vehicles send the second call request to the remote monitoring device, the call permission determination unit is capable of establishing a call between the second call device and the first call device of at least one of the vehicles.

6. A vehicle having a camera that photographs at least one of the interior and exterior of the vehicle, and a first communication device, A second communication device provided in a remote monitoring device having a display that shows camera images acquired by the aforementioned camera is capable of communicating with the first communication device. When the remote monitoring device issues a first call request to at least one of the selected vehicles, the second communication device becomes capable of transmitting voice information to the first communication device of the selected vehicle, and when the vehicle issues a second call request to the remote monitoring device, the call permission determination unit of the remote monitoring device determines whether or not to establish a call between the second communication device and the first communication device of the vehicle that issued the second call request.

7. A remote monitoring device comprising a second communication device capable of communicating with a first communication device installed in a plurality of vehicles, each having a camera that captures at least one of the interior and exterior of the vehicle, and a display that shows camera images acquired by the cameras, When the remote monitoring device issues a first call request to at least one of the selected vehicles, the second call device becomes capable of transmitting voice information to the first call device of the selected vehicle. A remote monitoring device having a call permission determination unit that determines whether or not to establish a call between the second call device and the first call device of the vehicle that made the second call request when the vehicle makes a second call request to the remote monitoring device.

Images