Vehicle remote monitoring system, vehicle and remote monitoring device

Abstract

This invention provides a vehicle remote monitoring system, vehicle, and remote monitoring device that enable a vehicle administrator to quickly recognize information that the vehicle wants to convey to the remote monitoring device administrator when it is difficult to establish communication between the remote monitoring device and the vehicle. [Solution] The system comprises a plurality of vehicles each having a vehicle-side display, an input device, and a first communication device, and a remote monitoring device having a second communication device, a transmitter, and an output device 25 that can wirelessly communicate with the first communication device. When a call is not established between the first communication device and the second communication device of a vehicle that has sent a call request to the remote monitoring device, the transmitter can send a transmission request to the vehicle for which a call has not been established, requesting the transmission of character information. The character information entered using the input device on the vehicle-side display of the vehicle that received the transmission request is transmitted to the remote monitoring device and output from the output device.

Description

Technical Field

[0005]

[0001] The present invention relates to a vehicle remote monitoring system, a vehicle, and a remote monitoring device.

Background Art

[0002] The following Patent Document 1 discloses an invention that enables a vehicle to send a call request to an operator (remote administrator) at a service center.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the invention of the above Patent Document 1, when call requests are sent from a plurality of vehicles to an operator, a call may not be established between some of the vehicles and the operator. In this case, it may take a long time for the operator to recognize the information that the passengers of some of these vehicles want to convey.

[0005] In consideration of the above facts, an object of the present invention is to provide a vehicle remote monitoring system, a vehicle, and a remote monitoring device that enable an administrator to quickly recognize information that a vehicle wants to convey to the administrator of the remote monitoring device when it is difficult to establish a call between the remote monitoring device and the vehicle.

Means for Solving the Problems

[0006] A vehicle remote monitoring system according to a first aspect of the present invention comprises a plurality of vehicles each having a vehicle-side display, an input device and a first communication device, and a remote monitoring device having a second communication device that can be wirelessly communicated with the first communication device, a transmitting device and an output device capable of outputting at least one of voice and text, wherein when a call is not established between the first communication device and the second communication device of a vehicle that has sent a call request to the remote monitoring device, the transmitting device can send a transmission request to the vehicle with which the call has not been established, requesting the transmission of text information, and the text information input using the input device to the vehicle-side display of the vehicle that received the transmission request is transmitted to the remote monitoring device and output from the output device.

[0007] A second aspect of the present invention is a vehicle remote monitoring system in the first aspect of the present invention, wherein the output device of the remote monitoring device is a display capable of displaying character information.

[0008] A third aspect of the vehicle remote monitoring system of the present invention is a vehicle remote monitoring system of the first or second aspect of the present invention, wherein the transmitting device is capable of transmitting to the vehicle, together with the transmission request, at least one message selected from a set of predefined messages.

[0009] A fourth aspect of the present invention is a vehicle remote monitoring system in a vehicle remote monitoring system in a third aspect of the present invention, wherein when the input device of the vehicle that has received a plurality of messages inputs a plurality of response pieces of information for each of the messages as character information to the vehicle-side display, each of the response pieces of information is sequentially transmitted to the remote monitoring device.

[0010] A fifth aspect of the present invention is a vehicle remote monitoring system in any of the first to fourth aspects of the present invention, in which the transmitting device can notify the vehicle that a call can be established while the character information is being input to the vehicle-side display.

[0011] A vehicle according to a sixth aspect of the present invention is a vehicle having a vehicle-side display, an input device and a first communication device, wherein a second communication device provided on a remote monitoring device having a transmitting device and an output device capable of outputting at least one of voice and text is able to communicate with the first communication device, and when a call is not established between the first communication device and the second communication device of the vehicle that has sent a call request to the remote monitoring device, the transmitting device can send a transmission request to the vehicle with which the call has not been established, requesting the transmission of text information, and the text information input using the input device to the vehicle-side display of the vehicle that has received the transmission request is transmitted to the remote monitoring device and output from the output device.

[0012] A seventh aspect of the present invention is a remote monitoring device comprising a first communication device provided in a plurality of vehicles having a vehicle-side display and an input device, a second communication device capable of communication, a transmitting device, and an output device capable of outputting at least one of voice and text, wherein when a call is not established between the first communication device and the second communication device of the vehicle that has sent a call request to the remote monitoring device, the transmitting device can send a transmission request to the vehicle with which the call has not been established, requesting the transmission of text information, and the text information input using the input device on the vehicle-side display of the vehicle that has received the transmission request is transmitted to the remote monitoring device and output from the output device. [Effects of the Invention]

[0013] In the vehicle remote monitoring system of the first aspect of the present invention, the vehicle of the sixth aspect, and the remote monitoring device of the seventh aspect, a situation may occur where, despite call requests being sent from multiple vehicles to the remote monitoring device, a call cannot be established between the first communication device of some vehicles and the second communication device of the remote monitoring device. In this case, the information that these some vehicles (and their occupants) wish to convey to the remote monitoring device is input as text information on the vehicle's display using the vehicle's input device. Furthermore, this text information is transmitted to the remote monitoring device and output from the output device. Therefore, the administrator of the remote monitoring device, who is in a call with another vehicle, can quickly recognize this text information.

[0014] In a second aspect of the vehicle remote monitoring system of the present invention, the output device is a display capable of displaying character information. Therefore, the administrator of the remote monitoring device can visually recognize the character information transmitted from the vehicle.

[0015] In a third aspect of the present invention, the remote monitoring system for vehicles allows the administrator of the remote monitoring device to send at least one message selected from a plurality of messages to the vehicle.

[0016] In the vehicle remote monitoring system according to the fourth aspect of the present invention, when the vehicle's input device inputs multiple response pieces of information for multiple received messages as character information to the vehicle-side display, each response piece of information is transmitted sequentially to the remote monitoring device. Therefore, compared to a system where all response pieces of information are transmitted simultaneously to the remote monitoring device only after all of them have been input to the vehicle-side display, the administrator of the remote monitoring device can quickly recognize each response piece of information.

[0017] A fifth aspect of the present invention allows a vehicle occupant who is inputting text information to a vehicle-side display using an input device to recognize that they will soon be able to communicate with a remote monitoring device. [Brief explanation of the drawing]

[0018] [Figure 1] This diagram shows the overall configuration of the vehicle remote monitoring system according to the embodiment. [Figure 2] It is a control block diagram of a control device of a remote monitoring device. [Figure 3] It is a functional block diagram of a control device. [Figure 4] It is a diagram showing a first display of a remote monitoring device. [Figure 5] It is a diagram showing a second display of a remote monitoring device. [Figure 6] It is an enlarged view of a part of the second display. [Figure 7] It is a control block diagram of an ECU of a bus. [Figure 8] It is a functional block diagram of an ECU. [Figure 9] It is a diagram showing a display of a bus. [Figure 10] It is an enlarged view of a question-and-answer display section of a bus display. [Figure 11] It is a diagram showing the question-and-answer display section after answer information is transmitted to the remote monitoring device. [Figure 12] It is a flowchart showing the processing executed by the control device of the remote monitoring device. [Figure 13] It is a flowchart showing the processing executed by the ECU of the bus.

Embodiments for Carrying Out the Invention

[0019] Hereinafter, embodiments of a vehicle remote monitoring system, a vehicle, and a remote monitoring device according to the present invention will be described with reference to the drawings.

[0020] As shown in FIG. 1, a vehicle remote monitoring system 10 (hereinafter referred to as system 10) of the embodiment has a remote monitoring device 20 and a plurality of buses (vehicles) 40A, 40B, 40C.

[0021] The remote monitoring device 20 shown in FIGS. 1 and 2 is installed in a building (not shown) owned by the administrator of the system 10. The remote monitoring device 20 includes a control device 21, a first display 24, a second display 25, an input device 27, and an audio input / output device 28.

[0022] As shown in Figure 2, the control device 21 has a hardware configuration that includes a CPU (Central Processing Unit) (processor) 21A, ROM (Read Only Memory) 21B, RAM (Random Access Memory) 21C, storage 21D, wireless communication interface (interface) 21E, and input / output interface 21G. The CPU 21A, ROM 21B, RAM 21C, storage 21D, wireless communication interface 21E, and input / output interface 21G are connected to each other so as to be able to communicate with one another via the internal bus 21Z. The CPU 21A and the CPU 41A, which will be described later, can acquire time-related information from a timer.

[0023] The CPU 21A is the central processing unit, which executes various programs and controls various components. The CPU 21A reads programs from ROM 21B or storage 21D and executes them using RAM 21C as the working area. The CPU 21A controls each component and performs various calculations according to the programs recorded in ROM 21B or storage 21D.

[0024] ROM21B stores various programs and data. For example, multiple programs are installed in ROM21B. ROM21B also records map data, call hold information, advance notification information, and vehicle ID information, which will be described later. RAM21C temporarily stores programs or data as a working area. Storage21D consists of a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs and data.

[0025] The map data includes information about the areas in which buses 40A, 40B, and 40C, which are registered vehicles in system 10, operate. Although only the three buses 40A, 40B, and 40C are depicted in Figure 1, many more buses are registered in system 10.

[0026] The wireless communication interface (I / F21E) is an interface for wireless communication with various devices. For example, the wireless communication interface (I / F21E) can communicate wirelessly with the wireless communication interface (I / F41E) on buses 40A, 40B, and 40C via a network (e.g., the Internet). The wireless communication interface (I / F21E) can send and receive audio data, camera image data, and sensor group detection values, as described later.

[0027] The input / output interface 21G is an interface for communicating with various devices. For example, the input / output interface 21G is connected to a first display 24, a second display (output device) 25, an input device 27, and an audio input / output device 28. The first display 24 and the second display 25 can display various images. For example, the first display 24 can display a map image 30, which will be described later. The second display 25 can display, for example, camera images, sensor group detection values, and response information (described later) received wirelessly by the wireless communication interface 21E from the wireless communication interface 41E on buses 40A, 40B, and 40C. The input device 27 can input various information to the control device 21. The input device 27 is, for example, a keyboard. The audio input / output device (output device) 28 is equipped with a microphone and a speaker.

[0028] Figure 3 shows an example of the functional configuration of the control device 21 in a block diagram. The control device 21 has a functional configuration consisting of a transmit / receive control unit 211, a SIP control unit 212, and a display control unit 213. These functions are realized by the CPU 21A reading and executing a program stored in the ROM 21B.

[0029] The transmit / receive control unit 211 controls the wireless communication interface 21E. The wireless communication interface 21E controlled by the transmit / receive control unit 211 can, for example, receive camera image data, sensor group detection values, and response information, and record data received from an external source in the ROM 21B and storage 21D. Furthermore, when a call using SIP, described later, is being performed, the wireless communication interface 21E can wirelessly transmit audio data input to the audio input / output device 28, and receive audio data received from the wireless communication interfaces 41E on buses 40A, 40B, and 40C.

[0030] When a second call request, described later, is issued from buses 40A, 40B, or 40C and a touch operation is performed on a transmission processing unit 37A, 37B, or 37C, described later, the transmission / reception control unit 211 wirelessly transmits the following question information and call hold information to the buses 40A, 40B, or 47C corresponding to the operated transmission processing unit 37A, 37B, or 37C.

[0031] Furthermore, when a second call request is issued from buses 40A, 40B, or 40C and a touch operation is performed on one of the pre-notification processing units 34A4, 34B4, or 34C4 (described later), the transmission / reception control unit 211 transmits the pre-notification information (described later) to the buses 40A, 40B, or 40C corresponding to the operated pre-notification processing unit 34A4, 34B4, or 34C4.

[0032] The SIP control unit 212 controls communication between the voice input / output device 28, which uses SIP (Session Initiation Protocol), and buses 40A, 40B, and 40C (speaker 44, microphone 45). In other words, the control unit 21 functions as a SIP server.

[0033] The SIP control unit 212 enables the transmission of voice information from the control device 21 to the selected vehicle (wireless communication I / F 41E) when a first call request is transmitted from the remote monitoring device 20 to the selected vehicle (wireless communication I / F 41E) by touch operation on the first call request operation units 34A2, 34B2, and 34C2, which will be described later. That is, the voice input to the voice input / output device 28 is output from the speaker (first call device) 44 of the selected vehicle. The selected vehicle is the bus corresponding to the first call request operation unit 34A2, 34B2, and 34C2 that was touched.

[0034] When the SIP control unit 212 receives a second call request (described later) from buses 40A, 40B, or 40C, and a touch operation is performed on the call authorization units 34A3, 34B3, or 34C3 (described later), the SIP control unit 212 establishes a call between the buses 40A, 40B, or 40C corresponding to the operated call authorization unit 34A3, 34B3, or 34C3 and the control device 21. That is, voice information input to the microphone (first call device) 45 of buses 40A, 40B, or 40C is output from the voice input / output device 28, and voice information input to the voice input / output device 28 is output from the speaker 44. On the other hand, the SIP control unit 212 does not authorize a call between the buses 40A, 40B, or 40C corresponding to the unoperated call authorization unit 34A3, 34B3, or 34C3 and the control device 21 (voice input / output device 28).

[0035] The display control unit 213 controls the first display 24 and the second display 25, both of which are touch panel type.

[0036] Figure 4 shows the first display 24 controlled by the display control unit 213. The first display 24 displays a map image 30 based on the map data described above. The map image 30 includes a road image 31, a bus image 32A representing bus 40A, a bus image 32B representing bus 40B, and a bus image 32C representing bus 40C. The display control unit 213 changes the positions of the bus images 32A, 32B, and 32C on the map image 30 based on the position information (GNSS information) transmitted from buses 40A, 40B, and 40C.

[0037] Figure 5 shows a second display 25 controlled by a display control unit 213. The second display 25 displays camera image display areas 33A, 33B, and 33C. Furthermore, the camera image display areas 33A, 33B, and 33C have a first display area 33A1, 33B1, 33C1, a second display area 33A2, 33B2, 33C2, and a third display area 33A3, 33B3, 33C3. The first display areas 33A1, 33B1, and 33C1 display camera images acquired by the in-vehicle cameras (cameras) 46 of the buses 40A, 40B, and 40C in real time. The second display areas 33A2, 33B2, and 33C2 display camera images acquired by the out-of-vehicle front cameras (cameras) (out-of-vehicle cameras) 47 of the buses 40A, 40B, and 40C in real time. The third display areas 33A3, 33B3, and 33C3 display camera images acquired in real time by the external rear cameras (cameras) 48 of buses 40A, 40B, and 40C.

[0038] Furthermore, the second display 25 is provided with bus information display units 35A, 35B, and 35C located to the right of the camera image display areas 33A, 33B, and 33C. Bus information display unit 35A corresponds to bus 40A, bus information display unit 35B corresponds to bus 40B, and bus information display unit 35C corresponds to bus 40C. Bus information display units 35A, 35B, and 35C display information regarding sensor group detection values ​​acquired by the sensor group 51 of buses 40A, 40B, and 40C, which will be described later, in real time. Specifically, bus information display units 35A, 35B, and 35C display information such as the vehicle speed, accelerator pedal opening, brake pedal depression amount, turn signal position, tire pressure, steering angle, and acceleration generated on the vehicle body for each bus 40A, 40B, and 40C.

[0039] Furthermore, the second display 25 displays the second call request reception display units 34A1, 34B1, 34C1, the first call request operation units 34A2, 34B2, 34C2, the call permission units 34A3, 34B3, 34C3, the pre-notification processing units 34A4, 34B4, 34C4, and the call termination operation units 34A5, 34B5, 34C5 to the right of the bus information display units 35A, 35B, and 35C, respectively. The second call request reception display unit 34A1, the first call request operation unit 34A2, the call permission unit 34A3, the pre-notification processing unit 34A4, and the call termination operation unit 34A5 correspond to bus 40A. The second call request reception display unit 34B1, the first call request operation unit 34B2, the call permission unit 34B3, the pre-notification processing unit 34B4, and the call termination operation unit 34B5 correspond to bus 40B. The second call request reception display unit 34C1, the first call request operation unit 34C2, the call permission unit 34C3, the pre-notification processing unit 34C4, and the call termination operation unit 34C5 correspond to bus 40C.

[0040] The second call request reception indicators 34A1, 34B1, and 34C1 are equipped with illumination fixtures and blink when the wireless communication I / F21E receives a second call request (call request) from buses 40A, 40B, and 40C.

[0041] As described above, when a touch operation is performed on the first call request operation units 34A2, 34B2, and 34C2, the first call request is wirelessly transmitted to the selected vehicle. Also, as described above, when a second call request is issued from buses 40A, 40B, and 40C, and the call permission units 34A3, 34B3, and 34C3 are operated, a call is established between the corresponding buses 40A, 40B, and 40C and the control device 21 (voice input / output device 28).

[0042] When a second call request is issued from buses 40A, 40B, or 40C and a touch operation is performed on the transmission processing units 37A, 37B, or 37C displayed on the second display 25, question information (transmission request) and call hold information are wirelessly transmitted to buses 40A, 40B, or 40C. The question information is displayed on the question display units 36A, 36B, or 36C formed on the second display 25. The question display unit 36A and the transmission processing unit 37A correspond to bus 40A, the question display unit 36B and the transmission processing unit 37B correspond to bus 40B, and the question display unit 36C and the transmission processing unit 37C correspond to bus 40C.

[0043] Storage 21D stores numerous standardized question messages (messages) consisting of predefined phrases. These standardized question messages represent information that the administrator wants to ask the crew (passengers) of buses 40A, 40B, and 40C. The standardized question messages include the following questions A to H, as well as numerous other standardized messages separate from questions A to H. These standardized messages are displayed in a designated area on the second display 25. Question A: Was there a collision? Question B: Are there any customers who are injured? Question C: Is the window glass broken? Question D: Is there a problem on the outside of the vehicle? Question E: Are the roads flooded? Question F: Are there any police officers? Question G: Are there any disputes between customers? Question H: Are criminal acts taking place inside the vehicle?

[0044] By using the input device 27, the administrator can select any standard question message from all the standard question messages displayed in the predetermined area and display it on the question display units 36A, 36B, and 36C. For example, as shown in Figure 6, the administrator can select question A as question 1 (Q1). Furthermore, by using the input device 27, the administrator can freely create various question messages. For example, as shown in Figure 6, the administrator can create the question message "Are there any customers experiencing bleeding?" as question 2 (Q2).

[0045] For example, the administrator can select a predetermined standard question message or create a question message based on the camera images displayed in the camera image display areas 33A, 33B, and 33C, and the information regarding the sensor group detection values ​​displayed in the bus information display units 35A, 35B, and 35C.

[0046] When a question is displayed on the question display units 36A, 36B, and 36C, and a touch operation is performed on the transmission processing units 37A, 37B, and 37C, the remote monitoring device 20 wirelessly transmits question information representing these questions and call hold information to the buses 40A, 40B, and 40C. The call hold information is a message such as, "We are currently on a call with another vehicle, so please wait a moment. Please answer the question we have sent."

[0047] Furthermore, when a second call request is issued from buses 40A, 40B, and 40C, and a touch operation is performed on the transmission processing units 37A, 37B, and 37C, and then a touch operation is performed on the pre-notification processing units 34A4, 34B4, and 34C4, the remote monitoring device 20 wirelessly transmits pre-notification information to buses 40A, 40B, and 40C. The pre-notification information may include messages such as, "Thank you for waiting. The call will now begin," or "The call will be available in 20 seconds."

[0048] When a touch operation is performed on the call termination operation units 34A5, 34B5, and 34C5, the SIP control unit 212 controls the communication between the remote monitoring device 20 and buses 40A, 40B, and 40C based on the second call request, rendering it impossible.

[0049] Furthermore, as shown in Figure 5, information sharing units 38A, 38B, and 38C are provided on the right side of the second display 25. Information sharing unit 38A corresponds to bus 40A, information sharing unit 38B corresponds to bus 40B, and information sharing unit 38C corresponds to bus 40C. As will be described later, the same image as the question answer display unit 55 of the display (vehicle-side display) 50, which will be described later and provided on buses 40A, 40B, and 40C that have received the question information, is displayed in real time on the information sharing units 38A, 38B, and 38C.

[0050] As shown in Figure 7, buses 40A, 40B, and 40C are equipped with an ECU 41, a GNSS (Global Navigation Satellite System) receiver 43, a speaker 44, a microphone 45, an in-vehicle camera 46, an external front camera 47, an external rear camera 48, a second call request operation unit 49S, a call termination operation unit 49F, a touch panel display 50, a sensor group 51, and a driver assistance operation unit 62. As shown in Figure 7, the ECU 41 has the same hardware configuration as the control unit 21. That is, the ECU 41 has a hardware configuration of a CPU 41A, ROM 41B, RAM 41C, storage 41D, wireless communication I / F 41E, input / output I / F 41G, and an internal bus 41Z. For example, multiple programs are installed in ROM 41B.

[0051] The input / output interface 41G of the ECU 41 is connected to a GNSS receiver 43, a speaker 44, a microphone 45, an in-vehicle camera 46, an out-of-vehicle front camera 47, an out-of-vehicle rear camera 48, a second call request operation unit 49S, a call termination operation unit 49F, a display 50, a sensor group 51, and a driver assistance operation unit 62.

[0052] The GNSS receiver 43 receives GNSS signals transmitted from GNSS satellites and acquires position information for buses 40A, 40B, and 40C equipped with an ECU 41. The acquired position information is wirelessly transmitted to the remote monitoring device 20 via a wireless communication interface 41E.

[0053] Speaker 44 can output various sounds. For example, speaker 44 can output audio based on call hold information. Furthermore, when speaker 44 receives a first call request or when a call is established between the remote monitoring device 20 and buses 40A, 40B, and 40C, it can output audio based on the audio input to the audio input / output device 28.

[0054] Various types of audio are input to microphone 45. For example, when audio is input to microphone 45 when communication is established between the remote monitoring device 20 and buses 40A, 40B, and 40C, this audio is output from the audio input / output device 28 of the remote monitoring device 20.

[0055] The in-vehicle camera 46 is installed inside the buses 40A, 40B, and 40C. For example, the in-vehicle camera 46 is installed on the ceiling of the buses 40A, 40B, and 40C. The in-vehicle camera 46 repeatedly acquires image data representing subjects inside the buses 40A, 40B, and 40C at predetermined intervals.

[0056] The external front camera 47 is installed at the front end of buses 40A, 40B, and 40C. The external front camera 47 repeatedly acquires image data representing an external subject located in front of the front end of buses 40A, 40B, and 40C at predetermined intervals.

[0057] The external rear camera 48 is installed at the rear end of buses 40A, 40B, and 40C. The external rear camera 48 repeatedly acquires image data representing an external subject located behind the rear end of buses 40A, 40B, and 40C at predetermined intervals.

[0058] The second call request operation unit 49S and the call termination operation unit 49F are, for example, mechanical buttons installed inside the buses 40A, 40B, and 40C. However, the second call request operation unit 49S and the call termination operation unit 49F may also be images (operational images) displayed on the display 50.

[0059] The display 50 is installed inside the buses 40A, 40B, and 40C. For example, the display 50 is installed on the interior side of the buses 40A, 40B, and 40C. The number of displays 50 installed in buses 40A, 40B, and 40C may be one or more. As shown in Figure 9, the display 50 has an operation information display unit 53. The operation information display unit 53 contains information regarding the names of the bus stops that each of the buses 40A, 40B, and 40C is scheduled to stop at, the scheduled arrival times at each bus stop, and the current time.

[0060] Furthermore, as shown in Figure 9, the right side of the display 50 displays a question and answer display unit 55 and an input unit (input device) 60. As shown in Figure 10, the question and answer display unit 55 includes a character display unit 56 that displays question information received from the remote monitoring device 20 and answer information (response information) (character information) for the question information, individual transmission buttons 57 for individually sending each piece of answer information displayed on the character display unit 56 to the remote monitoring device 20, and a batch transmission button 58 for sending all answer information to the remote monitoring device 20 at once. For example, if the question information (question message) shown in Figure 6 is sent to the bus 40A, the same question information as in Figure 6 will be displayed on the character display unit 56 as shown in Figure 10.

[0061] Furthermore, as shown in Figure 9, an input unit 60 for inputting various character information into the character display unit 56 is provided at the lower end of the display 50.

[0062] Sensor group 51 is a collective term for the sensors installed on buses 40A, 40B, and 40C. These sensors include, for example, a vehicle speed sensor, an accelerator pedal sensor, a brake pedal sensor, a turn signal sensor, a LiDAR sensor, a radar sensor, air pressure sensors installed on the tires of each wheel, a steering angle sensor that detects the steering angle of the steering wheel, and an acceleration sensor that detects acceleration in various directions generated on the vehicle body. Each of these sensors repeatedly performs a detection operation after a predetermined time has elapsed. In this specification, the detection value acquired by each sensor included in sensor group 51 is referred to as the sensor group detection value.

[0063] The driver assistance control unit 62 is a switch located on the instrument panel of buses 40A, 40B, and 40C.

[0064] Figure 8 shows an example of the functional configuration of 41 in a block diagram. ECU 41 has a functional configuration consisting of a transmit / receive control unit 411, a call control unit 412, and a driver assistance control unit 413. These functions are realized by the CPU 41A reading and executing a program stored in ROM 41B.

[0065] The transmit / receive control unit 411 controls the wireless communication interface 41E. The wireless communication interface 41E controlled by the transmit / receive control unit 411 can communicate wirelessly with the wireless communication interface 21E of the remote monitoring device 20 via the network. For example, by controlling the wireless communication interface 41E, the transmit / receive control unit 411 repeatedly wirelessly transmits image data, location information, and sensor group detection values ​​from the in-vehicle camera 46, the exterior front camera 47, and the exterior rear camera 48 to the remote monitoring device 20 at predetermined intervals.

[0066] Furthermore, the transmission / reception control unit 411 wirelessly transmits an automatic driving execution signal to the remote monitoring device 20 when the driving support operation unit 62 is turned ON and buses 40A, 40B, and 40C are performing Level 4 or Level 5 driving support control, as described later.

[0067] The call control unit 412 controls the speaker 44 and the microphone 45. For example, when the wireless communication interface 41E receives a first call request from the remote monitoring device 20, the call control unit 412 enables the function of the speaker 44.

[0068] Furthermore, when the second call request operation unit 49S is operated by a crew member, the call control unit 412 causes the wireless communication I / F 41E to wirelessly transmit the second call request to the remote monitoring device 20.

[0069] Furthermore, when the call termination operation unit 49F is operated by a crew member, the call control unit 412 terminates the call between the buses 40A, 40B, and 40C and the remote monitoring device 20 based on the second call request.

[0070] Furthermore, when a second call request is wirelessly transmitted to the remote monitoring device 20 and a touch operation is performed on the call permission units 34A3, 34B3, and 34C3, the call control unit 412 enables the functions of the speaker 44 and microphone 45.

[0071] Furthermore, when the wireless communication interface 41E receives voice data based on call hold information from the remote monitoring device 20, the call control unit 412 causes the speaker 44 to output voice based on this voice data.

[0072] When the driver assistance control unit 62, provided on buses 40A, 40B, and 40C, is turned ON, the driver assistance control unit 413 performs driver assistance control using data acquired by the GNSS receiver 43, the external front camera 47, the external rear camera 48, and the sensor group 51. This driver assistance control is performed by operating various devices on buses 40A, 40B, and 40C, including the brake system, engine, electric motor, steering system (not shown), and turn signals. In this specification, "driver assistance control" includes driver assistance control levels 1 to 5 as defined by the SAE (Society of Automotive Engineers). When a predetermined operation is performed on the driver assistance control unit 62, the driver assistance control unit 413 determines (selects) the level of driver assistance control to be performed from levels 1 to 5.

[0073] (Mechanism of action and effect) Next, the operation and effects of this embodiment will be described.

[0074] First, the processing flow of the remote monitoring device 20 in this embodiment will be explained using the flowchart in Figure 12. The CPU 21A of the remote monitoring device 20 repeatedly executes the process shown in the flowchart in Figure 12 after a predetermined amount of time has elapsed.

[0075] First, in step S10 (the word "step" will be omitted hereafter), the CPU 21A determines whether or not it has received camera image data and sensor group detection values ​​from any of the buses 40A, 40B, or 40C.

[0076] If CPU 21A determines Yes in S10, it proceeds to S11, where it displays the received camera images in the camera image display areas 33A, 33B, and 33C of the second display 25, and also displays the sensor group detection values ​​in the bus information display units 35A, 35B, and 35C.

[0077] When S10 determines No, or when CPU 21A has finished processing S11, it proceeds to S12 to determine whether or not it has received location information from any of the buses 40A, 40B, or 40C.

[0078] If CPU 21A determines Yes in S12, it proceeds to S13 and displays bus images 32A, 32B, and 32C at the positions corresponding to the received location information on the map image 30 of the first display 24.

[0079] When the result in S12 is determined to be No, or when the CPU 21A has finished processing in S13, it proceeds to S14 and determines whether or not it has received a second call request from any of the buses 40A, 40B, or 40C.

[0080] If CPU 21A determines "Yes" in S14, it proceeds to S15. At this time, CPU 21A blinks the second call request reception indicators 34A1, 34B1, and 34C1 corresponding to the buses 40A, 40B, and 40C that issued the second call request for a predetermined period of time.

[0081] In S15, the CPU 21A determines whether a touch operation has been performed on the call permission units 34A3, 34B3, and 34C3 corresponding to the blinking second call request reception display units 34A1, 34B1, and 34C1.

[0082] If CPU 21A determines Yes in S15, it proceeds to S16 and enables communication between the audio input / output device 28 of the remote monitoring device 20 and the speakers 44 and microphones 45 of buses 40A, 40B, and 40C corresponding to the touch-operated call permission units 34A3, 34B3, and 34C3.

[0083] After completing the processing in S16, CPU 21A proceeds to S17 and determines whether or not a call termination operation has been performed. That is, CPU 21A determines whether or not a touch operation has been performed on the call termination operation units 34A5, 34B5, and 34C5.

[0084] If CPU21A determines "Yes" in S17, it proceeds to S18 and disables communication between buses 40A, 40B, and 40C.

[0085] On the other hand, CPU 21A, which determined No in S15, proceeds to S19 to determine whether a touch operation was performed on any of the transmission processing units 37A, 37B, or 37C.

[0086] If CPU 21A determines "Yes" in S19, it proceeds to S20 and transmits question information and call hold information to the wireless communication I / F 41E of buses 40A, 40B, and 40C corresponding to the touched parts 34A4, 34B4, and 34C4.

[0087] After completing the processing in S20, CPU21A proceeds to S21 and determines whether or not it has received response information from buses 40A, 40B, and 40C.

[0088] If CPU 21A determines "Yes" in S21, it proceeds to S22 and displays the received response information on the information sharing units 38A, 38B, and 38C of the second display 25. For example, if the response information (A1, A2) shown in Figure 10 is entered in the character display unit 56 of the bus 40A's display 50, the character information shown in Figure 6 will be displayed on the information sharing unit 38A.

[0089] When the result in S19 or S21 is determined to be No, or when the processing in S22 is completed, the CPU 21A proceeds to S23 and determines whether or not a touch operation has been performed on the pre-notification processing units 34A4, 34B4, and 34C4.

[0090] If CPU 21A determines "Yes" in S23, it proceeds to S24 and sends advance notification information to buses 40A, 40B, and 40C. At this time, the administrator can touch the advance notification processing units 34A4, 34B4, and 34C4, taking into account the status of the answer information displayed on the information sharing unit 38A. For example, as shown on the information sharing unit 38A in Figure 6, the administrator may touch the advance notification processing unit 34A4 while the crew (passenger) of bus 40A is filling in the answer information A2. In this case, the advance notification information sent to bus 40A might be, for example, "You will be able to make a call in 20 seconds." Alternatively, as shown on the question and answer display unit 55 in Figure 11, the administrator may touch the advance notification processing unit 34A4 after the crew (passenger) of bus 40A has finished filling in the answer information A2. In this case, the advance notification information sent to bus 40A might be, for example, "Thank you for waiting. The call will now begin."

[0091] After completing the processing in S24, CPU 21A proceeds to S15. If it determines Yes in S15, CPU 21A proceeds to S16 and establishes a call between the buses 40A, 40B, and 40C corresponding to the touched pre-notification processing units 34A4, 34B4, and 34C4.

[0092] When the process in S18 is completed, or when the result in S17 and S23 is determined to be No, the CPU 21A temporarily terminates the process in the flowchart shown in Figure 12.

[0093] Next, we will explain the processes performed by CPU 41A on buses 40A, 40B, and 40C using the flowchart in Figure 13. CPU 41A repeatedly executes the processes shown in the flowchart in Figure 13 at predetermined intervals.

[0094] First, in S30, the CPU 41A controls the wireless communication interface 41E to wirelessly transmit camera image data acquired by the in-vehicle camera 46, the exterior front camera 47, and the exterior rear camera 48, as well as the sensor group detection values, to the remote monitoring device 20.

[0095] After completing the processing in S30, CPU 41A proceeds to S31 and determines whether or not the second call request operation unit 49S has been operated.

[0096] If the CPU 41A determines "Yes" in S31, it proceeds to S32 and wirelessly transmits the second call request to the remote monitoring device 20.

[0097] When the processing in S32 is complete, the CPU 41A proceeds to S33 and determines whether or not a call has been established between itself and the remote monitoring device 20.

[0098] When the system determines "Yes" in S33, the CPU 41A proceeds to S34 and enables the functions of the speaker 44 and microphone 45. As a result, the voice spoken by the administrator of the remote monitoring device 20 and input to the audio input / output device 28 is output from the speaker 44. Therefore, the occupants of buses 40A, 40B, and 40C can hear this voice. Furthermore, when voices spoken by the occupants are input to the microphone 45, this voice is output from the audio input / output device 28 of the remote monitoring device 20.

[0099] After completing the processing in S34, CPU 21A proceeds to S35 and determines whether or not a call termination operation has been performed. That is, CPU 41A determines whether or not the call termination operation unit 49F has been operated.

[0100] If CPU 21A determines "Yes" in S35, it proceeds to S36 and disables communication with the remote monitoring device 20.

[0101] On the other hand, if the result in S33 is No, the CPU 41A proceeds to S37 and determines whether the wireless communication I / F 41E has received question information and call hold information.

[0102] If the answer is "Yes" in S37, the CPU 41A proceeds to S38, where it displays the question information on the question answer display unit 55 of the display 50 and outputs call hold information to the speaker 44.

[0103] When the processing in S38 is completed, the CPU 41A proceeds to S39 and uses the input unit 60 to determine whether or not response information has been entered.

[0104] If the CPU 41A determines "Yes" in S39, it proceeds to S40 and displays the answer information on the question answer display unit 55 of the display 50.

[0105] When the processing in S40 is completed, the CPU 41A proceeds to S41 and determines whether a touch operation has been performed on the individual send button 57 or the batch send button 58.

[0106] If the CPU 41A determines "Yes" in S41, it proceeds to S42 and wirelessly transmits the response information to the remote monitoring device 20.

[0107] Furthermore, once all response information has been transmitted to the remote monitoring device 20, the question and answer display unit 55 will continue to display all question information and all answer information, as shown in Figure 11. The CPU 41A may maintain the question and answer display unit 55 in this state until, for example, the subsequent communication between the occupants of buses 40A, 40B, and 40C and the administrator of the remote monitoring device 20 is completed. That is, the question and answer display unit 55 may be maintained in this state until the aforementioned call termination operation units 34A5, 34B5, 34C5, or call termination operation unit 49F is operated.

[0108] When the processing in S42 is completed, CPU41A proceeds to S43 and determines whether or not it has received prior notification information.

[0109] If the response in S43 is "Yes", the CPU 41A proceeds to S44 and displays the advance notification information on the display 50. For example, the advance notification information may be displayed as a pop-up display superimposed on the question answer display unit 55. Also, as described above, if the advance notification information is received while the crew (passengers) of buses 40A, 40B, and 40C are filling in the answer information A2, text information such as "You will be able to talk in 20 seconds." will be displayed on the display 50. Also, if the advance notification information is received after the crew (passengers) of buses 40A, 40B, and 40C have finished filling in the answer information A2, text information such as "Thank you for waiting. The call will now begin." will be displayed on the display 50.

[0110] When processing S36 or S44 is completed, or when No is determined in S31, 35, or 43, CPU 41A temporarily terminates the processing of the flowchart in Figure 13.

[0111] As described above, in the system 10 of the embodiment, even though a second call request is sent to the remote monitoring device 20 from multiple buses 40A, 40B, and 40C, a situation may occur where a call is not established between some of the buses 40A, 40B, and 40C and the remote monitoring device 20. In this case, the response information that the crew (passengers) of these buses 40A, 40B, and 40C wish to convey to the remote monitoring device 20 is input as text information to the display 50 using the input unit 60. Furthermore, this response information is transmitted to the remote monitoring device 20 and displayed on the second display 25 (information sharing unit 38A). Therefore, for example, an administrator who is in a call with the crew of another bus 40A, 40B, or 40C using the audio input / output device 28 can quickly recognize this response information while continuing the call. As a result, when the administrator subsequently calls the crew of the bus 40A, 40B, or 40C that sent the response information, the administrator can have a smooth conversation with that crew member. Furthermore, crew members who have sent a second call request but are not immediately permitted to speak with the administrator are less likely to become emotionally unstable.

[0112] Furthermore, the remote monitoring device 20 is equipped with a second display 25 as an output device that outputs text information (response information) received from buses 40A, 40B, and 40C. Therefore, the administrator of the remote monitoring device 20 can visually recognize the response information transmitted from buses 40A, 40B, and 40C. As a result, the administrator can easily recognize this response information even when in a call with the crew of another bus 40A, 40B, or 40C.

[0113] Furthermore, the administrator of the remote monitoring device 20 can determine the conditions inside and outside buses 40A, 40B, and 40C that sent the second call request from the camera images of buses 40A, 40B, and 40C displayed in the camera image display areas 33A, 33B, and 33C of the second display 25, and can determine whether or not a malfunction or failure has occurred in buses 40A, 40B, and 40C based on the sensor group detection values ​​displayed in the bus information display units 35A, 35B, and 35C. Based on these determinations, the administrator can select an appropriate standard question message for each bus 40A, 40B, and 40C from all the standard question messages, or create an appropriate question message using the input device 27.

[0114] Furthermore, buses 40A, 40B, and 40C can sequentially transmit multiple answer information (A1, A2) entered into the question answer display unit 55 of the display 50 to the remote monitoring device 20 using the individual transmit buttons 57 corresponding to each answer information. Therefore, compared to the case where all answer information is transmitted to the remote monitoring device 20 all at once after all answer information has been entered into the question answer display unit 55, the administrator of the remote monitoring device 20 can quickly recognize each answer information.

[0115] Furthermore, the remote monitoring device 20 can transmit advance notification information to buses 40A, 40B, and 40C. Therefore, for example, a crew member who is in the process of inputting answer information into the question and answer display unit 55 using the input unit 60 can be aware in advance that they will soon be able to communicate with the remote monitoring device 20. In other words, if advance notification information is not transmitted to buses 40A, 40B, and 40C, there is a risk that the voice of the administrator may suddenly be output from the speaker 44 while the crew member is in the process of inputting answer information into the question and answer display unit 55, but in this embodiment, such a problem is less likely to occur.

[0116] The system 10, remote monitoring device 20, and buses 40A, 40B, and 40C according to the embodiment have been described above, but these can be modified as appropriate without departing from the spirit of the present invention.

[0117] The audio input / output device 28 of the remote monitoring device 20 may output the response information transmitted from buses 40A, 40B, and 40C as audio. In this case, the response information may or may not be displayed on the information sharing units 38A, 38B, and 38C.

[0118] The voices emitted by the occupants of buses 40A, 40B, and 40C may be input to the ECU 41 as response information via microphones installed inside the vehicle. In this case as well, this response information may be displayed on the information sharing units 38A, 38B, and 38C of the second display 25, or output as audio from the audio input / output device 28.

[0119] If a predetermined time has elapsed since the second call request reception display units 34A1, 34B1, and 34C1 began flashing, and the call permission units 34A3, 34B3, and 34C3 and the pre-notification processing units 34A4, 34B4, and 34C4 have not been operated, the CPU 21A may automatically transmit question information and call hold information to buses 40A, 40B, and 40C. In this case, the CPU 21A may also recognize the conditions inside and outside buses 40A, 40B, and 40C based on the camera image data and sensor group detection values ​​received from buses 40A, 40B, and 40C, and select from a set of standard question messages or create a predetermined question message based on this recognition result. Furthermore, in this case, the CPU 21A may also recognize the conditions inside and outside buses 40A, 40B, and 40C by using a learning model (machine learning) that determines the relationship between the camera image data and sensor group detection values ​​and the conditions inside and outside buses 40A, 40B, and 40C.

[0120] After the transmission processing units 37A, 37B, and 37C have been operated, and before the pre-notification processing units 34A4, 34B4, and 34C4 have been operated, the administrator of the remote monitoring device 20 may operate the call permission units 34A3, 34B3, and 34C3. That is, the administrator may establish a call between the remote monitoring device 20 and buses 40A, 40B, and 40C without transmitting pre-notification information to buses 40A, 40B, and 40C.

[0121] Information requesting the crew to answer a question, included in the call hold information transmitted from the remote monitoring device 20 to buses 40A, 40B, and 40C, may be displayed as text on the display 50.

[0122] For example, buses 40A, 40B, and 40C do not necessarily need to be able to perform driver assistance control. In other words, the driver may operate buses 40A, 40B, and 40C.

[0123] Call hold information may also be in the form of text or images. In this case, for example, text or images representing the call hold information will be displayed on a display (not shown) provided in buses 40A, 40B, and 40C. This text could be, for example, "We are currently on a call with another vehicle, please wait a moment," or text representing the estimated time until the call is established.

[0124] The remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the first call request operation units 34A2, 34B2, and 34C2. Similarly, the remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the call permission units 34A3, 34B3, and 34C3. Similarly, the remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the pre-notification processing units 34A4, 34B4, and 34C4. Similarly, the remote monitoring device 20 may be equipped with mechanical buttons having functions corresponding to the call termination operation units 34A5, 34B5, and 34C5.

[0125] Images (operation units) having functions equivalent to the second call request operation unit 49S and the call termination operation unit 49F may be displayed on the displays 50 of buses 40A, 40B, and 40C.

[0126] The character information entered into the display 50 using the input unit 60 may be sent to the remote monitoring device 20 along with the second call request. This message is displayed on the second display 25 of the remote monitoring device 20. This message may, for example, indicate the reason why the occupants of buses 40A, 40B, and 40C who entered the character information wish to speak with the administrator.

[0127] Buses 40A, 40B, and 40C may be equipped with at least one of the following: an in-vehicle camera 46, an exterior front camera 47, and an exterior rear camera 48. Furthermore, buses 40A, 40B, and 40C may be equipped with cameras that photograph the outside of the vehicle. The in-vehicle camera 46, exterior front camera 47, exterior rear camera 48, and cameras that photograph the outside of the vehicle may include multiple cameras. That is, for example, buses 40A, 40B, and 40C may be equipped with five in-vehicle cameras 46.

[0128] The number of displays provided on the remote monitoring device 20 does not have to be two; it may be one or three or more. If the remote monitoring device 20 has only one display, all images displayed on the first display 24 and the second display 25 will be displayed on this display.

[0129] Communication control between the remote monitoring device 20 and buses 40A, 40B, and 40C may be performed using a control method other than SIP. For example, communication control may be performed using Web Real-Time Communication. In this case, SIP may be combined with a control method other than SIP.

[0130] Multiple speakers 44 and microphones 45 may be provided on buses 40A, 40B, and 40C.

[0131] Furthermore, a bus may have multiple vehicles. For example, if a bus is composed of multiple vehicles connected to each other, each vehicle may be equipped with a speaker 44, a microphone 45, an in-vehicle camera 46, an exterior camera, and a display. In this case, the remote monitoring device 20 may send a first communication request to all vehicles, or it may send a first communication request to only some of the vehicles. That is, in this case, each vehicle can be a selected vehicle. Also, when a second call request is sent from each vehicle to the remote monitoring device 20 using the second call request operation unit 49S provided in each vehicle that makes up a single bus, the remote monitoring device 20 may establish a call with the speaker 44 and microphone 45 of some vehicles while not establishing a call with the remaining vehicles, taking into consideration the camera images acquired by the in-vehicle cameras 46 of each vehicle displayed on the second display 25. Furthermore, in this case, the remote monitoring device 20 may send question information and call hold information to the remaining vehicles. Furthermore, when a second call request is sent to the remote monitoring device 20 from each vehicle constituting a single bus, the remote monitoring device 20 may establish communication between the speakers 44 and microphones 45 of all vehicles.

[0132] The present invention may also be applied to vehicles of a different type than buses 40A, 40B, and 40C (for example, passenger cars).

[0133] The data such as map data stored in ROM21B may be stored on an external server (not shown), and the control device 21 may retrieve this data by accessing the external server via the Internet.

[0134] When a question is displayed on the question display units 36A, 36B, and 36C, and a touch operation is performed on the transmission processing units 37A, 37B, and 37C, the remote monitoring device 20 may transmit a question signal corresponding to each of these question information and a hold signal corresponding to the call hold information to the buses 40A, 40B, and 40C. The ROM 41B or storage 41D of the buses 40A, 40B, and 40C stores the above-mentioned standard question messages (question information) and call hold information. The question answer display unit 55 of the display 50 of the buses 40A, 40B, and 40C that receives a question signal displays the question information corresponding to the received question signal, which is recorded in the ROM 41B or storage 41D. In addition, a display (not shown) provided on the buses 40A, 40B, and 40C that receives a hold signal displays characters representing the call hold information recorded in the ROM 41B or storage 41D. [Explanation of Symbols]

[0135] 10. Vehicle Remote Monitoring System (System) 20 Remote monitoring device 21E Wireless communication I / F (transmission device) 25. Second display (display) (output device) 28. Voice Input / Output Device (Second Communication Device) (Output Device) 40A 40B 40C Bus (vehicle) 44. Speaker (First communication device) 45. Microphone (First communication device) 50. Display (vehicle-side display) 60 Input section (input device)

Claims

1. Multiple vehicles having a vehicle-side display, an input device and a first communication device, A remote monitoring device having a second communication device, a transmitting device, and an output device capable of outputting at least one of voice and text, which are wirelessly connected to the first communication device. Equipped with, When a call is not established between the first and second communication devices of the vehicle that has sent a call request to the remote monitoring device, the transmitting device can send a transmission request to the vehicle from which the call has not been established, requesting the transmission of text information. A vehicle remote monitoring system in which the character information entered using the input device on the vehicle-side display of the vehicle that received the transmission request is transmitted to the remote monitoring device and output from the output device.

2. The vehicle remote monitoring system according to claim 1, wherein the output device of the remote monitoring device is a display capable of displaying character information.

3. The vehicle remote monitoring system according to claim 1 or 2, wherein the transmitting device is capable of transmitting at least one message selected from a set of standard messages to the vehicle along with the transmission request.

4. The vehicle remote monitoring system according to claim 3, in which the input device of the vehicle that has received a plurality of the aforementioned messages inputs a plurality of response pieces of information for each of the aforementioned messages as character information to the vehicle-side display, and the response pieces of information are sequentially transmitted to the remote monitoring device.

5. The vehicle remote monitoring system according to claim 1 or 2, wherein the transmitting device can notify the vehicle that it is ready to establish a call while the aforementioned character information is being input to the vehicle's display.

6. A vehicle having a vehicle-side display, an input device, and a first communication device, A second communication device provided in a remote monitoring device having a transmitting device and an output device capable of outputting at least one of voice and text is capable of communicating with the first communication device. When a call is not established between the first and second communication devices of the vehicle that has sent a call request to the remote monitoring device, the transmitting device can send a transmission request to the vehicle from which the call has not been established, requesting the transmission of text information. A vehicle in which the character information entered using the input device on the vehicle-side display of the vehicle that received the transmission request is transmitted to the remote monitoring device and output from the output device.

7. A first communication device provided in multiple vehicles having a vehicle-side display and input device, A remote monitoring device having a second communication device capable of communication, a transmitting device, and an output device capable of outputting at least one of voice and text, When a call is not established between the first and second communication devices of the vehicle that has sent a call request to the remote monitoring device, the transmitting device can send a transmission request to the vehicle from which the call has not been established, requesting the transmission of text information. The remote monitoring device transmits the character information, which has been input to the vehicle-side display of the vehicle that received the transmission request using the input device, to the remote monitoring device and outputs it from the output device.

Images