Driving support device, system, program, and driving support method

Abstract

To improve safety of operation of an automatic driving vehicle.SOLUTION: A driving support device 10 comprises a control section which acquires a moving image obtained by imaging a sidewalk which is in contact with a roadway where an automatic driving vehicle 51 travels and on which pedestrians walk using a camera 30 and, when a pedestrian is detected in the acquired moving image, outputs a notification that the moving image is displayable to a terminal device 20 of an operator 52 of the automatic driving vehicle 51.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a driving support device, a system, a program, and a driving support method.

Background Art

[0002] Patent Document 1 discloses a notification system that recognizes the behavior of an autonomous vehicle scheduled to pass through a predetermined traffic area and notifies other vehicles or pedestrians than the autonomous vehicle of the availability of passage and warnings.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An operator inside the vehicle or at a remote location may support the operation of the autonomous vehicle. However, in the conventional technology, a video of a pedestrian is not presented to the operator. Therefore, the safety of the operation of the autonomous vehicle is not sufficient.

[0005] An object of the present disclosure is to improve the safety of the operation of an autonomous vehicle.

Means for Solving the Problems

[0006] The driving support device according to the present disclosure acquires a video obtained by photographing a sidewalk on which a pedestrian passes in contact with the lane on which the autonomous vehicle travels with a camera, and when a pedestrian is detected in the acquired video, outputs a notification to the effect that the video can be displayed to the terminal device of the operator of the autonomous vehicle, and includes a control unit.

[0007] The program according to the present disclosure Output a notification indicating that a video obtained by photographing a sidewalk where pedestrians pass, which is adjacent to the lane on which the automated driving vehicle travels, can be displayed. In response to an operation by the operator of the automated driving vehicle to instruct the display of the video, display the video. Cause a computer to execute operations including these.

[0008] The driving support method according to the present disclosure is as follows. The driving support device acquires a video obtained by photographing a sidewalk where pedestrians pass, which is adjacent to the lane on which the automated driving vehicle travels, using a camera. When the driving support device detects a pedestrian in the acquired video, it causes the terminal device of the operator of the automated driving vehicle to output a notification indicating that the video can be displayed. This includes the above.

Effect of the Invention

[0009] According to the present disclosure, the safety of the operation of the automated driving vehicle is improved.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0011] Hereinafter, some embodiments of the present disclosure will be described with reference to the drawings.

[0012] In each figure, the same or corresponding parts are denoted by the same reference numerals. In the description of each embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0013] The first embodiment, which is one embodiment of the present disclosure, will be described.

[0014] With reference to FIG. 1, the configuration of the system 1 according to this embodiment will be described.

[0015] The system 1 according to this embodiment includes a driving support device 10, a terminal device 20, and a camera 30. The driving support device 10 can communicate with the terminal device 20 via the network 40. The driving support device 10 can communicate with the camera 30 via the network 40.

[0016] The driving support device 10 is installed in a facility such as a data center. The driving support device 10 is a computer such as a server belonging to a cloud computing system or other computing system.

[0017] The terminal device 20 is used by an operator 52 who supports the operation of the autonomous vehicle 51. The operator 52 may be inside the vehicle or in an external facility such as an operation management room. When the operator 52 is inside the vehicle, the terminal device 20 is held by the operator 52. When the operator 52 is in an external facility, the terminal device 20 is installed in the external facility. The terminal device 20 is, for example, a mobile device such as a mobile phone, a smartphone, or a tablet, or a PC. "PC" is an abbreviation for personal computer.

[0018] As shown in FIG. 2, the camera 30 is installed at a position where it can photograph the sidewalk 55 on which the pedestrian 53 passes, in contact with the road lane 54 on which the autonomous vehicle 51 travels. In the example shown in FIG. 2, the position where the camera 30 is installed is near the blind spot from the road lane 54, but the camera 30 may be installed directly in the blind spot. In the example shown in FIG. 2, the position where the camera 30 is installed is near the crosswalk 56 on the road lane 54, but the camera 30 may be installed directly on the crosswalk 56. In the example shown in FIG. 2, when there is a vehicle traveling from left to right on the road lane 54, until the vehicle passes the crosswalk 56, the portion of the sidewalk 55 blocked by the building 57 becomes a blind spot from the road lane 54. The camera 30 is supported by a long pillar such as a cylinder so as to be located at a desired height. In addition to the camera 30, a light such as an LED, a speaker, or a display for notifying the pedestrian 53 of the approach of the autonomous vehicle 51 may be attached to this pillar. "LED" is an abbreviation for light-emitting diode.

[0019] The autonomous vehicle 51 is, for example, any type of automobile such as a gasoline vehicle, a diesel vehicle, a hydrogen vehicle, an HEV, a PHEV, a BEV, or an FCEV. "HEV" is an abbreviation for hybrid electric vehicle. "PHEV" is an abbreviation for plug-in hybrid electric vehicle. "BEV" is an abbreviation for battery electric vehicle. "FCEV" is an abbreviation for fuel cell electric vehicle. The autonomous vehicle 51 is an AV in this embodiment and is automated at a high level. "AV" is an abbreviation for autonomous vehicle. The level of automation is, for example, either level 3 or level 4 in the SAE level classification. "SAE" is the abbreviation of Society of Automotive Engineers.

[0020] Network 40 includes the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. Network 40 may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network is, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0021] Referring to FIGS. 1 and 2, the outline of this embodiment will be described.

[0022] In this embodiment, the driving support device 10 acquires the moving image VD obtained by photographing the sidewalk 55 with the camera 30. When the driving support device 10 detects a pedestrian 53 in the acquired moving image VD, it causes the terminal device 20 of the operator 52 of the autonomous vehicle 51 to output a notification ND indicating that the moving image VD can be displayed. The pedestrian 53 includes not only a person but also a moving body capable of passing through the sidewalk 55 such as a wheelchair. The terminal device 20 outputs the notification ND. The terminal device 20 displays the moving image VD in response to an operation of instructing the display of the moving image VD by the operator 52 of the autonomous vehicle 51.

[0023] According to this embodiment, in response to the request of the operator 52 of the autonomous vehicle 51, the moving image VD of the pedestrian 53 can be presented to the operator 52. Therefore, the safety of the operation of the autonomous vehicle 51 is improved.

[0024] In the example shown in FIG. 2, the operator 52 is riding in the autonomous vehicle 51. The operator 52 operates the terminal device 20 and checks the notifications from the terminal device 20. The pedestrian 53 is passing on the sidewalk 55 and is about to cross the crosswalk 56 on the roadway 54. Since the pedestrian 53 is behind the building 57 that blocks the sidewalk 55 in the traveling direction of the autonomous vehicle 51, it is difficult for the operator 52 to visually recognize the pedestrian 53. A camera 30 is installed near the blind spot from the roadway 54 that cannot be visually recognized by the operator 52 in this way. The camera 30 is photographing the sidewalk 55 with its back to the roadway 54. The camera 30 can transmit the video VD obtained by photographing the sidewalk 55 to the driving support device 10 via the network 40.

[0025] Referring to FIG. 3, the configuration of the driving support device 10 according to the present embodiment will be described.

[0026] The driving support device 10 includes a control unit 11, a storage unit 12, and a communication unit 13.

[0027] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. While controlling each part of the driving support device 10, the control unit 11 executes processing related to the operation of the driving support device 10.

[0028] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, RAM or ROM. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, SRAM or DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used for the operation of the driving assistance device 10 and data obtained by the operation of the driving assistance device 10.

[0029] The communication unit 13 includes at least one communication interface. The communication interface is, for example, a LAN interface. The communication unit 13 receives data used for the operation of the driving assistance device 10 and transmits data obtained by the operation of the driving assistance device 10.

[0030] The functions of the driving assistance device 10 are realized by executing the control program according to the present embodiment on a processor as the control unit 11. That is, the functions of the driving assistance device 10 are realized by software. The control program causes a computer to execute the operation of the driving assistance device 10, thereby causing the computer to function as the driving assistance device 10. That is, the computer functions as the driving assistance device 10 by executing the operation of the driving assistance device 10 according to the control program.

[0031] The program can be stored in a non-transitory computer-readable medium. The non-transitory computer-readable medium is, for example, a flash memory, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a ROM. The distribution of the program is performed, for example, by selling, transferring, or lending a portable medium such as an SD card, a DVD, or a CD-ROM storing the program. "SD" is an abbreviation of Secure Digital. "DVD" is an abbreviation of digital versatile disc. "CD-ROM" is an abbreviation of compact disc read only memory. The program may be distributed by storing the program in the server storage and transferring the program from the server to other computers. The program may be provided as a program product.

[0032] The computer temporarily stores, for example, a program stored in a portable medium or a program transferred from a server in the main memory device. Then, the computer reads the program stored in the main memory device by the processor and executes the processing according to the read program by the processor. The computer may directly read the program from the portable medium and execute the processing according to the program. The computer may sequentially execute the processing according to the received program each time the program is transferred from the server to the computer. The processing may be executed by a so-called ASP type service that realizes the function only by the execution instruction and the result acquisition without transferring the program from the server to the computer. "ASP" is an abbreviation of application service provider. The program is information for use in the processing by an electronic computer and includes those conforming to the program. For example, data that is not a direct instruction to the computer but has the property of defining the processing of the computer corresponds to "those conforming to the program".

[0033] Some or all of the functions of the driving support device 10 may be realized by a programmable circuit or a dedicated circuit as the control unit 11. That is, some or all of the functions of the driving support device 10 may be realized by hardware.

[0034] Referring to FIG. 4, the configuration of the terminal device 20 according to this embodiment will be described.

[0035] The terminal device 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, an output unit 25, and a positioning unit 26.

[0036] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. The programmable circuit is, for example, an FPGA. The dedicated circuit is, for example, an ASIC. The control unit 21 executes processes related to the operation of the terminal device 20 while controlling each part of the terminal device 20.

[0037] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 22 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. Data used for the operation of the terminal device 20 and data obtained by the operation of the terminal device 20 are stored in the storage unit 22.

[0038] The communication unit 23 includes at least one communication interface. The communication interface is, for example, an interface compatible with a mobile communication standard such as LTE, 4G standard, or 5G standard, or a LAN interface. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. The communication unit 23 receives data used for the operation of the terminal device 20 and transmits data obtained by the operation of the terminal device 20.

[0039] The input unit 24 includes at least one input interface. The input interface is, for example, a physical key, a capacitive key, a pointing device, a touch screen provided integrally with the display, a camera, or a microphone. The input unit 24 accepts an operation of inputting data used for the operation of the terminal device 20. Instead of being provided in the terminal device 20, the input unit 24 may be connected to the terminal device 20 as an external input device. As the connection interface, for example, an interface compatible with a standard such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0040] The output unit 25 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. The output unit 25 outputs data obtained by the operation of the terminal device 20. Instead of being provided in the terminal device 20, the output unit 25 may be connected to the terminal device 20 as an external output device. As the connection interface, for example, an interface compliant with a standard such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0041] The positioning unit 26 includes at least one GNSS receiver. "GNSS" is an abbreviation for global navigation satellite system. GNSS is, for example, GPS, QZSS, BDS, GLONASS, or Galileo. "GPS" is an abbreviation for Global Positioning System. "QZSS" is an abbreviation for Quasi-Zenith Satellite System. The satellites of QZSS are called quasi-zenith satellites. "BDS" is an abbreviation for BeiDou Navigation Satellite System. "GLONASS" is an abbreviation for Global Navigation Satellite System. The positioning unit 26 measures the position of the terminal device 20.

[0042] The functions of the terminal device 20 are realized by executing the terminal program according to the present embodiment with a processor as the control unit 21. That is, the functions of the terminal device 20 are realized by software. The terminal program causes a computer to execute the operation of the terminal device 20, thereby making the computer function as the terminal device 20. That is, the computer functions as the terminal device 20 by executing the operation of the terminal device 20 according to the terminal program.

[0043] Some or all of the functions of the terminal device 20 may be realized by a programmable circuit or a dedicated circuit as the control unit 21. That is, some or all of the functions of the terminal device 20 may be realized by hardware.

[0044] With reference to FIGS. 5 and 6, the operation of the system 1 according to the present embodiment will be described. This operation corresponds to the driving support method according to the present embodiment. FIG. 5 shows the operation of the driving support device 10. FIG. 6 shows the operation of the terminal device 20.

[0045] In S101 of FIG. 5, the control unit 11 of the driving support device 10 acquires the video VD. Specifically, the control unit 11 receives the video VD from the camera 30 via the communication unit 13. The control unit 11 continues to receive the video VD until at least the autonomous vehicle 51 passes the position where the camera 30 is installed while the autonomous vehicle 51 is traveling on the road 54.

[0046] In S102 of FIG. 5, the control unit 11 of the driving assistance device 10 detects the pedestrian 53 in the video VD acquired in S101. Specifically, the control unit 11 recognizes an object whose position changes between the frames of the video VD acquired in S101 as a moving object. As a method for recognizing an object in a frame, any method can be adopted. The control unit 11 determines that the larger the rectangular area when the recognized moving object is surrounded by a rectangle, the closer the moving object is to the camera 30, and the smaller the rectangular area, the farther the moving object is from the camera 30. The control unit 11 determines that when the rectangular area increases with the passage of time, the moving object is approaching the camera 30, and when it decreases, the moving object is moving away from the camera 30. The control unit 11 detects the moving object as the pedestrian 53 when the minor axis area is equal to or larger than a predetermined size and increases with the passage of time. The predetermined size is, for example, 10% of the size of the frame. The control unit 11 does not have to detect the moving object as the pedestrian 53 when the rectangular area is less than the predetermined size or decreases with the passage of time. When the pedestrian 53 is detected, the control unit 11 proceeds to the process of S103. The control unit 11 repeats the process of recognizing the moving object in the frame of the video VD sequentially acquired until the pedestrian 53 is detected.

[0047] In S103 of FIG. 5, the control unit 11 of the driving assistance device 10 transmits a notification NS indicating that the video VD can be transmitted to the terminal device 20 via the communication unit 13. At this time, the control unit 11 specifies the terminal device 20 as the terminal device that transmits the notification NS. As a method for specifying the terminal device that transmits the notification NS, any method can be adopted. As an example, the control unit 11 receives position data indicating the position of the terminal device 20 via the communication unit 13. The control unit 11 calculates the distance between the position indicated by the received position data and the position where the camera 30 is installed. The control unit 11 specifies the terminal device 20 as the terminal device that transmits the notification NS when the calculated distance is within a predetermined distance range and the calculated distance is decreasing. The control unit 11 does not specify the terminal device 20 as the terminal device that transmits the notification NS when the calculated distance is larger than the predetermined distance range or the calculated distance is increasing. The predetermined distance is, for example, 100 meters.

[0048] In S111 of FIG. 6, the control unit 21 of the terminal device 20 receives the notification NS from the driving support device 10. Specifically, the control unit 21 receives the notification NS transmitted from the driving support device 10 in S103 via the communication unit 23.

[0049] In S112 of FIG. 6, when the control unit 21 of the terminal device 20 receives the notification NS from the driving support device 10 in S111, it outputs a notification ND indicating that the video VD can be displayed. Specifically, the control unit 11 displays the notification ND on the screen of the display as the output unit 25. The notification ND includes graphic elements for receiving an operation for instructing the display of the video VD by the operator 52. The graphic elements are, for example, buttons such as a "video" button or an icon representing a pedestrian. The control unit 11 may output the notification ND as audio from the speaker as the output unit 25.

[0050] In S113 of FIG. 6, the control unit 21 of the terminal device 20 receives an operation for instructing the display of the video VD by the operator 52 via the input unit 24. The operation is, for example, a tap on a graphic element. When the operator 52 taps the graphic element, that is, when the control unit 21 receives the operation, the control unit 11 proceeds to the process of S114. If the operator 52 does not tap the graphic element, that is, if the control unit 21 does not receive the operation, the operation of the terminal device 20 ends.

[0051] In S114 of FIG. 6, the control unit 21 of the terminal device 20 transmits a request to the driving support device 10. Specifically, the control unit 21 transmits a notification indicating that the display of the video VD is instructed as a request to the driving support device 10 via the communication unit 23.

[0052] In S104 of FIG. 5, the control unit 11 of the driving support device 10 receives a request from the terminal device 20. Specifically, the control unit 11 receives the request transmitted from the terminal device 20 in S114 via the communication unit 13. When the request is received, the control unit 11 proceeds to the process of S105. If the control unit 11 does not receive the request, the operation of the driving support device 10 ends.

[0053] In S105 of FIG. 5, the control unit 11 of the driving support device 10 starts transmitting the video VD to the terminal device 20 via the communication unit 13 in response to the request received in S104. Specifically, the control unit 11 transmits the video VD in a streaming format. The control unit 11 continues to transmit the video VD to the terminal device 20 via the communication unit 13 until S107, which will be described later.

[0054] In S115 of FIG. 6, the control unit 21 of the terminal device 20 receives the video VD transmitted from the driving support device 10 in S105 via the communication unit 23.

[0055] In S116 of FIG. 6, the control unit 21 of the terminal device 20 displays the video VD received in S115. Specifically, the control unit 21 displays the video VD received in S115 on the screen using a display as the output unit 25. The operator 52 can confirm that the pedestrian 53 approaches the roadway 54 and the crosswalk 56 by viewing the video VD.

[0056] In S106 of FIG. 5, the control unit 11 of the driving support device 10 determines whether the end condition is satisfied. The end condition includes the condition that the autonomous vehicle 51 has passed the position where the camera 30 is installed. As a method for determining the end condition, any method can be adopted. As an example, the control unit 11 receives position data indicating the position of the terminal device 20 via the communication unit 13. The control unit 11 calculates the distance between the position indicated by the received position data and the position where the camera 30 is installed. As the calculated distance increases, the control unit 11 determines that the autonomous vehicle 51 has passed the position where the camera 30 is installed and the end condition is satisfied. As the calculated distance decreases, the control unit 11 determines that the autonomous vehicle 51 has not yet passed the position and the end condition is not satisfied. When it is determined that the end condition is satisfied, the control unit 11 proceeds to the process of S107. The control unit 11 repeats the process of S106 until the end condition is satisfied.

[0057] In S107 of FIG. 5, the control unit 11 of the driving support device 10 ends the transmission of the video VD. As a result, the display of the video VD ends on the terminal device 20.

[0058] After transmitting the notification NS in S103, the control unit 11 of the driving support device 10 may execute a process of determining whether the same end condition as in S106 is satisfied while waiting to receive a request from the terminal device 20 in S104. When the control unit 11 determines in S104 that the end condition is satisfied, it ends the waiting for receiving the request. Even if the control unit 11 receives a request after determining in S104 that the end condition is satisfied, it does not transmit the video VD. In the example of FIG. 2, when the control unit 11 determines that the autonomous vehicle 51 has passed the crosswalk 56 and the end condition is satisfied in S104, it ends the waiting for receiving the request. Even if the control unit 11 receives a request after determining that the autonomous vehicle 51 has passed the crosswalk 56 and the end condition is satisfied in S104, it does not transmit the video VD.

[0059] As described above, in the present embodiment, the control unit 11 of the driving support device 10 acquires the moving image VD obtained by photographing the sidewalk 55 with the camera 30. When the control unit 11 of the driving support device 10 detects a pedestrian 53 in the acquired moving image VD, it transmits a notification NS indicating that the moving image VD can be transmitted to the terminal device 20 via the communication unit 13, thereby causing the terminal device 20 to output a notification ND indicating that the moving image VD can be displayed. When the control unit 21 of the terminal device 20 receives the notification NS via the communication unit 23, it outputs the notification ND. When the control unit 21 of the terminal device 20 responds to an operation instructing the display of the moving image VD by the operator 52 of the autonomous vehicle 51 and requests the driving support device 10 to transmit the moving image VD, and receives the moving image VD via the communication unit 23, it displays the moving image VD.

[0060] In a tense scene where the autonomous vehicle 51 is driving in a blind spot, there may be no room for the operator 52 to look at the screen of the terminal device 20. According to the present embodiment, instead of the moving image VD being presented by default before passing through such a location, only the notification ND is output, and the operator 52 can view the moving image VD at his or her own discretion. Therefore, the safety of the operation of the autonomous vehicle 51 can be improved. However, when the operator 52 is at an external facility, there is a high possibility that there is more room for the operator 52 to look at the screen of the terminal device 20 compared to when the operator 52 is inside the vehicle. Therefore, when the operator 52 is at an external facility, the moving image VD may be presented by default.

[0061] As a modification of the present embodiment, the control unit 11 of the driving support device 10 may transmit the moving image VD regardless of the presence or absence of a request from the terminal device 20. A second embodiment, which is such a modification, will be described.

[0062] With reference to FIGS. 7 and 8, the operation of the system 1 according to the present embodiment will be described. This operation corresponds to the driving support method according to the present embodiment. FIG. 7 shows the operation of the driving support device 10. FIG. 8 shows the operation of the terminal device 20.

[0063] Since the processes of S201 and S202 in FIG. 7 are the same as the processes of S101 and S102 in FIG. 5, the description thereof is omitted.

[0064] In S203 of FIG. 7, when the control unit 11 of the driving support device 10 detects the pedestrian 53 in the video VD in S202, the control unit 11 starts transmitting the video VD to the terminal device 20 via the communication unit 13. At this time, the control unit 11 identifies the terminal device 20 as the terminal device that transmits the video VD. As a method for identifying the terminal device that transmits the video VD, any method can be adopted. As an example, the control unit 11 receives position data indicating the position of the terminal device 20 via the communication unit 13. The control unit 11 calculates the distance between the position indicated by the received position data and the position where the camera 30 is installed. When the calculated distance is within a predetermined distance range and the calculated distance is decreasing, the control unit 11 identifies the terminal device 20 as the terminal device that transmits the video VD. When the calculated distance is greater than the predetermined distance range or the calculated distance is increasing, the control unit 11 does not identify the terminal device 20 as the terminal device that transmits the video VD. The predetermined distance is, for example, 100 meters. The control unit 11 continues to transmit the video VD to the terminal device 20 via the communication unit 13 until S205, which will be described later, regardless of whether there is a request from the terminal device 20.

[0065] Since the process of S211 in FIG. 8 is the same as the process of S115 in FIG. 6, the description thereof is omitted.

[0066] In S212 of FIG. 8, when the control unit 21 of the terminal device 20 starts receiving the video VD from the driving support device 10 in S211, the control unit 21 outputs a notification ND indicating that the video VD can be displayed. Since the output and display example of the notification ND are the same as those in S112 of the first embodiment, the description thereof is omitted.

[0067] Since the process of S213 in FIG. 8 is the same as the process of S113 in FIG. 6, the description thereof is omitted.

[0068] In S214 of FIG. 8, when the control unit 21 of the terminal device 20 receives the operation by the operator 52 in S213, it starts displaying the moving image VD. Since the display example of the moving image VD is the same as that in S116 of the first embodiment, the description thereof is omitted.

[0069] Since the processes of S204 and S205 in FIG. 7 are the same as the processes of S106 and S107 in FIG. 5, the description thereof is omitted.

[0070] As described above, in the present embodiment, the control unit 11 of the driving support device 10 acquires the moving image VD obtained by photographing the sidewalk 55 with the camera 30. When the control unit 11 of the driving support device 10 detects the pedestrian 53 in the acquired moving image VD, it transmits the moving image VD to the terminal device 20 via the communication unit 13, thereby outputting a notification ND indicating that the moving image VD can be displayed. When the control unit 21 of the terminal device 20 starts receiving the moving image VD via the communication unit 23, it outputs the notification ND. The control unit 21 of the terminal device 20 starts displaying the moving image VD in response to an operation by the operator 52 of the autonomous vehicle 51 instructing the display of the moving image VD.

[0071] According to the present embodiment, as in the first embodiment, instead of the moving image VD being presented by default before passing through a blind spot, only the notification ND is output, and the operator 52 can view the moving image VD at his or her own discretion. Therefore, the safety of the operation of the autonomous vehicle 51 can be improved. Moreover, in the present embodiment, since the terminal device 20 continues to receive the moving image VD even while not displaying the moving image VD, it can instantaneously respond to the instruction when the operator 52 instructs the display of the moving image VD.

[0072] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks described in the block diagram may be integrated, or one block may be divided. Instead of executing two or more steps described in the flowchart in time series according to the description, each step may be executed in parallel or in a different order according to the processing ability of the device that executes the steps, or as necessary. In addition, changes can be made without departing from the spirit of the present disclosure.

[0073] Some embodiments of the present disclosure will be exemplified below. However, note that the embodiments of the present disclosure are not limited thereto. [Appendix 1] A driving support device includes a control unit that acquires a video obtained by photographing a sidewalk on which a pedestrian passes in contact with a lane on which an autonomous vehicle travels with a camera, and when a pedestrian is detected in the acquired video, outputs a notification to the effect that the video can be displayed to a terminal device of an operator of the autonomous vehicle. [Appendix 2] The driving support device further includes a communication unit that communicates with the terminal device, when the terminal device receives an operation for instructing display of the video by the operator, the terminal device transmits a request to the driving support device, The control unit transmits the video to the terminal device via the communication unit in response to the request. The driving support device according to Appendix 1. [Appendix 3] when the terminal device receives a notification to the effect that the video can be transmitted from the driving support device, the terminal device outputs a notification to the effect that the video can be displayed, when the control unit detects the pedestrian in the video, the control unit transmits a notification to the effect that the video can be transmitted to the terminal device via the communication unit. The driving support device according to Appendix 2. [Appendix 4] The driving support device further includes a communication unit that communicates with the terminal device, when the terminal device receives an operation for instructing display of the video by the operator, the terminal device starts displaying the video, The control unit continuously transmits the video to the terminal device via the communication unit regardless of whether there is a request from the terminal device. The driving support device according to Appendix 1. [Appendix 5] when the terminal device starts receiving the video from the driving support device, the terminal device outputs a notification to the effect that the video can be displayed, when the control unit detects the pedestrian in the video, the control unit starts transmitting the video to the terminal device via the communication unit. The driving support device according to Appendix 4. [Appendix 6] It further includes a communication unit for communicating with the terminal device, The control unit transmits the video to the terminal device via the communication unit in response to a request from the terminal device or regardless of the presence or absence of a request from the terminal device, and ends the transmission of the video when an end condition is satisfied. The driving assistance device according to any one of Supplementary Notes 1 to 5. [Supplementary Note 7] The driving assistance device according to Supplementary Note 6, wherein the end condition includes the condition that the autonomous vehicle has passed the position where the camera is installed. [Supplementary Note 8] The driving assistance device according to Supplementary Note 7, wherein the position is a blind spot from the lane or near the blind spot. [Supplementary Note 9] The driving assistance device according to Supplementary Note 7, wherein the position is a crosswalk on the lane or near the crosswalk. [Supplementary Note 10] A driving assistance device according to any one of Supplementary Notes 1 to 9, and The terminal device A system comprising. [Supplementary Note 11] Outputting a notification indicating that a video obtained by photographing a sidewalk on which pedestrians pass in contact with the lane on which the autonomous vehicle travels can be displayed, Displaying the video in response to an operation for instructing the display of the video by the operator of the autonomous vehicle A program for causing a computer to execute operations including. [Supplementary Note 12] The displaying When receiving the operation, transmitting a request to the driving assistance device, Receiving the video from the driving assistance device, Displaying the received video The program according to Supplementary Note 11 including. [Supplementary Note 13] The operations Further include receiving the video from the driving assistance device, The program described in Supplementary Note 11, wherein the displaying includes starting to display the received video upon receiving the operation. [Supplementary Note 14] The program according to any one of Supplementary Notes 11 to 13, wherein the notification includes a graphic element for receiving the operation. [Supplementary Note 15] The program according to Supplementary Note 14, wherein the operation is a tap on the graphic element. [Supplementary Note 16] The program according to Supplementary Note 14 or 15, wherein the graphic element is a button. [Supplementary Note 17] The program according to Supplementary Note 14 or 15, wherein the graphic element is an icon representing a pedestrian. [Supplementary Note 18] A driving support method including: an operation support device acquiring a video obtained by photographing, with a camera, a sidewalk on which pedestrians pass, which is adjacent to a lane on which an automated vehicle travels; and when the operation support device detects a pedestrian in the acquired video, causing a terminal device of an operator of the automated vehicle to output a notification indicating that the video can be displayed. [Supplementary Note 19] When the terminal device receives an operation for instructing the operator to display the video, the terminal device transmits a request to the operation support device, The driving support method further includes, according to Supplementary Note 18, the operation support device transmitting the video to the terminal device in response to the request. [Supplementary Note 20] When the terminal device receives an operation for instructing the operator to display the video, the terminal device starts displaying the video, The driving support method further includes, according to Supplementary Note 18, the operation support device continuously transmitting the video to the terminal device regardless of the presence or absence of a request from the terminal device.

Explanation of Signs

[0074] 1 System 10 Driving Support Device 11 Control Unit 12 Memory Unit 13 Communication Unit 20 Terminal Device 21 Control Unit 22 Memory Unit 23 Communication Unit 24 Input Unit 25 Output Unit 26 Positioning Unit 30 Camera 40 Network 51 Autonomous Vehicle 52 Operator 53 Pedestrian 54 Road Lane 55 Sidewalk 56 Crosswalk 57 Building

Claims

1. A driving support device comprising a control unit that acquires a video obtained by photographing a sidewalk on which pedestrians pass in contact with a lane on which an automated vehicle travels, and when a pedestrian is detected in the acquired video, outputs a notification to a terminal device of an operator of the automated vehicle indicating that the video can be displayed.

2. Further comprising a communication unit that communicates with the terminal device, wherein when the terminal device receives an operation instruction from the operator to display the video, the terminal device transmits a request to the driving support device, and the control unit transmits the video to the terminal device via the communication unit in response to the request. The driving support device according to claim 1.

3. when the terminal device receives a notification indicating that the video can be transmitted from the driving support device, the terminal device outputs a notification indicating that the video can be displayed, and the control unit transmits a notification indicating that the video can be transmitted to the terminal device via the communication unit when the pedestrian is detected in the video. The driving support device according to claim 2.

4. Further comprising a communication unit that communicates with the terminal device, wherein when the terminal device receives an operation instruction from the operator to display the video, the terminal device starts displaying the video, and the control unit continuously transmits the video to the terminal device via the communication unit regardless of whether a request is received from the terminal device. The driving support device according to claim 1.

5. when the terminal device starts receiving the video from the driving support device, the terminal device outputs a notification indicating that the video can be displayed, and the control unit starts transmitting the video to the terminal device via the communication unit when the pedestrian is detected in the video. The driving support device according to claim 4.

6. Further comprising a communication unit that communicates with the terminal device, wherein the control unit transmits the video to the terminal device via the communication unit in response to a request from the terminal device or regardless of whether a request is received from the terminal device, and when an end condition is satisfied, ends the transmission of the video. The driving support device according to claim 1.

7. The end condition includes the condition that the automated vehicle has passed the position where the camera is installed. The driving support device according to claim 6.

8. The position is a blind spot from the lane or near the blind spot. The driving support device according to claim 7.

9. The position is a crosswalk on the lane or near the crosswalk. The driving support device according to claim 7.

10. The driving assistance device according to any one of claims 1 to 9, and the terminal device comprising a system.

11. Outputting a notification indicating that a video obtained by photographing a sidewalk on which pedestrians pass in contact with the lane on which the autonomous vehicle travels can be displayed; displaying the video in response to an operation by the operator of the autonomous vehicle instructing the display of the video A program that causes a computer to execute operations including.

12. The displaying includes: when receiving the operation, sending a request to the driving assistance device; receiving the video from the driving assistance device; displaying the received video The program according to claim 11, including.

13. The operation further includes: receiving the video from the driving assistance device, The displaying includes starting to display the received video when receiving the operation. The program according to claim 11.

14. The notification includes a graphic element for receiving the operation. The program according to claim 11.

15. The operation is a tap on the graphic element. The program according to claim 14.

16. The graphic element is a button. The program according to claim 14.

17. The graphic element is an icon representing a pedestrian. The program according to claim 14.

18. The driving assistance device obtains a video obtained by photographing a sidewalk on which pedestrians pass in contact with the lane on which the autonomous vehicle travels by a camera; when the driving assistance device detects a pedestrian in the acquired video, causing the terminal device of the operator of the autonomous vehicle to output a notification indicating that the video can be displayed A driving assistance method including.

19. When the terminal device receives an operation by the operator instructing the display of the video, it sends a request to the driving assistance device. The driving assistance method includes: The driving assistance device further includes sending the video to the terminal device in response to the request. The driving assistance method according to claim 18.

20. When the terminal device receives an operation by the operator instructing the display of the video, it starts displaying the video. The driving assistance method includes: The driving support method according to claim 18, further comprising continuously transmitting the video to the terminal device regardless of the presence or absence of a request from the terminal device.

Images