Driving assistance device

Abstract

To reduce the annoyance of an operator in notifying the operator of a pedestrian.SOLUTION: A driving assistance device 10 includes a control unit that acquires a video captured with a camera 30 of a sidewalk which is adjacent to a roadway on which an automatic driving vehicle 51 is traveling and on which pedestrians walk, and when a pedestrian is detected in the acquired video, changes whether or not to notify whether the detected pedestrian is moving or stationary depending on whether or not an area in the video that includes the detected pedestrian satisfies notification conditions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a driving assistance device. [Background technology]

[0002] Patent Document 1 discloses an alert system that recognizes the behavior of autonomous vehicles that are scheduled to pass through a specified traffic area, and notifies vehicles other than autonomous vehicles or pedestrians whether they are allowed to pass through and issues warnings. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-050629 Summary of the Invention [Problem to be solved by the invention]

[0004] It is conceivable that an operator in the vehicle or at a remote location will assist the operation of an autonomous vehicle. For smooth assistance, it is desirable to notify the operator of pedestrians. However, if pedestrian notification is performed uniformly regardless of the situation, it may become a nuisance for the operator.

[0005] An object of the present disclosure is to reduce the annoyance felt by an operator when notifying the operator of a pedestrian. [Means for solving the problem]

[0006] The driving assistance device according to the present disclosure comprises: The system acquires video footage taken by a camera of a sidewalk adjacent to a roadway on which an autonomous vehicle is traveling and where pedestrians walk, and when a pedestrian is detected in the acquired video, the system is equipped with a control unit that changes whether or not to issue a notification as to whether the detected pedestrian is moving or stationary, depending on whether or not the area in the video that includes the detected pedestrian satisfies a notification condition. [Effects of the Invention]

[0007] According to the present disclosure, the annoyance felt by the operator when notifying the operator of pedestrians is reduced. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 illustrates an example of how a system according to an embodiment of the present disclosure is used. [Figure 3] 1 is a block diagram showing a configuration of a driving assistance device according to an embodiment of the present disclosure. [Figure 4] 4 is a flowchart illustrating an operation of the driving assistance device according to the embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.

[0010] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0011] The configuration of a system 1 according to this embodiment will be described with reference to FIG.

[0012] The system 1 according to this embodiment includes a driving assistance device 10, a terminal device 20, and a camera 30. The driving assistance device 10 is capable of communicating with the terminal device 20 via a network 40. The driving assistance device 10 is capable of communicating with the camera 30 via the network 40.

[0013] The driving assistance device 10 is installed in a facility such as a data center, etc. The driving assistance device 10 is a computer such as a server that belongs to a cloud computing system or other computing system.

[0014] The terminal device 20 is used by an operator 52 who assists in the operation of the autonomous vehicle 51. The operator 52 may be in the vehicle or in an external facility such as an operation control room. When the operator 52 is in the vehicle, the terminal device 20 is held by the operator 52. When the operator 52 is in an external facility, the terminal device 20 is installed in the external facility. The terminal device 20 is, for example, a mobile device such as a mobile phone, a smartphone, or a tablet, or a PC. "PC" is an abbreviation for personal computer.

[0015] As shown in FIG. 2, the camera 30 is installed in a position where it can capture an image of a sidewalk 55 adjacent to a roadway 54 on which an autonomous vehicle 51 travels and where pedestrians 53 walk. In the example shown in FIG. 2, the camera 30 is installed near a blind spot from the roadway 54, but the camera 30 may be installed directly in the blind spot. In the example shown in FIG. 2, the camera 30 is installed near a crosswalk 56 on the roadway 54, but the camera 30 may be installed directly on the crosswalk 56. In the example shown in FIG. 2, when a vehicle is traveling from left to right on the roadway 54, the portion of the sidewalk 55 that is blocked by a building 57 is in a blind spot from the roadway 54 until the vehicle passes the crosswalk 56. The camera 30 is supported by a long support such as a cylinder so as to be positioned at a desired height. In addition to the camera 30, the support may be equipped with a light such as an LED, a speaker, or a display to notify pedestrians 53 of the approach of the autonomous vehicle 51. "LED" is an abbreviation for light-emitting diode.

[0016] Network 40 may include the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. Network 40 may include at least one wireless network, at least one optical network, or any combination thereof. A wireless network may be, for example, an ad-hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0017] The autonomous vehicle 51 is any type of automobile, such as a gasoline vehicle, a diesel vehicle, a hydrogen vehicle, an HEV, a PHEV, a BEV, or an FCEV. "HEV" is an abbreviation for hybrid electric vehicle. "PHEV" is an abbreviation for plug-in hybrid electric vehicle. "BEV" is an abbreviation for battery electric vehicle. "FCEV" is an abbreviation for fuel cell electric vehicle. In this embodiment, the autonomous vehicle 51 is an AV, and driving is automated at a high level. "AV" is an abbreviation for autonomous vehicle. The level of automation is, for example, either level 3 or level 4 in the SAE level classification. "SAE" is an abbreviation for Society of Automotive Engineers.

[0018] The outline of this embodiment will be described with reference to FIGS.

[0019] In this embodiment, the driving assistance device 10 acquires a video VD obtained by capturing an image of the sidewalk 55 with the camera 30. When the driving assistance device 10 detects a pedestrian 53 in the acquired video VD, the driving assistance device 10 changes whether or not to notify whether the detected pedestrian 53 is moving or stationary, depending on whether or not an area AR including the detected pedestrian 53 in the video VD satisfies a notification condition. The pedestrian 53 is not limited to a person, but also includes a moving object that can travel on the sidewalk 55, such as a wheelchair.

[0020] According to this embodiment, the operator 52 of the automatically driven vehicle 51 can be notified of the pedestrian 53 depending on the situation. Therefore, the annoyance of the operator 52 in notifying the operator 52 of the pedestrian 53 can be reduced.

[0021] In the example shown in FIG. 2 , an operator 52 is riding in an autonomous vehicle 51. The operator 52 operates a terminal device 20 and checks a notification from the terminal device 20. A pedestrian 53 is walking on a sidewalk 55 and is about to cross a crosswalk 56 on a roadway 54. The pedestrian 53 is located behind a building 57 that blocks the sidewalk 55 in the direction of travel of the autonomous vehicle 51, making it difficult for the operator 52 to see the pedestrian 53. A camera 30 is installed near a blind spot from the roadway 54, where the operator 52 cannot see the pedestrian 53. The camera 30 is facing away from the roadway 54 and photographs the sidewalk 55. The camera 30 can transmit a video VD obtained by photographing the sidewalk 55 to the driving assistance device 10 via the network 40. As another example, the camera 30 may be installed facing the entrance of the building 57, or may be installed on a path of pedestrians exiting the building 57 and heading toward the roadway 54.

[0022] The configuration of the driving support device 10 according to this embodiment will be described with reference to FIG.

[0023] The driving assistance device 10 includes a control unit 11, a storage unit 12, and a communication unit 13.

[0024] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. An example of the programmable circuit is an FPGA. "FPGA" is an abbreviation for field-programmable gate array. An example of the dedicated circuit is an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 11 controls each part of the driving assistance device 10 and executes processing related to the operation of the driving assistance device 10.

[0025] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM or a ROM. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used in the operation of the driving assistance device 10 and data obtained by the operation of the driving assistance device 10.

[0026] The communication unit 13 includes at least one communication interface. The communication interface is, for example, a LAN interface. The communication unit 13 receives data used in the operation of the driving assistance device 10 and transmits data obtained by the operation of the driving assistance device 10.

[0027] The functions of the driving assistance device 10 are realized by executing a control program according to this embodiment on a processor serving as the control unit 11. That is, the functions of the driving assistance device 10 are realized by software. The control program causes a computer to execute the operations of the driving assistance device 10, thereby causing the computer to function as the driving assistance device 10. That is, the computer functions as the driving assistance device 10 by executing the operations of the driving assistance device 10 in accordance with the control program.

[0028] The program can be stored on a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical disks, magneto-optical recording media, and ROMs. The program can be distributed by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs that store the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the storage of a server and transferring it from the server to another computer. The program can also be provided as a program product.

[0029] A computer temporarily stores a program stored on a portable medium or transferred from a server in its main storage device. The computer then reads the program stored in the main storage device using a processor and executes processing in accordance with the read program. The computer may also read the program directly from a portable medium and execute processing in accordance with the program. The computer may also execute processing in accordance with the received program each time a program is transferred from a server to the computer. Processing may also be executed through a so-called ASP-type service that achieves its functions by issuing execution instructions and obtaining results without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. A program is information used for processing by a computer and includes something equivalent to a program. For example, data that is not a direct instruction to a computer but has properties that specify computer processing falls under the category of "something equivalent to a program."

[0030] Some or all of the functions of the driving assistance device 10 may be realized by a programmable circuit or a dedicated circuit as the control unit 11. In other words, some or all of the functions of the driving assistance device 10 may be realized by hardware.

[0031] The operation of the system 1 according to this embodiment will be described with reference to Fig. 4. This operation corresponds to the driving assistance method according to this embodiment.

[0032] In S1, the control unit 11 of the driving assistance device 10 acquires the video VD. Specifically, the control unit 11 receives the video VD from the camera 30 via the communication unit 13. The control unit 11 continues to receive the video VD while the autonomously driven vehicle 51 is traveling on the roadway 54, at least until the autonomously driven vehicle 51 passes the position where the camera 30 is installed.

[0033] In S2, the control unit 11 of the driving assistance device 10 detects a pedestrian 53 in the video VD acquired in S1. Specifically, the control unit 11 recognizes an object whose position changes between frames of the video VD acquired in S1 as a moving object. Any method can be adopted as a method for recognizing an object in a frame. The control unit 11 determines that the larger the rectangular area surrounding the recognized moving object, the closer the moving object is to the camera 30, and the smaller the rectangular area, the farther the moving object is from the camera 30. The control unit 11 determines that the moving object is approaching the camera 30 when the rectangular area increases over time, and that the moving object is moving away from the camera 30 when the rectangular area decreases over time. The control unit 11 detects the moving object as a pedestrian 53 if the rectangular area is equal to or larger than a predetermined size and is increasing over time. The predetermined size is, for example, 10% of the size of the frame. The control unit 11 may not detect the moving object as a pedestrian 53 if the rectangular area is smaller than the predetermined size or is decreasing over time. If a pedestrian 53 is detected, the control unit 11 proceeds to the process of S3. The control unit 11 repeats the process of recognizing a moving object in the frames of the moving image VD that are sequentially acquired until a pedestrian 53 is detected.

[0034] In S3, the control unit 11 of the driving assistance device 10 determines whether the area AR including the pedestrian 53 detected in S2 in the video VD acquired in S1 satisfies the notification condition. If it is determined that the notification condition is satisfied, the control unit 11 proceeds to the process of S4. If it is determined that the notification condition is not satisfied, the control unit 11 proceeds to the process of S7. The notification condition may be, for example, the following conditions:

[0035] In the first example, the notification condition includes a condition that the size of the area AR is equal to or smaller than a threshold value. The threshold value is, for example, 50% of the frame size. For example, the control unit 11 determines that the notification condition is met when the ratio of the area AR to the frame size is equal to or smaller than 50%. At this time, the control unit 11 determines that the pedestrian 53 is some distance away from the camera 30. At this time, the control unit 11 determines that the notification condition is not met when the ratio of the area AR is greater than 50%. At this time, the control unit 11 determines that the pedestrian 53 is very close to the camera 30.

[0036] In the second example, the notification condition includes a first condition that the head of the pedestrian 53 is included in the area AR. For example, the control unit 11 determines that the notification condition is satisfied when the head of the pedestrian 53 is included in the area AR. At this time, the control unit 11 determines that the pedestrian 53 is at a certain distance from the camera 30. At this time, the control unit 11 determines that the notification condition is not satisfied when the head of the pedestrian 53 is not included in the area AR, that is, when the head of the pedestrian 53 is not visible. At this time, the control unit 11 determines that the pedestrian 53 is very close to the camera 30.

[0037] In the third example, the notification condition includes a second condition that the feet of the pedestrian 53 are included in the area AR. For example, when the feet of the pedestrian 53 are included in the area AR, the control unit 11 determines that the notification condition is satisfied. At this time, the control unit 11 determines that the pedestrian 53 is at a certain distance from the camera 30. When the feet of the pedestrian 53 are not included in the area AR, that is, when the feet of the pedestrian 53 are not visible, the control unit 11 determines that the notification condition is not satisfied. At this time, the control unit 11 determines that the pedestrian 53 is very close to the camera 30.

[0038] In the fourth example, the notification condition includes both the first condition and the second condition. For example, the control unit 11 determines that the notification condition is satisfied when both the head and feet of the pedestrian 53 are included in the area AR. At this time, the control unit 11 determines that the pedestrian 53 is at a certain distance from the camera 30. The control unit 11 determines that the notification condition is not satisfied when the area AR includes the head of the pedestrian 53 but not the feet of the pedestrian 53. The control unit 11 determines that the notification condition is not satisfied when the area AR includes the feet of the pedestrian 53 but not the head of the pedestrian 53. The control unit 11 determines that the notification condition is not satisfied when the area AR includes neither the head nor the feet of the pedestrian 53. In these cases, the control unit 11 determines that the pedestrian 53 is in the immediate vicinity of the camera 30.

[0039] In S4, the control unit 11 of the driving assistance device 10 determines whether the pedestrian 53 detected in S2 is moving or stationary. Any method can be used to determine whether the pedestrian 53 is moving or stationary. As an example, the control unit 11 measures the distance the area AR moves within the frame during a certain period of time, and determines that the pedestrian 53 is moving when the measured distance exceeds a predetermined length. The control unit 11 determines that the pedestrian 53 is stationary when the measured distance is equal to or shorter than the predetermined length. The predetermined length may vary depending on the size of the area AR. As another example, the control unit 11 determines that the pedestrian 53 is moving when the ratio of the area AR to the frame size increases and the change in the ratio exceeds a predetermined value. The control unit 11 determines that the pedestrian 53 is stationary when the change in the ratio is equal to or shorter than the predetermined value. If the control unit 11 determines that the pedestrian 53 is moving, the control unit 11 proceeds to processing in S5. If the control unit 11 determines that the pedestrian 53 is stationary, the control unit 11 proceeds to processing in S6.

[0040] In S5, the control unit 11 of the driving assistance device 10 performs a first process. The first process is a process of displaying a first icon on the terminal device 20. The first icon is, for example, an icon indicating that the pedestrian 53 is moving. Specifically, when the area AR satisfies the notification condition and the control unit 11 determines that the pedestrian 53 is moving, the control unit 11 transmits a first icon display command to the terminal device 20 via the communication unit 13 as the first process. The first icon display command may include the first icon itself, or may include only data specifying the first icon if the first icon is pre-stored in the terminal device 20. Upon receiving the first icon display command, the terminal device 20 displays the first icon on the display in accordance with the received first icon display command. By checking the first icon displayed on the terminal device 20, the operator 52 can know in advance that the pedestrian 53 is in a blind spot and that the pedestrian 53 is moving, and can prepare for the pedestrian 53 suddenly appearing. After S5, the control unit 11 performs a process of S8.

[0041] In S6, the control unit 11 of the driving assistance device 10 performs a second process. The second process is a process of displaying a second icon on the terminal device 20. The second icon is an icon different from the first icon. The second icon is, for example, an icon indicating that the pedestrian 53 is stopped. Specifically, when the area AR satisfies the notification condition and the control unit 11 determines that the pedestrian 53 is stopped, the control unit 11 transmits a second icon display command to the terminal device 20 via the communication unit 13 as the second process. The second icon display command may include the second icon itself, or may include only data specifying the second icon if the second icon is pre-stored in the terminal device 20. Upon receiving the second icon display command, the terminal device 20 displays the second icon on the display in accordance with the received second icon display command. By checking the second icon displayed on the terminal device 20, the operator 52 can know in advance that the pedestrian 53 is in a blind spot but is stopped. After S6, the control unit 11 performs the process of S8.

[0042] In S7, the control unit 11 of the driving assistance device 10 performs a third process. The third process is a process of displaying a third icon on the terminal device 20. The third icon is an icon different from both the first icon and the second icon. The third icon is, for example, an icon indicating that it has not been determined whether the pedestrian 53 is moving or stationary. Specifically, when the area AR does not satisfy the notification condition, the control unit 11 transmits a third icon display command to the terminal device 20 via the communication unit 13 as the third process. The third icon display command may include the third icon itself, or may include only data specifying the third icon if the third icon is pre-stored in the terminal device 20. Upon receiving the third icon display command, the terminal device 20 displays the third icon on the display in accordance with the received third icon display command. The terminal device 20 may further display a mark indicating that it has not been determined whether the pedestrian 53 is moving or stationary. For example, the terminal device 20 may flash an LED light as the mark. By checking the third icon displayed on the terminal device 20, the operator 52 can know in advance that the pedestrian 53 is in the blind spot, and can also know that the pedestrian 53 is close to the camera 30 and therefore it has not been determined whether the pedestrian 53 is moving or stationary. After S7, the control unit 11 performs the process of S8.

[0043] In S7, the third process may be a process of displaying a second icon on the terminal device 20. Specifically, when the area AR does not satisfy the notification condition, the control unit 11 may, as the third process, transmit a second icon display command to the terminal device 20 via the communication unit 13. Upon receiving the second icon display command, the terminal device 20 displays the second icon on the display in accordance with the received second icon display command.

[0044] In S7, the third process may be a process of not displaying the icon on the terminal device 20. Specifically, when the area AR does not satisfy the notification condition, the control unit 11 may, as the third process, transmit an icon hide command to the terminal device 20 via the communication unit 13. Upon receiving the icon hide command, the terminal device 20 does not display anything in the area on the display where the first icon and the second icon can be selectively displayed, in accordance with the received icon hide command.

[0045] In S8, the control unit 11 of the driving assistance device 10 determines whether the termination condition is satisfied. The termination condition includes a condition that the autonomously driven vehicle 51 has passed the location where the camera 30 is installed. Any method can be used to determine the termination condition. As an example, the control unit 11 receives position data indicating the location of the terminal device 20 via the communication unit 13. The control unit 11 calculates the distance between the location indicated by the received position data and the location where the camera 30 is installed. If the calculated distance increases, the control unit 11 determines that the autonomously driven vehicle 51 has passed the location where the camera 30 is installed and the termination condition is satisfied. If the calculated distance decreases, the control unit 11 determines that the autonomously driven vehicle 51 has not yet passed the location and the termination condition is not satisfied. When it is determined that the termination condition is satisfied, the control unit 11 may send an icon hide command to the terminal device 20 via the communication unit 13. Alternatively, the terminal device 20 may terminate the display of the first icon, the second icon, or the like by independently determining that the autonomously driven vehicle 51 has passed the location where the camera 30 is installed. The control unit 11 repeats the processes from S2 onwards until the termination condition is satisfied.

[0046] As described above, in this embodiment, the control unit 11 of the driving assistance device 10 acquires a video VD obtained by capturing an image of the sidewalk 55 with the camera 30. When the control unit 11 of the driving assistance device 10 detects a pedestrian 53 in the acquired video VD, the control unit 11 changes whether or not to notify whether the detected pedestrian 53 is moving or stationary, depending on whether or not an area AR including the detected pedestrian 53 in the video VD satisfies a notification condition.

[0047] According to this embodiment, the operator 52 of the automatically driven vehicle 51 can be notified of the pedestrian 53 depending on the situation. Therefore, the annoyance of the operator 52 in notifying the operator 52 of the pedestrian 53 can be reduced.

[0048] As a modification of this embodiment, the frame rate of the camera 30 may be adjusted according to the remaining battery charge of the camera 30. The frame rate is the number of images included in the video VD per second. In this modification, the camera 30 lowers the frame rate as the remaining battery charge decreases. For example, when the remaining battery charge falls below 30%, the camera 30 switches to a first low power consumption mode, setting the frame rate to 80% of the normal rate. When the remaining battery charge falls below 20%, the camera 30 switches to a second low power consumption mode, setting the frame rate to 50% of the normal rate. When the remaining battery charge falls below 10%, the camera 30 switches to a third low power consumption mode, setting the frame rate to 30% of the normal rate. When the frame rate is below a reference value, the control unit 11 may not notify the pedestrian 53 whether they are moving or stationary. The reference value may be, for example, 80%, 50%, or 30% of the normal frame rate. That is, when camera 30 is in a specific low power consumption mode, control unit 11 does not need to notify whether pedestrian 53 is moving or stationary. Specifically, in S3, when the frame rate of moving image DV acquired in S1 is equal to or lower than a reference value, control unit 11 may proceed to the process of S7 even if it determines that the notification condition is satisfied.

[0049] In another variation of this embodiment, the control unit 11 of the driving assistance device 10 may acquire video images captured by each of the multiple cameras as the video image VD, and if a certain period of time has passed since the start of notification for any of the multiple cameras, the control unit 11 may transition to a mode in which notification is unconditionally stopped for that camera. In this variation, for a camera for which it has been determined in S3 that the notification conditions are not satisfied, the control unit 11 stores data identifying the camera and the date and time in the storage unit 12. The control unit 11 refers to the stored data, and for a camera for which it has been determined in S3 that the notification conditions are not satisfied for a certain period of time or more, the control unit 11 unconditionally proceeds to the processing of S7 after S2. The certain period is, for example, 24 hours.

[0050] As yet another modification of the present embodiment, the control unit 11 of the driving assistance device 10 may cause the terminal device 20 to display the video VD. In this modification, when transmitting a first icon display command to the terminal device 20 as a first process in S5, the control unit 11 also transmits the video VD acquired in S1 to the terminal device 20. Upon receiving the first icon display command and the video VD, the terminal device 20 displays the first icon and the received video VD on the display in accordance with the received first icon display command. When transmitting a second icon display command to the terminal device 20 as a second process in S6, the control unit 11 may also transmit the video VD acquired in S1 to the terminal device 20. Upon receiving the second icon display command and the video VD, the terminal device 20 displays the second icon and the received video VD on the display in accordance with the received second icon display command. When transmitting a third icon display command to the terminal device 20 as a third process in S7, the control unit 11 may also transmit the video VD acquired in S1 to the terminal device 20. Upon receiving the third icon display command and the video VD, the terminal device 20 displays the third icon and the received video VD on the display in accordance with the received third icon display command. The operator 52 can check the video of the pedestrian 53 hiding in the blind spot by looking at the displayed video VD.

[0051] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagrams may be integrated, or one block may be divided. Two or more steps shown in the flowcharts may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure. [Explanation of symbols]

[0052] 1 System 10 Driving assistance devices 11 Control section 12 Storage section 13 Communications Department 20 Terminal equipment 30 Camera 40 Network 51 Autonomous Vehicles 52 Operator 53 Pedestrians 54 Roadway 55 Sidewalk 56 Crosswalk 57 Building

Claims

1. A driving assistance device that acquires video footage taken by a camera of a sidewalk adjacent to a roadway on which an autonomous vehicle is traveling and where pedestrians walk, and that is equipped with a control unit that, when a pedestrian is detected in the acquired video, changes whether or not to notify the user that the detected pedestrian is moving or stationary, depending on whether or not an area in the video that includes the detected pedestrian satisfies a notification condition.

2. The driving assistance device according to claim 1 , wherein the notification condition includes a condition that the size of the area is equal to or smaller than a threshold value.

3. 2. The driving assistance device according to claim 1, wherein the notification condition includes a first condition that the area includes the pedestrian's head, a second condition that the area includes the pedestrian's feet, or both the first condition and the second condition.

4. Further, a communication unit that communicates with a terminal device of an operator of the autonomous driving vehicle, 2. The driving assistance device according to claim 1, wherein when the area satisfies the notification condition, if the control unit determines that the pedestrian is moving, the control unit causes the terminal device to display a first icon via the communication unit, and if the control unit determines that the pedestrian is stationary, the control unit causes the terminal device to display a second icon different from the first icon via the communication unit.

5. The driving assistance device according to claim 4, wherein the control unit, when the area does not satisfy the notification condition, causes the terminal device to display a third icon different from both the first icon and the second icon via the communication unit.

Images