Notification control device, notification control system, and notification control method

Abstract

To provide a notification control device capable of notifying a periphery of a vehicle of presence of a person not recognized by the vehicle.SOLUTION: A control unit 10 (a notification control device) includes: a roadside detection result acquisition unit 11 that acquires a roadside detection result that is a result obtained by detecting a person within a predetermined range in a roadside device; a vehicle detection result acquisition unit 12 that acquires a vehicle detection result that is a result obtained by detecting a person within the predetermined range in a vehicle; a determination unit 13 for determining whether or not the roadside detection result and the vehicle detection result coincide with each other; and a person notification control unit 14 for notifying a person in the predetermined range assuming that there is a person not recognized by the vehicle when it is determined that the roadside detection result and the vehicle detection result do not coincide with each other.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a notification control device, a notification control system, and a notification control method. [Background technology]

[0002] There is known a technology that uses roadside devices installed on the roadside to support safe vehicle travel at intersections, etc. For example, in recent years, development of smart poles equipped with cameras and displays has been progressing.

[0003] As a related technique, for example, Patent Document 1 discloses a vehicle-pedestrian wireless communication system in which direct wireless communication is performed between an in-vehicle wireless communication device mounted on a vehicle and a pedestrian portable wireless communication device carried by a pedestrian. The in-vehicle wireless communication device wirelessly transmits a pedestrian-to-pedestrian transmission signal including vehicle GPS information including the position of the vehicle. The in-vehicle wireless communication device also receives the signal transmitted by the pedestrian portable wireless communication device to obtain pedestrian GPS information. The in-vehicle wireless communication device then determines the degree of danger from the vehicle GPS information and the received pedestrian GPS information, and alerts the driver in a potentially dangerous situation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2004-220143 A Summary of the Invention [Problem to be solved by the invention]

[0005] When detecting a person around the vehicle using a sensor equipped in the vehicle, the sensor may not be able to detect the person correctly due to a malfunction of the sensor. For example, assume that a camera is used as the sensor. If the camera breaks down or water droplets adhere to the lens, the camera cannot capture the image properly. Also, if a person overlaps with a surrounding object, the camera can only capture a part of the person, so that the person may not be detected from the captured image. In such a case, the driver may not recognize the person who was not detected, and there is a risk of contact between the person and the vehicle. The technology disclosed in Patent Document 1 does not fully consider such problems.

[0006] In consideration of the above-mentioned problems, the object of the present disclosure is to provide a notification control device, a notification control system, and a notification control method capable of notifying people in the vicinity of a vehicle that there is a person who has not been recognized by the vehicle. [Means for solving the problem]

[0007] The notification control device according to the present disclosure includes: a roadside detection result acquisition unit that acquires a roadside detection result that is a result of detecting a person within a predetermined range by a roadside device; a vehicle detection result acquisition unit that acquires a vehicle detection result that is a result of detecting a person within the predetermined range in a vehicle; a determination unit that determines whether the roadside detection result and the vehicle detection result match; The vehicle detection system is equipped with a person notification control unit that, when it is determined that the roadside detection result and the vehicle detection result do not match, notifies people within the specified range that there is a person not recognized by the vehicle.

[0008] The notification control system according to the present disclosure comprises: A roadside device; An in-vehicle device; A server, The server, a roadside detection result acquisition unit that acquires a roadside detection result, which is a result of detecting a person within a predetermined range by the roadside device, from the roadside device; a vehicle detection result acquisition unit that acquires from the in-vehicle device a vehicle detection result that is a result of detecting a person within the predetermined range in a vehicle; a determination unit that determines whether the roadside detection result and the vehicle detection result match; The vehicle detection system has a person notification control unit that, when it is determined that the roadside detection result and the vehicle detection result do not match, notifies people within the specified range that there is a person not recognized by the vehicle.

[0009] The notification control method according to the present disclosure includes: a roadside detection result acquisition step of acquiring a roadside detection result which is a result of detecting a person within a predetermined range by a roadside device; a vehicle detection result acquisition step of acquiring a vehicle detection result which is a result of detecting a person within the predetermined range in a vehicle; a determination step of determining whether the roadside detection result and the vehicle detection result match; If it is determined that the roadside detection result and the vehicle detection result do not match, a person notification control step is executed in which a person not recognized by the vehicle is present and a person notification step is executed to notify the person within the specified range. Effect of the Invention

[0010] The notification control device, notification control system, and notification control method disclosed herein can notify those in the vicinity of the vehicle that there is a person who has not been recognized by the vehicle. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an overall configuration of a notification control system according to a first embodiment. [Diagram 2] 1 is a diagram showing an external appearance of a roadside device according to a first embodiment. [Diagram 3] FIG. 1 is a diagram showing the appearance of a vehicle according to a first embodiment. [Figure 4] 1 is a block diagram showing a functional configuration of a notification control system according to a first embodiment. [Diagram 5] 5 is a diagram showing a specific example of a process performed by the roadside device according to the first embodiment; FIG. [Figure 6] 5 is a flowchart showing a notification control process performed by a control unit of the roadside device according to the first embodiment. [Figure 7] FIG. 11 is a block diagram showing a functional configuration of a notification control system according to a second embodiment. [Figure 8] 11 is a flowchart showing a notification control process performed by a control unit of a roadside device according to a second embodiment. [Figure 9] FIG. 11 is a block diagram showing a functional configuration of a notification control system according to a third embodiment. [Figure 10] FIG. 13 is a diagram showing an example of an action plan input screen displayed on a display unit according to the third embodiment. [Figure 11] 13 is a flowchart showing a notification control process performed by a control unit of a roadside device according to a third embodiment. [Figure 12] FIG. 11 is a block diagram showing a functional configuration of a notification control system according to a fourth embodiment. [Figure 13] 13 is a flowchart showing a notification control process performed by an in-vehicle device of a vehicle according to a fourth embodiment. [Figure 14] FIG. 13 is a diagram showing an example of a recognition situation screen showing a person recognition situation in a vehicle according to the fourth embodiment. [Figure 15] FIG. 13 is a block diagram showing the overall configuration of a notification control system according to a fifth embodiment. [Figure 16] FIG. 13 is a block diagram showing the functional configuration of a notification control system according to a fifth embodiment. [Figure 17] 13 is a flowchart showing a notification control process performed by a control unit of a server according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals. For clarity of explanation, duplicate explanations will be omitted as necessary.

[0013] The problem to be solved by the present disclosure will be described in more detail. For example, assume that a pedestrian is attempting to cross a crosswalk without a traffic light and a moving vehicle is approaching. According to traffic rules, pedestrians have priority, so the pedestrian may cross the crosswalk assuming that the vehicle recognizes the pedestrian and that the vehicle will stop. However, the pedestrian may be located in a blind spot from the driver's seat of the vehicle, and the driver may not be able to recognize the pedestrian. For example, if there is a large bus stop in front of the crosswalk, the pedestrian may be hidden by the bus stop and the driver may not be able to recognize the pedestrian.

[0014] Furthermore, for example, when the vehicle is an autonomous vehicle or when the driver drives relying on safety features such as a drive recorder, the sensor may not be able to correctly detect pedestrians due to malfunctions, as described above. For example, pedestrians may not be correctly detected from the captured image due to water droplets on the camera, camera failure, or a pedestrian overlapping an object and therefore not being captured in part. If the vehicle cannot correctly recognize the pedestrian, the vehicle may proceed straight ahead, causing an accident between the vehicle and the pedestrian.

[0015] The notification control device, notification control system, and notification control method disclosed herein solve such problems by notifying people in the vicinity of the vehicle that there is a person that the vehicle has not recognized, thereby alerting those people in the vicinity of the vehicle.

[0016] <Embodiment 1> Fig. 1 is a block diagram showing the overall configuration of a notification control system 100 according to this embodiment. An overview of this embodiment will be described with reference to Fig. 1. The notification control system 100 includes a roadside device 1 and an in-vehicle device 20 mounted in a vehicle 2.

[0017] The roadside device 1 and the vehicle 2 are connected via a network N. The network N is, for example, a wireless communication line. For example, a wireless LAN (Local Area Network) or an Internet line is used as the network N. The type of communication is not limited. Furthermore, the number of vehicles 2 connected to the roadside device 1 is not limited to one. The roadside device 1 may be connected to multiple vehicles 2.

[0018] The roadside device 1 is a device installed on the roadside. The roadside device 1 may be installed near a road. The roadside device 1 may be installed on a sidewalk or a parking space adjacent to a road. The in-vehicle device 20 is a device mounted on a vehicle 2. The in-vehicle device 20 performs various controls in the vehicle 2. Detailed configurations of the roadside device 1 and the vehicle 2 will be described later.

[0019] The roadside device 1 and the vehicle 2 each include a sensor. The roadside device 1 and the vehicle 2 perform sensing using the sensor that each includes. The in-vehicle device 20 transmits the sensing result of the vehicle 2 to the roadside device 1. The roadside device 1 receives the sensing result from the in-vehicle device 20 and compares the received sensing result with the sensing result of the roadside device 1. The roadside device 1 performs notification control according to the comparison result.

[0020] Specifically, the roadside device 1 and the vehicle 2 detect a person within a predetermined range by sensing the predetermined range from their respective positions. The predetermined range is an area that is the target of sensing by both the sensor provided in the roadside device 1 and the sensor provided in the vehicle 2. The predetermined range may be, for example, an area where there is a high risk of a traffic event occurring between a pedestrian and the vehicle 2. A traffic event indicates a traffic accident or an event where there is a high possibility of a traffic accident occurring. The predetermined range may be, for example, an area within a predetermined distance from an intersection. The predetermined range is not limited to this, and may be set to any area. For example, the predetermined range may be an area within a predetermined distance from a crosswalk, an area where there are many pedestrians under a certain age, or an area where traffic accidents occur frequently.

[0021] In the following, the result of the roadside device 1 detecting a person within a specified range is referred to as the "roadside detection result." Also, the result of the vehicle 2 detecting a person within a specified range is referred to as the "vehicle detection result." The roadside detection result includes information on roadside-detected people, who are people detected by the roadside device 1. For example, the roadside detection result may include the number of roadside-detected people, characteristics of the roadside-detected people, etc. Similarly, the vehicle detection result includes information on vehicle-detected people, who are people detected by the vehicle 2. For example, the vehicle detection result may include the number of vehicle-detected people, characteristics of the vehicle-detected people, etc. The roadside detection result may include information on multiple roadside-detected people. Similarly, the vehicle detection result may include information on multiple vehicle-detected people.

[0022] For example, if cameras are used as sensors for both the roadside device 1 and the vehicle 2, the shooting ranges of the respective cameras are different. Therefore, the captured images obtained by the camera of the roadside device 1 and the camera of the vehicle 2 are different. Therefore, the people detected from each captured image are also different. Therefore, the roadside detection result and the vehicle detection result may differ.

[0023] The roadside device 1 judges whether the roadside detection result and the vehicle detection result match. If it is judged that the roadside detection result and the vehicle detection result match, it can be said that a person within the predetermined range is recognized by both the roadside device 1 and the vehicle 2. Conversely, if it is judged that the roadside detection result and the vehicle detection result do not match, it can be said that a person not recognized by either the roadside device 1 or the vehicle 2 exists within the predetermined range. A person not recognized by the vehicle 2 is more likely to have an accident or the like between the vehicle 2 and the person. In this embodiment, if it is judged that the roadside detection result and the vehicle detection result do not match, the roadside device 1 notifies the person within the predetermined range that a person not recognized by the vehicle 2 exists.

[0024] As an example of a mismatch between the roadside detection result and the vehicle detection result, a case where a person who is recognized by the vehicle 2 but not recognized by the roadside device 1 is present within a predetermined range is considered. Although not recognized by the roadside device 1, such a person is recognized by the vehicle 2, and therefore the risk of an accident or the like occurring between the person and the vehicle 2 is considered to be low. Therefore, such an example of a mismatch will not be mentioned here.

[0025] However, when the roadside detection result and the vehicle detection result do not match, the roadside device 1 may determine whether or not the person included in the vehicle detection result includes the person included in the roadside detection result. When the person included in the vehicle detection result includes the person included in the roadside detection result, the roadside device 1 does not need to perform the process according to this embodiment. When the person included in the vehicle detection result does not include the person included in the roadside detection result, the roadside device 1 performs the process according to this embodiment.

[0026] When the roadside device 1 determines that the roadside detection result does not match the vehicle detection result, it notifies people within a predetermined range that there is a person not recognized by the vehicle 2. For example, when the roadside device 1 determines that the number of pedestrians indicated by the roadside detection result does not match the number of pedestrians indicated by the vehicle detection result, it notifies people within a predetermined range that there is a person not recognized by the vehicle 2. For example, the roadside device 1 is provided with a display, and notifies all people within the predetermined range by displaying a notification on the display.

[0027] (Configuration of notification control system 100) The configuration of the notification control system 100 will be described in detail with reference to Fig. 2 to Fig. 4. Fig. 2 is a diagram showing the external appearance of the roadside device 1. Fig. 3 is a diagram showing the external appearance of the vehicle 2. Fig. 4 is a block diagram showing the functional configuration of the notification control system 100.

[0028] (Roadside equipment 1) The roadside device 1 will be described with reference to Fig. 2 and Fig. 4. In this embodiment, a smart pole equipped with a sensor and a communication function will be used as an example of the roadside device 1. However, the roadside device 1 is not limited to this, and may be configured by infrastructure such as a traffic light.

[0029] 2, the roadside device 1 includes a main body unit 1P, a sensor unit 15, a display unit 16, and an audio output unit 17. The main body unit 1P is configured as a pole-shaped (cylindrical) housing. However, the shape of the main body unit 1P is not limited to this, and may be configured arbitrarily. For example, the main body unit 1P may be a polygonal columnar shape.

[0030] The sensor unit 15 is provided, for example, on the upper part of the main body unit 1P. The sensor unit 15 is, for example, composed of a camera having a predetermined range as an imaging range. Therefore, the sensor unit 15 is attached at a height that allows imaging of the predetermined range. The display unit 16 notifies people within the predetermined range by display. The display unit 16 is provided at a size that allows people within the predetermined range to be visible. The audio output unit 17 notifies people within the predetermined range by audio. The audio output unit 17 may be provided at a position where the output audio can be easily heard by people within the predetermined range.

[0031] The main body 1P includes a control unit 10, a communication unit 18, and a storage unit 19 in a housing. These will be described later. Note that the configuration of the roadside device 1 shown in Fig. 2 is an example, and the configuration of the roadside device 1 may be changed as appropriate. For example, the positions of the control unit 10, the sensor unit 15, the display unit 16, the audio output unit 17, the communication unit 18, and the storage unit 19 are not limited to those shown in the figure.

[0032] The functional configuration of the roadside device 1 will be described with reference to FIG. 4. As described above, the roadside device 1 includes a control unit 10, a sensor unit 15, a display unit 16, an audio output unit 17, a communication unit 18, and a storage unit 19. The roadside device 1 includes a processor and a memory as components not shown. The storage unit 19, which is a storage device, stores a computer program in which the processing according to this embodiment is implemented. The processor can load the computer program from the storage device into the memory and execute the computer program. In this way, the processor realizes the functions of the roadside detection result acquisition unit 11, the vehicle detection result acquisition unit 12, the determination unit 13, and the person notification control unit 14 included in the control unit 10.

[0033] The roadside detection result acquisition unit 11, the vehicle detection result acquisition unit 12, the determination unit 13, and the person notification control unit 14 may each be realized by dedicated hardware. In addition, a part or all of the components of each device may be realized by a general-purpose or dedicated circuit, processor, etc., or a combination of these. These may be configured by a single chip, or may be configured by multiple chips connected via a bus. A part or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program. In addition, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a quantum processor (quantum computer control chip), etc. can be used as the processor.

[0034] In addition, when some or all of the components of the control unit 10 are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or distributed. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, etc., in a form in which each is connected via a communication network. In addition, the function of the control unit 10 may be provided in a SaaS (Software as a Service) format. The above-mentioned method of realizing each component of the control unit 10 is also the same for the in-vehicle device 20 described later.

[0035] The control unit 10 is an example of a notification control device according to the present disclosure. The control unit 10 performs a predetermined notification control process in the roadside device 1. The control unit 10 includes a roadside detection result acquisition unit 11, a vehicle detection result acquisition unit 12, a determination unit 13, and a person notification control unit 14.

[0036] The roadside detection result acquisition unit 11 acquires from the sensor unit 15 roadside detection results, which are the results of detecting people within a predetermined range by roadside devices. The roadside detection results include information on roadside-detected people. The roadside detection results may include, for example, the number of roadside-detected people and the characteristics of the roadside-detected people. The characteristics of the roadside-detected people are, for example, the appearance and movement of the roadside-detected people. The appearance of the roadside-detected people is, for example, the height, clothing, belongings, face, physique, or posture of the roadside-detected people. Not limited to these, any information may be used as the characteristics of the roadside-detected people.

[0037] Furthermore, the roadside detection result acquisition unit 11 may acquire information calculated from these pieces of information as the roadside detection result. For example, the roadside detection result acquisition unit 11 may calculate the height of the roadside detected person based on the result of image analysis. The roadside detection result acquisition unit 11 may estimate the BMI and the like from the physique information. The roadside detection result acquisition unit 11 may estimate the age of the person by using a known face recognition algorithm using AI (Artificial Intelligence) technology.

[0038] The vehicle detection result acquisition unit 12 acquires a vehicle detection result, which is a result of detecting a person within a predetermined range in a vehicle, from the in-vehicle device 20. The vehicle detection result acquisition unit 12 can acquire the vehicle detection result via the communication unit 18. An example of the vehicle detection result is the same as the roadside detection result described above.

[0039] The determination unit 13 determines whether the roadside detection result and the vehicle detection result match. For example, the roadside detection result and the vehicle detection result each include the number of people. The determination unit 13 determines whether the number of people indicated by the roadside detection result matches the number of people indicated by the vehicle detection result. Also, for example, the roadside detection result and the vehicle detection result each include the color of the clothes of the people. The determination unit 13 determines whether the color of the clothes of the people indicated by the roadside detection result matches the color of the clothes of the people indicated by the vehicle detection result.

[0040] For example, if the roadside detection result includes a person wearing red clothing, but the vehicle detection result does not include a person wearing red clothing, the determination unit 13 determines that the roadside detection result and the vehicle detection result do not match. Without being limited to this, the determination unit 13 may make a determination according to information on people indicated by the roadside detection result and the vehicle detection result.

[0041] When it is determined that the roadside detection result and the vehicle detection result do not match, the person notification control unit 14 determines that there is a person not recognized by the vehicle 2 and notifies people within a predetermined range. For example, the person notification control unit 14 generates display information for notifying that there is a person not recognized by the vehicle 2, and displays it on the display unit 16. The display information may be stored in the storage unit 19 in advance. For example, the person notification control unit 14 displays a message such as "Vehicle approaching! Caution!" on the display unit 16.

[0042] The contents of the notification may be set in advance, or may be changed as appropriate depending on the situation of the vehicle 2, etc. For example, the person notification control unit 14 may calculate a danger level indicating the level of danger, and determine the notification form depending on the danger level. For example, the person notification control unit 14 may determine the size and color of the characters depending on the danger level. Furthermore, the person notification control unit 14 may cause the characters to blink, etc.

[0043] A specific example will be used for explanation. For example, assume that there is a parent and child on a crosswalk, and the child hiding in the parent's shadow cannot be detected by the vehicle 2. In this case, the roadside detection result indicates that there are two people, but the vehicle detection result indicates that there is only one person. In such a case, the determination unit 13 determines that the roadside detection result and the vehicle detection result do not match. Furthermore, the person notification control unit 14 notifies all people around the crosswalk. For example, the person notification control unit 14 notifies by displaying the above-mentioned message on the display unit 16.

[0044] In this way, the person notification control unit 14 can alert all people within the predetermined range. The people within the predetermined range include roadside detected people detected by the sensor unit 15.

[0045] The person notification control unit 14 may perform notification by projecting the display information onto the road. By providing the roadside device 1 with a projector, the person notification control unit 14 can project the display information onto the road. The person notification control unit 14 may generate display information different from the display information to be displayed on the display unit 16.

[0046] Furthermore, the person notification control unit 14 may generate audio information for notifying the vehicle 2 that there is a person not recognized by the vehicle 2. The person notification control unit 14 causes the generated audio information to be output from the audio output unit 17. For example, the person notification control unit 14 causes the audio output unit 17 to output a sound such as "Vehicle approaching." By using a display and a sound, the person notification control unit 14 can make a more effective notification.

[0047] Alternatively, the person notification control unit 14 may notify a terminal carried by each person within a predetermined range. The terminal may be, for example, a smartphone, a mobile phone, a beacon, smart glasses, or a smart watch, but is not limited to these. The person notification control unit 14 notifies by display, sound, vibration, or the like. The content of the notification may be the same as the notification using the display unit 16 and the sound output unit 17. In this embodiment, the roadside device 1 can notify by the display unit 16 and the sound output unit 17, so that it is possible to reliably notify even a person who does not carry a terminal or a person who cannot confirm a notification received at the terminal.

[0048] The sensor unit 15 detects a person within a predetermined range in the roadside device 1. The sensor unit 15 outputs a roadside detection result, which is a result of the detection, to the control unit 10. The sensor unit 15 is, for example, a camera. In this case, the sensor unit 15 detects a person from a captured image acquired by shooting with the camera. The sensor unit 15 may detect a person within a predetermined range using any image analysis technology. The sensor unit 15 may detect a person using an image captured by another camera instead of or in addition to an image captured by a camera equipped in the roadside device 1. For example, the sensor unit 15 may use an image captured by a camera equipped in the vehicle 2. In the case where the vehicle 2 is equipped with a camera that captures, for example, the front, rear, left and right, the sensor unit 15 may detect a person using a captured image obtained by synthesizing the respective captured images. Alternatively, the sensor unit 15 may detect a person using a schematic image reconstructed from information recognized by the roadside device 1, rather than an image captured by a camera, etc.

[0049] The sensor unit 15 may be a distance measurement sensor that measures the distance to an object present around the roadside device 1. The distance measurement sensor may be, for example, a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging). In this case, the distance measurement sensor detects people in the vicinity by transmitting laser light around the roadside device 1 and receiving the laser light reflected by the people or objects. In addition to the LIDAR, various sensors capable of measuring the distance to an object, such as a stereo camera, a ToF (Time of Flight) camera, and a millimeter wave radar, may be used as the distance measurement sensor. The distance measurement sensor may also have a camera function. By using a camera and a distance measurement sensor in combination, the accuracy of person recognition can be improved. The sensor unit 15 may include multiple sensors.

[0050] The display unit 16 displays information related to the processing of the present disclosure. The display unit 16 is, for example, a display device such as a liquid crystal display. The display unit 16 may be a touch panel or the like having a function of an input unit that receives input from a pedestrian or the like.

[0051] The audio output unit 17 outputs information related to the processing according to the present disclosure as audio. The audio output unit 17 is, for example, a speaker. The audio output unit 17 may be a directional speaker. The audio output unit 17 may have a function of an input unit that accepts audio input from a pedestrian or the like.

[0052] The communication unit 18 communicates with the in-vehicle device 20. The communication unit 18 may be configured with a communication interface for performing wired or wireless communication. The communication unit 18 includes a transmitting unit that transmits data and a receiving unit that receives data. The communication unit 18 transmits and receives data related to the processing of this embodiment to and from the in-vehicle device 20.

[0053] The storage unit 19 stores various data and programs. At least a part of the storage unit 19 is configured with a non-volatile memory so that data is retained even when the power of the roadside device 1 is turned off. For example, the storage unit 19 stores programs for realizing each function of the control unit 10.

[0054] (Car 2) The vehicle 2 will be described with reference to Figs. 3 and 4. In this embodiment, a vehicle equipped with a sensor and a communication function will be described as an example of the vehicle 2. The vehicle 2 may include at least a driver as an occupant. The vehicle 2 may be an autonomous vehicle that does not include an occupant. The vehicle 2 may be, for example, a drone capable of autonomously traveling. In this embodiment, an example in which the vehicle 2 is an autonomous vehicle will be described.

[0055] 3, the vehicle 2 includes a vehicle body 2B, a sensor unit 25, and a display unit 26. The sensor unit 25 is provided, for example, on the upper front part of the vehicle body 2B. The sensor unit 25 performs sensing toward the front of the vehicle 2. The sensor unit 25 is configured, for example, with a camera having a predetermined range as an imaging range. Therefore, the sensor unit 25 is attached at a height that allows it to image the predetermined range.

[0056] The display unit 26 is provided on the front of the vehicle body 2B. The display unit 26 notifies people within a predetermined range by displaying information. The display unit 26 is provided in a size that allows people within the predetermined range to see it. Note that, although only the front of the vehicle 2 is shown in the figure, the rear may be configured in the same way. For example, a display unit 26 similar to that on the front may be provided on the rear of the vehicle 2. Also, a sensor unit 25 similar to that on the front may be provided on the rear of the vehicle 2.

[0057] The vehicle body 2B includes an in-vehicle device 20, an audio output unit 27, a communication unit 28, and a storage unit 29. The audio output unit 27 notifies people within a predetermined range by audio. The audio output unit 27 may be a horn or the like provided on the vehicle 2, or may be a speaker capable of outputting a message. The audio output unit 27 may be provided in a position where the output audio can be easily heard by people within a predetermined range.

[0058] 3 is an example, and may be appropriately modified. For example, the positions of the in-vehicle device 20, the sensor unit 25, the display unit 26, the audio output unit 27, the communication unit 28, and the storage unit 29 are not limited to those shown in the figure.

[0059] Referring again to FIG. 4, the functional configuration of the vehicle 2 will be described. As described above, the vehicle 2 includes the in-vehicle device 20, the sensor unit 25, the display unit 26, the audio output unit 27, the communication unit 28, and the storage unit 29. The vehicle 2 includes a processor and a memory as components not shown. The storage unit 29, which is a storage device, stores a computer program in which the processing according to this embodiment is implemented. The processor can load the computer program from the storage device into the memory and execute the computer program. In this way, the processor realizes the functions of the in-vehicle device 20. The in-vehicle device 20 can be realized in a manner similar to the control unit 10 described above.

[0060] The in-vehicle device 20 performs the process according to this embodiment in the vehicle 2. The in-vehicle device 20 acquires a vehicle detection result acquired by the sensor unit 25. The in-vehicle device 20 transmits the vehicle detection result to the roadside device 1 via the communication unit .

[0061] The sensor unit 25 detects a person within a predetermined range in the vehicle 2. The sensor unit 25 outputs a vehicle detection result, which is a result of the detection, to the in-vehicle device 20. The sensor unit 25 is, for example, a camera. In this case, the sensor unit 25 detects a person from a captured image acquired by shooting with the camera. The sensor unit 25 may detect a person within a predetermined range using any image analysis technology. For example, when the sensor unit 25 is a camera that captures the front, rear, left and right, the sensor unit 25 may detect a person using a captured image obtained by synthesizing the respective captured images. The sensor unit 25 may detect a person using an image captured by another camera instead of an image captured by a camera equipped in the vehicle 2, or in addition to an image captured by a camera equipped in the vehicle 2. Alternatively, the sensor unit 25 may detect a person using a schematic image reconstructed from information recognized by the vehicle 2, rather than a camera image or the like.

[0062] The sensor unit 25 may be a distance measurement sensor that measures the distance to an object present around the roadside device 1. The distance measurement sensor may be, for example, a LIDAR. In this case, the distance measurement sensor detects people in the vicinity by transmitting laser light to the surroundings of the vehicle 2 and receiving the laser light reflected by the people or objects. In addition to the LIDAR, various sensors capable of measuring the distance to an object, such as a stereo camera, a ToF camera, or a millimeter wave radar, may be used as the distance measurement sensor. The distance measurement sensor may also have a camera function. By using a camera and a distance measurement sensor in combination, the accuracy of person recognition can be improved. The sensor unit 25 may include multiple sensors.

[0063] The display unit 26 displays information related to the processing of the present disclosure. The display unit 26 is, for example, a display device such as a liquid crystal display. The display unit 16 may be a touch panel or the like having a function of an input unit that accepts input from a pedestrian or the like.

[0064] The audio output unit 27 outputs information related to the processing according to the present disclosure by voice. The audio output unit 27 is, for example, a horn or a speaker. The audio output unit 27 may be a speaker having directionality. The audio output unit 27 may have a function of an input unit that accepts voice input from a pedestrian or the like.

[0065] The communication unit 28 communicates with the roadside device 1. The communication unit 28 may be configured with a communication interface for performing wired or wireless communication. The communication unit 28 includes a transmitting unit that transmits data and a receiving unit that receives data. The communication unit 28 transmits and receives data related to the processing of this embodiment to and from the roadside device 1.

[0066] The storage unit 29 stores various data and programs. At least a part of the storage unit 29 is configured with a non-volatile memory so that data is retained even when the power of the vehicle 2 is turned off. For example, the storage unit 29 stores programs for realizing each function of the in-vehicle device 20.

[0067] (Specific examples) Fig. 5 is a diagram showing a specific example of processing performed by roadside device 1. Fig. 5 shows the vicinity of a road as seen from vehicle 2. The top of the figure indicates the traveling direction of vehicle 2. As shown in the figure, roadside device 1 is installed on the roadside. In addition, there are pedestrians P1 to P4 within a predetermined range. Here, pedestrian P3 can be recognized by roadside device 1, but cannot be recognized by vehicle 2 because he is hidden by a bus stop BS.

[0068] The sensor unit 15 of the roadside device 1 performs sensing within a predetermined range and detects pedestrians P1 to P4. The roadside detection result acquisition unit 11 acquires roadside detection results including information on the pedestrians P1 to P4 from the sensor unit 15. For example, the roadside detection result acquisition unit 11 acquires that the number of people detected on the roadside is 4.

[0069] Furthermore, the sensor unit 25 of the vehicle 2 performs sensing within a predetermined range and detects pedestrians P1, P2, and P4. The in-vehicle device 20 transmits a vehicle detection result including information on the pedestrians P1, P2, and P4 to the roadside device 1. The roadside device 1 receives the vehicle detection result from the in-vehicle device 20. As a result, the vehicle detection result acquisition unit 12 of the roadside device 1 acquires the vehicle detection result including information on the pedestrians P1, P2, and P4. For example, the vehicle detection result acquisition unit 12 acquires that the number of vehicle-detected people is 3 as the vehicle detection result.

[0070] The determination unit 13 of the roadside device 1 determines whether the roadside detection result and the vehicle detection result match. Here, the determination unit 13 determines whether the number of people detected on the roadside match the number of people detected by the vehicle. Since the number of people detected on the roadside and the number of people detected by the vehicle are 4 and 3, respectively, the determination unit 13 determines that the number of people detected on the roadside do not match the number of people detected by the vehicle. The person notification control unit 14 notifies people within a specified range that there is a person not recognized by the vehicle 2.

[0071] In the example of FIG. 5, the person notification control unit 14 displays a message "Vehicle approaching! Caution!" on the display unit 16. The person notification control unit 14 may also output the message by voice from the voice output unit 17. In this way, the pedestrians P1 to P4 can know that the vehicle 2 is approaching. The person notification control unit 14 may also display or output information by voice such as "There is one person not visible from the vehicle." In this way, the pedestrians P1 to P4 can pay more attention to the vehicle 2.

[0072] (Notification control process performed by the control unit 10 of the roadside device 1) The notification control process performed by the roadside device 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the notification control process performed by the control unit 10 of the roadside device 1.

[0073] First, the roadside detection result acquisition unit 11 acquires the roadside detection result from the sensor unit 15 of the roadside device 1 (S11). The roadside detection result is the result of detecting people within a predetermined range by the roadside device 1. The roadside detection result includes information on roadside detected people detected by the roadside device 1. For example, the roadside detection result may include the number of people detected on the roadside, characteristics of the people detected on the roadside, etc.

[0074] Next, the vehicle detection result acquisition unit 12 acquires the vehicle detection result from the in-vehicle device 20 (S12). The vehicle detection result is a result of detecting people within a predetermined range in the vehicle 2. The vehicle detection result includes information on the vehicle-detected people detected in the vehicle 2. For example, the vehicle detection result may include the number of vehicle-detected people, characteristics of the vehicle-detected people, etc.

[0075] Next, the determination unit 13 determines whether the roadside detection result and the vehicle detection result match (S13). Here, it is assumed that it is determined that the roadside detection result and the vehicle detection result match (YES in S13). In this case, the person within the predetermined range is recognized by both the roadside device 1 and the vehicle 2. Therefore, the determination unit 13 ends the process.

[0076] Assume that it is determined that the roadside detection result and the vehicle detection result do not match (NO in S13). In this case, the person notification control unit 14 determines that there is a person not recognized by the vehicle 2, and notifies the person within a predetermined range (S14). For example, the person notification control unit 14 causes the display unit 16 to display a message. The person notification control unit 14 may also cause the audio output unit 17 to output the message by audio.

[0077] As described above, in the notification control system 100 according to this embodiment, the roadside device 1 acquires the roadside detection result and the vehicle detection result, and determines whether or not the roadside detection result and the vehicle detection result match. If it is determined that the roadside detection result and the vehicle detection result do not match, the roadside device 1 determines that there is a person not recognized by the vehicle 2, and notifies people within a predetermined range. In this way, the notification control system 100 can notify the vicinity of the vehicle 2 that there is a person not recognized by the vehicle 2.

[0078] <Embodiment 2> (Configuration of notification control system 100a) Next, a notification control system 100a according to the second embodiment will be described. The notification control system 100a includes a roadside device 1a and an in-vehicle device 20 mounted on a vehicle 2. The overall configuration of the notification control system 100a is the same as that in Fig. 1, and is therefore not shown. The overall configuration of the notification control system 100a according to the present embodiment can be described by replacing the notification control system 100 in Fig. 1 with the notification control system 100a, and replacing the roadside device 1 with the roadside device 1a.

[0079] The configuration of the notification control system 100a will be described in detail with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of the notification control system 100a. The roadside device 1a includes a control unit 10a instead of the control unit 10 of the first embodiment.

[0080] The control unit 10a is an example of a notification control device according to the present disclosure. The control unit 10a differs from the control unit 10 in that it includes an identification unit 31. In addition, the person notification control unit 14a has a function different from that of the person notification control unit 14. By including the identification unit 31 in the roadside device 1a, the roadside device 1a can identify an undetected person who is included in the roadside detection result and not included in the vehicle detection result. The roadside device 1a notifies the identified undetected person, so that the undetected person can understand that he or she is not recognized by the vehicle 2. In the following, differences from the first embodiment will be mainly described, and overlapping points will be omitted as appropriate.

[0081] When it is determined that the roadside detection result and the vehicle detection result do not match, the identification unit 31 identifies an undetected person who is included in the roadside detection result but not included in the vehicle detection result. An undetected person is a person who is detected on the roadside but not a vehicle-detected person. In other words, an undetected person is recognized by the roadside device 1 but not by the vehicle 2. Therefore, there is a high possibility that an accident will occur between the undetected person and the vehicle 2.

[0082] The identification unit 31 identifies an undetected person based on the roadside detection result and the vehicle detection result. For example, the identification unit 31 identifies an undetected person based on the characteristics of a person included in each of the roadside detection person and the vehicle detection result. As described above, the characteristics of a person are, for example, the appearance and movement of a person. The appearance of a person is, for example, the height, clothing, belongings, face, physique, or posture of a person. Not limited to these, any information may be used as the characteristics of a person. The identification unit 31 outputs information of the identified undetected person to the person notification control unit 14a.

[0083] The identification unit 31 may identify a terminal carried by an undetected person based on pre-registered information, whereby the person notification control unit 14a can send a notification to the identified terminal.

[0084] For example, the identification unit 31 identifies undetected persons and terminals based on person information registered in advance in the storage unit 19. The person information is, for example, information that associates a person's features with a terminal owned by the person. The person information may include, for example, feature quantities that indicate the person's facial features and gait (way of walking) features as the person's features. This allows the identification unit 31 to identify undetected persons and terminals based on the roadside detection results and vehicle detection results. The identification unit 31 may acquire person information from a device other than the roadside device 1a.

[0085] The person notification control unit 14a notifies the undetected person identified by the identification unit 31 that the vehicle 2 has not detected the undetected person. The person notification control unit 14a may notify the terminal identified by the identification unit 31. For example, the person notification control unit 14a causes a message such as "You are not being detected by the vehicle!" to be displayed on the display unit of the identified terminal. The person notification control unit 14a may also cause the message to be output from an audio output unit of the identified terminal.

[0086] Furthermore, the person notification control unit 14a may output the above message using the display unit 16 and the audio output unit 17. For example, the person notification control unit 14a may cause the display unit 16 to display an image of the undetected person. The person notification control unit 14a may also output an audio message that can identify the undetected person, such as "Man in red clothing not detected." In this way, the person notification control unit 14a can notify the undetected person. Furthermore, since it is possible to alert people around the undetected person, it is possible to prevent accidents even if the undetected person does not notice the notification.

[0087] Furthermore, the person notification control unit 14a may use a directional speaker as the audio output unit 17 to notify only the undetected person. In this way, the undetected person can understand that he or she is not recognized by the vehicle 2.

[0088] (Notification control process performed by the control unit 10a of the roadside device 1a) The notification control process performed by the roadside device 1a will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the notification control process performed by the control unit 10a of the roadside device 1a. The notification control process shown in Fig. 8 differs from the notification control process according to the first embodiment described with reference to Fig. 6 in that it includes steps S21 and S22 after step S13. Steps S11 to S13 are similar to the process shown in Fig. 6, and therefore description thereof will be omitted.

[0089] If it is determined that the roadside detection result and the vehicle detection result do not match (NO in S13), the identification unit 31 identifies the undetected person (S21). For example, the identification unit 31 identifies the undetected person based on the characteristics of the person included in each of the roadside detection result and the vehicle detection result. The characteristics of the person are, for example, the appearance or movement of the person. The identification unit 31 may identify the terminal carried by the undetected person.

[0090] The person notification control unit 14a notifies the undetected person that the vehicle 2 has not detected the undetected person (S22). When the terminal is identified, the person notification control unit 14a may notify the terminal. Furthermore, the person notification control unit 14a may use the display unit 16 or the audio output unit 17 to output a message indicating that the vehicle 2 has not detected the undetected person.

[0091] As described above, in the notification control system 100a according to this embodiment, when the roadside device 1a determines that the roadside detection result and the vehicle detection result do not match, the roadside device 1a identifies an undetected person who is included in the roadside detection result but not included in the vehicle detection result. The roadside device 1a notifies the undetected person that the vehicle 2 has not detected the undetected person. In this way, the notification control system 100a can identify and notify the undetected person, so that the undetected person can pay more attention to the vehicle 2.

[0092] <Embodiment 3> (Configuration of notification control system 100b) Next, a notification control system 100b according to a third embodiment will be described. The notification control system 100b includes a roadside device 1b and an in-vehicle device 20 mounted on a vehicle 2. The overall configuration of the notification control system 100b is the same as that in Fig. 1, and is therefore not shown. The overall configuration of the notification control system 100b according to this embodiment can be described by replacing the notification control system 100 in Fig. 1 with the notification control system 100b, and replacing the roadside device 1 with the roadside device 1b.

[0093] The configuration of the notification control system 100b will be described in detail with reference to Fig. 9. Fig. 9 is a block diagram showing the functional configuration of the notification control system 100b. The roadside device 1b includes a control unit 10b instead of the control unit 10 of the first embodiment and the control unit 10a of the second embodiment.

[0094] The control unit 10b is an example of a notification control device according to the present disclosure. The control unit 10b differs from the control unit 10a in that it includes an action schedule acquisition unit 41 and a vehicle notification control unit 42. By including the action schedule acquisition unit 41 and the vehicle notification control unit 42 in the roadside device 1b, the roadside device 1b can acquire the action schedule of a person within a predetermined range and notify the vehicle 2 of the acquired action schedule. This allows the vehicle 2 to travel while taking into account the action schedule of an undetected person. Below, differences from the second embodiment will be mainly described, and overlapping points will be omitted as appropriate.

[0095] The behavior schedule acquisition unit 41 acquires the behavior schedule of the non-detected person after the notification by the person notification control unit 14a. The behavior schedule of the non-detected person indicates how the non-detected person who received the notification plans to act. For example, the behavior schedule includes the intention of the non-detected person, such as to cross the road or not to cross the road.

[0096] The activity schedule acquisition unit 41 acquires the activity schedule of the undetected person via, for example, the input unit of the roadside device 1b. Here, as an example, a case will be described in which the display unit 16 is a touch panel having a function of an input unit that accepts input from the undetected person. When the undetected person realizes from the notification that he or she has not been recognized by the vehicle 2, he or she inputs the activity schedule into the roadside device 1b by touching the display unit 16.

[0097] For example, the action schedule acquisition unit 41 displays an action schedule input screen including an image area for selecting whether or not to cross the road on the display unit 16. The image area for selecting whether or not to cross the road may be, for example, a button indicating an intention to cross and a button indicating an intention not to cross.

[0098] 10 is a diagram showing an example of an action plan input screen 16a displayed by the display unit 16. As shown in the figure, the action plan input screen 16a includes a "Cross" button A and a "Don't cross" button B. The action plan input screen 16a may also include buttons such as "Undecided" and "Don't know." The action plan acquisition unit 41 accepts the input of an action plan in response to pressing any of the buttons.

[0099] The activity schedule acquisition unit 41 may receive the activity schedule input by using a voice input. The activity schedule acquisition unit 41 may also receive the activity schedule input via a terminal carried by the undetected person. In this case, the undetected person transmits the activity schedule to the roadside device 1b using the terminal. The activity schedule acquisition unit 41 may detect the pressing of any of the buttons in a non-contact manner.

[0100] Furthermore, the behavior schedule acquisition unit 41 may acquire a behavior schedule using a method other than accepting input of a behavior schedule from an undetected person. For example, the behavior schedule acquisition unit 41 may acquire a behavior schedule based on information detected by the sensor unit 15. Here, it is assumed that the sensor unit 15 includes a camera. The behavior schedule acquisition unit 41 acquires the behavior schedule of the undetected person using an image captured by the camera. For example, the behavior schedule acquisition unit 41 predicts the behavior of the undetected person using any image analysis technology.

[0101] For example, in the example of FIG. 5, it is assumed that pedestrian P3 is an undetected person. Pedestrian P3 is walking toward the road. Therefore, the behavior schedule acquisition unit 41 determines that pedestrian P3 will cross the road. The behavior schedule acquisition unit 41 may calculate the walking speed of pedestrian P3, and determine that pedestrian P3 will cross the road if there is no change in the walking speed or if the walking speed has increased. Conversely, if the walking speed has decreased, the behavior schedule acquisition unit 41 may determine that pedestrian P3 will not cross the road.

[0102] 5, the pedestrian P1 is also an undetected person. When the undetected person faces the opposite side of the road like the pedestrian P1, the schedule acquisition unit 41 determines that the undetected person does not cross the road. Without being limited to these, the schedule acquisition unit 41 may acquire the schedule of the undetected person by using any technique.

[0103] Returning to Fig. 9, the vehicle notification control unit 42 notifies the vehicle 2 of the action plan acquired by the action plan acquisition unit 41. The in-vehicle device 20 of the vehicle 2 acquires the action plan notified from the vehicle notification control unit 42 and controls the driving of the vehicle 2. This enables the vehicle 2 to drive in accordance with the action plan.

[0104] Here, an example of processing in the vehicle 2 that has acquired the action plan of the undetected person will be specifically described. First, an explanation will be given assuming that there is one undetected person. It is assumed that the "Cross" button A is pressed on the action plan input screen 16a shown in FIG. 10. If the vehicle 2 is an autonomous vehicle, the in-vehicle device 20 stops the vehicle 2. If the vehicle 2 has already stopped, the vehicle 2 restricts the start of traveling. Also, if the vehicle 2 is not an autonomous vehicle, the in-vehicle device 20 displays a message on a display unit (not shown) provided in the vehicle that prompts the driver to stop the vehicle 2. For example, the in-vehicle device 20 displays a message such as "Pedestrian crossing". The in-vehicle device 20 may output the message by voice using an audio output unit (not shown) that outputs voice to the inside of the vehicle.

[0105] Also, assume that the "Do not cross" button B is pressed. If the vehicle 2 is an autonomous vehicle, the in-vehicle device 20 slows down the vehicle 2. For example, it is desirable for the in-vehicle device 20 to drive the vehicle 2 at a speed that allows the vehicle 2 to stop immediately even if an undetected person jumps out. Also, if the vehicle 2 is not an autonomous vehicle, the in-vehicle device 20 displays a message on a display unit inside the vehicle urging the driver to proceed carefully and at a slower speed. For example, the in-vehicle device 20 displays a message such as "Waiting for pedestrians". The in-vehicle device 20 may output the message as audio into the vehicle.

[0106] Also, assume that neither the "Cross" button A nor the "Don't cross" button B has been pressed. In other words, assume that the action plan of the undetected person has not been selected. If the vehicle 2 is an autonomous vehicle, the in-vehicle device 20 drives the vehicle 2 at the slowest possible speed. For example, the in-vehicle device 20 drives the vehicle 2 at a speed that allows the vehicle 2 to stop immediately even if an undetected person jumps out. Also, if the vehicle 2 is not an autonomous vehicle, the in-vehicle device 20 displays a message on a display unit inside the vehicle urging the driver to proceed carefully at the slowest possible speed. For example, the in-vehicle device 20 displays a message such as "Driving at the slowest possible speed because the action plan is unknown." The in-vehicle device 20 may output the message as audio to the inside of the vehicle.

[0107] Next, a case where there are multiple undetected persons will be described. When there are multiple undetected persons, the action schedule acquisition unit 41 acquires the action schedule of each of the multiple undetected persons. The vehicle notification control unit 42 notifies the vehicle 2 of the action schedule of each of the multiple undetected persons. When the "Cross" button A is pressed by at least one of the multiple undetected persons, the in-vehicle device 20 performs the same process as the process when the "Cross" button A is pressed described above.

[0108] When the "Do not cross" button B is pressed and not selected, the in-vehicle device 20 performs the same process as when not selected. Also, when multiple undetected persons input a matching action plan, the in-vehicle device 20 performs the same process as when there is only one undetected person.

[0109] (Notification control process performed by the control unit 10b of the roadside device 1b) The notification control process performed by the roadside device 1b will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the notification control process performed by the control unit 10b of the roadside device 1b. The notification control process shown in Fig. 11 differs from the notification control process according to the second embodiment described with reference to Fig. 8 in that it includes steps S31 and S32 after step S22. Steps S11 to S22 are similar to the process shown in Fig. 8, and therefore description thereof will be omitted.

[0110] The activity schedule acquisition unit 41 acquires the activity schedule of the undetected person after the notification (S31). For example, the activity schedule acquisition unit 41 may acquire the activity schedule from the display unit 16 having the function of an input unit, or may acquire the activity schedule from the terminal of the undetected person. The vehicle notification control unit 42 notifies the vehicle 2 of the activity schedule (S32). This allows the vehicle 2 to perform control in the in-vehicle device 20 according to the activity schedule.

[0111] As described above, in the notification control system 100b according to this embodiment, the roadside device 1b acquires the behavioral plan of the undetected person after notification, and notifies the acquired behavioral plan to the vehicle 2. In this way, the notification control system 100b can control the vehicle 2 according to the behavioral plan of the undetected person, so that the vehicle 2 can travel more safely.

[0112] <Embodiment 4> (Configuration of notification control system 100c) Next, a notification control system 100c according to a fourth embodiment will be described. The notification control system 100c includes a roadside device 1c and an in-vehicle device 20c mounted on a vehicle 2c. The overall configuration of the notification control system 100c is the same as that in Fig. 1, and is therefore not shown. The overall configuration of the notification control system 100c according to this embodiment can be described by replacing the notification control system 100 in Fig. 1 with the notification control system 100c, the roadside device 1 with the roadside device 1c, and the vehicle 2 with the vehicle 2c.

[0113] The configuration of the notification control system 100c will be described in detail with reference to Fig. 12. Fig. 12 is a block diagram showing the functional configuration of the notification control system 100c. The roadside device 1c includes a control unit 10c instead of the control unit 10 of the first embodiment.

[0114] In the first embodiment, the control unit 10 of the roadside device 1 acquires the roadside detection result and the vehicle detection result, judges whether or not the roadside detection result and the vehicle detection result match, and if it is judged that they do not match, it notifies the person within a predetermined range that there is a person that the vehicle 2 has not recognized. This embodiment differs from the first embodiment in that the above processing is performed by the in-vehicle device 20c of the vehicle 2.

[0115] The control unit 10c performs the process according to this embodiment in the roadside device 1c. The control unit 10c acquires the roadside detection result acquired by the sensor unit 15. The control unit 10c transmits the roadside detection result to the vehicle 2c via the communication unit 18. In this embodiment, the control unit 10c does not include the configurations of the roadside detection result acquisition unit 11, the vehicle detection result acquisition unit 12, the determination unit 13, and the person notification control unit 14 described above.

[0116] The vehicle-mounted device 20c is an example of a notification control device according to the present disclosure. The vehicle-mounted device 20c includes a roadside detection result acquisition unit 21, a vehicle detection result acquisition unit 22, a determination unit 23, and a person notification control unit 24. The roadside detection result acquisition unit 21, the vehicle detection result acquisition unit 22, the determination unit 23, and the person notification control unit 24 correspond to the roadside detection result acquisition unit 11, the vehicle detection result acquisition unit 12, the determination unit 13, and the person notification control unit 14 of the control unit 10 described in the first embodiment, respectively. Therefore, in the following, differences from the first embodiment will be mainly described, and overlapping points will not be described as appropriate.

[0117] The roadside detection result acquisition unit 21 acquires from the roadside device 1 a roadside detection result that is a result of the roadside device 1 detecting a person within a predetermined range. The vehicle detection result acquisition unit 22 acquires from the sensor unit 25 a vehicle detection result that is a result of the vehicle 2 detecting a person within a predetermined range. The determination unit 23 determines whether the roadside detection result and the vehicle detection result match. If it is determined that the roadside detection result and the vehicle detection result do not match, the person notification control unit 24 notifies the person within the predetermined range that there is a person not recognized by the vehicle 2.

[0118] (Notification control process performed by the in-vehicle device 20c of the vehicle 2c) The notification control process performed by the vehicle 2c will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the notification control process performed by the in-vehicle device 20c of the vehicle 2c. The notification control process shown in the figure corresponds to the notification control process of the control unit 10 described with reference to Fig. 6.

[0119] First, the roadside detection result acquisition unit 21 acquires the roadside detection result from the roadside device 1c (S41). Next, the vehicle detection result acquisition unit 22 acquires the vehicle detection result from the sensor unit 25 of the vehicle 2c (S42). Next, the determination unit 23 determines whether the roadside detection result and the vehicle detection result match (S43). If it is determined that the roadside detection result and the vehicle detection result match (YES in S43), the determination unit 23 ends the process.

[0120] If it is determined that the roadside detection result and the vehicle detection result do not match (NO in S43), the person notification control unit 24 determines that there is a person not recognized by the vehicle 2c, and notifies the person within a predetermined range (S44). For example, the person notification control unit 24 causes the display unit 26 to display a message. The person notification control unit 24 may also cause the audio output unit 27 to output the message by voice.

[0121] As described above, in the notification control system 100c according to the present embodiment, the vehicle 2c acquires the roadside detection result and the vehicle detection result, and determines whether or not the roadside detection result and the vehicle detection result match. If it is determined that the roadside detection result and the vehicle detection result do not match, the vehicle 2 determines that there is a person that the vehicle 2 does not recognize, and notifies people within a predetermined range. In this way, the notification control system 100c can notify the surroundings of the vehicle 2 that there is a person that the vehicle 2 does not recognize.

[0122] In addition, when notifying people within a predetermined range from the vehicle 2c as in the present embodiment, the vehicle-mounted device 20c can notify in various forms. Specific examples will be described below.

[0123] The vehicle 2c has a similar configuration to the vehicle 2 as shown in FIG. 3, and therefore has a display unit 26 on the front of the vehicle 2c. As described above, the person notification control unit 24 can notify people within a predetermined range by displaying a notification on the display unit 26. For example, the person notification control unit 24 displays a message such as "Pedestrians cannot be detected!" on the display unit 26. Without being limited to this, the person notification control unit 24 may display a notification by pointing a HUD (Head-Up Display) toward the outside of the vehicle 2c. Furthermore, the person notification control unit 24 may output a message using the audio output unit 27 in addition to the display unit 26.

[0124] Although not shown, the in-vehicle device 20c may have the functions of the control units 10a and 10b described in the second and third embodiments. That is, the present embodiment may be realized in combination with the second and third embodiments.

[0125] For example, the in-vehicle device 20c may have a function corresponding to the identification unit 31. Furthermore, the in-vehicle device 20c may have a function corresponding to the activity schedule acquisition unit 41 and the vehicle notification control unit 42. With these configurations, the in-vehicle device 20c can identify an undetected person when it is determined that the roadside detection result and the vehicle detection result do not match. Furthermore, the in-vehicle device 20c can notify the undetected person that the vehicle 2c has not detected the undetected person. The in-vehicle device 20c may identify the terminal of the undetected person and notify the identified terminal. Furthermore, the in-vehicle device 20c may acquire the activity schedule of the undetected person after the notification and notify the vehicle 2c of the activity schedule.

[0126] The person notification control unit 24 may acquire the action schedule of the vehicle 2c and notify the acquired action schedule of the vehicle 2c. For example, the person notification control unit 24 causes the display unit 26 or the audio output unit 27 to output information such as that the vehicle 2c is about to start or stop. This allows people within a predetermined range to know the action schedule of the vehicle 2c.

[0127] Furthermore, the person notification control unit 24 may display the recognition status of the person in the vehicle 2c on the display unit 26. FIG. 14 is a diagram showing an example of a recognition status screen 26a showing the recognition status of the person in the vehicle 2c. The recognition status screen 26a corresponds to FIG. 5 described above. The recognition status screen 26a is composed of an image in which a captured image of the road surroundings seen from the vehicle 2c and information showing the recognition status of the person in the vehicle 2c are superimposed. For example, the person notification control unit 24 acquires the position of the person within a predetermined range from the roadside device 1c, and displays the captured image of the sensor unit 25 superimposed on the acquired position of the person.

[0128] The person notification control unit 24 may display a vehicle-detected person detected in the vehicle 2c and a non-detected person not detected in the vehicle 2c in different display modes. For example, the person notification control unit 24 may surround a person included in a photographed image acquired by the camera with a predetermined frame line indicating that the person has been recognized by the vehicle 2c.

[0129] In the illustrated example, pedestrians P1, P2, and P4, who are vehicle-detected persons, are surrounded by frame lines L1 to L3. Pedestrian P3, who is an undetected person, is not surrounded by a frame line. The person notification control unit 24 may change the display mode of the frame lines L1 to L3 depending on the situations of the pedestrians P1, P2, and P4. For example, the person notification control unit 24 displays different frame lines depending on the risk of an accident occurring to the pedestrians P1, P2, and P4.

[0130] The person notification control unit 24 may change the color of the border line according to the danger level, or may change the thickness of the border line according to the danger level. For example, the person notification control unit 24 displays the border line using a color associated with the danger level, such as blue when the danger level is low, yellow when the danger level is medium, and red when the danger level is high.

[0131] For example, the person notification control unit 24 determines that the pedestrian P1 who is stopped while looking at the opposite side of the road is at low risk. The person notification control unit 24 determines that the pedestrian P2 who is moving as if he is about to cross the road is at high risk. The person notification control unit 24 determines that the pedestrian P4 who is in a state where it is unclear whether he will cross the road is at medium risk. Therefore, the person notification control unit 24 surrounds the pedestrian P1 with a blue frame L1, the pedestrian P2 with a red frame L2, and the pedestrian P4 with a yellow frame L3.

[0132] For example, a blue border indicates that the vehicle 2c can drive normally. For example, a yellow border indicates that the vehicle 2c should drive slowly. Furthermore, a red border indicates that the vehicle 2c should stop. The danger level is not limited to three levels, and may be set to any level. Furthermore, any color may be set according to the danger level.

[0133] The person notification control unit 24 causes the display unit 26 to display the recognition status screen 26a displaying the color-coded frame lines. In this manner, the person notification control unit 24 can display the people recognized by the vehicle 2c, the people not recognized by the vehicle 2c, and the planned movement of the vehicle 2c among the people within a predetermined range. A pedestrian in the vicinity of the vehicle 2c can visually confirm the display unit 26 to know the people recognized by the vehicle 2c and the people not recognized by the vehicle 2c. In addition, since the pedestrian can know the planned movement of the vehicle 2c, the pedestrian can be careful not to jump out or cross the road in an unreasonable manner, for example. In addition, if the pedestrian knows that the vehicle 2c is about to stop, the pedestrian can cross the road with peace of mind.

[0134] For example, if the vehicle 2c is not an autonomous vehicle, the pedestrian and the driver can communicate with each other through eye contact as to whether or not to give way. However, if the vehicle 2c is an autonomous vehicle, it is difficult for the vehicle 2c and the pedestrian to communicate with each other. In such a case, the vehicle 2c may remain stopped. However, even in such a case, the notification control system 100c according to the present embodiment allows the pedestrian to easily understand the planned actions of the vehicle 2c, and therefore the pedestrian can act safely. This realizes safe and smooth traffic.

[0135] Furthermore, the person notification control unit 24 may transmit the recognition situation screen 26a to the roadside device 1c and cause the recognition situation screen 26a to be displayed on the display unit 16 of the roadside device 1c. This allows the pedestrian to view the recognition situation screen 26a from both the roadside device 1c and the vehicle 2c.

[0136] The person notification control unit 24 may display the recognition status screen 26a on a display unit in the vehicle 2c. This allows the driver to visually check the recognition status screen 26a, and therefore the driver can recognize undetected persons who were not detected by the sensor unit 15. This makes it possible to alert the driver and passengers of the vehicle 2c.

[0137] <Embodiment 5> (Configuration of notification control system 100d) Next, a notification control system 100d according to the fifth embodiment will be described. Fig. 15 is a block diagram showing an overall configuration of the notification control system 100d according to the present embodiment. The notification control system 100d includes a roadside device 1d, a vehicle 2d, and a server 5. The notification control system 100d differs from the notification control system 100 shown in Fig. 1 and the notification control systems 100a to 100c (not shown) in that the notification control system 100d includes the server 5.

[0138] The roadside device 1d, the vehicle 2d, and the server 5 are connected to each other via a network N. The network N is, for example, a wireless communication line. For example, a wireless LAN or an Internet line is used as the network N. The type of communication is not limited. Furthermore, the number of roadside devices 1d and vehicles 2d connected to the server 5 is arbitrary. The server 5 may be connected to multiple roadside devices 1d and multiple vehicles 2d.

[0139] In the first to third embodiments, the notification control processing according to the present disclosure is performed in the roadside devices 1, 1a, and 1b. In the fourth embodiment, the notification control processing according to the present disclosure is performed in the vehicle-mounted device 20c. In this embodiment, the notification control processing according to the present disclosure is performed in the server 5. Therefore, each functional unit of the server 5 is equivalent to, for example, the configuration of the control unit 10 of the roadside device 1 described in the first embodiment (see FIG. 4) or the configuration of the vehicle-mounted device 20c of the vehicle 2c described in the fourth embodiment (see FIG. 12).

[0140] The configuration of the notification control system 100d will be described in detail with reference to FIG. 16. FIG. 16 is a block diagram showing the functional configuration of the notification control system 100d. The roadside device 1d includes a control unit 10d instead of the control unit 10 of the first embodiment. The control unit 10d performs the processing according to this embodiment in the roadside device 1d. The control unit 10d acquires the roadside detection result acquired by the sensor unit 15. The control unit 10d transmits the roadside detection result to the server 5 via the communication unit 18. In this embodiment, the control unit 10d does not include the configurations of the roadside detection result acquisition unit 11, the vehicle detection result acquisition unit 12, the determination unit 13, and the person notification control unit 14 of the first embodiment.

[0141] Further, the in-vehicle device 20d mounted on the vehicle 2d includes the in-vehicle device 20d instead of the in-vehicle device 20 of the first embodiment. The in-vehicle device 20d performs the processing according to this embodiment in the vehicle 2d. The in-vehicle device 20d acquires the vehicle detection result acquired by the sensor unit 25. The in-vehicle device 20d transmits the vehicle detection result to the server 5 via the communication unit 28. In this embodiment, the in-vehicle device 20d does not include the configurations of the roadside detection result acquisition unit 21, the vehicle detection result acquisition unit 22, the determination unit 23, and the person notification control unit 24 in the fourth embodiment.

[0142] The server 5 includes a control unit 50, which is an example of a notification control device according to the present disclosure. The server 5 includes a control unit 50, a communication unit 58, and a storage unit 59. The server 5 includes a processor and a memory as components not shown. The storage unit 59, which is a storage device, stores a computer program in which the processing according to this embodiment is implemented. The processor can load the computer program from the storage device into the memory and execute the computer program. In this way, the processor realizes the functions of the roadside detection result acquisition unit 51, the vehicle detection result acquisition unit 52, the determination unit 53, and the person notification control unit 54 included in the control unit 50. The control unit 50 can be realized in the same manner as the control unit 10 described above.

[0143] The roadside detection result acquisition unit 51 acquires from the roadside device 1d a roadside detection result that is a result of the roadside device detecting people within a predetermined range. The vehicle detection result acquisition unit 52 acquires from the in-vehicle device 20d a vehicle detection result that is a result of the vehicle 2d detecting people within a predetermined range. The determination unit 53 determines whether the roadside detection result and the vehicle detection result match. If the determination result by the determination unit 53 does not match, the person notification control unit 54 notifies the people within the predetermined range that there is a person that the vehicle 2d does not recognize.

[0144] (Notification control process performed by the control unit 50 of the server 5) The notification control process performed by the server 5 will be described with reference to Fig. 17. Fig. 17 is a flowchart showing the notification control process performed by the control unit 50 of the server 5.

[0145] First, the roadside detection result acquisition unit 51 acquires the roadside detection result from the roadside device 1d (S51). Next, the vehicle detection result acquisition unit 52 acquires the vehicle detection result from the in-vehicle device 20d (S52). Next, the determination unit 53 determines whether the roadside detection result and the vehicle detection result match (S53). If it is determined that the roadside detection result and the vehicle detection result match (YES in S53), the determination unit 53 ends the process.

[0146] If it is determined that the roadside detection result and the vehicle detection result do not match (NO in S53), the person notification control unit 54 determines that there is a person not recognized by the vehicle 2d, and notifies the person within a predetermined range (S54). For example, the person notification control unit 54 causes the display unit 16 of the roadside device 1d to display a message. The person notification control unit 54 may also cause the audio output unit 17 of the roadside device 1d to output the message by voice. The person notification control unit 54 may also cause the display unit 26 and audio output unit 27 of the vehicle 2d to output the message by display or voice.

[0147] As described above, in the notification control system 100d according to the present embodiment, the server 5 acquires the roadside detection result from the roadside device 1d and acquires the vehicle detection result from the in-vehicle device 20d. The server 5 judges whether the acquired roadside detection result matches the vehicle detection result. When it is judged that the roadside detection result does not match the vehicle detection result, the server 5 notifies the person within a predetermined range that there is a person not recognized by the vehicle 2d. In this way, the notification control system 100d can notify the surroundings of the vehicle 2d that there is a person not recognized by the vehicle 2d. In this way, the server 5 compares the roadside detection result with the vehicle detection result, thereby reducing the load on the processing of the roadside device 1d and the vehicle 2d.

[0148] The function of the server 5 according to the present embodiment may be provided in the roadside device 1d or in the vehicle 2d. An example in which the roadside device 1d has the function of the server 5 corresponds to the first embodiment. An example in which the vehicle 2d has the function of the server 5 corresponds to the fourth embodiment. That is, the function of the notification control device according to the present disclosure may be provided in a roadside device such as a smart pole, in an in-vehicle device, or in a predetermined server.

[0149] <Hardware configuration example> Each functional component of the above-mentioned roadside devices 1, 1a, 1b, 1c, 1d, vehicles 2, 2c, 2d, and server 5 may be realized by hardware (e.g., hardwired electronic circuits, etc.) that realizes each functional component, or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). For example, any process in the present disclosure can be realized by having a CPU execute a computer program.

[0150] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in various types of non-transitory computer readable medium or tangible storage medium. By way of example and not limitation, non-transitory computer readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on various types of transitory computer readable medium or communication medium. By way of example and not limitation, transitory computer readable medium or communication medium includes electrical, optical, acoustic, or other form of propagated signal.

[0151] The present disclosure is not limited to the above-mentioned embodiments, and can be modified as appropriate without departing from the spirit of the present disclosure. The above-mentioned embodiments 1 to 5 can be implemented in any combination. For example, as described above, embodiment 4 and embodiment 2 may be implemented in combination. Also, embodiment 4 and embodiment 3 may be implemented in combination. Any combination can be implemented without being limited to these. Also, for example, in the above description, the roadside device and the vehicle are provided with a display unit, but this is not limiting. For example, the roadside device and the vehicle may be configured to have only an audio output unit without a display unit.

[0152] This disclosure includes matters that contribute to the realization of the Sustainable Development Goals (SDGs) of "Good Health and Well-Being for All" and contribute to reducing traffic accidents. [Explanation of symbols]

[0153] 1, 1a, 1b, 1c, 1d Roadside equipment 1P main body 2, 2c, 2d vehicles 2B Body 5 Server 10, 10a, 10b, 10c, 10d Control section 11 Roadside detection result acquisition unit 12 Vehicle detection result acquisition unit 13 Judgment section 14, 14a Person notification control section 15 Sensor section 16 Display section 16a Action plan input screen 17 Audio output section 18 Communications Department 19 Memory section 20, 20c, 20d onboard equipment 21 Roadside detection result acquisition unit 22 Vehicle detection result acquisition unit 23 Judgment section 24 Person notification control section 25 Sensor section 26 Display section 26a Recognition status screen 27 Audio output section 28 Communications Department 29 Memory section 31 Specific section 41 Action Plan Acquisition Department 42 Vehicle notification control unit 50 Control section 51 Roadside detection result acquisition unit 52 Vehicle detection result acquisition unit 53 Judgment section 54 Person notification control section 58 Communications Department 59 Memory section 100, 100a, 100b, 100c, 100d Notification Control System A, B buttons BS bus stop L1, L2, L3 borders N Network P1, P2, P3, P4 Pedestrians

Claims

1. a roadside detection result acquisition unit that acquires a roadside detection result that is a result of detecting a person within a predetermined range by a roadside device; a vehicle detection result acquisition unit that acquires a vehicle detection result that is a result of detecting a person within the predetermined range in a vehicle; a determination unit that determines whether the roadside detection result and the vehicle detection result match; and a person notification control unit that notifies a person within the predetermined range when it is determined that the roadside detection result and the vehicle detection result do not match, assuming that a person not recognized by the vehicle is present. Notification control device.

2. an identification unit that, when it is determined that the roadside detection result and the vehicle detection result do not match, identifies an undetected person that is a person that is included in the roadside detection result but not included in the vehicle detection result; The person notification control unit notifies the undetected person that the vehicle has not detected the undetected person. The notification control device according to claim 1 .

3. An action schedule acquisition unit that acquires an action schedule of the non-detected person after the notification, The vehicle notification control unit notifies the vehicle of the planned action. The notification control device according to claim 2 .

4. A roadside device; An in-vehicle device; A server, The server, a roadside detection result acquisition unit that acquires a roadside detection result, which is a result of detecting a person within a predetermined range by the roadside device, from the roadside device; a vehicle detection result acquisition unit that acquires from the in-vehicle device a vehicle detection result that is a result of detecting a person within the predetermined range in a vehicle; a determination unit that determines whether the roadside detection result and the vehicle detection result match; and a person notification control unit that notifies a person within the predetermined range when it is determined that the roadside detection result does not match the vehicle detection result, assuming that a person not recognized by the vehicle is present. Notification control system.

5. a roadside detection result acquisition step of acquiring a roadside detection result which is a result of detecting a person within a predetermined range by a roadside device; a vehicle detection result acquisition step of acquiring a vehicle detection result which is a result of detecting a person within the predetermined range in a vehicle; a determination step of determining whether the roadside detection result and the vehicle detection result match; a person notification control step of notifying a person within the predetermined range when it is determined that the roadside detection result and the vehicle detection result do not match, assuming that a person not recognized by the vehicle is present. Notification control methods.

Images