Notification device, notification method, and program

Abstract

To provide a notification device capable of notifying a person having a possibility to jump out into a travel road of a vehicle of warning at appropriate timing.SOLUTION: A notification device installed in a vehicle acquires attribute information related to a person positioned within a range of a predetermined distance from the vehicle. The notification device classifies the person based on the attribute information. The notification device adjusts notification timing with respect to the person based on a result of the classification.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a notification device, a notification method, and a program.

Background Art

[0002] When a driver of a vehicle recognizes that a person has jumped out onto the vehicle's traveling road, the driver performs driving operations such as stepping on the brake. Related technologies are disclosed in Patent Document 1. Patent Document 1 discloses a technology for identifying an approaching vehicle approaching a crossing prediction area where a pedestrian is predicted to pass from the start to the end of crossing the road, predicting whether the approaching vehicle will collide with the pedestrian, and notifying of the danger of collision when it is predicted that the approaching vehicle will collide with the pedestrian.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For warnings such as when there is a possibility that a person may jump out onto the traveling road of a vehicle as described above, it is desirable to notify at an appropriate timing according to the attributes of the person.

[0005] The present invention has been made in consideration of such circumstances, and one of its purposes is to provide a notification device, a notification method, and a program capable of notifying a warning to a person who may jump out onto the traveling road of a vehicle at an appropriate timing.

Means for Solving the Problems

[0006] The notification device, notification method, and program according to this invention employ the following configuration. (1) A notification device according to one aspect of the present invention comprises an attribute acquisition unit that acquires attribute information relating to a person located within a predetermined distance from a moving object, a classification unit that classifies the person based on the attribute information, and an adjustment unit that adjusts the timing of notification to the person based on the result of the classification.

[0007] (2) In the embodiment of (1) above, the attribute information includes at least information relating to the person’s age, and the classification unit outputs the result of the classification relating to the person’s age based on the age.

[0008] (3) In the embodiment of (1) or (2) above, the device comprises a person recognition unit that recognizes the person and an attribute estimation unit that estimates attribute information of the person based on the result of the person recognition.

[0009] (4) In any embodiment of (1) to (3) above, the device comprises a movement state estimation unit that estimates the direction and speed of movement of the person, and the adjustment unit further adjusts the timing of the notification to the person based on the direction and speed of movement of the person.

[0010] (5) In any embodiment of (1) to (3) above, the adjustment unit comprises a movement state estimation unit that detects the movement speed of the person, and the adjustment unit further adjusts the timing of the notification to the person based on the movement speed of the person.

[0011] (6) In any embodiment of (1) to (5) above, the system includes a road recognition unit that recognizes the intersection state of the road on which the moving body is traveling with another road, and the adjustment unit adjusts the timing of the notification to the person based on the intersection state.

[0012] (7) In any embodiment of (1) to (5) above, the system includes a track recognition unit that recognizes the intersection state of the track on which the moving body is traveling with another track, and the adjustment unit adjusts the timing of the notification to the person based on the intersection state and the position of the person.

[0013] (8) In any of the embodiments described in (1) to (7) above, the adjustment unit adjusts the timing so that it can notify as soon as possible if it is unable to obtain the attribute information within a predetermined time from the time the person is recognized.

[0014] (9): In the embodiment of (7) above, the adjustment unit adjusts the timing so that it can notify as soon as possible if it cannot recognize the intersection of the travel path with other travel paths.

[0015] (10): In the embodiment of (5) above, the adjustment unit adjusts the timing so that it can notify as soon as possible if it cannot detect the speed of movement of the person.

[0016] (11): In any embodiment of (1) to (10) above, the adjustment unit identifies a collision margin time corresponding to a plurality of age groups indicated by the result of the classification of the person, which is the time from when the person enters the road on which the moving body is traveling until it collides with the moving body, and adjusts the timing of the notification to a time when the collision margin time can be secured.

[0017] (12): In any embodiment of (1) to (11) above, the adjustment unit identifies a collision margin time, which is the time from when the person enters the path on which the moving body is traveling until the person collides with the moving body, based on the classification of the person's speed of movement, and adjusts the timing of the notification to a time when the collision margin time can be secured.

[0018] (13) In any of the aspects (6), (7), and (9) above, based on the intersection state of the travel path with other travel paths and the position of the person, the adjustment unit specifies a collision margin time indicating the time until the person exits the travel path on which the moving body travels and collides with the moving body, and adjusts the timing of the notification at a time when the collision margin time can be ensured.

[0019] (14) The notification method according to one aspect of this invention is such that a computer acquires attribute information regarding a person located within a predetermined distance range from a moving body, classifies the person based on the attribute information, and adjusts the timing of the notification to the person based on the result of the classification.

[0020] (15) The program according to one aspect of this invention causes a computer to execute a process of acquiring attribute information regarding a person located within a predetermined distance range from a moving body, a process of classifying the person based on the attribute information, and a process of adjusting the timing of the notification to the person based on the result of the classification.

Advantages of the Invention

[0021] (1) According to the aspects (1) to (15), based on the classification of a person who may jump out onto the travel path on which the moving body is traveling, etc., it is possible to notify the person with a warning, etc. at an appropriate timing.

[0022] (2) According to the aspect (2), based on the classification by the age of a person who may jump out onto the travel path on which the moving body is traveling, etc., it is possible to notify the person with a warning, etc. at an appropriate timing.

[0023] (3) According to the aspect (3), based on the attributes of the person specified based on the result of recognizing the person, it is possible to notify the person with a warning, etc. at an appropriate timing.

[0024] According to the aspect of (4), based on the moving direction and moving speed of a person who may jump out onto the traveling path of the moving body, etc., it is possible to notify a warning, etc. at an appropriate timing for the person.

[0025] According to the aspect of (5), based on the moving speed of a person who may jump out onto the traveling path of the moving body, etc., it is possible to notify a warning, etc. at an appropriate timing for the person.

[0026] According to the aspect of (6), based on the intersection state of the traveling path of the moving body with other traveling paths, it is possible to notify a warning, etc. at an appropriate timing for a person who may jump out onto the traveling path.

[0027] According to the aspect of (7), based on the intersection state of the traveling path of the moving body with other traveling paths and the position of the person, it is possible to notify a warning, etc. at an appropriate timing for a person who may jump out onto the traveling path.

[0028] According to the aspect of (8), when a person cannot be recognized, it is possible to notify a warning, etc. at the earliest time.

[0029] According to the aspect of (9), when the shape of the traveling path of the moving body cannot be recognized, it is possible to notify a warning, etc. at the earliest time.

[0030] According to the aspect of (10), when the moving speed of a person cannot be detected, it is possible to notify a warning, etc. at the earliest time.

[0031] According to the aspect of (11), it is possible to adjust the notification timing to a time when a collision margin time calculated according to the age of the person can be ensured.

[0032] According to the aspect of (12), it is possible to adjust the notification timing to a time when a collision margin time calculated based on the moving speed of the person can be ensured.

[0033] According to the embodiment of (13), the notification timing can be adjusted to a time when a collision margin calculated based on the intersection of the road the moving object is traveling on with other roads and the position of the person can be secured. [Brief explanation of the drawing]

[0034] [Figure 1] This document outlines a notification system according to an embodiment of the present invention. [Figure 2] This is a schematic diagram of a notification system according to an embodiment of the present invention. [Figure 3] This is a hardware configuration diagram of a notification device according to an embodiment of the present invention. [Figure 4] This is the first figure showing the functional block of a notification device according to an embodiment of the present invention. [Figure 5] This figure shows an example of a notification timing adjustment method according to an embodiment of the present invention. [Figure 6] This figure shows the processing flow of a notification device according to an embodiment of the present invention. [Figure 7] This is a second figure showing the functional block of a notification device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0035] The following describes embodiments of the notification device, notification method, and program of the present invention with reference to the drawings. The notification device is a device installed on a mobile body. A mobile body may include any mobile body that can move on the road surface of a travel path, including three-wheeled or four-wheeled vehicles, two-wheeled vehicles, micromobility devices, etc. In the following description, the mobile body will be assumed to be a four-wheeled vehicle, and the vehicle on which the notification device is installed will be referred to as vehicle M.

[0036] [overview] Figure 1 illustrates the schematic of a notification system according to an embodiment of the present invention. As shown in Figure 1, the notification device 1, which constitutes the notification system 100, is mounted on a vehicle M. When the notification device 1 detects a pedestrian H that may suddenly appear on the road, it sends a notification, such as a warning, to a notification device, such as a terminal 2 carried by the pedestrian H.

[0037] Figure 2 is a schematic diagram of a notification system according to an embodiment of the present invention. As shown in Figure 1, the vehicle M is equipped with at least a camera 10 and a notification device 1 as devices for constituting the notification system 100 of this embodiment. The camera 10 and the notification device 1 are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. Note that the configuration shown in Figure 1 is merely an example, and other configurations may be added.

[0038] Camera 10 is a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). In this embodiment, camera 10 is installed, for example, on the front bumper of vehicle M, and camera 10 takes pictures of the area in front of the vehicle at predetermined intervals such as 10 milliseconds and outputs the captured images to notification device 1. The field of view of camera 10 may be set so that, for example, a person walking or running on the sidewalk 5m ahead on the left side in the direction of travel and a person walking or running on the sidewalk 5m ahead on the right side in the direction of travel are captured. Camera 10 may also be a stereo camera.

[0039] Notification device 1 establishes a communication connection with terminal 2 of a pedestrian H who may step into the roadway. By establishing a communication connection with terminal 2, notification device 1 issues a warning or other notification if pedestrian H, who is carrying terminal 2, is likely to step into front of vehicle M. Notification device 1 may establish a communication connection with terminal 2 directly, or it may establish a communication connection via another device such as server device 3.

[0040] [Notification device] Figure 3 is a hardware configuration diagram of the notification device. As shown in Figure 3, notification device 1 is a computer equipped with hardware such as a CPU (Central Processing Unit) 101, ROM (Read Only Memory) 102, RAM (Random Access Memory) 103, HDD (Hard Disk Drive) 104, and communication module 105. Terminal 2 and server device 3 may have a similar hardware configuration.

[0041] Figure 4 is the first diagram showing the functional blocks of the notification device. The notification device 1 includes, for example, an attribute acquisition unit 11, a travel path recognition unit 12, a movement state estimation unit 13, a position detection unit 14, a classification unit 15, an adjustment unit 16, a notification unit 17, a person recognition unit 18, a control unit 19, and a storage unit 111. The storage unit 111 stores, for example, a trained model. These elements are realized, for example, by a hardware processor such as a CPU 101 executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD 104 or ROM 102, or it may be stored in a removable storage medium (non-transient storage medium) such as a DVD or CD-ROM and installed when the storage medium is mounted in a drive device. The memory unit 111 is implemented by, for example, ROM 102, RAM 103, flash memory, SD card, HDD 104, registers, etc.

[0042] The attribute acquisition unit 11 acquires attribute information about a person (pedestrian H) located within a predetermined distance from the vehicle M. The road recognition unit 12 recognizes the intersection status of the road the vehicle M is traveling on with other roads. Specifically, the road recognition unit 12 recognizes whether the intersection status with other roads within a predetermined distance in the direction of travel of the vehicle M is a crossroads or a straight road without intersections with other roads. The movement state estimation unit 13 determines whether a pedestrian H located within a predetermined distance from the vehicle M is walking or running. The position detection unit 14 detects the position of the pedestrian H with reference to the direction of travel of the vehicle M traveling on the road. The classification unit 15 classifies, based on the attribute information of pedestrians H located within a predetermined distance from vehicle M, whether the age of pedestrian H belongs to the first age range (12 years old or younger), the second age range (13 years old or older and 59 years old or younger), or the third age range (60 years old or older). The adjustment unit 16 adjusts the timing of the warning notification to pedestrian H based on the classification result. The notification unit 17 issues a warning. The person recognition unit 18 recognizes people such as pedestrians H from the captured image. The control unit 19 performs various controls other than those of the other functional units described above. The memory unit 111 stores various information necessary for processing the notification device 1.

[0043] [Methods for adjusting notification timing] Figure 5 shows an example of a method for adjusting the timing of warning notifications. The adjustment unit 16 adjusts different notification timings according to the processing results of the road recognition unit 12, the movement state estimation unit 13, the position detection unit 14, and the classification unit 15. Specifically, the adjustment unit 16 adjusts the notification timing depending on whether the road the vehicle M is traveling on intersects with other roads at a crossroads or a straight road, whether the pedestrian H belongs to the first, second, or third age range, whether the pedestrian H is walking or running, and whether the pedestrian H's position relative to the vehicle M traveling on the road is to the right or left.

[0044] For example, if the recognition result of the road path recognition unit 12 is an intersection, the estimation result of the movement state estimation unit 13 is within the first age range, and the position of pedestrian H is to the right of the direction of travel, the adjustment unit 16 determines that the estimated time to pedestrian H, who is located on the sidewalk, enters the vehicle M's path and collides with the vehicle M is 5.6 seconds, regardless of whether pedestrian H is walking or running. If the recognition result of the road path recognition unit 12 is an intersection, the estimation result of the movement state estimation unit 13 is within the second age range, and the position of pedestrian H is to the right of the direction of travel, the adjustment unit 16 determines that the estimated time to pedestrian H, who is located on the sidewalk, enters the vehicle M's path and collides with the vehicle M is 4.2 seconds, regardless of whether pedestrian H is walking or running. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a crossroads, the estimation result of the movement state estimation unit 13 is within the third age range, and the position of pedestrian H is to the right of the direction of travel, the collision margin time, which indicates the expected estimated time from when pedestrian H is on the sidewalk until he steps onto the road of vehicle M and collides with vehicle M, is 4.3 seconds, regardless of whether pedestrian H is walking or running.

[0045] Furthermore, if the recognition result of the road path recognition unit 12 is a crossroads, the estimation result of the movement state estimation unit 13 is within the first age range, and the position of pedestrian H is on the left side relative to the direction of travel, the adjustment unit 16 identifies a collision margin time of 2.0 seconds, which indicates the estimated time until pedestrian H, located on the sidewalk, steps onto the vehicle M's path and collides with the vehicle M, regardless of whether pedestrian H is walking or running. If the recognition result of the road path recognition unit 12 is a crossroads, the estimation result of the movement state estimation unit 13 is within the second age range, and the position of pedestrian H is on the left side relative to the direction of travel, the adjustment unit 16 identifies a collision margin time of 1.5 seconds, which indicates the estimated time until pedestrian H, located on the sidewalk, steps onto the vehicle M's path and collides with the vehicle M, regardless of whether pedestrian H is walking or running. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a crossroads, the estimation result of the movement state estimation unit 13 is within the third age range, and the position of pedestrian H is to the left of the direction of travel, the collision margin time, which indicates the expected estimated time from when pedestrian H is on the sidewalk until he steps onto the road of vehicle M and collides with vehicle M, is 1.6 seconds, regardless of whether pedestrian H is walking or running.

[0046] For example, if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the first age range, and the position of pedestrian H is to the right of the direction of travel, the adjustment unit 16 identifies a collision margin of 5.6 seconds, which is the estimated time it will take for pedestrian H, who is located on the sidewalk, to enter the vehicle M's path and collide with the vehicle M, regardless of whether pedestrian H is walking or running. If the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the second age range, and the position of pedestrian H is to the right of the direction of travel, the adjustment unit 16 identifies a collision margin of 4.2 seconds, which is the estimated time it will take for pedestrian H, who is located on the sidewalk, to enter the vehicle M's path and collide with the vehicle M, regardless of whether pedestrian H is walking or running. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the third age range, and the position of pedestrian H is to the right of the direction of travel, then regardless of whether pedestrian H is walking or running, the collision margin time, which is the estimated time from when pedestrian H is on the sidewalk until he enters the road path of vehicle M and collides with vehicle M, is 4.3 seconds.

[0047] For example, if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the first age range, the position of pedestrian H is to the left of the direction of travel, and pedestrian H is walking, the adjustment unit 16 specifies a collision margin time of 2.0 seconds, which is the estimated time until pedestrian H, located on the sidewalk, steps onto the road of vehicle M and collides with vehicle M. If the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the first age range, the position of pedestrian H is to the left of the direction of travel, and pedestrian H is running, the adjustment unit 16 specifies a collision margin time of 3.6 seconds, which is the estimated time until pedestrian H, located on the sidewalk, steps onto the road of vehicle M and collides with vehicle M. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the second age range, the position of pedestrian H is to the left of the direction of travel, and pedestrian H is walking, the collision margin time, which indicates the estimated time from when pedestrian H, located on the sidewalk, enters the road of vehicle M and collides with vehicle M, is 1.5 seconds. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the second age range, the position of pedestrian H is to the left of the direction of travel, and pedestrian H is running, the collision margin time, which indicates the estimated time from when pedestrian H, located on the sidewalk, enters the road of vehicle M and collides with vehicle M, is 2.7 seconds. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the third age range, the position of pedestrian H is to the left of the direction of travel, and pedestrian H is walking, the collision margin time, which indicates the estimated time until pedestrian H, located on the sidewalk, steps onto the road of vehicle M and collides with vehicle M, is 1.6 seconds. The adjustment unit 16 determines that if the recognition result of the road path recognition unit 12 is a straight road, the estimation result of the movement state estimation unit 13 is within the third age range, the position of pedestrian H is to the left of the direction of travel, and pedestrian H is running, the collision margin time, which indicates the estimated time until pedestrian H, located on the sidewalk, steps onto the road of vehicle M and collides with vehicle M, is 2.8 seconds.

[0048] As described above, by adjusting the notification timing of the adjustment unit 16, when pedestrian H is on the right side in the direction of travel of vehicle M, the notification timing for issuing warnings, etc., is set to be earlier compared to when pedestrian H is on the left side in the direction of travel of vehicle M. A person such as pedestrian H on the left side in the direction of travel of vehicle M is likely to be within the field of view of vehicle M's camera 10, and is likely to be sufficiently recognized by the recognition results of camera 10. Therefore, the notification device 1 according to this embodiment can issue warnings, etc., with higher priority to pedestrian H, who is unlikely to be within the field of view of camera 10. Furthermore, as described above, by adjusting the notification timing of the adjustment unit 16, when pedestrian H is in the first age range (children aged 13 or younger), the notification timing for issuing warnings, etc., is set to be earlier compared to other age ranges. Therefore, the notification device 1 according to this embodiment can issue warnings, etc., with higher priority when pedestrian H's age range is within the age range of children.

[0049] The memory unit 111 may store collision margins in a data table stored in the memory unit 111, linked to combinations of information such as the age range of pedestrian H (first age range to third age range), the intersection state of the road on which vehicle M is traveling with other roads (crossroads, straight road), the position information of pedestrian H relative to the direction of travel of vehicle M (right side, left side), and the movement state of pedestrian H (walking, running). The adjustment unit 16 may obtain the collision margins recorded in the data table, linked to combinations of information such as the age range of pedestrian H (first age range to third age range), the intersection state of the road on which vehicle M is traveling with other roads (crossroads, straight road), the position information of pedestrian H relative to the direction of travel of vehicle M (right side, left side), and the movement state of pedestrian H (walking, running), and specify the relevant time.

[0050] [Attribute detection process] The attribute acquisition unit 11 obtains the age of pedestrian H from the attribute information of pedestrian H included in the personal attribute information acquired from terminal 2 or server device 3. The attribute acquisition unit 11 may also obtain the age recognized by the person recognition unit 18 from the captured image.

[0051] [Road Recognition Processing] The road recognition unit 12 generates a learning model by pre-training on images captured by the camera 10 in front of the road. Using this learning model, the road recognition unit 12 recognizes the intersection state of the road traveled by the vehicle M with other roads and determines whether it is a crossroads or a straight road. The learning model is generated by a learning device that uses a large number of relationships between captured images and the correct information on the intersection state of the road and other roads shown in the captured images through machine learning. The storage unit 111 of the notification device 1 stores the learning model generated by the learning device. For example, the road recognition unit 12 of the notification device 1 inputs the captured image into a neural network generated using the learning model, and as a result determines whether the state of the road and other roads shown in the captured image is a crossroads. If it is determined to be a crossroads, the road recognition unit 12 outputs information indicating that it is a crossroads to the adjustment unit 16; otherwise, it outputs information indicating that it is a straight road to the adjustment unit 16. The road recognition unit 12 may use the captured image and the learned model to clearly determine whether the road ahead is a crossroads or a straight road.

[0052] Alternatively, the road recognition unit 12 may recognize whether the road ahead at a predetermined distance, such as 30m, is a straight road or a crossroads, without using a learning model. For example, the control unit 19 of the vehicle M's notification device 1 acquires position information calculated by the notification device 1 or a position detection function separately provided in the vehicle M, which receives signals from GNSS satellites. The control unit 19 periodically accesses the server device 3 at predetermined intervals, such as every second, and sends an information acquisition request that includes the position information. Based on the position information included in multiple consecutive information acquisition requests, the server device 3 identifies the position information and the direction of travel of the vehicle M, and identifies information about the state of the road (crossroads or straight road) within a predetermined distance in the direction of travel from the vehicle M's position information. The server device 3 sends the information about the state of the road ahead to the notification device 1 that sent the information acquisition request. The road recognition unit 12 may then recognize the state of the road ahead based on the information received from the server device 3.

[0053] The road recognition unit 12 may acquire information indicating the condition of the road ahead from the information received from the terminal 2. Alternatively, the road recognition unit 12 may acquire information indicating the condition of the road located at a predetermined distance ahead in the direction of travel of the vehicle M from a navigation device mounted on the vehicle M that displays a map and the current location on that map.

[0054] [Movement state estimation process] The movement state estimation unit 13 obtains the movement speed and direction of pedestrian H from information contained in personal attribute information acquired from terminal 2 or server device 3. The movement state estimation unit 13 may also calculate the movement direction and movement speed of pedestrian H based on the position information of people captured in a series of consecutive images.

[0055] [Location detection process] The position detection unit 14 determines whether pedestrian H is located on the right or left side of the direction of travel of vehicle M based on the position information included in the personal attribute information. The position detection unit 14 may also determine whether pedestrian H is located on the right or left side of the direction of travel of vehicle M based on the position of a person in the captured image. The position detection unit 14 recognizes the road from the captured image and determines that the person is located on the left side if the person is located on the left side of the road, and determines that the person is located on the right side if the person is recognized on the right side of the road.

[0056] [First Embodiment] Figure 6 is a diagram showing the processing flow of the notification device. Next, the processing flow of notification device 1 will be explained step by step. Terminal 2 has a dedicated application program installed in advance for connecting to the notification system 100. When pedestrian H runs this program on terminal 2, terminal 2 is equipped with a notification function. The following processing of terminal 2 is the processing of this notification function. Terminal 2 detects location information. The location information may be location information calculated by the location detection function equipped on terminal 2 after receiving signals from GNSS satellites. The location information may be information indicating the latitude, longitude, and altitude of terminal 2. Terminal 2 may calculate the speed and direction of movement of terminal 2 from these multiple location information based on the transition of location information over time. Terminal 2 obtains the gender and age of pedestrian H, which are set in advance and stored in terminal 2. Terminal 2 generates personal attribute information including a terminal ID to identify terminal 2, location information, speed, direction of movement, and the gender and age of pedestrian H. Terminal 2 transmits the personal attribute information to server device 3. Server device 3 stores the personal attribute information.

[0057] The control unit 19 of the notification device 1 of vehicle M acquires location information calculated by the notification device 1 and a location detection function separately provided in vehicle M that receives signals from GNSS (Global Navigation Satellite System) satellites. The control unit 19 periodically accesses the server device 3 at predetermined intervals, such as every second, and sends an information acquisition request that includes location information. Based on the location information included in the information acquisition request, the server device 3 acquires personal attribute information that includes location information within a predetermined distance from that location information. The predetermined distance may be, for example, 100 meters or 200 meters. The server device 3 sends the personal attribute information to the notification device 1 that sent the information acquisition request. The notification device 1 receives the personal attribute information.

[0058] The notification device 1 may obtain personal attribute information directly from the terminal 2 without going through the server device 3. For example, the control unit 19 of the notification device 1 transmits an information acquisition request at predetermined intervals. This transmission is broadcast by a wireless signal via the communication module 105. A terminal 2 of a pedestrian H located within a predetermined distance (e.g., 100 meters or 200 meters) from the vehicle M can receive the information acquisition request signal. Upon receiving the information acquisition request signal, the terminal 2 transmits the generated personal attribute information to the notification device 1 in the vehicle M. The notification device 1 can then acquire the personal attribute information.

[0059] The control unit 19 acquires captured images from the camera 10 at predetermined short intervals, such as every few tens of milliseconds. The control unit 19 outputs the captured images to the person recognition unit 18. The person recognition unit 18 recognizes a person from the captured image. The recognition of a person in the image may be, for example, a process to recognize a person's face. The process to recognize a face may use known techniques. The person recognition unit 18 outputs information to the adjustment unit 16 indicating whether or not a person was recognized from the captured image.

[0060] The control unit 19 of the notification device 1 determines whether or not it was able to acquire personal attribute information (step S101). The control unit 19 may perform a process to determine whether it was able to acquire personal attribute information within a predetermined time from the time when a person's face was recognized in the captured image. If the control unit 19 was able to acquire personal attribute information, it decided to adjust the notification timing. If the control unit 19 could not acquire personal attribute information, it decided to send a notification without adjusting the notification timing.

[0061] When adjusting the notification timing, the attribute acquisition unit 11 acquires personal attribute information from the control unit 19. The attribute acquisition unit 11 acquires the age of pedestrian H included in the personal attribute information. The attribute acquisition unit 11 outputs the age of pedestrian H to the classification unit 15. The classification unit 15 classifies the age range of pedestrian H based on the age of pedestrian H (step S102). As a result, the classification unit 15 determines whether the age of pedestrian H belongs to the first age range, the second age range, or the third age range. The classification unit 15 outputs the age range of pedestrian H to the adjustment unit 16.

[0062] The road recognition unit 12 acquires images from the camera 10. The road recognition unit 12 uses the captured images and a learning model to input the captured images into a neural network or the like using the learning model. As a result, the road recognition unit 12 outputs information on whether the intersection of the road the vehicle M is traveling on with other roads is a crossroads or a straight road. In this output process, the road recognition unit 12 determines, based on the captured images, whether it was able to recognize whether the intersection of the road the vehicle M is traveling on with other roads is a crossroads or a straight road (step S103). If the road recognition unit 12 outputs information indicating a crossroads or a straight road as the intersection state of the road with other roads, it determines that it was recognized and outputs the recognition result to the adjustment unit 16.

[0063] The position detection unit 14 calculates the direction of travel vector of the vehicle M in three-dimensional space based on the transition of the vehicle M's position information. Based on the direction of travel vector relative to the vehicle M's current position and the position information of the pedestrian H, the position detection unit 14 determines whether the pedestrian H is located on the right or left side of the direction of travel (step S104). If the pedestrian H is located on the right side of the direction of travel, the position detection unit 14 outputs the information for the right side, and if the pedestrian H is located on the left side of the direction of travel, the information for the left side, as the position information of the pedestrian H to the adjustment unit 16.

[0064] The movement state estimation unit 13 determines whether it has acquired the movement speed of pedestrian H (step S105). The movement state estimation unit 13 may detect and acquire the movement speed from personal attribute information. The movement state estimation unit 13 may calculate the movement speed based on the position information included in the personal attribute information acquired continuously from terminal 2 at predetermined intervals. The movement state estimation unit 13 may also estimate the movement speed using the transition of the position of a person in the captured image and techniques such as optical flow. If the movement state estimation unit 13 has acquired the movement speed of pedestrian H, it outputs that movement speed to the adjustment unit 16. If the personal attribute information includes the direction of movement, the movement state estimation unit 13 outputs the direction of movement information to the adjustment unit 16. The movement state estimation unit 13 may calculate the direction of movement in the same way as the movement speed, or it may estimate the direction of movement of pedestrian H using the captured image and optical flow techniques.

[0065] The adjustment unit 16 determines whether pedestrian H is running or walking based on the speed of movement (step S106). For example, the adjustment unit 16 may determine that pedestrian H is walking if the speed of movement of pedestrian H is less than a predetermined speed such as 1 meter per second, and determine that pedestrian H is running if the speed is 1 meter per second or more.

[0066] The adjustment unit 16 reads and identifies the time to collision (TTC) recorded in the timing list stored in the memory unit 111, linked to the following information: the age range of the pedestrian H (first age range to third age range), the intersection status of the road on which the vehicle M is traveling with other roads (crossroads, straight road), the position information of the pedestrian H relative to the direction of travel of the vehicle M (right side, left side), and the movement state of the pedestrian H (walking, running) (step S107). The adjustment unit 16 outputs the acquired time to collision and terminal ID to the notification unit 17. This process is one aspect of the process in which the adjustment unit 16 of the notification device 1 identifies the time to collision, which indicates the time from when the pedestrian H enters the road on which the vehicle M is traveling until it collides with the vehicle M, and adjusts the notification timing to a time when the said time to collision can be secured.

[0067] The notification unit 17 determines the predicted collision time based on the current position of vehicle M, the vehicle M's speed, the vehicle M's movement vector, the pedestrian H's current position, and the pedestrian H's movement speed and movement vector (step S108). The calculation of the predicted collision time can be done using known techniques with this information. The notification unit 17 transmits a warning signal including the terminal ID at a notification timing obtained by subtracting the collision buffer time from the predicted collision time (step S109). The notification unit 17 may transmit the warning signal immediately if the time from the current time to the predicted collision time is shorter than the collision buffer time.

[0068] If the time from the current time to the predicted collision time is longer than the collision buffer time, the notification unit 17 may send a warning signal at a notification timing calculated by subtracting the time obtained by adding the collision buffer time to the predicted collision time from the predicted collision time, taking into account the time α from when the warning signal is sent until the terminal 2 receives the signal and emits a warning sound. Alternatively, even if the time from the current time to the predicted collision time is longer than the collision buffer time, the notification unit 17 may immediately send a warning signal to the terminal 2 that includes the predicted collision time and the collision buffer time. In this case, the terminal 2 may calculate the notification timing of the warning by subtracting the collision buffer time from the predicted collision time and perform the process of sounding a warning sound at that notification timing.

[0069] In the above-described process, if personal attribute information cannot be obtained in step S101, if the intersection of the road on which vehicle M is traveling with other roads cannot be recognized in step S103, or if the movement speed of pedestrian H cannot be detected in step S106 and it is not possible to determine whether pedestrian H is running or walking, the adjustment unit 16 obtains the collision margin time recorded in the data table to calculate the earliest predetermined notification timing (step S110). The adjustment unit 16 uses this collision margin time to transmit a warning signal in the same manner as above. The collision margin time is greater than any of the above times. This allows the warning signal to be transmitted as quickly as possible.

[0070] According to the above process, the notification device 1 can notify pedestrian H approaching vehicle M of a warning at a time before the estimated collision time at which collision between vehicle M and pedestrian H is estimated to occur, using a collision buffer time. Furthermore, according to the above process, the notification device 1 can set a collision buffer time according to the age range to which pedestrian H belongs and the position relative to the direction of travel of vehicle M, thereby eliminating the possibility of warnings sounding unnecessarily early or late, and notifying pedestrian H of a warning at an appropriate time. In addition, according to the above process, since the warning signal is sent to terminal 2 equipped with a dedicated application program, it is not necessary to send a warning signal to all pedestrian H, and warnings can be notified only to pedestrian H (children, disabled persons, etc.) who have installed the dedicated application on their terminal.

[0071] [First variation] In the above-described process, terminal 2 may perform actions such as activating a vibrator or outputting a warning message to a display, instead of, or in combination with, sounding a warning. Terminal 2 may be a dedicated terminal 2 that can be used with a notification system 100 other than a smartphone, or it may be, for example, a small wireless earphone or smart glasses.

[0072] [Second variation] Furthermore, in the process described above, the adjustment unit 16 identifies the collision margin from a data table based on the current position of the vehicle M, the vehicle M's speed, the vehicle M's direction vector, the pedestrian H's current position, and the pedestrian H's movement speed and direction vector. However, the adjustment unit 16 may also calculate the collision margin based on the current position of the vehicle M, the vehicle M's speed, the vehicle M's direction vector, the pedestrian H's current position, and the pedestrian H's movement speed and direction vector. The collision margin will differ depending on the pedestrian's position. Therefore, a collision margin can be set according to the pedestrian's position, and a warning can be issued.

[0073] [Third variation] Figure 7 is a second diagram showing the functional blocks of the notification device. The above process was explained using an example where age, which is attribute information of a person, is included in the personal attribute information. However, the notification device 1 may also include an attribute estimation unit 112, which estimates which of the first to third age ranges the person in the captured image belongs to based on the recognition result (estimated age) of the person in the captured image.

[0074] [Fourth variation] In the process described above, the adjustment unit 16 determines the notification timing without using the direction of movement of the pedestrian H. However, the adjustment unit 16 may further determine the notification timing using the direction of movement. In this case, the adjustment unit 16 determines whether the direction of movement of the pedestrian H intersects with the direction of travel of the vehicle traveling on the road. Only when the direction of movement of the pedestrian H intersects with the direction of travel of the vehicle traveling on the road, the adjustment unit 16 obtains the collision margin time recorded in the timing list stored in the storage unit 111, linked to the following information: the age range of the pedestrian H (first age range to third age range), the intersection state of the road on which the vehicle M is traveling with other roads (crossroads, straight road), the position information of the pedestrian H relative to the direction of travel of the vehicle M (right side, left side), and the movement state of the pedestrian H (walking, running). If the direction of movement of the pedestrian H does not intersect with the direction of travel of the vehicle traveling on the road, the adjustment unit 16 may determine not to send a notification and stop sending the warning signal. The adjustment unit 16 may calculate the collision margin time using a predetermined calculation formula based on the age range of the pedestrian H (first age range to third age range), the intersection state of the road on which the vehicle M is traveling with other roads (crossroads, straight road), the position information of the pedestrian H relative to the direction of travel of the vehicle M (right side, left side), the movement state of the pedestrian H (walking, running), and the direction of movement of the pedestrian H.

[0075] [Fifth variation] In the process described above, the adjustment unit 16 determines the notification timing based on the intersection state of the road on which the vehicle M is traveling with other roads and the position of the pedestrian H. However, the adjustment unit 16 may also determine the notification timing without using the position of the pedestrian H. In this case, the adjustment unit 16 may obtain the collision margin time recorded in the timing list stored in the storage unit 111, linked to the following information: the age range of the pedestrian H (first age range to third age range), the intersection state of the road on which the vehicle M is traveling with other roads (crossroads, straight road), and the movement state of the pedestrian H (walking, running).

[0076] [Sixth variation] In the process described above, the notification device 1 identifies the location information of terminal 2 from personal attribute information received from terminal 2. However, the location information of terminal 2 may also be detected by the notification device 1. For example, the communication module 105 of the notification device 1 may detect the direction of arrival and signal strength of the wireless signal transmitted from terminal 2, and detect the location information of terminal 2 based on that information. Alternatively, the communication module 105 of the notification device 1 may detect the location information of terminal 2 based on the wireless signal transmitted from terminal 2 using other known methods. The person recognition unit 18 of the notification device 1 recognizes the face of pedestrian H at the position in the image corresponding to the detected location information of terminal 2, and the movement state estimation unit 13 may estimate the movement state (direction and speed of movement) of pedestrian H based on the movement information of the face captured in multiple captured images, or the attribute estimation unit 112 may estimate the age of the person based on the face information in the captured images.

[0077] [Seventh variation] In the process described above, notification device 1 transmits a warning signal to terminal 2 carried by the pedestrian. However, notification device 1 may also transmit a warning signal to notification destination device 2 located near the pedestrian. Notification destination device 2 may be, for example, an advertising display set up around the pedestrian. In this case, the control unit 19 of notification device 1 transmits a destination information request, including its own location information, to server device 3 at predetermined intervals. Server device 3 obtains the location information of the notification destination device and obtains the network address information of notification destination device 2 located at a predetermined distance, such as 100m, from its location information. The network address information is, for example, the address information of a communication network for transmitting information to the notification destination device. Server device 3 transmits the network address information of the communication destination device to notification device 1. Notification device 1 obtains the network address information of notification destination device 2. Alternatively, notification device 1 may obtain the network address of a notification destination device located within a predetermined distance range relative to the location information of terminal 2, or the notification destination device closest to the location information of vehicle M, from a data table that associates the location information and network address of notification destination devices stored in the device's memory unit 111 in advance. When notification device 1 transmits a warning signal in the above process, it uses the acquired network address information of the notification destination device to transmit the warning signal to the notification destination device. The notification destination device that receives the warning signal may display warning information or output a warning sound. This makes it possible to display warning information or output a warning sound on a notification destination device such as an advertising display near pedestrian H. Notification device 1 may also transmit the warning signal to terminal 2 and other notification destination devices simultaneously. This has the effect of making it easier to convey the warning to pedestrian H.

[0078] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes the program stored in the memory device, Obtain attribute information about a person located within a predetermined distance from a moving object. Based on the attribute information, classify the person, Based on the results of the above classification, the timing of notification to the person will be adjusted. A notification device configured in such a way.

[0079] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0080] 1 Notification device 2 terminals 3 Server equipment 10 Cameras 11 Attribute acquisition part 12. Road Recognition Unit 13. Movement state estimation unit 14 Position detection unit 15 Classification section 16 Adjustment part 17 Notification Department 18 Person recognition section 19 Control Unit 111 Storage section 112 Attribute estimation section H Pedestrian M Vehicle

Claims

1. An attribute acquisition unit that acquires attribute information about a person located within a predetermined distance from a moving object, A classification unit that classifies the person based on the attribute information, An adjustment unit identifies a collision margin time, which is the time it takes for the person to enter the path on which the moving object is traveling and to collide with the moving object, and adjusts the timing of notification to the person at a time when the collision margin time can be secured, based on the classification result and the collision margin time. Equipped with, The adjustment unit adjusts the timing so that it can notify as quickly as possible if it cannot obtain the attribute information within a predetermined time from the time the person is recognized. Notification device.

2. The attribute information includes at least information regarding the age of the person, The classification unit outputs the result of the classification regarding the age of the person based on the age. The notification device according to claim 1.

3. A person recognition unit that recognizes the aforementioned person, An attribute estimation unit that estimates attribute information of the person based on the results of the recognition of the person, The notification device according to claim 1 or 2, comprising:

4. The system includes a movement state estimation unit that estimates the direction and speed of movement of the person, The adjustment unit further adjusts the timing of the notification to the person based on the person's direction of movement and speed of movement. A notification device according to any one of claims 1 to 3.

5. The unit comprises a movement state estimation unit that detects the movement speed of the person, The adjustment unit further adjusts the timing of the notification to the person based on the person's movement speed. A notification device according to any one of claims 1 to 3.

6. A road recognition unit recognizes the intersection state of the road on which the moving object is traveling with other roads, Equipped with, The adjustment unit adjusts the timing of the notification to the person based on the interaction state. A notification device according to any one of claims 1 to 5.

7. A road recognition unit recognizes the intersection state of the road on which the moving object is traveling with other roads, Equipped with, The adjustment unit adjusts the timing of the notification to the person based on the interaction state and the position of the person. A notification device according to any one of claims 1 to 5.

8. The adjustment unit adjusts the timing so that it can notify the driver as quickly as possible if it cannot recognize the intersection of the aforementioned road with other roads. The notification device according to claim 7.

9. An attribute acquisition unit that acquires attribute information relating to a person located within a predetermined distance from a moving object, A classification unit that classifies the person based on the attribute information, A road recognition unit recognizes the intersection state of the road on which the moving object is traveling with other roads, An adjustment unit determines a collision margin time, which is the time from when the person enters the path on which the moving object is traveling until it collides with the moving object, based on the aforementioned intersection state and the position of the person, and adjusts the timing of notification to the person at a time when the collision margin time can be secured, based on the classification result and the collision margin time. Equipped with, The adjustment unit adjusts the timing so that it can notify the driver as quickly as possible if it cannot recognize the intersection of the aforementioned road with another road. Notification device.

10. The adjustment unit adjusts the timing so that it can notify as quickly as possible if it cannot detect the person's movement speed. The notification device according to claim 5.

11. An attribute acquisition unit that acquires attribute information relating to a person located within a predetermined distance from a moving object, A classification unit that classifies the person based on the attribute information, A movement state estimation unit that detects the movement speed of the person, The system includes an adjustment unit that, based on the speed of movement of the person, identifies a collision margin time indicating the time from when the person enters the path on which the moving object is traveling until it collides with the moving object, and adjusts the timing of notification to the person at a time when the collision margin time can be secured, based on the classification result and the collision margin time. The adjustment unit adjusts the timing so that it can notify as quickly as possible if it cannot detect the speed of the person's movement. Notification device.

12. The adjustment unit identifies a collision buffer time, which is the time from when a person enters the path traveled by the moving object until they collide with the moving object, according to the multiple age groups indicated by the classification of the person, and adjusts the timing of the notification to a time when the collision buffer time can be secured. A notification device according to any one of claims 1 to 11.

13. The adjustment unit, based on the classification of the person's movement speed, identifies a collision margin time, which is the time it takes for the person to enter the path on which the moving object is traveling and collide with the moving object, and adjusts the timing of the notification to a time when the collision margin time can be secured. A notification device according to any one of claims 1 to 12.

14. The adjustment unit determines a collision buffer time, which is the time it takes for the person to enter the path on which the moving object is traveling and collide with the moving object, based on the intersection of the travel path with other travel paths and the position of the person, and adjusts the timing of the notification to a time when the collision buffer time can be secured. A notification device according to any one of claims 6, 7, or 8.

15. Computers Obtain attribute information about a person located within a predetermined distance from a moving object. Based on the attribute information, classify the person, The collision margin time, which indicates the time from when the person enters the path on which the moving object is traveling until it collides with the moving object, is identified, and based on the classification result and the collision margin time, the timing of notification to the person is adjusted to a time when the collision margin time can be secured. If attribute information cannot be obtained within a predetermined time from the time the person is recognized, the timing will be adjusted to ensure that notification is given as soon as possible. Notification method.

16. On the computer, A process for acquiring attribute information about a person located within a predetermined distance from a moving object, A process for classifying the person based on the attribute information, The process involves identifying a collision buffer time, which is the time it takes for the person to enter the path on which the moving object is traveling and to collide with the moving object, and adjusting the timing of notification to the person at a time when the collision buffer time can be secured, based on the classification result and the collision buffer time. If attribute information cannot be obtained within a predetermined time from the time the person is recognized, the process adjusts the timing to ensure that notification is given as soon as possible. A program that executes the command.

Images