Notification device, notification method, notification program, and recording medium

Abstract

To provide a notification device that, in giving guidance using a feature visible from on the road, when the feature is not visible from on the road, can prevent a user from being annoyed.SOLUTION: A guidance device comprises: an image acquisition unit that acquires a surrounding image that is an image of the surroundings of a movable body photographed from the movable body; and a notification unit that, when in a first area where a predetermined feature is predicted to appear in the surrounding image, which is determined on the basis of map information including positional information of the predetermined feature and the position of the movable body at the time point when the surrounding image is photographed, an object different from the predetermined feature appears, notifies a user of information on the type of the object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an alarm device, an alarm method, an alarm program, and a recording medium. [Background technology]

[0002] Conventionally, navigation devices that display a searched route and provide route guidance are known. For example, Patent Document 1 discloses a navigation device that displays roads at points or sections where scenic objects can be visually recognized in a display format different from that of other roads. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-107025 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, when route guidance is provided by a navigation device such as that described above, one issue is that when actually driving along a road at a point or section where a scenic object is visible, the scenic object may not be visible because it is obstructed by, for example, another vehicle or a newly constructed building, which may confuse the user.

[0005] The present invention has been made in consideration of the above-mentioned points, and one of its objects is to provide an alarm device that can prevent a user from becoming confused when, for example, guidance is provided using features that are visible from the road, and the features cannot be seen from the road. [Means for solving the problem]

[0006] The invention described in claim 1 is characterized in that it comprises an image acquisition unit that acquires a surrounding image, which is an image of the surroundings of a moving body photographed from the moving body, and an alarm unit that, when an object different from a specified feature appears in a first area in the surrounding image in which the specified feature is expected to appear, which is determined based on map information including position information of the specified feature and the position of the moving body at the time the surrounding image is photographed, alarm unit that alarms information regarding the type of the object.

[0007] The invention described in claim 10 is an alarm method performed by an alarm device, comprising: an image acquisition step in which an image acquisition unit acquires a surrounding image, which is an image of the surroundings of a moving body photographed from the moving body; and an alarm step in which the alarm unit notifies information regarding the type of object when an object different from a specified feature appears in a first area in the surrounding image in which the specified feature is expected to appear, which is determined based on map information including position information of the specified feature and the position of the moving body at the time the surrounding image was photographed.

[0008] The invention described in claim 11 is an alarm program executed by an alarm device having a computer, which causes the computer to execute an image acquisition step of acquiring a surrounding image, which is an image of the surroundings of a moving body photographed from the moving body, and an alarm step of notifying information regarding the type of object when an object other than a specified feature appears in a first area in the surrounding image in which the specified feature is expected to appear, which is determined based on map information including position information of the specified feature and the position of the moving body at the time the surrounding image was photographed. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing an overview of a notification system including a notification device according to a first embodiment. [Diagram 2] 1 is a diagram showing a front seat portion of a vehicle in which a notification device according to a first embodiment is mounted. [Diagram 3] 1 is a block diagram showing a configuration of a notification device according to a first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of data included in map information according to the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of data included in map information according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of a guide map according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of a traveling state of a moving object according to the first embodiment. [Figure 8] FIG. 4 is a diagram showing an example of a surrounding image used for guidance according to the first embodiment. [Figure 9] FIG. 4 is a diagram showing another example of the traveling state of the moving body according to the first embodiment. [Figure 10] FIG. 11 is a diagram showing another example of the surrounding image used for guidance according to the first embodiment. [Figure 11] FIG. 4 is a diagram showing another example of the traveling state of the moving body according to the first embodiment. [Figure 12] 4 is a flowchart showing an example of a routine executed by the notification device of the first embodiment. [Figure 13] 5 is a flowchart showing an example of a subroutine executed by the notification device of the first embodiment. [Figure 14] FIG. 11 is a diagram showing an example of data included in map information used for route guidance in the second embodiment. [Figure 15] FIG. 11 is a diagram showing an example of data included in map information used for route guidance in the second embodiment. [Figure 16] FIG. 11 is a diagram showing an example of a route guidance map according to a second embodiment. [Figure 17] FIG. 11 is a diagram showing an example of a driving situation during route guidance according to the second embodiment. [Figure 18] FIG. 11 is a diagram showing another example of a driving situation during route guidance according to the second embodiment. [Figure 19] 10 is a flowchart showing an example of a routine executed by a notification device of a second embodiment. [Figure 20] 10 is a flowchart showing an example of a subroutine executed by the notification device of the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description and accompanying drawings, substantially the same or equivalent parts are designated by the same reference numerals. EXAMPLES

[0011] A configuration of a feature guide device 10 as a notification device according to a first embodiment will be described with reference to FIGS. 1 to 5. FIG.

[0012] Fig. 1 shows an information exchange system 100 including a feature guide device 10. As shown in Fig. 1, the information exchange system 100 is configured to include a feature guide device 10 mounted on a vehicle M, and a server S. The feature guide device 10 transmits and receives data to and from the server S via a network NW.

[0013] The feature guide device 10 provides feature guidance by presenting information about features around the road to a user aboard the vehicle M while the vehicle M is traveling on the road.

[0014] For example, the server S holds map information including information about features. For example, the feature guide device 10 transmits information about features around a road obtained while the vehicle M is traveling to the server S. The server S is configured to be able to update the information about features in the map information in accordance with the information transmitted from the feature guide device 10.

[0015] In this specification, "geographical features" refers to all natural or man-made objects such as trees, rivers, houses, roads, railways, etc. Typically, such geographical features are famous landmarks, and include natural objects such as mountains and rocks, as well as buildings such as temples, shrines, and high-rise buildings.

[0016] 2 is a diagram showing a front seat portion of a vehicle M equipped with a feature guide device 10 according to the first embodiment. The feature guide device 10 is disposed in a center console in a dashboard DB of the vehicle M.

[0017] The feature guide device 10 is communicatively connected to a microphone 11, a touch panel display 13, a speaker 15, and a camera 17 installed in the vehicle M. The feature guide device 10 is also connected to a GPS receiver 19. The microphone 11 and the touch panel display 13 are disposed, for example, on a dashboard DB. The speaker 15 is provided, for example, on the A-pillar AP or on the interior side of a door.

[0018] The microphone 11 is an acoustic device that converts voice into an electrical signal. The microphone 11 is installed at a position between the driver's seat and the passenger seat on the dashboard DB, and transmits the voice of a user sitting in the front seat of the vehicle M to the feature guide device 10.

[0019] The touch panel display 13 is configured by combining a display that displays a screen based on the control of the feature guide device 10, and a touch panel that transmits signals to the feature guide device 10 representing input operations by the user.

[0020] The speaker 15 is a device that outputs sound based on a signal transmitted from the feature guide device 10 .

[0021] The feature guide device 10 accepts operational input from a user via a microphone 11 or a touch panel display 13. The feature guide device 10 displays information about the current position of the vehicle M and features on the touch panel display 13. The feature guide device 10 also presents the information about features to the user by voice from a speaker 15.

[0022] The camera 17 is disposed, for example, on the dashboard DB. The camera 17 is oriented so as to be able to capture an ambient image, which is an image of the surroundings of the vehicle M. The camera 17 has an imaging direction in front of the vehicle M, i.e., the traveling direction of the vehicle M. When the traveling direction of the vehicle M is the forward direction, the camera 17 is disposed so as to be able to capture an ambient image in a range including the forward direction, the right front direction, and the left front direction. Therefore, the camera 17 can capture an image within the field of view of a user sitting in the front seat of the vehicle M when looking forward.

[0023] The camera 17 is not limited to being disposed on the dashboard DB, and may be disposed anywhere as long as it can capture an image of the surroundings in front of the vehicle M. For example, the camera 17 may be disposed behind the rear-view mirror RM, that is, on the surface facing the windshield FG. The camera 17 may also be disposed on the outer surface (exterior surface) of the vehicle M, for example, on the hood or the front bumper.

[0024] The GPS receiver 19 is disposed on, for example, the dashboard DB, receives signals (GPS signals) from GPS (Global Positioning System) satellites, and transmits them to the feature guide device 10.

[0025] 3 is a block diagram showing the configuration of the feature guide device 10. As shown in FIG. 3, the feature guide device 10 is configured by connecting each unit via a system bus 21.

[0026] The input unit 23 is an interface for acquiring data from devices provided in the vehicle M. The input unit 23 is connected to the microphone 11 and the touch panel of the touch panel display 13, which are devices provided in the vehicle M. The feature guide device 10 accepts input from the microphone 11 or the touch panel via the input unit 23.

[0027] 3, the input unit 23 is connected to the camera 17. The feature guide device 10 acquires an image of the surroundings of the vehicle M captured by the camera 17 via the input unit 23.

[0028] The input unit 23 is also connected to the GPS receiver 19 and the sensor 24. The sensor 24 is a group of sensors including, for example, an acceleration sensor, an angular velocity sensor, and a distance sensor. The feature guide device 10 acquires data such as the position and speed of the vehicle M from the GPS receiver 19 and the sensor 24 via the input unit 23. For example, the data such as the position and speed of the vehicle M is used to acquire the current position for feature guidance by the feature guide device 10.

[0029] The output unit 25 is connected to the display and speaker 15 of the touch panel display 13 provided in the vehicle M. The output unit 25 is an interface for supplying various types of information to the display and speaker 15.

[0030] For example, the feature guide device 10 supplies images of messages, maps, and the like to be presented to the user to a display via the output unit 25. For example, the feature guide device 10 supplies audio of guidance information to the speaker 15 via the output unit 25 to provide guidance regarding features to the user.

[0031] The storage unit 27 is a storage device configured, for example, by a hard disk device, a solid state drive (SSD), a flash memory, etc. The storage unit 27 stores various programs executed in the feature guide device 10. Note that the various programs may be acquired, for example, from the server S or another server device via a network, or may be recorded on a recording medium and read via various drive devices.

[0032] The storage unit 27 also stores a map information database (hereinafter referred to as map information DB) 27A. The map information DB 27A stores map information used for feature guidance by the feature guidance device 10. The map information also includes information on features around roads used for feature guidance. For example, the map information includes location information of the features and images of the features taken from the road.

[0033] Specifically, the map information stores road sections (hereinafter also referred to as road links) included in the map in association with features visible from the road sections.

[0034] 4 is a diagram showing an example of linking data between road sections and features included in map information stored in the map information DB27A as table TB1. In table TB1, each of RL1 to RL4 in the "Link ID" column indicates each of a plurality of road links. In the "Visible feature" column, features FT1 to FT4 that are visible from a vehicle traveling on each road section are associated with each road section.

[0035] For example, as shown in FIG. 4, there are road links to which a plurality of features are associated (RL3), and there are road links to which no features are associated (RL4).

[0036] Fig. 5 is a diagram showing an example of location information of features included in map information as a table TB2. As shown in Fig. 5, location information is associated with each visible feature. The location information may be global coordinates such as latitude and longitude, or local coordinates such as coordinates relative to a reference position.

[0037] In addition, the map information stored in the map information DB 27A stores images of features captured from the road section in association with each feature associated with each road section. In other words, the map information includes images of the features associated with the road section. The images are used, for example, to determine whether or not a surrounding image captured from the vehicle M traveling on a road includes a feature that is visible from the road section being traveled.

[0038] The storage unit 27 stores an image database (hereinafter referred to as image DB) 27B, which is a database that collects images of objects that may be visible from a moving body during movement. For example, the image DB 27B stores image data of objects such as various types of vehicles and buildings with various appearances that may be obstacles when visually recognizing features from a moving body, photographed from various angles.

[0039] For example, the image data stored in the image DB 27B is used to identify the type of an object that is captured in a surrounding image captured from the vehicle M and that obstructs the visibility of a feature. That is, the image DB 27B stores sample images of obstacles.

[0040] The control unit 29 is a computer that includes a CPU (Central Processing Unit) 29A, a ROM (Read Only Memory) 29B, and a RAM (Random Access Memory) 29C, and controls the operation of the feature guide device 10. The CPU 29A reads out and executes various programs stored in the ROM 29B and the storage unit 27, thereby implementing various functions.

[0041] For example, the control unit 29 reads out and executes a feature guidance program serving as the notification program of this embodiment from the storage unit 27. The feature guidance program proceeds, for example, as follows.

[0042] First, the control unit 29 searches for whether or not there is a feature visible from the road on which the vehicle M is traveling, by referring to the map information stored in the map information DB.

[0043] The control unit 29 acquires an image of the surroundings taken from the vehicle M by the camera 17 if a visible feature is present.

[0044] The control unit 29 specifies an expected feature area (first area) in which a visible feature is expected to appear within the surrounding image, and sets the area as a judgment area. The expected feature area is determined based on the position of the visible feature and the position of the vehicle M at the time the surrounding image is captured. The control unit 29 compares the image of the visible feature stored in the map information DB 27A with the image within the judgment area, and judges whether or not a visible feature appears in the judgment area.

[0045] When a visible feature appears in the judgment area, the control unit 29 provides the user with information providing guidance regarding the visible feature by outputting audio from the speaker 15 or by displaying an image on the touch panel display 13.

[0046] When an object different from the visible feature is captured in the determination area, the control unit 29 identifies the type of the different object. For example, the control unit 29 identifies the type of the object appearing in the determination area by comparing the image of the determination area with images of various objects stored in the image DB 27B of the storage unit 27.

[0047] The control unit 29 presents guidance information including information about the type of the identified object to the user. If the type of the object corresponds to a predetermined type, the control unit 29 transmits information including the type of the object to the server S.

[0048] In this way, when executing the feature guidance program, if a visible feature does not appear in the judgment area in the surrounding image, i.e., if the user is unable to see the visible feature, the control unit 29 notifies the user or an external server of information including the type of object that is preventing the visibility of the visible feature.

[0049] Furthermore, the control unit 29 transmits information about features obtained while the vehicle M is traveling to the server S. For example, when a building not included in the map information is newly constructed and this construction impedes the visibility of features that were originally included in the map information, the control unit 29 transmits information about the new building, for example, position information about the new building, to the server S.

[0050] The transmitting / receiving unit 31 is a network adapter such as a NIC (Network Interface Card) connected to a wireless device (not shown). The transmitting / receiving unit 31 transmits and receives data between the feature guide device 10 and the outside in accordance with commands from the control unit 29.

[0051] An example of feature guidance executed by the feature guidance device 10 of this embodiment will be described with reference to Figs. 6 to 10.

[0052] Fig. 6 is a diagram showing an example of a feature guide map displayed on a display. The feature guide map is, for example, a map image based on map information, on which the current position of the vehicle (vehicle mark) and the like are superimposed. In Fig. 6, a display showing the current position of the vehicle M and feature FT1 is superimposed on a road section RL1 on the image of the map.

[0053] 6, a vehicle M is traveling on a road section RL1 where a feature FT1 is visible. For example, based on the current position and traveling direction of the vehicle M and the position information of the feature FT1 included in the map information, the visual orientation of FT1 relative to the traveling direction of the vehicle M and the distance between the vehicle M and the feature FT1 can be calculated.

[0054] Fig. 7 shows a driving situation of a vehicle M traveling on a road section RL1. As shown in Fig. 7, a feature FT1 is located in the left front direction of the vehicle M. In the situation shown in Fig. 7, a user aboard the vehicle M can visually recognize the feature FT1 in the left front direction.

[0055] Fig. 8 is a diagram showing a surrounding image S1 of a vehicle M acquired in the situation exemplified in Fig. 7. For example, a feature FT1 is associated with a road section RL1, and when the vehicle M travels on the road section RL1, the control unit 29 acquires a surrounding image S1 of the vehicle M from the camera 17 and identifies a determination area AR, which is an expected feature area in which the feature FT1 is expected to appear in the surrounding image S1.

[0056] For example, the control unit 29 specifies the determination area AR based on the orientation of the feature FT1 relative to the shooting direction at the time when the surrounding image S1 was captured and the distance between the position of the vehicle M at the time when the surrounding image S1 was captured and the feature FT1. For example, the orientation and distance can be acquired using the vehicle position (the current position and traveling direction of the vehicle M) and position information of the feature FT1 included in the map information (e.g., FIG. 5).

[0057] In addition, the control unit 29 determines whether or not a feature FT1 is included in the determination area AR. For example, the control unit 29 determines whether or not the feature FT1 is included by comparing an image of the feature FT1 pre-stored in the map information DB of the storage unit 27 with the determination area AR of the surrounding image S1.

[0058] As described above, when the traveling direction of the vehicle M is set to the forward direction, the camera 17 is disposed so as to be able to capture a surrounding image in a range including the front, right front, and left front, and is able to capture a surrounding image S1 within the range of the field of view when the user sitting in the front seat of the vehicle M is looking forward. Therefore, when the feature FT1 is captured in the surrounding image including the determination area AR, it can be considered that the user can see the feature FT1 from the vehicle M.

[0059] 8, the control unit 29 determines that the feature FT1 is captured in the determination area AR. In this case, the control unit 29 presents guidance information to the user that the feature FT1 is visible via the speaker 15 or the display. For example, an announcement such as "You can see the large torii gate to your left" is played from the speaker 15 or shown on the display.

[0060] FIG. 9 illustrates a case where a bus OB1 is traveling diagonally ahead of the vehicle M on the left in addition to the traveling situation illustrated in FIG.

[0061] 9, the view to the left front from vehicle M is blocked by bus OB1. In other words, bus OB1 existing between vehicle M and feature FT1 is an obstacle that prevents the user in vehicle M from seeing FT1. In this case, the user in vehicle M cannot see feature FT1.

[0062] Fig. 10 is a diagram showing a surrounding image S2 of the vehicle M acquired in the situation exemplified in Fig. 9. When the control unit 29 acquires the surrounding image S2 from the camera 17, it specifies the determination area AR by the above-mentioned method, and determines whether or not a feature FT1 is captured in the specified determination area AR.

[0063] 10, the feature FT1 is not captured in the determination area AR of the surrounding image S2. In other words, an object different from the feature FT1 appears in the determination area AR of the surrounding image S2.

[0064] In this case, the control unit 29 identifies the type of the object appearing in the determination area AR. For example, the control unit 29 identifies the type of the object appearing in the determination area AR by referring to the image DB 27B. For example, the image DB 27B stores images of a large number of objects each classified into one of a plurality of types.

[0065] For example, the control unit 29 identifies the type of object appearing in the determination area AR by comparing the image of the determination area AR in the surrounding image S2 with each of the various images stored in the image DB 27B.

[0066] 10, a bus OB1, which is an object different from the feature FT1, is captured in the determination area AR, and therefore the type of the object is determined to be, for example, a "bus." The type of object to which the bus OB1 belongs is not limited to a "bus" and may be, for example, a large vehicle.

[0067] When an object different from the feature FT1 appears in the determination area AR, the control unit 29 notifies the user of information regarding the type of object that is obstructing the visibility of the feature FT1. For example, the control unit 29 displays a message including the type of object that is obstructing the visibility of the feature FT1 on the display or outputs the message by voice from the speaker 15. For example, a message such as "There is a large torii gate behind the bus on the front left" is announced or displayed.

[0068] The message including the type of object may be, for example, a message such as "The large torii gate is hidden by a bus ahead on the left," which indicates that the feature is not visible due to the presence of the object.

[0069] By being notified of such information including the type of object, the user can know why feature FT1 is not visible when traveling along road section RL1, i.e., when vehicle M is in a position where feature FT1 is visible.

[0070] Furthermore, the control unit 29 may issue, together with the information including the type of object, movement instruction information for moving the vehicle M so that the feature FT1 can be visually recognized, i.e., so that the feature FT1 appears in the surrounding image. For example, the control unit 29 may issue, as the movement instruction information, a notification encouraging the driver to change lanes. Also, for example, the control unit 29 may issue, as the movement instruction information, a notification encouraging the driver to slow down and increase the distance between the vehicles.

[0071] Specifically, as movement instruction information, messages such as "There is a large torii gate behind the bus ahead of you on the left. You may be able to see it if you change lanes" or "There is a large torii gate behind the bus ahead of you. You may be able to see it if you leave a larger distance between you and the bus" may be announced or displayed.

[0072] For example, a database relating to countermeasures according to situations may be stored in advance in the storage unit 27. The contents of the movement instruction information may be determined based on the database.

[0073] In this way, by notifying the user of the movement instruction information together with the information including the type of object, the user can know what to do to make the feature FT1 visible when the feature FT1 is not visible.

[0074] Fig. 11 illustrates a situation in which, in the driving situation illustrated in Fig. 7, instead of a bus OB1 traveling diagonally in front of the left of the vehicle M, a high-rise building OB2 is present along the road in the left front direction of the vehicle M. In the driving situation in Fig. 11, the view in the left front direction from the vehicle M is blocked by the high-rise building OB2, so the user inside the vehicle M cannot see the feature FT1.

[0075] 11, the control unit 29 also determines that the feature FT1 is not included in the determination area AR of the surrounding image, that is, an object different from the feature FT1 appears. Then, as described above, the type of the object appearing in the determination area AR is identified. The type of the object for the high-rise building OB2 is identified as, for example, a "high-rise building" or a "building".

[0076] The control unit 29 notifies the user of information about the type of object that is obstructing the visibility of the feature FT1, similar to the case of Fig. 10. For example, a message such as "There is a large torii gate at the back of the building ahead on the left" is announced or displayed.

[0077] Furthermore, the control unit 29 may determine whether the object is a predetermined object. The predetermined object is, for example, a fixed object that does not move or change in a short period of time, such as real estate including fixed objects on land, such as buildings and bridges.

[0078] If the object that obstructs the visibility of a visible feature is a fixed object such as real estate, the fixed object is likely not included in the map information. Therefore, the control unit 29 transmits information on the type of the fixed object, the position information of the fixed object, and information indicating the visible feature whose visibility is obstructed by the fixed object to the server S. Furthermore, in addition to these pieces of information, an image of the fixed object captured by the camera 17 may be transmitted to the server S. Note that the position information of the fixed object may be calculated from, for example, the relative position of the fixed object from the vehicle M obtained by a known position measuring device such as a LIDAR (Light Detection and Ranging) sensor mounted on the vehicle M, and the position information of the vehicle M obtained by the GPS receiver 19.

[0079] When the server S receives the information on the fixed objects and the information indicating the visible features, the server S adds the information on the fixed objects to the map information held by the server S. This makes it possible to keep the map information up to date even when, for example, a new building is constructed.

[0080] The server S may, for example, exclude visible features whose visibility is obstructed by the fixed object from features visible from the corresponding road section. For example, in the example of Fig. 11, the server S excludes the feature FT1 from features visible from the road section RL1.

[0081] Certain obstacles such as real estate constantly block the visibility of feature FT1 from road section RL1, not only when vehicle M but also when other vehicles are traveling on road section RL1.

[0082] Therefore, by excluding such objects from the visible features associated with the road section RL1 in the map information, it is possible to avoid a situation in which a visible feature cannot be seen.

[0083] In this way, the feature guidance device 10 transmits information indicating visible features and information regarding objects that obstruct the visibility of the visible features to the server S, thereby, for example, avoiding a situation in which the visible feature cannot be seen from the road section to which the visible feature is associated, making feature guidance smoother from the next time onwards.

[0084] Furthermore, by adding location information and types of objects that obstruct the visibility of visible features to the map data, the map data can be kept up to date.

[0085] 12 is a flowchart showing the feature guidance routine RT1 executed by the control unit 29 of the feature guidance device 10. For example, when the power to the feature guidance device 10 is turned on, the control unit 29 starts the feature guidance routine RT1.

[0086] When the control unit 29 starts the feature guidance routine RT1, it determines whether or not there is a feature visible from the road section on which the vehicle M is currently traveling in the map data (step S101). In step S101, for example, the control unit 29 refers to the map information DB 27A, and determines that a visible feature exists when there is a visible feature associated with the road section on which the vehicle M is currently traveling.

[0087] When the control unit 29 determines that a visible feature is present (step S101: YES), it acquires an image of the surroundings of the vehicle M, which is a moving body, from the camera 17 (step S102). In step S102, the control unit 29 functions as an image acquisition unit that executes acquisition of the surrounding image (image acquisition step).

[0088] After executing step S102, the control unit 29 identifies a feature expected area in which a visible feature is expected to appear in the surrounding image (step S103). The feature expected area is a judgment area for judging whether or not a visible feature is captured in the surrounding image. In step S103, for example, the control unit 29 identifies the judgment area based on the orientation of the visible feature with respect to the shooting direction at the time when the surrounding image S1 was captured and the distance between the vehicle M and the visible feature. In step S103, the control unit 29 functions as an area identification unit.

[0089] In step S103, for example, if multiple visible features are associated with the road section on which vehicle M is traveling, the control unit 29 selects the feature that is closest to the current position of vehicle M, and identifies the judgment area for the selected visible feature.

[0090] After executing step S103, the control unit 29 determines whether or not an object other than the visible feature is captured in the determination area in the surrounding image (step S104).

[0091] In step S104, for example, the control unit 29 determines whether or not an object different from the visible feature appears in the determination area of ​​the surrounding image S1 by comparing an image of the visible feature identified in step S101 among images pre-stored in the map information DB 27A of the storage unit 27 with the determination area of ​​the surrounding image S1. In step S104, the control unit 29 functions as a determination unit that determines whether or not an object different from the visible feature appears in the feature expected area in the surrounding image.

[0092] If the control unit 29 determines in step S104 that no feature different from the visible feature is captured (step S104: NO), that is, if it determines that a visible feature is captured, it outputs normal guide information regarding the visible feature (step S105).

[0093] For example, in step S105, the control unit 29 notifies the user that the visible feature is visible and guidance information indicating the direction in which the visible feature is visible by audio output from the speaker 15 or image display on the display. In step S105, for example, an announcement such as "You can see the large torii gate to your left" is made.

[0094] If the control unit 29 determines in step S104 that an object other than a visible object is captured in the image (step S104: YES), that is, if it determines that a visible object is not captured in the image, it identifies the type of object captured in the judgment area (step S106).

[0095] In step S106, for example, image analysis is performed on the determination region of the surrounding image to identify the type of object that is different from the visible feature.

[0096] In step S106, for example, the control unit 29 identifies the type of object appearing in the surrounding image by comparing the judgment area of ​​the surrounding image with images of various objects stored in the image DB27B of the memory unit 27, for example by pattern matching.

[0097] In step S106, the control unit 29 functions as an object type identification unit that identifies the type of object appearing in the determination area by performing image analysis on the surrounding image.

[0098] After executing step S106, the control unit 29 outputs guide information including the type of the object (step S107). In step S107, for example, the control unit 29 notifies the user of information indicating the type of the object captured in the determination area by audio output from the speaker 15 or image display on the display.

[0099] For example, in step S107, guidance information is presented by voice or on a screen display, indicating that a visible feature exists behind the object identified in step S106.

[0100] In step S107, the control unit 29 functions as a notification unit that performs notification of information about the type of object (notification step) when an object different from the predetermined object appears in the determination area. Also, the control unit 29 notifies information about the type of object when the vehicle M is in a position where the predetermined object can be seen.

[0101] By providing information about the type of object that is different from the visible feature, the user can know the reason why the visible feature cannot be seen. Furthermore, when the vehicle M is in a position where the visible feature can be seen, the guidance information can be presented in real time, so that the user's doubts and anxieties caused by the inability to see the visible feature can be resolved on the spot.

[0102] After executing step S107, the control unit 29 executes a transmission subroutine (S108) and transmits information including the type of object to the outside. In step S108, for example, in addition to the type of object captured in the determination area, information indicating visible features that cannot be viewed because of the object, i.e., features that were the subject of the determination in step S104, etc. are transmitted to the external server S.

[0103] After executing step S105, after executing step S108, or when it is determined in step S101 that there is no visible feature (step S101: NO), the control unit 29 ends the feature guidance routine RT1 and repeatedly executes the feature guidance routine RT1.

[0104] FIG. 13 is a flowchart showing a transmission subroutine RT2, which is an example of a subroutine executed in step S108 of the feature guide routine RT1.

[0105] When the control unit 29 starts the transmission subroutine RT2, it determines whether or not the type of the object captured in the determination area specified in step S106 corresponds to a predetermined object (step S201). The predetermined object is, for example, a fixed object such as real estate, which constantly obstructs the visibility of a feature associated with the road section as a visible feature.

[0106] For example, in step S201, a table for determining whether or not an object is a specified object is stored in memory unit 27, and based on the table, it is determined whether or not the type of object depicted in the determination area corresponds to a specified object.

[0107] When the control unit 29 determines in step S201 that the object is a specified object (step S201: YES), it transmits to the outside information indicating the type of object appearing in the judgment area and the visible feature that was the subject of the judgment in step S104 (step S202).

[0108] In step S202, for example, information indicating the object type and the visible features is transmitted to the server S, and the map information held in the server S is updated.

[0109] After executing step S202, or when it is determined in step S201 that the object is not a predetermined object (step S201: NO), the control unit 29 ends the transmission subroutine RT2.

[0110] The feature guidance by the feature guidance device 10 of this embodiment may be executed as part of the flow of route guidance by a navigation device.

[0111] As described above in detail, the feature guide device 10 as a notification device of the first embodiment can acquire an image of the surroundings of a moving body taken from the current position of the moving body by a camera arranged on the moving body, identify a feature predicted area in which a predetermined feature is predicted to appear in the surrounding image, and, if an object different from the predetermined feature appears in the feature predicted area, notify information including the type of the object to a user or an external server, etc. The predetermined feature is a feature that can be determined from map information to be visible from the current position of the moving body, and is a feature that is associated in the map information with the road section on which the vehicle M of this embodiment is currently traveling.

[0112] Therefore, according to the notification device of Example 1, when the visibility of a specified feature is obstructed by an object other than the specified feature, it is possible to notify a user or a server of the type of object that is obstructing the visibility.

[0113] Therefore, when guidance is given using features that are visible from the road, it is possible to provide an alarm device that can prevent a user from becoming confused when the features cannot be seen from the road.

[0114] When the notification device of this embodiment notifies the user of a type of object that is different from a specified feature, any anxiety or dissatisfaction the user may have due to not being able to see a visible feature can be alleviated, potentially reducing the user's psychological stress.

[0115] In addition, when the notification device of the present embodiment notifies the server of the type of object different from the predetermined feature, it is possible to keep the map information up to date. In addition, it is possible to avoid a situation where a visible feature is constantly invisible due to the construction of a new building, etc., and it is possible to perform route guidance with high accuracy by using the updated map information. EXAMPLES

[0116] The function of a route guidance device 50 as a notification device according to the second embodiment will be described with reference to Figs. 14 to 20. The route guidance device 50 has the same configuration as the feature guidance device 10 described with reference to Figs. 1 to 3, but part of the information stored in the storage unit 27 and part of the processing executed by the control unit 29 are different. Hereinafter, a description of parts common to the first embodiment will be omitted.

[0117] The control unit 29 performs route guidance by executing a route guidance program serving as a notification program in this embodiment. When the route guidance program is executed, for example, the control unit 29 accepts input of route search conditions such as a destination via the microphone 11 or the touch panel display 13, and generates a route based on map information. Thereafter, the control unit 29 performs route guidance according to the generated route.

[0118] A route guidance program is stored in the storage unit 27. In addition, data in which landmark features serving as landmarks are associated with each guide point in route guidance is stored in a map information DB 27A of the storage unit 27. In addition, position information of the landmark features is stored in the map information DB 27A.

[0119] 14 is a diagram showing an example of data of landmark features associated with guide points stored in the map information DB 27A as a table TB3. In the table TB3, each of the guide points GP1, GP2, etc., written in the column of the guide point ID, is associated with a landmark feature that serves as a landmark at each guide point.

[0120] In this specification, the guidance point is a point where a driving operation such as turning right or left is required or a point where a warning is required on the driving route of the vehicle M. For example, in the route guidance of this embodiment, it is a point where guidance such as encouraging the vehicle M to turn right or left is given by voice when the vehicle M arrives within a predetermined distance.

[0121] In this specification, a feature that serves as a landmark at a guidance point is referred to as a landmark feature. For example, a landmark feature includes a building that serves as a landmark for confirming that the intersection is one where a right or left turn should be made. For example, a feature that exists within a specified distance from the guidance point, a feature facing the guidance point, a feature that has a height equal to or greater than a specified height, etc. can be a landmark feature at a guidance point.

[0122] In table TB3, guide points are associated with landmark features that are visible from road sections adjacent to the guide points. Note that different landmark features may be associated with each road section connected to a guide point.

[0123] Fig. 15 is a diagram showing, as a table TB4, an example of position information of landmark features, which is part of the map information stored in the map information DB27A of the storage unit 27. As shown in Fig. 15, a landmark feature ID indicating each landmark feature is associated with the position information of the landmark feature specified by the landmark feature ID.

[0124] Fig. 16 is a diagram showing an example of a route guidance map displayed on a display in this embodiment. In Fig. 16, the current position of the vehicle M, an intersection N1 which is a guidance point on the route A, and a landmark feature LM associated with the intersection N1 are displayed superimposed on an image of the route A of the vehicle M including the road section RL2.

[0125] 16, the vehicle M is traveling on a road section RL2 where the landmark feature LM is visible. For example, based on the current position and traveling direction of the vehicle M and the position information of the landmark feature LM included in the map data, the visible direction of the landmark feature LM relative to the traveling direction of the vehicle M and the distance between the vehicle M and the landmark feature LM can be calculated.

[0126] Fig. 17 is a diagram showing an example of a driving state of the vehicle M during the route guidance shown in Fig. 16. In the situation in Fig. 17, the vehicle M is driving in the road section RL2 of the route A according to the guidance by the route guidance device 50, and is approaching the intersection N1. The route A is a route that turns right at the intersection N1.

[0127] A landmark feature LM, which is a feature that serves as a landmark when confirming an intersection at which to turn right, is visible from the current position of the vehicle M. When the current position of the vehicle M approaches within a predetermined distance of the landmark feature LM, the control unit 29 of the route guidance device 50 acquires an image of the surroundings of the vehicle M from the camera 17, and specifies a determination area within the surrounding image in which the landmark feature LM is expected to appear.

[0128] When the route guidance device 50 determines that a landmark feature is captured in the determination area, it uses the landmark feature LM as a landmark and makes an announcement to encourage a right turn at the intersection N1. In the situation of Fig. 17, for example, a voice saying "50 meters ahead, turn right at the intersection where the XX hotel is located" is output from the speaker 15. Using the landmark feature LM as a landmark, the driver of the vehicle M can recognize that the intersection where the landmark feature LM is located is an intersection where a right turn should be made.

[0129] Fig. 18 is a diagram showing another example of the driving situation of vehicle M during the route guidance shown in Fig. 16. In Fig. 18, in addition to the situation in Fig. 17, a bus OB3 is driving in front of and to the left of vehicle M. From the current position of vehicle M, the bus OB3 becomes an obstacle and the landmark feature LM cannot be seen.

[0130] In the situation of FIG. 18, when the control unit 29 of the route guidance device 50 specifies the determination area by the above-mentioned method, a bus OB3, which is an object different from the landmark feature LM, appears in the determination area.

[0131] The route guidance device 50 determines that the landmark feature LM is not in the determination area, i.e., that the landmark feature LM is a different object, and outputs guidance information including information about the different object, i.e., an obstacle that prevents the landmark feature LM from being seen.

[0132] 18, for example, a voice such as "50 meters ahead, turn right at the intersection where the XX Hotel is located. The XX Hotel is on the other side of the bus ahead on the left" is output from the speaker 15. This allows the driver of the vehicle M to get a rough idea of ​​the location of the landmark feature LM even if he or she cannot see the landmark feature LM.

[0133] 19 is a flowchart showing a route guidance routine RT3, which is an example of processing executed by the control unit 29 of the route guidance device 50. The control unit 29 starts the route guidance routine RT3, for example, when the route of the vehicle M is determined.

[0134] When the control unit 29 starts the route guidance routine RT3, it judges whether or not the current position of the vehicle M has reached within a predetermined distance of the next guidance point (step S301). In step S301, for example, the control unit 29 reads map information from the map information DB 27A, searches for the nearest guidance point from the current position of the vehicle M based on the position information of each guidance point on the route, calculates the distance between the guidance point and the current position of the vehicle M, and judges whether or not it is within the predetermined distance.

[0135] When the control unit 29 determines in step S301 that the vehicle has arrived within a predetermined distance of the next guidance point (step S301: YES), the control unit 29 refers to map information (e.g., FIG. 14) and searches for a landmark feature associated with the next guidance point (step S302).

[0136] After executing step S302, the control unit 29 determines whether or not there is a landmark feature that serves as a landmark at the next guide point (step S303).

[0137] If the control unit 29 determines in step S303 that there is no landmark feature (step S303: NO), it supplies normal guide information that does not use a landmark feature to the speaker 15 or the display via the output unit 25 (step S304). In step S304, for example, an announcement is made to encourage a right turn at the intersection N1, using a sign indicating the intersection name or a traffic light as a landmark.

[0138] When the control unit 29 determines in step S303 that a landmark feature exists (step S303: YES), it executes a landmark feature guidance subroutine (step S305) and outputs guide information according to the situation.

[0139] When the control unit 29 determines in step S301 that the vehicle has not arrived within a predetermined distance of the next guidance point (step S301: NO), after executing step S304 or step S305, the control unit 29 determines whether the vehicle has arrived at the destination (step S306).

[0140] When the control unit 29 determines in step S306 that the vehicle M has not arrived at the destination (step S306: NO), the control unit 29 returns to step S301 and determines again whether the current position of the vehicle M is within a predetermined distance of the next guide point. When the control unit 29 determines in step S306 that the vehicle M has arrived at the destination (step S306: YES), the control unit 29 ends the route guidance routine RT3.

[0141] Fig. 20 is a flowchart showing a landmark feature guidance subroutine RT4, which is an example of a subroutine executed in step S305 of the route guidance routine RT3. The landmark feature guidance subroutine RT4 has the same contents as steps S102 to S108 of the feature guidance routine RT1 described in Fig. 12, and differs only in that it handles landmark features instead of visible features, so a detailed description will be omitted.

[0142] When the control unit 29 starts the landmark feature guidance subroutine RT4, it acquires a surrounding image of the vehicle M from the camera 17 (step S401), and identifies a judgment area in which a landmark feature is expected to appear in the surrounding image (step S402). As described above, in step S402, the judgment area is identified based on, for example, the orientation of the landmark feature relative to the shooting direction at the time when the surrounding image was shot, and the distance between the vehicle M and the landmark feature.

[0143] When the control unit 29 determines that an object other than the landmark feature is not shown in the determination area (step S403: NO), it outputs guidance information using the landmark feature as a landmark (step S404). For example, in the case of guidance encouraging a right turn at an intersection using a hotel as a landmark feature as a landmark, an announcement such as "50 m ahead, turn right at the intersection where the XX hotel is located" is made.

[0144] When the control unit 29 determines that an object other than the landmark feature is captured in the determination area (step S403: YES), it identifies the type of the object captured in the determination area (step S405) and outputs guide information including the type of the object (step S406). For example, if the landmark feature is "XX Hotel" and the type of the object is a bus, an announcement such as "XX Hotel is on the other side of the bus ahead on the left" is made.

[0145] Thereafter, the control unit 29 executes a transmission subroutine (step S407) and transmits information on the type of object to the server S. For example, the transmission subroutine in step S407 may have the same content as the transmission subroutine RT2 described in FIG. 13. Also, unlike the transmission subroutine RT2, the transmission subroutine in step S407 may transmit data on the obstacle to the server not only when the obstacle obstructing the visibility of the landmark feature is a fixed object, but also when the obstacle is a moving object such as another vehicle. As a result, data on the frequency with which the landmark feature cannot be viewed due to traffic conditions, for example, when the visibility is obstructed by a large truck, is accumulated in the server S. For example, the data accumulated in this way is used as a basis for determining whether or not a landmark feature whose visibility is frequently obstructed due to traffic conditions is excluded from the landmark features at the guidance point.

[0146] In addition, in the landmark feature guidance subroutine RT4, the transmission subroutine 407 may be omitted.

[0147] Furthermore, the feature guidance of the embodiment 1 may be combined and executed with the route guidance of the embodiment 2. For example, in the route guidance, guidance on a feature associated with a road section on which the vehicle M is traveling may be performed at a point other than the guidance point, and when an object that obstructs the visibility of the feature appears in a determination area in the surrounding image, information on the type of the object may be notified to the user.

[0148] As described above, according to the route guidance device 50 as the notification device of the second embodiment, when guidance is given at a guide point using a feature as a landmark, if an object such as another vehicle becomes an obstacle that prevents the landmark feature from being seen, the type of the obstacle object can be identified and information including the type of the object can be notified. Therefore, even if the user cannot see the landmark feature, the user can know the reason why the landmark feature cannot be seen.

[0149] In addition, when route guidance is provided using features visible from the road as landmarks, it is possible to provide an alarm device that can prevent a user from becoming confused when the features cannot be seen from the road.

[0150] Furthermore, the route guidance device 50 can notify the user of the location of the landmark object along with the type of the obstacle. This allows the user to get a rough idea of ​​the location of the landmark object.

[0151] In the above-mentioned first and second embodiments, the moving body is a vehicle, but the present invention is not limited to this. The moving body may be a human. The notification device of the present invention may provide notification regarding route guidance for travel by foot. The notification device of the present invention is not limited to being mounted on a vehicle or other mobile device, but may be mounted on a portable device or a wearable device such as a smartphone or tablet carried by a user. It is sufficient that the notification device is configured to be able to acquire a surrounding image captured from the current position of the moving body.

[0152] In the above first and second embodiments, examples have been described in which the processes of the area identification unit, the determination unit, and the object type identification unit are performed by the control unit 29, but these processes may be performed on the server S. The notification device of the present invention may be a device that transmits image data to the server S, receives information indicating the type of object from the server S, and notifies the user.

[0153] In the above first and second embodiments, examples in which data is transmitted and received from the server S have been described, but the present invention is not limited to this, and the notification device of the present invention does not need to communicate with an external server.

[0154] It should be noted that the road section in the above embodiment may be each of the lanes of the road section.

[0155] In addition, the map information in the map information DB27A in the above embodiment may be updated, for example, periodically, or may be updated by obtaining update data via the network NW when the map information in the server S is updated.

[0156] The configurations in the above-described embodiments are merely examples, and can be appropriately selected and modified depending on the application, etc. [Explanation of symbols]

[0157] 10 Route search device 11. Mike 13 Touch panel display 17 Camera 19 GPS receiver 22 Sensors 23 Input section 24 System Bus 25 Output section 27 Memory section 29 Control Unit 31 Transmitter / receiver

Claims

[Claim 1] A guidance device that guides a route of a moving object using a predetermined feature as a landmark, an image acquisition unit that acquires a surrounding image, which is an image of the surroundings of the moving object captured from the moving object; a notification unit that notifies a passenger of the moving body of guidance including information on the predetermined feature and information on a type of the object when an object different from the predetermined feature appears in a first area in which the predetermined feature is expected to appear in the surrounding image, the first area being determined based on map information including position information of the predetermined feature and the position of the moving body at the time the surrounding image was captured, thereby preventing the predetermined feature from being seen; A guidance device comprising:

Images