Guidance device, guidance method, guidance program, and recording medium
The notification device addresses navigation system failures by identifying obstructing objects and providing real-time guidance, enhancing user experience and map data accuracy.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- PIONEER IP
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-07
AI Technical Summary
Existing navigation systems fail to provide effective guidance when landscape objects become visually obstructed, leading to user confusion during travel.
A notification device that acquires ambient images using a camera, identifies potential obstruction objects, and provides information on the type of obstructing objects to the user, updating map data to maintain accurate guidance.
Prevents user confusion by informing about obstructing objects and providing instructions to regain visibility of landmarks, ensuring smoother and more accurate route guidance.
Smart Images
Figure 2026113700000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a notification device, a notification method, a notification program, and a recording medium.
Background Art
[0002] Conventionally, a navigation device that provides route guidance while displaying a searched route is known. For example, Patent Document 1 discloses a navigation device that displays a road at a point or section where a landscape object can be visually recognized in a display form different from other roads.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] For example, in route guidance by the above-described navigation device, when actually traveling on a road at a point or section where a landscape object can be visually recognized, the landscape object may not be visually recognized due to, for example, other vehicles or newly constructed buildings, and there is a possibility that the user may be confused. This is one of the problems.
[0005] The present invention has been made in view of the above points, and for example, when providing guidance using a ground feature visible from a road, it is possible to prevent the user from being confused when the ground feature cannot be visually recognized from the road. One of the objectives is to provide a notification device.
Means for Solving the Problems
[0006] The invention described in claim 1 is characterized by comprising: an image acquisition unit that acquires an ambient image which is an image of the area around a moving object taken from the moving object; and a notification unit that notifies information regarding the type of object when an object different from the predetermined object appears in a first region in the ambient image where the predetermined object is expected to appear, which is determined based on map information including location information of a predetermined object and the position of the moving object at the time the ambient image was taken.
[0007] The invention described in claim 10 is a notification method performed by a notification device, comprising: an image acquisition step in which an image acquisition unit acquires an ambient image which is an image of the area around a moving object taken from the moving object; and a notification step in which a notification unit notifies information regarding the type of object when an object different from the predetermined object appears in a first region in the ambient image where the predetermined object is expected to appear, which is determined based on map information including location information of a predetermined object and the position of the moving object at the time the ambient image was taken.
[0008] The invention described in claim 11 is a notification program executed by a notification device equipped with a computer, characterized in that the computer is caused to perform an image acquisition step of acquiring an ambient image which is an image of the area around a moving object taken from the moving object; and a notification step of notifying information about the type of object when an object different from the predetermined object appears in a first region in the ambient image where the predetermined object is expected to appear, which is determined based on map information including location information of a predetermined object and the position of the moving object at the time the ambient image was taken. [Brief explanation of the drawing]
[0009] [Figure 1] This figure shows an overview of the notification system, including the notification device according to Example 1. [Figure 2] This figure shows the front seat area of a vehicle equipped with the notification device according to Example 1. [Figure 3] This is a block diagram showing the configuration of the notification device according to Example 1. [Figure 4] This figure shows an example of the data included in the map information related to Example 1. [Figure 5] This figure shows an example of the data included in the map information related to Example 1. [Figure 6] This figure shows an example of a guide map related to Example 1. [Figure 7] This figure shows an example of the movement of the mobile body according to Example 1. [Figure 8] This figure shows an example of a surrounding image used in the guidance for Example 1. [Figure 9] This figure shows another example of the movement of the mobile body according to Example 1. [Figure 10] This figure shows another example of a surrounding image used in the guidance according to Example 1. [Figure 11] This figure shows another example of the movement of the mobile body according to Example 1. [Figure 12] This flowchart shows an example of a routine performed by the notification device of Example 1. [Figure 13] This flowchart shows an example of a subroutine executed by the notification device of Example 1. [Figure 14] This figure shows an example of the data included in the map information used for route guidance in Example 2. [Figure 15] This figure shows an example of the data included in the map information used for route guidance in Example 2. [Figure 16] This figure shows an example of a route guidance map according to Example 2. [Figure 17] This figure shows an example of the driving conditions during route guidance according to Example 2. [Figure 18] This figure shows another example of the driving conditions during route guidance according to Example 2. [Figure 19] This flowchart shows an example of a routine performed by the notification device of Example 2. [Figure 20] This flowchart shows an example of a subroutine executed by the notification device of Example 2.
Best Mode for Carrying Out the Invention
[0010] The embodiments of the present invention will be described in detail below. In the following description and the accompanying drawings, the same reference numerals are assigned to substantially identical or equivalent parts.
Embodiment
[0011] While referring to FIGS. 1 to 5, the configuration of the ground object guidance device 10 as the notification device according to Embodiment 1 will be described.
[0012] FIG. 1 shows an information exchange system 100 including the ground object guidance device 10. As shown in FIG. 1, the information exchange system 100 includes the ground object guidance device 10 mounted on the vehicle M and the server S. The ground object guidance device 10 transmits and receives data to and from the server S via the network NW.
[0013] The ground object guidance device 10 performs ground object guidance by presenting information on ground objects around the road to the user riding in the vehicle M when the vehicle M travels on the road.
[0014] For example, the server S holds map information including information on ground objects. For example, the ground object guidance device 10 transmits information on ground objects around the road obtained when the vehicle M travels to the server S. The server S is configured to be able to update the information on the ground objects in the map information according to the information transmitted from the ground object guidance device 10.
[0015] In this specification, "ground object" refers to all natural or artificial objects such as trees, rivers, houses, roads, railways, etc. Typically, the ground object is a famous landmark or the like, including natural objects such as mountains and rocks, and buildings such as temples, shrines, and high-rise buildings.
[0016] FIG. 2 is a view showing the front seat portion of the vehicle M on which the ground object guidance device 10 according to Embodiment 1 is mounted. The ground object guidance device 10 is disposed in the center console within the dashboard DB of the front seat of the vehicle M.
[0017] The terrain information device 10 is communicatively connected to a microphone 11, a touch panel display 13, a speaker 15, and a camera 17 installed inside the vehicle M. The terrain information device 10 is also connected to a GPS receiver 19. The microphone 11 and the touch panel display 13 are located, for example, on the dashboard DB. The speaker 15 is located, for example, on the A-pillar AP or on the interior side of the door.
[0018] Microphone 11 is an audio device that converts sound into electrical signals. Microphone 11 is installed on the dashboard DB between the driver's seat and the passenger seat and transmits the voice of the user sitting in the front seat of vehicle M to the terrain guidance device 10.
[0019] The touch panel display 13 is composed of a display that shows screen information based on the control of the terrain information device 10 and a touch panel that transmits signals representing user input operations to the terrain information device 10.
[0020] Speaker 15 is a device that outputs sound based on signals transmitted from the geographical feature guidance device 10.
[0021] The feature information device 10 receives user input via the microphone 11 or the touch panel display 13. The feature information device 10 displays the vehicle M's current location and information about features on the touch panel display 13. The feature information device 10 also presents information about features to the user via voice from the speaker 15.
[0022] Camera 17 is positioned, for example, on the dashboard DB. Camera 17 is oriented to capture ambient images, which are images of the area around vehicle M. Camera 17 is positioned to capture images in front of vehicle M, i.e., in the direction of vehicle M's movement. When the direction of vehicle M's movement is considered to be forward, camera 17 is positioned to capture ambient images in a range that includes the front, the front right, and the front left. Therefore, camera 17 is capable of capturing images within the field of view of a user sitting in the front seat of vehicle M while looking forward.
[0023] Furthermore, the camera 17 is not limited to being located on the dashboard DB, but may be installed anywhere as long as it can capture images of the surrounding area in front of the vehicle M. For example, the camera 17 may be installed behind the rearview mirror RM, that is, on the surface facing the windshield FG. Alternatively, the camera 17 may be installed on the outer surface (exterior surface) of the vehicle M, for example, on the hood or front bumper.
[0024] The GPS receiver 19 is located, for example, on a dashboard database, and receives signals (GPS signals) from GPS (Global Positioning System) satellites and transmits them to the geographical information device 10.
[0025] Figure 3 is a block diagram showing the configuration of the terrain guidance device 10. As shown in Figure 3, the terrain guidance device 10 is configured with each part connected via a system bus 21.
[0026] The input unit 23 is an interface for acquiring data from equipment installed in the vehicle M. The input unit 23 is connected to the microphone 11 and the touch panel of the touch panel display 13, which are equipment installed in the vehicle M. The feature guidance device 10 receives input from the microphone 11 or the touch panel via the input unit 23.
[0027] As shown in Figure 3, the input unit 23 is connected to the camera 17. The feature guidance device 10 acquires an image of the vehicle M's surroundings, captured by the camera 17, via the input unit 23.
[0028] Furthermore, the input unit 23 is connected to the GPS receiver 19 and the sensor 24. The sensor 24 is a group of sensors consisting of, for example, an acceleration sensor, an angular velocity sensor, and a distance sensor. The feature guidance device 10 acquires data such as the position and speed of the vehicle M from the GPS receiver 19 and the sensor 24 via the input unit 23. For example, the data such as the position and speed of the vehicle M is used by the feature guidance device 10 to acquire the current position for feature guidance.
[0029] The output unit 25 is connected to the display and speaker 15 of the touch panel display 13 provided in the vehicle M. The output unit 25 is an interface for supplying various information to the display and speaker 15.
[0030] For example, the feature information device 10 supplies images such as messages and maps to the display via the output unit 25. For example, the feature information device 10 supplies audio guidance information to the speaker 15 via the output unit 25 to provide guidance about features to the user.
[0031] The storage unit 27 is a storage device composed of, for example, a hard disk drive, an SSD (solid state drive), or flash memory. The storage unit 27 stores various programs that are executed in the geographical feature guidance device 10. These programs may be obtained from, for example, a server S or other server device via a network, or they may be recorded on a recording medium and read via various drive devices.
[0032] Furthermore, the memory unit 27 stores a map information database (hereinafter referred to as the map information DB) 27A. The map information DB 27A stores map information used for feature guidance by the feature guidance device 10. This map information also includes information on features located around roads used for feature guidance. For example, this map information includes location information of features and images of features taken from the road.
[0033] Specifically, the map information stores maps that include road sections (hereinafter also referred to as road links), with the geographical features visible from those road sections associated with them.
[0034] Figure 4 shows an example of the data linking road sections and features included in the map information stored in map information DB27A, as shown in table TB1. In table TB1, each of RL1 to RL4 in the "Link ID" column represents one of multiple road links. In the "Visible Features" column, features FT1 to FT4 that are visible from a vehicle traveling along each road section are associated with each road section.
[0035] For example, as shown in Figure 4, some road links have multiple features associated with them (RL3), while others do not have any features associated with them (RL4).
[0036] Figure 5 is a diagram showing an example of location information for features included in map information, represented as Table TB2. As shown in Figure 5, location information is associated with each visible feature. The location information may be global coordinates such as latitude and longitude, or local coordinates such as coordinates relative to a reference location.
[0037] Furthermore, the map information stored in the map information DB27A stores an image of each feature associated with each road section, taken from that road section. In other words, the map information includes images of the features associated with the road section. These images are used, for example, to determine whether or not a feature visible from the road section being traveled is captured in an ambient image taken from a vehicle M traveling on the road.
[0038] The memory unit 27 stores an image database (hereinafter referred to as the image DB) 27B, which is a database of images of objects that may be visible from a moving vehicle while it is in motion. For example, the image DB 27B stores image data taken from various angles of various types of vehicles and buildings of various appearances that may be obstacles when a moving vehicle is viewing a landmark.
[0039] For example, the image data stored in image DB27B is used to identify the type of object when an object visible in the surrounding image taken from vehicle M obstructs the visibility of other features. In other words, image DB27B stores sample images of obstacles.
[0040] The control unit 29 is a computer that controls the operation of the geographical feature guidance device 10, and includes a CPU (Central Processing Unit) 29A, a ROM (Read Only Memory) 29B, and a RAM (Random Access Memory) 29C. The CPU 29A reads and executes various programs stored in the ROM 29B and the memory unit 27 to realize various functions.
[0041] For example, the control unit 29 reads the feature guidance program, which serves as the notification program in this embodiment, from the storage unit 27 and executes it. The feature guidance program proceeds as follows, for example.
[0042] First, the control unit 29 searches for whether or not there are any visible features on the road the vehicle M is traveling on by referring to the map information stored in the map information DB.
[0043] The control unit 29 acquires an image of the surroundings captured by the camera 17 from the vehicle M if visible features are present.
[0044] The control unit 29 identifies a feature prediction area (first area) in the surrounding image where visible features are expected to appear and designates it as a determination area. This feature prediction area is determined based on the location of the visible features and the position of the vehicle M at the time the surrounding image was taken. The control unit 29 compares the image of the visible features stored in the map information DB 27A with the image within the determination area to determine whether or not a visible feature is present in the determination area.
[0045] If a visible feature appears in the determination area, the control unit 29 provides the user with information regarding the visible feature by outputting audio from the speaker 15 or displaying an image on the touch panel display 13.
[0046] The control unit 29 identifies the type of object if an object different from a visible feature is captured in the determination area. For example, the control unit 29 identifies the type of object appearing in the determination area by comparing the image of the determination area with images of various objects stored in the image DB 27B of the storage unit 27.
[0047] The control unit 29 presents guidance information to the user, including information about the type of the identified object. Furthermore, if the object's type matches a predetermined category, it transmits information including the object's type to the server S.
[0048] Thus, when the control unit 29 executes the feature guidance program, if no visible feature appears in the determination area of the surrounding image, that is, if the user is unable to see a visible feature, it notifies the user or an external server of information including the type of object obstructing the visibility of the visible feature.
[0049] Furthermore, the control unit 29 transmits information about features obtained while the vehicle M is in motion to the server S. For example, if a building that is not present in the map information is newly constructed and this obstructs the visibility of features that were originally present in the map information, the control unit 29 transmits information about the new building, such as the location information of the new building, to the server S.
[0050] The transmitting / receiving unit 31 is a network adapter such as a NIC (Network Interface Card) connected to a wireless device (not shown). The transmitting / receiving unit 31 transmits and receives data between the terrain guidance device 10 and the outside world according to commands from the control unit 29.
[0051] Referring to Figures 6 to 10, an example of feature guidance performed by the feature guidance device 10 of this embodiment will be described.
[0052] Figure 6 shows an example of a feature map displayed on a screen. A feature map is, for example, a map image based on map information, with the vehicle's current location (vehicle marker) superimposed on it. In Figure 6, the road section RL1 on the map image is superimposed with the vehicle's current location and feature FT1.
[0053] In the example shown in Figure 6, vehicle M is traveling on road section RL1 where feature FT1 is visible. For example, based on the vehicle M's current position, direction of travel, and the location information of feature FT1 included in the map information, the direction in which FT1 is visible relative to the vehicle M's direction of travel and the distance between vehicle M and feature FT1 can be calculated.
[0054] Figure 7 shows the driving conditions of vehicle M while traveling on road section RL1. As shown in Figure 7, feature FT1 is located to the left front of vehicle M. In the situation shown in Figure 7, the user in vehicle M can see feature FT1 to the left front.
[0055] Figure 8 shows the surrounding image S1 of vehicle M acquired in the situation illustrated in Figure 7. For example, if feature FT1 is associated with road section RL1, when vehicle M travels along road section RL1, the control unit 29 acquires the surrounding image S1 of vehicle M from camera 17 and identifies the determination area AR, which is the feature prediction area in the surrounding image S1 where feature FT1 is expected to be visible.
[0056] For example, the control unit 29 identifies a determination area AR based on the orientation of feature FT1 with respect to the shooting direction at the time the surrounding image S1 was taken, and the distance between the position of the vehicle M and feature FT1 at the time the image was taken. For example, the orientation and distance can be obtained using the vehicle's position (the current position and direction of travel of the vehicle M) and the position information of feature FT1 included in the map information (for example, Figure 5).
[0057] Furthermore, the control unit 29 determines whether or not the feature FT1 is visible in the determination area AR. For example, the control unit 29 determines whether or not the feature FT1 is visible by comparing an image of the feature FT1 that is pre-stored in the map information DB of the storage unit 27 with the determination area AR of the surrounding image S1.
[0058] As described above, the camera 17 is positioned to capture surrounding images in a range including the front, right front, and left front when the direction of travel of the vehicle M is forward, and can capture surrounding images S1 within the field of view when a user sitting in the front seat of the vehicle M is looking forward. Therefore, if the feature FT1 is visible in the surrounding image including the judgment area AR, the user can be considered to be able to see the feature FT1 from the vehicle M.
[0059] In the example shown in Figure 8, the control unit 29 determines that feature FT1 is visible in the AR determination area. In this case, the control unit 29 provides the user with information indicating that feature FT1 is visible via the speaker 15 or display. For example, an announcement such as "You can see the large torii gate to the left" is played from the speaker 15 or displayed on the display.
[0060] Figure 9 shows the case where, in addition to the driving situation illustrated in Figure 7, bus OB1 is traveling diagonally to the left and in front of vehicle M.
[0061] In the situation shown in Figure 9, the view from vehicle M to the left front is obstructed by bus OB1. In other words, bus OB1, which is located between vehicle M and terrain object FT1, acts as an obstacle that prevents vehicle M from seeing FT1. In this case, the user in vehicle M cannot see terrain object FT1.
[0062] Figure 10 shows an image S2 of the surroundings of vehicle M acquired in the situation illustrated in Figure 9. When the control unit 29 acquires the surrounding image S2 from the camera 17, it identifies the determination area AR using the method described above and determines whether or not the feature FT1 is visible in the identified determination area AR.
[0063] As shown in Figure 10, the object FT1 is not visible in the judgment region AR of the surrounding image S2. In other words, an object different from the object FT1 appears in the judgment region AR of the surrounding image S2.
[0064] In this case, the control unit 29 identifies the type of object that appears in the judgment area AR. For example, the control unit 29 identifies the type of object that appears in the judgment area AR by referring to the image DB 27B. For example, the image DB 27B stores images of many objects, each classified into one of several types.
[0065] For example, the control unit 29 identifies the type of object depicted in the determination area AR by comparing the image of the determination area AR of the surrounding image S2 with each of the various images stored in the image DB 27B.
[0066] In the example in Figure 10, the determination area AR contains a bus OB1, which is a different object from the feature FT1, so the object type is determined to be, for example, "bus". The object type to which bus OB1 belongs is not limited to "bus"; it could also be, for example, a large vehicle.
[0067] The control unit 29, when an object different from feature FT1 appears in the judgment area AR, notifies the user of information regarding the type of object obstructing the visibility of feature FT1. For example, the control unit 29 displays a message including the type of object obstructing the visibility of feature FT1 on the display or outputs it as audio from the speaker 15. For example, a message such as "There is a large torii gate behind the bus to the left front" is announced or displayed.
[0068] Messages that include the type of object may also include messages such as "The large torii gate is not visible due to the presence of the object," which indicate that the feature is not visible due to the presence of the object.
[0069] By providing information including the type of object, the user can understand why the feature FT1 is not visible when driving on road section RL1, that is, when vehicle M is in a position to see the feature FT1.
[0070] Furthermore, the control unit 29 may also provide information including the type of object, along with movement instruction information to move the vehicle M so that the feature FT1 can be seen, that is, so that the feature FT1 appears in the surrounding image. For example, the control unit 29 may provide movement instruction information to encourage a lane change. Alternatively, for example, the control unit 29 may provide movement instruction information to encourage slowing down and increasing the distance between vehicles.
[0071] Specifically, as movement instruction information, messages such as "There is a large torii gate behind the bus to your left. You may be able to see it if you change lanes" or "There is a large torii gate behind the bus ahead. You may be able to see it if you maintain a safe distance" may be announced or displayed.
[0072] For example, a database of appropriate countermeasures for different situations may be stored in the storage unit 27 beforehand. The content of the movement instruction information may be determined based on this database.
[0073] In this way, by providing movement instruction information along with information including the type of object, users can understand what to do to make feature FT1 visible if it is not visible.
[0074] Figure 11 shows a scenario in the same driving situation as illustrated in Figure 7, but instead of bus OB1 traveling diagonally to the left and in front of vehicle M, a high-rise building OB2 is located along the road to the left and in front of vehicle M. In the driving situation shown in Figure 11, the view to the left and in front of vehicle M is obstructed by high-rise building OB2, so the user inside vehicle M cannot see landmark FT1.
[0075] In the case of Figure 11, the control unit 29 also determines that feature FT1 is not visible in the judgment area AR of the surrounding image, that is, that an object different from feature FT1 has appeared. Then, as described above, it identifies the type of object that has appeared in the judgment area AR. For example, the type of object for the high-rise building OB2 is identified as "high-rise building" or "building".
[0076] The control unit 29, as in the case of Figure 10, informs the user of the type of object obstructing the visibility of feature FT1. For example, a message such as "There is a large torii gate behind the building to the left and in front" is announced or displayed.
[0077] Furthermore, the control unit 29 may determine whether the object is a predetermined object. A predetermined object is, for example, a fixed object that does not move or change in a short period of time, and includes real estate such as buildings and bridges that are fixed to the land.
[0078] If an object obstructing the visibility of a visible feature is a fixed object such as real estate, it is highly likely that the fixed object is not included in the map information. Therefore, the control unit 29 transmits to the server S information regarding the type of object, the location information of the fixed object, and information indicating the visible feature whose visibility is obstructed by the fixed object. Furthermore, in addition to this information, an image of the fixed object captured by the camera 17 may also be transmitted to the server S. The location information of the fixed object may be calculated, for example, from the relative position of the fixed object from the vehicle M obtained by a known position measuring device such as a LIDAR (Light Detection and Ranging) sensor mounted on the vehicle M, and the location information of the vehicle M acquired by the GPS receiver 19.
[0079] When server S receives information about fixed objects and information indicating visible features, it adds the information about those fixed objects to the map information it holds. This allows the map information to be kept up-to-date, even when, for example, a new building is constructed.
[0080] Server S may, for example, exclude a visible feature whose visibility is obstructed by the fixed object from the features visible from the corresponding road section. For example, in the example in Figure 11, feature FT1 is excluded from the features visible from road section RL1.
[0081] Certain obstacles, such as real estate, will constantly obstruct the visibility of the feature FT1 from road section RL1, not only for vehicle M but also for other vehicles traveling on road section RL1.
[0082] Therefore, by excluding such objects from the visible features associated with road section RL1 in the map information, it is possible to avoid situations where visible features are not visible.
[0083] In this way, the feature guidance device 10 transmits information indicating visible features and information about objects that obstruct the visibility of such features to the server S. This prevents situations such as the visible features not being visible from the road section to which they are associated, and ensures smoother feature guidance from the next time onward.
[0084] Furthermore, by adding location and type information of objects that obstruct the visibility of visible features to the map data, the map data can be kept up-to-date.
[0085] Figure 12 is a flowchart showing the feature guidance routine RT1 executed by the control unit 29 of the feature guidance device 10. The control unit 29 starts the feature guidance routine RT1 when, for example, power is turned on to the feature guidance device 10.
[0086] When the control unit 29 starts the feature guidance routine RT1, it determines whether or not there are any features visible from the road section that the vehicle M is currently traveling on in the map data (step S101). In step S101, for example, the control unit 29 refers to the map information DB27A and determines that there are visible features if there are visible features associated with the road section that the vehicle M is currently traveling on.
[0087] When the control unit 29 determines that visible objects exist (step S101: YES), it acquires an image of the area around the moving vehicle M from the camera 17 (step S102). In step S102, the control unit 29 functions as an image acquisition unit that performs the acquisition of surrounding images (image acquisition step).
[0088] After step S102 is executed, the control unit 29 identifies a feature prediction area in the surrounding image where visible features are expected to appear (step S103). This feature prediction area is a determination area for determining whether or not visible features are captured in the surrounding image. In step S103, for example, the control unit 29 identifies the determination area based on the orientation of the visible features with respect to the shooting direction at the time the surrounding image S1 was taken and the distance between the vehicle M and the visible features. In step S103, the control unit 29 functions as a region identification unit.
[0089] In step S103, for example, if there are multiple visible features associated with the road section on which the vehicle M is traveling, the control unit 29 selects the feature closest to the current position of the vehicle M and identifies a determination area for the selected visible feature.
[0090] After step S103 is executed, the control unit 29 determines whether or not an object different from a visible feature is captured in the determination area of the surrounding image (step S104).
[0091] In step S104, for example, the control unit 29 compares an image of a visible feature identified in step S101 from among the images pre-stored in the map information DB27A of the storage unit 27 with the determination area of the surrounding image S1 to determine whether or not an object different from a visible feature appears in the determination area. In step S104, the control unit 29 functions as a determination unit that determines whether or not an object different from a visible feature appears in the feature prediction area of the surrounding image.
[0092] If the control unit 29 determines in step S104 that no features other than visible features are captured in the image (step S104: NO), that is, if it determines that visible features are captured in the image, it outputs normal guide information regarding visible features (step S105).
[0093] For example, in step S105, the control unit 29 notifies the user that a visible feature is visible and provides guidance information indicating the direction in which the visible feature is visible, either through audio output from the speaker 15 or by displaying an image on the display. In step S105, for example, an announcement such as "You can see the large torii gate to the left" is made.
[0094] If the control unit 29 determines in step S104 that an object different from a visible object is captured in the image (step S104: YES), that is, if it determines that no visible object is captured, it identifies the type of object captured in the determination area (step S106).
[0095] In step S106, for example, image analysis is performed on the judgment area of the surrounding image to identify the type of object that is different from the visible features.
[0096] In step S106, for example, the control unit 29 identifies the type of object appearing in the surrounding image by comparing the determination area of the surrounding image with images of various objects stored in the image DB27B of the storage unit 27, for example, by pattern matching.
[0097] In step S106, the control unit 29 functions as an object type identification unit that identifies the type of object appearing in the determination area by performing image analysis on the surrounding image.
[0098] After executing step S106, the control unit 29 outputs guide information including the type of object (step S107). In step S107, for example, the control unit 29 informs the user of information indicating the type of object visible in the determination area by audio output from the speaker 15 or by displaying an image on the display.
[0099] For example, in step S107, guidance information indicating that there is a visible feature beyond the object identified in step S106 is presented via audio or screen display.
[0100] In step S107, the control unit 29 functions as a notification unit that performs a notification (notification step) of information regarding the type of object if a feature different from the predetermined feature appears in the determination area. The control unit 29 also notifies information regarding the type of object when the vehicle M is in a position where the predetermined feature can be seen.
[0101] By providing information about the type of object different from the visible features, users can understand why the visible features are not visible. Furthermore, since this guidance information can be displayed in real time when vehicle M is in a position where the visible features are visible, any questions or concerns users may have due to the inability to see the features can be resolved immediately.
[0102] After executing step S107, the control unit 29 executes a transmission subroutine (S108) and transmits information including the type of object to the outside. In step S108, for example, information such as the type of object visible in the judgment area, as well as visible features that are not visible due to the object, i.e., features that were the subject of judgment in step S104, is transmitted to the external server S.
[0103] If the control unit 29 determines after step S105, after step S108, or in step S101 that there are no visible features (step S101: NO), it terminates the feature guidance routine RT1 and repeatedly executes the feature guidance routine RT1.
[0104] Figure 13 is a flowchart showing an example of a transmission subroutine RT2, which is executed in step S108 of the feature guidance routine RT1.
[0105] When the control unit 29 starts the transmission subroutine RT2, it determines whether the type of object visible in the determination area identified in step S106 corresponds to a predetermined object (step S201). A predetermined object is, for example, a fixed object such as real estate that permanently obstructs the visibility of features associated with visible features in the road section.
[0106] For example, in step S201, a table for determining whether or not an object is a predetermined object is stored in the storage unit 27, and based on this table, it is determined whether or not the type of object shown in the determination area corresponds to a predetermined object.
[0107] If the control unit 29 determines in step S201 that it is a predetermined object (step S201: YES), it transmits to the outside information indicating the type of object captured in the determination area and the visible feature that was the subject of determination in step S104 (step S202).
[0108] In step S202, for example, information indicating the type of object and visible features is sent to the server S, and the map information held by the server S is updated.
[0109] After executing step S202, or if the control unit 29 determines in step S201 that the object is not a predetermined object (step S201: NO), it terminates the transmission subroutine RT2.
[0110] In this embodiment, the feature guidance provided by the feature guidance device 10 may be performed as part of the route guidance flow provided by the navigation device.
[0111] As described in detail above, the feature guidance device 10 as a notification device in Embodiment 1 acquires an image of the surroundings of the moving object, taken from the current position of the moving object by a camera placed on the moving object, identifies a feature prediction area in the surrounding image where a predetermined feature is expected to be visible, and if an object different from the predetermined feature is visible in the feature prediction area, information including the type of the object can be notified to the user or an external server, etc. The predetermined feature is a feature that can be determined from map information to be visible from the current position of the moving object, and is a feature that is associated in map information with the road section on which the vehicle M in this embodiment is currently traveling.
[0112] Therefore, according to the notification device of Example 1, if the visibility of a predetermined feature is obstructed by an object different from the predetermined feature, the user or server can be notified of the type of object causing the obstruction.
[0113] Therefore, when providing guidance using landmarks visible from the road, it is possible to provide a notification device that can prevent users from becoming confused when such landmarks are not visible from the road.
[0114] In this embodiment, when the notification device notifies the user of an object of a different type than a predetermined feature, it may be possible to alleviate user anxiety or dissatisfaction caused by the inability to see a visible feature, thereby reducing the user's psychological stress.
[0115] Furthermore, when the notification device of this embodiment notifies the server of an object type different from a predetermined feature, it becomes possible to keep the map information up-to-date. In addition, it is possible to avoid situations where visible features become permanently invisible due to the construction of new buildings, etc., and by using updated map information, route guidance can be provided with high accuracy. [Examples]
[0116] Referring to Figures 14 to 20, the functions of the route guidance device 50 as a notification device according to Embodiment 2 will be described. The route guidance device 50 is configured similarly to the feature guidance device 10 described using Figures 1 to 3, but some of the information stored in the memory unit 27 and some of the processing performed by the control unit 29 are different. The parts that are common with Embodiment 1 will be omitted from the explanation below.
[0117] The control unit 29 executes a route guidance program, which serves as a notification program in this embodiment, to provide route guidance. When the route guidance program is executed, for example, the control unit 29 receives input of route search conditions, such as a destination, via the microphone 11 or the touch panel display 13, and generates a route based on map information. Subsequently, the control unit 29 provides route guidance according to the generated route.
[0118] The memory unit 27 stores the route guidance program. The map information DB 27A of the memory unit 27 stores data that associates landmark features with each guidance point in the route guidance process. The map information DB 27A also stores the location information of these landmark features.
[0119] Figure 14 shows an example of data for landmark features associated with guide points stored in the map information DB27A, represented as Table TB3. In Table TB3, each guide point, such as GP1, GP2, etc., listed in the Guide Point ID column, is associated with a landmark feature that serves as a reference point at that respective guide point.
[0120] In this specification, a guidance point is a point along the vehicle M's route where driving operations such as turning right or left are required, or where a warning is necessary. For example, in the route guidance of this embodiment, a guidance point is a point where, when vehicle M reaches a predetermined distance, voice guidance is given prompting the driver to turn right or left.
[0121] Furthermore, in this specification, a geographical feature that serves as a landmark at a guided point is referred to as a "landmark feature." For example, landmark features include buildings and other structures that serve as landmarks when confirming that an intersection requires a right or left turn. For example, features located within a predetermined distance from the guided point, features facing the guided point, and features with a height above a predetermined height can all be considered landmark features at the guided point.
[0122] In Table TB3, the guide point and the road section adjacent to the guide point are associated with landmark features that are visible from the guide point. Note that different landmark features may be associated with each road section connected to the guide point.
[0123] Figure 15 is a diagram showing an example of location information for landmark features, which is part of the map information stored in the map information DB27A of the storage unit 27, as table TB4. As shown in Figure 15, each landmark feature is associated with a landmark feature ID and the location information of the landmark feature identified by the landmark feature ID.
[0124] Figure 16 shows an example of a route guidance map displayed on the display in this embodiment. In Figure 16, the current position of vehicle M, the intersection N1 which is a guidance point on route A, and the landmark feature LM associated with intersection N1 are displayed superimposed on an image of route A of vehicle M, which includes road section RL2.
[0125] As shown in Figure 16, vehicle M is traveling on a road section RL2 where the landmark feature LM is visible. For example, based on the vehicle M's current position, direction of travel, and the location information of the landmark feature LM included in the map data, the direction in which the feature LM is visible relative to the vehicle M's direction of travel and the distance between vehicle M and the landmark feature LM can be calculated.
[0126] Figure 17 shows an example of the driving situation of vehicle M during the route guidance shown in Figure 16. In the situation shown in Figure 17, vehicle M is traveling along road section RL2 within route A, following the guidance of the route guidance device 50, and is approaching intersection N1. Route A is a route that turns right at intersection N1.
[0127] From the vehicle M's current position, the landmark feature LM, which serves as a landmark when confirming the intersection to turn right at, is visible. When the vehicle M approaches a predetermined distance from the landmark feature LM, the control unit 29 of the route guidance device 50 acquires an image of the area around the vehicle M from the camera 17 and identifies a determination area in the surrounding image where the landmark feature LM is expected to be visible.
[0128] When the route guidance device 50 determines that a landmark is visible in the detection area, it uses the landmark LM as a landmark and makes an announcement urging the driver to turn right at intersection N1. In the situation shown in Figure 17, for example, the speaker 15 outputs the voice message, "50m ahead, turn right at the intersection where the XX Hotel is located." The driver of vehicle M can use the landmark LM as a landmark and recognize that the intersection where the landmark LM is located is the intersection where they should turn right.
[0129] Figure 18 shows another example of the driving situation of vehicle M during the route guidance shown in Figure 16. In Figure 18, in addition to the situation in Figure 17, bus OB3 is driving to the left front of vehicle M. From vehicle M's current position, bus OB3 is an obstruction, making it impossible to see the landmark object LM.
[0130] In the situation shown in Figure 18, when the control unit 29 of the route guidance device 50 identifies a determination area using the method described above, a bus OB3, which is a different object from the landmark feature LM, appears in that determination area.
[0131] The route guidance device 50 determines that the landmark feature LM is not visible in the determination area, that is, that it is an object different from the landmark feature. It then outputs guidance information that includes information about the different object, i.e., an obstacle that obstructs the visibility of the landmark feature LM.
[0132] For example, in the situation shown in Figure 18, the speaker 15 outputs voice messages such as, "Turn right at the intersection 50m ahead where the XX Hotel is located. The XX Hotel is beyond the bus to your left." This allows the driver of vehicle M to get a rough idea of the location of the landmark LM, even if they cannot see the landmark LM.
[0133] Figure 19 is a flowchart showing an example of a route guidance routine RT3, which is performed by the control unit 29 of the route guidance device 50. The control unit 29 starts the route guidance routine RT3, for example, when the route of vehicle M is determined.
[0134] When the control unit 29 starts the route guidance routine RT3, it determines whether the vehicle M's current position has reached a predetermined distance from the next guidance point (step S301). In step S301, for example, the control unit 29 reads map information from the map information DB27A, searches for the guidance point closest to the vehicle M's current position based on the position information of each guidance point on the route, calculates the distance between the guidance point and the vehicle M's current position, and determines whether it is within the predetermined distance.
[0135] In step S301, the control unit 29 determines that it has reached a predetermined distance from the next guide point (step S301: YES), and then, referring to map information (for example, Figure 14), searches for a landmark associated with the next guide point (step S302).
[0136] After executing step S302, the control unit 29 determines whether or not there is a landmark object at the next guide point (step S303).
[0137] In step S303, if the control unit 29 determines that there are no landmarks (step S303: NO), it supplies normal guide information that does not use landmarks to the speaker 15 or display via the output unit 25 (step S304). In step S304, for example, an announcement is made to encourage turning right at intersection N1, using a sign indicating the name of the intersection or a traffic light as a landmark.
[0138] In step S303, the control unit 29 determines that a landmark object exists (step S303: YES), then executes the landmark object guidance subroutine (step S305), and outputs guidance information according to the situation.
[0139] If the control unit 29 determines in step S301 that the vehicle has not reached the next guide point within a predetermined distance (step S301: NO), it determines whether or not the vehicle has arrived at the destination after the execution of step S304 or step S305 (step S306).
[0140] If the control unit 29 determines in step S306 that the destination has not been reached (step S306: NO), it returns to step S301 and determines again whether the current position of vehicle M is within a predetermined distance to the next guidance point. If the control unit 29 determines in step S306 that the destination has been reached (step S306: YES), it terminates the route guidance routine RT3.
[0141] Figure 20 is a flowchart showing the landmark feature guidance subroutine RT4, which is an example of a subroutine executed in step S305 of the route guidance routine RT3. The landmark feature guidance subroutine RT4 is similar in content to steps S102 to S108 of the feature guidance routine RT1 described in Figure 12, differing only in that it handles landmark features instead of visible features, so the detailed content will be omitted.
[0142] When the control unit 29 starts the landmark feature guidance subroutine RT4, it acquires an image of the area around the vehicle M from the camera 17 (step S401) and identifies a determination area, which is the area in the surrounding image where the landmark feature is expected to be visible (step S402). As described above, in step S402, the determination area is identified, for example, based on the orientation of the landmark feature with respect to the shooting direction at the time the surrounding image was taken and the distance between the vehicle M and the landmark feature.
[0143] If the control unit 29 determines that no object other than the landmark is visible in the determination area (step S403: NO), it outputs guidance information using the landmark as a landmark (step S404). For example, if the guidance is to encourage a right turn at an intersection using the hotel, which is the landmark, it will make an announcement such as, "50m ahead, turn right at the intersection where the XX Hotel is located."
[0144] When the control unit 29 determines that an object different from the landmark is visible in the determination area (step S403: YES), it identifies the type of object visible in the determination area (step S405) and outputs guide information including the type of object (step S406). For example, if the landmark is "○○ Hotel" and the type of object is a bus, an announcement such as "○○ Hotel is beyond the bus to the left and in front" will be made.
[0145] Subsequently, the control unit 29 executes a transmission subroutine (step S407) and transmits information about the type of object to the server S. For example, the transmission subroutine in step S407 may have the same content as the transmission subroutine RT2 described in Figure 13. Also, unlike the transmission subroutine RT2, the transmission subroutine in step S407 may transmit data about the obstacle to the server not only when the obstacle obstructing the visibility of the landmark is a fixed object, but also when it is a moving object such as another vehicle. As a result, data on how often the landmark is not visible due to traffic conditions, such as when visibility is obstructed by a large truck, is accumulated in the server S. For example, the accumulated data can be used as a basis for deciding whether to exclude landmarks that are frequently obstructed by traffic conditions from the landmarks at that guidance point.
[0146] In addition, the transmission subroutine 407 may be omitted in the landmark feature guidance subroutine RT4.
[0147] Furthermore, the route guidance of Example 2 may be combined with the feature guidance of Example 1. For example, in route guidance, guidance may be provided at points other than the guidance point, regarding features associated with the road section on which the vehicle M is traveling, and when an object that obstructs the visibility of the feature appears in the judgment area of the surrounding image, information regarding the type of the object may be notified to the user.
[0148] As explained above, according to the route guidance device 50 as a notification device of Embodiment 2, when providing guidance at a guidance point using landmarks as reference points, if other vehicles or other objects obstruct visibility and the landmarks become invisible, the device can identify the type of object causing the obstruction and provide information including the type of object. Therefore, even if the landmarks are not visible, the user can find out the reason why they are not visible.
[0149] Furthermore, when providing route guidance using landmarks visible from the road, it is possible to provide a notification device that can prevent users from becoming confused when such landmarks are not visible from the road.
[0150] Furthermore, the route guidance device 50 can also inform the user of the type of obstacle object and the location of the landmark object. This allows the user to get a rough idea of the location of the landmark object.
[0151] In Examples 1 and 2 described above, the moving object was described as a vehicle, but the invention is not limited to this. The moving object may be a person. The notification device of the present invention may provide notifications regarding route guidance for walking. The notification device of the present invention is not limited to being mounted on a vehicle or other movable device, but may also be mounted on a portable device such as a smartphone or tablet carried by the user, or a wearable device, etc. It is sufficient that the device is configured to acquire a surrounding image taken from the current location of the moving object.
[0152] In the above-described examples 1 and 2, the processing of the region identification unit, determination unit, and object type identification unit was explained in the control unit 29, but these processes may also be performed on the server S. The notification device of the present invention may also be a device that transmits image data to the server S and receives information indicating the type of object from the server S and provides notification.
[0153] Although the above-described examples of sending and receiving data with server S were explained in Examples 1 and 2, the present invention is not limited to these examples, and the notification device does not necessarily have to communicate with an external server.
[0154] In addition, the road section in the above embodiment may be each of the lanes within the road section.
[0155] Furthermore, the map information in the map information DB27A in the above embodiment may be updated periodically, for example, by obtaining update data via the network NW when the map information in the server S is updated.
[0156] The configurations in the above-described embodiments are merely illustrative examples and can be appropriately selected and modified depending on the application and other factors. [Explanation of Symbols]
[0157] 10. Pathfinding device 11 Mike 13 Touch panel display 17 Cameras 19 GPS receiver 22 sensors 23 Input section 24 System Bus 25 Output section 27 Memory section 29 Control Unit 31 Transmitter / Receiver
Claims
1. A guidance device that provides route guidance for a moving object using designated landmarks as markers, An image acquisition unit that acquires a surrounding image which is an image of the surroundings of the moving object taken from the moving object, A notification unit provides, in conjunction with the route guidance, information to the occupants of the mobile body, including that the predetermined feature is not visible and information regarding the type of object, when an object different from the predetermined feature appears in a first region in the surrounding image where the predetermined feature is expected to appear, which is determined based on map information including the location information of the predetermined feature and the position of the mobile body at the time the surrounding image was taken, and the object obstructs the visibility of the predetermined feature. Equipped with, The guidance device is characterized in that the notification unit includes in the guidance information movement instruction information for moving the moving body so that an object different from the predetermined feature does not obstruct the visibility of the predetermined feature.
2. The guidance device according to claim 1, characterized in that the notification unit notifies information prompting a lane change as movement instruction information.
3. The aforementioned surrounding image is an image captured by a camera that has a predetermined direction of shooting from the moving object. The region identification unit identifies the first region based on the orientation of the predetermined feature with respect to the shooting direction and the distance between the moving object and the predetermined feature at the time the surrounding image was taken, The guidance device according to claim 1 or 2, characterized in that the notification unit notifies the guidance in conjunction with the route guidance when an object different from the predetermined feature appears in the first region identified by the region identification unit and obstructs the visibility of the predetermined feature.
4. The map information includes an image of the predetermined feature, and the guidance device has a determination unit that determines whether an object different from the predetermined feature appears in the area of the surrounding image and obstructs the visibility of the predetermined feature by comparing the image of the predetermined feature with an image of the first region in the surrounding image. The guidance device according to any one of claims 1 to 3, characterized in that the notification unit notifies the guidance in conjunction with the route guidance when the determination unit determines that an object different from the predetermined feature has appeared and obstructs the visibility of the predetermined feature.
5. It has an object type identification unit that identifies the type of object by performing image analysis on the surrounding image, The guidance device according to any one of claims 1 to 4, characterized in that the notification unit notifies the guidance, which includes information relating to the type of object identified by the object type identification unit, in conjunction with the route guidance.
6. A guidance method that uses designated landmarks to guide a moving object along its route, The image acquisition unit performs an image acquisition step in which it acquires an ambient image, which is an image of the area around the moving object taken from the moving object, A notification step in which the notification unit notifies the occupants of the mobile body, along with the route guidance, that the predetermined feature is not visible and provides information regarding the type of object, when an object different from the predetermined feature appears in a first region in the surrounding image where the predetermined feature is expected to appear, which is determined based on map information including the location information of the predetermined feature and the position of the mobile body at the time the surrounding image was taken, and the object obstructs the visibility of the predetermined feature; Includes, The notification step is characterized by including in the guidance information movement instruction information for moving the moving body so that an object different from the predetermined feature does not obstruct the visibility of the predetermined feature.
7. A guidance program that is executed by a guidance device equipped with a computer and provides route guidance for a moving object using predetermined landmarks, wherein the computer has, Image acquisition step: Acquires a surrounding image which is an image of the surroundings of the moving object taken from the moving object, A notification step, accompanied by route guidance, in which, if an object different from the predetermined feature appears in a first region in the surrounding image where the predetermined feature is expected to appear, determined based on map information including the location information of the predetermined feature and the position of the moving body at the time the surrounding image was taken, and obstructs the visibility of the predetermined feature, a notification is given to the occupant of the moving body, accompanied by route guidance, which includes information that the predetermined feature is not visible and information regarding the type of the object. Make it run, The notification step is a guidance program characterized by including in the guidance information movement instruction information for moving the moving body so that an object different from the predetermined feature does not obstruct the visibility of the predetermined feature.
8. A computer-readable recording medium characterized by storing the guidance program described in claim 7.