Information notification system and information notification method

Abstract

To improve convenience for a user concerning a technique for notifying a user of a vehicle of a speed limit.SOLUTION: An information notification system notifies a user of a vehicle of information. The information notification system determines whether a first condition is established, which shows that one or more time designation speed limit signs to be recognized by a camera mounted on the vehicle designate a whole time zone of one day on the whole. When the first condition is not established, the information notification system notifies at least the user of a legal speed limit on a road where the one or more time designation speed limit signs are arranged based on reference information related to the legal speed limit in an area where the vehicle is used.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a technique for notifying information to a user of a vehicle. In particular, the present disclosure relates to a technique for notifying a user of a vehicle of information regarding a speed limit.

Background Art

[0002] Patent Document 1 discloses a sign display device for a vehicle. The sign display device detects a sign in front of the vehicle by a camera. When first sign information regarding a first maximum speed in a first section and second sign information regarding a second maximum speed in a second section are detected, the sign display device displays the first sign information and the second sign information side by side.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A technique for recognizing a speed limit sign and notifying a user of a vehicle of the speed limit specified by the speed limit sign is useful. However, among the types of speed limit signs, there is also a time-specified speed limit sign that specifies the speed limit in a specific time zone of a day. Even if the speed limit specified by the time-specified speed limit sign is notified to the user, the speed limit outside the specific time zone is not known to the user. For example, when the current time is outside a specific time zone, the notification of the speed limit in the specific time zone is not particularly useful. There is room for improvement in the technique for notifying a user of a vehicle of the speed limit.

Means for Solving the Problems

[0005] A first aspect relates to an information notification system for notifying information to a user of a vehicle. The information notification system is One or more processors, One or more storage devices that store reference information regarding the legal speed limits in the area where the vehicle is used. It is equipped with. One or more processors, The first condition is met, which is that one or more time-limited speed limit signs recognized by the camera mounted on the vehicle collectively specify all time periods of the day. If the first condition is not met, the user will be notified of the statutory speed limit on roads where one or more time-limited speed limit signs are installed, based on the reference information.

[0006] The second aspect relates to information notification methods that use computers to inform vehicle users of information. The method of information notification is: The first condition is met, which is that one or more time-limited speed limit signs recognized by the camera mounted on the vehicle collectively specify all time periods of the day. If the first condition is not met, the user shall be notified of the statutory speed limit on roads where one or more time-limited speed limit signs are installed, based on reference information regarding statutory speed limits in the area where the vehicle is used. Includes. [Effects of the Invention]

[0007] According to this disclosure, if the recognized time-limited speed limit sign does not specify all time periods of the day as a whole, at least the statutory speed limit will be communicated to the vehicle user. This communication of the statutory speed limit is useful to the user and improves user convenience. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram illustrating the outline of the information notification system according to the first embodiment. [Figure 2] This is a conceptual diagram illustrating an example of information notification processing related to time-specified speed limit signs according to the first embodiment. [Figure 3] This is a conceptual diagram illustrating another example of the information notification process regarding time-limited speed limit signs according to the first embodiment. [Figure 4] This is a block diagram showing an example configuration of the information notification system according to the first embodiment. [Figure 5] This is a conceptual diagram illustrating an example of reference information relating to the first embodiment. [Figure 6] This is a conceptual diagram illustrating another example of reference information relating to the first embodiment. [Figure 7] This is a flowchart showing the information notification process regarding time-specified speed limit signs according to the first embodiment. [Figure 8] This is a conceptual diagram illustrating an example of information notification processing related to time-specified speed limit signs according to the second embodiment. [Figure 9] This is a conceptual diagram illustrating another example of the information notification process regarding time-limited speed limit signs according to the second embodiment. [Figure 10] This is a flowchart showing the information notification process regarding time-specified speed limit signs according to the second embodiment. [Modes for carrying out the invention]

[0009] Embodiments of this disclosure will be described with reference to the attached drawings.

[0010] 1. First Embodiment 1-1. Overview of the Information Notification System FIG. 1 is a conceptual diagram for explaining the outline of an information notification system 100 according to the present embodiment. The information notification system 100 notifies the user O of the vehicle 1 of information regarding the vehicle 1. For example, the user O of the vehicle 1 is an operator of the vehicle 1. The operator of the vehicle 1 may be a driver riding in the vehicle 1, or may be a remote operator remotely operating (remote driving, remote assistance) the vehicle 1. The user O of the vehicle 1 may be a passenger riding in the vehicle 1. The vehicle 1 may be an autonomous vehicle. The information notification system 100 may be mounted on the vehicle 1, may be included in a remote operator terminal operated by a remote operator, or may be distributedly arranged between the vehicle 1 and the remote operator terminal.

[0011] For example, the information notification system 100 notifies the user O of information regarding the speed limit of the road on which the vehicle 1 is traveling. The speed limit of the road on which the vehicle 1 is traveling is obtained, for example, from a speed limit sign S installed on that road. The information notification system 100 recognizes the speed limit sign S installed on the road on which the vehicle 1 is traveling, and notifies the user O of the speed limit specified by the recognized speed limit sign S.

[0012] For the recognition of the speed limit sign S, the camera 2 mounted on the vehicle 1 is used. The camera 2 photographs the surrounding situation of the vehicle 1 and acquires an image IMG showing the surrounding situation of the vehicle 1. The information notification system 100 acquires the image IMG obtained by the camera 2, and recognizes the speed limit sign S by analyzing the image IMG. Typically, the information notification system 100 recognizes the speed limit sign S from the image IMG by using image recognition AI (Artificial Intelligence). The image recognition AI is generated in advance through a learning method such as deep learning. For example, the image recognition AI is learned so as to be able to detect various signs from the image IMG and further identify the types of the detected signs. As another example, the image recognition AI may be learned so as to be able to detect the speed limit sign S from the image IMG. In any case, the speed limit sign S can be recognized by using the camera 2 and the image recognition AI.

[0013] The information notification system 100 includes a notification device for notifying user O of information. The information notification system 100 notifies user O of the speed limit specified by the recognized speed limit sign S through the notification device.

[0014] For example, the notification device includes a display device. The information notification system 100 displays the image of the recognized speed limit sign S itself on the display device. Alternatively, the information notification system 100 may read the number described on the speed limit sign S based on the image of the recognized speed limit sign S. In other words, the information notification system 100 may recognize the speed limit specified by the recognized speed limit sign S based on the image of the recognized speed limit sign S. In this case, the information notification system 100 displays the recognized speed limit on the display device. The display of the speed limit may be an icon or characters (numbers).

[0015] As another example, the notification device may include a speaker. In this case, the information notification system 100 outputs voice information notifying of the speed limit from the speaker.

[0016] The technology of notifying the speed limit in this way is useful for user O of vehicle 1.

[0017] 1-2. Time-Specified Speed Limit Sign Among the types of speed limit signs S, there is also a "time-specified speed limit sign ST" that specifies the speed limit in a specific time period of a day. Typically, the time-specified speed limit sign ST includes, in addition to the main body part that specifies the speed limit, an auxiliary part that specifies the time period to which the speed limit applies. The image recognition AI can determine whether the speed limit sign S is a time-specified speed limit sign ST based on the presence or absence of the auxiliary part that specifies the time period. The image recognition AI may recognize the time period specified by the time-specified speed limit sign ST.

[0018] Even if the speed limit specified by the time-limited speed limit sign ST is notified to the user O of vehicle 1, the user O will not know the speed limit outside of that specific time period. For example, if the current time is outside of the specific time period, notification of the speed limit for that time period is not particularly useful. Therefore, this embodiment provides a technology that can further improve convenience for the user O of vehicle 1.

[0019] Figure 2 is a conceptual diagram illustrating an example of information notification processing regarding a time-limited speed limit sign ST. The information notification system 100 recognizes a time-limited speed limit sign ST installed on the road on which vehicle 1 is traveling. In the example shown in Figure 2, the time period specified by the time-limited speed limit sign ST is from 6:00 to 19:00. When such a time-limited speed limit sign ST is recognized, the information notification system 100 obtains information on the "statutory speed limit" on the road where the time-limited speed limit sign ST is installed.

[0020] More specifically, the information notification system 100 holds "reference information REF" regarding the legal speed limit in the area where vehicle 1 is used. Here, the area where vehicle 1 is used refers to a country or state. Traffic laws are established on a regional basis. Reference information REF is pre-generated based on the traffic laws established for each region. Reference information REF may be generated to cover multiple regions.

[0021] For example, reference information REF associates the design of a speed limit sign S used in a region with the legal speed limit on the road where that speed limit sign S is installed. The design of the speed limit sign S includes the number of the speed limit specified by that sign S. For example, a speed limit sign S specifying a speed limit of 100 km / h or more is installed on an expressway (motor vehicle-only road). The legal speed limit on expressways in that region is determined by traffic laws. Therefore, reference information REF can be generated that associates the design of the speed limit sign S with the legal speed limit.

[0022] However, the above example is not the only example of a reference (REF). Various examples of references (REF) will be described later.

[0023] The information notification system 100 acquires information on the statutory speed limit on the road where the recognized time-specified speed limit sign ST is installed, based on the reference information REF. In the example shown in Figure 2, the information notification system 100 notifies the user O of vehicle 1 of the statutory speed limit in addition to the speed limit specified by the recognized time-specified speed limit sign ST.

[0024] As a result, user O of vehicle 1 can know not only the speed limit applicable during a specific time period, but also the statutory speed limit applicable outside of that time period. Such information on statutory speed limits is useful to user O and improves user convenience. For example, if the current time is outside of a specific time period, user O can know the statutory speed limit applicable at the current time, which is preferable.

[0025] Figure 3 is a conceptual diagram illustrating another example of information notification processing regarding time-limited speed limit signs ST. In the example shown in Figure 3, multiple time-limited speed limit signs ST-1 and ST-2, specifying different time periods, are recognized simultaneously. The time period specified by time-limited speed limit sign ST-1 is from 6:00 to 19:00, and the time period specified by time-limited speed limit sign ST-2 is from 19:00 to 6:00. In other words, the multiple time-limited speed limit signs ST-1 and ST-2 collectively specify all time periods of the day.

[0026] In the example shown in Figure 3, the information notification system 100 notifies the user O of vehicle 1 of multiple speed limits specified by multiple time-specified speed limit signs ST-1 and ST-2. This allows user O to know the speed limits for all time periods of the day.

[0027] Since user O can know the speed limits for all time periods, it is not necessarily required to notify user O of the legal speed limit. Therefore, the information notification system 100 may refrain from processing the acquisition of legal speed limit information based on the reference information REF. Alternatively, the information notification system 100 may refrain from processing the notification of the legal speed limit to user O. In either case, the processing load on the information notification system 100 is reduced. In addition, since user O is not notified of information more than necessary, the feeling of annoyance to user O is suppressed.

[0028] 1-3. Example of an Information Notification System Configuration Figure 4 is a block diagram showing an example configuration of the information notification system 100 according to this embodiment. The information notification system 100 includes a communication interface 110, a notification device 120, one or more processors 130 (hereinafter simply referred to as processor 130), and one or more storage devices 140 (hereinafter simply referred to as storage devices 140).

[0029] The communication interface 110 communicates with the outside of the information notification system 100. For example, the communication interface 110 communicates with the camera 2 mounted on the vehicle 1. If user O is a remote operator who remotely controls (remotely drives, remotely assists) the vehicle 1, and the information notification system 100 is included in the remote operator terminal, the communication interface 110 communicates with the vehicle 1.

[0030] The notification device 120 notifies the user O of vehicle 1 of various information. If user O is the driver in vehicle 1, the notification device 120 is installed in vehicle 1. If user O is a remote operator, the notification device 120 is included in the remote operator terminal. For example, the notification device 120 includes a display device. Examples of display devices include displays, head-up displays (HUDs), etc. Another example is that the notification device 120 may include a speaker.

[0031] The processor 130 performs various processes. For example, the processor 130 includes a CPU (Central Processing Unit).

[0032] The storage device 140 stores various types of information. Examples of storage devices 140 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0033] The control program 200 is a computer program for controlling the information notification system 100. The processor 130 executes the control program 200, thereby enabling various processes by the information notification system 100. The control program 200 is stored in the storage device 140. Alternatively, the control program 200 may be recorded on a computer-readable recording medium.

[0034] The processor 130 acquires an image IMG captured by a camera 2 mounted on the vehicle 1 via the communication interface 110. The image IMG is stored in the storage device 140.

[0035] Furthermore, the processor 130 acquires navigation information NAV from the vehicle 1 via the communication interface 110. The navigation information NAV includes map information and location information of the vehicle 1. The navigation information NAV may also include information about the region (country, state, etc.) where the vehicle 1 is located. The navigation information NAV is stored in the storage device 140.

[0036] The storage device 140 further stores reference information REF. Reference information REF is information about the legal speed limits in the region (country, state, etc.) where the vehicle 1 is used. Reference information REF may be generated to cover multiple regions.

[0037] Figure 5 is a conceptual diagram illustrating an example of reference information REF. In the example shown in Figure 5, reference information REF associates the design of a speed limit sign S used in a region with the legal speed limit on the road where that speed limit sign S is installed. The design of the speed limit sign S includes the number of the speed limit specified by that sign S. For example, a speed limit sign S specifying a speed limit of 100 km / h or more is installed on an expressway (motor vehicle-only road). The legal speed limit on the expressway in that region is determined by traffic laws. Therefore, reference information REF can be generated that associates the design of the speed limit sign S with the legal speed limit.

[0038] The reference information REF exemplified in Figure 5 may be an AI model that takes an image of a speed limit sign S as input and outputs the legal speed limit. Such an AI model can be pre-generated through learning methods such as deep learning.

[0039] Figure 6 is a conceptual diagram illustrating another example of reference information REF. In the example shown in Figure 6, reference information REF shows the correspondence between road types and legal speed limits in the area. In other words, reference information REF shows the legal speed limits for each road type in the area.

[0040] As another example, the reference information REF may be map information in which the legal speed limit for each location is registered.

[0041] 1-4. Processing Flow Figure 7 is a flowchart showing the information notification process for time-limited speed limit sign ST according to this embodiment.

[0042] In step S100, the processor 130 recognizes one or more time-limited speed limit signs ST based on the image IMG obtained by the camera 2 mounted on the vehicle 1. Typically, the image recognition AI described above is used to recognize the time-limited speed limit signs ST. In the example shown in Figure 2, one time-limited speed limit sign ST is recognized. In the example shown in Figure 3, multiple time-limited speed limit signs ST-1, ST-2, which specify different time periods, are recognized simultaneously.

[0043] In step S110, the processor 130 determines whether the first condition is met. The first condition is that one or more recognized time-limited speed limit signs ST collectively specify all time periods of the day. In the example shown in Figure 2, the first condition is not met, while in the example shown in Figure 3, the first condition is met. If the first condition is met (step S110; Yes), the process proceeds to step S120. On the other hand, if the first condition is not met (step S110; No), the process proceeds to step S130.

[0044] In step S120, the processor 130 notifies the user O via the notification device 120 of the multiple speed limits specified by each of the recognized multiple time-specified speed limit signs ST (see Figure 3). At this time, preferably, the processor 130 refrains from the process of acquiring information on the legal speed limit based on the reference information REF. Alternatively, the processor 130 refrains from the process of notifying the user O of the legal speed limit.

[0045] In step S130, the processor 130 identifies the legal speed limit on the road where the recognized time-limited speed limit sign ST is installed. More specifically, the processor 130 identifies the legal speed limit based on the reference information REF stored in the storage device 140. If necessary, the processor 130 determines the region (country, state, etc.) where vehicle 1 is located based on the navigation information NAV and uses the reference information REF for that region.

[0046] The processing in the case of reference information REF shown in Figure 5 is as follows: Reference information REF associates the design of the speed limit sign S with the legal speed limit on the road where the speed limit sign S is installed. Reference information REF may also be an AI model that takes an image of the speed limit sign S as input and outputs the legal speed limit. The processor 130 obtains the legal speed limit associated with the design of the time-specified speed limit sign ST based on the image of the recognized time-specified speed limit sign ST and the reference information REF.

[0047] The processing in the case of reference information REF shown in Figure 6 is as follows. Reference information REF indicates the legal speed limit for each road type. First, the processor 130 estimates the road type of the road where the recognized time-specified speed limit sign ST is installed. For example, the design of the time-specified speed limit sign ST includes the speed limit number. Time-specified speed limit signs ST that specify a speed limit of 100 km / h or more are installed on expressways. Therefore, the processor 130 can estimate the road type of the road where the recognized time-specified speed limit sign ST is installed based on its design. Alternatively, the processor 130 may identify the road type from navigation information NAV (map information and vehicle 1 location information). Then, the processor 130 obtains the legal speed limit set for the road type based on the reference information REF.

[0048] As yet another example, the reference information REF may be map information in which the legal speed limit for each location is registered. In this case, the processor 130 obtains the legal speed limit at the location of vehicle 1 based on the location information of vehicle 1 and the reference information REF.

[0049] If the statutory speed limit is determined on a road where a time-limited speed limit sign ST is installed (Step S130; Yes), the process proceeds to Step S140. In some situations, it may not be possible to determine the statutory speed limit. In that case (Step S130; No), the process proceeds to Step S150.

[0050] In step S140, the information notification system 100 notifies the user O via the notification device 120 of both the speed limit specified by one or more recognized time-specified speed limit signs ST and the statutory speed limit (see Figure 2).

[0051] In step S150, the information notification system 100 notifies the user O via the notification device 120 of the speed limit specified by one or more recognized time-specified speed limit signs ST.

[0052] 1-5. Effects As described above, according to this embodiment, one or more time-limited speed limit signs ST are recognized by the camera 2 mounted on the vehicle 1. Furthermore, it is determined whether the first condition is met, which is that the one or more recognized time-limited speed limit signs ST collectively specify all time periods of the day.

[0053] If the first condition is not met, the statutory speed limit is notified to the user O of vehicle 1. As a result, user O of vehicle 1 can know not only the speed limit that applies during a specific time period, but also the statutory speed limit that applies outside of that time period. Such information on the statutory speed limit is useful to user O and improves user O's convenience. For example, if the current time is outside of a specific time period, user O can know the statutory speed limit that applies at the current time, which is preferable.

[0054] On the other hand, if the first condition is met, multiple speed limits specified by each of the multiple time-specified speed limit signs ST are notified to user O. In other words, the speed limits for all time periods of the day are notified to user O. In this case, it is not necessarily required to notify user O of the statutory speed limit. By refraining from the process of acquiring information on the statutory speed limit, or by refraining from the process of notifying user O of the statutory speed limit, the processing load of the information notification system 100 can be reduced. In addition, since user O is not notified of more information than necessary, the feeling of annoyance to user O is suppressed.

[0055] 2. Second Embodiment In the second embodiment, we consider the case where the notification device 120 is designed to notify only one type of speed limit. For example, due to display space limitations, the notification device 120 (display device) may only be able to notify one type of speed limit.

[0056] In the second embodiment, the processor 130 notifies the user O of vehicle 1 of only one of the speed limits specified by one or more recognized time-specified speed limit signs ST and the statutory speed limit. Basically, the processor 130 notifies the user O of the highest of the candidate speed limits.

[0057] Figure 8 shows a situation similar to that in Figure 2. The processor 130 recognizes a single time-limited speed limit sign ST that specifies a certain time period (6:00 to 19:00). Since the first condition is not met, the processor 130 obtains information on the legal speed limit. Generally, the speed limit specified by the time-limited speed limit sign ST is lower than the legal speed limit. Therefore, the processor 130 notifies the user O of vehicle 1 of the legal speed limit without notifying the user O of the speed limit specified by the time-limited speed limit sign ST. Since the user O is notified of the legal speed limit information that cannot be obtained from the time-limited speed limit sign ST, the convenience for the user O is improved. Note that the time-limited speed limit sign ST can be seen by the user O himself.

[0058] Figure 9 shows a situation similar to that shown in Figure 3. The processor 130 simultaneously recognizes multiple time-specified speed limit signs ST-1 and ST-2 that specify different time periods. The first condition is met. In this case, the processor 130 notifies the user O of vehicle 1 of one of the multiple speed limits specified by each of the multiple time-specified speed limit signs ST-1 and ST-2. For example, the processor 130 notifies the user O of the highest of the multiple speed limits.

[0059] As a modified example, the processor 130 may notify the user O of vehicle 1 of one of several speed limits applicable to the current time. The current time is obtained from a clock. Information on the time periods specified by each of the multiple time-specified speed limit signs ST-1 and ST-2 is obtained by analyzing images of the multiple time-specified speed limit signs ST-1 and ST-2. The processor 130 selects one time-specified speed limit sign ST that specifies a time period including the current time, and notifies the user O of the speed limit specified by the selected time-specified speed limit sign ST.

[0060] Figure 10 is a flowchart showing the information notification process for the time-specified speed limit sign ST according to the second embodiment. Compared to the flowchart in the first embodiment (see Figure 7), step S120 is replaced with step S120', and step S140 is replaced with step S140'. Explanations that overlap with the first embodiment are omitted as appropriate.

[0061] In step S120', the processor 130 notifies the user O of one of the multiple speed limits specified by each of the recognized time-specified speed limit signs ST (see Figure 9).

[0062] In step S140', the information notification system 100 notifies the user O of the statutory speed limit without notifying the user of the speed limit specified by the time-specified speed limit sign ST (see Figure 8). [Explanation of Symbols]

[0063] 1 vehicle 2 cameras 100 Information Notification System 110 Communication Interface 120 Notification device 130 processors 140 Storage device 200 control programs O User S Speed ​​limit sign ST Time-limited speed limit sign IMG image Navigation Information REF Reference Information

Claims

1. An information notification system that notifies vehicle users of information, One or more processors, One or more storage devices that store reference information regarding the legal speed limit in the area in which the vehicle is used. Equipped with, The one or more processors described above are: The first condition is met, which is that one or more time-limited speed limit signs recognized by the camera mounted on the vehicle collectively specify all time periods of the day. If the first condition is not met, the system shall, based on the reference information, notify the user of at least the statutory speed limit on roads where one or more time-specified speed limit signs are installed. Information notification system.

2. An information notification system according to claim 1, If the first condition is not met, the one or more processors shall notify the user of both the speed limit specified by the one or more time-specified speed limit signs and the statutory speed limit. Information notification system.

3. The information notification system according to claim 2, If multiple time-specified speed limit signs specifying different time periods are recognized and the first condition is met, the one or more processors notify the user of the multiple speed limits specified by each of the multiple time-specified speed limit signs. Information notification system.

4. The information notification system according to claim 3, If the first condition is met, the one or more processors refrain from obtaining the legal speed limit, or refrain from notifying the user of the legal speed limit. Information notification system.

5. An information notification system according to claim 1, If the first condition is not met, the one or more processors shall notify the user of the statutory speed limit without notifying the user of the speed limit specified by the one or more time-specified speed limit signs. Information notification system.

6. The information notification system according to claim 5, If multiple time-limited speed limit signs specifying different time periods are recognized and the first condition is met, the one or more processors shall notify the user of one of the multiple speed limits specified by each of the multiple time-limited speed limit signs. Information notification system.

7. The information notification system according to claim 6, The one or more processors notify the user of the highest of the multiple speed limits. Information notification system.

8. An information notification system according to any one of claims 1 to 7, The one or more processors acquire the statutory speed limit on the road where the one or more time-limited speed limit signs are installed, based on the design of the one or more time-limited speed limit signs and the reference information. Information notification system.

9. An information notification system according to claim 8, The aforementioned reference information relates the design of speed limit signs used in the region to the statutory speed limits on the roads where the speed limit signs are installed. The one or more processors obtain the statutory speed limit associated with the design of the one or more time-limited speed limit signs based on the reference information. Information notification system.

10. An information notification system according to claim 8, The aforementioned reference information indicates the statutory speed limit for each type of road. The one or more processors described above are: Based on the design of the one or more time-limited speed limit signs mentioned above, the road type of the road on which the one or more time-limited speed limit signs are installed is estimated. Based on the aforementioned reference information, obtain the statutory speed limit set for the aforementioned road type. Information notification system.

11. An information notification method that uses a computer to notify the vehicle user of information, The first condition is met, which is that one or more time-limited speed limit signs recognized by the camera mounted on the vehicle collectively specify all time periods of the day. If the first condition is not met, the user shall be notified of the statutory speed limit on roads where one or more time-specified speed limit signs are installed, based on reference information regarding the statutory speed limit in the area in which the vehicle is used. including Information notification method.

Images