Information display control device, information display control method, information display control program, and recording medium

Abstract

One of the objectives is to provide an information display control device that can present information related to road signs and other signs according to the situation, thereby providing passengers with more appropriate and useful information. [Solution] The system includes a presentation control unit that controls the presentation of guide information, including information about signs or markings, including road signs or markings, to the passengers of a moving vehicle, and the presentation control unit is characterized in that when the moving vehicle is in autonomous driving mode, it causes the guide information to include more information than when the moving vehicle is not in autonomous driving mode.

Description

Technical Field

[0001] The present invention relates to an information presentation control device, and for example, relates to an information presentation control device, an information presentation control method, and a program that perform control related to presentation of information directed to an occupant of a mobile object.

Background Art

[0002] During travel of a mobile object, it is important for occupants including the driver of the mobile object to recognize sign displays including road markings and road signs (road signs and the like, hereinafter also simply referred to as "signs and displays") and grasp the meanings indicated by the road signs and the like. Devices that assist occupants in recognizing road signs and the like by presenting information related to road signs and the like present around the mobile object to the occupants are in use.

[0003] For example, Patent Document 1 discloses a road sign and the like display device comprising: an extraction unit that extracts road signs and the like from an image captured by an in-vehicle camera; a recording unit that records images of the road signs and the like; a display unit for displaying a car navigation image; and an image output unit that performs image editing so that the images of the road signs and the like and the car navigation image can be displayed simultaneously, and outputs the edited images to the display unit.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0005] For occupants of a mobile object, what kind of information related to which road sign and the like is beneficial varies depending on the situation. In the road sign and the like display device as described above, that information unsuited to the situation may also be displayed is cited as an example of a problem.

[0006] This invention has been made in view of the above-mentioned points, and one of its objectives is to provide an information display control device that can display information related to road signs, etc., according to the situation, and can provide more appropriate and useful information to passengers. [Means for solving the problem]

[0007] The invention described in claim 1 comprises a detection unit for detecting signs or markings, including road signs or road markings, that are present around a moving body, and a presentation control unit for controlling the presentation of guide information, which includes information relating to a first sign or marking detected by the detection unit, to a passenger of the moving body, wherein the presentation control unit causes the guide information to be presented in a manner based on the passenger's past level of interest in signs or markings of the same type as the first sign or marking.

[0008] The invention described in claim 10 is a method for causing an information presentation control device to present guide information including information about signs or markings, including road signs or road markings, comprising: a detection step in which a detection unit detects the signs or markings present around a moving body; and a presentation control step in which a presentation control unit controls the presentation of guide information including information about a first sign or marking detected by the detection unit to a passenger of the moving body, wherein the presentation control unit causes the guide information to be presented in a manner based on the passenger's past level of interest in signs or markings of the same type as the first sign or marking in the presentation control step.

[0009] The invention described in claim 11 is an information presentation control program executed by an information presentation control device equipped with a computer, wherein the computer functions as a detection unit that detects signs or markings, including road signs or road markings, that are present around a moving body, and a presentation control unit that controls the presentation of guide information, including information relating to a first sign or marking detected by the detection unit, to the passenger of the moving body, and the presentation control unit causes the guide information to be presented in a manner based on the passenger's past level of interest in signs or markings of the same type as the first sign or marking. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example of the configuration of an information presentation control device in Embodiment 1 of the present invention. [Figure 2A] This figure shows an example of a display screen presented by the information presentation control device of Example 1. [Figure 2B] This figure shows an example of a display screen presented by the information presentation control device of Example 1. [Figure 3A] This figure shows an example of information related to control in the information presentation control device of Example 1. [Figure 3B] This figure shows an example of information related to control in the information presentation control device of Example 1. [Figure 3C] This figure shows an example of information related to control in the information presentation control device of Example 1. [Figure 4] This flowchart shows an example of the presentation control routine by the information presentation control device in Example 1. [Figure 5] This block diagram shows an example of the configuration of an information presentation control device in Embodiment 2 of the present invention. [Figure 6A] This figure shows an example of information related to control in the information presentation control device of Example 2. [Figure 6B] This figure shows an example of information related to control in the information presentation control device of Example 2. [Figure 6C] This figure shows an example of information related to control in the information presentation control device of Example 2. [Figure 7] This is a flowchart showing an example of a presentation control routine by the information presentation control device of Embodiment 2. [Figure 8] This is a block diagram showing an example of the configuration of an information presentation control system according to Embodiment 3 of the present invention. [Figure 9A] This is a block diagram showing an example of the configuration of an information presentation control device according to Embodiment 3 of the present invention. [Figure 9B] This is a block diagram showing an example of the configuration of an information presentation control server according to Embodiment 3 of the present invention. [Figure 10] This is a diagram showing an example of information related to control in the information presentation control device of Embodiment 3. [Figure 11] This is a flowchart showing an example of a presentation control routine by the information presentation control system of Embodiment 3. [Figure 12] This is a block diagram showing an example of the configuration of an information presentation control device according to Embodiment 4 of the present invention. [Figure 13A] This is a diagram showing an example of a display screen presented by the information presentation control device of Embodiment 4. [Figure 13B] This is a diagram showing an example of a display screen presented by the information presentation control device of Embodiment 4. [Figure 13C] This is a diagram showing an example of a display screen presented by the information presentation control device of Embodiment 4. [Figure 14] This is a flowchart showing an example of a presentation control routine by the information presentation control device of Embodiment 4. [Mode for Carrying Out the Invention]

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail. In the following description and the accompanying drawings, substantially the same or equivalent parts are denoted by the same reference numerals. [Embodiments]

[0012] Referring to Figure 1, the configuration of the information presentation control device 10, which is Embodiment 1 of the present invention, will be described. The information presentation control device 10 is mounted on a moving body such as a vehicle and controls the presentation of guide information, which is information including road signs and road markings present around the moving body, to passengers, including the driver of the moving body. For example, the guide information includes the content of traffic regulations or instructions indicated by road signs or road markings. In this embodiment, an example in which the information presentation control device 10 is mounted on a vehicle will be described.

[0013] The storage unit 11 is a storage device such as a hard disk, flash memory, SSD (Solid State Drive), or RAM (Random Access Memory). The storage unit 11 appropriately stores programs executed in the information presentation control device 10, data necessary for the operation of the information presentation control device 10, data generated during the operation of the information presentation control device 10, and data taken in from external sources. For example, the storage unit 11 stores navigation information such as map information or route information. The storage unit 11 also stores information including the content of traffic regulations and instructions indicated by road signs or road markings.

[0014] The control unit 13 includes a central processing unit (CPU) and performs the necessary controls for the operation of the information display control device 10. For example, the control unit 13 reads map information or route information related to the mobile device on which the information display control device 10 is mounted from the storage unit 11 and performs the necessary controls when displaying this information.

[0015] The control unit 13 includes a presentation control unit 13A. The presentation control unit 13A is a functional unit that acquires information necessary from each unit when presenting guide information, which includes information about road signs, etc., including road signs or road markings present around the mobile unit, to the passenger of the mobile unit, and generates and supplies information for presenting the guide information to each unit. For example, the presentation control unit 13A generates information that associates road signs, etc., present around the mobile unit with the content of such road signs, etc., stored in the storage unit 11. For example, the guide information includes the content of traffic regulations or instructions indicated by road signs or road markings.

[0016] The display unit 15 is a display device such as a liquid crystal display, an organic EL display, or a head-up display (HUD), and is provided so that it can be seen by the passenger. The display unit 15 displays a screen showing information such as navigation information and guide information in response to instructions from the control unit 13. For example, the display unit 15 displays map information of the surrounding area where the vehicle is traveling. The display unit 15 also displays images of road signs and other signs present around the vehicle, as well as text indicating the traffic regulations indicated by those road signs.

[0017] The audio output unit 16 is an audio device that outputs information displayed by the display unit 15, such as navigation information and guide information, as audio. For example, it outputs a portion of the route information as audio. Furthermore, for example, the audio output unit 16 outputs the name and content of road signs, etc., as audio.

[0018] The driving information acquisition unit 17 is a group of devices that includes at least one device for acquiring information related to the movement of a moving object. The driving information acquisition unit 17 includes, for example, a location identification device such as a GPS (Global Positioning System) device, and acquires information related to the position of the moving object. The driving information acquisition unit 17 may also include sensors such as an acceleration sensor and a gyroscope sensor, and may acquire information such as the acceleration and direction of travel of the moving object. For example, the control unit 13 generates route information using the location information acquired by the driving information acquisition unit 17.

[0019] The operation input unit 19 is an input device such as a remote controller or a touch panel. The operation input unit 19 receives operation input from the passenger regarding the display unit 15 or the audio output unit 16. For example, the operation input unit 19 receives operation input such as an input requesting the display of route information or an input of the destination.

[0020] Furthermore, the operation input unit 19, which functions as an operation unit, accepts inputs such as stopping the display of the guide information image, which is an image displayed when guide information is shown by the display unit 15. In addition, the operation input unit 19, which functions as an operation unit, may also accept inputs to change the volume or stop the guide information audio, which is audio related to road signs, etc., announced by the audio output unit 16.

[0021] The peripheral imaging unit 21 is an imaging device such as an in-vehicle camera, which captures the area around a moving object and supplies the image data obtained to the control unit 13. For example, the peripheral imaging unit 21 captures the scenery in front of the moving object and supplies the image data to the control unit 13. The control unit 13 generates navigation information including the image from the image data captured by the peripheral imaging unit 21 and displays it on the display unit 15.

[0022] Furthermore, the control unit 13 detects road signs and the like from image data of the scenery in front of the moving object, including road signs and the like, captured by the peripheral imaging unit 21. The control unit 13 analyzes the image data, for example, by template matching, and detects road signs and the like included in the image. In this case, the peripheral imaging unit 21 functions together with the control unit 13 as a detection unit that detects road signs and the like present around the moving object. The peripheral imaging unit 21 may be equipped with multiple cameras, and may be configured and arranged to capture images in multiple directions, such as in front of and to the side of a vehicle.

[0023] The in-vehicle monitor unit 23 is a device such as a camera that photographs the interior of the vehicle or a sensor that detects changes in the interior. For example, the in-vehicle monitor unit 23 supplies data to the control unit 13 that observes the direction or movement of the driver's gaze or the movement of their body, such as their head. It may also be possible to identify the driver from the images taken by the camera inside the vehicle.

[0024] For example, the display control unit 13A can obtain the length of time the driver gazed at road signs, etc., located around the moving object (sign gaze time) from data relating to the direction or movement of the driver's gaze. Also, for example, the display control unit 13A can obtain the length of time the driver gazed at the guide information displayed on the display unit 15 (guide gaze time) from data relating to the direction or movement of the driver's gaze.

[0025] The driver identification information acquisition unit 25 is a recognition device capable of recognizing passengers, including the driver. For example, the driver identification information acquisition unit 25 may be a device that recognizes terminals carried by passengers, such as smartphones or wearable devices. Alternatively, the driver identification information acquisition unit 25 may be a biometric authentication device capable of recognizing veins or fingerprints.

[0026] For example, when a driver gets into a vehicle, the driver identification information acquisition unit 25 detects the identification information of the smartphone the driver is carrying and supplies it to the control unit 13. For example, this identification information is a unique identification number for the device, such as the IMEI (International Mobile Equipment Identifier).

[0027] For example, when a driver enters the vehicle for the first time, the identification information detected for the first time by the driver identification information acquisition unit 25 is stored in the storage unit 11. Then, when the driver enters the vehicle again, the driver identification information acquisition unit 25 detects the identification information of the smartphone again. The presentation control unit 13A compares this with the driver's identification information stored in the storage unit 11 to identify the driver. In other words, the driver identification information acquisition unit 25 and the presentation control unit 13A function as identification units that identify the driver.

[0028] As described above, the in-vehicle monitor unit 23 acquires data on gaze time, which is the length of time the driver gazes at road signs, etc., or the length of time the driver gazes at guide information corresponding to said road signs, etc. This gaze time serves as an indicator of the degree of the driver's interest in said road signs, etc. For example, based on this gaze time, a degree of interest indicating the driver's level of interest can be calculated.

[0029] For example, if a driver does not understand road signs well or is unfamiliar with them, their level of interest is likely to be high. For instance, if the level of interest is high, it can be considered that the driver has a high need for guide information corresponding to the road sign, that is, information including the content of the traffic regulations indicated by the road sign. On the other hand, if the level of interest is low, it can be considered that the driver does not have much need for guide information corresponding to the road sign, and therefore has a low need.

[0030] For example, when the display control unit 13A displays guide information on the display unit 15, the control unit can display the guide information in a manner that is relevant to the driver's level of interest, thereby enabling the guide information to be presented in a manner that is appropriate to the driver's needs. For example, if the driver is highly interested in road signs, the guide information screen corresponding to the road signs will be displayed in a larger, more prominent manner. If the driver is less interested, the guide information screen will be displayed in a less prominent manner.

[0031] Figure 2A shows an example in which guide information is displayed on the display screen 30 of the display unit 15 in a less conspicuous (less emphasized) manner. In Figure 2A, an image 31 of the area in front of the moving object is displayed. The image 31 of the area in front of the moving object includes a road sign 31A. Below the image 31 of the area in front, guide information 32 corresponding to the road sign 31A is displayed. The guide information 32 includes an image 32A showing the road sign 31A and text 32B indicating the content of the road sign 31A. In addition, a region 33 of navigation information such as map information may be displayed in conjunction with the guide information in a part of the display screen 30.

[0032] Figure 2B shows an example in which guide information is displayed on the display screen 30 of the display unit 15 in a more prominent (emphasized) manner than in Figure 2A. In Figure 2B, as in Figure 2A, an image 31 of the area in front of the moving object and an area 33 containing navigation information are displayed. The image 31 of the area in front of the moving object includes a road sign 31A. Below the image 31 of the area in front, guide information 34 corresponding to the road sign 31A is displayed.

[0033] In Figure 2B, the guide information 34 includes an image 34A showing a road sign 31A and text 34B indicating the content of the road sign 31A. Image 34A is displayed larger and more prominently than image 32A in Figure 2A. Text 34B uses larger characters than text 32B in Figure 2A and is displayed in a more emphasized manner. Also, as in Figure 2A, the navigation information area 33 is displayed.

[0034] Figure 3A shows an example of sign detection data, which is information used by the presentation control unit 13A for controlling the presentation of guide information. This sign detection data is stored, for example, in the storage unit 11, and each time a new road sign or the like is detected, information corresponding to that road sign or the like is added.

[0035] In Figure 3A, the "Date and Time" when a road sign was detected, the "Details of the Road Sign" (which contains information about the road sign), and the "Type" are shown in correspondence. Some road signs include numbers indicating speed limits, etc. For example, road signs that differ only in the numbers indicating limits such as maximum speed, minimum speed, height, maximum width, weight, and time may be treated as the same type. Also, signs that differ only in the number or direction of the arrows displayed, and otherwise have the same meaning, may be treated as the same type.

[0036] In Figure 3A, "Type" refers to road signs and other signs that differ only in number being treated as the same type. For example, in Figure 3A, "Maximum Speed ​​60km," "Maximum Speed ​​80km," and "Maximum Speed ​​40km" all have "Maximum Speed" listed in the "Type" column.

[0037] Furthermore, the system includes "driver identification information," which is the driver's identification information at the time each road sign, etc., was detected, and the driver's "gaze time (sec)." This gaze time is, for example, the time the driver gazed at the image if the guide information was presented as an image. Alternatively, if no guide information was presented, it may be the time the driver gazed at the road sign, etc., located in front of the moving object.

[0038] In other words, in Figure 3A, road signs and other objects detected from images acquired by the peripheral imaging unit 21, the corresponding driver's gaze time, and driver identification information are all associated and stored.

[0039] Figure 3B shows an example in which the average gaze time of driver A over a predetermined period is calculated for each type of road sign based on sign detection data, and an interest level is assigned based on the calculation results. In Figure 3B, an example is shown in which an interest level is assigned as follows: "Low interest" when the average gaze time is less than 0.5 seconds, "Medium interest" when the average gaze time is 0.5 seconds or more but less than 1 second, and "High interest" when the average gaze time is 1 second or more.

[0040] In Figure 3B, the average gaze time for the "Maximum Speed" category is 0.8 seconds, and the level of interest is assigned a "Medium" rating. The average gaze time for the "Pedestrian Only" category is 0.4 seconds, and the level of interest is assigned a "Low" rating, while the average gaze time for the "Height Restriction" category is 1.2 seconds, and the level of interest is assigned a "High" rating.

[0041] Figure 3C shows an example of the criteria for determining the presentation mode according to the level of interest when the presentation control unit 13A controls the presentation of guide information in the manner based on the level of interest shown in Figure 3B. According to the criteria in Figure 3C, if the level of interest is "low," the guide information is not presented. If the level of interest is "medium," the guide information is normally presented in an inconspicuous, non-emphasized manner, as shown in Figure 2A. If the level of interest is "high," the guide information is presented in an emphasized manner, as shown in Figure 2B, which is more conspicuous and emphasized.

[0042] For example, when guide information is announced by the audio output unit 16, if the level of interest is "high", the announcement may be made in a manner that includes a louder voice or a more unusual announcement sound compared to when the level of interest is "medium". Also, when guide information is presented by the HUD, if the level of interest is "high", control may be performed to lower the transparency compared to when the level of interest is "medium".

[0043] Furthermore, if the level of interest is less than a first predetermined value (first predetermined value), the display control unit 13A will not display the guide information. If the level of interest is equal to or greater than the first predetermined value, but less than a second predetermined value (second predetermined value) that is greater than the first predetermined value, the guide information may be displayed in an inconspicuous manner. If the level of interest is equal to or greater than the second predetermined value, the guide information may be emphasized and displayed in a more conspicuous manner.

[0044] [Presentation control routine RT1] Referring to Figure 4, the presentation control routine RT1 executed by the presentation control unit 13A in the information presentation control device 10 will now be described. The presentation control unit 13A starts the presentation control routine RT1 when the accessory power supply (hereinafter referred to as ACC power supply) of the vehicle on which the information presentation control device 10 is installed is turned on. The presentation control unit 13A may also start the presentation control routine RT1 when the engine of the vehicle starts or when the vehicle starts moving.

[0045] When the presentation control unit 13A starts the presentation control routine RT1, it performs passenger authentication (step S11). In step S11, the presentation control unit 13A obtains an identification number unique to the terminal carried by the driver operating the vehicle from the driver identification information acquisition unit 25, and recognizes the driver by comparing this with the identification information stored in the storage unit 11. If the identification number is not stored in the storage unit 11, such as when the driver is riding in the vehicle for the first time, the identification number obtained in step S11 is stored in the storage unit 11 to register it as a new identification number.

[0046] After step S11 is executed, the display control unit 13A instructs the surrounding imaging unit 21 to start imaging the area around the vehicle and begins acquiring images of the area around the vehicle (step S12). After step S12 is executed, the display control unit 13A determines whether or not road signs, etc. have been detected (step S13). If the display control unit 13A determines in step S13 that road signs, etc. have not been detected (step S13: NO), it executes step S13 again and determines again whether or not road signs, etc. have been detected.

[0047] In step S13, the display control unit 13A determines that a road sign or the like has been detected (step S13: YES), and then determines whether or not to display guide information corresponding to the detected road sign or the like (step S14). In step S14, the display control unit 13A determines whether or not a certain amount of past information (Figure 3A) corresponding to the same type of road sign or the like that has been detected has been stored in the storage unit 11. In step S14, if it is determined that a certain amount of information including the driver's attention time when the same type of road sign or the like was detected in the past has not been stored as sign detection information, then it is determined that guide information should be displayed.

[0048] Furthermore, in step S14, if the display control unit 13A determines that a certain number of sign detection data have been accumulated, it refers to the driver's past gaze time or the past level of interest shown by the driver in the past, obtained from the said gaze time, from the sign detection data. For example, in step S14, if the referenced level of interest is "low" (Figure 3C), it is determined that guide information should not be presented. Also, for example, if the level of interest is below a predetermined threshold (below the first predetermined value), it is determined that guide information should not be presented.

[0049] In step S14, the presentation control unit 13A determines that guide information should be presented (step S14: YES), and then determines whether or not to highlight the guide information (step S15). For example, in step S15, the presentation control unit 13A determines whether or not the level of interest referenced in step S14 is "high" (Figure 3C). Also, for example, if in step S14 it is determined that a certain number of sign detection data have not been accumulated, it may be determined in step S15 to highlight the information.

[0050] If the display control unit 13A determines in step S15 that the guide information should not be highlighted (step S15: NO), it causes the display unit 15 to display the guide information normally (step S16). For example, in step S16, the guide information is displayed in an inconspicuous manner as shown in Figure 2A.

[0051] In step S15, the display control unit 13A determines that the guide information should be highlighted (step S15: YES), and then causes the display unit 15 to highlight the guide information (step S17). For example, in step S17, the guide information is presented in an highlighted manner as shown in Figure 2B.

[0052] After step S16 or step S17 is executed, the display control unit 13A determines whether the driver has finished confirming the guide information (step S18). For example, in step S18, the display control unit 13A obtains data regarding the direction of the driver's gaze from the in-vehicle monitor unit 23 and determines whether the confirmation has been completed. For example, in step S18, the display control unit 13A determines that the confirmation has been completed if the driver's gaze has been directed towards the display screen of the display unit 15 where the guide information image is displayed and then moved away from the display screen.

[0053] Furthermore, for example, in step S18, if the driver's gaze is not directed at the guide information image at all or is continuously directed at it, the display control unit 13A determines that the confirmation is complete after a certain amount of time has elapsed since the guide information image was displayed. In addition, in step S18, the confirmation of the guide information may be determined to be complete when the operation input unit 19 receives an operation to stop the display of the guide information image. The level of interest may also be calculated based on the time until such operation is received.

[0054] If the presentation control unit 13A determines in step S18 that the confirmation of the guide information has not been completed (step S18: NO), it repeats step S18. If the presentation control unit 13A determines in step S18 that the confirmation of the guide information has been completed (step S18: YES), it terminates the presentation of the guide information (step S19).

[0055] The display control unit 13A stores sign detection information (step S20) after the execution of step S19 or if it determines in step S14 that guide information should not be displayed (step S14: NO). For example, the sign detection information includes the date and time when the road sign was detected in the display control routine RT1, details of the road sign, type of road sign, the time the driver looked at the road sign or guide information (looking time), and the driver's identification information. In step S20, the display control unit 13A stores the sign detection information in the storage unit 11.

[0056] After executing step S20, the presentation control unit 13A returns to the beginning of the presentation control routine RT1 and repeats the presentation control routine RT1.

[0057] As described above, in the presentation control routine RT1, the presentation control unit 13A presents the guide information based on the driver's past level of interest. However, after the presentation, the routine may determine the driver's current level of interest in the road signs, etc., detected in step S13, based on the time spent looking at the road signs, etc. or the guide information, and then change the presentation of the guide information based on that current level of interest.

[0058] In other words, the presentation control unit 13A may change the presentation of guide information from a mode based on the driver's past level of interest to a mode based on the driver's current level of interest in detected road signs, etc.

[0059] For example, even if it is determined in step S14 that guide information should not be presented (step S14: NO), if the driver has spent a long time looking at the road signs, etc., detected in step S13 and is judged to have a high level of interest (for example, level of interest "medium (1)" or "high (2)"), then, for example, step S16 or step S17 may be executed to present the guide information.

[0060] Furthermore, even if, for example, it is determined in step S15 that no emphasis presentation is necessary and the information is presented normally (for example, with a level of interest of "medium (1)"), if the driver has spent a long time looking at road signs or normally displayed guide information and it is determined that the level of interest is high, step S17 may be executed to emphasize the guide information.

[0061] As described in detail above, the information presentation control device 10 of Embodiment 1 adopts the driver's level of interest in road signs, etc. as an indicator of how much the driver needs the guide information, including the content of road signs, etc., detected while the moving vehicle is in motion, that is, how beneficial it is for the driver to present the guide information to the driver, and can provide the guide information to the driver in a manner based on that level of interest. Therefore, it is possible to provide an information presentation control device that can provide useful information to the driver in an appropriate manner, according to the driver's level of understanding and familiarity with each of the road signs, etc. [Examples]

[0062] Referring to Figure 5, the configuration of the information presentation control device 40, which is Embodiment 2 of the present invention, will be described. Similar to the information presentation control device 10 of Embodiment 1, the information presentation control device 40 is mounted on a moving body such as a vehicle and controls the presentation of guide information, which is information including road signs and road markings present around the moving body, to passengers, including the driver of the moving body. For example, the guide information includes the content of traffic regulations or instructions indicated by road signs or road markings. In this embodiment, an example in which the information presentation control device 40 is mounted on a vehicle will be described.

[0063] As shown in Figure 5, the information display control device 40 differs from the information display control device 10 in that it does not have an in-vehicle monitor unit 23 and has a display control unit 13B, but otherwise has the same configuration. In this embodiment, the display control unit 13B tally up the detection count, which is the number of times road signs, etc., have been detected.

[0064] Figure 6A shows an example of the sign detection data table TB2 used by the presentation control unit 13B for controlling the presentation of guide information in Embodiment 2. The sign detection data table TB2 is stored in the storage unit 11. Each time a new road sign is detected, sign detection information corresponding to that road sign is added to the sign detection data table TB2. The sign detection information includes the date and time the road sign was detected, details such as the name and contents of the detected road sign, and driver identification information.

[0065] In Figure 6A, the "Date and Time" when a road sign was detected, the "Details of the Road Sign," which includes the name and other information of the road sign, and the "Type" of the road sign are listed in chronological order and correspond to each other. In Figure 6A, as in the case of the sign detection data table TB1 in Example 1, road signs with the same name but different displayed numbers are treated as the same type.

[0066] Furthermore, in Figure 6A, the "driver identification information," which is the driver's identification information when each of the road signs, etc., is detected, is shown in association with each. In this way, in Figure 6A, the road signs, etc., detected from the images acquired by the surrounding imaging unit 21 and the driver's identification information are stored in association with each other.

[0067] Figure 6B shows the results of the display control unit 13B aggregating the number of detections, which is the number of times road signs, etc., were detected while driver A was driving the vehicle, based on the sign detection data table TB2. In Figure 6B, "cumulative detection count" shows the number of detections for each "type" per 10 hours of driving time, which is the total time the same driver drove the vehicle.

[0068] The cumulative number of detections may also be the number of detections per predetermined driving time. Alternatively, it may be the number of detections per distance traveled, for example, the distance the driver has driven the vehicle, or the total number of detections since the driver first got into the vehicle. In this way, the display control unit 13B aggregates the number of detections for each driver and for each type of road sign, etc.

[0069] Figure 6B also shows an example of the results of calculating the encounter frequency, which is the frequency with which driver A encountered various types of road signs, etc., based on the cumulative number of detections. In Figure 6B, an example is shown in which the encounter frequency is calculated as follows: "1 (low)" if the cumulative number of detections is less than 3, "2 (medium)" if the cumulative number of detections is 3 or more but less than 20, and "3 (high)" if the cumulative number of detections is 20 or more.

[0070] Furthermore, the frequency of encounters may be calculated as the sum of the number of detections per unit of driving time, the number of detections per unit of driving distance, or the number of detections since the driver first got into the vehicle, for each type of road sign, etc., as described above. Alternatively, the frequency of encounters may be calculated as the ratio of the number of detections for each type to the total number of detections, which is the total number of road signs, etc. detected. In addition, the frequency of encounters may be calculated by combining these calculation methods.

[0071] In Figure 6B, the cumulative number of detections for the category "Maximum Speed" is 50, resulting in an encounter frequency of "3 (High)". The cumulative number of detections for "Pedestrian Only" is 18, resulting in an encounter frequency of "2 (Medium)". The encounter frequency for "With Roundabout" is 1, resulting in an encounter frequency of "1 (Low)".

[0072] Furthermore, the types of road signs detected and their frequency of encounter vary depending on the route the driver travels in the vehicle. For example, if the frequency of encounter is low, it is likely that the driver is unfamiliar with, unaccustomed to, or has little understanding of those infrequently encountered road signs. On the other hand, if the frequency of encounter is high, it is likely that the driver is familiar with, accustomed to, or has a high level of understanding of those frequently encountered road signs.

[0073] Therefore, if the frequency of encounters is low, it can be considered that the driver has a high degree of need for the guide information corresponding to the road sign, that is, information including the content of the traffic regulations indicated by the road sign. On the other hand, if the frequency of encounters is high, it can be considered that the driver does not have much need for the guide information corresponding to the road sign, and therefore has a low degree of need.

[0074] Figure 6C shows an example of the criteria for determining the presentation mode according to the encounter frequency when the presentation control unit 13B controls the presentation of guide information by displaying it as an image on the display screen of the display unit 15, based on the encounter frequency shown in Figure 6B. According to the criteria in Figure 6C, if the encounter frequency is "3 (high)", the guide information is not presented.

[0075] Furthermore, if the encounter frequency is "2 (medium)", the normal presentation control of guide information is performed in an inconspicuous, unemphasized manner, as illustrated in Figure 2A in Example 1. If the encounter frequency is "1 (low)", the emphasized presentation control of guide information is performed in a more conspicuous, emphasized manner, as illustrated in Figure 2B in Example 1.

[0076] Thus, when the display control unit 13B displays guide information on the display unit 15, by controlling the display of the guide information in a manner based on the driver's frequency of encountering such situations, it is possible to display the guide information in a manner that is appropriate to the driver's needs.

[0077] Furthermore, when the presentation control unit 13B notifies guide information via the audio output unit 16, it may notify in a manner accompanied by a louder voice or a more unusual notification sound when the encounter frequency is "1 (low)" compared to when the encounter frequency is "2 (medium)". Also, when the presentation control unit 1·BR>RB notifies guide information via the HUD, it may control the transparency to make the guide information easier to recognize when the encounter frequency is "1 (low)" compared to when the encounter frequency is "2 (medium)".

[0078] In addition, while Figures 6B and 6C illustrate an example where the presentation control unit 13B calculates the encounter frequency as "1 (low)," "2 (medium)," and "3 (high)," this is not the only way. The encounter frequency may be calculated as any other arbitrary value.

[0079] In other words, the presentation control unit 13B determines that guide information should not be presented if the encounter frequency is equal to or greater than a first predetermined value (first predetermined value). If the encounter frequency is less than the first predetermined value but equal to or greater than a second predetermined value (second predetermined value) smaller than the first predetermined value, the guide information is normally presented in an inconspicuous manner. If the encounter frequency is less than the second predetermined value, the guide information is emphasized and presented in a more conspicuous manner. This is one example of control over the presentation manner.

[0080] Furthermore, while the above describes an example in which the encounter frequency is calculated in three stages using two predetermined values, a first predetermined value and a second predetermined value, as thresholds, this is not the only option; there may be one or more predetermined values. In other words, it is sufficient to have established criteria for deciding whether to present guide information according to the encounter frequency.

[0081] For example, in addition to "1 (low)," "2 (medium)," and "3 (high)," a third predetermined value greater than the first predetermined value, "4 (extremely high)," may be defined as an even higher encounter frequency than "3 (high)." Furthermore, if the encounter frequency is "3 (high)," guide information may not be provided because it is considered familiar, but if the encounter frequency is very high, such as "4 (extremely high)," guide information may be provided because it is considered an important road sign, etc.

[0082] [Presentation control routine RT2] Referring to Figure 7, the presentation control routine RT2 executed by the presentation control unit 13B in the information presentation control device 40 will be described. The presentation control unit 13B starts the presentation control routine RT2 when the engine of the vehicle on which the information presentation control device 40 is installed is started. The display control unit 13B may also start the display control routine RT2 when the accessory power supply (hereinafter referred to as ACC power supply) of the vehicle is turned on.

[0083] When the presentation control unit 13B starts the presentation control routine RT2, it performs passenger authentication (step S31). In step S31, the presentation control unit 13B obtains passenger identification information from the driver identification information acquisition unit 25 and performs authentication.

[0084] For example, in step S31, the display control unit 13B obtains an identification number unique to the terminal carried by the driver operating the vehicle from the driver identification information acquisition unit 25, and compares this with the identification information stored in the storage unit 11 to recognize the driver. If the identification number is not stored in the storage unit 11, such as when the driver is riding in the vehicle for the first time, the identification number obtained in step S31 is stored in the storage unit 11 to register it anew.

[0085] After step S31 is executed, the display control unit 13B instructs the surrounding imaging unit 21 to start imaging the area around the vehicle and begins acquiring images of the area around the vehicle (step S32). After step S32 is executed, the display control unit 13B determines whether or not road signs, etc. have been detected (step S33). If the display control unit 13B determines in step S33 that road signs, etc. have not been detected (step S33: NO), it executes step S33 again and determines again whether or not road signs, etc. have been detected.

[0086] In step S33, the display control unit 13B determines that it has detected a road sign or the like (step S33: YES), and then determines whether or not to display guide information corresponding to the detected road sign or the like (step S34). In step S34, the display control unit 13B determines whether or not a certain number of the driver's sign detection information (Figure 6A) corresponding to the same type of road sign or the like that was detected has been stored in the storage unit 11. In step S34, if it is determined that a certain number of pieces of information including the number of detections has not been stored, it is determined that guide information should be displayed.

[0087] Furthermore, in step S34, if the display control unit 13B determines that a certain amount of information including the number of detected signs has been accumulated, it calculates the frequency with which the driver has encountered the same type of road sign in the past. Based on this frequency of encounters, it then determines whether or not to display guide information.

[0088] For example, in step S34, the presentation control unit 13B determines that guide information should not be presented if the calculated encounter frequency is "3 (high)" (Figure 6C) or is equal to or greater than the first predetermined value. For example, the presentation control unit 13B determines that guide information should be presented if the encounter frequency is "2 (medium)" or "1 (low)" or is less than a predetermined threshold (less than the first predetermined value).

[0089] In step S34, the display control unit 13B determines that guide information should be displayed (step S34: YES), and then determines whether or not to highlight the guide information (step S35). For example, in step S35, the display control unit 13B determines whether or not the encounter rate calculated in step S34 is "1 (low)" (Figure 6C). Also, for example, if in step S34 it is determined that a certain number of sign detection information have not been accumulated, it may be determined in step S35 to highlight the information.

[0090] If the display control unit 13B determines in step S35 that the guide information should not be highlighted (step S35: NO), it causes the display unit 15 to display the guide information normally (step S36). For example, in step S36, the guide information is displayed for a predetermined time in an inconspicuous manner as shown in Figure 2A.

[0091] In step S35, the display control unit 13B determines that the guide information should be highlighted (step S35: YES), and causes the display unit 15 to highlight the guide information (step S37). For example, in step S37, the guide information is displayed for a predetermined time in an highlighted manner as shown in Figure 2B.

[0092] The display control unit 13B terminates the display of guide information by the display unit 15 after the execution of step S36 or step S37 (step S38). In step S38, if the operation input unit 19 receives an operation to stop the display of the guide information image, the display control unit 13B may terminate the display of guide information even if the predetermined time has not elapsed. In addition, the history that such an operation has been received may be recorded and stored together with the sign detection information.

[0093] The display control unit 13B stores sign detection information (step S39) after the execution of step S38 or if it determines in step S34 that guide information should not be displayed (step S34: NO). For example, the sign detection information includes the date and time when the road sign was detected in the display control routine RT2, details of the road sign, type of road sign, total number of detected signs, and driver identification information. For example, in step S39, the display control unit 13B stores the sign detection information in the storage unit 11.

[0094] After executing step S39, the display control unit 13B determines whether or not the vehicle's engine is ON (step S40). If the display control unit 13B determines in step S40 that the engine is ON (step S40: YES), it returns to step S33 and determines again whether or not a road sign or the like has been detected.

[0095] If the presentation control unit 13B determines in step S40 that the engine is not ON (step S40: NO), it terminates the presentation control routine RT2.

[0096] In this embodiment, the information display control device 40 may also have an in-vehicle monitor unit 23, similar to the first embodiment, and may be capable of acquiring data such as the driver's eye movements and identifying the driver from images taken by a camera inside the vehicle.

[0097] As described in detail above, the information presentation control device 40 of this embodiment calculates the frequency of encounters by aggregating the number of road signs, etc. detected while the moving vehicle is in motion. This frequency of encounters can be used as an indicator of how much the driver needs guide information, including the content of the road signs, etc. detected while the moving vehicle is in motion, that is, how beneficial it is for the driver to receive guide information. Guide information can then be provided to the driver in a manner based on this frequency of encounters. Therefore, it is possible to provide an information presentation control device that can provide useful information to the driver in an appropriate manner, according to the driver's experience of encountering each of the road signs, etc. [Examples]

[0098] Referring to Figures 8, 9A, and 9B, the configuration of the presentation control system 100, which includes the information presentation control device 50 according to Embodiment 3 of the present invention, will be described. As shown in Figure 8, the presentation control system 100 includes the information presentation control device 50 and the information presentation control server 60 as an information processing device. The information presentation control device 50 is mounted on the vehicle M.

[0099] The information display control device 50 and the information display control server 60, which is an external device of the information display control device 50, can mutually send and receive data (communication connection) via a network (communication path). For example, communication protocols such as TCP / IP are used for such sending and receiving. Furthermore, this network can be constructed using mobile communication networks, wireless communication such as Wi-Fi (registered trademark), and internet communication including wired communication.

[0100] The information display control server 60 is capable of sending and receiving data to and from each of the multiple information display control devices 50 installed in multiple vehicles M.

[0101] Figure 9A shows the configuration of the information presentation control device 50. As shown in Figure 9A, the information presentation control device 50 differs from the information presentation control device 40 of Embodiment 2 in that it has a communication unit 51 and a presentation control unit 13C, but is otherwise configured similarly.

[0102] The communication unit 51 is a network adapter such as a NIC (Network Interface Card) for sending and receiving data to and from the outside in accordance with the instructions of the display control unit 13C. For example, the communication unit 51 transmits identification information of the driver riding in the vehicle M. Also, when a road sign or the like is detected while the vehicle M is in motion, the communication unit 51 transmits sign detection information, which is information about the road sign or the like. This sign detection information includes the name and details of the detected road sign or the like, or the type of the detected road sign or the like.

[0103] For example, information regarding the road signs, etc., is transmitted to the information presentation control server 60 using a communication protocol such as TCP / IP. The communication unit 51 also receives information regarding the presentation of guide information, including information regarding road signs, etc., to the driver of vehicle M.

[0104] When the presentation control unit 13C receives information regarding the presentation, it controls the display unit 15 or the audio output unit 16 to provide guide information to the driver according to the instructions contained in the information.

[0105] Figure 9B shows the configuration of the information presentation control server 60. The server communication unit 61 is a network adapter such as a NIC, and transmits and receives data with one or more vehicles M. The server communication unit 61 receives identification information of the driver riding in vehicle M and sign detection information related to road signs, etc., detected while vehicle M is in motion.

[0106] The server storage unit 62 is a storage device such as a hard disk, flash memory, SSD (Solid State Drive), or RAM (Random Access Memory). The server storage unit 62 stores various programs, such as the operating system and server software.

[0107] Furthermore, the server storage unit 62 stores a driver history database (driver history DB) 62A. The driver history DB 62A stores driver history information, which is information indicating the history of road signs, etc., encountered by each driver when riding in one or more vehicles M. For example, the driver history information includes the aggregated results of the number of detections for each type of road sign, etc.

[0108] The server control unit 63 is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM, etc., and functions as a computer. The server control unit 63 realizes various functions by reading and executing various programs stored in the server storage unit 62. The server control unit 63 includes a server presentation control unit 63A.

[0109] The server information control unit 63A controls the presentation or notification of guide information based on the driver history information stored in the driver history DB 62A. For example, the server information control unit 63A obtains driver identification information and detected road signs and other signs from the server communication unit 61 and adds them to the driver history information to tally the number of detected signs. For example, the number of detected signs is tallied for each type of road sign. The server display control unit 63A functions as a tallying unit that tallies the number of detected road signs for each type.

[0110] Furthermore, when the server display control unit 63A obtains sign detection information from the server communication unit 61, it calculates the encounter frequency from the aggregated number of detections included in the driver history information corresponding to the sign detection information, and determines whether or not to display guide information corresponding to the detected road sign, etc., based on the encounter frequency. The server display control unit 63A causes the server communication unit 61 to transmit data including the result of this determination to the information display control device 50.

[0111] Figure 10 shows an example of a sign detection data table TB3 stored in the driver history DB62A as driver history information. In Figure 10, information regarding the detection of road signs, etc., detected when driver A rode in different vehicles M1, M2, and M3 is shown in chronological order.

[0112] As shown in Figure 10, the same type of road sign is detected even when the driver is in a different vehicle. For example, a road sign of the type "Road Construction in Progress" is detected while the driver is in vehicle M2, and then again while the driver is in vehicle M3.

[0113] Furthermore, as in the cases of Examples 1 and 2, road signs and the like may be treated as belonging to the same category if they have the same name but different numbers or other markings. For example, in Figure 10, road signs indicating "Maximum Speed ​​60km" and "Maximum Speed ​​80km" are listed under the same category, "Maximum Speed."

[0114] Based on the driver history information shown in Figure 10, the server presentation control unit 63A aggregates the number of detected road signs, etc., calculates the frequency of encounters with road signs, etc., and determines the presentation mode of guide information based on the encounter frequency. The encounter frequency can be calculated as follows, for example, as shown in Figure 6B in Embodiment 2: "3 (high)" if it is above a first predetermined value, "2 (medium)" if it is below the first predetermined value but above a second predetermined value, and "1 (low)" if it is below the second predetermined value.

[0115] For example, the presentation method may be such that, as shown in Figure 6C in Example 2, it may be determined that guide information should not be presented when the encounter frequency is "3 (high)". Also, when the encounter frequency is "2 (medium)", it may be determined that guide information should be presented using a normal presentation, which is an inconspicuous presentation method, and when the encounter frequency is "1 (low)", it may be determined that guide information should be presented using an emphasized presentation, which is a conspicuous presentation method.

[0116] [Presentation control routine RT3] Referring to Figure 11, the presentation control routine RT3 executed in the presentation control system 100 will be described. The dashed lines in Figure 11 schematically show the data transmission and reception process. In Figure 11, the routine executed by the information presentation control device 50 is shown as routine RT3A, and the routine executed by the information presentation control server 60 is shown as routine RT3B.

[0117] The display control unit 13C of the information display control device 50 starts routine RT3A when the engine of vehicle M is started. Alternatively, the display control unit 13C may start routine RT3A when the ACC power of vehicle M is turned on.

[0118] When the presentation control unit 13C starts routine RT3A, it acquires the driver identification information of vehicle M from the driver identification information acquisition unit 25 and controls the communication unit 51 to transmit it to the information presentation control server 60 (step S51A). In step S51A, the driver identification information is transmitted by the communication unit 51 along with the device ID of the information presentation control device 50. In step S51A, the communication unit 51 functions as a transmission unit.

[0119] For example, in step S51A, the presentation control unit 13C obtains an identification number unique to the terminal carried by the driver from the driver identification information acquisition unit 25 and transmits it together with the device ID of the information presentation control device 50.

[0120] For example, in step S51A, the driver identification information acquisition unit 25 may acquire driver identification information by biometric authentication such as fingerprint authentication or iris authentication. In addition to the biometric authentication information, the driver identification information acquisition unit 25 may also acquire password input information from the driver and provide it to the presentation control unit 13C.

[0121] For example, the device ID is an identifier in the communication protocol used between the information display control device 50 and the information display control server 60. The information transmitted in step 51A may also include a vehicle ID, which is the identification information of the vehicle M.

[0122] When the information presentation control server 60 is started, the server presentation control unit 63A of the information presentation control server 60 starts routine RT3B. After starting routine RT3B, the server presentation control unit 63A waits for the reception of driver identification information (step S51B).

[0123] After executing step S51B, the server presentation control unit 63A determines whether or not it has received driver identification information (step S52B). In step S52B, the server presentation control unit 63A determines whether or not information including driver identification information and the device ID of the information presentation control device 50 has been received by the server communication unit 61. In step S52B, the server communication unit 61 functions as a receiving unit.

[0124] If the server presentation control unit 63A determines that it has not received driver identification information (step S52B: NO), it waits for such reception for a certain period of time and then executes step S52B again.

[0125] When the server presentation control unit 63A determines that it has received driver identification information (step S52B: YES), it determines whether the driver identification information is already registered in the driver history DB 62A of the server storage unit 62 (step S53B). The determination in step S53B is made by checking whether the driver identification information received in step S52B is recorded in the indicator detection data table TB3.

[0126] If the server presentation control unit 63A determines in step S53B that driver identification information is not registered (step S53B: NO), it registers the driver identification information received in step S52B (step S54B). The registration in step S54B is performed by newly writing and saving the received driver identification information in the indicator detection data table TB3.

[0127] After step S51A is executed, the display control unit 13C instructs the surrounding imaging unit 21 to start imaging the area around the vehicle and begins acquiring images of the area around the vehicle (step S52A). After step S52A is executed, the display control unit 13C determines whether or not road signs, etc. have been detected (step S53A). If the display control unit 13C determines in step S53A that road signs, etc. have not been detected (step S53A: NO), it executes step S53A again and determines again whether or not road signs, etc. have been detected.

[0128] If the display control unit 13C determines in step S53A that it has detected a road sign or the like (step S53A: YES), it transmits the detected road sign or the like information to the information display control server 60 (step S54A).

[0129] If the server presentation control unit 63A determines in step S53B that driver identification information has been registered (step S53B: YES) or after the execution of step S54B, it waits for indicator detection information (step S55B). Subsequently, the server presentation control unit 63A determines whether or not it has received indicator detection information (S56B).

[0130] If the server presentation control unit 63A determines in step S56B that it has not received any sign detection information (step S56B: NO), it repeats step S56B. If the server presentation control unit 63A determines in step S56B that it has received any sign detection information (step S56B: YES), it determines whether or not to present guide information (step S57B).

[0131] In step S57B, the driver history information stored in the driver history DB62A is used to calculate the encounter frequency (Figure 6B) and determine the presentation mode based on the encounter frequency (Figure 6C), similar to the process in Example 2. For example, in step S57B, the server presentation control unit 63A determines that guide information should not be presented if the encounter frequency is greater than or equal to a first predetermined value (first predetermined value), and determines that guide information should be presented if the encounter frequency is less than the first predetermined value.

[0132] Furthermore, the server presentation control unit 63A may determine in step S57B that it should present guide information if a certain amount of driver history information has not been accumulated.

[0133] In step S57B, the server presentation control unit 63A determines that guide information should be presented (step S57B: YES) and sends a guide information output request (presentation request), which is information that includes information requesting the output of guide information, to the information presentation control device 50 (step S58B). For example, in step S58B, the server presentation control unit 63A generates information that includes information requesting the output of guide information and information indicating the presentation method of the guide information, and has the server communication unit 61 transmit it. In step S58B, the server communication unit 61 functions as a transmission unit.

[0134] After executing step S54A, the presentation control unit 13C determines whether or not it has received a guide information output request (step S55A). In step S55A, the presentation control unit 13C determines whether or not the communication unit 51 has received a guide information output request. In step S55A, the communication unit 51 functions as a receiving unit. For example, in step S55A, the presentation control unit 13C repeats the determination of whether or not it has received a guide information output request for a predetermined time.

[0135] In step S55A, the display control unit 13C determines that it has received a request to output guide information (step S55A: YES), and then displays the guide information (step S56A). For example, in step S56A, the presentation control unit 13C causes the guide information to be displayed on the display screen of the display unit 15 for a predetermined time. The presentation method in step 56A may be either an inconspicuous normal presentation method or a conspicuous emphasized presentation method, according to the instructions included in the guide information output request. Alternatively, for example, in step S56A, the presentation control unit 13C may cause the audio output unit 16 to output audio of the guide information being read aloud.

[0136] After executing step S56A, the presentation control unit 13C terminates the presentation of guide information (step S57A). After executing step S57A or if the presentation control unit 13C determines in step S55A that it has not received a request to output guide information (step S55A: NO), it determines whether the engine of vehicle M is ON or not (step S58A).

[0137] If the display control unit 13C determines in step S58A that the engine is ON (step S58A: YES), it returns to step S53A and determines again whether or not it has detected a road sign or the like.

[0138] If the presentation control unit 13C determines in step S58A that the engine is not ON (step S58A: NO), it sends driving completion information to the information presentation control server 60 indicating that the vehicle M has finished driving (step S59A). After executing step S59A, the presentation control unit 13C terminates routine RT3A.

[0139] After executing step S58B, the server information control unit 63A determines whether or not it has received information that vehicle M has finished driving (step S59B). If the server information control unit 63A determines in step S59B that it has not received information that vehicle has finished driving (step S59B: NO), it returns to step S55B and waits again for sign detection information. If the server information control unit 63A determines in step S59B that it has received information that vehicle has finished driving (step S59B: YES), it terminates routine RT3B.

[0140] In this embodiment, the case in which the information display control server 60 determines whether or not to display guide information has been described, but this is not limited to this case. The determination may also be made by the display control unit 13C of the information display control device 50. For example, the server display control unit 63A of the information display control server 60 may, instead of a guide information output request, transmit information to the information display control device 50 including the number of detected road signs, etc., or the frequency of encounters.

[0141] In this case, the presentation control unit 13C of the information presentation control device 50 may aggregate the number of detections and determine whether or not to present guide information and the presentation method based on the number of detections or the frequency of encounters. It is sufficient if the number of detections for each driver for multiple vehicles is aggregated in the driver history DB 62A so that the frequency of encounters can be calculated by the server presentation control unit 63A or the presentation control unit 13C.

[0142] As described in detail above, the presentation control system 100 of Embodiment 3 allows the information presentation control server 60 to aggregate the frequency of encounters that drivers have encountered in the past while driving different vehicles, regardless of the type of vehicle, for each type of road sign, etc. Then, guide information can be presented to the driver in a manner that differs for each driver based on their respective encounter frequency.

[0143] Therefore, even if a single driver has the opportunity to ride in multiple vehicles, such as through car sharing or rental cars, and does not always ride in the same vehicle, it is possible to collect the frequency of encounters with road signs, etc., for each driver, and guide information can be presented based on each driver's encounter frequency.

[0144] Therefore, it is possible to provide an information presentation control device and an information presentation control server that can provide drivers with useful information in an appropriate manner. [Examples]

[0145] The information presentation control device 70 according to Embodiment 4 will be described below with reference to the attached drawings. Figure 12 shows the configuration of the information presentation control device 70. The information presentation control device 70 is configured similarly to the information presentation control device 40 according to Embodiment 2, but differs in that it has a presentation control unit 13D that can perform different control from the presentation control unit 13B.

[0146] The information display control device 70 is installed in a vehicle that can switch between automatic driving and manual driving modes. In this specification, automatic driving refers to a driving mode that does not require human intervention. In this specification, manual driving refers to a driving mode that requires human intervention (non-automatic driving), and includes partially manual driving. Partially manual driving refers to a driving mode in which at least part of the operation or surrounding environment checks must be performed by a human, and some form of human intervention is required.

[0147] Furthermore, in this specification, a vehicle that can switch between autonomous and manual driving refers to a vehicle that is capable of driving in autonomous mode and can be driven in manual mode when the surrounding road conditions, weather, or other environmental conditions are poor, or when autonomous driving is deactivated at the driver's discretion. For example, a vehicle capable of driving at Level 3 or higher according to the SAE J3016 standard falls under this category.

[0148] The display control unit 13D controls the display of guide information, including information about road signs, etc., based on whether the vehicle's driving mode is automatic driving or not. Specifically, when the vehicle is driving in automatic driving mode (automatic driving), the display control unit 13D displays guide information in a manner that shows more detailed content, including explanations of traffic rules, penalties for violating traffic rules, or information about a wider variety of road signs, etc.

[0149] During manual driving mode, the display control unit 13D displays simplified guide information containing less information, such as only the names of road signs. The determination of whether or not the vehicle is in automatic driving mode is made, for example, by acquiring information from the vehicle.

[0150] For example, a predetermined standard can be established to select which type of road sign, etc., corresponds to which guide information to be presented. For instance, if the vehicle is not in autonomous driving mode (non-autonomous driving), the display control unit 13D may display guide information for non-autonomous driving regarding a type of road sign, etc. that meets a first standard. Alternatively, if the vehicle is in autonomous driving mode, the display control unit 13D may display guide information for autonomous driving regarding a type of sign or marking that meets a second standard, which is easier to meet than the first standard.

[0151] For example, the same criteria as in Examples 1 to 3, such as the passenger's level of interest in road signs or the frequency with which the passenger encounters road signs, can be used. For example, guide information corresponding to types of road signs with a level of interest higher than a predetermined value, or types of road signs with an encounter frequency lower than a predetermined value, may be displayed. By setting the predetermined value lower during autonomous driving and higher during manual driving, it is possible to present guide information containing more information during autonomous driving and guide information containing less information during manual driving.

[0152] Furthermore, the functions of the presentation control unit 13D can also be realized in the information presentation control device 10 in Example 1 (Figure 1), the information presentation control device 40 in Example 2 (Figure 5), or the presentation control system 100 in Example 3 (Figures 8 and 9A).

[0153] Therefore, the information presentation control device 70 may use either the driver's level of interest in road signs, etc. or the frequency of encountering road signs, etc. as an indicator of necessity, which is the degree to which the passengers, including the driver, need the information, and present guide information to the driver in a manner based on the level of interest or the frequency of encountering road signs, etc.

[0154] Figure 13A shows an example of a display screen including guide information that is displayed on the display screen 30 of the display unit 15 during autonomous driving in Embodiment 4. In Figure 13A, an image 31 of the area in front of the vehicle is displayed. The image 31 of the area in front of the vehicle includes a road sign 31A indicating the maximum speed. Below the image 31 of the area in front of the vehicle, guide information 35 corresponding to the road sign 31A is displayed.

[0155] Guide information 35 includes an image 35A showing road sign 31A and text 35B indicating the detailed content of road sign 31A. Text 35B indicates not only the name of road sign 31A but also the meaning displayed by road sign 31A. In addition, a navigation information area 33, such as map information, may be displayed in a part of the display screen 30 together with the guide information 35.

[0156] Figure 13B shows another example of a display screen including guide information that is displayed on the display screen 30 of the display unit 15 during autonomous driving in Embodiment 4. In Figure 13B, an image 36 of the area in front of the vehicle is displayed. The image 36 of the area in front of the vehicle includes a road sign 36A indicating a stop sign. Below the image 36 of the area in front of the vehicle, guide information 37 corresponding to the road sign 36A is displayed.

[0157] Guide information 37 includes an image 37A showing road sign 36A and text 37B showing detailed information about road sign 36A. Text 37B shows not only the name of road sign 36A, but also its meaning and penalties.

[0158] Furthermore, when the guide information 37 in Figure 13B is presented, the voice output from the voice output unit 16 may also output a reading of the text 37B. For example, the guide information 37 may be displayed on the display screen 30 accompanied by the voice saying, "This is a stop sign. Even slowing down is a violation. The penalty points are 2, and the fine for a regular car is 7,000 yen."

[0159] Detailed information, such as that exemplified in Figure 13B, may be used for driver learning. Furthermore, in conjunction with this guide information, area 38 containing navigation information, such as map information, may be displayed in a portion of the display screen 30.

[0160] Figure 13C shows another example of the quiz mode display screen shown on the display screen 30 of the display unit 15 during autonomous driving in Embodiment 4. The quiz mode is used, for example, to check the driver's learning results or level of understanding. In Figure 13C, an image of a road sign and a question asking the driver to select the correct meaning of the road sign are displayed. For example, the driver can learn about road signs, etc., during autonomous driving when they do not need to be involved in driving, and this can be used to help them recognize road signs when they switch to manual driving.

[0161] [Presentation control routine RT4] Referring to Figure 14, an example of a routine executed in Embodiment 4 will be described, specifically the presentation control routine RT4 executed by the presentation control unit 13D. The presentation control unit 13D starts the presentation control routine RT4 when the engine of the vehicle equipped with the information presentation control device of Embodiment 4 is started. Alternatively, the presentation control unit 13D may start the presentation control routine RT4 when the ACC power of the vehicle is turned on.

[0162] When the presentation control unit 13D starts the presentation control routine RT4, it performs authentication of the passengers, including the driver of the vehicle (step S61). In step S61, the presentation control unit 13D obtains the passenger's identification information from the driver identification information acquisition unit 25 and performs authentication.

[0163] For example, in step S61, the presentation control unit 13D acquires the identification number of the terminal carried by the driver or the driver's biometric information, and authenticates the driver by comparing it with the identification information stored in the storage unit 11 and registered in advance. If the driver's identification information is not stored in the storage unit 11, the identification information acquired in step S61 is stored in the storage unit 11 and newly registered.

[0164] After step S61 is executed, the display control unit 13D instructs the surrounding imaging unit 21 to start imaging the area around the vehicle and begins acquiring images of the area around the vehicle (step S62). After step S62 is executed, the display control unit 13D determines whether or not road signs, etc. have been detected (step S63). If the display control unit 13D determines in step S63 that road signs, etc. have not been detected (step S63: NO), it executes step S63 again and determines again whether or not road signs, etc. have been detected.

[0165] In step S63, the display control unit 13D determines that it has detected a road sign or the like (step S63: YES), and then determines whether the vehicle's driving mode is automatic driving mode (step S64).

[0166] In step S64, the display control unit 13D determines that it is not in automatic driving mode (step S64: NO) and performs control to display guide information in a simplified manner that does not include detailed information (non-automatic driving guide information display mode) (step S65). In step S65, the display control unit 13D displays the guide information for a certain period of time in a simplified manner, for example, as shown in Figure 2A in Embodiment 1.

[0167] In step S64, the display control unit 13D determines that the vehicle is in automatic driving mode (step S64: YES) and then performs control to display detailed guide information containing a lot of information (automatic driving guide information display mode) (step S66). In step S66, the display control unit 13D displays detailed guide information, such as that shown in Figure 13A, or detailed guide information accompanied by voice, such as that shown in Figure 13B, for a certain period of time. In addition, in step S66, the display control unit 13D displays detailed guide information, such as that shown in Figure 13C, including a quiz mode, for a certain period of time.

[0168] In addition, during the presentation control in steps S65 and S66, the presentation control unit 13D may further control the presentation manner of the guide information based on the level of interest or the frequency of encounter. For example, it may decide not to present the guide information if the level of interest is low or the frequency of encounter is high.

[0169] After executing step S65 or step S66, the presentation control unit 13D saves a history of the presentation of guide information (step S67). In step S67, the presentation control unit 13D stores sign detection information, including the date and time of detection of newly detected road signs, etc., in the storage unit 11. In step S67, the presentation control unit 13D may also store in the storage unit 11 information regarding the presented guide information, such as whether the content was detailed or the driver's level of understanding.

[0170] After executing step S67, the display control unit 13D determines whether the engine of the vehicle is ON or OFF (step S68). If the display control unit 13D determines in step S68 that the engine is ON (step S68: YES), it returns to step S63 and determines again whether or not a road sign or the like has been detected.

[0171] In step S68, if the display control unit 13D determines that the engine is not ON (step S68: NO), it instructs the surrounding imaging unit 21 to stop imaging the area around the vehicle and terminates the display control routine RT4.

[0172] As described in detail above, according to the information display control device or information display control system of Embodiment 4 of the present invention, which is installed in a vehicle capable of switching between autonomous driving and non-autonomous driving modes, it is possible to provide the driver with more detailed information, including more information about road signs, etc., when the driver is in autonomous driving mode, in which the driver does not need to be involved in driving.

[0173] This system provides drivers with opportunities to review or learn about the content of road signs and related rules and penalties, enabling them to maintain knowledge of road signs even when using autonomous driving mode.

[0174] Therefore, according to the information presentation control device or information presentation control system of Example 4, it is possible to present information such as road signs in an appropriate amount and manner according to whether or not the vehicle is in autonomous driving mode, thereby providing useful information to the passenger. [Explanation of Symbols]

[0175] 10, 40, 50, 70 Information display control device 11 Storage section 13 Control Unit 13A Display Control Unit 15 Display section 16 Audio output section 17. Driving Information Acquisition Unit 19. Operation Input Section 21 Peripheral Photography Department 23 In-car monitor unit 25 Driver Recognition Information Acquisition Unit 51 Communications Department 60 Information display control server 61 Server Communication Unit 62 Server Storage 62A Driver History Database 63 Server Control Unit 63A Server Presentation Control Unit

Claims

1. The system includes a display control unit that controls the presentation of guide information, including information about signs or markings, including road signs or road markings, to the passengers of a moving vehicle. The information presentation control device is characterized in that, when the moving object is in autonomous driving mode, it presents the guide information which includes more information than when the moving object is not in autonomous driving mode.

2. The system includes a detection unit that detects the signs or markings present in the vicinity of the moving body, The information presentation control device according to claim 1, characterized in that the presentation control unit causes the device to present the guide information corresponding to the detected sign or mark when the sign or mark is detected by the detection unit.

3. The information display control device according to claim 1 or 2, characterized in that the display control unit causes the mobile vehicle to display the guide information, including an explanation of the traffic rules indicated by the sign or marking, when the mobile vehicle is in automatic driving mode.

4. The information display control device according to any one of claims 1 to 3, characterized in that, when the moving object is in autonomous driving mode, it causes the display control unit to display the guide information, including the penalties for violating the traffic rules indicated by the signs or markings.

5. The information display control device according to any one of claims 1 to 4, characterized in that the display control unit displays the guide information for non-autonomous driving for signs or markings of a type that meets the first criterion when the moving body is not in autonomous driving mode, and displays the guide information for autonomous driving for signs or markings of a type that meets the second criterion, which is a criterion that is easier to meet than the first criterion, when the moving body is in autonomous driving mode.

6. The information display control device according to claim 5, characterized in that, when the moving body is not in automatic driving mode, the display control unit sets a type of sign or marking that is of a type that is of a type that satisfies the first criterion, which is either of a type that is

7. A method for causing an information display control device to display guide information including information about signs or markings, including road signs or road markings, The presentation control unit includes a presentation control step that controls the presentation of the guide information to the passenger of the moving vehicle, The information presentation control step is characterized in that, when the moving body is in autonomous driving mode, the guide information containing more information than when the moving body is not in autonomous driving mode is presented.

8. An information presentation control program executed by an information presentation control device equipped with a computer, The aforementioned computer, A display control unit that controls the presentation of guide information, including information about signs or markings, including road signs or road markings, to the passengers of a moving vehicle. To make it function as, The information presentation control program is characterized in that, when the mobile body is in autonomous driving mode, it causes the mobile body to present guide information that includes more information than when the mobile body is not in autonomous driving mode.

9. A computer-readable recording medium characterized by storing the program described in claim 8.

Images