Information processing device, information processing method, and information processing program

Abstract

The present invention provides an information processing device that allows drivers of mobile vehicles to concentrate on driving by controlling the updating of information according to the situation when presenting them with information that is updated as it is received. [Solution] The display unit 214 acquires output information that is output to and can be updated, and also acquires status information indicating the level of autonomous driving of the vehicle. The control unit 211 controls whether or not to update the output information based on the level of autonomous driving indicated by the acquired status information.

Description

Technical Field

[0001] This application belongs to the technical field of an information processing apparatus or the like that performs update control of information output by an output means.

Background Art

[0002] With the recent spread of so-called mobile computing, users can obtain various types of information output by terminal devices. Such information includes information that is updated in real-time, such as weather information, exchange rate information, stock price information, operation information of transportation facilities, posting information to SNS (Social Networking Service), and POI (Point of Interest) information displayed on the map of a navigation device.

[0003] In addition, Patent Document 1 below discloses a technique for distributing information that better reflects the preferences of a user in consideration of the user's preferences and the situation at that time.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, in the above-described terminal devices that present information to users and the techniques described in Patent Document 1, it is common to present information that is updated in real-time to users while sequentially updating it in accordance with the update. However, for example, regarding the information that a navigation device of a moving body such as an automobile displays on a display device, if it is updated in real-time regardless of the driving situation or the intention of the passengers, there is a problem that, for example, a driver may be distracted by the updated information and unable to concentrate on driving.

[0006] Therefore, this application was made in view of these circumstances, and one example of its objective is to provide an information processing device that can allow drivers to concentrate on driving by controlling the updating of information according to the situation when presenting information that is updated as it is received to the driver of a moving vehicle. [Means for solving the problem]

[0007] To solve the above problems, the invention described in claim 1 comprises: an output information acquisition means for acquiring output information that is output to an output means provided on a mobile body and is updatable; a state information acquisition means for acquiring state information indicating a control state related to movement in the mobile body; and a control means for controlling whether or not to update the output information based on the acquired state information.

[0008] To solve the above problems, the invention described in claim 7 is an information processing method performed in an information processing device, comprising: an output information acquisition step of acquiring output information that is output to an output means provided on a mobile body and is updatable; a state information acquisition step of acquiring state information indicating a control state related to movement in the mobile body; and a control step of controlling whether or not to update the output information based on the acquired state information.

[0009] To solve the above problems, the invention described in claim 8 causes the computer connected to the output means to function as an information processing device according to any one of claims 1 to 6. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the outline configuration of the information processing device according to the embodiment. [Figure 2] This figure shows an example of a screen in the display of a navigation device according to an embodiment. [Figure 3] This is a block diagram showing the outline configuration of a navigation device according to an embodiment. [Figure 4]This is an external view illustrating the arrangement of the display and other components in the navigation device according to the embodiment, as seen from the driver's seat. [Figure 5] This is a flowchart illustrating an example of the operation of the navigation device according to the embodiment. [Figure 6] This flowchart shows the details of an example of the operation of the navigation device according to the embodiment. [Modes for carrying out the invention]

[0011] The form for implementing this application will be explained using Figure 1.

[0012] As shown in Figure 1, the information processing device 1 according to this embodiment includes an output information acquisition means 111A, a state information acquisition means 111B, a control means 111C, an ambient environment information acquisition means 111D, and a detection means 111E, and is connected to an external output means 2.

[0013] In this configuration, the output information acquisition means 111A acquires output information that is output to and updatable by the output means 2 provided on the mobile body. The output information that is output to the output means 2 in an updatable manner includes weather information, exchange rate information, stock price information, transportation service operation information, and social media posting information.

[0014] The state information acquisition means 111B acquires state information indicating the control state related to the movement of a moving object. A moving object is, for example, an automobile, bicycle, railway, ship, aircraft, etc. The control state refers to, for example, the extent to which the judgment or driving skills of the user of the moving object (for example, the driver if the moving object is a vehicle) are involved in the movement of the moving object, and is the so-called state of automatic driving of that moving object.

[0015] The control means 111C then controls whether or not to update the output information based on the state information acquired by the state information acquisition means 111B. Here, "updating" the output information means: i) A process of updating the output information already output to the output means 2 to new output information (for example, a process of updating the already output exchange rate information A to other exchange rate information B), and ii) A process of controlling whether to output the output information that has not yet been output to the output means 2 (that is, the output means 2 switches from a state where no output information is output to the output means 2 to a state where new output information is output, etc.) Both are included (the same applies hereinafter).

[0016] Therefore, according to the information processing apparatus 1 according to the embodiment, since the permission or non - permission of updating the output information is controlled based on the control state related to the movement in the moving body, the permission or non - permission of updating the output information can be controlled finely and appropriately, and the driver can be focused on driving.

[0017] Also, in the information processing apparatus 1 according to the embodiment, the control state of the moving body is the state of automation of the operation of the moving body in which a plurality of levels are preset, and the state information acquired by the state information acquisition means 111B is configured to be state information indicating which level of automation the current automation of the operation of the moving body is.

[0018] Therefore, the control state related to the movement of the moving body is the state of automation of its operation in which a plurality of levels are predefined, and the state information indicates which level of automation the current automation of the operation of the moving body is. Thus, according to the level of automation of the operation of the moving body, the permission or non - permission of updating the output information can be controlled finely and appropriately, and the driver can be focused on driving.

[0019] In the information processing apparatus 1 according to the further embodiment, the control state of the moving body is either a fully automated driving state in which all of acceleration, steering, and braking in its operation are automated and the driver is not involved in the operation at all, or a driving state of the moving body other than the fully automated driving state. The state information acquired by the state information acquisition means 111B is state information indicating any of these driving states. The control means 111C controls to prohibit the update of the output information when it is indicated by the state information that the above driving state is a driving state of the moving body other than the fully automated driving state.

[0020] Therefore, since the control state regarding the movement of the moving body is either a fully automated driving state or a driving state of the moving body other than the fully automated driving state, the state information is state information indicating any of these driving states, and the update of the output information is prohibited when it is shown that the driving state of the moving body is a driving state of the moving body other than the fully automated driving state, the update of the output information can be controlled finely and appropriately, and the driver can be concentrated on driving.

[0021] Furthermore, in the information processing apparatus 1 according to the embodiment, the surrounding environment information acquisition means 111D acquires surrounding environment information regarding the surrounding environment of the moving body.

[0022] Then, the control means 111C controls the permissibility of updating the output information based on the control state of the moving body indicated by the state information acquired by the state information acquisition means 111B and the surrounding environment regarding the surrounding environment information acquired by the surrounding environment information acquisition means 111D.

[0023] Therefore, since the permissibility of updating the output information is controlled based on the control state of the moving body and the surrounding environment, the permissibility of updating the output information can be controlled more finely and appropriately, and the driver can be concentrated on driving.

[0024] Furthermore, in the information processing device 1 according to the embodiment, the control state of the mobile body is either a fully automated driving state in which acceleration, steering, and braking in the operation of the mobile body are all automated and the driver is not involved in the operation at all, or a driving state of the mobile body other than the fully automated driving state, and the state information acquired by the state information acquisition means 111B is state information that indicates either of the above driving states.

[0025] The control means 111C controls the update of the output information when the status information indicates that the operating state of the mobile body is in a fully automated operating state, and controls the update of the output information when the status information indicates that the operating state is in an operating state other than a fully automated operating state, and the surrounding environment information acquired by the surrounding environment information acquisition means 111D indicates that the surrounding environment is not suitable for updating the output information.

[0026] Therefore, the system controls the updating of output information when the control state for the movement of the mobile body is either a fully automated driving state or a driving state of the mobile body other than the fully automated driving state, the state information indicates either of those driving states, and it is indicated that the driving state of the mobile body is a fully automated driving state. However, it prohibits updating the output information when it is indicated that the driving state is a driving state other than a fully automated driving state and it is indicated that the surrounding environment is not suitable for updating the output information. This allows for fine-grained and appropriate control over whether or not to update the output information, enabling the driver to concentrate on driving.

[0027] Furthermore, in the information processing device 1 according to this embodiment, the detection means 111E detects the actions of passengers other than the driver of the moving vehicle.

[0028] The control means 111C then controls the system so that, if the output information is the target of an operation detected by the detection means 111E, the output information can be updated regardless of the surrounding environment, as determined by the surrounding environment information acquisition means 111D.

[0029] Therefore, when the output information is subject to the actions of a passenger other than the driver, the system controls it to allow updating of the output information regardless of the surrounding environment, so that the output information can be updated appropriately when it is subject to the actions of a passenger other than the driver. [Examples]

[0030] Next, specific embodiments corresponding to the above-described embodiments will be explained with reference to Figures 2 to 6. The embodiments described below are examples in which the above-described embodiments are applied to an in-vehicle device mounted on a vehicle (an example of a "mobile body"). Examples of in-vehicle devices according to the embodiments described below include a navigation device that guides the movement of the vehicle on which the in-vehicle device is mounted, and a display device that displays content such as videos and SNS posts inside the vehicle.

[0031] (I) Overview of the in-vehicle device according to the example First, we will briefly describe the characteristic information update function of the NV in-vehicle device according to the embodiment. The information update function is a function that controls the updating of output information, which is updated over time, when it is displayed on the screen.

[0032] The in-vehicle device NV according to the embodiment acquires time information, exchange rate information, and SNS posting information, which are output information that is updated over time. The timing of acquiring each piece of information may be periodic, or it may be whenever the information is updated. SNS posting information may include, for example, information indicating posts containing keywords set by the user. That is, when the keyword "Hokkaido" is set, the in-vehicle device NV acquires information indicating posts related to Hokkaido.

[0033] Next, the in-vehicle device NV displays the acquired output information in separate areas as shown in Figure 2. Specifically, it displays time information in the time display area 401 of screen 400, exchange rate information in the exchange rate display area 402, and SNS post information in the post information display area 403. At this time, if the surrounding environment of the vehicle on which the in-vehicle device NV is installed is not suitable for updating information during its movement, the in-vehicle device NV will not update the SNS post information displayed in the post information display area 403 with newly acquired SNS post information. Here, the situation in which the surrounding environment is not suitable for updating information during the vehicle's movement refers to situations such as heavy rain, traffic congestion on the road being traveled on, or extremely narrow roads, or any other surrounding environment that imposes psychological, physical, or mental stress on the driver of the vehicle. Furthermore, the in-vehicle device NV will not update the SNS post information displayed in the post information display area 403 with newly acquired SNS post information, even if new SNS post information is acquired, if the SNS post information is the target of the user's actions, that is, if the user's gaze is directed towards the SNS post information, or if the user is performing an operation on the post information display area 403 where the SNS post information is displayed (for example, scrolling). On the other hand, with respect to time information, the in-vehicle device NV will update the time information displayed in the time display area 401, etc., with newly acquired time information, regardless of whether the surrounding environment of the vehicle indicates that the situation is not suitable for updating information while the vehicle is moving, and regardless of whether the time information is the target of the user's actions (the same applies to exchange rate information and the exchange rate display area 402). In other words, the in-vehicle device NV displays the latest information for time information.

[0034] Furthermore, the NV in-vehicle device controls whether or not to update the output information according to the level of automation in the vehicle in which the NV is installed. In this case, the level of automation can basically be set arbitrarily, but as an example, levels 1 to 4 are shown in Table 1 below.

[0035] [Table 1]

[0036] The automation levels shown in Table 1 are defined on page 5 of the "Public-Private ITS Concept and Roadmap 2016-2020: Towards the Realization of Automated Driving on Expressways and Unmanned Automated Driving Mobility Services in Limited Areas," published by the Headquarters for the Promotion of an Advanced Information and Communications Network Society on May 20, 2016. According to the definitions in Table 1, a system that achieves Level 1 automation is called a "Safe Driving Support System," a system that achieves Levels 2 and 3 automation is called a "Semi-Automated Driving System," and a system that achieves Level 4 automation is called a "Safe Automated Driving System." The "Information Provision Type" classification in Table 1 includes "Safe Driving Support Devices (In-Vehicle Equipment)" that provide warnings to the driver in this information provision type. Furthermore, at any level of automation in Table 1, the vehicle driver can intervene in the control of the automated driving system at any time. That is, for example, in Level 4 automation, the driver can stop the automated driving system as needed using a system deactivation / stop button, etc. However, if the driver intervenes in the automated driving system during Level 4 automation, it ceases to be Level 4 automation at that point.

[0037] (II) Configuration of the in-vehicle device according to the embodiment Next, the configuration of the in-vehicle device NV according to the embodiment will be described using Figure 3. As shown in Figure 3, the in-vehicle device NV according to the embodiment is configured to include a control unit 211, a storage device 212 consisting of an HDD or the like, an input device 213, a display unit 214, a bus line 215, an input / output interface unit 220, a vehicle speed sensor 221, an angular velocity sensor 222, an acceleration sensor 223, a steering angle sensor 224, a GPS (Global Positioning System) receiving unit 225, a data transmission / reception unit 226, a gaze detection camera 227, an external camera 228, and an external sensor 229. Of the above configuration, an example of the arrangement of the input device 213, the display unit 214 including the screen 400, the gaze detection camera 227, and the external sensor 229 is, for example, as illustrated in Figure 4, where the external sensor 229 is located outside the vehicle as viewed from the driver's seat inside the vehicle, the input device 213 is located at the position of the steering wheel H, and the display unit 214 and gaze detection camera 227 are located at the front panel FP next to the steering wheel H. Furthermore, if there is an update portion in the image HD of the head-up display displayed (projected) at the driver's line of sight through the windshield FG, the update control according to this embodiment can also be applied.

[0038] In the above configuration, the vehicle speed sensor 221 detects the current speed of the vehicle using speed detection processing, for example, using a vehicle speed pulse acquired from a vehicle equipped with the on-board device NV, and outputs speed data. The angular velocity sensor 222 detects the angular velocity of the vehicle, for example, the angular velocity of changes in direction, and outputs angular velocity data per unit time and relative bearing data. The acceleration sensor 223 detects the acceleration of the vehicle, for example, in the longitudinal direction, and outputs acceleration data per unit time, etc. The steering angle sensor 224 detects the steering angle of the vehicle and outputs steering angle data, etc. The GPS receiver 225 receives navigation radio waves from GPS satellites and outputs GPS positioning data such as latitude, longitude, altitude data which are the vehicle's position information, absolute bearing data of the vehicle's direction of travel, and GPS speed data. The data transmission / reception unit 226 performs processing related to the transmission and reception of data with a server device via a network. The gaze detection camera 227 is installed inside the vehicle equipped with the NV in-vehicle device and is a camera for detecting the gaze of the user (referring to the driver and passengers; the same applies hereinafter), and outputs the captured image to the control unit 211. The exterior camera 228 is installed outside the vehicle equipped with the NV in-vehicle device and is a camera for detecting the surrounding environment of the vehicle, and outputs the captured image to the control unit 211. The exterior sensor 229 is a sensor for detecting the temperature, humidity, or rainfall outside the vehicle, and outputs the detection result to the control unit 211 via the input / output interface unit 220 and the bus line 215.

[0039] The storage device 212 stores various programs, such as the operating system and application programs. These programs may be obtained, for example, from a server device via a network, or they may be read and executed from recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), or USB (Universal Serial Bus) memory.

[0040] The storage device 212 also stores map image data for displaying maps on the display unit 214, map information used when searching for routes, road link information, and other similar data. The storage device 212 also stores information indicating so-called near-miss points, such as intersections, railway crossings, and highway merging points where accidents are likely to occur. Furthermore, the storage device 212 stores operation history information showing the user's operation history with the in-vehicle device NV, or gaze movement history information showing the user's gaze movement history. At this time, the operation history information and gaze movement history information are stored together with identification information indicating which user they belong to. This identification information is identified by the control unit 211 based on the captured image output by the gaze detection camera 227.

[0041] In addition to the above, the storage device 212 stores road information related to the map information and road link information, including road type information indicating the type of road (e.g., expressway, general road, or one-way road), road width information indicating the width of the road, shoulder presence / absence information indicating the presence or absence of a shoulder on each road, shoulder width information indicating the width of the shoulder if one exists, lane number information indicating the number of lanes on each road, center lane position information indicating the position of the center lane on each road, parking space presence / absence information indicating the presence or absence of a parking space on each road, and parking space position information indicating the location of the parking space if one exists. Furthermore, the storage device 212 stores parking information including parking type information indicating the type of parking lot (service area, shopping mall parking lot, etc.) and parking location information indicating the location of each parking lot. At this time, the parking location information includes position information indicating the longitude / latitude of the vertices of each side constituting the parking lot when the corresponding parking lot is viewed as a planar shape (range), and is configured so that the area within the range connecting each vertex can be recognized as a parking lot based on this position information. More specifically, parking location information is configured such that, for example, if the planar shape (range) of the corresponding parking lot is rectangular, the area within the range of the four vertices is recognized as a parking lot based on the positional information indicated by the longitude / latitude of those four points.

[0042] The information stored in the storage device 212 may also be stored in a server device (not shown) outside the in-vehicle device NV, and received and used as needed.

[0043] The input device 213 consists of a touch sensor, touch panel, keyboard, mouse, pressure sensor, jog dial, and other controllers, and receives user input operations and transmits operation signals indicating the content of those operations to the control unit 211.

[0044] The display unit 214 displays various display data under the control of the control unit 211. The display unit 214 is composed of a graphics controller 214a, a buffer memory 214b consisting of memory such as VRAM (Video RAM), and a display 214c consisting of a liquid crystal display or the like. In this configuration, the graphics controller 214a controls the entire display unit 214 based on control data sent from the control unit 211 via the bus line 215. The buffer memory 214b temporarily stores image information that can be displayed immediately. Then, based on the image data output from the graphics controller 214a, an image is displayed on the display 214c.

[0045] The control unit 211 consists of a CPU 211a that controls the entire control unit 211, a ROM 211b that stores control programs and the like in advance for controlling the control unit 211, and a RAM 211c that temporarily stores various data. The control unit 211 is connected to the vehicle speed sensor 221, angular velocity sensor 222, acceleration sensor 223, steering angle sensor 224, and GPS receiver 225 via a bus line 215 and an input / output interface unit 220. Based on the speed data, angular velocity data, relative direction data, steering angle data, GPS positioning data, absolute direction data of the vehicle's direction of travel, acceleration data, etc. output from each of these sensors, the control unit 211 controls the entire NV in-vehicle system and controls the operation of various components such as the display unit 214.

[0046] The control unit 211 acquires time information from the system clock and exchange rate information and SNS post information from a server device (not shown). The exchange rate information acquired is information showing the exchange rate between currencies set by the user, and the SNS post information acquired is information showing posts containing keywords set by the user. The control unit 211 acquires time information and exchange rates at predetermined intervals, and acquires SNS post information each time a new post is uploaded to the server device (not shown).

[0047] The control unit 211 identifies the configuration of the screen currently displayed on the display 214c. For example, it identifies the location and display content of the parts of the screen that are subject to updating (e.g., the time display area 401, the currency exchange area 402, and the posted information display area 403 in Figure 2), and the location of the parts of the screen that can be operated by the user. The control unit 211 also identifies whether the acquired information is for updating the information displayed in which of the parts subject to updating. Meanwhile, the control unit 211 identifies the surrounding environment, such as the weather and road conditions outside the vehicle, based on the captured image output by the external camera 228 and the detection results from the external sensor 229. The control unit 211 also identifies the direction of the user's gaze based on the captured image output by the gaze detection camera 227. Specifically, it identifies which user is looking at which area of ​​the screen displayed on the display 214c: the time display area 401, the currency exchange area 402, or the posted information display area 403.

[0048] The control unit 211 then determines whether or not to update the display contents of the time display area 401, the exchange rate display area 402, and the post information display area 403 based on the acquired time information, exchange rate information, and SNS post information. For example, it can determine whether or not to update the part that is subject to updating (hereinafter sometimes referred to as the "updated part") based on the acquired information, based on at least one of the following rules (1) to (5). (1) If the acquired information is for updating the section that the user is currently focusing on, the update using that acquired information will be suspended (meaning the update at that time will be prohibited; the same applies hereinafter). (2) If the acquired information is information for updating the update portion that is the target of the user's operation, the update using the acquired information will be suspended. (3) If the current location is a near-miss point and the user is the driver of a vehicle equipped with the in-vehicle device according to the third embodiment, the update based on the acquired information will be withheld, taking into account the conditions relating to the more detailed embodiment described later (hereinafter referred to as "detailed conditions relating to the embodiment") and the current level of automation of the vehicle's operation (see Table 1). On the other hand, even if the current location is a near-miss point, if the user is a passenger other than the driver, the update will be performed regardless of the level of automation. (4) If the current surrounding environment is not suitable for updating information while the vehicle equipped with the NV in-vehicle device is in motion, that is, as described above, if there is heavy rain in the surrounding area, if the road being traveled on is congested, or if the road being traveled on is extremely narrow, and the user is the driver, the update based on the acquired information will be suspended, taking into account the detailed conditions of the embodiment and the level of automation. On the other hand, even if the current surrounding environment is not suitable for updating information while the vehicle is in motion, if the user is a passenger, the update will be performed regardless of the level of automation. (5) If the acquired information is for updating the update section that was operated (looked at) by the user within a arbitrarily set time, and the user is the driver, the update based on the acquired information will be suspended. On the other hand, even if the acquired information is for updating the update section that was operated (looked at) by the user within a arbitrarily set time, if the user is a passenger, the update will be performed.

[0049] However, if the acquired information is information for which it is preferable to display the latest information, such as time information, then the information will always be updated with the acquired information, even if it falls under the rules of (1) to (5) above (i.e., regardless of whether it is subject to user action, whether the surrounding environment is suitable for updating information while the vehicle is moving, or the level of automation described above). On the other hand, if SNS posting information falls under the rules of (1) to (5) above (i.e., if it is subject to user action, or if the surrounding environment is not suitable for updating information while the vehicle is moving), then the information will not be updated with the acquired information, depending on the level of automation described above.

[0050] (III) Example of operation of an in-vehicle device according to the embodiment Next, an example of the operation of the in-vehicle device NV according to the embodiment will be explained using the flowchart in Figure 5. It is assumed that calibration of the gaze detection of the vehicle occupant by the gaze detection camera 227 has been completed. The control unit 211 detects the user's gaze, stores the history of gaze movement in the storage device 212, and stores the operation history in the storage device 212 when the user performs an operation.

[0051] First, the control unit 211 acquires output information (time information, exchange rate information, or SNS post information) (step S11). Next, the control unit 211 identifies the update portion (time display area 401, exchange rate display area 402, or post information display area 403) to be updated based on the output information acquired in step S11 (step S12). For example, if the acquired output information is time information, the control unit 211 identifies the time display area 401 as the update portion, and if the acquired output information is SNS post information, it identifies the post information display area 403 as the update portion.

[0052] Next, the control unit 211 determines whether the output information acquired in step S11 does not need to be updated (step S13). For the sake of explanation in this embodiment, it is assumed that time information is output information that does not need to be updated, while exchange rate information and SNS post information are not output information that does not need to be updated (i.e., they are output information that needs to be updated). If the control unit 211 determines that the acquired output information does not need to be updated (time information) (step S13: YES), it updates the information in the update portion (time display area 401) identified in step S12 using the acquired output information (time information) (step S24), and proceeds to step S11. On the other hand, if the control unit 211 determines that the acquired output information does not need to be updated (i.e., it determines that it is exchange rate information or SNS post information) (step S13: NO), it then acquires the user's current gaze information and operation information (step S14).

[0053] Next, the control unit 211 acquires user information indicating the user of the in-vehicle device NV based on images from the gaze detection camera 227 and information from a seat sensor (not shown) (step S15). In this case, user information includes, for example, whether the driver is sitting in the driver's seat, or whether a passenger other than the driver is sitting in the passenger seat or the rear seat.

[0054] Next, the control unit 211 determines, based on the gaze information acquired in step S14, whether the driver's gaze is directed towards the update portion identified in step S12 (step S16). If the control unit 211 determines that the driver's gaze is directed towards the update portion (step S16: YES), it refrains from updating the information displayed in the update portion identified in step S12 using the output information acquired in step S11 (step S25) and proceeds to step S11. As a result, even if new output information is acquired for the update portion that the driver is looking at, the information in the update portion will not be updated by that output information, thus deepening the driver's understanding of the content of the output information. On the other hand, if the control unit 211 determines that the driver's gaze is not directed towards the update portion (step S16: NO), it then determines, based on the operation information acquired in step S14, whether the target of the driver's operation is the update portion identified in step S12 (step S17). In this case, for example, if the output information obtained in step S11 is SNS post information and the driver is performing a scroll operation on the post information display area 403, then "YES" is determined.

[0055] If the control unit 211 determines that the driver's target of operation is the update section identified in the process of step S12 (step S17: YES), it refrains from updating the information displayed on the update section identified in the process of step S12 using the output information acquired in step S11 (step S25) and proceeds to the process of step S11. As a result, even if new output information is acquired for the update section being operated by the driver, the information on the update section will not be updated by that output information, thus deepening the driver's understanding of the content of the output information. On the other hand, if the control unit 211 determines that the driver's current target of operation is not the update section identified in the process of step S12 (step S17: NO), it then acquires the driver's operation history information, gaze movement history information, current location information, and surrounding environment information (step S18). Specifically, the control unit 211 acquires driver operation history information and gaze movement history information from the storage device 212, acquires latitude, longitude, and altitude data output by the GPS receiver 225 as current location information, and acquires surrounding environment information by analyzing images of the outside of the vehicle output by the external camera 228 and detection results output by the external sensor 229. Surrounding environment information indicates whether the current surrounding environment is in an unsuitable state for updating information during vehicle movement, and is calculated and acquired by the control unit 211 based on the images captured by the external camera 228 and detection results output by the external sensor 229, respectively, based on the weather and road conditions outside the vehicle. For example, if there is heavy rain, the road being traveled on is congested, or the road being traveled on is extremely narrow, surrounding environment information indicating that the situation is unsuitable for updating information during vehicle movement will be acquired. The control unit 211 may transmit the vehicle's current location via the network to a server device (not shown) that holds information indicating weather and road conditions for each location, obtain the information corresponding to the current location from the server device, and calculate surrounding environment information based on that information.The control unit 211 may also transmit the vehicle's current position via the network to a server device that calculates inappropriate information for each position, indicating the degree to which the surrounding environment is unsuitable for updating information during vehicle movement, and obtain the inappropriate information from the server device as surrounding environment information.

[0056] Next, the control unit 211 determines whether the level of automation in the current driving state of the vehicle is level 4 as shown in Table 1 above, that is, whether the vehicle equipped with the NV in-vehicle device is moving in a fully automated driving state (step S19). For this determination in step S19, for example, automated driving state information output from an automated driving control unit (not shown) can be used. The process of detecting the level of automation itself can be carried out using the same method as in the past. If, in the determination in step S19, the level of automation in the current driving state of the vehicle is level 4 as shown in Table 1 above (step S19: YES), the control unit 211 proceeds to step S24 and updates the necessary update parts.

[0057] On the other hand, if the level of automation in step S19 is a level other than level 4 (step S19: NO), the control unit 211 then determines, based on the user information acquired in step S15, whether the current user of the in-vehicle device NV is the driver (step S20). If the determination in step S20 is that the current user of the in-vehicle device NV is a passenger other than the driver (step S20: NO), the control unit 211 proceeds to step S24 to update the necessary update parts. On the other hand, if the determination in step S20 is that the current user of the in-vehicle device NV is the driver (step S20: YES), the control unit 211 then determines, based on the driver's operation history information and gaze movement history information acquired in step S18, whether the update part identified in step S12 is an update part that was operated on or looked at within a predetermined time from the present (step S21). For example, the control unit 211 determines whether the update part was operated on or looked at within the last 30 seconds. If the control unit 221 determines that the update section has been operated on or looked at within a predetermined time (step S21: YES), it refrains from updating the information displayed on the update section identified by the processing in step S12 using the output information acquired in step S11 (step S25) and proceeds to the processing in step S11. As a result, even if new output information is acquired for the update section that the driver was looking at or operating on up to the predetermined time, the information on the update section will not be updated by that output information, so when the driver looks at the information again to view the information, they can view the same output information as before. On the other hand, if the control unit 211 determines that the update section has not been operated on or looked at within a predetermined time (step S21: NO), it then determines whether the current location is a near-miss point or its vicinity (for example, within 50 meters of a near-miss point; the same applies hereinafter) based on the current location information acquired in the processing in step S18 and the information indicating a near-miss point stored in the storage device 212 (step S22).

[0058] If, in the determination in step S22, it is determined that the current location is a near-miss point or its vicinity (step S22: YES), the control unit 211 then determines which level of automation the vehicle is currently operating at, excluding level 4 shown in Table 1 (see step S19), that is, whether the vehicle equipped with the NV in-vehicle device is moving at any level of automation other than fully automated driving (step S26). This determination in step S26 can also be made using, for example, the automated driving state information described above, similar to the determination in step S19. If, in the determination in step S26, the level of automation the vehicle is currently operating at is level 3 shown in Table 1 (step S26: 3), the control unit 211 proceeds to step S23, which will be described later. On the other hand, in the determination in step S26, if the level of automation in the current driving state of the vehicle is level 1 or level 2 as shown in Table 1 above (step S26: 1, 2), the control unit 211 then makes a determination to postpone updating the information displayed in the update section identified by the processing in step S12, based on the output information acquired in step S11, based on the detailed conditions related to the embodiment (step S27). If the determination in step S27 is to postpone updating the information based on the detailed conditions (step S27: YES), the control unit 211 postpones updating the information (step S25) and proceeds to the processing in step S11. As a result, if the current position is a near-miss point, the level of automation is low, and the information update is postponed based on the detailed conditions, even if new output information is acquired, the information in the update section will not be updated by that output information, so the driver can concentrate on driving. On the other hand, if the control unit 211 decides not to postpone updating the information based on the detailed conditions in step S27 (i.e., to update the information) (step S27: NO), it proceeds to step S23, which will be described later.

[0059] On the other hand, if the determination in step S22 determines that the current location is not a near-miss point or its vicinity (step S22: NO), the control unit 211 determines, based on the surrounding environment information acquired in step S18, whether the surrounding environment is in a situation where it is highly unsuitable for updating information during vehicle movement (step S23).

[0060] If, in the determination in step S23, it is determined that the surrounding environment is not one where the degree of the condition is high (step S23: NO), the control unit 211 updates the information in the update portion (exchange rate display area 402 or post information display area 403) identified in the process of step S12 using the output information (exchange rate information or SNS post information) obtained in the process of step S11 (step S24), and proceeds to the process of step S11.

[0061] On the other hand, if the determination in step S23 determines that the surrounding environment is highly unsuitable for updating information during vehicle movement (step S23: YES), the control unit 211 then determines again which level of automation the current vehicle driving state is at, excluding level 4 shown in Table 1 (see step S19), that is, whether the vehicle equipped with the NV in-vehicle device is moving at any level of automation other than fully automated driving (step S28). This determination in step S28 can also use, for example, the automated driving state information, as in the determinations in step S19 or step S26. If the determination in step S28 determines that the current vehicle driving state is at level 2 or level 3 shown in Table 1 (step S28: 2, 3), the control unit 211 proceeds to step S24. On the other hand, in the determination in step S28, if the level of automation in the current driving state of the vehicle is level 1 as shown in Table 1 above (step S28:1), the control unit 211 then makes a determination to postpone updating the information displayed in the update section identified by the processing in step S12, based on the output information acquired in step S11, based on the detailed conditions related to the same embodiment as in step S27 (step S29). If the determination in step S29 is to postpone updating the information based on the detailed conditions (step S29:YES), the control unit 211 postpones updating the information (step S25) and proceeds to the processing in step S11. As a result, in the surrounding environment where the degree of automation is high, the level of automation is low, and the information update is postponed based on the detailed conditions, even if new output information is acquired, the information in the update section will not be updated by that output information, so the driver can concentrate on driving. On the other hand, if the determination in step S29 is not to postpone updating the information based on the detailed conditions (i.e., to update the information) (step S29:NO), the control unit 211 proceeds to step S24.

[0062] Furthermore, the determination in step S28 is based on the premise that the control unit 211 can determine the degree based on the captured image output by the external camera 228 and the detection result output by the external sensor 229. However, in special cases where the control unit 211 cannot determine the degree based on the captured image output by the external camera 228 and the detection result output by the external sensor 229, for example due to an extremely strong storm, it is preferable for the control unit 211 to proceed to step S29 for all automation levels other than level 4, that is, to skip the determination in step S28.

[0063] Next, the decision to withhold based on the detailed conditions of the embodiment in steps S27 and S29 will be explained in detail using Figure 6. Note that the decision to withhold based on the detailed conditions of the embodiment in step S27 and the decision to withhold based on the same detailed conditions in step S29 are the same, so they will be explained together using Figure 6.

[0064] As shown in Figure 6, in each case where the determination in step S26 is "1, 2" (in the case of step S27) or where the determination in step S28 is "1" (in the case of step S29), the control unit 211 first acquires the road information and parking information from the storage device 212 (step SA), and then detects the current location of the vehicle on which the in-vehicle device NV is installed based on the current location information from the GPS receiver 225 (step SB). Next, the control unit 211 determines whether or not the vehicle is currently located off the road based on the current location detected in step SB (step SC).

[0065] Here, the term "off the road" in relation to the embodiments refers to, for example, a location or area where parking or stopping of a vehicle equipped with the NV2 in-vehicle device according to the second embodiment or the in-vehicle device according to the third embodiment is permitted under the Road Traffic Act (Act No. 105 of June 25, 1960). More specifically, the term "off the road" refers to, for example, the following locations or areas. • Roadside areas defined in the Road Traffic Act, excluding "roadside areas where parking and stopping are prohibited" and "roadside areas exclusively for pedestrians" as defined in the Road Traffic Act. - In cases where a separate sidewalk is provided alongside the roadway, the area of ​​the roadway outside the outer edge line when viewed from the centerline of the roadway. • So-called parking lots or parking spaces (for example, time-limited parking spaces on roads, etc.) In the determination in step SC described above, if the vehicle is not currently located outside the road (i.e., the vehicle is located within a road other than the road outside the road) (step SC: NO), the control unit 211 determines "YES" for the determination in step S27 or step S29, and proceeds to step S25 in Figure 5.

[0066] On the other hand, in the determination of step SC, if the vehicle is currently located outside the road (step SC: YES), the control unit 211 then determines whether the vehicle is currently stopped or not based on the vehicle's position detected in step SB (step SD). In the determination of step SD, if the vehicle is not currently stopped (i.e., the vehicle is moving) (step SD: NO), the control unit 211 determines "YES" for the determination in step S27 or step S29 and proceeds to step S25.

[0067] On the other hand, in the determination of step SD, if the vehicle is currently stopped (step SD: YES), the control unit 211 determines "NO" for the determination of step S27 or step S29, and proceeds to step S23 (in the case of step S27) or step S24 (in the case of step S29).

[0068] As the operation examples of the in-vehicle device NV, including the determinations in step S26 and step S28 described above, are performed mainly by the control unit 211, for example, in the determination in step S16, if the driver's gaze is not directed towards the update part (see Figure 5 step S16: NO), and the driver's target of operation is not the update part (see Figure 5 step S17: NO), then if the level of automation in the current driving state of the vehicle is level 4 as shown in Table 1 (see step S19: YES), the vehicle is in a fully automated driving state, and therefore the information of the update part is updated (see Figure 5 step S24).

[0069] Furthermore, for example, if, in the determination in step S16 above, the driver's gaze is not directed towards the part to be updated (see Figure 5 step S16: NO), and the driver's target of operation is not the part to be updated (see Figure 5 step S17: NO), and the level of automation in the current driving state of the vehicle is level 3 as shown in Table 1 above (Figure 5 step S26: NO), and there is no traffic congestion on the road and the weather is clear (Figure 5 step S23: NO), then even if the vehicle's current position at the time the output information for updating is acquired in step S11 above is at or near a near-miss point (see Figure 5 step S22: YES), it is considered that there is no problem, and the information for the part to be updated is updated (see Figure 5 step S24).

[0070] Furthermore, for example, if, in the determination in step S16 above, the driver's gaze is not directed towards the part to be updated (see Figure 5 step S16: NO), and the driver's target of operation is not the part to be updated (see Figure 5 step S17: NO), and the level of automation in the current driving state of the vehicle is level 3 as shown in Table 1 above (Figure 5 step S28: 2, 3), then even if there is no congestion on the road but the external sensor 229 detects a rainfall amount that would hinder the vehicle's movement (see Figure 5 step S23: YES), it is considered that there is no problem, and the information in the part to be updated is updated (see Figure 5 step S24).

[0071] As described above, in the control of the update of output information by the control unit 211 of the in-vehicle device NV according to the embodiment (an example of "output information acquisition means," "state information acquisition means," "control means," "surrounding environment information acquisition means," and "detection means"), the possibility of updating the output information is controlled based on the control state related to movement in the vehicle. Therefore, the possibility of updating the output information can be controlled in a fine-grained and appropriate manner, allowing the driver to concentrate on driving.

[0072] Furthermore, in the control of whether or not the output information can be updated by the control unit 211, the control state is one of several pre-set levels of automation in the operation of the vehicle (see Table 1), and the state information acquired by the control unit 211 indicates which level of automation the vehicle's operation is currently at. Therefore, the ability to update the output information can be controlled in a fine-grained and appropriate manner according to the level of automation in the operation of the vehicle, allowing the driver to concentrate on driving.

[0073] Furthermore, the control unit 211 controls whether or not to update the output information based on the vehicle's control state and the surrounding environment. This allows for more precise and appropriate control over whether or not to update the output information, enabling the driver to concentrate on driving.

[0074] Furthermore, the control unit 211 controls whether or not to update the output information. It controls the updating of the output information when it is indicated that the vehicle's driving state is in a fully automated driving state, while prohibiting the updating of the output information when it is indicated that the driving state is an automated state corresponding to automation level 1 (see Figure 5 step S28:1) and the surrounding environment is not suitable for updating the output information (see Figure 5 step S23:YES). This allows for fine-grained and appropriate control over whether or not to update the output information, enabling the driver to concentrate on driving.

[0075] Furthermore, in the control unit 211's control of whether or not to update the output information, if the output information is subject to the actions of a passenger other than the driver (Figure 5, step S20: NO), the control is set to allow the update of the output information regardless of the surrounding environment. Therefore, if the output information is subject to the actions of a passenger other than the driver, the output information can be updated appropriately.

[0076] In the special case described above, where the control unit 211 controls whether or not to update the output information, and the vehicle's control state is either a fully automated driving state corresponding to automation level 4, or an automated driving state other than the said fully automated driving state, and the state information acquired by the control unit 211 indicates one of these driving states, and the control unit 211 indicates that the vehicle's driving state is a driving state other than the fully automated driving state, then the control unit 211 controls whether or not to update the output information in a fine-grained and appropriate manner, allowing the driver to concentrate on driving.

[0077] Furthermore, in the above embodiment, step S22 determines whether the current location is a near-miss point, and step S23 determines whether the surrounding environment is in an inappropriate state for updating information during vehicle movement. However, even if the current location is a near-miss point, the surrounding environment is also considered to be in an inappropriate state for updating information during vehicle movement. Therefore, the "surrounding environment is inappropriate for updating information during movement" in the present invention also includes the case where the current location is a near-miss point.

[0078] (IV) Variation Next, we will describe some variations of the above embodiment. Note that the variations described below can be combined as appropriate.

[0079] (IV.1) First variation In the above embodiment, the display 214c was exemplified as a device for outputting information, but other output devices (for example, a HUD (Head-Up Display; see Figure 4)) may also be provided, and the updates of information output by multiple output devices may be controlled. In this case, the control unit 211 will be able to identify the configuration of the screens displayed on the display 214c and the HUD.

[0080] (IV.2) Second variation In the above embodiment, the gaze of the user (driver, passenger) is detected based on the captured image output by the gaze detection camera 227, but the gaze of the user may also be detected based on the data output by the electrooculography sensor.

[0081] (IV.3) Third variation In the above embodiment, the process in step S22 determines whether the current location is a near-miss point or not. However, it may also be determined whether the current location is approaching a near-miss point or not. In this case, if the current location is approaching a near-miss point or is a near-miss point, the update will be postponed, allowing the user (driver) to concentrate on driving.

[0082] (IV.4) Fourth variation In the above embodiment, after the update is postponed in step S25, the process moves to step S11 to acquire the next output information. As a result, the output information acquired in the most recent step S11 is not used for the update and is discarded. However, in order to ensure that the update is performed using the output information acquired in step S11, the output information acquired in step S11 is stored in the storage device 212, and when executing step S24, the stored output information is used for the update in order from the oldest output information (FIFO (First In First Out)).

[0083] (IV.5) Fifth variation The above embodiment describes a case where the information processing device is an in-vehicle device (NV) mounted on a vehicle, but it can be applied to any device that moves with a moving object and outputs information. For example, it can be applied to PCs (personal computers), notebook PCs, smartphones, tablets, etc.

[0084] (IV.6) Sixth variation In the flowchart of Figure 5 relating to the above embodiment, steps S16 and S17, steps S18 to S23, and steps S26 to S29 may be rearranged while maintaining the order within their respective processing (step) groups. [Explanation of Symbols]

[0085] 1. Information Processing Device 111A Output information acquisition means 111B Means for acquiring status information 111C Control means 111D Means for acquiring ambient environment information 111E Detection means 2 Output means 211 Control Unit 211a CPU 211b ROM 211c RAM 212 Storage device 213 Input device 214 Display Unit 214a Graphics Controller 214b Buffer memory 214c display 215 Bus Line 220 Input / Output Interface Section 221 Vehicle speed sensor 222 angular velocity sensors 223 Accelerometer 224 Steering angle sensor 225 GPS receiver 226 Data transmission / reception unit 227 Eye-tracking camera 228 Exterior car camera 229 External vehicle sensors NV in-vehicle equipment

Claims

[Claim 1] An output information acquisition means that acquires output information that is output to and can be updated by an output means provided on a mobile body, A state information acquisition means for acquiring state information indicating the control state related to the movement of the aforementioned moving body, A control means that controls whether or not to update the output information based on the acquired state information, An information processing device characterized by comprising:

Images