Information processing method, information processing device, and program

Abstract

The present invention comprises: a determination process for determining, on the basis of a linear movement path (first path) from a first position to a second position and content displayed on a display unit, whether or not the visibility of specific content is impaired if an agent image, which is displayed at the first position on the display unit, moves along the first path when the agent image moves to the second position on the basis of the state of an occupant; and a control process for executing a first movement process or the like for moving the agent image along a movement path (second path) which is different from the first path and which does not impair the visibility of the specific content if the visibility of the specific content is impaired.

Description

Information processing method, information processing device, and program

[0001] The present invention relates to an information processing method, an information processing device, and a program for controlling an agent capable of communicating with a user.

[0002] Conventionally, technologies for conveying various types of information to a user in a vehicle have been proposed. For example, JP2020-055348A proposes a technology for moving the display position of an agent image to the display position of a provided image when the image is displayed on a display unit to guide the gaze of the occupant to the display position of the provided image.

[0003] In the above-described conventional technology, when the display position of the agent image is moved to the display position of the image to be provided to the occupant, there is a risk that the content displayed on the display unit may be obscured by the moving agent image. For example, if content related to the vehicle's driving route, vehicle driving control, etc. is displayed on the display unit, there is a risk that the moving agent image may obscure that content. Therefore, it is important to ensure that the visibility of specific content is not obstructed by the moving agent image.

[0004] An object of the present invention is to appropriately move an agent image on a display unit so as not to impair the visibility of specific content.

[0005] One aspect of the present invention is an information processing method for displaying an agent image on a display unit that communicates with a vehicle occupant, and controlling the display state of the agent image based on the state of the occupant. This information processing method includes, when an agent image displayed at a first position on the display unit moves to a second position based on the state of the occupant, a determination process for determining whether or not movement of the agent image along the first path will obstruct visibility of specific content, based on a linear movement path (first path) from the first position to the second position and content displayed on the display unit, and, if the visibility of the specific content will not be obstructed, moving the agent image along the first path in a first display mode, and, if the visibility of the specific content will be obstructed, moving the agent image along the first path in a first display mode, and, if the visibility of the specific content will be obstructed, moving a movement path (second path) different from the first path that does not obstruct visibility of the specific content. a control process for executing one of a first movement process for moving the agent image along a first display mode (a first moving mode), a second movement process for moving the agent image from a first position to a second position in a second display mode that is a display mode different from the first display mode and does not impede the visibility of the specific content, and a third movement process for moving the agent image from the first position to the second position at a second movement timing different from the first movement timing in the first display mode and that does not impede the visibility of the specific content, or at a second movement speed different from the first movement speed in the first display mode and that does not impede the visibility of the specific content.

[0006] FIG. 1 is a simplified diagram showing an example of the configuration of the interior of a vehicle. FIG. 2 is a simplified diagram showing an example of the configuration of the interior of a vehicle. FIG. 3 is a simplified diagram showing an example of the configuration of the interior of a vehicle. FIG. 4 is a diagram showing an example of transition when an agent image is moved. FIG. 5 is a block diagram showing an example of the system configuration of an information processing system. FIG. 6 is a diagram showing an example of transition of an agent image. FIG. 7 is a diagram showing an example of transition of an agent image. FIG. 8 is a diagram showing an example of transition of an agent image. FIG. 9 is a diagram showing an example of transition of an agent image. FIG. 10 is a flowchart showing an example of agent movement processing. FIG. 11 is a flowchart showing an example of agent movement processing.

[0007] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0008] [Example of installation of display unit] Fig. 1 is a simplified diagram showing an example of the configuration of the interior of a vehicle C1. Fig. 1 shows the interior of the vehicle C1 in front of the driver's seat and passenger seat (not shown) as viewed from the rear in the longitudinal direction of the vehicle C1. To facilitate explanation, Fig. 1 omits illustrations of only the dashboard 2, steering wheel 3, windshield 4, rearview mirror 5, and output device 200.

[0009] The output device 200 is an output device installed inside the vehicle C1, and executes various output operations based on instructions from the information processing device 110 (see FIG. 5 ). The output device 200 is an device installed on the dashboard 2, and includes a display unit 201, a sound output unit 202, and a reception unit 203 (see FIG. 5 ). In FIG. 1 , an output device that is long in the left-right direction of the vehicle C1 is shown as an example of the output device 200. The output device 200 is an device whose display area extends, for example, from near the center of the dashboard 2 to the driver's seat.

[0010] For example, the output device 200 is an in-vehicle system including one or more devices capable of providing various types of information. For example, the output device 200 may be at least one of a navigation device, an audio device, a DVD device, a TV tuner, an IVI (In-Vehicle Infotainment), etc. Images displayed on each of the output devices 200 may be displayed using a HUD (Head Up Display) implemented on the windshield 4 or other display devices. Alternatively, an information processing device such as a portable device that the driver D1 can carry (or a device that can be installed in the vehicle C1), for example, a smartphone, a tablet terminal, or a portable personal computer, may be used.

[0011] Furthermore, the output device 200 executes an output operation that enables communication with users (including the driver D1) riding in the vehicle C1. For example, an agent image AG1 (see FIGS. 2 and 3) that enables communication with the occupants is displayed on the display unit 201 of the output device 200, and various effects are executed using the agent image AG1. Note that an agent image that enables communication with the occupants may be displayed on the windshield 4 (HUD display area) or another display device, and various effects may be executed using the agent image.

[0012] [Display example of agent image] Figures 2 and 3 are simplified diagrams showing an example of the configuration inside the vehicle C1. Note that Figures 2 and 3 are similar to the configuration example shown in Figure 1, so detailed description will be omitted here.

[0013] 2 and 3 show an example in which content CT1 and an agent image AG1 are displayed on the display unit 201 of the output device 200. The content CT1 is content that displays map information including a current location indicator PL1 indicating the current location of the vehicle C1 and route information RI1 indicating the travel route of the vehicle C1. For example, the route information RI1 is displayed in a display mode different from that of a map (for example, an arrow colored in blue, red, etc.). Although not shown, it is also possible to display map information with a different viewpoint as appropriate. For example, the content CT1 is displayed based on a navigation function.

[0014] The agent image AG1 is an image representing an object capable of communicating with the user. In this embodiment, an example is shown in which an image imitating a human face is used as the agent image AG1. However, the agent image AG1 is not limited to this. For example, the agent image AG1 may be an image imitating the entire body or part of a human (e.g., the upper body), an image imitating an animal such as a rabbit or pig (or the face of that animal), or an image imitating a virtual creature (e.g., the face of an anime character) (or the face of that animal). In this way, a mimicked image can be used as the agent image AG1.

[0015] The output device 200 executes various operations related to driving assistance (for example, operations related to the content CT1 and the agent image AG1) based on instructions from the information processing device 110 (see FIG. 5 ). For example, the output device 200 outputs various information related to driving assistance when the driver D1 performs driving operations, surrounding facilities, etc. based on the control of the information processing device 110.

[0016] Driving assistance using the agent image AG1 is expected to include, for example, a notification of a moving object ahead or behind. For example, to notify a moving object ahead, a voice output such as "Be careful, there is a railroad crossing ahead" or "There is a person ahead" can be used. Various actions can be performed using the agent image AG1 at the time of each of these outputs. That is, driving assistance can be performed using the agent image AG1. Furthermore, for example, the output device 200 executes various processes using the agent image AG1 based on the control of the information processing device 110. For example, the output device 200 executes a communication process for various exchanges with the driver D1 using the agent image AG1. For example, the output device 200 uses the agent image AG1 to exchange various conversations with the driver D1 or provide information preferred by the driver D1.

[0017] For example, as shown in Fig. 2, it is conceivable that in response to the agent image AG1 uttering voice information S1 "Turn right at the next intersection," the driver D1 utters voice S2 "Is there a highway beyond that?" Then, in response to the voice S2 uttered by the driver D1, the agent image AG1 utters voice information S3 "We'll get on the highway next" (see Fig. 3). Note that these exchanges are simplified and are not limited to these.

[0018] In this way, when an occupant of vehicle C1 makes some kind of user input, it is conceivable to move the agent image AG1 near the occupant who made the user input, making it easier to provide assistance to the occupant. For example, suppose that when agent image AG1 is displayed at the position shown in FIG. 2, driver D1 utters voice S2, "Is that a highway up ahead?" In this case, it is conceivable to move agent image AG1 near driver D1 (for example, in the direction of driver D1's line of sight), as shown in FIG. 3. In other words, it is conceivable to move agent image AG1 to a position where the user is operating it or a position where the user wants to see it. A method for moving agent image AG1 in this case will be described with reference to FIG. 4.

[0019] [Example of Movement of Agent Image] FIG. 4 is a diagram showing an example of transition when the agent image AG1 is moved on the display unit 201 of the output device 200. As shown in FIG.

[0020] In FIG. 4A, when the agent image AG1 is moved on the display unit 201, the agent image from which the agent image AG1 is moved is shown as agent image AG1a, and the agent image to which the agent image AG1 is moved is shown as agent image AG1e.

[0021] In Figure 4(B), dotted lines MT1 and MT2 indicate the range of the movement trajectory of agent image AG1 when it moves linearly from agent image AG1a to agent image AG1e on display unit 201. Also in Figure 4(B), parts of the movement trajectory of agent image AG1 are schematically indicated by dotted circles AG1a to AG1e. In this way, Figure 4(B) shows the relationship between the first movement trajectory AG1a to AG1e of agent image AG1 from the first position AG1a to the second position AG1e and the currently displayed content CT1.

[0022] Here, an example is shown in which a linear movement path is set from the initial position (first position AG1a) of the agent image AG1 to a response position (second position AG1e), which is the position to which the agent image AG1 is moved in response to user input (e.g., manual input, voice input).

[0023] Each position on the display surface of the display unit 201 can be managed as coordinate information in the agent management DB 132 (see FIG. 5 ), for example. For example, the output content determination unit 126 (see FIG. 5 ) can acquire the current position of the agent image AG1 by sequentially storing the positions of each image (content image, agent image AG1) displayed on the display surface of the display unit 201 in the agent management DB 132. Furthermore, the movement mode determination unit 127 can calculate the range of the movement trajectory of the agent image AG1 moving on the display surface of the display unit 201 based on the display region (display area) of the agent image AG1, the current position of the agent image AG1, and the position to which the agent image AG1 is to be moved. A known calculation method can be used to calculate this range.

[0024] For example, it is conceivable to move the agent image AG1 based on the line of sight of the occupant of the vehicle C1, the position of the occupant's hands, etc. For example, when the occupant performs a desired operation on the display unit 201, it is possible to move the agent image AG1 to a position that is easy for the occupant to operate based on the occupant's behavior. For example, when the occupant operates with their hands, it is conceivable to move the agent image AG1 to a position close to their hands. For example, it is conceivable to move the agent image AG1 in a straight line from the start point to the destination point, as shown by arrow AW1.

[0025] In this way, by having the agent image AG1 follow the user's operation and moving the agent image AG1 continuously, it is possible to show that the agent image AG1 is responding to the user's operation. Furthermore, by looking at the agent image AG1, the user can easily visually grasp the information that the agent image AG1 is trying to convey. For this reason, it is conceivable that the user may feel uneasy if the agent image AG1 disappears even temporarily. However, there is a possibility that the agent image AG1 may overlap with information that should not be hidden while moving in response to the user's input.

[0026] For example, if the agent image AG1 is displayed in the foreground and the vehicle moves along the path indicated by the arrow AW1, the agent image AG1 will overlap the front side of the content CT1 (i.e., the display surface side of the display unit 201) as it moves. In this case, the vehicle occupant cannot see part of the content CT1 while the agent image AG1 is moving. In other words, the movement of the agent image AG1 will impede the visibility of the content CT1.

[0027] Therefore, in this embodiment, if the movement of the agent image AG1 impairs the visibility of specific content, the movement path of the agent image AG1 is changed, the display mode of the agent image AG1 is changed, or the movement timing or movement speed of the agent image AG1 is changed so as not to impair the visibility of the specific content.

[0028] [Configuration Example of Information Processing System] FIG. 5 is a block diagram showing an example of the system configuration of the information processing system 100 installed in the vehicle C1.

[0029] The information processing system 100 includes a sound acquisition unit 101, a driver image acquisition unit 102, a vehicle interior image acquisition unit 103, a vehicle exterior image acquisition unit 104, an information processing device 110, and an output device 200. The information processing device 110 is an example of a device that controls the output device 200, which is capable of communicating with occupants (including the driver D1) of the vehicle C1.

[0030] The information processing device 110 and the output device 200 are connected by a communication method using wired communication or wireless communication. The information processing device 110 is also connected to the network 20 by a communication method using wireless communication. The network 20 is a network such as a public line network or the Internet. The output device 200 may also be connected to the network 20 by a communication method using wireless communication. Although Fig. 5 shows an example in which the information processing device 110 and the output device 200 are configured as separate entities, the information processing device 110 and the output device 200 may also be configured as an integrated device.

[0031] The sound acquisition unit 101 is provided inside the vehicle C1, acquires sounds inside the vehicle C1, and outputs sound information related to the acquired sounds to the information processing device 110. As the sound acquisition unit 101, for example, one or more microphones or sound acquisition sensors can be used.

[0032] The driver image acquisition unit 102 captures an image of the driver D1 in the vehicle C1 to generate an image (image data), and outputs image information related to the generated image to the information processing device 110. The driver image acquisition unit 102 is provided at least inside the vehicle C1 and captures an image of the driver D1 in the vehicle C1 to generate an image (image data). The driver image acquisition unit 102 is configured, for example, with one or more camera devices or image sensors capable of capturing an image of the driver D1. For example, one driver image acquisition unit 102 may be provided in the front of the interior of the vehicle C1 (e.g., on the ceiling), and capture an image of the driver D1 from in front of the vehicle C1 to generate an image (image data). For example, the driver image acquisition unit 102 may be provided above the windshield 4, i.e., above the rearview mirror 5. The driver image acquisition unit 102 and the vehicle interior image acquisition unit 103 may be configured as the same device or as different devices.

[0033] The vehicle interior image acquisition unit 103 captures an image of a subject inside the vehicle C1 to generate an image (image data), and outputs image information related to the generated image to the information processing device 110. The vehicle interior image acquisition unit 103 is provided at least inside the vehicle C1 (e.g., on the ceiling) and captures an image of a subject inside the vehicle C1 to generate an image (image data). The vehicle interior image acquisition unit 103 is configured, for example, with one or more camera devices or image sensors capable of capturing an image of a subject. For example, one vehicle interior image acquisition unit 103 may be provided in the front of the vehicle C1 to capture an image of a subject from the front to generate an image (image data), or another vehicle interior image acquisition unit 103 may be provided in the rear of the vehicle C1 to capture an image of a subject from the rear to generate an image (image data).

[0034] The outside-vehicle image acquisition unit 104 captures images of subjects outside the vehicle C1 to generate images (image data), and outputs image information related to the generated images to the information processing device 110. Two or more outside-vehicle image acquisition units 104 may be provided, and all or some of the images from these outside-vehicle image acquisition units 104 may be used. For example, one outside-vehicle image acquisition unit 104 may be provided in front of the vehicle C1 to capture images of subjects in front of the vehicle C1 to generate images (image data), and another outside-vehicle image acquisition unit 104 may be provided behind the vehicle C1 to capture images of subjects behind the vehicle C1 to generate images (image data). Alternatively, one or more devices capable of capturing images of subjects in all directions from the vehicle C1 and subjects inside the vehicle C1, such as a 360-degree camera, may be used.

[0035] The driver image acquisition unit 102, the vehicle interior image acquisition unit 103, and the vehicle exterior image acquisition unit 104 are each configured with, for example, an image sensor that receives light from a subject collected by a lens and an image processing unit that performs predetermined image processing on image data generated by the image sensor. As the image sensor, for example, a CCD (Charge Coupled Device) type or a CMOS (Complementary Metal Oxide Semiconductor) type image sensor can be used.

[0036] The information processing device 110 includes a control unit 120, a storage unit 130, and a communication unit 140. The communication unit 140 exchanges various types of information with other devices using wired or wireless communication under the control of the control unit 120. For example, when the communication unit 140 receives driving assistance information, operation information of the agent image AG1, etc. from an external device (e.g., a server), it outputs the information to the control unit 120.

[0037] The control unit 120 controls each unit based on various programs stored in the storage unit 130. The control unit 120 is realized by a processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). Note that the vehicle ECU (Electronic Control Unit) of the vehicle C1 may also be used as the control unit 120, or a processing device different from the vehicle ECU may be provided as the control unit 120.

[0038] The control unit 120 executes various controls based on the information output from the sound acquisition unit 101, the driver image acquisition unit 102, the vehicle interior image acquisition unit 103, the vehicle exterior image acquisition unit 104, the communication unit 140, etc., and the information acquired by the vehicle information acquisition unit 125. For example, the control unit 120 executes a control process to control the operating state of the output device 200. Specifically, the control unit 120 includes an utterance acquisition unit 121, a driver status acquisition unit 122, a vehicle interior status acquisition unit 123, a vehicle exterior status acquisition unit 124, a vehicle information acquisition unit 125, an output content determination unit 126, a movement mode determination unit 127, and an output control unit 128.

[0039] The speech acquisition unit 121 executes a predetermined sound analysis process on the sound information output from the sound acquisition unit 101 to acquire speech information related to the speech of each user (including the driver D1) contained in the sound information, and outputs the speech information to the output content determination unit 126. This sound analysis process can be performed using a known sound analysis process.

[0040] The driver status acquisition unit 122 executes a predetermined image analysis process on the image information output from the driver image acquisition unit 102 to acquire various pieces of information related to the driver D1 contained in the image information, and outputs the acquired information to the output content determination unit 126. This image analysis process can be performed using a known image analysis process. As various pieces of information related to the driver D1, for example, the driver D1's facial expression, gaze, hand, face, body movements, and other actions of the driver D1 can be acquired. From these, it is possible to detect, for example, whether the driver D1 has gotten in and out of the vehicle C1, whether each seat in the vehicle C1 is occupied, and hand movements of the driver D1. In other words, the driver status acquisition unit 122 acquires driver status information related to the state of the driver D1.

[0041] The cabin status acquisition unit 123 executes a predetermined image analysis process on the image information output from the cabin image acquisition unit 103 to acquire various pieces of information about the cabin of the vehicle C1 contained in the image information, and outputs the acquired information to the output content determination unit 126. This image analysis process can be performed using a known image analysis process. As the various pieces of information about the cabin of the vehicle C1, for example, each user's facial expression, gaze, hand, face, body, and other actions can be acquired. That is, the cabin status acquisition unit 123 acquires user status information about the status of each user riding in the vehicle C1 and vehicle status information about the status of the vehicle C1.

[0042] The vehicle exterior situation acquisition unit 124 acquires various pieces of information about the outside of the vehicle C1 contained in the image information by performing a predetermined image analysis process on the image information output from the vehicle exterior image acquisition unit 104, and outputs the pieces of information to the output content determination unit 126. This image analysis process can be performed using known image analysis processes. As the various pieces of information about the outside of the vehicle C1, for example, it is possible to detect whether the vehicle C1 is traveling on a road, whether the vehicle C1 is stopped on a road, and traffic lights, signs, etc. that are present ahead of the vehicle C1. In other words, the vehicle exterior situation acquisition unit 124 acquires vehicle state information about the state of the vehicle C1.

[0043] The vehicle information acquisition unit 125 acquires information (vehicle state information) related to various vehicle states of the vehicle C1 and outputs the acquired vehicle state information to the output content determination unit 126. The vehicle state information can be acquired, for example, from a CAN (Controller Area Network) signal. The vehicle state information includes, for example, vehicle speed, acceleration, shift lever position (e.g., P range, D range), accelerator pedal depression amount, brake pedal depression amount, position information, and abnormality occurrence information. For example, it is possible to determine whether the vehicle C1 is stopped or moving based on the vehicle speed, acceleration, etc. Furthermore, various warning information can be displayed based on the abnormality occurrence information related to the vehicle C1.

[0044] The vehicle information acquisition unit 125 may also acquire sensor detection information output from various sensors installed in the vehicle C1. Examples of the sensors include a LiDAR (Light Detection and Ranging), a RADAR (Radio Detection and Ranging), sonar, a vehicle speed sensor, an acceleration sensor, a steering sensor, an accelerator position sensor, a position information acquisition sensor (position information acquisition unit), a seating sensor, a seat belt sensor, and the like. Known sensors can be used for each of these sensors. The LiDAR, RADAR, sonar, and the like are examples of sensors that detect the situation around the vehicle C1. The vehicle speed sensor, acceleration sensor, steering sensor, accelerator position sensor, and the like are examples of sensors that detect the driving operation status of the driver D1. These are merely examples, and other sensors may be used. Alternatively, only some of these sensors may be used.

[0045] The location information acquisition unit acquires location information regarding the location of the vehicle C1. For example, the location information acquisition unit can be realized by a GNSS (Global Navigation Satellite System) receiver that acquires location information using the GNSS. The location information includes data regarding the location, such as latitude, longitude, and altitude, at the time of receiving the GNSS signal. The location information may also be acquired by other location information acquisition methods. For example, the location information may be derived using information from nearby access points or base stations. The location information may also be acquired using a beacon.

[0046] The seating sensor (or seat sensor) is a sensor that detects whether or not an occupant is seated in each seat of the vehicle C1. The seat belt sensor is a sensor that detects whether or not an occupant is wearing a seat belt. For example, whether or not a driver D1 sitting in the driver's seat is wearing a seat belt can be detected using the seating sensor, the seat belt sensor, or the like.

[0047] The output content determination unit 126 determines the content of each piece of output information to be output from the output device 200 based on the information output from the utterance acquisition unit 121, the driver status acquisition unit 122, the vehicle interior status acquisition unit 123, the vehicle exterior status acquisition unit 124, the vehicle information acquisition unit 125, the reception unit 203, etc. Then, the output content determination unit 126 outputs the determined content of each piece of output information and the information used in determining the content of each piece of output information to the movement mode determination unit 127 and the output control unit 128. For example, the output content determination unit 126 acquires at least one of user status information regarding the status of each occupant (including the driver D1) riding in the vehicle C1 and vehicle status information regarding the status of the vehicle C1. Then, the output content determination unit 126 determines the content of each piece of information to be output from the output device 200 based on at least one of the acquired user status information and vehicle status information. For example, when the output content determination unit 126 detects, based on the user status information or the vehicle status information, assistance information to be communicated to the driver D1, the output content determination unit 126 determines to output the assistance information in order to communicate the assistance information to the driver D1. Furthermore, when the output content determination unit 126 detects, based on the user status information or the vehicle status information, that a predetermined communication (e.g., a conversation) is to be performed with any of the occupants of the vehicle C1, the output content determination unit 126 determines the output information for performing the communication. Furthermore, the output content determination unit 126 determines, for example, the display mode, display position, etc. of the agent image AG1 based on the user status information or the operation information received by the reception unit 203.

[0048] The movement mode determination unit 127 determines the movement mode of the agent image AG1 when moving the agent image AG1 based on the content of each piece of output information determined by the output content determination unit 126, the state of the driver D1 (e.g., driving load, line of sight direction), the state of the vehicle C1 (e.g., whether the surrounding environment is high driving load), etc. The movement mode determination unit 127 then outputs the determined movement mode of the agent image AG1 to the output control unit 128. For example, the movement mode determination unit 127 checks the relationship between the movement trajectory of the agent image AG1 moving from the origin to the destination and one or more pieces of content displayed on the display unit 201. The movement mode determination unit 127 then determines whether or not there is content currently being displayed between the origin and the destination, and if there is content currently being displayed between the origin and the destination, determines whether or not the content overlaps with the movement trajectory of the agent image AG1. For example, if there is content currently being displayed between the source and destination of the movement and that content overlaps with the movement trajectory of the agent image AG1, the movement mode determination unit 127 determines to execute a movement effect that moves the agent image AG1 so as not to impair the visibility of that content. This movement effect will be described in detail with reference to Figures 6 to 9. Furthermore, only specific content among the currently displayed content may be subject to the determination of whether or not there is an overlap.

[0049] [Regarding specific content] In this embodiment, if there is a possibility that the visibility of content (particularly specific content) may be impaired by the movement of the agent image AG1, as described above, the movement path of the agent image AG1, the display mode of the agent image AG1, the movement timing or movement speed of the agent image AG1, etc. will be changed. Therefore, the specific content will be described here.

[0050] For example, among the contents displayed on the display unit 201, content that includes particularly important information can be set as the specific content. This important information is, for example, information that is required by law to be continuously displayed (i.e., information that is required by law to be continuously displayed, or information that is not permitted by law to be hidden). This information is, for example, various indicator lights and various warning lights. For example, this is information such as a battery level warning light, a maximum speed sign indicator, a battery level gauge, a cruising range indicator, a speedometer, a position indicator (shift lever position), an odometer (total distance meter), a trip meter (segment distance meter), and a warning light indicating a vehicle malfunction. Note that this is an example of typical information, and other information (information that is required by law to be continuously displayed) may also be set as the important information.

[0051] Furthermore, for example, important information is information of high urgency. This information of high urgency is, for example, information related to driving assistance that is presented to the driver D1 while driving. For example, information for notifying the driver D1 of the traveling direction of the vehicle C1, information for notifying the driver D1 of an object (for example, a railroad crossing or an accident vehicle) that exists in the traveling direction of the vehicle C1, etc. are assumed.

[0052] Furthermore, for example, the important information may be information related to driving operations, driving control information related to driving control, abnormality occurrence information related to the occurrence of an abnormality in the vehicle C1, and the like.

[0053] The information related to the driving operation is, for example, map information (map content) displayed to guide the vehicle C1 in the traveling direction. This map information includes, in particular, a current location indicator indicating the current location of the vehicle C1 and a route indicator indicating the route the vehicle C1 will travel (e.g., the route from the current location to the destination). For example, the route the vehicle C1 will travel in the future is more important to the driver D1 than the route the vehicle C1 has traveled so far. Therefore, it is preferable to treat the route the vehicle C1 will travel in the future as important information. Furthermore, for example, when the vehicle C1 is being driven autonomously, it is expected that the route of the autonomous driving will be displayed in the map content. Furthermore, when the vehicle C1 is being driven autonomously, it is expected that the driver D1 will frequently check the driving route or perform operations other than driving. In such cases, it is expected that the agent image AG1 will move more frequently in response to manual operations by the driver D1. Furthermore, guidance information regarding the traveling direction of the vehicle C1 can also be included as specific content as information related to the driving operation.

[0054] Furthermore, for example, in a non-autonomous driving environment, when highly urgent information is displayed, it is expected that the passenger in the passenger seat will often perform the operation. In such cases, it is expected that there will be more opportunities for the agent image AG1 to move in response to manual operation by the passenger in the passenger seat. In this way, when the passenger in the passenger seat performs manual operation while the driver D1 is driving, it is conceivable that the agent image AG1 displayed on the driver D1 side will move in response to the operation of the passenger in the passenger seat. In this case, it is expected that the movement of the agent image AG1 will cause the driver D1 to lose sight of important information (specific content).

[0055] Furthermore, the driving control information related to driving control is, for example, information relating to the driving of the vehicle C1, the turning of the vehicle C1, the stopping of the vehicle C1, and the like.

[0056] The abnormality occurrence information content regarding the occurrence of an abnormality in the vehicle C1 is, for example, content including information regarding a malfunction of each part of the vehicle C1, such as content including information corresponding to various warning lights displayed on meters generally installed in front of the driver's seat.

[0057] The above describes an example in which specific content is set in advance, but specific content may also be set based on the state of an occupant looking at the display unit 201. For example, when there is an occupant looking at the display unit 201, content that exists in the line of sight (i.e., line of sight) of the occupant may be set as specific content. In this case, content may be set as specific content on the condition that the same content exists in the line of sight (i.e., line of sight) of the occupant for a predetermined period of time or more.

[0058] It should be noted that entertainment information (entertainment content) that displays network information such as SNS (social networking service), music information, entertainment video information, etc. can be set not to be specified as content.

[0059] [Configuration Example of Information Processing System] The output control unit 128 controls the operating state of the output device 200 based on the content of the output information determined by the output content determination unit 126 and the movement mode of the agent image AG1 determined by the movement mode determination unit 127. Each of these operations will be described in detail with reference to Figs. 6 to 11 etc.

[0060] The storage unit 130 is a storage medium that stores various types of information. For example, the storage unit 130 stores various types of information (e.g., a control program, an agent information DB 131, an agent management DB 132, a content management DB 133, and a map information DB) that are necessary for the control unit 120 to perform various processes. The storage unit 130 also stores various types of information acquired via the communication unit 140. The storage unit 130 can be, for example, a read-only memory (ROM), a random access memory (RAM), a static random access memory (SRAM), a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0061] The agent information DB 131 stores various types of information required to realize various operations of the agent image AG1 displayed on the output device 200. For example, image information related to the agent image AG1 to be displayed on the display unit 201 of the output device 200 and audio information related to the audio of the agent image AG1 to be output from the audio output unit 202 are stored in the agent information DB 131. In addition, for example, operation information and the like for operating the agent image AG1 on the output device 200 when various types of communication are performed is stored in the agent information DB 131.

[0062] Agent management DB 132 stores agent management information (coordinate information) for managing the display position of agent image AG1 displayed on display unit 201. For example, information regarding the position of agent image AG1 on the display surface of display unit 201 and the display area of ​​agent image AG1 is managed as agent management information.

[0063] The content management DB 133 stores content management information (coordinate information) for managing the display position of each piece of content (e.g., map content, music content) displayed on the display unit 201. For example, information regarding the position of each piece of content on the display surface of the display unit 201, the display area of ​​the content, and the type of content is managed as content management information.

[0064] The output device 200 is a device that can display the agent image AG1 based on an instruction from the information processing device 110 and convey various types of information to the driver D1 and the like using the agent image AG1.

[0065] The output device 200 includes a display unit 201, a sound output unit 202, and a reception unit 203. The display unit 201, the sound output unit 202, and the reception unit 203 are controlled based on a control unit (not shown) included in the output device 200.

[0066] The display unit 201 is a display unit that displays various images based on instructions from the information processing device 110 .

[0067] The sound output unit 202 outputs various sounds based on instructions from the information processing device 110. As the sound output unit 202, for example, one or more speakers can be used.

[0068] The reception unit 203 receives user input from an occupant of the vehicle C1 and outputs the received input to the control unit 120. The reception unit 203 may be, for example, a touch panel or various operation members. The display unit 201 and the reception unit 203 may be configured as a touch panel that allows a user to perform operation input by touching or approaching the display surface with their finger, or may be configured as a separate user interface. The display unit 201, the sound output unit 202, and the reception unit 203 are examples of user interfaces, and some of them may be omitted, or other user interfaces may be used.

[0069] [Example of Driving Load Detection] The degree of driving load can be determined based on whether the vehicle C1 is stopped or moving. For example, when the vehicle C1 is stopped, it can be determined that the driving load is low (e.g., the driving load is less than a threshold value). On the other hand, when the vehicle C1 is moving, it can be determined that the driving load is high (e.g., the driving load is equal to or greater than a threshold value). Whether the vehicle C1 is stopped or moving can be determined based on vehicle information (e.g., vehicle speed, acceleration, position of the shift lever (e.g., P range, D range), accelerator pedal depression amount, brake pedal depression amount) acquired by the vehicle information acquisition unit 125.

[0070] Furthermore, the driving load can be determined based on a traffic light ahead of the vehicle C1. For example, when the traffic light ahead of the vehicle C1 is red and the vehicle C1 is stopped, it can be determined that the driving load is low (for example, the driving load is below a threshold value). On the other hand, when the vehicle C1 is stopped but the traffic light ahead of the vehicle C1 turns green, it can be determined that the driving load is high (for example, the driving load is above a threshold value). Note that other than these determinations, it is possible to make the same determination as that based on whether the vehicle C1 is stopped or moving. Note that when the vehicle C1 is in autonomous driving mode, it can be determined that the driving load is low (for example, the driving load is below a threshold value) even when the vehicle C1 is moving.

[0071] The driving load of the driver D1 may be determined based on the driving operation of the driver D1, the surrounding conditions of the vehicle C1, and the like.

[0072] For example, the operation status of the driver D1, the status around the vehicle C1, etc. can be acquired by the driver status acquisition unit 122, the vehicle exterior status acquisition unit 124, the vehicle information acquisition unit 125, etc. For example, if the number of accelerator operations and the number of steering operations within a predetermined time are large, it can be estimated that the driving load of the driver D1 is high. In addition, the driving load of the driver D1 can be estimated based on the status around the vehicle C1, etc. For example, if the vehicle C1 is traveling on a winding road, a narrow road, a highly congested road, a road with many people, etc., it can be estimated that the driving load of the driver D1 is high. On the other hand, for example, if the vehicle C1 is traveling on a long, straight road, it can be estimated that the driving load of the driver D1 is low. In addition, the driving load of the driver D1 can be estimated based on the user's facial expression, the voice uttered by the driver D1, etc.

[0073] For example, steering entropy can be used as the driving operation of the driver D1. The steering entropy method is a measurement method that measures and estimates the load on the driver based on the smoothness of the driver's steering angle, and known calculation methods can be used. In addition, this steering entropy method can use a measurement result (measurement value) that is digitized as an information entropy value calculated based on time-series steering angle data.

[0074] Furthermore, for example, the surrounding conditions of the vehicle C1 may include the number of traffic participants around the vehicle C1, the weather around the vehicle C1, the darkness around the vehicle C1, the shape of the road around the vehicle C1, etc.

[0075] The driving load of the driver D1 can be determined using the above-mentioned values. For example, a predetermined calculation (e.g., addition) can be performed on the above-mentioned values, and the calculation result can be used to determine the driving load of the driver D1. Note that the driving load of the driver D1 may be determined using at least one of the above-mentioned values. Other known methods for determining the driving load can also be used.

[0076] [Example of Movement of Agent Image] FIGS. 6 and 7 are diagrams showing an example of the transition of the agent image AG1 when the agent image AG1 is moved from the first position PP1 to the second position SP1 on the display unit 201 of the output device 200. FIG.

[0077] 6A shows an example of a transition when an agent image AG1 is moved from a first position PP1 to a second position SP1 when content CT2, which is not a specific content, is displayed on the display unit 201. The content CT2 is, for example, music content that is displayed to allow various operations to be performed when listening to music. Also, in FIG. 6A, the movement trajectory of the agent image AG1 moving from the first position PP1 to the second position SP1 is schematically shown by agent images AG1a to AG1e.

[0078] As shown in FIG. 6A , when the agent image AG1 moves along a linear path from the first position PP1 to the second position SP1, it is possible that the moving agent image AG1 and the displayed content CT2 may overlap. For example, the moving agent images AG1b to AG1d may overlap the content CT2. In this case, the moving agent images AG1b to AG1d are displayed so as to overlap the front side of the content CT2 (i.e., the display surface side of the display unit 201), preventing the occupant from seeing all or part of the content CT2. As described above, in the case of content other than the specific content, even if the occupant temporarily loses visibility, it is expected that this will not interfere with driving or the like and will not cause the occupant any unpleasant impression. Furthermore, by continuing to display the moving agent image AG1, it becomes easier to recognize the identity of the agent image AG1, which is thought to increase the occupant's trust in and attachment to the agent image AG1. Therefore, when the content CT2 being displayed is not a specific content, the output control unit 128 executes display control to move the agent image AG1 along a linear movement path from the first position PP1 to the second position SP1.

[0079] For example, since it is possible to detect the line of sight of each occupant of vehicle C1, it is possible to determine whether any of the occupants are looking at content CT2. For example, if any of the occupants of vehicle C1 are looking at content CT2, continuing to display the moving agent image AG1 is thought to make it easier to recognize the identity of the agent image AG1. However, if none of the occupants of vehicle C1 are looking at content CT2, it is assumed that no occupant will look at the agent image AG1 that overlaps with content CT2 while moving, even if the moving agent image AG1 continues to be displayed. Therefore, if none of the occupants of vehicle C1 are looking at content CT2, the display of the moving agent image AG1 may be stopped. In other words, the agent image AG1 may be erased from the first position PP1 and then displayed at the second position SP1. Note that if it is detected that any of the occupants are looking at content CT2, the agent image AG1 may be moved without overlapping with content CT2. An example of this is shown in FIG. 11 .

[0080] 6(B), (C) and 7(A) to (C) show examples of transitions when an agent image AG1 is moved from a first position PP1 to a second position SP1 when a specific content, CT1, is displayed on the display unit 201. Fig. 6(B) shows an example of moving the agent image AG1 behind the content CT1 (i.e., the opposite side of the display surface of the display unit 201).

[0081] As shown in FIG. 6B , when the agent image AG1 moves along a linear path from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT1. In this case, if the displayed content CT1 is a specific content, it is important not to impair the visibility of the content CT1. Therefore, when the displayed content CT1 is a specific content, the output control unit 128 executes display control such that the agent image AG1 moves behind the content CT1 (i.e., on the opposite side of the display surface of the display unit 201) along the linear path from the first position PP1 to the second position SP1. This makes it possible to maintain the identity of the moving agent image AG1 while not impairing the visibility of the content CT1.

[0082] FIG. 6C shows an example in which a transparent or semi-transparent agent image AG1 is moved on the front side of the content CT1 (i.e., the display surface side of the display unit 201).

[0083] As shown in FIG. 6C , when the currently displayed content CT1 is a specific content, the output control unit 128 executes display control to display a transparent or semi-transparent agent image AG1 moving on the front side of the content CT1 (i.e., the display surface side of the display unit 201) along a linear movement path from the first position PP1 to the second position SP1. This makes it possible to maintain the identity of the moving agent image AG1 while not impeding the visibility of the content CT1. The transparency of the agent image AG1 can be set according to the importance of the overlapping content. For example, if the content is specific and cannot be hidden by law, the importance is determined to be high, and the transparency of the agent image AG1 is set high. That is, the agent image AG1 is set to be completely transparent. On the other hand, if the content is specific and it is considered acceptable for some of it to be hidden, the importance is determined to be low, and the transparency of the agent image AG1 is set low.

[0084] 7A and 7B show an example in which the agent image AG1 is moved so as to bypass the content CT1.

[0085] 7(A) and (B), when the content CT1 being displayed is a specific content, the output control unit 128 executes display control to display the agent image AG1 in a manner such that the agent image AG1 moves along a detour route that bypasses the content CT1 as a movement route from the first position PP1 to the second position SP1. In Figures 7(A) and (B), the moving agent image AG1 is shown by agent images AG1a to AG1f.

[0086] 7(A) and 7(B), when the agent image AG1 is moved along a detour route, it is possible that the range of the detour route is narrower than the display area of ​​the agent image AG1. In this case, it is possible to reduce the size of the agent image AG1 while it is moving, or to display a part of the agent image AG1 while it is moving behind the content CT1 (i.e., on the opposite side of the display surface of the display unit 201), or to display it outside the display surface of the display unit 201.

[0087] In the example shown in Figure 7 (A), the size of the moving agent image AG1 is reduced, and a part of the moving agent image AG1 is displayed outside the display surface of the display unit 201.

[0088] The example shown in Figure 7 (B) shows an example in which the size of the moving agent image AG1 is reduced, and a part of the moving agent image AG1 is displayed on the back side of the content CT1 (i.e., the opposite side of the display surface of the display unit 201) and on the outside of the display surface of the display unit 201.

[0089] FIG. 7C shows an example in which the agent image AG1 is moved so as to bypass the content CT3, which is a specific content.

[0090] As shown in Fig. 7(C), when the content CT3 being displayed is a specific content, the output control unit 128 executes display control to display the agent image AG1 in a manner such that the agent image AG1 moves along a detour route that bypasses the content CT3 as a movement route from the first position PP1 to the second position SP1. In Fig. 7(C), the moving agent image AG1 is shown by agent images AG1a to AG1d.

[0091] 7C, when the agent image AG1 is moved along a detour route, it is possible that the range of the detour route is larger than the display area of ​​the agent image AG1. In this case, it is possible to maintain the size of the agent image AG1 while it is moving.

[0092] Here, it is assumed that there are portions on the detour route where the agent image AG1 and the specific content do not overlap but are close to each other. For example, in the example shown in FIG. 7A , the lower portions of the moving agent images AG1b to AG1e are close to the upper portion of the specific content CT1. In such a case, it is assumed that the proximity of the agent image AG1 to the specific content reduces the visibility of the content. Therefore, if there is a close portion on the detour route where the agent image AG1 and the specific content are close to each other, the agent image AG1 in that close portion may be displayed transparently or semi-transparently. Here, the distance for determining proximity can be set appropriately based on experiments, simulations, or the like. For example, a distance of several millimeters to several centimeters can be used as the criterion for determining proximity.

[0093] As shown in FIGS. 6 and 7, when the agent image AG1 is moved, the movement trajectory of the agent image AG1 may be expressed by animation processing or the like.

[0094] This makes it possible to maintain the identity of the moving agent image AG1 while not impeding the visibility of the content CT1. The detour route and display mode of the agent image AG1 for a specific content can be set appropriately based on experiments, simulations, etc.

[0095] Here, there are cases where it is possible to set multiple detour routes that bypass a specific content. In such cases, it is possible to set the detour route based on a preset criterion. For example, it is possible to set the detour route based on the line of sight of the occupants. For example, the occupants who are viewing the specific content are identified based on the line of sight of each occupant. Then, based on the line of sight direction of the occupant who is viewing the specific content, the position (focus position) on the display surface of the display unit 201 that the occupant is expected to be looking at most is identified. Then, from among the multiple detour routes that bypass the specific content, the detour route closest to the focus position is determined. This allows the occupants to easily recognize the movement of the agent image AG1 while it is moving.

[0096] [Example of movement when multiple specific contents exist on the movement path] The above describes an example of movement of the agent image AG1 when one specific content currently being displayed overlaps with the moving agent image AG1. Here, when multiple specific contents are displayed on the display unit 201, it is also possible that two or more of these specific contents overlap with the moving agent image AG1. In this case, too, it is possible to move the agent image AG1 in the same way as in the examples shown in Figures 6(B) and (C) and Figures 7(A) to (C). This example is shown in Figure 8.

[0097] Figure 8 is a diagram showing an example of the transition of the agent image AG1 when the agent image AG1 is moved from the first position PP1 to the second position SP1 when multiple specific contents CT3 and CT4 are displayed on the display unit 201 of the output device 200.

[0098] As shown in Fig. 8, when the agent image AG1 moves along a linear path from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT3 and CT4. In this case, too, it is possible to move the agent image AG1 in the same way as in the examples shown in Figs. 6(B) and (C) and 7(A) to (C). Fig. 8 shows, as a modification of the example shown in Fig. 7(A), an example in which the size of the moving agent image AG1 is maintained and a portion of the moving agent image AG1 is displayed outside the display surface of the display unit 201.

[0099] [Example of movement of an agent image along a detour route when the agent image moves across multiple display units] The above describes an example of movement of the agent image AG1 when a specific content being displayed on the display unit 201 overlaps with the moving agent image AG1. It is also possible that when one or more specific contents are displayed on multiple display units (including the display unit 201), the specific content and the moving agent image AG1 overlap. In this case, too, it is possible to move the agent image AG1 in the same way as in the examples shown in Figures 6(B) and (C), 7(A) to (C), and 8. This example is shown in Figure 9.

[0100] Figure 9 is a diagram showing an example of the transition of the agent image AG1 when a specific content CT5 is displayed on the display unit 201 of the output device 200 and a specific content CT6 is displayed in the HUD display area of ​​the windshield 4, and the agent image AG1 is moved from a first position PP1 to a second position SP1.

[0101] Here, when various images are to be displayed in the HUD display area of ​​the windshield 4, a HUD display device for realizing the HUD is provided on the upper part of the dashboard 2 near the boundary with the windshield 4.

[0102] The HUD display device is a display device, such as a projector or optical system, that projects light onto a display area 4a of the windshield 4 and uses the reflected light to realize a HUD display that shows a virtual image to the driver D1. That is, the light projected from the HUD display device onto the display area 4a of the windshield 4 is reflected by the windshield 4, and the reflected light heads toward the eyes of the driver D1. The reflected light projected onto the display area 4a and entering the eyes of the driver D1 is then displayed together with actual objects visible through the windshield 4, superimposed on those objects. In this way, the HUD display device realizes a HUD display by displaying a virtual image using the windshield 4.

[0103] For example, the HUD display device displays the agent image AG1 in the display area 4a under the control of the information processing device 110 (see FIG. 5). The windshield 4 also functions as a display medium for the HUD of the vehicle C1.

[0104] As shown in FIG. 9 , when the agent image AG1 moves along a linear path (a straight line in three-dimensional space or a straight line in two-dimensional space based on the occupant's visual field) from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT5 and CT6. In this case, the agent image AG1 can be moved in the same manner as in the examples shown in FIGS. 6B, 6C, 7A to 7C, and 8. FIG. 9 shows an example in which the size of the moving agent image AG1 is maintained, as in the example shown in FIG. 8, and a display mode is used in which a portion of the moving agent image AG1 is displayed outside the display surface of the display unit 201. Here, when moving the agent image AG1 between multiple physically separated display units, it is preferable to set a detour route or the like within a range that the occupant viewing the agent image AG1 can create. For example, if a detour route would make the movement difficult to see, the agent image AG1 can be displayed transparently or semi-transparently.

[0105] 9 shows an example in which specific content is displayed on the display unit 201 and the HUD display area on the windshield 4, but is not limited to this. For example, if a display unit is provided on the steering wheel 3, it is also possible for the specific content to be displayed on that display unit. In this case, it is possible to execute a movement process for the agent image AG1 based on the overlap state between the specific content displayed on at least one of the display unit 201, the HUD display area on the windshield 4, and the display unit of the steering wheel 3 and the moving agent image AG1.

[0106] In this way, this embodiment can be understood as a display system comprising at least one or more display units that display the agent image AG1, and a control unit that controls the position at which the agent image AG1 is displayed on any of the display units.

[0107] The above example illustrates a case in which the movement processing of the agent image AG1 is changed when the agent image AG1 moving along a linear path overlaps with a specific content. Here, for example, if the agent image AG1 moving along a linear path does not overlap with the specific content but is close to it, it is conceivable that a passenger viewing the specific content may experience reduced visibility of the specific content due to the influence of the nearby agent image AG1. Therefore, the movement processing of the agent image AG1 may also be changed in a similar manner when the agent image AG1 moving along a linear path is close to the specific content. For example, the system may determine whether the linear path passes through an area (proximity area) within a predetermined distance of the specific content, and if it is determined that the linear path passes through the proximity area, the system may move the agent image AG1 along a path (detour route) that passes through a position farther away than the predetermined distance. The display mode of the proximity area may also be changed to move the agent image AG1. The movement processing in these cases is similar to the movement processing shown in Figures 6 to 9 .

[0108] [Example of Operation of Information Processing System] Figures 10 and 11 are flowcharts showing an example of agent migration processing in the information processing system 100. This agent migration processing is executed by the control unit 120 (see Figure 5) based on a program stored in the storage unit 130 (see Figure 5). This agent migration processing is also executed constantly at each control cycle. This agent migration processing will be explained with appropriate reference to Figures 1 to 9.

[0109] In step S501, the output content determination unit 126 determines whether a user input has been received. For example, if the reception unit 203 receives a manual operation by any of the occupants, if the speech acquisition unit 121 receives a voice operation by any of the occupants, or if the driver status acquisition unit 122 or the cabin status acquisition unit 123 receives a gesture operation by any of the occupants, it is determined that a user input has been received. If a user input has been received, the process proceeds to step S502. On the other hand, if a user input has not been received, monitoring continues.

[0110] In step S502, the output content determination unit 126 determines whether the user input received in step S501 is a voice input. That is, if the speech acquisition unit 121 receives a voice operation from any of the occupants, the user input is determined to be a voice input. If the user input is a voice input, the process proceeds to step S503. On the other hand, if the user input is not a voice input, the process proceeds to step S510 (see FIG. 11 ).

[0111] In step S503, the output content determination unit 126 acquires the current display position (first position) of the agent image AG1 and determines the destination display position (second position) of the agent image AG1. The first position can be acquired based on the display position of the agent image AG1 stored in the agent management DB 132. For example, the position of the occupant who made the user input (e.g., hand position, eye position) can be used as a reference, and the position on the surface of the display unit 201 closest to the occupant can be determined as the second position. The occupant who made the user input can be determined based on the relationship between the timing at which the utterance was acquired by the utterance acquisition unit 121 and the occupant's mouth movement acquired by the driver status acquisition unit 122 or the vehicle interior status acquisition unit 123. For example, if there is an occupant moving their mouth at the timing at which the utterance was made, it can be assumed that the occupant is making a sound as a user input. The presence or absence of an occupant in each seat of the vehicle C1 can be determined based on the detection values ​​of a seating sensor, a seat belt sensor, etc. Furthermore, the presence or absence of an occupant in each seat of the vehicle C1 may be determined based on images acquired by the driver state acquisition unit 122 or the vehicle interior state acquisition unit 123. Furthermore, if multiple microphones or the like are installed in the vehicle C1, the location of a sound generation can be identified based on sounds acquired by these microphones. Therefore, the location of the occupant who made the user input may be identified using a technology for identifying the location of a sound generation.

[0112] For example, if the user input is a voice input, the second position can be determined based on the eye position of the occupant who made the user input, rather than the hand position, because the user input is not a manual operation. In this case, the position on the display surface of the display unit 201 that is closest to the eye position of the occupant who made the user input can be determined as the second position. Also, for example, if the user input is something other than a voice input (e.g., a manual operation), the third position (shown in step S511) can be determined based on the hand position of the occupant who made the user input. In this case, the position on the display surface of the display unit 201 that is closest to the hand position of the occupant who made the user input can be determined as the third position (see step S511).

[0113] In step S504, the movement mode determination unit 127 determines whether the user input received in step S501 was made by the driver D1. For example, when the driver D1 is driving, manual operation is often not possible, so it is assumed that the user input is often made by voice. The method for identifying the occupant who made the user input is the same as the method shown in step S503. If the user input was made by the driver D1, the process proceeds to step S505. On the other hand, if the user input was made by an occupant other than the driver, the process proceeds to step S506.

[0114] In step S505, the movement mode determination unit 127 determines whether the driving load of the driver D1 who made the user input is less than a threshold value. The method for detecting the driving load is the same as the detection method described above. The threshold value shown here is determination information for determining whether the driving load of the driver D1 is high or low. This threshold value can be set appropriately based on experiments, simulations, etc. Here, an example is shown in which it is determined whether the driving load of the driver D1 is less than a threshold value. However, if autonomous driving is not being performed, it may be determined whether the vehicle C1 is moving or stopped. If the driving load of the driver D1 is less than the threshold value, the process proceeds to step S506. On the other hand, if the driving load of the driver D1 is equal to or greater than the threshold value, the process proceeds to step S510.

[0115] In step S506, the movement mode determination unit 127 checks the relationship between the first movement trajectory of the agent image AG1 moving from the first position to the second position determined in step S503 and one or more pieces of content displayed on the display unit 201. For example, the movement mode determination unit 127 determines whether or not there is content currently being displayed between the first position and the second position, and if there is content currently being displayed between the first position and the second position, determines whether or not that content overlaps with the first movement trajectory of the agent image AG1. Note that the method for determining whether or not the first movement trajectory of the agent image AG1 moving from the first position to the second position overlaps with the content currently being displayed is the same as the determination method shown in FIG. 4(B).

[0116] In step S507, the movement mode determination unit 127 determines whether or not a specific content exists among the displayed contents that overlap with the first movement trajectory of the agent image AG1 moving from the first position to the second position. If a specific content exists that overlaps with the first movement trajectory, the process proceeds to step S508. On the other hand, if a specific content does not exist that overlaps with the first movement trajectory, the process proceeds to step S509.

[0117] In step S508, the movement mode determination unit 127 determines a movement path and display mode for moving the agent image AG1 from the first position to the second position so as not to obstruct the visibility of the specific content determined in step S507 to overlap with the first movement trajectory of the agent image AG1. Then, the output control unit 128 moves the agent image AG1 from the first position to the second position in accordance with the determined movement path and display mode. For example, as shown in Figures 6(B) and 6(C) and 7 to 9, it is possible to move the agent image AG1 so as not to obstruct the visibility of the specific content CT1.

[0118] In step S509, the movement mode determination unit 127 determines to move the agent image AG1 along the first movement trajectory. Then, the output control unit 128 moves the agent image AG1 from the first position to the second position along the first movement trajectory. That is, the agent image AG1 is moved over the shortest distance. For example, as shown in FIG. 6A, it is possible to move the agent image AG1 so that it overlaps with the front side of the currently displayed content CT2 (i.e., the display surface side of the display unit 201).

[0119] In step S510, the movement mode determination unit 127 determines to move only the position of the agent image AG1 without executing a movement effect that moves the agent image AG1. That is, the movement mode determination unit 127 determines to erase the agent image AG1 displayed at the first position, and then to display the agent image AG1 at the second position. Then, the output control unit 128 erases the agent image AG1 displayed at the first position, and then displays the agent image AG1 at the second position.

[0120] As described above, when the driver D1 is driving, manual operation is often not possible, and it is therefore assumed that user input is often performed by voice. Furthermore, when the driver D1's driving load is high (above a threshold), it is considered that the driver D1 does not have time to see the movement of the agent image AG1, and therefore temporarily erasing the agent image AG1 is not likely to affect the presence of the agent image AG1. Furthermore, when the driver D1's driving load is high (above a threshold), it is considered that it is easier for the driver D1 to recognize the agent image AG1 if the agent image AG1 is moved instantaneously from a first position to a second position rather than by executing a movement effect. Therefore, in step S510, only the display position of the agent image AG1 is changed without executing a movement effect of the agent image AG1.

[0121] In step S511, the output content determination unit 126 acquires the current display position (first position) of the agent image AG1 and determines a destination display position (third position) of the agent image AG1. Here, assuming that the user input is manual input rather than voice input, an example is shown in which the position of the hand of the occupant who made the user input is used as a reference, and the position on the surface of the display unit 201 closest to the position of the occupant's hand is determined as the third position. The occupant who made the user input (manual input) can be determined based on the relationship between the timing at which the operation is accepted by the accepting unit 203 and the hand movement of the occupant acquired by the driver status acquiring unit 122 or the vehicle interior status acquiring unit 123. For example, if there is an occupant moving their hand near the display unit 201 at the timing at which the accepting unit 203 accepts the operation, it can be assumed that the occupant is performing a manual operation as a user input. Furthermore, when a touch operation is performed on the surface of the display unit 201, the hand position can be determined as the position at which the touch operation was performed.

[0122] For example, it is assumed that a selection operation for selecting an operation button related to content displayed on the display unit 201 is performed as a user input. In this case, in order to provide assistance with the operation, it is possible to move the agent image AG1 near the operation button that has been selected.

[0123] The processes in steps S512 to S514 and S516 correspond to the processes in steps S506 to S509, and therefore will not be described here.

[0124] In step S515, the movement mode determination unit 127 determines whether the occupant is viewing the displayed content (other than the specific content) that is determined to overlap with the first movement trajectory of the agent image AG1 in step S513. As described above, when the driver D1 is driving, manual operation is often not possible. Therefore, it is assumed that the occupant subject to each process of steps S511 to S516 is a occupant other than the driver D1 (or a driver D1 whose driving load is less than a threshold). It is also assumed that a occupant other than the driver D1 (or a driver D1 whose driving load is less than a threshold) is viewing and enjoying the content displayed on the display unit 201. Therefore, if there is a occupant viewing the displayed content, it is important not to impede the visibility of the content, even if the content is not the specific content. Therefore, if the occupant is viewing the displayed content, a movement effect is executed that does not impede the visibility of the content. Note that this movement effect can be the same as the movement effect for the specific content. That is, in step S514, the content the occupant is viewing is also subject to movement effect, along with the specific content.

[0125] Although the above describes an example in which the movement path or display mode of the agent image AG1 is changed so as not to impair the visibility of the specific content, the present embodiment is not limited thereto. For example, the movement timing or movement speed of the agent image AG1 may be changed so as not to impair the visibility of the specific content. For example, by moving the agent image AG1 at a speed faster than the normal movement speed, it is possible to avoid impairing the visibility of the specific content. For example, even if there is an overlap between the agent image AG1 and the specific content, it is possible to shorten the time during which the specific content is hidden by moving the agent image AG1 quickly along a movement path that includes the overlapping portion. Furthermore, for example, it is possible to determine whether any occupant of the vehicle C1 is viewing the specific content, and to move the agent image AG1 (at a normal speed or a speed faster than the normal speed) when the occupant is not viewing the specific content. For example, when moving the agent image AG1 along a travel path that includes an overlapping portion where the agent image AG1 and specific content overlap, it is possible to prevent the occupant from feeling that the specific content is not visible by moving the agent image AG1 at a time when the occupant is not looking at the specific content.

[0126] 10 and 11 show an example in which the control content is changed based on whether the user input is a voice input, but this embodiment is not limited to this and may be realized using other control examples. For example, the agent movement process (steps S503 to S510 or steps S511 to S516) may be executed even when any user input is received. Also, FIG. 10 shows a control process in which the movement mode of the agent image AG1 is changed based on the process of determining whether the driver's driving load is high or low (step S505). However, this embodiment is not limited to this and may be realized using other control examples. For example, the process of determining whether the driver D1's driving load is high or low (steps S505 and S510) may be omitted. Also, FIG. 11 shows a control process in which the movement mode of the agent image AG1 is changed based on the process of determining whether the occupant is viewing content (step S515). However, this embodiment is not limited to this and may be realized using other control examples. For example, the process of determining whether the occupant is viewing content (step S515) may be omitted.

[0127] Also, an example has been shown in which a control process is executed to change the movement manner of the agent image AG1 based on, for example, whether or not the agent image AG1 overlaps with specific content. However, the control process may be executed by other methods. For example, the control process may be executed using artificial intelligence (AI). For example, various situations related to the driver D1, the vehicle C1, etc., the movement manner of the agent to be executed in response to those situations, and the overlapping state between the agent image AG1 and specific content may be learned in advance, and this learning data may be used in the control process. For example, the learning data may be used for various situations occurring related to the driver D1, the vehicle C1, etc., and the overlapping state between the agent image AG1 and specific content, to determine the movement manner of the agent, and the agent image AG1 may be moved in the determined movement manner.

[0128] [Example of Effect of This Embodiment] As described above, in this embodiment, when the agent image AG1 is moved to a position corresponding to a user input in response to the user input, it is possible to move the agent image AG1 so as not to obstruct the visibility of specific content (important content). In other words, if specific content is displayed along the shortest movement route of the agent image AG1, the agent image AG1 is moved to avoid the specific content so as not to obstruct the visibility of the specific content. For example, it is possible to move the agent image AG1 along a detour route, change the shape of the agent image AG1, change the transparency of the agent image AG1, change the movement speed of the agent image AG1, or change the movement timing of the agent image AG1.

[0129] This allows the occupant who has made a user input to visually confirm (or feel a sense of presence) the agent image AG1 that moves in response to the user input, and the visibility of the specific content is not obstructed. This allows the agent image AG1 to provide appropriate operational assistance to the user. It is also possible to create an impression of intelligence for the agent image AG1 that moves without obscuring the specific content. This makes it possible to increase the trust and attachment to the agent image AG1, and to improve the ability of the agent image AG1 to convey information to each occupant. In this way, in this embodiment, the agent image AG1 can be appropriately moved on the display unit 201 so as not to obstruct the visibility of the specific content.

[0130] [Example of Executing Processing in Other Devices or Systems] Note that, although the above describes an example in which the determination processing, detection processing, control processing, etc. are executed in the information processing device 110 (or the information processing system 100), all or part of each of these processes may be executed in other devices. In this case, the information processing system is configured by the devices that execute part of each of these processes. For example, at least part of each process may be executed using various information processing devices and various electronic devices, such as in-vehicle devices, devices that can be used by the user (e.g., smartphones, tablet terminals, personal computers, car navigation devices, IVIs), and servers that can be connected via a predetermined network such as the Internet.

[0131] Furthermore, a part (or all) of the information processing system capable of executing the functions of the information processing device 110 (or the information processing system 100) may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[0132] [Configuration example and effects of this embodiment] The information processing method of this embodiment is an information processing method that displays an agent image AG1, which communicates with an occupant riding in vehicle C1, on display unit 201 (including each display unit such as the HUD display area of ​​the front window 4), and controls the display state of the agent image AG1 based on the state of the occupant. This information processing method includes, when an agent image AG1 displayed at a first position (initial position) on the display unit 201 moves to a second position (a position to which the agent image AG1 moves in response to a user input) based on the state of the occupant, a determination process for determining whether or not the visibility of a specific content is obstructed when the agent image AG1 moves along the first path based on a linear movement path (first path) from the first position to the second position and the content displayed on the display unit 201 (steps S506, S507, S512, S513), and if the visibility of the specific content is not obstructed, moving the agent image AG1 along the first path in a first display mode (steps S509, S516). If the visibility of the specific content is obstructed, moving the agent image AG1 along the first path in a first display mode (steps S509, S516). The control process includes control processes (steps S508, S509, S514, S516) for executing one of the following: a first movement process for moving the agent image AG1 along a movement path (second path) that does not obstruct the visibility of the specific content; a second movement process for moving the agent image AG1 from a first position to a second position in a display mode (second display mode) that is different from the first display mode and does not obstruct the visibility of the specific content; and a third movement process for moving the agent image AG1 from the first position to the second position at a second movement timing that is different from the first movement timing in the first display mode and does not obstruct the visibility of the specific content, or at a second movement speed that is different from the first movement speed in the first display mode and does not obstruct the visibility of the specific content (steps S508, S514). For example, the first movement process is performed by the movement processes shown in FIGS. 7 to 9. Furthermore, for example, the second movement process is performed by the movement processes shown in FIGS. 6B and 6C. The program according to this embodiment is a program that causes a computer to execute each of these processes.In other words, the program according to this embodiment is a program that causes a computer to realize each function that can be executed by the information processing device 110 .

[0133] With this configuration, the agent image AG1, which moves based on the state of the occupant, can be visually confirmed (or its presence, if it is transparent), and the visibility of specific content is not obstructed. In other words, the agent image AG1 can be appropriately moved on the display unit 201 so as not to obstruct the visibility of specific content.

[0134] In the information processing method according to the present embodiment, the first movement process moves the agent image AG1 along a detour route that bypasses a specific content as the second route. For example, the agent image AG1 can be moved by the movement processes shown in FIGS. 7 to 9.

[0135] According to this configuration, the agent image AG1 that bypasses the specific content can be visually confirmed near the specific content.

[0136] In the information processing method according to this embodiment, in the first movement process, if there is a portion on the detour route where the agent image AG1 overlaps with a specific content, or if there is a portion on the detour route where the agent image AG1 is close to a specific content, a display mode is used in which the agent image AG1 in the overlapping portion or the close portion is made transparent or semi-transparent, or a display mode is used in which the agent image AG1 in the overlapping portion is displayed as if it were positioned behind the specific content (in the depth direction of the display unit 201). For example, as shown in Fig. 7(B), if there is a portion on the detour route where the agent image AG1 overlaps with a specific content CT1, a display mode can be used in which the agent image AG1 in the overlapping portion is displayed as if it were positioned behind the specific content CT1.

[0137] According to this configuration, when the agent image AG1 and the specific content overlap or come close to each other on the detour route, it is possible to prevent the specific content from becoming difficult to see due to the overlap or closeness.

[0138] In the information processing method according to this embodiment, in the first movement process, if there is a portion of the agent image AG1 on the detour route that extends beyond the display surface of the display unit 201, that portion of the agent image AG1 is not displayed. For example, as shown in Fig. 7(A), if there is a portion of the agent image AG1 on the detour route that extends beyond the display surface of the display unit 201 (the upper portions of agent images AG1b to AG1e), it is possible to prevent the display of that portion of the agent image AG1.

[0139] According to this configuration, even if the route along which the agent image AG1 moves on a detour is narrow, it is possible to display the agent image AG1 while maintaining the size, etc., of the agent image AG1.

[0140] In the information processing method according to this embodiment, the determination process (steps S506, S507, S512, and S513) may determine that the moving agent image AG1 obstructs the visibility of a specific content when the agent image AG1 moving along the first path is adjacent to or overlaps with the specific content. In this case, the second movement process may move the agent image AG1 along the first path, and display the overlapping portion of the agent image AG1 as if it were positioned behind the specific content, or may display the adjacent or overlapping portion of the agent image AG1 as if it were transparent or semi-transparent, as the second display mode. For example, as shown in FIG. 6B , the agent image AG1 may move along the first path, and display the overlapping portion of the agent image AG1 as if it were positioned behind the specific content. Furthermore, as shown in FIG. 6C , the overlapping portion of the agent image AG1 may be displayed as if it were transparent or semi-transparent. The agent image AG1 in the vicinity of a specific content may be displayed transparently or semi-transparently.

[0141] According to this configuration, it is possible to move the agent image AG1 along the first route, and the portion close to or overlapping with the specific content can be displayed as if it is positioned behind the specific content, or the agent image AG1 can be displayed as transparent or semi-transparent. This makes it possible to move the agent image AG1 along the same movement route as in the case of content other than the specific content, and therefore it is possible to perform a movement presentation that makes it easy for the occupant to predict the movement.

[0142] In the information processing method according to this embodiment, the agent image AG1 may be moved at a second moving speed faster than the first moving speed in the third moving process. However, if the specific content is information that is required by law to be continuously displayed, this movement control is not executed.

[0143] With this configuration, even if there is an overlapping portion between the agent image AG1 and the specific content, it is possible to shorten the time that the specific content is not visible by quickly moving the agent image AG1 along a movement path that includes the overlapping portion. This makes it possible to appropriately move the agent image AG1 on the display unit 201 so as not to impede the visibility of the specific content.

[0144] The information processing method according to this embodiment may further include a detection process for detecting the gaze of an occupant based on an image including the occupant's eyes. For example, the movement mode determination unit 127 may detect the gaze of an occupant (including the driver D1) based on images acquired by the driver state acquisition unit 122 and the vehicle interior state acquisition unit 123. A known gaze detection technique may be employed for this gaze detection. The third movement process may determine whether the occupant is looking at specific content based on the occupant's gaze, and the timing when the occupant is not looking at the specific content may be set as the second movement timing.

[0145] According to this configuration, when moving the agent image AG1 along a travel route that includes an overlapping portion where the agent image AG1 and specific content overlap, it is possible to move the agent image AG1 at a timing when the occupant is not looking at the specific content. This makes it possible to appropriately move the agent image AG1 on the display unit 201 so as not to obstruct the visibility of the specific content.

[0146] In the information processing method according to this embodiment, the specific content can be at least one of map content displayed to guide the direction of travel of vehicle C1, route information content showing the route from the current location of vehicle C1 to the destination in the map content, driving control information content related to driving control of vehicle C1, abnormality occurrence information content related to the occurrence of an abnormality in vehicle C1, and information that is required by law to be continuously displayed.

[0147] According to this configuration, important information can be set as specific content.

[0148] The information processing method according to this embodiment may further include a driving load determination process (step S505) for determining the driving load of the driver D1 based on at least one of environmental information related to the environment outside the vehicle C1 and driving behavior information related to the driving behavior of the driver D1. In this case, the determination process (steps S506 and S507) determines whether the visibility of specific content is impaired if the driving load is less than a threshold. Then, in the control process, if the driving load is equal to or greater than the threshold, the agent image AG1 displayed at the first position is erased and then the agent image AG1 is displayed at the second position (step S510). If the driving load is less than the threshold, a movement process of the agent image AG1 is executed (steps S508 and S508) based on the determination result of the determination process (steps S506 and S507).

[0149] For example, when the driving load of driver D1 is high, it is considered that driver D1 does not have time to see the movement of the agent image AG1, and therefore temporarily erasing the agent image AG1 is not considered to affect the presence of the agent image AG1. Also, when the driving load of driver D1 is high, it is considered that it is easier for driver D1 to recognize the agent image AG1 if it is moved instantaneously from a first position to a second position rather than by executing a movement effect. In this way, it is possible to appropriately perform the movement effect of the agent image AG1 based on the driving load of driver D1.

[0150] The information processing device 110 is an information processing device that displays an agent image AG1 for communicating with a passenger in a vehicle C1 on a display unit 201 (including each display unit such as a HUD display area on the windshield 4), and controls the display state of the agent image AG1 based on the state of the passenger. The information processing device 110 includes a movement mode determination unit 127 (an example of a determination unit) that, when an agent image AG1 displayed at a first position (initial position) on the display unit 210 moves to a second position (a position to which the agent image AG1 moves in response to a user input) based on the state of the passenger, determines whether or not movement of the agent image AG1 along the first path will obstruct the visibility of specific content, based on a linear movement path (first path) from the first position to the second position and the content displayed on the display unit 201; and, when the visibility of the specific content is not obstructed, moves the agent image AG1 along the first path in a first display mode, and, when the visibility of the specific content is obstructed, moves the agent image AG1 along a movement path different from the first path that obstructs the visibility of the specific content. The information processing device 110 includes an output control unit 128 (an example of a control unit) that executes one of: a first movement process that moves the agent image AG1 along a movement path (second path) that does not impair visibility of the specific content; a second movement process that moves the agent image AG1 from a first position to a second position in a display mode (second display mode) that is different from the first display mode and does not impair visibility of the specific content; and a third movement process that moves the agent image from the first position to the second position at a second movement timing that is different from the first movement timing in the first display mode and does not impair visibility of the specific content, or at a second movement speed that is different from the first movement speed in the first display mode and does not impair visibility of the specific content. Note that the information processing device 110 may be a device built into the output device 200 or may be a device separate from the output device 200. Furthermore, instead of the information processing device 110, an information processing system configured with a plurality of devices that can execute each process realized by the information processing device 110 may be implemented.

[0151] With this configuration, the agent image AG1, which moves based on the state of the occupant, can be visually confirmed (or its presence, if it is transparent), and the visibility of specific content is not obstructed. In other words, the agent image AG1 can be appropriately moved on the display unit 201 so as not to obstruct the visibility of specific content.

[0152] Note that each processing procedure shown in this embodiment is an example for realizing this embodiment, and the order of some of the processing procedures may be changed within the scope that makes it possible to realize this embodiment, and some of the processing procedures may be omitted or other processing procedures may be added.

[0153] Each process in this embodiment is executed based on a program that causes a computer to execute various processing procedures. This embodiment can also be understood as an embodiment of a program that realizes the function of executing each process and a recording medium that stores the program. For example, an update process for adding a new function to an information processing device can store the program in the storage device of the information processing device. This makes it possible to cause the updated information processing device to execute each process described in this embodiment.

[0154] Although the embodiments of the present invention have been described above, the above embodiments merely show application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

Claims

1. An information processing method for displaying an agent image that communicates with a vehicle occupant on a display unit, and controlling a display state of the agent image based on a state of the occupant, comprising: a determination process for determining, when the agent image displayed at a first position on the display unit moves to a second position based on the state of the occupant, whether or not the visibility of specific content is obstructed when the agent image moves along a first route, which is a linear movement route from the first position to the second position, and based on content displayed on the display unit; When the visibility of the specific content is not impaired, the agent image is moved along the first route in a first display mode; If the visibility of the specific content is impaired, a first movement process of moving the agent image along a second route, which is a movement route different from the first route and does not obstruct the visibility of the specific content; a second movement process of moving the agent image from the first position to the second position in a second display mode that is a display mode different from the first display mode and does not impair visibility of the specific content; a control process for executing either a second movement timing different from a first movement timing in the first display mode and not interfering with the visibility of the specific content, or a third movement process for moving the agent image from the first position to the second position at a second movement speed different from a first movement speed in the first display mode and not interfering with the visibility of the specific content, Information processing methods.

2. The information processing method according to claim 1, In the first movement process, the agent image is moved along a detour route that bypasses the specific content as the second route. Information processing methods.

3. The information processing method according to claim 2, In the first movement process, if there is a portion on the detour route where the agent image overlaps with the specific content, or if there is a portion on the detour route where the agent image is close to the specific content, the display mode is set to make the agent image in the overlapping portion or the close portion transparent or semi-transparent, or to display the agent image in the overlapping portion as if it were positioned behind the specific content. Information processing methods.

3. The information processing method according to claim 2, In the first movement process, if there is a portion of the agent image that protrudes from the display surface of the display unit on the detour route, the protruding portion of the agent image is not displayed. Information processing methods.

2. The information processing method according to claim 1, In the determination process, when the agent image moving along the first route is close to or overlaps with the specific content, it is determined that the moving agent image obstructs visibility of the specific content; In the second movement process, moving the agent image along the first path; The second display mode is a display mode in which the agent image in the overlapping portion with the specific content is displayed as if it were positioned behind the specific content, or a display mode in which the agent image in the proximity or overlapping portion with the specific content is displayed transparent or semi-transparent. Information processing methods.

2. The information processing method according to claim 1, In the third movement process, the agent image is moved at a second movement speed that is faster than the first movement speed. Information processing methods.

2. The information processing method according to claim 1, a detection process for detecting a line of sight of the occupant based on an image including the eyes of the occupant, In the third movement process, it is determined whether the occupant is watching the specific content based on a line of sight of the occupant, and a time when the occupant is not watching the specific content is set as the second movement timing. Information processing methods.

8. An information processing method according to claim 1, The specific content is at least one of map content displayed to guide the vehicle in the traveling direction, route information content indicating a route from the current location of the vehicle to the destination in the map content, driving control information content related to driving control of the vehicle, abnormality occurrence information content related to the occurrence of an abnormality in the vehicle, and information content that is required by law to be continuously displayed. Information processing methods.

8. An information processing method according to claim 1, a driving load determination process for determining a driving load of the driver based on at least one of environmental information relating to an environment outside the vehicle and driving behavior information relating to a driving behavior of the driver, In the determination process, when the driving load is less than a threshold, it is determined whether visibility of the specific content is impaired; In the control process, When the driving load is equal to or greater than a threshold, the agent image displayed at the first position is erased, and then the agent image is displayed at the second position; If the driving load is less than a threshold, a process of moving the agent image is executed based on the determination result of the determination process. Information processing methods.

1. An information processing device that displays an agent image that communicates with a vehicle occupant on a display unit and controls a display state of the agent image based on a state of the occupant, a determination unit that, when the agent image displayed at a first position on the display unit moves to a second position based on the state of the occupant, determines whether or not the visibility of specific content is impaired when the agent image moves along a first route, which is a linear movement route from the first position to the second position, based on the content displayed on the display unit; When the visibility of the specific content is not impaired, the agent image is moved along the first route in a first display mode; If the visibility of the specific content is impaired, a first movement process of moving the agent image along a second route, which is a movement route different from the first route and does not obstruct the visibility of the specific content; a second movement process of moving the agent image from the first position to the second position in a second display mode that is a display mode different from the first display mode and does not impair visibility of the specific content; a control unit that executes either a second movement timing that is different from the first movement timing in the first display mode and does not impair visibility of the specific content, or a third movement process that moves the agent image from the first position to the second position at a second movement speed that is different from the first movement speed in the first display mode and does not impair visibility of the specific content, Information processing device. 　 A program executed by a computer that causes a display unit to display an agent image that communicates with a vehicle occupant, and controls a display state of the agent image based on a state of the occupant, a determination process for determining, when the agent image displayed at a first position on the display unit moves to a second position based on the state of the occupant, whether or not the visibility of specific content is obstructed when the agent image moves along a first route, which is a linear movement route from the first position to the second position, and based on content displayed on the display unit; When the visibility of the specific content is not impaired, the agent image is moved along the first route in a first display mode; If the visibility of the specific content is impaired, a first movement process of moving the agent image along a second route, which is a movement route different from the first route and does not obstruct the visibility of the specific content; a second movement process of moving the agent image from the first position to the second position in a second display mode that is a display mode different from the first display mode and does not impair visibility of the specific content; a control process for executing either a second movement timing different from a first movement timing in the first display mode and not interfering with the visibility of the specific content, or a third movement process for moving the agent image from the first position to the second position at a second movement speed different from a first movement speed in the first display mode and not interfering with the visibility of the specific content; A program that causes a program to execute the following.

Images