Information processing method, information processing apparatus, and program

Abstract

The control device includes a determination process in which, when an avatar image originally displayed at a first position in a display section moves to a second position based on a state of an occupant, it is determined whether visibility of a specific content is obstructed when the avatar image moves along a straight-line-shaped movement path (first path) from the first position to the second position and the content displayed in the display section, and a control process in which, when the visibility of the specific content is obstructed, a first movement process for moving the avatar image along a movement path (second path) different from the first path and not obstructing the visibility of the specific content is executed.

Description

Technical Field

[0001] This invention relates to an information processing method, information processing device, and program for controlling an intelligent agent capable of communicating with a user. Background Technology

[0002] Previously, a technology for conveying various information to users in a vehicle has been proposed. For example, JP2020-055348A proposes a technology in which, when a provision image for an occupant is displayed on a display unit, the display position of an agent image is moved to the display position of the provision image in order to guide the occupant's gaze to the display position of the provision image. Summary of the Invention

[0003] In the aforementioned prior art, when the display position of the agent image is moved to the position of the image provided to the occupants, the content displayed on the display unit may be obscured by the moving agent image. For example, when content related to the vehicle's driving path or vehicle driving control is displayed on the display unit, there is also a possibility that this content may be obscured by the moving agent image. Therefore, it is important to ensure that the visibility of specific content is not obstructed by the moving agent image.

[0004] The object of the present invention is to move the image of the intelligent agent in a display unit in a manner that does not obstruct the visibility of specific content.

[0005] One aspect of the present invention is an information processing method that displays an agent image for communication with occupants of a vehicle on a display unit, and controls the display state of the agent image based on the occupant's state. The information processing method includes: a determination process, in which, when an agent image originally displayed at a first position moves to a second position on the display unit based on the occupant's state, determines whether the movement of the agent image along the first path obstructs the visibility of specific content based on a linear movement path (first path) from the first position to the second position and the content displayed on the display unit; and a control process, in which, if the visibility of the specific content is not obstructed, the agent image is moved along the first path in a first display manner; and if the visibility of the specific content is obstructed, any one of a first movement process, a second movement process, and a third movement process is executed, wherein the first movement process is used to move the agent image along the first path... The agent image moves from the first position to the second position using a different movement path (second path) that does not obstruct the visibility of specific content. The second movement process is used to move the agent image from the first position to the second position using a second display mode that is different from the first display mode and does not obstruct the visibility of specific content. The third movement process is used to move the agent image from the first position to the second position using a second movement timing that is different from the first movement timing in the first display mode, or a second movement speed that is different from the first movement speed in the first display mode. The second movement timing is a movement timing that does not obstruct the visibility of specific content, and the second movement speed is a movement speed that does not obstruct the visibility of specific content. Attached Figure Description

[0006] Figure 1 This is a simplified diagram illustrating an example of the structure inside a vehicle's passenger compartment.

[0007] Figure 2 This is a simplified diagram illustrating an example of the structure inside a vehicle's passenger compartment.

[0008] Figure 3 This is a simplified diagram illustrating an example of the structure inside a vehicle's passenger compartment.

[0009] Figure 4 This is a diagram illustrating a migration example where the image of the agent is moved.

[0010] Figure 5 This is a block diagram illustrating an example of the system architecture of an information processing system.

[0011] Figure 6 This is a diagram illustrating an example of the transfer of an agent's image.

[0012] Figure 7 This is a diagram illustrating an example of the transfer of an agent's image.

[0013] Figure 8 This is a diagram illustrating a migration example of an agent graph.

[0014] Figure 9 This is a diagram illustrating an example of the transfer of an agent's image.

[0015] Figure 10 This is a flowchart illustrating an example of agent movement processing.

[0016] Figure 11 This is a flowchart illustrating an example of agent movement processing. Detailed Implementation

[0017] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0018] [Example of display unit setup]

[0019] Figure 1 This is a simplified diagram illustrating an example of the internal structure of the carriage of vehicle C1. Furthermore, in Figure 1 The image shows a simplified view of the interior of vehicle C1 from the rear, in a position forward of the driver's seat and front passenger seat (not shown). Additionally, in... Figure 1 For ease of explanation, illustrations other than those for the instrument panel 2, steering wheel 3, front window 4, rearview mirror 5, and output device 200 are omitted.

[0020] Output device 200 is an output device installed inside vehicle C1, based on information processing device 110 (see reference). Figure 5 The output device 200 is a device installed on the instrument panel 2, and includes a display unit 201, a sound output unit 202, and a receiving unit 203 (see reference 201). Figure 5 ).exist Figure 1 The diagram shows an example of an output device 200 that is longer in the left-right direction of the vehicle C1. The output device 200 is, for example, a device that extends from near the center of the dashboard 2 to the driver's seat as a display area.

[0021] For example, output device 200 is an in-vehicle system consisting of one or more devices capable of providing various information. As output device 200, at least one of the following can be used: navigation device, audio device, DVD device, TV tuner device, IVI (In-Vehicle Infotainment). Furthermore, images displayed on each output device 200 can also be displayed using a HUD (Head-Up Display) implemented in the windshield 4 or other display devices. Alternatively, portable devices that can be held by the driver D1 (or devices that can be installed in the vehicle C1), such as smartphones, tablets, portable personal computers, and other information processing devices, can also be used.

[0022] Additionally, the output device 200 performs output actions that enable communication with users (including driver D1) riding in the vehicle C1. For example, the display unit 201 of the output device 200 displays an image of an intelligent agent AG1 capable of communicating with the occupants (see reference). Figure 2 , Figure 3 The system can execute various performances using the agent image AG1. Additionally, an agent image capable of communicating with the occupants can be displayed in the front window 4 (HUD display area) or other display devices, and various performances using that agent image can be executed.

[0023] [Example of displaying an image of an intelligent agent]

[0024] Figure 2 , Figure 3 This is a simplified diagram illustrating an example of the internal structure of the carriage of vehicle C1. Furthermore, Figure 2 , Figure 3 and Figure 1 The structural examples shown are the same, so detailed explanations are omitted here.

[0025] exist Figure 2 , Figure 3 The diagram shows an example where content CT1 and agent image AG1 are displayed on the display unit 201 of the output device 200. Content CT1 displays map information including a current position identifier PL1 indicating the current position of vehicle C1 and path information RI1 indicating the driving path of vehicle C1. For example, the path information RI1 is displayed in a display method different from that of a map (e.g., with arrows in blue, red, or other colors). Furthermore, although the illustration is omitted, it is also possible to appropriately display map information with a changed viewpoint. For example, content CT1 can be displayed based on navigation functions.

[0026] The agent image AG1 represents an image of an object capable of communicating with a user. In this embodiment, an image mimicking a human face is shown as an example of the agent image AG1. However, the agent image AG1 is not limited to this; for example, an image mimicking a full body or part of a human (e.g., the upper body) can be used as the agent image AG1, as can an image mimicking an animal (or the face of such an animal) such as a rabbit or pig, or an image mimicking the face of a virtual creature (e.g., an animated character) (or the face thereof) can be used as the agent image AG1. In this way, biomimetic images can be used as the agent image AG1.

[0027] Output device 200 is based on information processing device 110 (see reference) Figure 5 The output device 200 performs various actions related to driving assistance (e.g., actions related to content CT1 and intelligent agent image AG1) based on the instructions of the information processing device 110. For example, the output device 200 outputs various information related to driving assistance, surrounding facilities, etc., when the driver D1 is performing driving operations, based on the control of the information processing device 110.

[0028] Furthermore, as a driving assistance system utilizing the intelligent agent image AG1, it is conceivable to provide notifications of moving objects in front or behind. For example, as a notification of a moving object in front, it could output a voice message such as "There's a crossing ahead, be careful!" or "There's someone ahead!" When making these outputs, the intelligent agent image AG1 can be used to perform various actions. That is, the intelligent agent image AG1 can be used to perform driving assistance. Additionally, for example, the output device 200 can perform various processes using the intelligent agent image AG1 based on the control of the information processing device 110. For example, the output device 200 uses the intelligent agent image AG1 to perform communication processes for various interactions with the driver D1. For example, the output device 200 uses the intelligent agent image AG1 to conduct various conversational exchanges with the driver D1, or to provide information that the driver D1 prefers.

[0029] For example, Figure 2 As shown, consider the following scenario: Agent AG1 issues voice information S1, "Turn right at the next intersection," and in response, driver D1 issues voice information S2, "Is that the highway?" Furthermore, consider the following scenario: Agent AG1 issues voice information S3, "Get on the highway next," in response to driver D1's voice information S2 (see...). Figure 3 Furthermore, these exchanges are simplified and not limited to this.

[0030] In this way, consider a scenario where an occupant of vehicle C1 makes certain user input, and the intelligent agent image AG1 moves to the vicinity of the occupant who made the user input, making it easier to assist that occupant. For example, imagine that when the intelligent agent image AG1 is displayed on... Figure 2 In the situation shown, driver D1 issued the voice message S2, "Is that a highway over there?" In this case, consider the following... Figure 3 As shown, the agent image AG1 is moved to the vicinity of the driver D1 (e.g., in the direction of the driver D1's gaze). That is, consider moving the agent image AG1 to the location where the user is performing an operation, the location the user wants to view. (Refer to...) Figure 4 The method for moving the agent image AG1 in this case is described.

[0031] [Example of agent image movement]

[0032] Figure 4 This is a diagram illustrating a migration example in which the agent image AG1 is moved in the display unit 201 of the output device 200.

[0033] exist Figure 4 In (A), the agent image at the starting point of the movement when the agent image AG1 is moved in the display unit 201 is represented as agent image AG1a, and the agent image at the destination of the movement is represented as agent image AG1e.

[0034] exist Figure 4 In (B), dashed lines MT1 and MT2 show the range of the movement trajectory of the agent image AG1 when it moves linearly from the agent image AG1a to the agent image AG1e in the display unit 201. Additionally, in Figure 4 In (B), a portion of the movement trajectory of the agent image AG1 is schematically shown using dashed circles AG1a to AG1e. Like this, in... Figure 4 (B) shows the relationship between the first movement trajectory AG1a to AG1e of the agent image AG1 from the first position AG1a to the second position AG1e and the content CT1 displayed.

[0035] Here, an example is shown of a linear movement path set from the initial position (first position AG1a) of the agent image AG1 to the response position (second position AG1e) to which the agent image AG1 is to be moved in response to user input (e.g., manual input, voice input).

[0036] Furthermore, each position on the display surface of the display unit 201 can be managed by an agent, for example, DB 132 (see reference). Figure 5 It is managed in the form of coordinate information. For example, the output content determination unit 126 (refer to...) Figure 5By sequentially storing the positions of each image (content image, agent image AG1) originally displayed on the display surface of display unit 201 in agent management DB 132, the current position of agent image AG1 can be obtained. Movement mode determination unit 127 can calculate the range of the movement trajectory of agent image AG1 moving on the display surface of display unit 201 based on the display area (display area) of agent image AG1, the current position of agent image AG1, and the position of the destination of agent image AG1. Known calculation methods can be used for calculating this range.

[0037] For example, the agent image AG1 could be moved based on the occupant's line of sight, hand position, etc. For example, if the occupant performs a desired operation on the display unit 201, the agent image AG1 could be moved to a position easily accessible to the occupant based on their actions. For example, if the occupant is operating with their hand, the agent image AG1 could be moved to a position close to their hand. For example, it could be moved linearly from the starting point to the destination, as indicated by arrow AW1.

[0038] In this way, by responding to user actions and making the agent image AG1 follow and move continuously, it can be shown that the agent image AG1 has reacted to the user's actions. Furthermore, by observing the agent image AG1, the user can visually grasp the information that the agent image AG1 wants to convey. Therefore, it is conceivable that the user would become uneasy if the agent image AG1 temporarily disappeared. However, the agent image AG1 may overlap with information that it does not want to obscure during its movement in response to user input.

[0039] For example, when the agent image AG1 is displayed in the foreground and moved along the path indicated by arrow AW1, the moving agent image AG1 overlaps with the surface of the content CT1 (i.e., the display surface side of the display unit 201). In this case, the occupant cannot see a portion of the content CT1 during the movement of the agent image AG1. That is, the movement of the agent image AG1 obstructs the visibility of the content CT1.

[0040] Therefore, in this embodiment, when the movement of the agent image AG1 obstructs the visibility of specific content, the movement path of the agent image AG1 is changed, or the display mode of the agent image AG1 is changed, or the movement timing or speed of the agent image AG1 is changed, so that the visibility of the specific content is not obstructed.

[0041] [Example of an information processing system structure]

[0042] Figure 5This is a block diagram illustrating an example of the system structure of the information processing system 100 installed in vehicle C1.

[0043] The information processing system 100 includes a voice acquisition unit 101, a driver image acquisition unit 102, a passenger compartment image acquisition unit 103, an exterior image acquisition unit 104, an information processing device 110, and an output device 200. The information processing device 110 is an example of a device that controls the output device 200, which is capable of communicating with the occupants (including the driver D1) of the vehicle C1.

[0044] Furthermore, the information processing device 110 and the output device 200 are connected via wired or wireless communication. Additionally, the information processing device 110 is connected to the network 20 via wireless communication. The network 20 is a public landline network, the Internet, or similar network. Furthermore, the output device 200 can also be connected to the network 20 via wireless communication. Figure 5 The example shown is an example in which the information processing device 110 and the output device 200 are configured as separate units, but the information processing device 110 and the output device 200 can also be configured as a single unit.

[0045] The sound acquisition unit 101 is installed inside the vehicle C1 and is used to acquire the sound inside the vehicle C1. The sound acquisition unit 101 outputs sound information related to the acquired sound to the information processing device 110. As the sound acquisition unit 101, one or more microphones or sound acquisition sensors can be used, for example.

[0046] The driver image acquisition unit 102 is used to capture images of the driver D1 riding in the vehicle C1 and generate images (image data). The driver image acquisition unit 102 outputs image information related to the generated images to the information processing device 110. The driver image acquisition unit 102 is installed at least inside the vehicle C1, and captures images of the driver D1 riding in the vehicle C1 and generates images (image data). The driver image acquisition unit 102 is, for example, composed of one or more camera devices or image sensors capable of capturing images of the driver D1. For example, a driver image acquisition unit 102 can be installed at the front of the interior of the vehicle C1 (e.g., the roof) to capture images of the driver D1 from the front of the vehicle C1 and generate images (image data). For example, the driver image acquisition unit 102 can be installed above the front window 4, that is, above the rearview mirror 5. Furthermore, the driver image acquisition unit 102 and the passenger compartment image acquisition unit 103 can be the same device or configured as different devices.

[0047] The vehicle compartment image acquisition unit 103 is used to capture images of subjects inside the vehicle C1 and generate images (image data). The vehicle compartment image acquisition unit 103 outputs image information related to the generated images to the information processing device 110. The vehicle compartment image acquisition unit 103 is installed in at least the interior (e.g., the roof) of the vehicle C1, capturing images of subjects inside the vehicle C1 and generating images (image data). The vehicle compartment image acquisition unit 103 may consist of, for example, one or more camera devices or image sensors capable of capturing subjects. For example, one vehicle compartment image acquisition unit 103 may be installed at the front of the vehicle C1 to capture images of subjects from the front and generate images (image data), and other vehicle compartment image acquisition units 103 may be installed at the rear of the vehicle C1 to capture images of subjects from the rear and generate images (image data).

[0048] The exterior image acquisition unit 104 is used to capture images of subjects outside the vehicle C1 and generate images (image data). The exterior image acquisition unit 104 outputs image information related to the generated images to the information processing device 110. Alternatively, two or more exterior image acquisition units 104 may be provided, and all or some of the images from these units may be used. For example, one exterior image acquisition unit 104 may be installed at the front of the vehicle C1 to capture images of subjects in front of the vehicle C1 and generate images (image data), and other exterior image acquisition units 104 may be installed at the rear of the vehicle C1 to capture images of subjects behind the vehicle C1 and generate images (image data). Furthermore, one or more devices capable of capturing subjects present in all directions of the vehicle C1 and subjects inside the vehicle C1, such as a 360-degree camera, may be used.

[0049] Furthermore, the driver image acquisition unit 102, the passenger compartment image acquisition unit 103, and the exterior image acquisition unit 104 are, for example, composed of an image sensor (image sensor) and an image processing unit. Light from the subject, focused by a lens, is incident on the image sensor, and the image processing unit performs prescribed image processing on the image data generated by the image sensor. For example, CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) type image sensors can be used as the image sensor.

[0050] The information processing device 110 includes a control unit 120, a storage unit 130, and a communication unit 140. Based on the control of the control unit 120, the communication unit 140 uses wired or wireless communication to exchange various types of information with other devices. For example, when the communication unit 140 receives driving assistance information, action information of the intelligent agent image AG1, etc., from an external device (e.g., a server), it outputs these pieces of information to the control unit 120.

[0051] The control unit 120 is used to control each unit based on various programs stored in the storage unit 130. The control unit 120 is implemented by a processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). Alternatively, the vehicle ECU (Electronic Control Unit) of the vehicle C1 can be used as the control unit 120, or a processing device different from the vehicle ECU can be used as the control unit 120.

[0052] The control unit 120 performs various controls based on information output from the voice acquisition unit 101, driver image acquisition unit 102, passenger compartment image acquisition unit 103, exterior image acquisition unit 104, communication unit 140, etc., and information acquired by the vehicle information acquisition unit 125. For example, the control unit 120 performs control processing for controlling the operating state of the output device 200. Specifically, the control unit 120 includes a speech acquisition unit 121, a driver status acquisition unit 122, a passenger compartment status acquisition unit 123, an exterior status acquisition unit 124, a vehicle information acquisition unit 125, an output content determination unit 126, a movement mode determination unit 127, and an output control unit 128.

[0053] The speech acquisition unit 121 is used to acquire speech information related to the speech of each user (including driver D1) contained in the sound information output from the sound acquisition unit 101 by performing a prescribed sound analysis process on the sound information. The speech acquisition unit 121 outputs the speech information to the output content determination unit 126. Known sound analysis processes can be used for this sound analysis process.

[0054] The driver status acquisition unit 122 is used to acquire various information related to the driver D1 contained in the image information output from the driver image acquisition unit 102 by performing a prescribed image analysis process on the image information. The driver status acquisition unit 122 then outputs this information to the output content determination unit 126. Known image analysis processes can be used for this image analysis process. For example, various actions of the driver D1 related to the driver D1's facial expression, gaze, hand, face, and body movements can be acquired. Thus, for example, it is possible to detect the driver D1's boarding and alighting from the vehicle C1, the presence or absence of seats in the vehicle C1, and the driver D1's hand movements. In other words, the driver status acquisition unit 122 acquires driver status information related to the driver D1's state.

[0055] The carriage status acquisition unit 123 acquires various information related to the interior of the vehicle C1 contained in the image information output from the carriage image acquisition unit 103 by performing a prescribed image analysis process on the image information. The carriage status acquisition unit 123 then outputs this information to the output content determination unit 126. Known image analysis processes can be used for this image analysis process. For example, various actions of each user related to their facial expressions, gaze, hand, face, and body movements can be acquired. In other words, the carriage status acquisition unit 123 acquires user status information related to the state of each user riding in the vehicle C1, as well as vehicle status information related to the state of the vehicle C1.

[0056] The exterior condition acquisition unit 124 acquires various information related to the exterior of vehicle C1 contained in the image information output from the exterior image acquisition unit 104 by performing a prescribed image analysis process on the image information. The exterior condition acquisition unit 124 then outputs this information to the output content determination unit 126. Known image analysis processes can be used for this image analysis process. Examples of the various information related to the exterior of vehicle C1 include whether vehicle C1 is driving on a road, whether vehicle C1 is parked on a road, and traffic lights and signs located in front of vehicle C1. In other words, the exterior condition acquisition unit 124 acquires vehicle state information related to the state of vehicle C1.

[0057] The vehicle information acquisition unit 125 acquires information (vehicle status information) related to various vehicle states of vehicle C1, and outputs the acquired vehicle status information to the output content determination unit 126. For example, vehicle status information can be acquired from CAN (Controller Area Network) signals. Furthermore, vehicle status information includes, for example, vehicle speed, acceleration, gear shift lever position (e.g., P gear, D gear), accelerator pedal depressor level, brake pedal depressor level, position information, and abnormal occurrence information. For example, it can determine whether vehicle C1 is parked or moving based on vehicle speed, acceleration, etc. Additionally, various warning messages can be displayed based on various abnormal occurrence information related to vehicle C1.

[0058] In addition, the vehicle information acquisition unit 125 can also acquire sensor detection information output from various sensors installed in the vehicle C1. Examples of such sensors include LiDAR (Light Detection and Ranging), RADAR (Radio Detection and Ranging), Sonar, vehicle speed sensor, acceleration sensor, steering sensor, accelerometer position sensor, position information acquisition sensor (position information acquisition unit), seating sensor, seatbelt sensor, etc. Known sensors can be used for each of these. Furthermore, LiDAR, RADAR, and Sonar are examples of sensors used to detect the surrounding conditions of the vehicle C1. Additionally, vehicle speed sensor, acceleration sensor, steering sensor, and accelerometer position sensor are examples of sensors used to detect the driving operation of the driver D1. These are just examples; other sensors can also be used. Alternatively, only a subset of these sensors may be used.

[0059] The location information acquisition unit is used to acquire location information related to the location of vehicle C1. For example, this can be achieved using a GNSS receiver that utilizes GNSS (Global Navigation Satellite System) to acquire location information. Furthermore, this location information includes various location-related data such as latitude, longitude, and altitude at the time of receiving the GNSS signal. Alternatively, location information can also be acquired using other methods. For example, location information can be derived using information from nearby access points or base stations. Additionally, beacons can be used to acquire location information.

[0060] Seat sensors (or seat sensors) are sensors that detect whether occupants are seated in each seat of vehicle C1. Seatbelt sensors are sensors that detect whether occupants in each seat of vehicle C1 are wearing seatbelts. For example, seat sensors and seatbelt sensors can be used to detect whether the driver D1, seated in the driver's seat, is wearing a seatbelt.

[0061] The output content determination unit 126 determines the content of each output information output from the output device 200 based on information output from the speech acquisition unit 121, driver status acquisition unit 122, passenger compartment status acquisition unit 123, external status acquisition unit 124, vehicle information acquisition unit 125, and receiving unit 203. Furthermore, the output content determination unit 126 outputs the determined content of each output information and the information used in determining this content to the mode of transport determination unit 127 and the output control unit 128. For example, the output content determination unit 126 acquires at least one of user status information related to the status of each occupant (including driver D1) of the vehicle C1 and vehicle status information related to the status of the vehicle C1. Moreover, the output content determination unit 126 determines the content of each information output from the output device 200 based on at least one of the acquired user status information and vehicle status information. For example, when the output content determination unit 126 detects auxiliary information to be conveyed to the driver D1 based on user status information or vehicle status information, it determines to output the auxiliary information and conveys it to the driver D1. Additionally, when the output content determination unit 126 detects that a prescribed communication (e.g., a conversation) needs to be performed with any occupant of the vehicle C1 based on user status information or vehicle status information, it determines the output information used to perform that communication. Furthermore, the output content determination unit 126 determines the display method and display position of the intelligent agent image AG1, for example, based on user status information or operation information received by the receiving unit 203.

[0062] The movement mode determination unit 127 determines the movement mode for moving the agent image AG1 based on the content of each output information determined by the output content determination unit 126, the state of the driver D1 (e.g., driving load, line of sight), and the state of the vehicle C1 (e.g., whether the surrounding environment is one of high driving load). Furthermore, the movement mode determination unit 127 outputs the determined movement mode of the agent image AG1 to the output control unit 128. For example, the movement mode determination unit 127 confirms the relationship between the movement trajectory of the agent image AG1 from the starting point to the destination and one or more contents displayed on the display unit 201. Moreover, the movement mode determination unit 127 determines whether there is currently displayed content between the starting point and the destination; if there is currently displayed content between the starting point and the destination, it determines whether that content overlaps with the movement trajectory of the agent image AG1. For example, if there is displayed content between the starting point and the destination, and this content overlaps with the movement trajectory of the agent image AG1, the movement mode determination unit 127 decides to execute a movement performance that moves the agent image AG1 in a manner that does not obstruct the visibility of the content. (See reference...) Figures 6 to 9 To explain the moving performance in detail. Alternatively, you can use only specific content within the displayed content as the criterion for determining whether there is overlap.

[0063] [Regarding specific content]

[0064] In this embodiment, if the movement of the agent image AG1 may obstruct the visibility of content (especially specific content), as described above, changes may be made to the movement path of the agent image AG1, the display mode of the agent image AG1, the timing of the movement of the agent image AG1, or the speed of the movement. Therefore, specific content will be described here.

[0065] For example, content containing particularly important information displayed on the display unit 201 can be designated as specific content. This important information may be, for example, information that is legally required to be continuously displayed (i.e., information that is legally required to be continuously displayed, or information that cannot be legally obscured). This information may include various indicator lights and warning lights. Examples include battery level warning lights, maximum speed indicator displays, battery level gauges, remaining driving range displays, speedometers, position indicators (gear shift lever positions), total odometers, short-distance odometers, and warning lights indicating adverse vehicle conditions. Furthermore, this is just a representative example of information; other information (information that is legally required to be continuously displayed) may also be designated as important information.

[0066] Additionally, important information may be information of high urgency. This high-urgency information may be related to driving assistance presented to the driver D1 during driving. For example, information to inform the driver C1 of the direction of travel, or information to inform the driver of objects (e.g., road crossings, accident vehicles) in the direction of travel of vehicle C1.

[0067] In addition, important information includes, for example, information related to driving operation, driving control information related to driving control, and abnormal occurrence information related to the occurrence of abnormalities in vehicle C1.

[0068] Information associated with driving operations includes, for example, map information (map content) displayed to guide the direction of travel of vehicle C1, particularly map information including a current position marker indicating the current position of vehicle C1 within the map, and a path marker indicating the path traveled by vehicle C1 (e.g., the path from the current position to the destination). For example, for driver D1, the path that vehicle C1 will take in the future is more important than the path that vehicle C1 has already taken, so it is preferable to regard the path that vehicle C1 will take in the future as important information. In addition, it is envisioned, for example, that when autonomous driving is being performed in vehicle C1, the autonomous driving path is displayed in the map content. Furthermore, it is envisioned that when autonomous driving is being performed in vehicle C1, driver D1 will frequently confirm the driving path or perform operations other than driving. In such cases, it is also envisioned that the intelligent agent image AG1 will move more frequently in response to the manual operation of driver D1. In addition, as information associated with driving operations, guidance information related to the direction of travel of vehicle C1 can also be included as specific content.

[0069] Furthermore, for example, in situations where autonomous driving is not in progress, it is envisioned that the passenger on the front seat would frequently operate the system when displaying information of high urgency. In such cases, it is also envisioned that the agent image AG1 would move more frequently in response to the passenger's manual operation. Thus, it can be assumed that when the passenger on the front seat performs a manual operation while the driver D1 is driving, the agent image AG1, originally displayed on the driver D1's side, would move in response to the passenger's operation. In this case, it is envisioned that the driver D1 would miss important information (specific content) due to the movement of the agent image AG1.

[0070] In addition, driving control information associated with driving control includes information related to the movement of vehicle C1, turning of vehicle C1, and stopping of vehicle C1.

[0071] In addition, the abnormality information related to the occurrence of an abnormality in vehicle C1 includes information related to malfunctions in various parts of vehicle C1. For example, it typically includes information corresponding to various warning lights displayed on the instrument panel located in front of the driver's seat.

[0072] The above illustrates an example of pre-setting specific content, but specific content can also be set based on the state of the occupant viewing the display unit 201. For example, content that is present in front of the occupant's line of sight (i.e., the direction of their gaze) while the occupant is viewing the display unit 201 can be set as specific content. In this case, the content can also be set as specific content if it exists in front of the occupant's line of sight (i.e., the direction of their gaze) for a specified period of time or more.

[0073] In addition, it is possible to set it so that entertainment information (entertainment content) such as SNS (Social networking service) network information, music information, and entertainment video information are not displayed as specific content.

[0074] [Example of an information processing system structure]

[0075] The output control unit 128 controls the operation state of the output device 200 based on the content of the output information determined by the output content determination unit 126 and the movement mode of the agent image AG1 determined by the movement mode determination unit 127. Furthermore, regarding these operations, see [reference needed]. Figures 6 to 11 We will provide a detailed explanation later.

[0076] Storage unit 130 is a storage medium for storing various types of information. For example, storage unit 130 stores various types of information required by control unit 120 for various processes (e.g., control program, agent information DB 131, agent management DB 132, content management DB 133, map information DB). Additionally, storage unit 130 stores various types of information acquired via communication unit 140. Storage unit 130 can be, for example, ROM (Read Only Memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or combinations thereof.

[0077] The agent information DB 131 stores various information required to perform various actions of the agent image AG1 displayed on the output device 200. For example, image information related to the agent image AG1 displayed on the display unit 201 of the output device 200 and voice information related to the voice of the agent image AG1 output from the sound output unit 202 are stored in the agent information DB 131. In addition, action information, such as action information for activating the agent image AG1 in the output device 200 when performing various communications, is stored in the agent information DB 131.

[0078] The agent management DB 132 stores agent management information (coordinate information) for managing the display position of the agent image AG1 displayed on the display unit 201. For example, information related to the position of the agent image AG1 on the display surface of the display unit 201 and the display area of ​​the agent image AG1 is managed as agent management information.

[0079] The content management database 133 stores content management information (coordinate information) used to manage the display positions of various contents (e.g., map content, music content) displayed on the display unit 201. For example, information related to the position of each content on the display surface of the display unit 201, the display area of ​​the content, and the type of content is managed as content management information.

[0080] The output device 200 is a device that can display the agent image AG1 based on instructions from the information processing device 110 and use the agent image AG1 to convey various information to the driver D1 and others.

[0081] The output device 200 includes a display unit 201, a sound output unit 202, and a receiving unit 203. Furthermore, the display unit 201, the sound output unit 202, and the receiving unit 203 are controlled based on a control unit (not shown) provided with the output device 200.

[0082] Display unit 201 is a display unit that displays various images based on instructions from information processing device 110.

[0083] The sound output unit 202 is used to output various voices based on instructions from the information processing device 110. One or more speakers can be used as the sound output unit 202, for example.

[0084] The receiving unit 203 receives user input from the occupants of vehicle C1 and outputs the received input to the control unit 120. The receiving unit 203 can be, for example, a touch panel or various operating components. Furthermore, the display unit 201 and the receiving unit 203 can be configured as a touch panel that allows the user to input operations by touching or approaching the display surface with their fingers, or they can be configured as separate user interfaces. Additionally, the display unit 201, the sound output unit 202, and the receiving unit 203 are examples of user interfaces; some of them may be omitted, or other user interfaces may be used.

[0085] [Example of driving load detection]

[0086] The level of driving load can be determined based on whether vehicle C1 is parked or moving. For example, if vehicle C1 is parked, the driving load can be determined to be low (e.g., driving load less than a threshold). On the other hand, if vehicle C1 is moving, the driving load can be determined to be high (e.g., driving load above a threshold). Whether vehicle C1 is parked or moving can be determined based on vehicle information (e.g., vehicle speed, acceleration, gear shift position (e.g., P gear, D gear), accelerator pedal depress amount, brake pedal depress amount) obtained by vehicle information acquisition unit 125.

[0087] Furthermore, the driving load can be determined based on the traffic light ahead of vehicle C1. For example, if the traffic light ahead of vehicle C1 is red and vehicle C1 is stationary, the driving load can be determined to be low (e.g., driving load less than a threshold). On the other hand, if vehicle C1 is stationary but the traffic light ahead of vehicle C1 changes to green, the driving load can be determined to be high (e.g., driving load above a threshold). In addition to these determinations, the same determinations based on whether vehicle C1 is stationary or in motion can be made. Furthermore, even when in autonomous driving mode, if vehicle C1 is in motion, the driving load can still be determined to be low (e.g., driving load less than a threshold).

[0088] In addition, the driving load of driver D1 can also be determined based on the driving operation of driver D1 and the surrounding conditions of vehicle C1.

[0089] For example, the driver's operating status and the surrounding conditions of the vehicle C1 can be acquired through the driver status acquisition unit 122, the external condition acquisition unit 124, and the vehicle information acquisition unit 125. For example, if the number of accelerator operations and steering operations is high within a specified time, the driver's workload of D1 can be estimated. Furthermore, the driver's workload of D1 can be estimated based on the surrounding conditions of the vehicle C1. For example, if the vehicle C1 is traveling on a winding road, a narrow road, a congested road, or a road with many people, the driver's workload of D1 can be estimated to be high. On the other hand, if traveling on a long, straight road, for example, the driver's workload of D1 can be estimated to be low. Additionally, the driver's workload of D1 can be estimated based on the user's facial expressions, the sounds made by the driver D1, etc.

[0090] For example, steering entropy can be used for driver D1's driving operation. The steering entropy method is a measurement method that measures and estimates the driver's load based on the smoothness of the driver's steering angle, and can use known calculation methods. In addition, in this steering entropy method, the numerical measurement results (measured values) can be used as information entropy values ​​calculated based on time-series steering angle data.

[0091] In addition, for example, as the surrounding conditions of vehicle C1, it is possible to use the number of traffic participants around vehicle C1, the weather around vehicle C1, the darkness around vehicle C1, the road shape around vehicle C1, etc.

[0092] The driving load of driver D1 can be calculated using the values ​​described above. For example, a prescribed operation (e.g., addition) can be performed on the values ​​described above, and the driving load of driver D1 can be calculated using the result of the operation. Alternatively, at least one of the values ​​described above can be used to calculate the driving load of driver D1. Furthermore, other known methods for determining driving load can be used.

[0093] [Example of agent image movement]

[0094] Figure 6 , Figure 7 This is a diagram illustrating an example of the migration of the agent image AG1 when the agent image AG1 is moved from the first position PP1 to the second position SP1 in the display unit 201 of the output device 200.

[0095] exist Figure 6 (A) illustrates a migration example where the agent image AG1 moves from the first position PP1 to the second position SP1 when content CT2, which is not specific content, is displayed on the display unit 201. Content CT2, for example, is music content displayed for various operations while listening to music. Furthermore, in... Figure 6 In (A), the movement trajectory of agent image AG1 from the first position PP1 to the second position SP1 is schematically shown using agent images AG1a to AG1e.

[0096] like Figure 6 As shown in (A), it is also envisioned that when the agent image AG1 moves along a straight path from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT2. For example, the moving agent images AG1b to AG1d overlap with the content CT2. In this case, the moving agent images AG1b to AG1d are displayed in a manner that overlaps with the surface side of the content CT2 (i.e., the display surface side of the display unit 201), so the occupant cannot see all or part of the content CT2. Here, as described above, in the case of content other than specific content, it is also envisioned that even if the occupant is temporarily unable to visually recognize it, it will not hinder driving, etc., thus rarely causing an unpleasant impression on the occupant. In addition, it can be considered that by continuously displaying the moving agent image AG1, the identity of the agent image AG1 is easily recognized, and the trust and attachment to the agent image AG1 increases. Therefore, when the content CT2 displayed is not specific content, the output control unit 128 performs display control to move the agent image AG1 along a linear movement path from the first position PP1 to the second position SP1.

[0097] For example, since the gaze of each occupant of vehicle C1 can be detected, it can be determined whether an occupant is viewing content CT2. For example, if it can be assumed that any occupant of vehicle C1 is viewing content CT2, the identity of the agent image AG1 can be easily identified by continuously displaying the moving agent image AG1. However, if none of the occupants of vehicle C1 are viewing content CT2, it is conceivable that although the moving agent image AG1 is continuously displayed, there is no occupant viewing the moving agent image AG1 that overlaps with content CT2. Therefore, if none of the occupants of vehicle C1 are viewing content CT2, the display of the moving agent image AG1 can be stopped. That is, the agent image AG1 can be displayed at the second position SP1 after being removed from the first position PP1. Furthermore, if it is detected that any occupant is viewing content CT2, the agent image AG1 can be moved in a manner that does not overlap with content CT2. Figure 1 This example is shown in the image.

[0098] exist Figure 6 (B), (C) Figure 7In (A) to (C), examples are shown of a migration where the agent image AG1 is moved from a first position PP1 to a second position SP1 when content CT1, which is specific content, is displayed on the display unit 201. Figure 6 Example (B) shows an example of moving the agent image AG1 on the back side of the content CT1 (i.e., the opposite side of the display surface of the display unit 201).

[0099] like Figure 6 As shown in (B), when the agent image AG1 moves along a straight path from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT1. In this case, it is important not to obstruct the visibility of the displayed content CT1 when it is specific content. Therefore, when the displayed content CT1 is specific content, the output control unit 128 performs display control configured such that the agent image AG1 moves along the straight path from the first position PP1 to the second position SP1 on the back side of the content CT1 (i.e., the opposite side of the display surface of the display unit 201). This maintains the identity of the moving agent image AG1 and does not obstruct the visibility of the content CT1.

[0100] exist Figure 6 Example (C) shows a transparent or semi-transparent agent image AG1 moving on the surface side of the content CT1 (i.e., the display surface side of the display unit 201).

[0101] like Figure 6 As shown in (C), when the displayed content CT1 is specific content, the output control unit 128 performs display control configured such that a transparent or semi-transparent agent image AG1 moves along a straight path from the first position PP1 to the second position SP1 on the surface of the content CT1 (i.e., the display surface side of the display unit 201). This maintains the uniformity of the moving agent image AG1 and does not obstruct the visibility of the content CT1. Furthermore, the transparency of the agent image AG1 can be set according to the importance of the overlapping content. For example, in the case of specific content that cannot be obscured by regulations, its importance is determined to be high, and the transparency of the agent image AG1 is set to high. That is, the agent image AG1 is set to be completely transparent. On the other hand, for example, in the case of specific content that can be considered to be acceptable even if partially obscured, its importance is determined to be low, and the transparency of the agent image AG1 is set to low.

[0102] exist Figure 7 Examples of moving the agent image AG1 around the content CT1 are shown in (A) and (B).

[0103] like Figure 7 As shown in (A) and (B), when the content CT1 displayed is specific content, the output control unit 128 performs display control configured as follows: the agent image AG1 moves along a detour path that bypasses the content CT1, serving as a movement path from the first position PP1 to the second position SP1. Figure 7 In (A) and (B), the moving agent image AG1 is represented by agent images AG1a to AG1f.

[0104] Here, as Figure 7 As shown in (A) and (B), when the agent image AG1 is moved along a detour path, it is also envisioned that the range of the detour path is narrower than the display area of ​​the agent image AG1. In this case, it is possible to reduce the size of the moving agent image AG1, or to set it to a display mode in which a portion of the moving agent image AG1 is displayed on the back side of the content CT1 (i.e., the opposite side of the display surface of the display unit 201), or to set it to a display mode in which it is displayed on the outside of the display surface of the display unit 201.

[0105] exist Figure 7 In the example shown in (A), an example is shown where the size of the moving agent image AG1 is reduced and a portion of the moving agent image AG1 is displayed outside the display surface of the display unit 201.

[0106] exist Figure 7 In the example shown in (B), an example is shown where the size of the moving agent image AG1 is reduced and a portion of the moving agent image AG1 is displayed on the back side of the content CT1 (i.e., the opposite side of the display surface of the display unit 201) and outside the display surface of the display unit 201.

[0107] exist Figure 7 Example (C) is shown below: the agent image AG1 is moved in a way that bypasses the content CT3, which is a specific content.

[0108] like Figure 7 As shown in (C), when the content CT3 displayed is specific content, the output control unit 128 executes display control configured as follows: the agent image AG1 moves along a detour path that bypasses the content CT3, serving as a movement path from the first position PP1 to the second position SP1. Figure 7 In (C), the moving agent image AG1 is represented by agent images AG1a to AG1d.

[0109] Here, as Figure 7As shown in (C), when the agent image AG1 is moved along a detour path, it is also assumed that the range of the detour path is larger than the display area of ​​the agent image AG1. In this case, the size of the moving agent image AG1 can be maintained.

[0110] Here, we also envision that in a circuitous path, the agent image AG1 has a portion that does not overlap with specific content but is close to it. For example, in Figure 7 In the example shown in (A), the lower parts of the moving agent images AG1b to AG1e are close to the upper part of the specific content CT1. In such a case, it is also conceivable that the visibility of the content would decrease due to the proximity of the agent image AG1 to the specific content. Therefore, if there is a proximity portion in the detour path where the agent image AG1 is close to the specific content, it can be set to a display mode in which the agent image AG1 in that proximity portion is transparent or semi-transparent. Here, the distance for proximity determination can be appropriately set based on experiments or simulations. For example, a distance of a few millimeters to a few centimeters can be used as the proximity determination criterion.

[0111] In addition, such as Figure 6 , Figure 7 As shown, when the agent image AG1 is moved, the movement trajectory of the agent image AG1 can also be represented by animation processing, etc.

[0112] Therefore, the consistency of the moving agent image AG1 can be maintained without hindering the visibility of the content CT1. The detour path and display method of the agent image AG1 for specific content can be appropriately set based on experiments or simulations.

[0113] Here, there are also cases where multiple paths can be set as detours to bypass specific content. In such cases, detours can be set based on pre-defined criteria. For example, detours can be set based on the occupants' line of sight. For instance, the occupants watching specific content can be identified based on their line of sight. Furthermore, based on the direction of the line of sight of the occupants watching specific content, the position on the display surface of the display unit 201 that is expected to be viewed most by the occupants (the position of interest) can be determined. Moreover, the detour path closest to the position of interest is determined from among the multiple detours bypassing the specific content. As a result, the occupants can easily recognize the moving performance of the moving agent image AG1.

[0114] [Example of movement when multiple specific items exist within the movement path]

[0115] The above illustrates an example of the movement of the agent image AG1 when a specific piece of content on the display overlaps with the moving agent image AG1. Here, when multiple specific pieces of content are displayed on the display unit 201, it is also envisioned that two or more of these specific pieces of content overlap with the moving agent image AG1. In this case, it is also possible to... Figure 6 (B), (C) Figure 7 The examples shown in (A) to (C) similarly move the agent image AG1. Figure 8 This example is shown in the image.

[0116] Figure 8 This is a diagram illustrating an example of the migration of the agent image AG1 when multiple specific contents CT3 and CT4 are displayed on the display unit 201 of the output device 200, and the agent image AG1 is moved from the first position PP1 to the second position SP1.

[0117] like Figure 8 As shown, when the agent image AG1 moves along a straight path from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT3 and CT4. In this case, it is also possible to... Figure 6 (B), (C) Figure 7 The examples shown in (A) to (C) similarly move the agent image AG1. Figure 8 In the middle, as Figure 7 A variation of the example shown in (A) illustrates an example in which the moving agent image AG1 is displayed in such a manner that the size of the moving agent image AG1 is maintained and a portion of the moving agent image AG1 is displayed outside the display surface of the display unit 201.

[0118] [An example of movement along a circuitous path when the agent image moves across multiple displays]

[0119] The above illustrates an example of the movement of the agent image AG1 when specific content displayed on display unit 201 overlaps with the moving agent image AG1. Here, when one or more specific contents are displayed on multiple display units (including display unit 201), it is also conceivable that the specific content overlaps with the moving agent image AG1. In this case, it is also possible to... Figure 6 (B), (C) Figure 7 (A) to (C) Figure 8 The example shown similarly moves the agent image AG1. In Figure 9 This example is shown in the image.

[0120] Figure 9This is a diagram illustrating an example of the migration of the agent image AG1 when a specific content CT5 is displayed in the display unit 201 of the output device 200 and a specific content CT6 is displayed in the HUD display area of ​​the front window 4, and the agent image AG1 is moved from the first position PP1 to the second position SP1.

[0121] Here, in order to display various images in the HUD display area of ​​the front window 4, a HUD display device for implementing the HUD is provided on the upper part of the instrument panel 2 and near the boundary with the front window 4.

[0122] A HUD display device is a display device, such as a projector or optical system, used to project light onto the display area 4a of the front window 4 and use the reflected light to allow the driver D1 to see a virtual image. That is, light projected from the HUD display device onto the display area 4a of the front window 4 is reflected by the front window 4, and the reflected light goes towards the driver D1's eyes. Furthermore, the reflected light projected onto the display area 4a and entering the driver D1's eyes is displayed along with the actual object seen through the front window 4 in a manner that overlaps with that object. In this way, the HUD display device uses the front window 4 to display a virtual image, thereby achieving a HUD display.

[0123] For example, the HUD display device is based on the information processing device 110 (see reference). Figure 5 The intelligent agent image AG1 is displayed in display area 4a under the control of the system. Additionally, the front window 4 functions as the display medium for the vehicle C1's HUD.

[0124] like Figure 9 As shown, when the agent image AG1 moves along a straight path (a straight line in three-dimensional space or a straight line in two-dimensional space based on the occupant's vision) from the first position PP1 to the second position SP1, the moving agent image AG1 overlaps with the displayed content CT5 and CT6. In this case, it is also possible to... Figure 6 (B), (C) Figure 7 (A) to (C) Figure 8 The example shown similarly moves the agent image AG1. In Figure 9 In, with Figure 8 The example shown also illustrates a display method in which the size of the moving agent image AG1 is maintained and a portion of the moving agent image AG1 is displayed outside the display surface of the display unit 201. Here, when the agent image AG1 is moved between multiple physically separate display units, it is preferable to set a circuitous path, such as a route that can be created by the occupant viewing the agent image AG1, to move the agent image AG1. For example, in the case of a moving performance that is difficult to see on a circuitous path, it is also possible to set a display method in which the agent image AG1 is transparent or semi-transparent.

[0125] In addition, Figure 9 The example shown is of displaying specific content in the HUD display areas of the display unit 201 and the windshield 4, but it is not limited to this. For example, when a display unit is provided on the steering wheel 3, it is also conceivable to display specific content in that display unit. In this case, the movement processing of the intelligent agent image AG1 can be performed based on the overlap state between the specific content displayed on at least one of the display unit 201, the HUD display area of ​​the windshield 4, and the display unit of the steering wheel 3 and the moving intelligent agent image AG1.

[0126] In this way, this embodiment can be mastered as a display system having at least one or more display units that display the agent image AG1 and a control unit that controls the position of displaying the agent image AG1 on any display unit.

[0127] Furthermore, the above illustrates an example of changing the movement processing of agent image AG1 when it overlaps with specific content while moving along a straight path. Here, for example, when agent image AG1 moving along a straight path does not overlap with specific content but is close to it, it is also conceivable that the visibility of the specific content for a passenger viewing it is reduced due to the influence of the approaching agent image AG1. Therefore, when agent image AG1 moving along a straight path is close to specific content, the movement processing of agent image AG1 can be changed in the same way. For example, it is determined whether the straight path passes through an area (proximity area) within a predetermined distance relative to the specific content. When it is determined that the straight path passes through the proximity area, agent image AG1 is moved along a path (detour path) that passes through a position further than the predetermined distance. Alternatively, the display method of the proximity area can be changed to move agent image AG1. The movement processing in these cases is similar to... Figures 6 to 9 The movement process is the same as shown above.

[0128] [Example of actions in an information processing system]

[0129] Figure 10 , Figure 11 This is a flowchart illustrating an example of agent movement processing in the information processing system 100. Furthermore, this agent movement processing is controlled by the control unit 120 (see reference 120). Figure 5 Based on storage unit 130 (refer to) Figure 5 The program stored in [the database] is executed. Furthermore, the agent's movement processing is always executed in each control cycle. Additionally, appropriate reference [is made]. Figures 1 to 9 This will illustrate the agent's movement processing.

[0130] In step S501, the output content determination unit 126 determines whether user input has been received. For example, if the receiving unit 203 receives a manual operation from any passenger, the speech acquisition unit 121 receives a voice operation from any passenger, or the driver status acquisition unit 122 or the passenger compartment status acquisition unit 123 receives a gesture operation from any passenger, it is determined that user input has been received. If user input has been received, the process proceeds to step S502. On the other hand, if no user input has been received, monitoring continues.

[0131] In step S502, the output content determination unit 126 determines whether the user input received in step S501 is voice input. That is, if the speech acquisition unit 121 receives a voice operation from any passenger, it determines that the user input is voice input. If the user input is voice input, the process proceeds to step S503. On the other hand, if the user input is not voice input, the process proceeds to step S510 (see reference). Figure 11 ).

[0132] In step S503, the output content determination unit 126 acquires the current display position (first position) of the agent image AG1 and determines the display position (second position) as the destination of the agent image AG1's movement. Furthermore, the first position can be acquired based on the display position of the agent image AG1 stored in the agent management DB 132. For example, the second position can be determined based on the position of the occupant who has made user input (e.g., hand position, eye position). Moreover, regarding the occupant who has made user input, the determination can be based on the relationship between the timing of speech acquired by the speech acquisition unit 121 and the mouth movements of the occupant acquired by the driver status acquisition unit 122 or the passenger compartment status acquisition unit 123. For example, if an occupant's mouth is moving at the time of speech, it can be assumed that the occupant is uttering voice input as user input. Additionally, the presence of occupants in each seat of the vehicle C1 can be determined based on detection values ​​from seat sensors, seatbelt sensors, etc. Alternatively, the presence of occupants in each seat of vehicle C1 can be determined based on images acquired by the driver status acquisition unit 122 or the passenger compartment status acquisition unit 123. Furthermore, if multiple microphones are installed in vehicle C1, the location of the sound can be determined based on the sound acquired by these microphones. Therefore, the technique for determining the location of these sounds can also be used to determine the location of an occupant who has provided user input.

[0133] For example, when the user input is voice input, since it is not a manual operation, the second position can be determined based on the position of the occupant's eyes rather than their hand. In this case, the second position can be determined as the position on the display surface of the display unit 201 closest to the occupant's eyes. Alternatively, if the user input is other than voice input (e.g., a manual operation), the third position can be determined based on the position of the occupant's hand (shown in step S511). In this case, the third position can be determined as the position on the display surface of the display unit 201 closest to the occupant's hand (see step S511).

[0134] In step S504, the movement mode determination unit 127 determines whether the user input received in step S501 was made by the driver D1. For example, assuming that the driver D1 is driving, manual operation is often not possible, so user input via voice is more common. The method for determining the occupant who made the user input is the same as the method shown in step S503. If the user input was made by the driver D1, the process proceeds to step S505. On the other hand, if the user input was made by an occupant other than the driver, the process proceeds to step S506.

[0135] In step S505, the movement mode determination unit 127 determines whether the driving load of driver D1, who has received user input, is less than a threshold. Furthermore, the method for detecting the driving load is the same as described above. Additionally, the threshold shown here is used to determine whether driver D1's driving load is high or low. This threshold can be appropriately set based on experiments or simulations. Furthermore, while an example of determining whether driver D1's driving load is less than the threshold is shown here, it is also possible to determine whether vehicle C1 is moving or stationary when autonomous driving is not in progress. If driver D1's driving load is less than the threshold, the process proceeds to step S506. On the other hand, if driver D1's driving load is above the threshold, the process proceeds to step S510.

[0136] In step S506, the movement mode determination unit 127 confirms the relationship between the first movement trajectory of the agent image AG1, which moved from the first position to the second position as determined in step S503, and one or more contents displayed on the display unit 201. For example, the movement mode determination unit 127 determines whether there is displayed content between the first position and the second position. If there is displayed content between the first position and the second position, it determines whether that content overlaps with the first movement trajectory of the agent image AG1. Furthermore, the method for determining whether the first movement trajectory of the agent image AG1, which moved from the first position to the second position, overlaps with the displayed content is similar to... Figure 4 The determination method shown in (B) is the same.

[0137] In step S507, the movement mode determination unit 127 determines whether there is specific content in the display that overlaps with the first movement trajectory of the agent image AG1 moving from the first position to the second position. If specific content overlapping with the first movement trajectory exists, the process proceeds to step S508. On the other hand, if no specific content overlapping with the first movement trajectory exists, the process proceeds to step S509.

[0138] In step S508, the movement mode determination unit 127 determines the movement path and display mode for moving the agent image AG1 from the first position to the second position in a manner that does not impede the visibility of specific content determined in step S507 to overlap with the first movement trajectory of the agent image AG1. Furthermore, the output control unit 128 moves the agent image AG1 from the first position to the second position according to the determined movement path and display mode. For example, Figure 6 (B), (C) Figures 7 to 9 As shown, the agent image AG1 can be moved in a manner that does not obstruct the visibility of specific content CT1.

[0139] In step S509, the movement mode determination unit 127 determines that the agent image AG1 should move along the first movement trajectory. Furthermore, the output control unit 128 moves the agent image AG1 from a first position to a second position along the first movement trajectory. That is, the agent image AG1 is moved with the shortest possible distance. For example, as... Figure 6 As shown in (A), the intelligent agent image AG1 can be moved in a manner that overlaps with the surface side (i.e., the display surface side of the display unit 201) of the displayed content CT2.

[0140] In step S510, the movement mode determination unit 127 decides to move only the position of the agent image AG1 without performing a movement animation that moves the agent image AG1. That is, the movement mode determination unit 127 decides to display the agent image AG1 at a second position after clearing it from the first position. Furthermore, the output control unit 128 displays the agent image AG1 at the second position after clearing it from the first position.

[0141] As described above, it is assumed that when driver D1 is driving, manual operation is mostly impossible, so user input via voice is more common. Furthermore, when driver D1's driving load is high (above a threshold), it can be assumed that driver D1 does not have sufficient time to observe the movement of the agent image AG1; therefore, it can be assumed that temporarily removing the agent image AG1 will not affect its presence. Additionally, when driver D1's driving load is high (above a threshold), it can be assumed that moving the agent image AG1 instantly from a first position to a second position is easier for driver D1 to recognize than performing a movement animation. Therefore, in step S510, only the display position of the agent image AG1 is changed, without performing a movement animation for the agent image AG1.

[0142] In step S511, the output content determination unit 126 acquires the current display position (first position) of the agent image AG1 and determines the display position (third position) as the destination of the agent image AG1's movement. Here, assuming the user input is manual input instead of voice input, an example is shown where the third position is determined based on the position of the hand of the occupant who performed the user input, as a reference. Furthermore, the occupant who performed the user input (manual input) can be determined based on the relationship between the timing of the operation received by the receiving unit 203 and the movement of the passenger's hand acquired by the driver status acquisition unit 122 or the passenger compartment status acquisition unit 123. For example, if an occupant's hand is moving near the display unit 201 at the timing of the operation received by the receiving unit 203, it can be assumed that the occupant is performing a manual operation as user input. Additionally, if a contact operation is performed on the surface of the display unit 201, the position of that hand can be set as the position where the contact operation occurred.

[0143] For example, imagine performing a selection operation on an operation button related to the content displayed on the display unit 201 as user input. In this case, to assist with the operation, the agent image AG1 can be moved to the vicinity of the operation button on which the selection operation was performed.

[0144] The processes in steps S512 to S514 and S516 correspond to the processes in steps S506 to S509, so the explanation here is omitted.

[0145] In step S515, the movement mode determination unit 127 determines whether the occupant is viewing content (other than specific content) on a display that was determined in step S513 to overlap with the first movement trajectory of the agent image AG1. As described above, it is assumed that the driver D1 is mostly unable to perform manual operation while driving, therefore the occupant targeted by each process in steps S511 to S516 is an occupant other than the driver D1 (or the driver D1 whose driving load is less than a threshold). Furthermore, in the case of an occupant other than the driver D1 (or the driver D1 whose driving load is less than a threshold), it is assumed that they are viewing and enjoying the content displayed on the display unit 201. Therefore, in the case of an occupant viewing the content on the display, it is important that the visibility of the content is not obstructed, even if the content is not specific. Therefore, when an occupant is viewing the content on the display, a movement performance that does not obstruct the visibility of the content is executed. In addition, it is possible to set this movement performance to be the same as the movement performance of specific content. That is, in step S514, the content that the occupant is viewing is set as the target of the movement performance along with the specific content.

[0146] Furthermore, the above examples illustrate changing the movement path or display mode of the agent image AG1 in a manner that does not obstruct the visibility of specific content, but this embodiment is not limited to this. For example, the visibility of specific content can also be ensured by changing the timing and speed of the movement of the agent image AG1. For example, by moving the agent image AG1 at a speed faster than usual, the visibility of specific content can be ensured without obstructing it. For example, even when there is a portion where the agent image AG1 overlaps with specific content, the time when the specific content is not visible can be shortened by moving the agent image AG1 quickly along the movement path that includes the overlapping portion. For example, it is possible to determine whether any occupant of vehicle C1 is viewing specific content, and to move the agent image AG1 (at a normal speed or a speed faster than usual) when the occupant is not viewing the specific content. For example, when moving the agent image AG1 along a movement path that includes the overlapping portion where the agent image AG1 overlaps with the specific content, by moving the agent image AG1 when the occupant is not viewing the specific content, it is possible to suppress the situation where the occupant feels that the specific content is not visible.

[0147] In addition, Figure 10 , Figure 11The example shown illustrates changing control content based on whether the user input is voice input, but this embodiment is not limited to this and can be implemented using other control examples. For example, agent movement processing (steps S503 to S510 or steps S511 to S516) can be performed even when any user input is made. Furthermore, in Figure 10 The embodiment illustrates a control process that changes the movement mode of the agent image AG1 based on a determination of the driver's driving load (step S505). However, this embodiment is not limited to this and can be implemented using other control examples. For instance, the determination of the driver D1's driving load (steps S505 and S510) can be omitted. Furthermore, in... Figure 11 The diagram illustrates a control process that changes the movement of the agent image AG1 based on a determination process (step S515) based on whether the occupant is viewing content. However, this embodiment is not limited to this and can be implemented using other control examples. For instance, the determination process (step S515) on whether the occupant is viewing content may be omitted.

[0148] Furthermore, an example of control processing that alters the movement of agent image AG1 based on factors such as whether it overlaps with specific content is shown. However, this control processing can also be performed using other methods. For example, control processing can be performed using artificial intelligence (AI). For instance, it is possible to pre-learn various situations related to driver D1, vehicle C1, etc., the movement of the agent corresponding to these situations, and the overlap state between agent image AG1 and specific content, and use this learned data for control processing. For example, for various situations arising regarding driver D1, vehicle C1, etc., and the overlap state between agent image AG1 and specific content, the learned data can be used to determine the movement mode of the agent, and the agent image AG1 can move in the determined movement mode.

[0149] [Example of the effect of this implementation method]

[0150] In this embodiment, when the agent image AG1 is moved to a position corresponding to user input in response to that input, the agent image AG1 can be moved in a manner that does not obstruct the visibility of specific content (important content). That is, when specific content is displayed midway along the shortest path of the agent image AG1, the agent image AG1 can be moved to avoid that specific content, thus ensuring its visibility. For example, the agent image AG1 can be moved along a circuitous path, or its shape can be changed, or its transparency can be changed, or its movement speed or timing can be altered.

[0151] Therefore, passengers who have provided user input can visually (or through a sense of presence) confirm the movement of the agent image AG1 in response to the user input, and the visibility of specific content is not obstructed. Thus, the agent image AG1 can provide appropriate operational assistance to the user. Furthermore, the agent image AG1, moving without obscuring specific content, can perform intelligent actions. This increases trust and attachment to the agent image AG1, thereby improving the ability to convey information from the agent image AG1 to each passenger. In this embodiment, the agent image AG1 can be moved appropriately in the display unit 201 without obstructing the visibility of specific content.

[0152] [Examples of performing processing on other devices or systems]

[0153] Furthermore, while examples of decision processing, detection processing, and control processing being performed in the information processing device 110 (or information processing system 100) have been shown above, all or part of these processes may also be performed in other devices. In this case, the information processing system is constituted by the devices that perform a portion of these processes. For example, at least a portion of each process can be performed using various information processing devices and various electronic devices, such as in-vehicle devices, user-usable devices (e.g., smartphones, tablets, personal computers, car navigation systems, IVI), and servers connected via a network such as the Internet.

[0154] Furthermore, part (or all) of the information processing system capable of performing the functions of information processing device 110 (or information processing system 100) can be provided through an application that can be provided via a defined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[0155] [Structure and Effects of This Embodiment]

[0156] The information processing method involved in this embodiment is an information processing method that displays an agent image AG1 that communicates with the occupants of the vehicle C1 on the display unit 201 (including the HUD display area of ​​the front window 4 and other display units) and controls the display state of the agent image AG1 based on the occupants' state. The information processing method includes: a determination process (steps S506, S507, S512, S513), in which, when the agent image AG1 originally displayed at a first position (initial position) in the display unit 201 moves to a second position (the position to be moved to in response to user input) based on the occupant's state, it is determined whether the agent image AG1 obstructs the visibility of specific content when moving along the first path, based on the linear movement path (first path) from the first position to the second position and the content displayed on the display unit 201; and a control process (steps S508, S509, S514, S516), in which, if the visibility of specific content is not obstructed, the agent image AG1 is moved along the first path in a first display mode (steps S509, S516), and if the visibility of specific content is obstructed, the agent image AG1 is moved along the first path in a first display mode. In the case of a certain condition, any one of the first movement processing, the second movement processing, and the third movement processing is executed (steps S508 and S514). The first movement processing is used to move the agent image AG1 along a movement path (second path) different from the first path and not obstructing the visibility of specific content. The second movement processing is used to move the agent image AG1 from the first position to the second position in a display mode different from the first display mode and not obstructing the visibility of specific content (second display mode). The third movement processing is used to move the agent image AG1 from the first position to the second position at a second movement timing different from the first movement timing in the first display mode and not obstructing the visibility of specific content, or at a second movement speed different from the first movement speed in the first display mode and not obstructing the visibility of specific content. For example, in the first movement processing, the following steps are executed: Figures 7 to 9 The movement process is shown. Additionally, for example, it is performed in the second movement process. Figure 6 The movement processes shown in (B) and (C) are described. Furthermore, the program involved in this embodiment is a program that causes a computer to execute these processes. In other words, the program involved in this embodiment is a program that enables a computer to perform various functions that can be executed by the information processing device 110.

[0157] According to this structure, the agent image AG1, which moves based on the occupant's state, can be confirmed visually (by its presence in the case of transparency) without hindering the visibility of specific content. That is, in the display unit 201, the agent image AG1 can be moved appropriately in a manner that does not hinder the visibility of specific content.

[0158] In the information processing method of this embodiment, in the first movement process, a detour path bypassing specific content is used as the second path to move the agent image AG1. For example, it is possible to... Figures 7 to 9 The motion processing shown is used to move the agent image AG1.

[0159] According to this structure, it is possible to visually confirm an agent image AG1 that bypasses specific content in the vicinity of specific content.

[0160] In the information processing method of this embodiment, during the first movement process, when there is a portion in the detour path where the agent image AG1 overlaps with specific content, or when there is a portion in the detour path where the agent image AG1 is close to specific content, the display method is configured to make the agent image AG1 in the overlapping or close portion transparent or semi-transparent, or to display the agent image AG1 in the overlapping portion behind the specific content (inside the depth direction of the display unit 201). For example, such as Figure 7 As shown in (B), when there is a portion in the detour path where the agent image AG1 overlaps with the specific content CT1, it is possible to set a display mode in which the agent image AG1 in the overlapping portion is positioned behind the specific content CT1.

[0161] According to this structure, when the agent image AG1 overlaps or approaches specific content in a detour path, it is possible to prevent the specific content from being difficult to view due to the overlap or proximity.

[0162] In the information processing method according to this embodiment, during the first movement process, if there is a portion of the agent image AG1 that extends beyond the display surface of the display unit 201 in the detour path, the portion of the agent image AG1 that extends beyond the display surface is not displayed. For example, Figure 7 As shown in (A), when there is a portion of the agent image AG1 that extends beyond the display surface of the display unit 201 (the upper portion of the agent images AG1b to AG1e) in the detour path, the agent image AG1 that extends beyond the display surface can be prevented from being displayed.

[0163] According to this structure, even when the movement path of the agent image AG1 moving on a detour is narrow, the agent image AG1 can be displayed while maintaining its size.

[0164] In the information processing method of this embodiment, it is also possible that, in the determination process (steps S506, S507, S512, S513), if the agent image AG1 moving along the first path is close to or overlaps with specific content, it is determined that the moving agent image AG1 obstructs the visibility of the specific content. In this case, in the second movement process, the agent image AG1 can be moved along the first path, and the following display mode can be set as the second display mode: a display mode in which the agent image AG1 overlapping with the specific content is placed behind the specific content, or a display mode in which the agent image AG1 that is close to or overlaps with the specific content is transparent or semi-transparent. For example, Figure 6 As shown in (B), the agent image AG1 can be moved along a first path, and is configured to be displayed in a manner that positions the agent image AG1, which overlaps with specific content, behind the specific content. Additionally, for example, as... Figure 6 As shown in (C), the display mode can be configured to make the portion of the agent image AG1 overlapping with specific content transparent or semi-transparent. Alternatively, the display mode can also be configured to make the portion of the agent image AG1 close to specific content transparent or semi-transparent.

[0165] According to this structure, the agent image AG1 can be moved along a first path, and can be configured to display in a way that places the portion close to or overlapping with the specific content behind the specific content, or to display the agent image AG1 in a transparent or semi-transparent manner. Thus, the agent image AG1 can move along the same path as in the case of content other than the specific content, thereby enabling the execution of a motion performance that facilitates motion prediction for the occupants.

[0166] In the information processing method described in this embodiment, the third movement process may involve using a movement speed faster than the first movement speed as the second movement speed to move the agent image AG1. However, if the specific content is information that is legally required to be continuously displayed, this movement control is not performed.

[0167] According to this structure, even when there is a portion where the agent image AG1 overlaps with specific content, the time when the specific content cannot be seen can be shortened by rapidly moving the agent image AG1 along the movement path that includes the overlapping portion. Therefore, the agent image AG1 can be moved appropriately in the display unit 201 in a manner that does not obstruct the visibility of the specific content.

[0168] The information processing method according to this embodiment may also include a detection process in which the occupant's gaze is detected based on an image including the occupant's eyes. For example, the movement mode determination unit 127 can detect the occupant's (including driver D1's) gaze based on images acquired by the driver status acquisition unit 122 and the passenger compartment status acquisition unit 123. Furthermore, known gaze detection techniques can be used for this gaze detection. Moreover, in the third movement process, it may be possible to determine whether the occupant is viewing specific content based on their gaze, and to set a second movement opportunity when the occupant is not viewing specific content.

[0169] According to this structure, when the agent image AG1 is moved along a movement path that includes an overlapping portion where the agent image AG1 overlaps with specific content, the agent image AG1 can be moved even when the occupant is not viewing the specific content. Therefore, the agent image AG1 can be moved appropriately in the display unit 201 in a manner that does not obstruct the visibility of the specific content.

[0170] In the information processing method of this embodiment, the specific content may be at least one of the following: map content displayed to guide the travel direction of vehicle C1, path information content indicating the movement path from the current position of vehicle C1 to the destination in the map content, driving control information content related to the driving control of vehicle C1, abnormal occurrence information content related to the abnormal occurrence of vehicle C1, and information that is required to be continuously displayed by law.

[0171] Based on this structure, important information can be set as specific content.

[0172] The information processing method according to this embodiment may further include a driving load determination process (step S505). In the driving load determination process (step S505), the driving load of driver D1 is determined based on at least one of environmental information related to the external environment of vehicle C1 and driving action information related to the driving action of driver D1. In this case, in the determination process (steps S506 and S507), if the driving load is less than a threshold, it is determined whether the visibility of specific content is obstructed. Moreover, in the control process, if the driving load is above the threshold, after the agent image AG1 displayed at the first position is eliminated, the agent image AG1 is displayed at the second position (step S510). If the driving load is less than the threshold, the movement processing of the agent image AG1 is performed based on the determination result in the determination process (steps S506 and S507) (steps S508 and S508).

[0173] For example, when driver D1 is under high driving load, it can be assumed that driver D1 does not have sufficient time to observe the movement of agent image AG1. Therefore, it can be assumed that even if agent image AG1 is temporarily removed, its presence will not be affected. Furthermore, when driver D1 is under high driving load, it can be assumed that moving agent image AG1 instantaneously from a first position to a second position is easier for driver D1 to recognize than performing a movement sequence. In this way, the movement sequence of agent image AG1 can be appropriately performed based on driver D1's driving load.

[0174] The information processing device 110 is an information processing device that displays an agent image AG1 for communication with the occupants of the vehicle C1 on the display unit 201 (including various display units such as the HUD display area of ​​the front window 4) and controls the display state of the agent image AG1 based on the occupant's state. The information processing device 110 includes: a movement mode determination unit 127 (an example of a determination unit), which determines whether the agent image AG1, originally displayed at a first position (initial position), moves to a second position (the position to be moved to in response to user input) on the display unit 201 based on the occupant's state and a linear movement path (first path) from the first position to the second position; and an output control unit 128 (an example of a control unit), which, if the view of the specific content is not obstructed, causes the agent image AG1 to move along the first path in a first display mode; and if the view of the specific content is obstructed, the output control unit 128 causes the agent image AG1 to move along the first path in a first display mode; and if the view of the specific content is obstructed, the output control unit 128 causes the agent image AG1 to move along the first path in a first display mode. In this case, the output control unit 128 executes any one of the first movement processing, the second movement processing, and the third movement processing. The first movement processing moves the agent image AG1 along a movement path (second path) different from the first path and not obstructing the visibility of specific content. The second movement processing moves the agent image AG1 from the first position to the second position in a display mode (second display mode) different from the first display mode and not obstructing the visibility of specific content. The third movement processing moves the agent image AG1 from the first position to the second position at a second movement timing different from the first movement timing in the first display mode and not obstructing the visibility of specific content, or at a second movement speed different from the first movement speed in the first display mode and not obstructing the visibility of specific content. Furthermore, the information processing device 110 may be a device built into the output device 200, or it may be a device different from the output device 200. Alternatively, as an alternative to the information processing device 110, an information processing system may be configured consisting of multiple devices capable of performing the various processes implemented by the information processing device 110.

[0175] According to this structure, the image AG1 of the agent, which moves based on the occupant's state, can be confirmed visually (in the case of transparency, through the presence of the agent AG1) without obstructing the visibility of specific content. That is, the image AG1 of the agent can be moved appropriately in the display unit 201 in a manner that does not obstruct the visibility of specific content.

[0176] Furthermore, each processing procedure shown in this embodiment represents one example of implementing this embodiment. Within the scope of being able to implement this embodiment, the order of some processing procedures can be changed, some processing procedures can be omitted, or other processing procedures can be added.

[0177] Furthermore, each process in this embodiment is executed based on a program for causing a computer to perform various processing procedures. This embodiment can also be understood as an embodiment of a program that implements the functions of these processes and a recording medium that stores the program. For example, by updating the information processing device to add new functions, the program can be stored in the storage device of the information processing device. Thus, the updated information processing device can perform the processes shown in this embodiment.

[0178] The embodiments of the present invention have been described above. However, the above embodiments are merely examples of application of the present invention and are not intended to limit the technical scope of the present invention to the specific structures of the above embodiments.

Claims

1. An information processing method comprising displaying an agent image for communicating with occupants of a vehicle on a display unit, and controlling the display state of the agent image based on the state of the occupants. The information processing method includes: The determination process involves determining whether the smart agent image, originally displayed at a first position, moves to a second position in the display unit based on the occupant's state. This determination is based on the first path, a straight movement path from the first position to the second position, and the content displayed on the display unit. as well as The control process moves the agent image along the first path in a first display mode without obstructing the visibility of the specific content. If the visibility of the specific content is obstructed, any one of the first, second, and third movement processes is executed. The first motion processing is used to move the agent image along a second path, which is a different motion path from the first path. The second path is a motion path that does not obstruct the visibility of the specific content. The second movement processing is used to move the agent image from the first position to the second position in a second display mode different from the first display mode. The second display mode is a display mode that does not obstruct the visibility of the specific content. The third motion processing is used to move the intelligent agent image from the first position to the second position at a second motion timing different from the first motion timing in the first display mode, or at a second motion speed different from the first motion speed in the first display mode. The second motion timing is a motion timing that does not obstruct the visibility of the specific content, and the second motion speed is a motion speed that does not obstruct the visibility of the specific content.

2. The information processing method according to claim 1, wherein, In the first movement process, a detour path that bypasses the specific content is used as the second path to move the agent image.

3. The information processing method according to claim 2, wherein, In the first motion process, when there is a portion in the detour path that causes the agent image to overlap with the specific content, or when there is a portion in the detour path that causes the agent image to be close to the specific content, the display mode is configured to make the agent image in the overlapping portion or the close portion transparent or semi-transparent, or to display the agent image in the overlapping portion behind the specific content.

4. The information processing method according to claim 2, wherein, In the first movement process, if the agent image in the detour path has a portion that extends beyond the display surface in the display unit, the portion of the agent image that extends beyond the display unit is not displayed.

5. The information processing method according to claim 1, wherein, In the determination process, if the agent image moving along the first path is close to or overlaps with the specific content, it is determined that the agent image making the movement obstructs the visibility of the specific content. In the second movement process, the agent image is moved along the first path, and the following display mode is set as the second display mode: a display mode in which the agent image of the portion overlapping with the specific content is positioned behind the specific content, or a display mode in which the agent image of the portion close to or overlapping with the specific content is transparent or semi-transparent.

6. The information processing method according to claim 1, wherein, In the third motion process, a motion speed faster than the first motion speed is used as the second motion speed to move the agent image.

7. The information processing method according to claim 1, wherein, It also includes a detection process in which the occupant's gaze is detected based on an image containing the occupant's eyes. In the third movement process, it is determined whether the occupant is viewing the specific content based on the occupant's line of sight, and the second movement timing is set when the occupant is not viewing the specific content.

8. The information processing method according to any one of claims 1 to 7, wherein, The specific content refers to at least one of the following: map content displayed to guide the vehicle's direction of travel, path information content representing the movement path from the vehicle's current location to its destination in the map content, driving control information content related to the vehicle's driving control, anomaly information content related to the occurrence of anomalies in the vehicle, and information content that is required to be continuously displayed by law.

9. The information processing method according to any one of claims 1 to 7, wherein, It also includes a driving load determination process, in which the driver's driving load is determined based on at least one of environmental information related to the vehicle's external environment and driving action information related to the driver's driving actions. In the determination process, if the driving load is less than a threshold, it is determined whether the visibility of the specific content is impeded. In the control process, If the driving load exceeds a threshold, after the agent image displayed at the first position is removed, the agent image is displayed at the second position. When the driving load is less than a threshold, the motion processing of the agent image is performed based on the determination result in the determination process.

10. An information processing apparatus that causes a display unit to display an image of an intelligent agent for communicating with occupants of a vehicle, and controls the display state of the intelligent agent image based on the state of the occupants. The information processing device includes: The determination unit, when the agent image originally displayed at a first position moves to a second position on the display unit based on the occupant's state, determines whether the agent image obstructs the visibility of specific content when moving along the first path, based on a linear movement path (i.e., a first path) from the first position to the second position and the content displayed on the display unit; and The control unit, without obstructing the visibility of the specific content, moves the agent image along the first path in a first display mode; if the visibility of the specific content is obstructed, the control unit executes any one of the first, second, and third movement processes. in, The first motion processing is used to move the agent image along a second path, which is a different motion path from the first path, and the second path is a motion path that does not obstruct the visibility of the specific content. The second movement processing is used to move the agent image from the first position to the second position in a second display mode different from the first display mode. The second display mode is a display mode that does not obstruct the visibility of the specific content. The third motion processing is used to move the intelligent agent image from the first position to the second position at a second motion timing different from the first motion timing in the first display mode, or at a second motion speed different from the first motion speed in the first display mode. The second motion timing is a motion timing that does not obstruct the visibility of the specific content, and the second motion speed is a motion speed that does not obstruct the visibility of the specific content.

11. A program executed by a computer, wherein, The computer causes the display unit to show an image of an intelligent agent for communicating with the occupants of the vehicle, and controls the display state of the intelligent agent image based on the state of the occupants. The program causes the program to perform the following processing: The determination process involves determining whether the smart agent image, originally displayed at a first position, moves to a second position in the display unit based on the occupant's state. This determination is based on the first path, a straight movement path from the first position to the second position, and the content displayed on the display unit. as well as The control process moves the agent image along the first path in a first display mode without obstructing the visibility of the specific content. If the visibility of the specific content is obstructed, any one of the first, second, and third movement processes is executed. The first motion processing is used to move the agent image along a second path, which is a different motion path from the first path. The second path is a motion path that does not obstruct the visibility of the specific content. The second movement processing is used to move the agent image from the first position to the second position in a second display mode different from the first display mode. The second display mode is a display mode that does not obstruct the visibility of the specific content. The third motion processing is used to move the intelligent agent image from the first position to the second position at a second motion timing different from the first motion timing in the first display mode, or at a second motion speed different from the first motion speed in the first display mode. The second motion timing is a motion timing that does not obstruct the visibility of the specific content, and the second motion speed is a motion speed that does not obstruct the visibility of the specific content.

Images