Information processing method and information processing apparatus

Abstract

An information processing method that provides notification information to users using an agent apparatus having a display unit for displaying face images. The information processing method includes a control process for controlling the display form of the display unit. In the control process, when the display unit is caused to display notification information, switchover rendering using the display unit is executed, and the notification information is shown by the display unit after the display of a face image is erased.

Description

Technical Field

[0001] The present invention relates to an information processing method and an information processing apparatus for notifying a user of information.

Background Art

[0002] Conventionally, there are robots capable of communicating with users. For example, a device has been proposed that displays information such as a character string on a display device that displays a face image to communicate with the user (see, for example, JP2019-124855A).

Summary of the Invention

[0003] In the above-described conventional technology, it is possible to gradually change the face image and completely hide the face image to display information such as a character string on the display device. Here, assume a case where notification information (notification information) to be notified is displayed on a display device on which a face image that gives the user a sense of being alive is displayed. In this case, there is a risk that the sense of being alive may be insufficient after the switching time when transitioning from information (face image) that gives a sense of being alive to notification information that does not give a sense of being alive. In such a case, it is also assumed that the interest in the notification information displayed on the display device decreases due to the lack of the sense of being alive, and it may be difficult to appropriately notify the user of the notification information. Therefore, when displaying information to be notified to the user, it is important to increase the user's interest in the notification of the information at the time of switching and to more appropriately notify the user of the information.

[0004] An object of the present invention is to more appropriately notify a user of information.

[0005] One aspect of the present invention is an information processing method for providing notification information to a user using an agent device having a display device that displays a face image. This information processing method includes a control process for controlling the display mode of the display device, The display unit includes a first display unit that displays a face image and a second display unit that displays notification information to be provided to the user. In this control process, when displaying notification information on the second display unit, the first display unit is used to perform a switching effect to guide the user's attention by changing the face image to a predetermined expression, and after the face image is removed from the first display unit, the notification information is displayed on the second display unit. [[ID=2 (29]]

Brief Description of the Drawings

[0006] [Figure 1] Figure 1 is a simplified diagram showing an example of the interior layout of a vehicle. [Figure 2] Figure 2 is a simplified front view showing an example of the external configuration of an agent device. [Figure 3] Figure 3 shows an example of the system configuration of an information processing system installed in a vehicle. [Figure 4] Figure 4 shows an example of the transition when providing notification information using two display units. [Figure 5] Figure 5 shows an example of the transition when providing notification information using two display units. [Figure 6] Figure 6 shows an example of the transition when providing notification information using two display units. [Figure 7] Figure 7 shows an example of a transition when providing notification information using two display units realized by utilizing a flat display panel that is rotatably mounted on the face. [Figure 8] Figure 8 shows an example of the transition when providing notification information using a sliding display unit and a fixed display unit. [Figure 9] Figure 9 shows an example of the transition when providing notification information using two display units. [Figure 10] Figure 10 shows an example of the transition when providing notification information using a single display unit. [Figure 11] Figure 11 is a flowchart showing an example of a notification information output process in an information processing device. [Figure 12] Figure 12 is a flowchart showing an example of broadcast information output processing in an information processing device. [Figure 13] Figure 13 is a flowchart showing an example of broadcast information output processing in an information processing device. [Modes for carrying out the invention]

[0007] Embodiments of the present invention will be described below with reference to the attached drawings.

[0008] [Example of agent device installation] Figure 1 is a simplified diagram showing an example of the interior configuration of vehicle C1. Note that Figure 1 shows an example of the exterior configuration as viewed from the rear of vehicle C1, specifically the area in front of the driver's seat and passenger seat (not shown). Furthermore, for the sake of clarity, Figure 1 omits illustrations of anything other than the dashboard 2, steering wheel 3, front windshield 4, rearview mirror 5, camera 101, and agent equipment 200.

[0009] The agent device 200 is a small robot installed on the dashboard 2 of the vehicle C1. In this embodiment, an example is shown in which a robot modeled after a human is used as the agent device 200. Note that Figure 1 shows an example in which the agent device 200 is installed on the dashboard 2, but it is not limited to this. For example, the agent device 200 may be installed on the top of the front windshield 4. Also, Figure 1 shows an example in which a robot modeled after a human is used as the agent device 200, but it is not limited to this. For example, the agent device 200 may be a robot modeled after an animal such as a rabbit or a pig, a robot modeled after a virtual creature (for example, the face of an anime character), or a robot modeled after another object (for example, a television-type device, a radio-type device). In this way, it is possible to use an anthropomorphic agent as the agent device 200.

[0010] The agent device 200 performs various operations based on instructions from the information processing device 110 (see Figure 3). For example, based on the control of the information processing device 110, the agent device 200 outputs various information related to driving assistance when the user is operating the vehicle. This driving assistance could include notifications of moving objects in front of or behind the vehicle. For example, as a notification of a moving object in front, it could output a voice message such as, "There's a level crossing ahead, so be careful," or "There's a person ahead." In this way, the agent device 200 performs driving assistance.

[0011] Here, "notification" means conveying or informing someone of some kind of information. Furthermore, information that should be notified to the user of vehicle C1 (the occupant of vehicle C1) will be referred to as notification information. Notification information may be conveyed to the user by image display or by audio output. In this embodiment, an example of conveying notification information to the user by image display is mainly shown. Note that the notification shown in this embodiment may also be referred to as notification, transmission, etc.

[0012] Camera 101 is installed on the ceiling inside vehicle C1 and captures images of subjects inside vehicle C1 to generate images (image data). Camera 101 is composed of, for example, one or more camera devices or image sensors capable of capturing images of subjects. For example, camera 101 can be installed on the upper part of the front windshield 4, i.e., above the rearview mirror 5. Although Figure 1 shows an example with at least one camera 101, two or more imaging devices may be used, and images from all or some of these imaging devices may be used. Furthermore, the installation location of each imaging device is not limited to the example shown in Figure 1 and can be changed as appropriate. In addition, one or more devices capable of acquiring images of subjects present in all directions around vehicle C1 and subjects inside vehicle C1, such as a 360-degree camera, may be used.

[0013] [Example of external configuration of agent device] Figure 2 is a simplified front view showing an example of the external configuration of the agent device 200. In this embodiment, when the agent device 200 is ON, an example is shown in which a face image, notification information, etc., are displayed on the display unit 210. The agent device 200 can be turned ON in response to the ON operation of the vehicle C1. The agent device 200 can also be turned ON in response to user operation related to the agent device 200. Note that the ON / OFF operation of the vehicle C1 refers to the ON or OFF operation of the start key related to starting or stopping the vehicle C1.

[0014] Figure 2(A) shows a front view of the agent device 200 in its normal state. The agent device 200 is composed of a roughly box-shaped body part 202 and a roughly box-shaped face part 201, which are arranged vertically via a connecting part 203. The face part 201 is also provided with a display unit 210 for displaying various images. A body image representing the body is displayed on the surface of the body part 202. Various display media such as a light-emitting sheet, a liquid crystal display, or an organic EL (Electro-Luminescence) can be used as the display unit 210. The connecting part 203 is recognized as the part corresponding to the neck of a living organism.

[0015] The display unit 210 of the face unit 201 displays each part that makes up the agent's face, such as the eye unit 211, nose unit 212, and mouth unit 213. In this embodiment, an example is shown in which the agent device 200 is installed on the dashboard 2 of the vehicle C1 with the eye unit 211 facing inward. The side of the agent device 200 on which the display unit 210 is provided will be referred to as the front side of the agent device 200, and the side opposite to the front side of the agent device 200 will be referred to as the rear side of the agent device 200. Furthermore, in the agent device 200, the left side of Figure 2 will be referred to as the right side of the agent device 200, and the right side of Figure 2 will be referred to as the left side of the agent device 200.

[0016] Figure 2(B) shows a front view of the agent device 200 with each part of the face 201 facing left. Figure 2(C) shows a front view of the agent device 200 with each part of the face 201 facing right. As shown in Figure 2(B), by moving the eye 211 to the left, shifting the pupil portion of the eye 211 to the left, and making the mouth 213 a vertically elongated oval, it is possible to create the expression of the face 201 facing left. Also, as shown in Figure 2(C), by moving the eye 211 to the right, shifting the pupil portion of the eye 211 to the right, and making the mouth 213 a vertically elongated oval, it is possible to create the expression of the face 201 facing right.

[0017] FIG. 2(D) shows a front view of the agent device 200 in a state where the face portion 201 is expressing an effect for making the user feel a sense of liveliness as a switching effect at the timing before displaying the notification information. As an effect for making the user feel a sense of liveliness, for example, it is possible to provide a display that conveys a surprised expression to the user.

[0018] For example, as shown in FIG. 2(D), by making the eye portion 211 an ellipse that is long in the vertical direction and shifting the black eye portion of the eye portion 211 downward, and making the mouth portion 213 an ellipse that is long in the vertical direction, it is possible to create an expression of a surprised face portion 201. Further, by displaying an exclamation mark image 214 for attracting attention or outputting voice information of surprise, it is possible to further express a surprised state. Thereby, it is assumed that the user will direct attention to the display unit 210 as to what will happen in the future due to the agent device 200 being surprised. Also, by making the agent device 200 show a surprised expression, it becomes possible to make the user feel a sense of liveliness. Thereby, it is possible to direct the user's interest to the display unit 210 with respect to the display of the notification information and to enhance the user's interest. Thus, as an effect for making the user feel a sense of liveliness, it is possible to draw the user's attention by making use of biological functions such as the eyes, nose, and mouth. That is, these effects can also be understood as effects for guiding the user's attention.

[0019] In this way, by executing an effect using various images such as the exclamation mark image 214, it becomes possible to further draw the user's attention. Also, it is possible to execute various effects such as an effect using other means than an image such as the exclamation mark image 214, for example, an effect using the movement of the face portion 201, to draw the user's attention. The effect using the movement of the face portion 201 of the agent device 200 will be described in detail with reference to FIGS. 5 to 9.

[0020] Figures 2(E) and 2(F) show a front view of the agent device 200 with notification information (notification images 400 and 410) displayed on the display unit 210. Figure 2(E) shows an example of displaying notification image 400 on the display unit 210 to notify the user of wearing a seat belt. Figure 2(F) shows an example of displaying notification image 410 on the display unit 210 to notify the user that the battery level of vehicle C1 is low. Note that these notification information are just examples, and other information may be displayed as notification information. For example, a notification image to notify that one of the doors of vehicle C1 is ajar may be displayed on the display unit 210. As notification information regarding a halted door, it is possible to display image information indicating the door on the display unit 210.

[0021] Note that the shape, display mode, and audio output mode of these agent devices 200 are examples only, and other shapes, display modes, and audio output modes are also possible. Furthermore, although Figure 2 shows an example in which the face part 201 and the body part 202 are configured as separate components, the face part 201 and the body part 202 may also be configured as a single housing.

[0022] Furthermore, while this embodiment shows an example of displaying a face image as an image representing a living creature (living creature representation), other images (for example, the agent's whole body, a mechanical object, or a virtual face) may also be displayed as images representing a living creature.

[0023] Here, the agent device 200 installed on the dashboard 2 is a three-dimensional robot that has been anthropomorphized, and is therefore assumed to have a strong presence that will be recognized by the user of vehicle C1. For example, when it is necessary to notify the user of vehicle C1 of some information, it is conceivable to use the agent device 200 to do so. For example, it is conceivable to switch from displaying a face image that gives the user a sense of life to displaying notification information on the display unit 210. For example, it is conceivable to display notification information shown on Figure 2(E) or (F) on the display unit 210 at the same time that a face image shown on Figure 2(A) to (C) is displayed on the display unit 210. In this case, there is a risk that the sense of life will be lacking after the switch from a face image that gives a sense of life to notification information that does not give a sense of life. In such a case, it is conceivable that the user's interest in the notification information displayed on the display unit 210 will decrease due to the lack of a sense of life, and it may become difficult to appropriately notify the user of the notification information. Therefore, when displaying notification information, it is important to make the user feel a sense of realism during the transition, thereby increasing the user's interest in the display of notification information and enabling the notification information to be conveyed to the user more appropriately.

[0024] Therefore, in this embodiment, when displaying notification information on the display unit 210, a switching effect is performed using the display unit 210, and the display of the face image on the display unit 210 is erased before the notification information is displayed on the display unit 210. For example, when displaying notification information shown in either Figure 2(E) or (F) on the display unit 210 at the same time that the face image shown in any of Figures 2(A) to (C) is displayed on the display unit 210, the switching effect is performed by displaying the face image shown in Figure 2(D) on the display unit 210 before the notification information is displayed. In this way, by performing a switching effect that makes the user feel that the information is real before the notification information is displayed, it is possible to increase the user's interest in the display of the notification information and to notify the user of the notification information more appropriately.

[0025] Note that the switching animation shown in Figure 2 is just one example, and other switching animations may be implemented. For example, Figure 10 shows another example of a switching animation using the same display screen. In addition, a switching animation using multiple display screens may be implemented, for example. Figure 4 shows an example of this. These switching animations can be implemented using software-based switching processing. Furthermore, these switching animations make it possible to exclusively display face images and notification information.

[0026] Furthermore, for example, a switching effect using the physical movement of the face 201 may be implemented. Examples of this are shown in Figures 5 to 9. These switching effects can be implemented by switching processes that utilize physical movements such as rotation and translation. These switching effects also make it possible to display the face image and the notification information exclusively. Even when these switching effects using physical movements are implemented, it is possible to make the display position of the face image and the display position of the notification information the same. In this case, since the notification information is displayed at the position of the face image after the switching effect that makes the user feel that it is a living thing is implemented, it is possible to further increase the user's interest in the notification information and make it possible to notify the user of the notification information more appropriately.

[0027] [Example of an information processing system configuration] Figure 3 shows an example of the system configuration of the information processing system 100 installed in vehicle C1.

[0028] The information processing system 100 comprises a camera 101, a location information acquisition sensor 102, an audio input unit 103, sensors 104, an information processing device 110, and an agent device 200. The information processing device 110 and the agent device 200 are connected by a communication method using wired communication or wireless communication. The information processing device 110 is also connected to the network 20 by a communication method using wireless communication. The network 20 is a network such as a public telephone network or the Internet. The agent device 200 may also be connected to the network 20 by a communication method using wireless communication. Although Figure 3 shows an example in which the information processing device 110 and the agent device 200 are configured as separate units, the information processing device 110 and the agent device 200 may also be configured as a single unit.

[0029] Camera 101 captures an image of a subject and generates an image (image data) based on the control of the information processing device 110, and outputs image information related to the generated image to the information processing device 110. Camera 101 is installed inside at least one part of the vehicle C1 and captures an image of a subject inside the vehicle C1 and generates an image (image data). In Figure 1, camera 101 is shown installed inside the vehicle C1. As described above, camera 101 is composed of, for example, one or more camera devices or image sensors capable of capturing an image of a subject. For example, one camera 101 may be installed in front of the vehicle C1 to capture an image of a subject from the front of the vehicle C1 and generate an image (image data), or another camera 101 may be installed in the rear of the vehicle C1 to capture an image of a subject behind the vehicle C1 and generate an image (image data).

[0030] The position information acquisition sensor 102 acquires position information regarding the location of vehicle C1 and outputs the acquired position information to the information processing device 110. For example, this can be implemented using a GNSS receiver that acquires position information using GNSS (Global Navigation Satellite System). The position information includes various position data such as latitude, longitude, and altitude at the time of receiving the GNSS signal. Position information may also be acquired by other methods. For example, position information may be derived using information from access points or base stations in the vicinity. Position information may also be acquired using beacons. For example, based on the information acquired by the position information acquisition sensor 102, it is possible to determine the state of vehicle C1, such as driving, stopped, or reversing.

[0031] Furthermore, for example, it is possible to determine, based on the location information acquired by the location information acquisition sensor 102, whether vehicle C1 is located near a facility outside of vehicle C1, such as a coffee shop.

[0032] The audio input unit 103 is located inside the vehicle C1 and, based on the control of the information processing device 110, acquires sounds from inside the vehicle C1 and outputs sound information related to the acquired sounds to the information processing device 110. For example, one or more microphones or sound acquisition sensors can be used as the audio input unit 103.

[0033] The sensors 104 are various sensors installed on the vehicle C1, and the detection information acquired by each sensor is output to the information processing device 110. Examples of sensors include a human presence sensor, distance sensor, vehicle speed sensor, acceleration sensor, seat occupancy sensor, seat belt sensor, door sensor, battery sensor, etc. Note that these are just examples, and other sensors may be used. Also, only some of these sensors may be used.

[0034] A human presence sensor is a sensor that detects the presence, number, location, and status of people inside vehicle C1. For example, it is possible to use a human presence sensor that utilizes infrared light, ultrasound, visible light, or an image sensor. For instance, if a person is seated in the driver's seat, passenger seat, or rear seat, their presence can be detected by the human presence sensor. Furthermore, by using both a human presence sensor and a seating sensor, it is possible to improve the accuracy of detecting the seating status of each seat.

[0035] The distance sensor is a sensor that detects the distance to a person inside vehicle C1, the distance to an object to be notified, etc. For example, various sensors such as distance measuring sensors can be used as distance sensors.

[0036] The seat occupancy sensor (or seat sensor) is a sensor that detects whether or not there is an occupant seated in each seat of vehicle C1. The seat belt sensor is a sensor that detects whether or not an occupant seated in each seat of vehicle C1 is wearing a seat belt. The door sensor is a sensor that detects whether or not each door of vehicle C1 is ajar. The battery sensor is a sensor for measuring the remaining charge of the battery installed in vehicle C1. For each of these sensors, known sensors can be used.

[0037] The information processing device 110 comprises a control unit 120, a storage unit 130, and a communication unit 140. The communication unit 140 exchanges various types of information with other devices using wired or wireless communication based on the control of the control unit 120.

[0038] The control unit 120 controls each part based on various programs stored in the memory unit 130. The control unit 120 is implemented by a processing unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). For example, the processing speed can be increased by performing image processing using a GPU. Furthermore, the processing speed can be increased even further by performing parallel processing using a GPU. The vehicle ECU (Electronic Control Unit) of vehicle C1 may be used as the control unit 120, or a processing unit different from the vehicle ECU may be provided as the control unit 120. The control unit 120 also includes a conversion function that converts text to speech. This conversion function is implemented by, for example, TTS (Text to Speech).

[0039] The control unit 120 performs control processing to control the operating state of the agent device 200 based on the information output from the camera 101, position information acquisition sensor 102, voice input unit 103, sensors 104, communication unit 140, etc. Specifically, the control unit 120 includes a display switching determination unit 121 and an agent control unit 122.

[0040] The display switching determination unit 121 determines whether or not to switch the content to be displayed on the display unit 210 based on the information output from the camera 101, position information acquisition sensor 102, voice input unit 103, sensors 104, communication unit 140, etc., and outputs the determination result to the agent control unit 122. Specifically, the display switching determination unit 121 determines the timing of switching from the face image to the notification information.

[0041] For example, the display switching determination unit 121 determines whether or not notification information has been generated based on the information output from the camera 101, position information acquisition sensor 102, voice input unit 103, sensors 104, communication unit 140, etc. For example, based on the seat belt sensor that detects whether or not a seat belt is being worn and the seating sensor (or seat sensor) that detects whether or not there is an occupant in each seat of the vehicle C1, it is possible to determine whether or not the occupants in each seat of the vehicle C1 are wearing seat belts. If it is determined that an occupant is seated but is not wearing a seat belt, it is determined that notification information has been generated.

[0042] The display switching determination unit 121 may adjust the switching timing from the face image to the notification information based on the importance of the notification information, or it may adjust the switching timing from the face image to the notification information based on the operating state of the face image (for example, whether or not a predetermined effect is being performed). These methods for adjusting the switching timing will be explained in detail with reference to Figures 12 and 13.

[0043] The agent control unit 122 controls the operating state of the agent device 200 based on the determination result from the display switching determination unit 121 and the information output from the camera 101, position information acquisition sensor 102, voice input unit 103, sensors 104, communication unit 140, etc. For example, the agent control unit 122 displays the parts that make up the face of the anthropomorphized agent (eyes 211, nose 212, mouth 213) on the display unit 210. The agent control unit 122 also displays information that should be notified to the occupants of vehicle C1 (notification information) on the display unit 210. In addition, the agent control unit 122 outputs the voice of the anthropomorphized agent, voices corresponding to the information that should be notified to the occupants of vehicle C1, etc., from the sound output unit 220.

[0044] For example, the agent control unit 122 determines the notification information to be displayed on the display unit 210 and the notification information to be output from the sound output unit 220, and executes control to output that notification information. Notification information concerning objects located inside the vehicle C1 is stored in the notification information DB 132. Examples of the output of each of these notification information are shown in Figures 2, 4 to 10. For example, if the notification information concerns wearing a seat belt, an image showing a seat belt (notification image 400 (see Figure 2(E))) is displayed as the notification information. If the notification information concerns the battery level, an image showing the battery (notification image 410 (see Figure 2(F))) is displayed as the notification information. If the notification information concerns a door that is not fully closed, an image showing the door is displayed as the notification information.

[0045] Furthermore, for example, when the agent control unit 122 switches the display state of the display unit 210 from a face image to notification information, it executes control to cause the agent device 200 to perform a switching animation. This switching animation will be explained in detail with reference to Figures 4 to 10.

[0046] The memory unit 130 is a storage medium for storing various types of information. For example, the memory unit 130 stores various types of information necessary for the control unit 120 to perform various processes (e.g., control programs, agent information DB 131, notification information DB 132, map information DB 133). The memory unit 130 also stores various types of information acquired via the communication unit 140. As the memory unit 130, for example, ROM (Read Only Memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0047] The agent information DB131 stores information necessary to realize various operations of the agent device 200. For example, the face image information displayed on the display unit 210 (e.g., the eyes 211, nose 212, and mouth 213) and the audio information output from the sound output unit 220 are stored in the agent information DB131. In addition, for example, operation information for operating the face unit 201 (see Figures 5 to 9) when a switching effect is executed is stored in the agent information DB131.

[0048] The notification information DB132 stores information necessary for outputting notification information about objects located inside or outside the vehicle C1. For example, in the case of notification information about a door being ajar, image information showing the door and audio information for notifying that the door is ajar are stored as notification information in the notification information DB132. Also, for example, in the case of notification information about wearing a seat belt, image information showing the seat belt (notification image 400 (see Figure 2(E))) and audio information for notifying that the seat belt should be worn are stored as notification information in the notification information DB132. Also, for example, in the case of notification information about the battery level, image information showing the battery (notification image 410 (see Figure 2(F))) and audio information for notifying that the seat belt should be worn are stored as notification information in the notification information DB132.

[0049] Map information DB133 stores map information such as road information necessary for route guidance of vehicle C1. This map information includes road gradient, road intersections, number of road lanes, road width information, and road topography information. The map information also includes road signs indicating speed limits, one-way streets, etc., pedestrian crossings, lane markings, etc. The map information may also include information on road structures (e.g., traffic lights, utility poles), facilities such as buildings, and surrounding tourist information. Agent information DB131, notification information DB132, and map information DB133 may be stored in the storage unit 130 of vehicle C1, or they may be obtained from external devices via the network 20.

[0050] The agent device 200 is a robotic device that performs various operations based on instructions from the information processing device 110. The agent device 200 includes a display unit 210, a sound output unit 220, and a drive unit 230. The display unit 210, the sound output unit 220, and the drive unit 230 are controlled by a control unit (not shown) provided in the agent device 200.

[0051] The display unit 210 is a display unit that displays various images based on instructions from the information processing device 110. For example, the display unit 210 can be an OLED (Electroluminescence) panel, an LCD (Liquid Crystal Display) panel, or other display panel. The display unit 210 may be configured as a touch panel that allows the user to perform input by touching or bringing their finger close to the display surface, or it may be configured as a separate user interface.

[0052] In Figure 2, an example is shown in which the agent device 200 has one display unit 210, but the agent device 200 may have multiple display units. Examples of the agent device 200 having multiple display units are shown in Figures 4 to 9. Each display unit shown in Figures 4 to 9 corresponds to the display unit 210 shown in Figure 3.

[0053] The sound output unit 220 outputs various sounds based on instructions from the information processing device 110. For example, one or more speakers can be used as the sound output unit 220. Note that the display unit 210 and the sound output unit 220 are examples of a user interface, and some of them may be omitted, or other user interfaces may be used.

[0054] The drive unit 230 drives each part of the agent device 200 based on instructions from the information processing device 110. For example, the drive unit 230 is a drive device that realizes a mechanism for rotating the face unit 201 (see Figures 5, 6, and 9). The drive unit 230 is also a drive device that realizes a mechanism for moving the display panel 270 (see Figure 7) on the face unit 201. Furthermore, the drive unit 230 is a drive device that realizes a mechanism for moving the display unit 300 (see Figure 8). For example, the drive unit 230 is composed of motors, servo motors, etc., capable of driving each part. The drive unit 230 will be described in detail with reference to Figures 5 to 9.

[0055] [Example of providing notification information using multiple display units] Figure 2 shows an example of providing notification information using one display unit 210. As mentioned above, it is also possible to provide notification information using multiple display units. Therefore, Figures 4 to 9 show examples of providing notification information using multiple display units. Figures 4 to 9 show an example of outputting notification information regarding battery level (notification image 410 (see Figure 2(F))). Figure 10 shows an example of outputting notification information regarding seat belt use (notification image 400 (see Figure 2(E))).

[0056] [An example of providing notification information using multiple display units arranged vertically.] Figure 4 shows an example of a transition when providing notification information using two display units 241 and 242. In Figure 4, an example is shown in which the agent device 200 is equipped with two display units 241 and 242 arranged vertically. Specifically, display unit 241 is installed on the face unit 201, and display unit 242 is installed on the body unit 202. In other words, the example shown in Figure 4 is the agent device 200 shown in Figure 2, but with display unit 241 installed in place of display unit 210, and display unit 242 installed on the body unit 202.

[0057] The display unit 241 will mainly display facial images, and the display unit 242 will mainly display notification information. For example, under normal circumstances, various facial images will be displayed on the display unit 241, similar to the examples shown in Figures 2(A) to (C).

[0058] Furthermore, the display switching determination unit 121 determines whether or not information that should be notified to the user of vehicle C1 (notification information) has been generated, based on the information output from the camera 101, position information acquisition sensor 102, voice input unit 103, sensors 104, communication unit 140, etc. If notification information has been generated, the display switching determination unit 121 outputs that fact to the agent control unit 122.

[0059] Next, if the display switching determination unit 121 determines that notification information has been generated, the agent control unit 122 executes control to display the notification information on the display unit 242. Specifically, the agent control unit 122 executes a switching effect using the display unit 241, and after erasing the display of the face image on the display unit 241, it displays the notification information on the display unit 242. Alternatively, instead of executing a switching effect using the display unit 241, a switching effect using both the display unit 241 and the display unit 242, or a switching effect using only the display unit 242 may be executed.

[0060] For example, when notification information is generated, as shown in Figure 4(A), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect, before displaying the notification information. Specifically, the agent control unit 122 executes the switching effect by controlling the display state of the display unit 241 so that the face image on the display unit 241 shows a surprised facial expression (see Figure 2(D)).

[0061] Next, as shown in Figure 4(B), the agent control unit 122 erases the display of the face image on the display unit 241 and causes the display unit 242 to display the notification image 410 as notification information.

[0062] In this way, when displaying notification information, by displaying a surprised facial expression on the display unit 241 when switching from the facial image to the notification information, it is possible to make the user feel that it is a living thing and increase the user's interest in the notification information. This makes it possible to notify the user of the notification information more appropriately.

[0063] [An example of a housing with display units on both sides that rotates left and right to provide notification information.] Figure 5 shows an example of the transition when notification information is provided using two display units 252 and 253. In Figure 5, the agent device 200 is equipped with a housing 250 (face unit 201) that can rotate in the left and right directions, with a display unit 252 (see Figures 5(A) and 5(B)) installed on one side of the housing 250 and a display unit 253 (see Figure 5(C)) installed on the other side of the housing 250. The display unit 252 mainly displays face images, and the display unit 253 mainly displays notification information.

[0064] Specifically, the neck portion 251 acts as a rotational support, connecting the housing 250 (face portion 201) and the body portion 202. The neck portion 251 is connected to the drive unit 230 (see Figure 3), and the rotational movement of the drive unit 230 allows the housing 250 to rotate left and right (in the direction of arrow A1) with the neck portion 251 as the pivot axis. The rotation range of the housing 250 can be set to, for example, 180 to 360 degrees. For example, by rotating it about 180 degrees left and right, the front and rear directions of the housing 250 can be reversed. Normally, the display unit 252 side is set to face forward, and various face images are displayed on the display unit 252, similar to the examples shown in Figures 2(A) to (C).

[0065] Similar to Figure 4, the display switching determination unit 121 determines whether or not notification information has been generated, and if notification information has been generated, it outputs that fact to the agent control unit 122.

[0066] Next, if the display switching determination unit 121 determines that notification information has been generated, the agent control unit 122 executes control to display the notification information on the display unit 253. Specifically, the agent control unit 122 executes a switching effect using the housing 250, and after erasing the display of the face image on the display unit 252, it displays the notification information on the display unit 253.

[0067] For example, when notification information is generated, as shown in Figures 5(A) and 5(B), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect, before displaying the notification information. Specifically, the agent control unit 122 executes the switching effect by controlling the display state of the display unit 252 so that the face image on the display unit 252 shows a surprised facial expression (see Figure 2(D)).

[0068] Next, as shown in Figure 5(B), the agent control unit 122 erases the display of the face image on the display unit 252 and rotates the housing 250 to the right (in the direction of arrows A2 and A3) with the neck portion 251 as the pivot axis. During this rotation, the face image on the display unit 252 may be displayed, and the face image on the display unit 252 may be erased when this rotation is completed. In addition, before or during this rotation, audio information to give the user a sense of life may be output from the sound output unit 220.

[0069] Next, as shown in Figure 5(C), the agent control unit 122 causes the display unit 253, which is on the user side of the vehicle C1, to display a notification image 410 as notification information. In this way, when notification information is generated, a switching animation is performed using the housing 250 (display units 252 and 253), and after the display of the face image on the display unit 252 is erased, the notification information (notification image 410) is displayed on the display unit 253.

[0070] In this way, when displaying notification information, a surprised facial image is displayed on the display unit 252 when switching from the facial image to the notification information, and the housing 250 (face unit 201) is rotated. This rotation of the housing 250 makes it possible to represent the movement of the agent device 200's face turning backward, thereby increasing the user's attention to the agent device 200. Furthermore, it is possible to give the user the impression that the notification information is displayed on the head or back of the agent device 200 after the face of the agent device 200 has turned backward. This makes the user feel that it is a living thing and increases the user's attention to the notification information being displayed. In addition, the display position of the facial image and the display position of the notification information are in the same position from the user's perspective. Therefore, it is possible to display the notification information in a position that is easy for the user to see after looking at the facial image. As a result, it is possible to notify the user of the notification information more appropriately.

[0071] [An example of providing notification information by rotating a housing with display units on both sides in the vertical direction.] Figure 6 shows an example of a transition when providing notification information using two display units 264 and 265. In Figure 6, the agent device 200 is equipped with a housing 260 (face unit 201) that can rotate in the vertical direction, with a display unit 264 (see Figure 6(A)) installed on one side of the housing 260 and a display unit 265 (see Figure 6(C)) installed on the other side of the housing 260. The display unit 264 mainly displays a face image, and the display unit 265 mainly displays notification information.

[0072] Specifically, the housing 260 is installed inside the frame portion 261 having a square shape when viewed from the front, and both left and right ends of the housing 260 in the lateral direction and the inner surface of the frame portion 261 are connected by the rotation support portions 262 and 263. Note that since the positions where the rotation support portions 262 and 263 are provided correspond to the ears of the face portion 201, the rotation support portions 262 and 263 may be configured as the ear portions of the face portion 201. The rotation support portions 262 and 263 are connected to the drive portion 230 (see FIG. 8), and by the rotational operation of the drive portion 230, the face portion 201 is rotated in the vertical direction (arrow A10 direction) with the rotation support portions 262 and 263 as the rotation axes. In this way, the housing 260 is made rotatable in the vertical direction inside the frame portion 261. Note that the rotation range of the housing 260 (face portion 201) can be set, for example, in a range of 180 degrees to 360 degrees. For example, by rotating about 180 degrees in the vertical direction, it is possible to reverse the front-rear direction of the housing 260. In the normal state, it is set such that the display portion 264 side becomes the front side, and various face images are displayed on the display portion 264 as in the examples shown in FIGS. 2(A) to (C).

[0073] Similar to FIG. 4, the display switching determination unit 121 determines whether or not notification information has occurred, and when the notification information has occurred, outputs the fact to the agent control unit 122.

[0074] Next, when it is determined by the display switching determination unit 121 that the notification information has occurred, the agent control unit 122 executes control for causing the display portion 265 to display the notification information. Specifically, the agent control unit 122 executes a switching effect using the housing 260, and after erasing the display of the face image on the display portion 264, causes the display portion 265 to display the notification information.

[0075] For example, when notification information is generated, as shown in Figures 6(A) and 6(B), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect before displaying the notification information. Specifically, the agent control unit 122 executes the switching effect by controlling the display state of the display unit 264 so that the face image on the display unit 264 shows a surprised facial expression (see Figure 2(D)).

[0076] Next, as shown in Figure 6(B), the agent control unit 122 erases the display of the face image on the display unit 264 and rotates the housing 260 vertically (in the direction of arrows A11 and A12) using the rotation support units 262 and 263 as pivot axes. During this rotation, the face image on the display unit 264 may be displayed, and the face image on the display unit 264 may be erased when this rotation is completed. In addition, before or during this rotation, audio information to give the user a sense of life may be output from the sound output unit 220.

[0077] Next, as shown in Figure 6(C), the agent control unit 122 displays the notification image 410 as notification information on the display unit 265, which is on the user side of the vehicle C1. In this way, when notification information is generated, a switching animation is performed using the housing 260 (display units 264 and 265), and after the display of the face image on the display unit 264 is erased, the notification information (notification image 410) is displayed on the display unit 265.

[0078] Thus, when displaying notification information, a surprised facial expression is displayed on the display unit 264 when switching from the facial image to the notification information, and the housing 260 (face unit 201) is rotated. This rotation of the housing 260 makes it possible to represent the movement of the agent device 200's face looking downwards, thereby increasing the user's attention to the agent device 200. Furthermore, after the agent device 200's face looks downwards, it is possible to give the user the impression that the notification information is being displayed on the agent device 200's head or back. This makes the user feel that it is a living thing and increases the user's attention to the notification information being displayed. In addition, the display position of the facial image and the display position of the notification information are in the same position from the user's perspective. Therefore, the notification information can be displayed in a position that is easy for the user to see after looking at the facial image. As a result, notification information can be conveyed to the user more appropriately.

[0079] Note that the left-right rotation mechanism shown in Figure 5 is just one example; other mechanisms may be employed in which a rotation axis is located at the center of the front-to-back and left-to-right directions of a plate-shaped housing equipped with display units on both sides, and the housing rotates around this rotation axis. Similarly, the up-and-down rotation mechanism shown in Figure 6 is just one example; other mechanisms may be employed in which a rotation axis is located at the center of the front-to-back and up-and-down directions of a plate-shaped housing equipped with display units on both sides, and the housing rotates around this rotation axis.

[0080] Figures 5 and 6 show an example in which display units are provided on both sides of the housing 250 and 260. However, it is also possible to make the user feel that the housing 250 and 260 are living creatures by making them sway back and forth. Therefore, a display unit may be provided on only one side of the housing 250 and 260, and the swaying motion of the housing 250 and 260 may be performed as a switching effect. In this case, when notification information is generated, the switching effect of swaying the housing is performed, and after the display of the face image on one display unit is erased, the notification information is displayed on that display unit.

[0081] In this way, it is possible to attract the user's attention by rotating the face unit 201 or making it sway back and forth, and then perform notification actions to the user. This allows for appropriate notification to be given to the user. In this way, when the face unit 201 itself starts moving, it is possible to guide the user's attention by making them wonder what will be displayed.

[0082] [Example of providing notification information using multiple display units with movable display panels] Figure 7 shows an example of a transition when providing notification information using two display units realized by utilizing a flat plate-shaped display panel 270 (see Figures 7(B) and 7(C)) that is rotatably mounted on the face portion 201.

[0083] The display panel 270 has a first display section 271 on one side and a second display section 272 on the opposite side. The display panel 270 is also provided with a rotating support section 273 at one end, and is mounted on the display section 280 so that it can rotate around the rotating support section 273 as the axis of rotation. The display section 280 is installed on the face section 201.

[0084] Furthermore, the rotation support unit 273 is connected to the drive unit 230 (see Figure 3), and the rotational movement of the drive unit 230 allows the display panel 270 to be rotated vertically (in the direction of arrow A20 (see Figure 7(B))) using the rotation support unit 273 as the pivot axis. The rotation range of the display panel 270 can be set to, for example, 180 degrees.

[0085] The display device, consisting of a display panel 270 and a display unit 280, is controlled by the rotation of the display panel 270 to switch between a first state, which displays a face image, and a second state, which displays notification information. As shown in Figure 7(A), in the first state, the display panel 270 is positioned in the upper area of ​​the display unit 280, forming a display area consisting of the first display unit 271 of the display panel 270 and the lower display area 281 of the display unit 280. As shown in Figure 7(D), in the second state, the display panel 270 is positioned in the lower area of ​​the display unit 280, forming a display area consisting of the second display unit 272 of the display panel 270 and the upper display area 282 of the display unit 280. In the first state, a face image is mainly displayed, and in the second state, notification information is mainly displayed.

[0086] Thus, the movable display panel 270 functions like a lid that covers the upper display area of ​​the display unit 280, or the lower display area of ​​the display unit 280. For this reason, the display panel 270 can also be referred to as a folding display unit or a lid.

[0087] Similar to Figure 4, the display switching determination unit 121 determines whether or not notification information has been generated, and if notification information has been generated, it outputs that fact to the agent control unit 122.

[0088] Next, if the display switching determination unit 121 determines that notification information has been generated, the agent control unit 122 executes control to display the notification information on the display device consisting of the display panel 270 and the display unit 280. Specifically, the agent control unit 122 executes a switching effect using the display panel 270 and the display unit 280, and after erasing the display of the face image in the first state, displays the notification information in the second state.

[0089] For example, when notification information is generated, as shown in Figures 7(A) to (C), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect, before displaying the notification information. Specifically, as shown in Figure 7(A), the agent control unit 122 executes the switching effect by controlling the display state of the display panel 270 and the display unit 280 so that the face image in the first state shows a surprised facial expression (see Figure 2(D)).

[0090] Next, as shown in Figures 7(B) and 7(C), the agent control unit 122 erases the display of the face image in the first display section 271 of the display panel 270 and the lower display area 281 of the display section 280, and rotates the display panel 270 downward (in the direction of arrows A21 and A22) using the rotation support section 273 as the pivot axis. During this rotation, the face image may be displayed, and the face image may be erased at the end of this rotation. In addition, before the start of this rotation, or during this rotation, audio information to give the user a sense of realism may be output from the sound output section 220.

[0091] Next, as shown in Figure 7(D), the agent control unit 122 displays the notification image 410 as notification information in the second state, where the display panel 270 is positioned in the lower area of ​​the display unit 280. That is, the notification image 410 is displayed in the display area formed by the second display unit 272 of the display panel 270 and the upper display area 282 of the display unit 280. In this way, when notification information is generated, the display panel 270 is rotated to transition from the first state to the second state as a switching effect, and after the display of the face image in the first state is erased, the notification information (notification image 410) is displayed in the second state. Note that, as shown in Figure 7(C), the notification information (notification image 410) may also be displayed while the display panel 270 is rotating.

[0092] Thus, when displaying notification information, a surprised facial expression image is displayed on the display unit 264 and the display panel 270 is rotated when switching from the facial image to the notification information. This rotation of the display panel 270 makes it possible to express an interesting movement as if the agent device 200 is removing its mask and revealing its true form, thereby increasing the user's interest in the agent device 200. Furthermore, after the agent device 200 removes its mask, it is possible to give the user an interesting impression as if the notification information is displayed on the agent device 200's true face. This makes the user feel that it is a living thing and increases the user's interest in the notification information being displayed. In addition, the display position of the facial image and the display position of the notification information are in the same position from the user's perspective. Therefore, it is possible to display the notification information in a position that is easy for the user to see after looking at the facial image. This makes it possible to notify the user of the notification information more appropriately.

[0093] In Figure 7, an example is shown in which the display panel 270 is rotated vertically using the rotation support portion 273, which extends horizontally, as the pivot axis, but the system is not limited to this. For example, the display panel 270 may be rotated horizontally using the rotation support portion, which extends vertically, as the pivot axis.

[0094] [Example of providing notification information using a display unit that can be moved by sliding] Figure 8 shows an example of the transition when providing notification information using a sliding display unit 300 and a fixed display unit 310. The display unit 300 is a plate-shaped housing equipped with a display screen 301, and the display unit 310 is a plate-shaped housing equipped with a display screen 311.

[0095] The display unit 300 is installed in front of the display unit 310 so as to be movable in the vertical direction. Specifically, the display unit 300 is mounted on rails 305 and 306 fixed to the front of the body unit 202 so as to be slidable in a direction substantially parallel to the display surface of the display unit 310. The sliding mechanism realized by the rails 305 and 306 can employ known technologies.

[0096] Furthermore, the size of the display screen 301 of the display unit 300 and the size of the display screen 311 of the display unit 310 may be the same (or nearly the same), or they may be different sizes. For example, the size of the display screen 301 of the display unit 300 can be such that it covers at least a portion of the display screen 311 of the display unit 310.

[0097] Furthermore, the display unit 300 is controlled to transition between a first state (see Figure 8(A)) that displays a face image and a second state (see Figure 8(C)) that displays notification information. As shown in Figure 8(A), in the first state, the display unit 300 is positioned in front of the display unit 310 so as to cover the display screen 311 of the display unit 310. Also, as shown in Figure 8(C), in the second state, the display unit 300 from the first state is slid in a direction approximately parallel to the display screen 311 of the display unit 310 so that the user can view the display screen 301 of the display unit 300 and the display screen 311 of the display unit 310.

[0098] Furthermore, the display unit 300 is directly or indirectly connected to the drive unit 230 (see Figure 3), and can be moved vertically (in the direction of arrow A30 (see Figure 8(B))) along the rail sections 305 and 306 by the drive of the drive unit 230.

[0099] Similar to Figure 4, the display switching determination unit 121 determines whether or not notification information has been generated, and if notification information has been generated, it outputs that fact to the agent control unit 122.

[0100] Next, if the display switching determination unit 121 determines that notification information has been generated, the agent control unit 122 executes control to display the notification information on the display unit 310. Specifically, the agent control unit 122 executes a switching effect using the display unit 300, and after clearing the display of the face image on the display unit 300, it displays the notification information on the display unit 310.

[0101] For example, when notification information is generated, as shown in Figures 8(A) and 8(B), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect, before displaying the notification information. Specifically, as shown in Figure 8(A), the agent control unit 122 executes the switching effect by controlling the display state of the display unit 300 so that the face image on the display unit 300 shows a surprised facial expression (see Figure 2(D)).

[0102] Next, as shown in Figure 8(B), the agent control unit 122 erases the face image on the display unit 300 and moves the display unit 300 downward (in the direction of arrow A31) along the rails 305 and 306. During this movement, the face image may be displayed, and it may be erased when this movement is completed. In addition, before or during this movement, audio information to give the user a sense of realism may be output from the sound output unit 220.

[0103] Next, as shown in Figure 8(C), the agent control unit 122 causes the display unit 310 to display a notification image 410 as notification information. In this way, when notification information is generated, the display unit 300 is slid downward (in the direction of arrow A31) along the rails 305 and 306 to perform a transition from the first state to the second state as a switching performance, and after the display of the face image on the display unit 300 is erased, the notification information (notification image 410) is displayed on the display unit 310.

[0104] In this way, by sliding the display unit 300 vertically along the rail sections 305 and 306, it is possible to give the user the impression that the display screen 301 of the display unit 300 is swiping over the display screen 311 of the display unit 310.

[0105] Thus, when displaying notification information, a surprised facial expression image is displayed on the display unit 300 and the display unit 300 is moved when switching from the facial image to the notification information. This movement of the display unit 300 removes the filter that was hiding the face of the agent device 200, making it possible to express an interesting movement that reveals its true form, thereby increasing the user's interest in the agent device 200. Furthermore, after the filter that was hiding the face of the agent device 200 is removed, it is possible to give the user an interesting impression that the notification information is displayed on the true face of the agent device 200. This makes the user feel that it is a living thing and increases the user's interest in the notification information being displayed. In addition, the display position of the facial image and the display position of the notification information are in the same position from the user's perspective. Therefore, it is possible to display the notification information in a position that is easy for the user to see after looking at the facial image. This makes it possible to notify the user of the notification information more appropriately.

[0106] In the example shown in Figure 8, the display unit 300 in the second state is positioned in front of the body unit 202, and from the user's perspective, a part of the body unit 202 is covered by the display unit 300, thus obscuring a portion of the body unit 202. However, when the display unit 300 is in the second state, the display unit 300 may be housed inside the body unit 202 so that the body unit 202 is not obscured from the user's perspective. Also, when the display unit 300 in the second state is positioned in front of the body unit 202, the display unit 300 may display the same image (or another image related to the agent device 200) as the part of the body unit 202 that is covered by the display unit 300 from the user's perspective. As a result, even when notification information is displayed on the face unit 201, the user can recognize the body unit 202, making it possible to give the user a sense of lifelikeness towards the agent device 200.

[0107] [An example of providing notification information by rotating a rectangular display device vertically.] Figure 9 shows an example of the transition when notification information is provided using two display units 324 and 325. In Figure 9, an example is shown in which the agent device 200 is equipped with a housing 320 (face unit 201) that can rotate in the vertical direction. Figure 9(D) shows a perspective view of the external appearance of the housing 320 (face unit 201) that can rotate in the vertical direction. As shown in Figure 9(D), a display unit 324 (see Figure 9(A)) is installed on one side of the housing 320, and a display unit 325 (see Figure 9(C)) is installed on the other side of the housing 320 adjacent to the side of the display unit 324. The display unit 324 mainly displays a face image, and the display unit 325 mainly displays notification information.

[0108] Note that the example shown in Figure 9 is a modified version of Figure 6, and differs in the shape of the housing 320, the positional relationship of the two display units within the housing 320, etc.

[0109] Specifically, the housing 320 is installed inside the frame portion 321, which is shaped like a square when viewed from the front. The left and right ends of the housing 320 and the inner surface of the frame portion 321 are connected by rotating support portions 322 and 323. The positions where the rotating support portions 322 and 323 are installed correspond to the ears of the face portion 201, so the rotating support portions 322 and 323 may be configured as the ear portions of the face portion 201. The rotating support portions 322 and 323 are connected to the drive unit 230 (see Figure 3), and the rotational operation of the drive unit 230 rotates the housing 320 vertically (in the direction of arrow A40) using the rotating support portions 322 and 323 as pivot points. In this way, the housing 320 can be rotated vertically inside the frame portion 321. The rotation range of the housing 320 can be set to, for example, a range of 180 to 360 degrees. For example, the two display units 324 and 325 can be switched by rotating them approximately 90 degrees vertically. Normally, as shown in Figure 9(A), the display unit 324 is set to face forward, and various facial images are displayed on the display unit 324, similar to the examples shown in Figures 2(A) to (C).

[0110] Similar to Figure 4, the display switching determination unit 121 determines whether or not notification information has been generated, and if notification information has been generated, it outputs that fact to the agent control unit 122.

[0111] Next, if the display switching determination unit 121 determines that notification information has been generated, the agent control unit 122 executes control to display the notification information on the display unit 325. Specifically, the agent control unit 122 executes a switching effect using the housing 320, and after erasing the face image on the display unit 324, it displays the notification information on the display unit 325.

[0112] For example, when notification information is generated, as shown in Figures 9(A) and 9(B), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect, before displaying the notification information. Specifically, as shown in Figure 9(A), the agent control unit 122 executes the switching effect by controlling the display state of the display unit 324 so that the face image on the display unit 300 shows a surprised facial expression (see Figure 2(D)).

[0113] Next, as shown in Figure 9(B), the agent control unit 122 erases the face image on the display unit 324 and rotates the housing 320 downward (in the direction of arrow A41) using the rotation support units 322 and 323 as pivot axes. During this rotation, the face image on the display unit 324 may be displayed, and the face image on the display unit 324 may be erased when this rotation is completed. In addition, before or during this rotation, audio information to give the user a sense of life may be output from the sound output unit 220.

[0114] Next, as shown in Figure 9(C), the agent control unit 122 displays the notification image 410 as notification information on the display unit 325, which is on the user side of the vehicle C1. In this way, when notification information is generated, a switching animation is performed using the housing 320 (display units 324 and 325), and after erasing the face image on the display unit 324, the notification information (notification image 410) is displayed on the display unit 325.

[0115] In this way, when displaying notification information, when switching from the face image to the notification information, a face image with a surprised expression is displayed on the display unit 324, and the housing 320 is moved. This rotation of the housing 320 makes it possible to represent the movement of the agent device 200's face looking downwards, thereby increasing the user's attention to the agent device 200. Furthermore, after the agent device 200's face looks downwards, it is possible to give the user the impression that the notification information is being displayed on the agent device 200's head or back. This makes the user feel that it is a living thing, and increases the user's attention to the notification information being displayed. In addition, the display position of the face image and the display position of the notification information are approximately the same as seen by the user. Furthermore, it is possible to make the display size of the notification information larger than the display size of the face image. Therefore, it is possible to display the notification information in a large size in a position that is easy for the user who was looking at the face image to see. In this way, it is possible to notify the user of the notification information more appropriately.

[0116] Furthermore, as shown in Figure 9(D), the face image is displayed on the elongated horizontal display unit 324, and the notification information is displayed on the vertically wider display unit 325. This makes it possible for the user to visually and intuitively understand that more important information than the face image is displayed on the display unit 325. In addition, by rotating the display unit 324 (which displays the face image) and displaying the notification information on the relatively large display unit 325 adjacent to it above, it is possible to give the user the interesting impression that the notification information is displayed on the back (or abdomen in front of the face) of the agent device 200 near its face. These features give the user a sense of realism and can increase the user's interest in the notification information.

[0117] In the example shown in Figure 9, a face image is displayed on the horizontally elongated display unit 324, and notification information is displayed on the vertically wide display unit 325. However, the system is not limited to this. For example, notification information may be displayed on the horizontally elongated display unit 324, and a face image may be displayed on the vertically wide display unit 325, or the face image and notification information may be displayed on two other adjacent surfaces.

[0118] Furthermore, while Figure 9 shows an example in which two display units are provided on two adjacent faces of a rectangular prism-shaped housing 320, two display units provided on two adjacent faces of a cubic-shaped housing may also be used.

[0119] Furthermore, Figure 9 shows an example in which the rectangular prism-shaped housing 320 is rotated vertically, but the system is not limited to this. For example, the rectangular prism-shaped housing, which is set to produce a vertically elongated face image, may be rotated horizontally.

[0120] [Example of providing notification information using a single display unit] Figures 4 to 9 show examples of displaying notification information by switching the screen content of multiple display units. Here, by switching the screen content displayed on a single display unit, it is also possible to make the user perceive the face image display screen and the notification information display screen as different screens. Therefore, Figure 10 shows an example of displaying notification information by switching the screen content of a single display unit.

[0121] Figure 10 shows an example of a transition when providing notification information using a single display unit 210. In Figure 10, an example is shown in which the display unit 210 has a display function that enables it to display each image with different display characteristics. Display characteristics refer to the characteristics of displaying an image, such as image quality, color, resolution, and whether or not it is stereoscopic. For example, different display characteristics can be achieved by switching between stereoscopic images and planar images. Also, for example, different display characteristics can be achieved by switching between high-resolution and low-resolution images. Furthermore, for example, different display characteristics can be achieved by switching between color images and black and white images.

[0122] For example, by displaying a face image as a three-dimensional image on the display unit 241 and displaying notification information as a two-dimensional image on the display unit 241, it becomes possible to display face images and notification information with different display characteristics. Alternatively, by displaying a face image as a high-resolution image on the display unit 241 and displaying notification information as a low-resolution image on the display unit 241, it becomes possible to display face images and notification information with different display characteristics. In this case, the high-resolution face image is, for example, a colorful and beautiful image, and the low-resolution face image is, for example, at least one of a black and white image, a pixel image, or a mosaic image. Alternatively, by displaying a face image as a color image on the display unit 241 and displaying notification information as a black and white image on the display unit 241, it becomes possible to display face images and notification information with different display characteristics.

[0123] Similar to Figure 4, the display switching determination unit 121 determines whether or not notification information has been generated, and if notification information has been generated, it outputs that fact to the agent control unit 122.

[0124] Next, if the display switching determination unit 121 determines that notification information has been generated, the agent control unit 122 executes control to display the notification information on the display unit 210. Specifically, the agent control unit 122 executes a switching effect using the display unit 210, and after erasing the face image on the display unit 210, it displays the notification information on the display unit 210.

[0125] For example, when notification information is generated, as shown in Figure 10(A), the agent control unit 122 executes a switching effect (switching effect) to give the user a sense of realism as a switching effect, before displaying the notification information. Specifically, as shown in Figure 10(A), the agent control unit 122 executes the switching effect by controlling the display state of the display unit 210 so that the face image on the display unit 210 shows a surprised facial expression (see Figure 2(D)).

[0126] Next, as shown in Figure 10(B), the agent control unit 122 erases the face image on the display unit 210 and displays the notification image 400 as notification information on the display unit 210. When transitioning the display from the face image to the notification information, a switching effect may be implemented to make the user feel that the display screen for the face image and the display screen for the notification information are different screens. For example, as described above, it is possible to implement a switching effect such as switching from a 3D image to a 2D image, switching from high resolution to low resolution, or switching from a color image to a black and white image. Alternatively, the switching effect may be implemented by changing the color scheme or changing the resolution. In this way, the display unit 210 can realize different display characteristics (first display characteristics, second display characteristics).

[0127] Furthermore, when transitioning the display from a face image to notification information, a transition effect may be executed using software-based image processing. For example, an effect may be executed in which the face image gradually disappears, followed by an effect in which the notification information gradually appears. By executing such an effect of the face image gradually disappearing, it is possible to give the user a sense of anticipation that some kind of information will be displayed, and to direct the user's attention to the display unit 210. In addition, before or during this transition, audio information may be output from the sound output unit 220 to give the user a sense of realism.

[0128] Thus, when displaying notification information, when switching from the face image to the notification information, the display unit 210 displays a face image with a surprised expression, and displays the notification information with different display characteristics. This makes the user feel that it is a living thing and increases the user's interest in the notification information. In addition, the difference in display characteristics allows the user to clearly distinguish between the face image and the notification information. Therefore, it becomes possible to notify the user of the notification information more appropriately.

[0129] In Figure 10, an example is shown in which notification information is displayed with different display characteristics from the face image by switching the screen content of one display unit 210. However, in embodiments in which notification information is displayed by switching the screen content of multiple display units (for example, Figures 4 to 9), each of the multiple display units may be configured with different display characteristics.

[0130] For example, the first display unit can be an LCD (Liquid Crystal Display) and the second display unit can be an OLED (Organic Electro Luminescence Diode). Alternatively, the first display unit can be a glossy screen and the second display unit a matte screen. Furthermore, the first display unit can be a flat screen and the second display unit a curved screen. Also, the first display unit can be a stereoscopic screen capable of displaying three-dimensional images, and the second display unit can be a screen capable of displaying flat images. Additionally, the sizes of the first and second display units can be different. For example, if the first display unit is an LCD and the second display unit is an OLED, the visual image may be the same, but the brightness will differ, allowing for different effects to be displayed. In these cases, when displaying notification information on the second display unit, a switching effect must be performed on at least one of the first and second display units. For example, the switching effect can be achieved by moving the casings of the first and second display units, or by displaying a surprised expression on the first display unit.

[0131] [Example of information processing device operation] Figure 11 is a flowchart showing an example of the notification information output process in the information processing device 110. This notification information output process is executed by the control unit 120 based on a program stored in the memory unit 130. This notification information output process is executed continuously at each control cycle. This notification information output process will be explained with reference to Figures 1 to 10 as appropriate.

[0132] In step S501, the display switching determination unit 121 performs an analysis process to analyze the input information. This input information includes information input to the information processing device 110 from the camera 101, position information acquisition sensor 102, voice input unit 103, sensors 104, etc., and information acquired from the outside via the communication unit 140 and input to the information processing device 110.

[0133] In step S502, the display switching determination unit 121 determines, based on the results of the input information analysis, whether or not there is information (notification information) that should be notified to the occupants of vehicle C1.

[0134] For example, the battery level of vehicle C1 can be detected based on the battery sensor. If it is determined that the battery level falls below a threshold, it is determined that there is notification information available to alert the vehicle that the battery level is low.

[0135] For example, based on a seat belt sensor that detects whether a seat belt is being worn and a seat occupancy sensor (or seat sensor) that detects the presence or absence of an occupant in each seat of vehicle C1, it is possible to determine whether an occupant in each seat of vehicle C1 is wearing a seat belt. If it is determined that an occupant is seated but not wearing a seat belt, it is determined that there is notification information available for that occupant.

[0136] Furthermore, for example, a partially closed door on vehicle C1 can be detected based on a partially closed door sensor that detects each door of vehicle C1 being partially closed. If a partially closed door is detected, it is determined that there is notification information for the partially closed door.

[0137] Furthermore, for example, based on information obtained from an external device, such as an information provision server, via the communication unit 140, and the current location of vehicle C1 obtained by the location information acquisition sensor 102, it is possible to determine whether there are any landmarks, commercial facilities, etc., around vehicle C1. For example, if it is determined that there is a commercial facility, such as a coffee shop, around vehicle C1, it is determined that there is information available as notification information about that coffee shop. Also, for example, if it is determined that there is a tourist facility, such as ABC Castle, around vehicle C1, it is determined that there is information available as notification information about ABC Castle.

[0138] In step S503, the display switching determination unit 121 determines whether or not information that should be notified to the occupants of vehicle C1 has occurred. If information that should be notified to the occupants of vehicle C1 has occurred, the process proceeds to step S504. On the other hand, if no information that should be notified to the occupants of vehicle C1 has occurred, the notification information output processing operation is terminated.

[0139] In step S504, the agent control unit 122 performs a switching animation from the face image to the notification information. For example, as shown in Figures 4(A), 5(A)(B), 6(A)(B), 7(A) to (C), 8(A)(B), 9(A)(B), and 10(A), a switching animation is performed using each part of the agent device 200. In this case, audio information related to the switching animation may be output from the sound output unit 220. In Figures 5 to 9, an example is shown in which both the surprised facial expression and the movement of the face unit 201 are performed as the switching animation, but only the movement of the face unit 201 may be performed as the switching animation.

[0140] In step S505, the agent control unit 122 causes the agent device 200 to perform output processing to output notification information. For example, as shown in Figures 4(B), 5(C), 6(C), 7(D), 8(C), 9(C), and 10(B), the notification images 400 and 410 are displayed on the display unit 210. In this case, audio information related to the notification information may be output from the sound output unit 220.

[0141] In step S506, the agent control unit 122 determines whether the notification information output in step S505 has reached its end. If it has reached its end, the process proceeds to step S507. If it has not reached its end, the process returns to step S505.

[0142] Here, the timing for the termination of the notification information can be when a predetermined action is taken by the occupant of vehicle C1 corresponding to the notification information, when a predetermined time has elapsed since the display of the notification information, or when the current location of vehicle C1 is outside the display area of ​​the notification information.

[0143] For example, if the notification information output in step S505 is related to the battery level, the predetermined action will be either increasing the battery level or charging the battery. An increase in the battery level can be determined based on the information detected by the battery sensor. Battery charging can be determined based on the communication between the charging equipment and vehicle C1 via the charging cable when the charging cable is connected to vehicle C1.

[0144] Furthermore, for example, if the notification information output in step S505 is notification information regarding seat belt fastening, the predetermined action is for the occupant seated in the seat that was the subject of the notification to fasten their seat belt. Whether or not this seat belt fastening action has been performed can be determined based on the information detected by the seat belt sensor.

[0145] Furthermore, for example, if the notification information output in step S505 concerns a partially closed door on vehicle C1, the predetermined action is to close the door that was the subject of the notification. Whether or not this door closing action is performed can be determined based on the detection information from the door sensor.

[0146] Furthermore, for example, if the notification information output in step S505 concerns commercial facilities, tourist facilities, etc., located around vehicle C1, then moving vehicle C1 to the location of the notification becomes a predetermined action. Whether or not to move vehicle C1 to the location of the notification can be determined based on the location information of vehicle C1 acquired by the location information acquisition sensor 102. In this case, the action of the occupant making a voice statement about the location of the notification also becomes a predetermined action. For example, if the notification information concerns commercial facilities located around vehicle C1, such as XYZ Coffee Shop, then making a voice statement about XYZ Coffee Shop, such as "XYZ Coffee is nice" or "XYZ Coffee looks delicious," can be a predetermined action. Also, for example, if the notification information concerns tourist facilities located around vehicle C1, such as ABC Castle, then making a voice statement about ABC Castle, such as "ABC Castle is nice" or "I want to see ABC Castle," can be a predetermined action. Whether or not these voice-related actions are performed can be determined based on the degree of match (or similarity) between the voice information acquired by the voice input unit 103 and predetermined keywords stored in the storage unit 130. For determining the degree of match (or similarity) of the voices, known voice recognition technologies can be employed.

[0147] Furthermore, for example, if the notification information output in step S505 concerns commercial facilities, tourist facilities, etc., located around vehicle C1, the notification information may be terminated when a predetermined time has elapsed since the display of the notification information. Alternatively, the notification information may be terminated when the current location of vehicle C1 moves outside the display area of ​​the notification information. For example, if the notification information concerns commercial facilities located around vehicle C1, such as XYZ Coffee Shop, the notification information can be terminated when vehicle C1 moves outside a predetermined area based on XYZ Coffee Shop, for example, outside a radius of 2 km centered on XYZ Coffee Shop.

[0148] In step S507, the agent control unit 122 executes a notification information termination process to end the display of the notification information output in step S505, and after this termination process, executes a display process to display the face image. For example, if the notification information concerns a commercial facility or a tourist facility, it executes a deletion process to erase the notification information and then executes a display process to display the face image. In the examples shown in Figures 5 to 9, the display unit that displays the face image moves when the notification information is displayed. Therefore, in the face image display process in step S507, the display unit that displays the face image is returned to its original position before the face image is displayed.

[0149] [Example of changing the timing of switching to notification information based on the importance of the notification information] Figure 12 is a flowchart showing an example of the notification information output process in the information processing device 110. This notification information output process is executed by the control unit 120 based on a program stored in the storage unit 130. This notification information output process is executed continuously at each control cycle. This notification information output process will be explained with reference to Figures 1 to 11 as appropriate.

[0150] This output control process is a modified version of the notification information output process shown in Figure 11. Specifically, it differs from the notification information output process shown in Figure 11 in that steps S511 and S512 have been added. Other aspects are common to the notification information output process shown in Figure 11, and therefore, the same reference numerals as in Figure 11 are used, and their explanations are omitted.

[0151] In step S511, the display switching determination unit 121 determines whether the importance of the notification information to be output is high or low. The importance of the notification information can be determined, for example, based on the urgency of the notification information. For example, information regarding the driving of vehicle C1 and the safety of the occupants of vehicle C1 is assumed to be of high urgency. Therefore, the importance of the notification information regarding the driving of vehicle C1 and the safety of the occupants of vehicle C1 is set to high. For example, the importance of the notification information regarding seat belt use, ajar door status, battery level, etc. is set to higher than the standard. On the other hand, the provision of information about the surroundings of vehicle C1, tourist information, gourmet information, etc. is assumed to be of low urgency. Therefore, the importance of the notification information regarding information about the surroundings of vehicle C1, tourist information, gourmet information, etc. is set to lower than the standard. For example, the importance of the notification information regarding facilities around vehicle C1 (e.g., restaurants, commercial facilities) is set to low. The level of importance of the notification information is stored in the notification information DB 132 (see Figure 3).

[0152] If the importance of the broadcast information to be output is high, the process proceeds to step S504. On the other hand, if the importance of the broadcast information to be output is low, the process proceeds to step S512.

[0153] In step S512, the agent control unit 122 executes an animation to smoothly erase the face image. Here, the animation to smoothly erase the face image is achieved, for example, by smoothly displaying the gestures of the face image. For example, by expressing the surprised gesture of the face image shown in Figure 4(A) with relatively slow movements and then erasing the face image, it is possible to smoothly erase the face image. In this way, in the case of notification information of low importance, there is little need to notify the user urgently, so it is possible to enhance the lifelike appearance of the agent device 200 and then proceed to the output processing of the notification information. That is, it is possible to smoothly erase the face image and then display the notification information.

[0154] In the examples shown in Figures 5 to 9, the display unit that shows the face image is moved when displaying notification information. In such cases, it is conceivable that the face image may become difficult to see due to the movement of the display unit. For this reason, in the examples shown in Figures 5 to 9, the process of smoothly erasing the face image when switching notification information may be omitted.

[0155] Furthermore, if the importance of the information to be announced is low, the face image and the information to be announced may be displayed simultaneously for a short period. Alternatively, if the importance of the information to be announced is low, the face image may be gradually erased and the information to be gradually revealed.

[0156] [Example of delaying the timing of switching to notification information based on whether or not facial images are enhanced] Figure 13 is a flowchart illustrating an example of the notification information output process in the information processing device 110. This notification information output process is executed by the control unit 120 based on a program stored in the memory unit 130. This notification information output process is executed continuously at each control cycle. This notification information output process will be explained with reference to Figures 1 to 12 as appropriate.

[0157] This output control process is a modified version of the notification information output process shown in Figure 12. Specifically, it differs from the notification information output process shown in Figure 12 in that step S512 is omitted and steps S521 and S522 are added. Other aspects are common to the notification information output process shown in Figure 12, and therefore, the same reference numerals as in Figure 12 are used, and their explanations are omitted.

[0158] In step S521, the display switching determination unit 121 determines whether or not the face image is being displayed. For example, if a predetermined interaction (e.g., a conversation) is taking place between the agent device 200 and the occupant of vehicle C1, it is determined that the face image is being displayed. Also, for example, if the agent device 200 is performing a display to demonstrate its autonomy (e.g., sleeping, tired), it is determined that the face image is being displayed. For example, if the agent device 200 is displaying sleepy behavior while simultaneously displaying advertisements for a ramen shop near vehicle C1 as notification information, it may cause discomfort to the user. In this case, the user may lose trust in or affection for the agent device 200 as a living being. Therefore, in order to emphasize the autonomy of the agent device 200 and to create a sense of unity in the expression of autonomy, when the agent device 200 is performing a display to demonstrate its autonomy, it is prevented from outputting low-priority notification information. This makes it possible to increase trust in and affection for the agent device 200 as a living being, and to increase interest in the notification information. If a facial image is being processed, the process proceeds to step S522. On the other hand, if a facial image is not being processed, the process proceeds to step S504.

[0159] In step S522, the display switching determination unit 121 determines whether or not the notification information determined in step S503 is currently being generated. That is, it is possible that the output of notification information may become unnecessary for some reason. If the output of notification information becomes unnecessary in this way, it is determined that the notification information is not currently being generated. If the notification information is currently being generated, the process returns to step S521. On the other hand, if the notification information is not currently being generated, the notification information output process is terminated.

[0160] For example, if the notification information concerns an advertisement for a commercial facility, outputting that notification information becomes unnecessary when vehicle C1 has moved far away from the commercial facility. Also, for example, if the notification information concerns an advertisement for a commercial facility, outputting that notification information may be unnecessary when a predetermined time (for example, about 10 minutes) has elapsed since the notification information was generated.

[0161] Here, it is important to prevent the switching animations of the agent device 200 from interfering with the driver while driving. For example, if the driver is in the driver's seat and driving, the driver may be aware of the switching animations of the agent device 200, but may not be looking in the direction of the agent device 200. Therefore, it is preferable to perform the switching animations of the agent device 200 while the vehicle C1 is stopped. However, if the target is an occupant other than the driver in the driver's seat, it may be performed while the vehicle C1 is moving.

[0162] [Examples of agent devices that can be installed outside of vehicles] The above describes an example of an agent device 200 installed in vehicle C1. However, this embodiment can also be applied to agent devices that are removable from vehicle C1, agent devices that can be installed outside of a vehicle, etc. For example, a user who possesses a portable agent device may place the agent device on the dashboard 2 of vehicle C1 when getting into vehicle C1, and then carry the agent device with them when getting out of vehicle C1. It is also conceivable that a user may use the agent device within their home.

[0163] For example, within a home, it is conceivable to output notification information regarding various appliances installed in the home. For instance, it is envisioned that notifications would be issued when the bath is ready or when cooking is finished. It is also envisioned that notifications would be issued if the front door or windows are open. Furthermore, it is envisioned that notifications would be issued if the gas stove is left on. In these cases, it is envisioned that notification information regarding the bath, cooking appliances, doors, windows, etc., would be provided. The level of importance can be set based on criteria such as safety and urgency. For example, for items with high urgency, the level of importance can be set higher than the standard, and for items with low urgency, the level of importance can be set lower than the standard. For example, items with high urgency and those that require safety maintenance include the gas stove (to prevent forgetting to turn off the flame), the front door, and windows (to notify when they are open). On the other hand, items with low urgency include, for example, the bath (to notify when the bath is ready) and cooking appliances (to notify when cooking is finished).

[0164] [Example of representing agent devices in 2D] The above explanation used a three-dimensional agent device 200 as an example. However, if it is possible to express the lifelike nature of the agent device, this embodiment may also be applied by displaying the agent image as a two-dimensional image. In this case, both the face and body parts are displayed as two-dimensional images.

[0165] [Examples of the effects of this embodiment] Here, when switching the display target of the agent device 200 from a face image to notification information, simply switching from a face image to notification information may result in the agent device 200 lacking a sense of lifelikeness because the face image and notification information are not clearly separated. For example, if notification information is mechanically displayed on the face unit 201 that is intended to recognize a face image, the user may perceive the face unit 201 as a machine. Also, if notification information is displayed immediately after the display of a face image, the user may perceive the face image display area as a mechanical display screen. In these cases, the user's attachment to the agent device 200 may decrease, and their interest in the notification information displayed on the agent device 200 may decline. In this case, it may become difficult to appropriately convey the notification information to the user.

[0166] Furthermore, when switching the display target of the agent device 200 from a face image to notification information, displaying the face image and notification information together may confuse the user. Therefore, in this embodiment, instead of displaying the face image and notification information together, a switching animation is performed at the timing from the display of the face image to the display of the notification information, and the face image and notification information are displayed separately. In other words, the switching animation is performed so that the face image and notification information are separated between the display of the face image and the display of the notification information.

[0167] In this way, a transition animation is performed after (or while) the face image is displayed, and notification information is displayed after this transition animation. This allows the user to prepare for the notification information displayed after the transition animation. In other words, by clearly distinguishing between the face image display screen, which evokes emotional and sentimental feelings, and the notification information display screen, which conveys predetermined information, the user can predict to some extent what will be displayed after the transition animation. For example, if a transition animation is performed after the face image is displayed, the user can predict that important notification information will be displayed after this transition animation. In this way, by displaying notification information after the transition animation, it becomes possible to predict the notification information, and the confusion caused to the user can be mitigated. This makes it possible to convey notification information appropriately.

[0168] Here, we consider the case where the agent device 200 outputs notification information without performing a switching animation, and the case where it outputs notification information after performing a switching animation. In this case, it is assumed that users will perceive the agent device 200 as more lifelike when the notification information is output after performing a switching animation than when it is output without performing a switching animation. Therefore, in this embodiment, the notification information is output after performing a switching animation. This makes it possible to make the user perceive the agent device 200 as lifelike and to convey the notification information appropriately.

[0169] Furthermore, in this embodiment, by ensuring autonomy while performing actions that make the agent device 200 appear intelligent, it becomes possible to efficiently convey notification information while maintaining a lifelike appearance.

[0170] Furthermore, in this embodiment, when highly urgent notification information, such as highly important notification information, is generated, it is possible to quickly perform a switching animation to exclusively display the face image and the notification information before displaying the notification information. In this way, by quickly executing the switching animation, at least in emergencies, it is possible to make the user feel that the agent device 200 is lifelike, and to quickly convey the notification information.

[0171] In the above example, the notification information output processing and other operations are shown to be performed by the agent device 200 and the information processing device 110 (or information processing system 100). However, all or part of these operations may be performed by other devices. In this case, the information processing system is composed of devices that perform part of these operations. For example, at least part of each operation can be performed using various information processing devices and electronic devices such as in-vehicle devices, user-accessible devices (e.g., smartphones, tablet terminals, personal computers, car navigation systems, IVI), and servers that can be connected via a predetermined network such as the Internet.

[0172] Furthermore, some (or all) of the information processing system capable of executing the functions of the information processing device 110 (or information processing system 100) may be provided by an application that can be provided via a predetermined network such as the Internet. This application may be, for example, SaaS (Software as a Service).

[0173] [Example configuration and effects of this embodiment] The information processing method according to this embodiment is an information processing method that provides notification information to a user using an agent device 200 having a display unit 210 (an example of a display device) that displays a face image. This information processing method includes control processing (steps S503 to S507) that controls the display mode of the display unit 210. In the control processing, when the notification information is to be displayed on the display unit 210, a switching effect using the display unit 210 is performed, and the notification information is displayed after the display of the face image is erased. Furthermore, the program according to this embodiment is a program that causes a computer to execute each of these processes. In other words, the program according to this embodiment is a program that causes a computer to realize each of the functions that the information processing device 110 can execute.

[0174] With this configuration, when switching from a facial image to notification information, the display unit 210 can be used to perform a switching animation, such as displaying a surprised expression, which makes the user feel that it is a living thing and increases the user's interest in the notification information. This makes it possible to notify the user of the notification information more appropriately.

[0175] In the information processing method according to this embodiment, the display unit 210 includes a first display unit (display units 241, 252, 264, 271 and 281, 300, 324) that displays a face image, and a second display unit (display units 242, 253, 265, 272 and 282, 310, 325) that displays notification information. In the control processing (steps S504, S505), when the notification information is to be displayed on the second display unit, a switching effect is performed on at least one of the first and second display units, and the display of the face image on the first display unit is erased before the notification information is displayed on the second display unit. For example, as a switching effect using the first display unit, a surprised expression can be displayed, or as a switching effect using the first and second display units, the housing equipped with the first and second display units can be rotated. Furthermore, for example, by displaying an image with different display characteristics in at least one of the first and second display units, it is also possible to perform a switching effect using the first or second display unit.

[0176] This configuration allows for a transition animation when switching from a facial image to notification information, creating a sense of realism for the user and increasing their interest in the notification information. Furthermore, the difference in the display units for the facial image and the notification information allows the user to clearly distinguish between them. Therefore, it becomes possible to deliver notification information to the user more appropriately.

[0177] In the information processing method according to this embodiment, the control process (step S504) executes a switching effect that moves the display unit 210. For example, as shown in Figures 5(A)(B), 6(A)(B), 7(A) to (C), 8(A)(B), and 9(A)(B), a switching effect is executed that moves the display unit of the agent device 200.

[0178] With this configuration, when switching from a facial image to notification information, the display unit 210 is moved to create a sense of realism for the user, thereby increasing the user's interest in the notification information. As a result, notification information can be delivered to the user more appropriately.

[0179] In the information processing method according to this embodiment, the display unit 210 (an example of a display device) consists of a cubic or rectangular prism-shaped housing in which a first display unit (display units 252, 264, 324) for displaying a face image is provided on one face, and a second display unit (display units 253, 265, 325) for displaying notification information is provided on a different face. In the control processing (steps S504, S505), when displaying notification information on the second display unit, a rotation operation is performed as a switching effect to switch the face of the display device visible to the user from one face to the other face, and after erasing the display of the face image on the first display unit, the notification information is displayed on the second display unit.

[0180] With this configuration, when switching from a facial image to notification information, the rotation of the display unit 210 is performed as a switching effect, making the movement of the face appear more lifelike to the user and increasing the user's interest in the notification information. As a result, the notification information can be conveyed to the user more appropriately.

[0181] In the information processing method according to this embodiment, the display unit 210 (an example of a display device) consists of a flat plate-shaped housing on which a first display unit (display units 252, 264) is provided on one side and a second display unit (display units 253, 265) is provided on the other side opposite to the first side. In the control processing (step S504), a rotation operation is performed as a switching effect to rotate the first display unit and the second display unit by 180 degrees around a rotation axis. For example, as shown in Figures 5(A)(B) and 6(A)(B), a switching effect is performed to rotate the display unit of the agent device 200.

[0182] With this configuration, when switching from a face image to notification information, the rotation of the display unit 210 is performed as a switching effect, making it possible to give the user the impression that the agent's face is moving in a lifelike manner, such as turning around or looking downwards. This makes it possible to increase the user's interest in the notification information and to notify the user more appropriately.

[0183] In the information processing method according to this embodiment, the display unit 210 (an example of a display device) consists of a cubic or rectangular prism-shaped housing on which a display unit 324 (an example of a first display unit) is provided on one face and a display unit 325 (an example of a second display unit) is provided on the other face adjacent to the first face. In the control processing (step S504), a rotation operation is performed to rotate the display units 324 and 325 by 90 degrees around a rotation axis as a switching effect. For example, as shown in Figures 9(A) and 9(B), a switching effect is performed to rotate the display units 324 and 325 of the agent device 200.

[0184] This configuration gives the user the intriguing impression that notification information is displayed on the back (or abdomen) behind the face of the agent device 200. This gives the user a sense of realism and can increase their interest in the notification information.

[0185] In the information processing method according to this embodiment, the display unit 210 (an example of a display device) comprises a flat display panel 270 having a first display unit 271 on one side and a second display unit 272 on the opposite side of the first side, and a display unit 280 (an example of a third display unit), the display panel 270 is mounted on the display unit 280 so as to be rotatable with one end as the axis of rotation, and the display unit 210, by the rotational movement of the display panel 270, displays a first display area composed of the first display unit 271 and a part of the display unit 280. The system is controlled to be in one of two display states: a first state in which a face image is displayed in the area, or a second state in which notification information is displayed in a second display area composed of the second display unit 272 and a part of the display unit 280. In the control process (steps S504, S505), when notification information is to be displayed on the display unit 210, the display panel 270 is rotated to transition from the first state to the second state as a switching effect, and the display of the face image in the first display area is erased before the notification information is displayed in the second display area. For example, as shown in Figures 7(A) to (C), a switching effect is performed in which the display panel 270 of the agent device 200 is rotated.

[0186] With this configuration, the display panel 270 is rotated to execute a switching effect between the face image and the notification information. The unexpected movement of the display panel 270 makes the user feel that it is living, and increases the user's interest in the notification information. As a result, it becomes possible to notify the user of the notification information more appropriately.

[0187] In the information processing method according to this embodiment, the display unit (an example of a display device) comprises a display unit 300 (an example of a first display unit) that displays a face image and a display unit 310 (a second display unit) that displays notification information. The display unit 300 is installed to transition between a first state in front of the display unit 310, covering the display surface of the display unit 310, and a second state in which it slides in a direction substantially parallel to the display surface of the display unit 310, allowing the user to view both the display surface of the display unit 300 and the display surface of the display unit 310. In the control processing (steps S504, S505), when notification information is to be displayed on the display unit 310, a transition effect is performed by sliding the display unit 300 to transition from the first state to the second state, and after erasing the display of the face image on the display unit 300, the notification information is displayed on the display unit 310.

[0188] With this configuration, the display unit 300 is slid to perform a switching effect from the face image to the notification information. The unexpected movement of the display unit 300 makes the user feel that it is a living thing, and increases the user's interest in the notification information. As a result, it becomes possible to notify the user of the notification information more appropriately.

[0189] In the information processing method according to this embodiment, the display unit 210 (an example of a display device) is capable of displaying each image with different display characteristics by switching between at least one of three-dimensional images and planar images, high-resolution and low-resolution images, and color images and monochrome images. In the control processing (steps S504, S505), when displaying notification information on the display unit 210, a switching effect is performed to display a predetermined image with different display characteristics than when displaying the face image, and after erasing the display of the face image on the display unit 210, the notification information is displayed on the display unit 210.

[0190] This configuration allows the user to perceive the information as more lifelike by displaying it with different display characteristics when switching from a facial image to notification information, thereby increasing the user's interest in the notification. Furthermore, the difference in display characteristics allows the user to clearly distinguish between the facial image and the notification information. Therefore, it becomes possible to deliver notification information to the user more appropriately.

[0191] In the information processing method according to this embodiment, the display unit 210 (an example of a display device) includes a first display unit (display units 241, 252, 264, 271 and 281, 300, 324) that displays a face image, and a second display unit (display units 242, 253, 265, 272 and 282, 310, 325) that displays notification information. The first and second display units have at least one different display characteristic among three-dimensional images and two-dimensional images, high-resolution and low-resolution images, and color images and black-and-white images. In the control processing (steps S504, S505), when notification information is to be displayed on the second display unit, a switching effect is performed on at least one of the first and second display units, and the display of the face image on the first display unit is erased before the notification information is displayed on the second display unit.

[0192] This configuration allows the user to perceive a sense of realism and increase their interest in the notification information by displaying the notification information on a second display unit with different display characteristics when switching from the face image to the notification information. Furthermore, the difference in display characteristics allows the user to clearly distinguish between the face image and the notification information. Therefore, it becomes possible to deliver notification information to the user more appropriately.

[0193] In the information processing method according to this embodiment, the control process (steps S504, S511, S512) determines the importance of the notification information based on its urgency, and changes the timing of switching from displaying a face image to displaying the notification information based on the importance of the notification information.

[0194] This configuration allows for the rapid output of high-priority notification information and the timely output of lower-priority notification information. Therefore, it becomes possible to more effectively communicate notification information to users.

[0195] In the information processing method according to this embodiment, in the control processing (steps S504, S511, S512, S521, S522), if the importance of the notification information is higher than a standard, the display switches from the face image display to the notification information display immediately after the notification information is generated. If the importance of the notification information is lower than a standard, and the face image displayed on the display unit 210 (an example of a display device) is performing a predetermined effect, the display switches from the face image display to the notification information display after the predetermined effect is completed. If the face image displayed on the display unit 210 is not performing a predetermined effect, the display switches from the face image display to the notification information display after the notification information is generated.

[0196] This configuration allows for the rapid output of highly important notification information, while less important information can be output at an appropriate time based on the facial image's presentation. Therefore, it becomes possible to more effectively communicate notification information to the user.

[0197] The information processing device 110 is an information processing device that provides notification information to a user using an agent device 200 having a display unit 210 (an example of a display device) that displays a face image. The information processing device 110 includes a control unit 120 that controls the display mode of the display unit 210. When the control unit 120 displays notification information on the display unit 210, it performs a switching effect using the display unit 210, and then displays the notification information on the display unit 210 after erasing the display of the face image. Alternatively, it may be an information processing system capable of executing each process realized by the information processing device 110.

[0198] With this configuration, when switching from a facial image to notification information, the display unit 210 can be used to perform a switching animation, such as displaying a surprised expression, which makes the user feel that it is a living thing and increases the user's interest in the notification information. This makes it possible to notify the user of the notification information more appropriately.

[0199] The processing steps shown in this embodiment are merely examples of how to implement this embodiment. The order of some of the processing steps may be changed, some of the processing steps may be omitted, or other processing steps may be added, as long as the embodiment is feasible.

[0200] Furthermore, each process shown in this embodiment is executed based on a program that causes a computer to execute each processing procedure. For this reason, this embodiment can also be understood as an embodiment of a program that realizes the function of executing each of these processes, and a recording medium that stores that program. For example, an update process to add a new function to an information processing device can cause that program to be stored in the storage device of the information processing device. This makes it possible to have the updated information processing device perform each of the processes shown in this embodiment.

[0201] Although embodiments of the present invention have been described above, these embodiments only represent a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

[0202] This application claims priority based on Japanese Patent Application No. 2022-135323, filed with the Japan Patent Office on 26 August 2022, and all contents of that application are incorporated herein by reference.

Claims

1. An information processing method for controlling an agent device having a display unit that displays a facial image and notification information to be provided to the user, This includes a control process for controlling the display mode of the display unit, The display unit comprises a first display unit for displaying a facial image and a second display unit for displaying notification information to be provided to the user. In the control process described above, when the notification information is to be displayed on the second display unit, a switching effect is performed using the first display unit to guide the user's attention by changing the facial image to a predetermined expression, and after the display of the facial image is erased from the first display unit, the notification information is displayed on the second display unit. Information processing methods.

2. The information processing method according to Claim 1, In the control process, as the switching effect, an effect is performed in which the display device comprising the first display unit and the second display unit is moved. Information processing methods.

3. The information processing method according to Claim 2, The display device consists of a cubic or rectangular prism-shaped housing in which the first display unit is provided on one surface and the second display unit is provided on another surface different from the first surface. In the control process described above, when the notification information is to be displayed on the second display unit, a rotation operation is performed as the switching effect to switch the side of the display device visible to the user from the first side to the other side, and the notification information is to be displayed on the second display unit after the display of the face image on the first display unit has been erased. Information processing methods.

4. The information processing method according to Claim 3, The display device consists of a flat plate-shaped housing on which the first display unit is provided on one surface and the second display unit is provided on the other surface which is the surface opposite to the first surface. In the control process described above, a rotational operation is performed to rotate the display device 180 degrees around the rotation axis as the switching effect. Information processing methods.

5. The information processing method according to Claim 3, The display device consists of a cubic or rectangular parallelepiped housing on which the first display unit is provided on one face and the second display unit is provided on the other face adjacent to the first face. In the control process described above, a rotational operation is performed to rotate the display device 90 degrees around the rotation axis as the switching effect. Information processing methods.

6. The information processing method according to claim 2, The display device comprises a flat display panel having a first display section on one side and a second display section on the opposite side of the first display section, and a third display section. The display panel is provided on the third display section so as to be rotatable with one end as the axis of rotation, The display device is controlled by the rotation of the display panel to enter either a first state in which the face image is displayed in a first display area composed of the first display unit and a part of the third display unit, or a second state in which the notification information is displayed in a second display area composed of the second display unit and a part of the third display unit. In the control process, when the notification information is to be displayed on the display device, the display panel is rotated to transition from the first state to the second state, and the notification information is displayed in the second display area after the display of the face image in the first display area has been erased. Information processing methods.

7. The information processing method according to Claim 1, The first display unit is installed to transition between a first state in front of the second display unit, covering the display surface of the second display unit, and a second state in which it slides in a direction substantially parallel to the display surface of the second display unit, allowing the user to view both the display surface of the first display unit and the display surface of the second display unit. In the control process described above, when the notification information is to be displayed on the second display unit, the first display unit is slid to transition from the first state to the second state, and the notification information is displayed on the second display unit after the display of the face image on the first display unit has been erased. Information processing methods.

8. An information processing method according to any one of claims 1 to 7, The first display unit and the second display unit are capable of displaying each image with different display characteristics by switching between at least one of the following: three-dimensional images and planar images, high-resolution and low-resolution images, and color images and black-and-white images. In the control process described above, when the notification information is to be displayed on the second display unit, the switching effect is performed to display the face image of the predetermined expression on the first display unit, which has different display characteristics from the face image displayed before the change to the predetermined expression. Information processing methods.

9. An information processing method according to any one of claims 1 to 7, In the control process described above, the importance of the notification information is determined based on the urgency of the notification information, and the timing of switching from the display of the face image to the display of the notification information is changed based on the importance. Information processing methods.

10. The information processing method according to claim 9, In the aforementioned control process, If the importance of the aforementioned notification information is higher than the standard, immediately after the notification information is generated, the display will switch from the face image to the notification information. When the importance of the notification information is lower than the standard, and the face image displayed on the first display unit is performing a predetermined animation, the display will switch from the face image to the notification information after the predetermined animation is completed. When the face image displayed on the first display unit is not performing the predetermined animation, the display will switch from the face image to the notification information after the notification information is generated. Information processing methods.

11. An information processing device for controlling an agent device having a first display unit for displaying a facial image and a second display unit for displaying notification information to be provided to the user, The system includes a control unit that controls the display modes of the first display unit and the second display unit, When the control unit displays the notification information on the second display unit, it performs a switching effect using the first display unit to guide the user's attention by changing the facial image to a predetermined expression, and then displays the notification information on the second display unit after erasing the display of the facial image from the first display unit. Information processing device.

Images