Systems and programs, etc.

Abstract

To provide a technique for allowing, for example, a user to be in contact with various performances of a character by providing a technique that is different from conventional techniques.SOLUTION: A display device 100 is provided, comprising a control unit configured to provide control to alter a relationship between a user and a given character according to a user input, and output character information on the character according to the relationship.SELECTED DRAWING: Figure 1A

Description

Technical Field

[0004] , , , ,

[0005] , , ,

[0003] , , ,

[0001] The present invention relates to, for example, systems and programs.

Background Art

[0002] Patent Document 1 describes a stereoscopic video display device that does not use special glasses. This stereoscopic video display device includes a light source, a liquid crystal display panel, and a parallax barrier disposed on the observer side of the liquid crystal display panel.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One object of the present invention is to provide a technique different from the conventional one, for example, to provide a technique that enables a user to recognize the relationship between himself / herself and a character through character information.

[0005] The purpose of the present invention is not limited thereto, and the applicant intends to obtain rights through divisional applications, amendments, etc., for configurations that aim to obtain the effects derived from the components of the configuration disclosed in this specification and the drawings, etc. For example, problems that can be described as "can be achieved" in this specification are disclosed in this specification if they are reinterpreted as "the problem is...". Each problem is described independently, and the applicant intends to obtain rights to each configuration for solving each problem independently through divisional applications, amendments, etc. Even if a problem is implicitly understood from the description in the specification, the applicant intends to include a part of the configuration described in this specification in the claims through amendment or divisional application. Furthermore, configurations that solve problems by combining these independent problems are also disclosed, and the applicant intends to obtain rights to them. [Means for solving the problem]

[0006] This invention includes several means for solving the above problems, but it is preferable to provide a system that outputs character information about a predetermined character in response to input from the user, and in particular, it is preferable to do so as follows.

[0007] (1) It would be preferable to provide a system having a control unit that controls the relationship between the user and a predetermined character in response to input from the user and outputs character information relating to the character according to the relationship. In this way, users can recognize their relationship with the character through the character's information.

[0008] User input can be provided as input to the display unit that shows images, and in particular as input to the character displayed on the display unit. Alternatively, input to the character can be provided as the user's voice directed at the character, and in particular as the user's speech directed at the character. Character information can be information that allows the user to recognize the character's existence, and in particular as video information for displaying the character on the display unit, audio information for outputting the character's voice, or a combination of these. The relationship between the user and the character can include elements related to emotions such as human relationships, and in particular as the degree of liking or intimacy, and in particular as the character's mood towards the user.

[0009] (2) The control unit may change the relationship in response to an operation by the user to touch a part of the character's body displayed on the display unit. In this way, users can access character information that changes in response to their interactions with the character's body, and can also recognize their relationship with the character.

[0010] (3) The control unit may change the relationship in response to the user's voice input to the character as input from the user. In this way, users can access character information that changes in response to their voice input towards the character, and recognize their relationship with the character.

[0011] (4) The control unit may worsen the relationship between the character and the user in response to input from the user expressing an action that the character finds unpleasant. In this way, users can recognize that their relationship with the character has deteriorated in response to their actions that the character found offensive, by interacting with the character's information, and thus gain a sense of real-life interpersonal relationships.

[0012] (5) The control unit may change the relationship by changing the mood of the character toward the user. In this way, users can recognize that the character's mood has changed as a result of their input by interacting with the character's information, and gain a sense of real-life interpersonal relationships.

[0013] (6) The control unit may output the character information including a phrase that reflects the emotions corresponding to the relationship. In this way, users can recognize their relationship with a character from the phrases included in the character's information, and gain a sense of what a real interpersonal relationship feels like.

[0014] (7) When the control unit receives an input requesting the character to perform a predetermined function, it may output the character information including a response to the request according to the relationship. In this way, when a user requests a character to perform a specific function, they receive a response that is appropriate to their relationship with the character at that time, giving them a sense of a real interpersonal relationship.

[0015] (8) The control unit may restrict the execution of the predetermined function if it is in the first state indicating that the relationship has deteriorated. In this way, when the user's relationship with the character deteriorates, the execution of the functions they requested is restricted, giving them a sense of real-life interpersonal relationships where the character doesn't readily listen to their requests.

[0016] (9) The control unit may change the relationship to a second state in which the relationship is improved compared to the first state, in response to a predetermined input from the user received when the relationship is in a first state indicating that the relationship has deteriorated. In this way, even if the user's relationship with a character is deteriorating, they can improve that relationship through their own actions, giving them a sense of real-life interpersonal relationships.

[0017] (10) When the control unit receives the predetermined input, it may change from the first state to the second state with a predetermined probability. In this way, even when users take actions to improve their relationship with a character, it doesn't necessarily mean that the relationship between the two will improve, allowing them to experience a feeling similar to real interpersonal relationships.

[0018] (11) The predetermined input may be an input of an action that expresses an apology to the character. In this way, by expressing an apology to the character, the user can feel as if their relationship with the character has improved, giving them the impression that they have made up with that character.

[0019] (12) When the control unit changes the relationship in a direction that improves or worsens it, it may output the character information of one of the multiple patterns of character information corresponding to the direction in which the relationship is changed. In this way, users can perceive a variety in how characters are portrayed when their relationship with them improves or deteriorates, giving them a sense of real-life interpersonal relationships.

[0020] (13) The plurality of patterns includes a first pattern and a second pattern, and the control unit may output character information for performing a performance in which at least one of the character's voice and the character's movements are different between the first pattern and the second pattern. By doing so, the user can feel the diversity of the effects as the voice or movement of the character changes according to the relationship with the character, and can obtain a feeling similar to that of an actual interpersonal relationship.

[0021] (14) The control unit may change the dialogue pattern in the dialogue process between the character regarding a predetermined type and the user according to the relationship. By doing so, in the dialogue process regarding a predetermined type, the user can change the dialogue pattern according to the relationship with the character, and thus can interact with the character while obtaining a feeling similar to that of an actual interpersonal relationship.

[0022] (15) The control unit outputs the character information for the character to perform an effect of displaying the time, and may output the character information for performing an effect of any one of a plurality of patterns with different movements of the character. By doing so, the user can not only simply know the time, but also obtain a feeling that the character is teaching the time, and can be exposed to various effects.

[0023] ​​​​​​​(17) The multiple patterns of the character's movement include a third pattern and a fourth pattern, wherein the display of the character and the position of the object at the end of the third pattern's performance are the same as the display of the character and the position of the object at the start of the fourth pattern's performance, and the control unit may output the character information for performing the fourth pattern's performance in a timely sequence with the third pattern's performance. In this way, even when the third and fourth animation patterns are time-sequential, users are less likely to notice any inconsistencies in the display at the transitions between these animations, making it easier to perceive the character's movements as natural. It is possible.

[0025] (18) The control unit may leave a predetermined time between the end of the third pattern's performance and the start of the fourth pattern's performance. In this way, even when the third and fourth animation patterns are time-sequential, users are less likely to notice any inconsistencies in the display at the transitions between these animations, making it easier to perceive the character's movements as natural.

[0026] (19) The control unit may output the character information for performing the effect of the character sleeping. This approach makes it easier for users to feel as if the character is still present or living alongside them, even when the character is asleep.

[0027] (20) When the character is asleep, the control unit may disable changing the relationship in response to user input. This way, the user can recognize that because the character is asleep, the character is unaware of any actions that might change the relationship with the character.

[0028] (21) If the control unit is in a first state indicating that the relationship has deteriorated, it may change the relationship to a second state which is an improvement over the first state, on the condition that the character is shown sleeping. In this way, users can perceive the character's personality as one that allows them to forget unpleasant things when they sleep.

[0029] (22) The characters include a first character and a second character. When the control unit receives a predetermined input from the user, it outputs character information relating to the first character in accordance with the predetermined input. On the other hand, when the first character is performing a sleeping animation, it may output character information relating to the second character in accordance with the predetermined input. In this way, when the first character is asleep, the user can get the feeling that the second character is taking care of them instead.

[0030] (23) The characters include a first character and a second character, and the control unit may output character information for the first character for outputting voice based on pre-recorded voice data, and output character information for the second character for outputting voice after performing voice synthesis processing. In this way, even if the first character's voice, based on pre-recorded audio data, cannot output certain sounds, the second character's voice can do so. The user can then naturally accept that the second character is handling things on behalf of the first character.

[0031] (24) When the control unit receives input from the user requesting the first character to perform a predetermined function, if the request is for the first function to be performed, it may perform the first function by outputting the voice of the first character, and if the request is for the second function to be performed, it may perform the second function by outputting the voice of the second character. In this way, the user can receive the results of a first function that can be performed using the voice of the first character based on pre-recorded voice data from the first character, and the results of a function that cannot be performed using the voice of the first character, from the second character instead of the first character, and receive both in a natural manner.

[0032] (25) When the control unit receives an input requesting the execution of the second function, it may output character information relating to the first character for performing a presentation that indicates the execution of the second function using the second character, and then execute the second function by outputting the voice of the second character. In this way, the user can experience the scene in which the second character accepts the request from the first character and the second action is performed, as well as the result of that action.

[0033] (26) The control unit may output character information relating to the second character so that the performance using the second character is started when the output of sound in the performance using the first character has finished. In this way, the user recognizes that the second character will accept their request through the first character, and can smoothly receive the results of the second character's execution of their request.

[0034] (27) The control unit may output character information for performing the character's actions to describe the functions that can be performed. In this way, users can perceive the available functions as if they were being explained by a character.

[0035] (28) If the control unit detects an indication of a malfunction, it may output character information for the character to provide an explanation regarding the malfunction. In this way, users can more easily accept the occurrence of malfunctions, even if they are inconvenient for them, because the character explains them to them.

[0036] (29) The system includes a display unit for displaying an aerial image, and the control unit may output character information to the display unit for displaying the character in an aerial image. In this way, users can get the feeling that they are interacting with a character that has a floating or virtual feel to it.

[0037] (30) It is preferable to have a non-contact sensor that detects operations on the aerial image. In this way, users can input controls through aerial manipulation and enjoy the virtual experience.

[0038] (31) The display unit may include a tilt changing unit that displays the aerial image on the user side and changes the tilt of the display of the aerial image on the user side. This allows users to change the tilt of the aerial image display to view it more effectively. For example, a user can view the aerial image directly in front of them, which can enhance immersion in their interactions with the character.

[0039] (32) Having a housing that has a space enclosed by an inner wall, and having the display unit inside the housing, The tilt changing unit may change the tilt of the display unit and the aerial image display by changing the tilt of the housing. In this way, users can change the tilt of the aerial image by changing the tilt of the display unit. Since the tilt of the aerial image changes in conjunction with the physical tilt of the display unit, the configuration for changing the tilt of the aerial image can be simplified.

[0040] (33) The control unit has a function to adjust the brightness level of the display unit according to the ambient brightness, and it is preferable that the content of the brightness level adjustment be set individually for when the ambient brightness is bright and when it is dark. This allows users to set the display brightness separately for bright and dark environments, enabling them to view images at a brightness level that is easy to see and that reflects their preferences at each brightness level.

[0041] (34) The control unit may output the character information corresponding to the inclination of the aerial image display. In this way, users can access diverse character information that changes depending on the tilt of the aerial image display.

[0042] (35) It would be preferable to provide a program that causes a computer to function as the control unit of any of the above systems. This approach allows us to offer technologies different from conventional ones; for example, users can recognize their relationship with a character through character information.

[0043] The inventions described in (1) to (35) above can be combined in any way. For example, one could combine all or part of the configuration of the invention described in (1) with at least part of the configuration of at least one of the inventions described in (2) and onward. In particular, one could combine the invention described in (1) with at least part of the configuration of at least one of the inventions described in (2) and onward. Alternatively, one could extract any configuration from the inventions described in (1) to (35) and combine the extracted configurations. The applicant of this application intends to acquire rights to inventions that include these configurations. Furthermore, even if there are descriptions such as "in the case of..." or "when...", these are not meant to indicate that the configuration is limited to that case or time. These are merely examples of better configurations, and the applicant intends to acquire rights to configurations that do not fall under these cases or times. Also, even if there is a sequence of descriptions, the order is not limited to that sequence. Configurations with some parts deleted or the order rearranged are also disclosed, and the applicant intends to acquire rights to them as well. [Effects of the Invention]

[0044] According to the present invention, it is possible to provide a technology different from conventional methods, for example, that a user can recognize the relationship between themselves and a character through character information.

[0045] The effects of the present invention are not limited thereto, and the effects produced by the components of the structure disclosed in this specification and drawings are also disclosed. The present invention intends to obtain rights to the components that produce such effects through divisional applications, amendments, etc. For example, the phrases "can do..." in this specification are descriptions that clearly indicate the effects produced, and there are components that produce effects even without such descriptions. Furthermore, there are effects that can be grasped by the component even without such descriptions. [Brief explanation of the drawing]

[0046] [Figure 1A] This is a perspective view of the display device 100. [Figure 1B] This is an explanatory diagram of the character displayed on the display device 100. [Figure 1C] This is an explanatory diagram of the character displayed on the display device 100. [Figure 2A] This is an explanatory diagram of the usage status and operating environment of the display device 100. [Figure 2B] This is an explanatory diagram showing how to use the display device 100. [Figure 3] This is a perspective view of the display device 100. [Figure 4A] These are views of the display device 100 from various angles. [Figure 4B] This shows the external appearance of the display device 100 from different directions than those shown in Figure 4A. [Figure 5] This is a perspective view of the front case 111. [Figure 6A] These are views of the front case 111 from various angles. [Figure 6B] This shows the external appearance of the front case 111 from different directions than those shown in Figure 6A. [Figure 7] This is a cross-sectional view of the front case 111. [Figure 8] This is an explanatory diagram illustrating the different tilt states of the display device 100. [Figure 9] This is an explanatory diagram illustrating the relationship between the display device 100, which has its housing 110 tilted, and the cables 910 and 920 connected to the display device 100. [Figure 10] This is a cross-sectional view of the display device 100. [Figure 11] This is an exploded view of the display device 100. [Figure 12] This is an exploded view illustrating the relationship between the front case 111, the speaker 1104, and the speaker enclosure. [Figure 13] This is a diagram illustrating the shape of a speaker enclosure. [Figure 14] This is a perspective view of the shape of the lower part of the front case 111. [Figure 15] This is an explanatory diagram illustrating the connection status of the electrical components of the display device 100. [Figure 16] This is a system block diagram illustrating the electrical configuration of the display device 100. [Figure 17] This is an explanatory diagram of the data stored in the storage unit 1670 of the display device 100. [Figure 18] This is an explanatory diagram of the startup process of the display device 100. [Figure 19A] This is an explanatory diagram of the detection area and detection target of the non-contact sensor 1102. [Figure 19B] This is an explanatory diagram of the detection area and detection target of the non-contact sensor 1102. [Figure 20] This is an explanatory diagram of the settings screen and standby screen related to the clock mode of the display device 100. [Figure 21] This diagram illustrates the settings screen and standby screen related to the zoom function of the display device 100. [Figure 22] This is an explanatory diagram of the settings screen for the voice recognition function of the display device 100. [Figure 23] This is an explanatory diagram of the setting screen for the sound setting function of the display device 100. [Figure 24] This is an explanatory diagram of the setting screen for the brightness setting function of the display device 100. [Figure 25] This is an explanatory diagram of the setting screen for the alarm function of the display device 100. [Figure 26] This is an explanatory diagram of the standby screen when an alarm is set on the display device 100. [Figure 27] This is an explanatory diagram of the screen for changing or deleting alarm settings on the display device 100. [Figure 28] This is an explanatory diagram of the user information setting screen of the display device 100. [Figure 29] This is an explanatory diagram of other screens displayed via the settings screen of the display device 100. [Figure 30] This is an explanatory diagram of the screen displayed on the display device 100 when the user 200 is able to interact with the character 131. [Figure 31] This is an explanatory diagram of the performance content table, which shows the content of the performance to be performed when the trigger phrase is recognized by the display device 100. [Figure 32]This is an explanatory diagram of the performance content table, which shows the content of the performance to be performed after the trigger phrase is recognized by the display device 100. [Figure 33] This is an explanatory diagram of the display content table, which shows the content of the effects performed when the display device 100 detects a prank. [Figure 34] This is an explanatory diagram of the display content table, which shows the content of the effects performed when the display device 100 detects a prank. [Figure 35] This is an explanatory diagram of the body parts of character 131 used in the display device 100 for detecting mischievous behavior. [Figure 36] This is an explanatory diagram of the performance content table, which shows the content of the performance to be performed when an act of apology is detected by the display device 100. [Figure 37A] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good morning". [Figure 37B] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good morning". [Figure 37C] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good morning". [Figure 37D] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good morning". [Figure 37E] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good morning". [Figure 37F] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good morning". [Figure 38] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Hello". [Figure 39] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "Good evening". [Figure 40A]This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "goodnight". [Figure 40B] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "goodnight". [Figure 41] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "calling out" or "thank you". [Figure 42A] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "I'm leaving." [Figure 42B] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "I'm leaving." [Figure 43] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "I'm home" or "temperature". [Figure 44] This is an explanatory diagram of the presentation content table, which shows the presentation content when user 200 utters a phrase of the type "cute". [Figure 45A] This is an explanatory diagram of the animation that shows the character's performance when the time mode of the display device 100 is ON. [Figure 45B] This is an explanatory diagram of the animation that shows the character's performance when the time mode of the display device 100 is ON. [Figure 45C] This is an explanatory diagram of the animation that shows the character's performance when the time mode of the display device 100 is ON. [Figure 45D] This is an explanatory diagram of the animation that shows the character's performance when the time mode of the display device 100 is ON. [Figure 46] This is an explanatory diagram of the standby screen displayed at night on the display device 100 when the clock mode is ON. [Figure 47A] This diagram shows the animations displayed when the character 131 of the display device 100 goes to sleep. [Figure 47B] This diagram shows the animations displayed when the character 131 of the display device 100 goes to sleep. [Figure 48A] This diagram shows the animation content for character 131 of the display device 100 when it wakes up. [Figure 48B] This diagram shows the animation content for character 131 of the display device 100 when it wakes up. [Figure 49A] This is an explanatory diagram of the display content table showing the display content related to the timer function of the display device 100. [Figure 49B] This is an explanatory diagram of the display content table showing the display content related to the timer function of the display device 100. [Figure 49C] This is an explanatory diagram of the display content table showing the display content related to the timer function of the display device 100. [Figure 49D] This is an explanatory diagram of the display content table showing the display content related to the timer function of the display device 100. [Figure 49E] This is an explanatory diagram of the display content table showing the display content related to the timer function of the display device 100. [Figure 50] This is an explanatory diagram of the standby screen while the timer function of the display device 100 is running. [Figure 51] This is an explanatory diagram of the screen used for notification when character 131 of the display device 100 is asleep. [Figure 52] This is an explanatory diagram of the screen used for notification when character 131 of the display device 100 is asleep. [Figure 53] This is an explanatory diagram of the presentation content table showing the presentation content related to the alarm function of the display device 100. [Figure 54A] This is an explanatory diagram of the display content table showing the display content related to the weather information function of the display device 100. [Figure 54B] This is an explanatory diagram of the display content table showing the display content related to the weather information function of the display device 100. [Figure 55] This is an explanatory diagram of the presentation content table showing the presentation content related to the anniversary notification function of the display device 100. [Figure 56]This is an explanatory diagram of the screen related to the gallery function of the display device 100. [Figure 57A] This is an explanatory diagram of the presentation content table showing the presentation content related to the external device control function of the display device 100. [Figure 57B] This is an explanatory diagram of the presentation content table showing the presentation content related to the external device control function of the display device 100. [Figure 58] This is an explanatory diagram of a display device 100 connected to network 1810. [Figure 59] This is a block diagram showing the electrical configuration of user terminal 1830. [Figure 60] This is an explanatory diagram of the home screen of user terminal 1830. [Figure 61] This is a diagram illustrating the main settings screen of user terminal 1830. [Figure 62] This is a diagram illustrating the settings screen for alarm settings on user terminal 1830. [Figure 63] This is a diagram illustrating the other settings screens of user terminal 1830. [Figure 64] This is an explanatory diagram of the outing function. [Figure 65] This is an explanatory diagram of the outing function. [Figure 66] This is an explanatory diagram showing variations in the tilt adjustment section. [Figure 67] This is an explanatory diagram of the display device 100 according to the second embodiment. [Figure 68] This is an explanatory diagram of the display device 100 according to the third embodiment. [Figure 69] This is an explanatory diagram of the display device 100 according to the fourth embodiment. [Figure 70] This is an explanatory diagram of the display device 100 according to the fifth embodiment. [Figure 71] This is an explanatory diagram of the display device 100 according to the sixth embodiment. [Modes for carrying out the invention]

[0047] Hereinafter, embodiments of the system of the present invention applied to the display device 100 will be described with reference to the drawings. The embodiments shown below are examples of embodiments of the present invention, and the present invention is not limited to these embodiments. In the drawings referenced in these embodiments, the same parts or parts having similar functions are denoted by the same or similar reference numerals (reference numerals with A, B, etc., added after a number), and repeated explanations may be omitted. In addition, in the figures referenced in the following description, the scale may be different from that of the actual parts in order to make each component, each area, etc., recognizable.

[0048] [1. Overview of the display device 100] Figure 1 is a diagram illustrating the overview of a display device 100 according to one embodiment. The display device 100 displays Yupiteru's mascot character, Kirishima Rei (hereinafter referred to as "Rei"), as a hologram (a 2D hologram in the display device 100), allowing the user to enjoy conversations and images with Rei. When Rei is displayed on the screen, Rei blinks. In the following description, the display device 100 will be described in the case where Rei is displayed on the standby screen. The standby screen is the normal screen when the power of the display device 100 is turned ON.

[0049] Figure 1A is a perspective view of the display device 100. Figure 1A(a) is a perspective view of the display device 100, illustrating the state in which the display device 100 is not displaying the aerial image 130. Figure 1A(b) is a perspective view of the display device 100 with the aerial image 130 displayed, illustrating the state in which the aerial image 130 is displayed on the display device 100. Figure 1A(c) is a front view of the display device 100 shown in Figure 1A(a), with an image-forming plane 140 for the aerial image 130 added. The image-forming plane 140 is indicated by a dashed line. Figure 1A(d) is a front view of the display device 100 shown in Figure 1A(b), with an image-forming plane 140 for the aerial image 130 added. The image-forming plane 140 is indicated by a dashed line. Figures 1B and 1C illustrate the characters displayed on the display device 100.

[0050] Display device 100 is a display device based on the concept of displaying Kirishima Rei, the mascot character of the applicant, Yupiteru Corporation, as a hologram, allowing users to enjoy conversations with the character and images of the character. Display device 100 displays the character (character 131, described later) as an aerial image 130, making the character appear to float in space like a hologram. In this sense, the aerial image 130 may be perceived as a floating image that gives the user a sense of floating when viewed. Thus, display device 100 displays the aerial image 130 in such a way that the user can recognize the existence of a character that gives a sense of three-dimensionality. In this sense, display device 100 may also be called a virtual figure, virtual signage, etc.

[0051] The display device 100 has the following structural features, for example: - To display a character projected like a hologram using an aerial image 130, allowing the user to face the character (for example, directly in front of it) and touch the character. • Equipped with a non-contact sensor to detect non-contact operations, and to detect operations performed by the user by touching the aerial image 130 (aerial operation or gesture operation, as described later).

[0052] Furthermore, the display device 100 performs functions to realize simulated communication between the character and the user. Such communication functions include, for example, the user conversing with the character, or the user touching the character's body with their hand and the character reacting to that.

[0053] Regarding features related to communication, for example, it has the following characteristics: • A feature where a character announces the time. This function connects to the internet via Wi-Fi or similar and communicates with a designated server device to provide weather information via a character. • Features voice recognition (e.g., local voice recognition) that allow users to converse and interact with characters. • The relationship between the user and the character, such as the character's feelings towards the user, can change based on the content of conversations and dialogues, as well as the user's touch actions. - The character's movements and phrases will change according to the relationship between the user and the character, allowing users to enjoy living together with a character full of humanity. • A function to control external devices such as home appliances.

[0054] However, the display device 100 is not required to have all of these functions; it may not have at least some of these functions, or it may have functions other than those listed above.

[0055] Next, we will describe Rei, an example of the first character, with reference to Figures 1B and 1C. Character 131 is an image displaying Rei's appearance. In the following description, when we refer to "character," we may be referring to the image of the character displayed on the display device 100, or to the character itself (for example, Rei herself). An overview of Rei's day is shown in Figure 1B. As a cybernetic girl, Rei changes clothes and sleeps in conjunction with real time. Figures 1B(a) and 1B(b) show examples of character 131 displayed when Rei is awake. Rei is an early riser and gets up at 5:00. During the time Rei is awake, from 8:00 to 20:00, character 131 is displayed wearing a skirt, as shown in Figure 1B(a). During the times when Ray is awake, specifically from 5:00 to 8:00 and from 20:00 to 0:00, character 131 is displayed wearing a different outfit, pajamas, as shown in Figure 1B(b). Ray sleeps at night (0:00 to 5:00). During this time, character 131 is displayed wearing pajamas and appearing to be asleep, as shown in Figure 1B(c). While Ray is awake, user 200 can interact with Ray, but interaction with Ray is restricted while Ray is asleep. During this time, the user may be able to interact with Teru (an example of a second character), a character representing Ray's pet cat, on Ray's behalf. Teru is displayed as character 150, for example, as shown in Figure 1B(d). Under certain circumstances, the user may also meet and communicate with Teru even while Ray is awake.

[0056] Ray's emotions can change. If the user does something that Ray dislikes, Ray may get angry (i.e., become grumpy). In this case, Ray may, for example, sulk and give grumpy responses, or become unable to respond at all. Figure 1C(a) shows character 131 as it appears when Ray is not in a bad mood, i.e., in a normal state. Figure 1C(b) shows character 131 as it appears when Ray is grumpy, i.e., angry. When Ray is grumpy, calling her by name and apologizing may make her forgive you. Also, Ray has a personality that might forget what happened the day before and cheer up after a good night's sleep. Details regarding this interaction between the user and character 131 will be explained in detail in the section [3. Examples of software processing of the display device 100] below.

[0057] [2. Configuration of the display device 100] Next, the configuration of the display device 100 will be described. In the embodiment shown in Figure 1A, the user (user 200 shown in Figure 2A) can perceive the aerial image 130 as if it were floating in space. In particular, the display device 100 displays the aerial image 130 as a stereoscopic image that allows the user to perceive depth. The display device 100 has a housing 110 that opens towards the user 200, and displays the aerial image 130 in the space inside the housing.

[0058] User 200 can see the aerial image 130, which gives a sense of three-dimensionality, appearing to leap out from the box materialized in the casing 110 towards User 200 (the front side). As will be explained in more detail later, the fact that the inside of the casing 110 is maintained in a relatively dark environment also contributes to User 200 getting the feeling that the character is floating. In this way, the display device 100 makes it easier for User 200 to recognize the object displayed in the aerial image 130 as a three-dimensional object (see Figure 1A(d) in particular). This display also contributes to User 200 feeling a greater sense of familiarity and attachment to the object in the aerial image 130.

[0059] Figures 1A(b) and 1A(d) show that the display device 100 displays the character's image as an aerial image 130. The display device 100 is also configured to perform the communication functions described above. As a result, the user 200 can get the feeling that they are interacting with the character face-to-face.

[0060] The image plane 140, shown by a dashed line in Figure 1A(d), indicates the imaging position of the aerial image 130. The image plane 140 and the aerial image 130 exist on the same plane. In other words, the image plane 140 is the position where the aerial image 130 is imaged. The image plane 140 can also be understood as the display surface of a virtual image. To make this clear, that is, to make it clear that the aerial image 130 is not in front of or behind the image plane 140, but precisely on the image plane 140 itself, in Figure 1A(d), one of the diagonal lines representing the image plane 140 is drawn to pass on the front side of the aerial image 130, and the other diagonal line passes on the back side of the aerial image 130.

[0061] The display device 100 is a display device that is small enough in size and weight to be held and carried by a user 200 with one hand. The display device 100 has a vertical housing structure that is longer in the height direction (i.e., up and down direction) than in the width direction (i.e., left and right direction). In this embodiment, the external appearance of the display device 100 is a rectangular parallelepiped. The dimensions of the display device 100 are, for example, 171.5 mm in the height direction, 97 mm in the width direction, and 93 mm in the depth direction. The weight of the display device 100 is, for example, approximately 565 g. The user 200 places the display device 100 on a flat surface such as a desk 230 or a table (hereinafter referred to as the "contact surface") and uses the display device 100.

[0062] The display device 100 is configured to change the tilt of the aerial image 130 displayed to the user 200. The display device 100 is configured to change the tilt of the aerial image 130 relative to the user 200. The tilt of the display relative to the user 200 may refer, for example, to the tilt of the aerial image 130 or the image plane 140 with respect to the direction of the user 200's line of sight. The display device 100 is particularly configured to change the tilt of the display in the front-to-back direction, and in particular, it is well-configured to change the aerial image 130 so that it faces directly toward the user 200's line of sight.

[0063] The display device 100 is configured so that the user 200 can adjust whether or not the aerial image 130 is tilted and the magnitude of that tilt. The aerial image 130 is not limited to an image displayed in a planar manner, but may also be a curved surface or a three-dimensional image. Other display methods for aerial images will be described later. In this embodiment, "tilt of the aerial image 130" may be read as "tilt of the imaging surface 140 of the aerial image 130" or "tilt of the aperture surface 700" (see Figure 7; the aperture surface 700 will be described later), etc. As a result, even if the user 200's eye level and the height of the display device 100's installation position are different, the user 200 can view the aerial image 130 directly by adjusting the tilt of the display of the aerial image 130. This point will be explained in more detail using Figure 2A.

[0064] In the embodiments shown in Figures 1A(a) and 1A(d), the aerial image 130 is an animated female character, but it may also be a male character, a non-human animal (e.g., a cat), a robot, a virtual character, or any other character. While the aerial image 130 may be animation-based, it is not limited to this; it may also be photograph-based, computer graphics-based, or a combination of these. Furthermore, two or more characters may be displayed simultaneously as the aerial image 130. The aerial image 130 may be a video image with some kind of movement (motion), but it may also be a still image.

[0065] In Figures 1A(c) and 1A(d), the imaging surface 140 is located at the position of the aperture surface 700 (see Figure 7) provided on the front side (user 200 side) of the display device 100, and is approximately the same size as (or may be the same size as) the aperture surface 700. Therefore, the display device 100 is configured to display the aerial image 130 within the range of the aperture surface 700 when the user 200 is looking directly at the imaging surface 140.

[0066] The imaging surface 140 is preferably displayed at or near the position of the aperture surface 700 within the concave space (the first space 300 described later) that is opened by the aperture surface 700, as will be described later, but it may also be displayed at a position offset in front of or behind it. However, the inventors have confirmed that it is easier to obtain a sense of three-dimensionality in the aerial image 130 when the imaging surface 140 is located within the first space 300 compared to when the imaging surface 140 is located closer to the user 200. The size of the imaging surface 140 may differ from the size of the range of the aperture surface 700. "Imaging surface 140" may be read as "aerial image 130" as appropriate.

[0067] The inventors have found that by displaying the aerial image 130 at or near the opening surface 700 on the side of the first space 300 inside the housing 110 that is closest to the user 200, the effect of enhancing the sense of three-dimensionality of the aerial image 130 perceived by the user 200, such as the feeling of the character floating or popping out, is enhanced in combination with the first space 300. The sense of popping out can be described as the feeling that the image of the character appears to float out from the jet-black housing. Furthermore, when the aerial image 130 is displayed at or near the opening surface 700, the user 200 can see the aerial image 130 at a position close to themselves. Therefore, this is desirable for enhancing immersion in the use of the display device 100, such as for communication with the character. The term "nearby" refers, for example, to a range of approximately 10% before and after the distance from the lens plate 1001 (described later) to the image plane.

[0068] Figure 1A(a) shows that the front case 111 and rear case 112 of the housing 110 of the display device 100, as well as the tilt adjustment section 120, have different color schemes. The front case 111 should preferably be a low-luminosity color such as black. This can improve the sense of floating that the user 200 feels when viewing the aerial image 130 shown in Figure 1A(b). Furthermore, the front case 111 is surface-treated to suppress light reflection, for example, by a textured or matte finish (i.e., non-glossy). This can prevent the user 200's face from being reflected in the aerial image 130, thereby enhancing the sense of immersion when using the display device 100.

[0069] The rear case 112 is preferably a color with a higher brightness than the front case 111, such as a glossy white or silver. Furthermore, the rear case 112 may be polished to give it a lustrous surface. This makes it easier for the user 200 to perceive the display device 100 as having a premium feel and a futuristic design, and to appreciate the rarity and novelty of the aerial image display technology 130.

[0070] Figure 2A is a diagram illustrating the usage state and usage environment of the display device 100 according to one embodiment. In Figure 2A, the display device 100 is placed on a desk 230. The display device 100 is positioned so that the aerial image 130 is tilted toward the user 200 seated in a chair 220. Preferably, the display device 100 can be configured to change the tilt of its housing 110 so that the user 200 can view the aerial image 130 from a position at least 35 cm away from the display surface (i.e., the imaging surface 140) of the aerial image 130, and directly face the aerial image 130.

[0071] In particular, the housing 110 should be configured to allow its tilt to be changed continuously (for example, seamlessly) or steplessly. This makes it easier for the user 200 to adjust the housing 110 to their preferred tilt while the display device 100 remains on the desk 230. When adjusting the tilt of the housing 110, the user 200 may, for example, hold it by grasping the top with both hands. The tilt of the housing 110 after adjustment is maintained continuously. The display device 100 should be configured such that the tilt of the display unit 1000 located inside the display device 100 is changed in conjunction with the change in the tilt of the housing 110, thereby adjusting the tilt of the aerial image 130 displayed on the user 200 side. In this specification, "display unit" refers to an example of an "image display unit".

[0072] For example, the tilt adjustment unit 120 is located below the housing 110 and changes the tilt of the housing 110. The tilt of the tilt adjustment unit 120 is fixed (i.e., unchanging) when installed on the desk 230, and the relative tilt of the housing 110 with respect to the tilt adjustment unit 120 can be changed. The tilt adjustment unit 120 is configured to change the front-to-back tilt of the housing 110 so that the aerial image 130 is displayed facing the user 200's line of sight. For example, the tilt of the display of the aerial image 130 is configured to be changed by a predetermined angle range towards the rear, with the horizontal direction Dh as the reference (0 degrees).

[0073] Tilting the housing 110 backward means tilting it so that the upper part of the housing 110 is further back than the lower part. The housing 110 should be configured to be able to change to any angle within the range of 0 to 25 degrees. The relative positional relationship between the user 200's eyes and the aerial image 130 changes depending on the installation environment, such as the height of the display device 100 and the position of the user 200's eyes. In this way, the user 200 can face the aerial image 130 directly and view it in various installation environments, thereby enhancing the user 200's sense of immersion when using the display device 100.

[0074] In the embodiment shown in Figure 2A, the user 200 is looking at the display device 100 at a downward angle Au with respect to the horizontal direction Dh, and the user 200's line of sight Du coincides with the perpendicular Lv to the aerial image 130 on the display device 100. In this state, the user 200 can directly view the aerial image 130. Furthermore, even if the user 200 adjusts their posture or shifts their body forward or backward, the user 200's line of sight Du remains within the range Ap of the preferred field of view of the aerial image 130, so the user 200 can continue to perceive the aerial image 130 without discomfort. The range Ap is, for example, a 20-degree angle range with an angle range of 10 degrees above and below the perpendicular Lv.

[0075] The sizes of the user 200, chair 220, desk 230, etc., and their relative positions shown in Figure 2A are merely illustrative examples. By changing the placement of the display device 100 and the tilt of the housing 110, the user 200 can install the display device 100 in a desired location in various usage environments and use it without any discomfort. Since the display device 100 has a compact configuration with the dimensions and weight described above, it can be placed on surfaces other than the desk 230, such as furniture like shelves, desktop computer cases, monitor stands, etc.

[0076] Figure 2B is a diagram illustrating how to use the display device 100. The display device 100 accepts non-contact operations as input from the user. One example of a non-contact operation is specifying a position on the imaging surface 140 (which may be read as the aerial image 130). Such non-contact operations by the user are detected using a non-contact sensor 1102, which will be described later. The non-contact sensor 1102 detects the position and movement of the user's fingers (fingers F in Figure 2B(a)). The configuration of the non-contact sensor 1102 will be described in detail later. Typically, the non-contact sensor 1102 detects parts of the user's body, such as the user's fingers, but an indicator such as a pen device may also be used. User input is not limited to this, and may also include input by operating physical buttons (for example, pressing the main switch 310) or input by the user's voice (typically the user's voice).

[0077] As shown in Figure 2B(b), non-contact user operations include, for example, touch operations and gesture operations. A touch operation is not an operation that touches a physical surface, but an operation in which a finger (finger F in Figure 2B(b)) touches a position on a virtual aerial image 130 (which may also be called the imaging surface 140) and then lifts it off. A gesture operation is an operation performed by moving the fingers, and in this embodiment, a flick operation (also called a flick) is assumed. A flick operation is an operation in which a finger (finger F in Figure 2B(b)) touches the aerial image 130 (which may also be called the imaging surface 140) and then moves the touching finger F in a sweeping motion. In Figure 2B(b), an operation of moving finger F up and down is shown as a flick operation, but operations in directions such as left and right or diagonally should also be detected as flick operations.

[0078] Figure 3 is a perspective view of a display device 100 according to one embodiment. Figure 3(a) is a perspective view of the display device 100 as seen from the right-diagonal direction relative to the user 200 (front side). Figure 3(b) is a perspective view of the display device 100, viewed from the left diagonal direction relative to the rear side.

[0079] As described above, the display device 100 has a first space 300 as a concave space that opens to the user 200 through the opening surface 700. The display device 100 further has a main switch 310 at a position that the user 200's fingers can reach when they place their fingers into the first space 300 through the opening surface 700. The main switch 310 is an operating part of the display device 100 that is positioned to be particularly easy for the user 200 to see and operate. The main switch 310 is an example of a reception unit that receives input from the user.

[0080] The main switch 310 is an operating unit for performing predetermined operations related to the functions of the display device 100, and it is preferable that it be assigned to control the main functions of the display device 100. Such functions include a function for the user 200 to speak to the character, or in other words, a function to instruct the start (i.e., turn on) of the voice recognition function used in the communication functions described above. The main switch 310 may also be assigned other functions, for example, functions that are used relatively frequently, such as starting or stopping the display device 100. Furthermore, the main switch 310 may be assigned a function to change the display of the aerial image 130. In this specification, "main switch" refers to an example of an "operating unit."

[0081] A sub-switch 320 is provided on the right side of the display device 100 (i.e., the right side as seen from the user 200's perspective) of the outer periphery of the display device 100 (i.e., the housing 110). The sub-switch 320 is a volume control operation unit located at least towards the rear of the center in the depth direction from the front to the rear of the housing 110. In another embodiment, the sub-switch 320 may function as another adjustment switch, various buttons, etc., for example, for adjusting display brightness, in place of or in addition to the volume control switch. In this specification, "sub-switch" refers to an example of a "second operating unit."

[0082] Furthermore, in addition to the sub-switch 320, sensors (hereinafter simply referred to as sensors) 330 are provided to detect various parameters via a sensor window. Specifically, on the right side of the display device 100, further back than the volume control sub-switch 320, there is a sensor window for at least one sensor 330.

[0083] In the embodiment shown in Figure 3, a temperature sensor 1631 (see Figure 16) and an illuminance sensor 1632 (see Figure 16) are used as sensors 330. Alternatively or additionally, at least one sensor 330 can be a humidity sensor, pressure sensor (barometric pressure sensor), carbon dioxide concentration sensor, carbon monoxide concentration sensor, TVOC (total volatile organic compound) sensor, PM2.5 sensor, ozone sensor, etc. In this specification, "sensors" and "sensors" refer to an example of a "sensor unit." Furthermore, in addition to components configured and implemented as so-called "sensors" that detect various environmental parameters, it is also possible to use components that can be used for sensor purposes. For example, a microphone is originally a component for inputting the user's voice, but it can also be used as an external sound detection sensor.

[0084] The display device 100 may have a function to output information to the user 200 based on information obtained from the sensors 330, and may be used, for example, for control of communication functions. The display device 100 may have a configuration to notify the user 200 of information obtained from the sensors 330. If, for example, a temperature sensor is used as the sensor 330, the display device 100 may have the character utter phrases related to temperature, such as "It's hot!" or "It's cold!", based on the temperature information.

[0085] The display device 100 may perform effects such as changing the character's costume according to the temperature. If, for example, an illuminance sensor is used as one of the sensors 330, the display device 100 may change the brightness of the aerial image 130 display (for example, the luminescence of the LCD 1003) according to the illuminance measured by the illuminance sensor. Furthermore, as will be described later, the display device 100 may be configured to control devices connected to the display device 100 based on information obtained from the sensors 330.

[0086] The display device 100 has a power switch 340 on the rear side of the outer periphery of the display device 100 (i.e., the housing 110). By positioning the power switch 340 in a location close to the rear side, where it cannot be seen by the user 200 located on the front side, the appearance of the display device 100 as seen by the user 200 is simplified. In the embodiment shown in Figure 3, the operating elements other than the main switch 310 are removed from the front and top surfaces of the display device 100, which are easily visible to the user 200. As a result, when the user 200 is viewing the aerial image 130, they will not perceive, or will not perceive, the presence of other sub-switches 320, sensors 330, and power switch 340, making it easier for them to concentrate on viewing the aerial image 130. This enhances the user 200's sense of immersion when using the display device 100.

[0087] Furthermore, in the embodiment shown in Figure 3, by arranging the sub-switches 320 and sensors 330 on the outer periphery of the display device 100 as described above, the upper part of the display device 100 can be made lighter and more compact. For example, if a large storage space for electronic components were to be secured in the upper part of the display device 100, it may give the impression that the top of the display device 100 is bloated. In contrast, the display device 100 is designed to give the user 200 the impression that its top is compact, thereby ensuring the aesthetic appearance of the display device 100 and enhancing the user 200's sense of immersion when using the display device 100.

[0088] Furthermore, in the embodiment shown in Figure 3, since the sub-switch 320 is located on the user 200 side compared to the sensors 330, the sensor window is less likely to be covered by the user's fingers even when the user 200 is operating the sub-switch 320, thus minimizing any adverse effects on detection by the sensor 330.

[0089] Figure 4A shows the external appearance of the display device 100 according to one embodiment, viewed from various directions. Figure 4B shows the external appearance of the display device 100 according to one embodiment, viewed from different directions than those in Figure 4A.

[0090] Figure 4A(a) is a top view (plan view) of the display device 100. Figure 4A(b) is a left side view of the display device 100. Figure 4A(c) is a front view of the display device 100. Figure 4B(d) is a right side view of the display device 100. Figure 4B(e) is a rear view of the display device 100. Figure 4B(f) is a bottom view of the display device 100.

[0091] Figure 4A(a) shows that the front width Wf and rear width Wb of the display device 100 are different, with the rear width Wb being smaller than the front width Wf. Furthermore, the top surface of the display device 100 does not have any operating elements such as switches, resulting in a simple appearance. As a result, the top surface of the display device 100 has a very smooth overall structure. By eliminating mechanical components from the top of the display device 100, the display device 100 can have a lighter top section. Figure 4A(a) further shows that the switch protrudes from the right side, and that the sum of the width of the display device 100 at the location where the switch is installed and the protruding width of the switch does not exceed the front width Wf.

[0092] With the above configuration, when the user 200 faces the aerial image 130 and views the display device 100 from the user 200's side (front side), the user 200 cannot see the sub-switch 320 and sensors 330 located on the right side of the display device 100, or their presence is made very difficult to see. As a result, from the user 200's perspective, the unnecessary switches (buttons) exposed at the location of the housing 110 are not visible or are difficult to see, giving a clean impression. In addition, the user 200 is less likely to obtain unnecessary information about the appearance from the outer perimeter of the display device 100, allowing them to concentrate on viewing the displayed aerial image 130 and increasing their sense of immersion in using the display device 100.

[0093] In Figure 4A(b), no operating elements such as switches are provided on the left side of the outer surface of the display device 100, and the left side of the display device 100 as a whole has a very smooth configuration. In Figure 4A(c), a first space 300 is formed on the outer circumferential surface of the display device 100, on the front side (user 200 side) of the display device 100, surrounded by four inner wall portions 400. In this embodiment, the four inner wall portions 400 are planar in shape and are substantially orthogonal to each other (they may be orthogonal). However, the connection points of each of the four inner wall portions 400 may be curved. Furthermore, the inner wall portion 400 may be composed of a single continuously curved inner wall portion 400, for example, a circular or elliptical shape.

[0094] In this embodiment, the inner wall portion 400 is inclined from the front side (user 200 side) towards the back side and toward the center. Therefore, the first space 300 has a truncated square pyramidal shape with the opening surface 700 as its base. This configuration will be explained in more detail using diagrams.

[0095] In Figure 4B(d), the sub-switch 320 and sensors 330 are provided on the right side of the outer surface of the display device 100. These configurations are the same as those already described in relation to Figure 3.

[0096] In Figure 4B(e), a power switch 340 and two connection ports are provided on the outer surface of the display device 100, specifically at a location exposed on the rear side. In this embodiment, one of the connection ports is a power jack for inserting a power cable 910, and the other connection port is for connecting, for example, a debug cable.

[0097] In Figure 4B(f), the bottom surface 1020 of the outer periphery of the display device 100 is provided with four anti-slip features (only one of which is labeled with a reference numeral) and legs 810 (see Figure 8).

[0098] The anti-slip material is preferably made of a substance such as rubber that has a higher frictional force than the material (e.g., plastic) that makes up the rest of the base surface 1020. It is even better if the anti-slip material protrudes slightly from the rest of the base surface 1020, as this concentrates the weight of the display device 100 on the protruding anti-slip material, increasing the frictional force of the anti-slip material against the contact surface, thereby preventing slipping and tipping.

[0099] In this embodiment, the legs 810 are configured to protrude toward the ground surface of the display device 100. The legs 810 may be configured as a tilt stand that protrudes toward the ground surface when rotated around a rotation axis from a state in which the user 200 is housed in the bottom area. In another embodiment, the legs 810 may be push-knob-shaped legs that protrude outward (for example downward) when pushed further by the user 200 while housed in the bottom area. If the leg portion 810 can be adjusted in stages or continuously (e.g., seamlessly) to adjust its protrusion width, the user 200 can adjust the tilt of the display device 100 more flexibly.

[0100] Figure 5 is a perspective view of the front case 111. Figure 5(a) is a perspective view of the front case 111 as seen from the front. Figure 5(b) is a perspective view of the front case 111 from the rear.

[0101] Figure 5 shows a perspective view of the front case 111 that forms the front surface of the outer periphery of the display device 100. As described above, the front case 111 has four inner wall sections 400 that surround a concave first space 300 that opens to the user 200 through the opening surface 700. The four inner wall sections 400 consist of an upper inner wall section 510, a left inner wall section 520, a lower inner wall section 530, and a right inner wall section 540. The front edges of the upper inner wall section 510, the left inner wall section 520, the lower inner wall section 530, and the right inner wall section 540 define the front opening surface 700 of the first space 300. The rear opening surface 710 of the first space 300 is closed by, for example, the front panel 1101 described later.

[0102] The four inner wall sections 400 are inclined such that the distance between opposing inner wall sections 400 decreases as the distance from the opening surface 700 (see Figure 7) to the back side of the display device 100 increases. This configuration, in which the inner wall sections 400 gradually narrow towards the back side of the display device 100, enhances the sense of depth that the user 200 perceives regarding the first space 300. With the above configuration, the user 200 can more easily see the inner wall 400 along with the aerial image 130 displayed within the opening surface 700, and can more easily perceive the depth of the first space 300 inside the housing 110. As a result, the user 200 can more easily get the feeling that the aerial image 130 is located close to them, and in combination with the first space 300 inside the housing 110, the user 200 can more easily perceive a stronger sense of three-dimensionality in the aerial image 130.

[0103] A slit-shaped opening is formed in the left inner wall portion 520. Hereafter, in this specification, such an opening will be referred to as a non-contact sensor opening 550, but this is just one example of an "opening".

[0104] On the left inner wall portion 520, a dummy groove 570L is formed parallel to the non-contact sensor opening 550, extending along the left inner wall portion 520 on the rear side (rear side) of the display device 100, beyond the non-contact sensor opening 550. Continuing from the dummy groove 570L, a dummy groove 570(570T) is formed on the upper inner wall portion 510. Continuing from the dummy groove 570(570T), a dummy groove 570(570R) is also formed on the right inner wall portion 540.

[0105] The configuration, in which dummy grooves 570 extending in parallel to the non-contact sensor opening 550 are formed along the left inner wall portion 520, the upper inner wall portion 510, and the right inner wall portion 540, makes it easier for the user 200 to recognize the non-contact sensor opening 550 as part of the design of the inner wall surface. In particular, the lower inner wall portion 530 is an inner wall portion that is relatively easy for the user 200 to see, similar to the left inner wall portion 520 and the right inner wall portion 540, but the dummy grooves 570 do not extend to the lower inner wall portion 530. Therefore, the dummy grooves 570, in combination with the non-contact sensor opening 550, are easily recognized as a symmetrical design. In another embodiment, dummy grooves may also be provided in the lower inner wall portion 530 so as to be continuous with the dummy grooves 570 provided in the left inner wall portion 520, the upper inner wall portion 510, and the right inner wall portion 540.

[0106] The dummy groove 570 is a portion that is recessed relatively upward on the upper inner wall portion 510, a portion that is recessed relatively to the left on the left inner wall portion 520, and a portion that is recessed relatively to the right on the right inner wall portion 540. These recesses are perceived as steps by the user 200, and together the non-contact sensor opening 550 and the dummy groove 570 are easily recognized as part of the design of the inner wall surface. Therefore, in another embodiment, a similar, or even enhanced or improved, visual effect can be obtained by providing a step-like step that is similarly perceived as a step, instead of or in addition to the dummy groove 570 described above. For example, in a configuration where a step-shaped step is provided instead of the dummy groove 570 described above, the step-shaped step is formed on the upper inner wall 510, the left inner wall 520, and the right inner wall 540 such that the rear side of the display device 100 protrudes more towards the first space 300 surrounded by the inner wall 400 than the front side, so that the user 200 can visually recognize the step. Furthermore, by designing the inclination of each inner wall (upper inner wall 510, left inner wall 520, lower inner wall 530, and right inner wall 540) to be constant before and after the step, the appearance of the display device can be simplified, and manufacturing costs can be reduced.

[0107] In a configuration where the inclination of each inner wall section is changed before and after the step, it becomes possible to design, for example, the direction of reflection of light emitted by the non-contact sensor (described later) on the surface of the inner wall section with greater flexibility. In yet another embodiment, for example, a step may be provided instead of the dummy groove 570 described above, and the upper inner wall portion 510 and the lower inner wall portion 530, and the left inner wall portion 520 and the right inner wall portion 540 may be designed to be parallel to each other on the front side of the step.

[0108] Furthermore, a sound-emitting hole 640, consisting of multiple openings for emitting sound from the speaker 1104, is formed at an angle to the front edge of the left inner wall portion 520. The sound-emitting hole 640 is an example of a sound-transmitting section. The sound-transmitting section is preferably a part that transmits sound more easily than the left inner wall portion 520, for example, by using a mesh-like material. The non-contact sensor 1102 in this embodiment is also called, for example, an Air Touch sensor, and can detect non-contact operations such as the movement of a user's fingers in the air. The opening 550 for the non-contact sensor corresponds to the length of one side of the detection area of ​​the non-contact sensor 1102, and light from the non-contact sensor 1102 passes through it. The non-contact sensor 1102 is an example of a reception section that receives input from a user.

[0109] Similar to the left inner wall portion 520, the right inner wall portion 540 also has a sound-emitting hole 640 for the speaker 1104, which is inclined relative to the front edge of the right inner wall portion 540. This sound-emitting hole 640 is also an example of a sound-transmitting section. The sound-transmitting section should be a part that allows sound to pass through more easily than the right inner wall portion 540, for example, by using a mesh-like material. An opening 560 for the main switch is formed in the lower inner wall portion 530.

[0110] As will be described later, the front case 111 in the embodiment of Figure 5 has a non-contact sensor 1102 along the vertical direction of the left inner wall portion 520, but for the sake of visual uniformity, it is also possible to form a similar slit-shaped opening as a dummy opening in the right inner wall portion 540. In another embodiment, a single continuous slit-shaped opening may be formed in the left inner wall portion 520, the upper inner wall portion 510, and the right inner wall portion 540, integrating the non-contact sensor opening 550 and the dummy opening.

[0111] The spacing between the opposing inner walls of the front case 111 gradually narrows from the opening surface 700 on the user 200 side towards the opening surface 710 on the rear side of the front case 111. This shape allows the surrounding frame (the four inner wall sections 400 (510, 520, 530, 540) of the front case 111) to give the user 200 a more emphasized sense of perspective in relation to the aerial image 130 displayed near the opening surface 700 on the user 200 side. As a result, user 200 is less likely to get the impression that the displayed aerial image 130 is floating more or that it is closer to the user than it actually is, beyond a greater depth.

[0112] Figure 6A shows examples of the external appearance of the front case 111 as viewed from various directions. Figure 6B shows examples of the appearance of the front case 111 from different directions than those shown in Figure 6A.

[0113] Figure 6A(a) is an example of a top view (plan view) of the front case 111. Figure 6A(b) is a left side view of the front case 111. Figure 6A(c) is a front view of the front case 111. Figure 6B(d) is a right side view of the front case 111. Figure 6B(e) is a rear view of the front case 111. Figure 6B(f) is a bottom view of the front case 111.

[0114] In Figure 6(a), the upper inner wall portion 510 of the front case 111 has, more specifically, a sound collection hole 620 and a microphone housing portion 610 formed on the side opposite to the inner wall surface of the inner wall portion 400 (510, 520, 530, 540) surrounding the first space 300 that opens to the user side of the display device 100 (hereinafter simply referred to as "the side opposite to the inner wall surface of inner wall portion 400," that is, the back side of the inner wall portion). The sound collection hole 620 is an example of a sound-transmitting portion. The sound-transmitting portion should be a part that allows sound to pass through more easily than the upper inner wall portion 510, for example, by using a mesh-like material.

[0115] In Figure 6A(b), the left inner wall portion 520 of the front case 111, more specifically on the side opposite to the inner wall surface of the inner wall portion, has a non-contact sensor opening 550, a sensor housing portion 630, a sound emission hole 640, and a speaker enclosure 650.

[0116] The other components of Figure 6A(c) are the same as those already described in relation to Figures 4A(c) and 5(a). The dummy groove 570 is formed along the left inner wall 520, the upper inner wall 510, and the right inner wall 540. Note that the dashed line B-B' shown in Figure 6A(c) is a hypothetical line used to explain the relationship with the cross-sectional view in Figure 7(a), and is not actually visible to the naked eye.

[0117] In Figure 6B(d), a sound vent 640 and a speaker enclosure 650 are formed on the right inner wall portion 540 of the front case 111, more specifically on the side opposite to the inner wall surface of the inner wall portion 400, similar to the left inner wall portion 520 of the front case 111 in Figure 6A(b).

[0118] Unlike the left inner wall portion 520 of the front case 111 shown in Figure 6A(b), the right inner wall portion 540 of the front case 111 shown in Figure 6B(d) does not have a non-contact sensor opening 550 and a sensor housing portion. However, as shown in Figure 6B(d), a dummy opening 660 may be provided in the right inner wall portion 540 of the front case 111 at the position indicated by the dashed line corresponding to the non-contact sensor opening 550 of the left inner wall portion 520, so as to give a symmetrical appearance when viewed from the user 200 side.

[0119] In embodiments that include such dummy grooves or other steps, when the display device 100 is viewed from the user 200's side, similar slit-shaped openings exist on both the left and right inner wall portions 400, resulting in a symmetrical appearance and making it easier to avoid the user 200 feeling any visual discomfort.

[0120] In the embodiment shown in Figure 6, the front case 111 is configured to have one sensor on the left inner wall portion 520. However, in another embodiment, a sensor may be provided on the upper inner wall portion 510 or the lower inner wall portion 530, or on the right inner wall portion 540. Furthermore, by providing two or more sensors, it is possible to increase the detection accuracy and the types of gestures that can be detected.

[0121] For example, by providing sensors at different distances from the opening surface 700 (see Figure 7) on the left inner wall portion 520 and the right inner wall portion 540, or by providing sensors at different distances from the opening surface 700 on the upper inner wall portion 510 and the right inner wall portion 540, it is possible to configure the device to detect the degree to which the user 200's finger is pressed in the depth direction (e.g., the amount (length) of the press, the speed of the press), for example, a pressing operation that causes the display device 100 to undulate in the front-to-back direction.

[0122] Figure 6B(e) shows the front case 111 viewed from the rear towards the front. Similar to Figure 6(a), the upper inner wall 510 of the front case 111 has a microphone housing 610, and the lower inner wall 530 of the front case 111 has an opening 560 for the main switch. Figure 6B(e) further shows the rear opening surface 710 of the first space 300, which is surrounded by the inner wall portions 400 (510, 520, 530, 540) of the front case 111.

[0123] Note that the dashed line C-C' shown in Figure 6B(e) is a hypothetical line used to explain the relationship with the cross-sectional view in Figure 7(b), and is not actually visible to the naked eye.

[0124] From Figure 6B(f), it can be seen that, more specifically, on the inner wall portion 530 at the bottom of the front case 111, on the side opposite to the inner wall surface of the inner wall portion 400, there is an opening 560 for the main switch, similar to that in Figure 6B(e).

[0125] In the embodiment shown in Figure 6, the sound vents 640 are provided in the lower regions of the left inner wall 520 and the right inner wall 540. However, in another embodiment, they may be provided in the central or upper regions of the left inner wall 520 and the right inner wall 540, for example. In the configuration where the sound vents 640 for the speaker 1104 are provided in the central regions of the left inner wall 520 and the right inner wall 540, the sound vents 640 for the speaker 1104 are positioned behind the sensor housing 630 (i.e., on the back side of the display device 100). The speaker 1104 is mounted on the front case 111 so that the sound emitted from the speaker 1104 is transmitted from the opening surface 700 of the first space 300 of the display device 100 to the user 200. This point will be explained in more detail with reference to Figure 11.

[0126] Figure 7 is a cross-sectional view of the front case 111. Figure 7(a) is a cross-sectional view of the front case 111 along the dashed line B-B' in Figure 6A(c). Figure 7(b) is a cross-sectional view of the front case 111 along the dashed line C-C' in Figure 6A(e).

[0127] In Figure 7(a), the upper inner wall portion 510 is inclined at an inclination angle At such that the first space 300 enclosed by the inner wall portions 400 (510, 520, 530, 540) of the front case 111 is reduced in size from the front side to the rear side of the display device 100. Similarly, the lower inner wall portion 530 is inclined at an inclination angle At such that the first space 300 enclosed by the inner wall portions 400 (510, 520, 530, 540) of the front case 111 is reduced in size from the front side to the rear side.

[0128] In the embodiment shown in Figure 7(a), the inclination angle At of the upper inner wall portion 510 is smaller than the inclination angle At of the lower inner wall portion 530. However, it is also possible to make the inclination angle At of the upper inner wall portion 510 and the inclination angle At of the lower inner wall portion 530 equal, or to make the inclination angle At of the upper inner wall portion 510 larger than the inclination angle At of the lower inner wall portion 530. In another embodiment, for example, by adjusting the inclination angle At of the upper inner wall portion 510 and the inclination angle At of the lower inner wall portion 530 so that the sum of the inclination angle At of the upper inner wall portion 510 and the inclination angle At of the lower inner wall portion 530 is within a range of a predetermined value, such as the viewing angle of the LCD 1003 used, the user 200's line of sight Du can be naturally guided within the viewing angle.

[0129] Furthermore, the inventors of this application have discovered, in connection with the space-saving configuration described with reference to Figure 10, that a very compact configuration can be achieved when the inclination angle At of the lower inner wall portion 530 is between 45 and 50 degrees, particularly around 48.5 degrees.

[0130] In Figure 7(b), the left inner wall 520 and the right inner wall 540 are inclined at an inclination angle Al for the left inner wall 520 and an inclination angle Ar for the right inner wall 540, such that the first space 300 enclosed by the inner wall 400 (510, 520, 530, 540) of the front case 111 is reduced from the front to the rear. In the embodiment shown in Figure 7(b), the inclination angle Al of the left inner wall portion 520 and the inclination angle Ar of the right inner wall portion 540 are equal in magnitude, but the left inner wall portion 520 and the right inner wall portion 540 may be inclined at different angles.

[0131] In another embodiment, for example, by adjusting the inclination angle Al of the left inner wall 520 and the inclination angle Ar of the right inner wall 540 so that the sum of the magnitudes of the inclination angle Al of the left inner wall 520 and the inclination angle Ar of the right inner wall 540 is within a range of a predetermined value, such as the viewing angle of the LCD 1003 used, the user 200's line of sight Du can be naturally guided within the viewing angle.

[0132] Figure 8 illustrates different tilt states of the display device 100 according to one embodiment. Figure 8(a) shows an example of the display device 100 in an upright position. Figure 8(b) shows an example of the display device 100 tilted to its maximum (25°) by the first tilt changing mechanism of the tilt changing unit 120. Figure 8(c) shows an example of the display device 100 tilted by a predetermined angle (5°) by the second tilt changing mechanism of the tilt changing unit 120. Figure 8(d) shows an example of the display device 100 tilted to a maximum of 25° + 5° = 30° using both the first and second tilt changing mechanisms.

[0133] In the embodiment shown in Figure 8, the tilt changing section 120 of the display device 100 has a first tilt changing mechanism and a second tilt changing mechanism. The second tilt changing mechanism is an example of the second tilt changing section.

[0134] As will be explained in more detail using Figure 11, the first tilt-changing mechanism consists of the support surface 1110 of the mount base 800 and the bottom case 1010 of the housing 110. The second tilt-changing mechanism consists of the leg portion 810 provided on the bottom surface 1020 of the mount base 800. The first tilt-changing mechanism and the second tilt-changing mechanism are independent of each other. For example, after tilting the first tilt-changing mechanism to its maximum extent, the second tilt-changing mechanism can be used to further tilt the display device 100 to the rear as viewed from the user 200. Furthermore, the mounting base 800, the legs 810, and the bottom surface 1020, like the front case 111, are treated to reduce light reflection, such as by a textured finish or a matte finish (i.e., a non-glossy surface).

[0135] In the embodiment shown in Figure 8, the bottom surface 1020 of the mount base 800 is also the bottom surface 1020 of the display device 100. Note that the mount base 800 is part of the base, and the mount base itself is sometimes referred to as the base. In Figure 8(a), user 200 is positioned on the left side of the page and looks at the display device 100 towards the right side of the page. In this case, the right side of Figure 8(a) is the "rear side from the user 200's perspective" and the "back side of the display device 100." The left side is the "front side of the display device 100."

[0136] The first tilt adjustment mechanism can continuously (for example, seamlessly) change the tilt of the housing 110 relative to the mount base 800 or the ground surface from the upright state shown in Figure 8(a) to the maximum tilt state shown in Figure 8(b). It is preferable to configure the first tilt adjustment mechanism to produce a click sensation or click sound when tilted to the maximum tilt state, as this makes it easier for the user 200 to avoid applying excessive force. In this case, the inclination of the housing 110 refers to the inclination of the aperture surface 700 or the imaging surface 140 of the aerial image 130 with respect to a straight line perpendicular to the ground surface of the display device 100.

[0137] In the embodiment shown in Figure 8, the user 200 can easily adjust the tilt of the housing 110 from the upright state in Figure 8(a) to the tilted state in Figure 8(c) using the second tilt changing mechanism. In the embodiment shown in Figure 8(c), the second tilt adjustment mechanism allows for tilt adjustment at a fixed angle (5°), but it can also be set to a larger or smaller angle. Furthermore, a configuration that allows for multi-stage angle adjustment or seamless and continuous angle adjustment can also be adopted.

[0138] Because the display device 100 has a first tilt adjustment mechanism and a second tilt adjustment mechanism, it is possible to effectively utilize the limited design space of the display device 100 and achieve a wide angle adjustment range. Therefore, the user 200 can easily adjust the display of the aerial image 130 to a desired tilt with a wider adjustment range, according to their own usage environment.

[0139] In the state where the legs 810, which are the second tilt-changing mechanism as shown in Figures 8(c) and 8(d), are extended, the display device 100 is in contact with the ground surface at the legs 810 and the rear side region of the bottom surface 1020. Similar to the anti-slip measures in Figure 4B(f), if the parts that come into contact with the ground surface in the states shown in Figures 8(c) and 8(d) are made of a material with greater friction, such as rubber, it is desirable that the display device 100 be installed more stably.

[0140] In the embodiment shown in Figure 8, both the left and right sides of the housing 110 are angled upwards towards the rear. This prevents the housing 110 from contacting the mounting base 800, the desk 230, the shelf, or other surfaces even when the housing 110 is tilted relative to the mounting base 800. This configuration minimizes the increase in height required to implement the tilt adjustment section 120. With this configuration, for example, a relatively large aerial image 130 can be displayed to the user 200 within the space required for installing the display device 100.

[0141] Furthermore, even if the degree of inclination is the same, for example, the user 200 can select between a first configuration in which the bottom surface 1020 of the display device 100 is grounded according to Figures 8(a) and 8(b), and a second configuration in which the legs 810 and the rear side region of the bottom surface 1020 of the display device 100 are grounded according to Figures 8(c) and 8(d).

[0142] Figure 9 is a diagram illustrating the relationship between a tilted display device 100 and the cables 910 and 920 connected to the display device 100 according to one embodiment. Figure 9(a) is a view of the tilted display device 100 shown in Figure 8(b), viewed from the back towards the front, parallel to the bottom surface of the display device 100. Figure 9(b) shows the tilted display device 100 as shown in Figure 8(b), viewed from the rear right side towards the front left side, parallel to the top surface of the display device 100.

[0143] In Figure 9, the bottom surface 1020 of the display device 100 is formed by the mount base 800, and two cables 910 and 920 are connected to the display device 100. As mentioned above, in the embodiment of Figure 9, the bottom surface 1020 of the mount base 800 (i.e., the bottom surface 1020 of the tilt adjustment section 120) also serves as the bottom surface 1020 of the display device 100. In the embodiment of Figure 9, the cables are a power cable 910 for power supply with power lines and a data cable 920 for data transmission and reception with signal lines, but other cables may also be used.

[0144] The tilt adjustment section 120 further has a recess 900 on its rear side. In this embodiment, the recess 900 is located on the rear side of the mounting base 800, beyond the housing 110, and is open upwards. The recess 900 houses the cable connector, and cables 910 and 920 are inserted into the connector portion of the display device 100 in the direction of the arrows in the diagram. Regardless of the tilt of the housing 110, they are inserted through the gap provided by the recess 900 on the back side of the display device 100, so they do not come into contact with the mount base 800 during insertion, or are less likely to come into contact.

[0145] Similarly, the cables 910 and 920 connected to the display device 100 are configured not to come into contact with, or to come into contact with, the tilt-changing section 120 (for example, the upper surface of the recess 900), regardless of the tilt of the housing 110. For example, since the connection part to the display device 100 is made of a rigid material that does not deform (including not deforming at all), the cables 910 and 920 can maintain a non-contact state with the tilt-changing section 120 within the space provided by the recess 900 on the back side of the display device 100, regardless of the tilt of the housing 110, under the force applied to the connection part of the cables 910 and 920 in a typical usage environment.

[0146] The rigid portions of cables 910 and 920 should have a length that allows them to extend outward from the recess 900 on the back side, regardless of the tilt of the housing 110, before transitioning to the flexible cable portion. Such a configuration makes it possible to suppress stress on cables 910 and 920 and the connection points (e.g., the jack) regardless of how the tilt of the housing 110 changes, thus making it easier to avoid damage to cables 910 and 920.

[0147] Figure 10 is a cross-sectional view of a display device 100 according to one embodiment. Figure 10 shows a cross-sectional view of the display device 100 as seen along the line A-A' in Figures 4A(c) and 4B(e).

[0148] The display device 100 has a housing 110 and a tilt changing section (first tilt changing mechanism and second tilt changing mechanism) 120 that constitute its outer periphery. The housing 110 has a front case 111 and a rear case 112 as components visible to the user 200. The housing 110 further has a bottom case 1010 that functions as a sliding part, which also serves as part of the first tilt changing mechanism.

[0149] In the embodiment shown in Figure 10, the front case 111 forms the front side (user side 200) of the housing 110 of the display device 100. The rear case 112 forms the top surface, both left and right sides, and the back surface of the housing 110 of the display device 100. The bottom case 1010 forms the bottom surface of the housing 110. The display unit 1000 is housed in the internal space of the housing 110 defined by the front case 111, the rear case 112, and the bottom case 1010.

[0150] In the embodiment shown in Figure 10, the display unit 1000 includes a lens plate 1001, a main board 1002, an LCD (liquid crystal display) 1003, and an LCD holder 1004. In this specification, "main board" refers to an example of a "control board," and "LCD" refers to an example of a "display."

[0151] In the embodiment shown in Figure 10, LCD 1003 is a 5-inch liquid crystal display that is longer in the vertical direction than in the horizontal direction. However, the display can be any configuration having a display surface that emits image light, and for example, an organic EL display or a display employing a digital mirror device may be used. LCD 1003 is an example of a "display unit". In the embodiment shown in Figure 10, the distance D1 from the surface of the LCD 1003 to the center of the lens plate 1001 is equal to the distance D2 from the center of the lens plate 1001 to the image plane 140. Also, in the embodiment shown in Figure 10, the position of the image plane 140 is approximately equal to the position of the aperture surface 700 of the front case 111.

[0152] In another embodiment, the position of the imaging surface 140 may be designed to be offset from the position of the aperture surface 700 of the front case 111. In yet another embodiment, the position of the imaging surface 140 may be designed to be changeable from the position of the aperture surface 700 in the front case 111. In such a configuration, the user 200 can make adjustments to better suit their preferences.

[0153] In the embodiment shown in Figure 10, the distance D1 from the surface of the LCD 1003 to the center of the lens plate 1001 and the distance D2 from the center of the lens plate 1001 to the image plane 140 should both be selected within a range of approximately 30 mm to 40 mm. By appropriately selecting the above distances D1 and D2, it is possible to make the edges and dots of the aerial image 130 less noticeable and to reduce the blurring of the aerial image 130.

[0154] The inclination of the inner wall portions 400 (510, 520, 530, 540) of the front case 111 changes at an angle less than or equal to the viewing angle of the LCD 1003. However, if it is approximately the same as the viewing angle, it is good because it can guide the user 200's line of sight to a range in which the aerial image 130 can be clearly seen. This effectively reduces the situation in which the user 200 views the aerial image 130 at an angle, or in which the image is difficult to see. Furthermore, because the inner wall portions 400 (510, 520, 530, 540) of the front case 111 are inclined inward, the outer frame (also called the bezel area) of the LCD 1003 located inside the housing 110 can be positioned in the user 200's blind spot.

[0155] In this specification, "lens plate" refers to an example of an "optical system." The lens plate 1001 is configured, for example, as a microlens array plate in which multiple minute lenses are arranged. The lens plate 1001 is an optical component also called a 3D lens. Each minute lens projects an image or video displayed by a display such as the LCD 1003 onto the opening side of the front case 111, thereby displaying an aerial image 130 to the user 200.

[0156] A display method that combines a display and an optical system makes it possible to display aerial images at a position different from the display surface. In this embodiment, a first space 300 can be provided between the optical system composed of the lens plate 1001 and the aerial image 130 (image plane 140). The first space 300 is mainly located behind the aerial image 130, but it has a variety of uses and effects, such as providing the user 200 with an enhanced sense of depth as explained in relation to Figure 5, being used as a propagation space for sound output by the speaker 1104 described later, being used as a path for accessing the operation section such as the main switch 310 formed as a touch sensor, and being used as a space for providing a member to hide the internal components of the display device 100.

[0157] Furthermore, the fact that the optical system is composed of a lens plate 1001 and does not have a reflector contributes to making it easier to implement the display device 100 in a vertically elongated housing structure such as a rectangular parallelepiped.

[0158] The display method for the aerial image 130, which combines the LCD 1003 and the lens plate 1001, does not require electrical or electronic conversions in the process of converting the image light emitted by the LCD 1003 (i.e., the image displayed by the LCD 1003) into the aerial image 130. Therefore, it is possible to display the aerial image 130 with a simple configuration. In addition, the total number of components of the entire display device 100, the computational load, and the power consumption can be reduced, thereby reducing manufacturing costs and operating costs.

[0159] In the embodiment shown in Figure 10, the distance D3 from the center of the lens plate 1001 to the center of the non-contact sensor aperture 550 is shorter than the distance D2 from the center of the lens plate 1001 to the image plane 140. In another embodiment, the distance D3 from the center of the lens plate 1001 to the center of the non-contact sensor aperture 550 can also be designed to be equal to the distance D2 from the center of the lens plate 1001 to the image plane 140.

[0160] The main circuit board 1002, located on the rear side of the center of the display device 100, is positioned upright and approximately parallel (including parallel; the same applies hereinafter) to the lens plate 1001 and the LCD 1003. Therefore, in this embodiment, the control circuit board is provided in a space formed behind the imaging plane 140 of the aerial image 130, such that its surface is approximately parallel to the imaging plane 140 of the aerial image 130. The main board 1002 is arranged vertically and housed within the casing 110. By arranging the main board 1002 almost parallel to the lens plate 1001 and LCD 1003, it is possible to reduce the number of components that extend in the front-to-back direction within the display device 100, thereby allowing for more flexible use of space.

[0161] In this embodiment, by arranging the main board 1002 vertically, the size of the control board can be increased according to the size of the imaging surface 140 of the aerial image 130 while suppressing an increase in the dimensions of the display device 100 as seen from the user 200 side. For example, it is possible to create a housing space for the main board 1002 at the bottom or top of the LCD 1003 and install the control board within such a housing space. However, in this case, it is necessary to secure a larger space in the front-to-back and left-to-right directions of the display device 100, which increases the size of the display device 100. In contrast, with a display device 100 in which the main board 1002 is mounted vertically, as described above, the enlargement of the top can be suppressed, resulting in a more compact appearance and a more desirable aesthetic appearance.

[0162] The internal space of the housing 110 further includes a weight 1030 and a supercapacitor 1070 (also called an electric double-layer capacitor) as a means of energy storage, located above or above the bottom case 1010, and below the center of gravity of the display device 100 (towards the bottom case 1010).

[0163] In the embodiment shown in Figure 10, the weight 1030 and the supercapacitor 1070, which have a relatively high specific gravity compared to the other components, are provided below the display device 100. This lowers the center of gravity of the entire display device 100 and improves its stability. As a result, the display device 100 can maintain a stable state regardless of the tilt of the housing 110 of the display device 100. This also helps to prevent tipping over.

[0164] The supercapacitor 1070 supplies power to ensure that user information and other data are reliably retained even in the event of an unexpected power loss, and can be used to close files in such a situation. A rechargeable battery may be used instead of or in addition to the supercapacitor 1070. The supercapacitor 1070, like the weight 1030, is located below the display device 100. Therefore, the supercapacitor 1070 also functions as a weight that lowers the center of gravity of the entire display device 100, thereby further improving the stability of the display device 100's posture.

[0165] Furthermore, if the weight 1030 and the supercapacitor 1070 are placed in front of the center of gravity of the display device 100 (towards the front case 111), the display device 100 can be installed even more stably, regardless of the tilt of the housing 110 of the display device 100.

[0166] A tilt adjustment section 120 is provided below the housing 110. More specifically, below the bottom case 1010, which is part of the housing 110 and part of the tilt adjustment section 120, a mount base 800 is provided that, together with the bottom case 1010, constitutes the first tilt adjustment mechanism of the tilt adjustment section 120. The mount base 800 is an example of a base, and in the embodiment shown in Figure 10, the bottom case 1010 is mounted on the mount base 800.

[0167] The front case 111 has an opening 550 for a non-contact sensor. Through the opening 550, the non-contact sensor 1102, such as an IR (infrared) sensor or a photoelectric sensor, emits light toward its detection range (hereinafter also referred to as the "detection area"). The surface treatment of the front case 111 should be performed in a manner that reduces false detections by the non-contact sensor 1102. The front case 111 further has an opening 560 for the main switch.

[0168] The main switch 310 is positioned so as not to overlap with the detection range of the non-contact sensor 1102. More specifically, in the embodiment shown in Figure 10, the upper end position 1060 of the main switch 310 is located below the lower end position 1050 of the detection area of ​​the non-contact sensor 1102. This configuration effectively reduces the possibility that the non-contact sensor 1102 might detect an input operation on the aerial image 130, even if the user 200 is attempting to operate the main switch 310.

[0169] Since the non-contact sensor 1102 is located inside the front case 111 and on the front side, the entire left-right direction of the display device 100 can be covered as a detection target. In the up-down direction of the display device 100, a portion of the lower area is excluded from the detection range, so that, for example, when the main switch 310 is pressed, the non-contact sensor 1102 does not react. It is preferable that the size of the area excluded from the detection range be changeable or adjustable through updates to the display device 100 (including updates to the firmware of electronic components included in the display device 100) or user settings. Additionally or alternatively, it is even better if the upper area of ​​the display device 100, not just the lower area, can also be excluded from the detection range in the up-down direction.

[0170] In the area shown by diagonal lines below the front case 111 in Figure 10, the space between the bottom surface 1020 and the lower inner wall portion 530 increases as the lower inner wall portion 530 moves from the opening surface 700 on the user 200 side toward the rear side of the display device 100. The curved surfaces of the bottom case 1010 and the mount base 800 each extend to the vicinity of the lower inner wall portion 530 of the front case 111 within the second space 1080 (1080b) enclosed by the dashed line. By having a part of the first tilt changing mechanism of the tilt changing section 120 house within the second space 1080 (1080b), a space-saving and compact configuration of the display device 100 can be achieved.

[0171] In the embodiment shown in Figure 10, the microphone 1040 is located in an area near the center of the upper inner wall portion 510 in the left-right direction. In this configuration, the microphone 1040 is mounted such that its directivity points downward from the horizontal, regardless of the tilt of the housing 110. In another embodiment, the microphone 1040 may be located in an area near the center of the upper inner wall portion 510 in the left-right direction, and in such a configuration, the microphone 1040 is mounted such that its directivity points upward from the horizontal, regardless of the tilt of the housing 110.

[0172] In the embodiment shown in Figure 10, the microphone 1040 described above may have an elliptical sound collection range that can collect sound with good sensitivity. In another embodiment, the microphone 1040 may change the shape of the range that can collect sound with good sensitivity, for example, from an elliptical shape to a circular shape, depending on the change in the tilt of the display of the aerial image 130.

[0173] With the above configuration, the display device 100 can detect the voice of the user 200 viewing the aerial image 130 with higher accuracy using the microphone 1040. More specifically, it becomes easier to collect the voice of the user 200 using the display device 100 regardless of the tilt of the aerial image 130, and less likely to collect voices from other directions (e.g., noise or the voices of people other than the user 200). As a result, the occurrence of misrecognition in the voice recognition function is suppressed, and conversations and interactions between the user 200 and the character become smoother. However, in another embodiment, if there is an intention to collect voices from a wide range, a microphone 1040 with wide directivity may be used.

[0174] In yet another embodiment, a drive unit may be provided to adjust the position of the microphone 1040 so that its directivity is directed more toward the user 200, in accordance with the tilt of the aerial image 130 display. This makes it possible to detect the voice of the user 200 viewing the aerial image 130 more clearly.

[0175] The above configuration is not limited to the microphone 1040, but can also be applied to other directional sensors. Such sensors may include, for example, a camera that detects the gaze and movement of user 200, a pyroelectric sensor that changes the brightness of the display depending on the presence or absence of user 200 for energy saving (an example of a human presence sensor), or a thermal camera that detects the body temperature of user 200.

[0176] As explained using Figure 5, the inner wall portions 400 (510, 520, 530, 540) of the front case 111 are inclined such that the distance between opposing inner wall portions 400 (510 and 530, 520 and 540) decreases as the distance from the opening surface 700 on the user 200 side toward the back of the display device 100 increases. The second space 1080 (1080t, 1080b) existing between the inner wall portion 400 and the outer surface of the display device 100 (which may also be called the outer wall portion) expands from the opening surface 700 on the user 200 side toward the back of the display device 100, as the distance from the opening surface 700 increases, in accordance with the reduction in the distance between the opposing inner wall portions 400 (510 and 530, 520 and 540). By arranging the components of the display device 100 within such a second space 1080 (1080t, 1080b), it becomes possible to achieve a very compact and space-saving configuration.

[0177] In the embodiment shown in Figure 10, a microphone 1040 is located in the second space 1080 (1080t), which is shown by a dashed line at the top. Furthermore, the main switch unit is located in the second space 1080 (1080b), which is shown by a dashed line at the bottom. In addition, a part of the tilt adjustment section 120 also extends to the second space 1080 (1080b). For example, the convex guide 1120 formed on the support surface 1110 of the mount base 800 (described later) and the concave notch (not shown) formed complementary to the bottom case 1010 may also extend to the second space 1080 (1080b).

[0178] This configuration allows components such as parts to be efficiently arranged within the second space 1080 secured between the inclined inner wall portion 400 (510, 520, 530, 540) and the outer surface of the display device 100, thereby enabling a miniaturization of the entire display device 100. The second space 1080 also exists to the right and left of the display device 100 and is used to arrange non-contact sensors 1102, speakers 1104, and the like.

[0179] The above configuration, in which the inner wall portions 400 (510, 520, 530, 540) of the front case 111 are inclined, makes it possible to give the user 200 a sense of depth and floating through the aerial image 130. In addition, it is possible to realize a compact display device 100 configuration even with a tilt adjustment unit 120 for tilting the display of the aerial image 130.

[0180] Figure 11 is an exploded view of a display device 100 according to one embodiment. Similar to the embodiment shown in Figure 10, the housing 110 consists of a front case 111 and a rear case 112, the display unit 1000 consists of a lens plate 1001, a main circuit board 1002, an LCD 1003, and an LCD holder 1004, and the tilt adjustment section 120 consists of a bottom case 1010 and a mount base 800. In the embodiment shown in Figure 11, the mount base 800 has a support surface 1110 and a guide 1120 on its upper curved surface, and a leg portion 810 on its lower bottom surface 1020.

[0181] The display unit 1000 is housed in the space within the enclosure 110, and the main board 1002 of the display unit 1000 is fixed to the rear case 112 on the back side of the display device 100 using screws. It is preferable that the main board 1002 be covered with a resin such as rubber, as this can protect it from the physical force applied when adjusting the tilt of the display.

[0182] The convex guide 1120 formed on the support surface 1110 of the mount base 800 engages with a concave notch (not shown) formed complementary to the bottom case 1010, assisting the bottom case 1010 to slide on the support surface 1110 of the mount base 800 in only one direction (the front-to-back direction of the display device 100).

[0183] In Figure 11, in addition to the display unit 1000, the housing 110 also houses a weight 1030, a non-contact sensor 1102, a microphone 1040, left and right speakers 1104, a button unit, a supercapacitor 1070, and the like. The housing 110 can also house, for example, a temperature sensor 1631 (see Figure 16), an illuminance sensor 1632 (see Figure 16), a humidity sensor, a pressure sensor (barometric pressure sensor), a carbon dioxide concentration sensor, a carbon monoxide concentration sensor, a TVOC (total volatile organic compound) sensor, a PM2.5 sensor, an ozone sensor, an acceleration sensor, a camera, and the like.

[0184] The weight 1030 is positioned on or above the bottom case 1010, below the center of gravity of the display device 100, for example, using double-sided tape. This makes it possible to install the display device 100 in a more stable state regardless of the tilting orientation.

[0185] By appropriately selecting the ratio of the weight of the weight 1030 to the weight of the display device 100, the curvature of the curved surfaces of the bottom case 1010 and the mount base 800, the material properties of the bottom case 1010 and the mount base 800, particularly the coefficient of friction, the width of the convex guide 1120 and the width of the concave notch, etc., stable stops can be achieved in any seamless and stepless inclination state of the housing 110. For example, the weight of the weight 1030 may be about 10% of the weight of the display device 100.

[0186] By selecting the width of the convex guide 1120 of the mount base 800 and the width of the concave notch of the bottom case 1010 so that the frictional force between them becomes a desired magnitude, more reliable stable stops of the housing 110 can be realized.

[0187] The speaker 1104 and the microphone 1040 are housed in the space between the opening surface 700 and the lens plate 1001 and between the inner wall portion 400 (510, 520, 530, 540) of the housing 110 and the outer peripheral surface of the housing 110. As detailed in relation to FIGS. 7(a) and 7(b), the four inner wall portions 400 (510, 520, 530, 540) of the front case 111 are inclined inward as they go from the front side to the back side, so that as the first space 300 gradually decreases due to the inclination of the inner wall portion 400 (510, 520, 530, 540), the second space 1080 between the corresponding inner wall portion 400 (510, 520, 530, 540) and the outer peripheral surface of the housing 110 increases correspondingly.

[0188] By arranging the speaker 1104 and the microphone 1040 in the second space 1080 between the inner wall portion 400 (510, 520, 530, 540) that increases as the distance from the opening surface 700 increases and the outer peripheral surface of the display device 100, the display device 100 can be made into a particularly space-saving and compact configuration. The speaker 1104 is configured as a sealed speaker by being housed in a sealed space surrounded by the speaker enclosure 650 and the lens plate 1001 of the front case 111. The speaker 1104 may be a dynamic speaker.

[0189] A part of the sound output from the speaker 1104 in the embodiment of FIG. 11 propagates directly in the direction of the user 200. Also, the remaining part of the sound output from the speaker 1104 propagates in the direction of the user 200 after being reflected at least once by the inner wall portions 400 (510, 520, 530, 540) of the front case 111 and the front case 111. Since at least a part of the reflected sound passes through the imaging surface 140 and propagates in the direction of the user 200, the user 200 can obtain a feeling that sound is output from the aerial image 130 such as a character displayed on the imaging surface 140.

[0190] In the embodiment of FIG. 11, a main switch unit (main button unit) is constituted by the main switch 310, the LED panel 1105, the milky white light guide plate 1106, the first sub-substrate (also referred to as a sensor substrate) 1107, and the like. The main switch unit is, for example, a capacitive touch sensor, and its detection range may be configured to be different from the size visible by the LED panel 1105, and in particular, may be configured to be larger than the size visible by the LED panel 1105. Thereby, while providing a comfortable touch operation range for the user 200, the design of the display device 100 can be determined flexibly.

[0191] The main switch unit is a switch for controlling a major predetermined function, such as the voice recognition of the display device 100. When user 200 presses (touches) the main switch 310, the display device 100 starts voice recognition. Voice recognition is local voice recognition performed using the voice recognition engine implemented on the main board 1002, but voice recognition may also be performed on the voice recognition server side by connecting to a voice recognition server via communication. Furthermore, the display device 100 changes the color of the light around the main switch 310 according to the status of the voice recognition process. This allows user 200 to easily understand the status of the voice recognition process.

[0192] The color of the light around the main switch 310 indicates, for example, when the always-on voice recognition mode (described later) is OFF. The indicator light should be green in the normal state, blue when the voice recognition mode is always ON or when the main switch 310 is pressed and the voice recognition is in standby mode, red when voice recognition is being processed (for example, during voice recognition), and light blue when the character's voice is being output from the display device 100 (for example, when the character is speaking).

[0193] Furthermore, the main predetermined function controlled by the main switch unit may be, for example, a gesture recognition control function instead of a voice recognition control function, or a function that combines both a voice recognition control function and a gesture recognition control function, as well as a function to switch between them. The gesture recognition function uses a camera provided in the display device 100 to recognize gestures such as sign language and body language of the user 200. With such a configuration, even a user 200 who has difficulty speaking can fully operate the display device 100.

[0194] In the embodiment shown in Figure 11, a front panel 1101 is attached to the user 200 side of the lens plate 1001. The front panel 1101 is an optical element that has both light-transmitting and light-blocking properties. The front panel 1101 is preferably a light-blocking sheet (also called a smoke panel or smoke sheet) with a light-blocking property of about 10% transmittance, and a light-blocking sheet with a light-blocking property of about 6% transmittance is preferable, as it makes it more difficult for the user 200 to recognize internal components such as the frame area of ​​the LCD 1003, and is even better for the user 200 to recognize the floating state of the aerial image 130. The color of the front panel 1101 is gray, but other colors are also acceptable. The optical properties of the front panel 1101 are selected for the purpose of maintaining the first space 300 in a dark state. This is because it affects the sense of floating of the aerial image 130 given to the user 200. By using such a front panel 1101, it is possible to create a configuration in which it is difficult for the user 200 to see the components housed inside the display device 100. In this specification, "front panel" refers to an example of a "dimming member."

[0195] As mentioned above, the front panel 1101 is better if it has a matte or textured surface treatment (i.e., a non-glossy finish) which reduces the reflectivity of light, thus avoiding reflections of the user 200. For example, the front panel 1101 is preferably made of a material with a lower reflectivity of specular reflection than the reflectivity of specular reflection of the lens plate 1001. This reduces the reflection of the user 200 on the lens plate 1001 located inside the housing 110, making it easier for the user 200 to concentrate on the aerial image 130. Instead of directly applying a surface treatment to the front panel 1101, an anti-reflective film may be applied to the front panel 1101 in addition.

[0196] Figure 12 is an exploded view illustrating the relationship between the front case 111, the speaker 1104, and the speaker enclosure 650. Figure 12(a) is a perspective view of an enlarged exploded view of the front case 111, speaker 1104, and speaker enclosure 650, seen from the front. Figure 12(b) is a perspective view of an enlarged exploded view of the front case 111, speaker 1104, and speaker enclosure 650, seen from the rear.

[0197] In Figure 12(a), a speaker housing 1210 is formed integrally with the front case 111 on the opposite side of the inner wall 400 (the back side of the inner wall 400). In the embodiment shown in Figure 12(a), the speaker 1104 is housed in the speaker housing 1210 and covered by the sealing plate 1103, thereby forming a sealed speaker enclosure 650 (speaker box).

[0198] The speaker 1104 may be positioned in the speaker housing 1210 with the magnet side facing the outer surface and the cone side facing the opposite side (the inner wall 400 side), with the sound emission hole 640 side facing the sound emission direction. Alternatively, the speaker may be positioned with the magnet side facing the inner wall 400 side and the cone side facing the opposite side (the outer surface side), with the sound emission hole 640 side facing a different direction. The speaker 1104 and speaker enclosure 650 may be configured as a sealed speaker as described above, but they may also be configured as a bass reflex speaker, a Kelton speaker, etc. It is desirable that they can emit sound with sufficient power even when there are space constraints for securing the speaker 1104 and speaker enclosure 650 for purposes such as miniaturizing the display device 100. The speaker 1104 and speaker enclosure 650 should be configured considering the type and characteristics of sound to be output by the display device 100, such as character voices and music.

[0199] The sealing plate 1103 is, for example, a plate-shaped member made of plastic, and is fixedly connected to the front case 111 by means of fixing based on adhesion using adhesive or double-sided tape, fixing based on shape such as fitting, or fixing based on frictional force generated by pressing the sealing plate 1103 into the corresponding part of the front case 111.

[0200] Figure 12(b) shows that, in addition to the configuration shown in Figure 12(a), the speaker enclosure 650 extends along two different inner wall sections 400 of the front case 111, namely the left inner wall section 520 and the lower inner wall section 530, and the right inner wall section 540 and the lower inner wall section 530. The right inner wall section 540 and the lower inner wall section 530 are connected to each other and are an example of two inner wall sections 400 facing in different directions. Figure 12(b) shows the x, y, and z axes added along the wall surface of the speaker enclosure 650.

[0201] Figure 13 is a diagram illustrating the shape of the speaker enclosure 650. Figure 13(a) is a diagram illustrating the shape of the speaker enclosure 650 as seen in the yz plane of Figure 12(b). Figure 13(b) is a diagram illustrating the shape of the speaker enclosure 650 as seen in the xz plane of Figure 12(b).

[0202] In Figure 13(a), the lower part of the speaker enclosure 650 extends further towards the rear than the upper part of the speaker enclosure 650. In Figure 13(b), the lower part of the speaker enclosure 650 has an L-shape, extending further toward the center of the lower inner wall portion 530 (towards the main switch opening 560) than the upper part of the speaker enclosure 650.

[0203] The speaker enclosure 650 surrounds the third space 1200 with a shape that extends along two interconnected inner walls, thereby forming a larger sealed speaker 1104 within the display device 100. In order to secure a wider third space 1200, the speaker enclosure 650 may also have a shape that extends along, for example, three or more inner walls 400, such as the left inner wall 520, the lower inner wall 530, and the right inner wall 540.

[0204] FIG. 14 is a perspective view of the shape of the lower part of the front case 111. FIG. 14(a) is a perspective view of the lower part of the front case 111 as viewed from the back side. FIG. 14(b) is a perspective view of the lower part of the front case 111 as viewed from the front side.

[0205] In FIG. 14(a), the lower part of the speaker enclosure 650 extends longer to the back side than the upper part of the speaker enclosure 650, and on the back side of the lower part of the speaker enclosure 650, it extends longer to the center side (main switch opening 560 side) of the lower inner wall part 530 on the side opposite to the inner wall surface of the lower inner wall part 530.

[0206] In FIG. 14(b), the upper part of the speaker enclosure 650 has a speaker housing part 1210, and below the speaker housing part 1210, the third space 1200 has an L-shaped form that extends longer to the center side (main switch opening 560 side) of the lower inner wall part 530. The speaker housing part 1210 is a housing space for the speaker 1104 provided within the third space 1200.

[0207] In the embodiment of FIG. 14, the speaker 1104 housed in the speaker housing part 1210 may be provided at a position where the sound emitting hole 640 is formed in the speaker housing part 1210, but it may also be provided at a different position. Also, although the speaker 1104 is attached substantially parallel to the left and right outer peripheral surfaces of the display device 100, the speaker 1104 housed in the speaker housing part 1210 may be attached substantially parallel to the inclination of the left inner wall part 520 or the right inner wall part 540. Also, depending on the selection of the material of the left inner wall part 520, the right inner wall part 540, etc., an application where these function as a diaphragm to improve the power of the audio output is also conceivable.

[0208] The configuration described using Figures 12 to 14 allows for a more spatially efficient design and the placement of a larger volume speaker enclosure 650 compared to configurations where the speaker enclosure 650 is designed in a simple cube or rectangular prism shape. This configuration suppresses the increase in the external dimensions of the display device 100, and therefore enables miniaturization of the housing 110. Furthermore, this speaker enclosure 650 makes it easier to improve the sound quality from the speaker 1104 and to ensure sufficient power for low-frequency audio output.

[0209] Furthermore, because the left and right inner walls 520 and 540 are inclined, sound is emitted from the left and right inner walls 520 and 540 toward the opening, and the sound waves converge in the center. This allows the display device 100 to output the speech sounds of the character displayed as an aerial image 130 with higher sound quality, and as if the character itself, located in the center of the image-forming surface 140, were actually speaking. As a result, the user 200 can perceive the aerial image 130 with a greater sense of immersion.

[0210] Figure 15 is a diagram illustrating the connection state of the electrical components of a display device 100 according to one embodiment. In the embodiment shown in Figure 15, the main board 1002 is connected to a non-contact sensor 1102, a microphone 1040, a speaker 1104, a first sub-board (a board for a capacitance sensor) 1107, a second sub-board (a board for a tact switch) 1500, and a supercapacitor 1070. As mentioned in relation to Figure 11, the vertical mounting of the main board 1002 allows other components to be arranged under relatively loose spatial constraints, particularly at the top and bottom of the display device 100, thereby improving design flexibility. Furthermore, the above configuration allows for a space-saving and slim appearance at the top and bottom of the display device 100.

[0211] Figure 16 is a system block diagram illustrating the electrical configuration of a display device 100 according to one embodiment. In the embodiment shown in Figure 16, the main board 1002 includes a processor module 1600, a communication module 1650, an eMMC memory 1610 (e.g., flash memory), a DDR memory 1620, a communication module 1650, and various connectors, etc. The main board 1002 is connected via connectors to a non-contact sensor 1102, an LCD 1003, a microphone 1040, a speaker 1104, a second sub-board 1500, an antenna 1660, a power adapter 1640, and other components.

[0212] The processor module 1600 has, for example, a CPU and a GPU, and controls various parts of the main board 1002. The processor module 1600, or the main board 1002 having it, functions as a control unit that is responsible for controlling the display device 100, and can therefore be read as the control unit of the display device 100. The display device 100 may further have two or more second sub-boards 1500. For example, the first sub-board 1107 may have a touch key circuit 1633, etc., and the second sub-board 1500 may have a temperature sensor 1631, an illuminance sensor 1632, a touch key circuit 1633, etc.

[0213] The main board 1002, the first sub-board 1107, and the second sub-board 1500 are mounted, for example, as a Printed Circuit Board Assembly (PCBA). It is preferable that the PCBA be covered with a resin such as rubber, as this protects the PCBA from the physical force applied when changing the tilt of the display. Furthermore, in a configuration where the main board 1002, the first sub-board 1107, and the second sub-board 1500 are composed of flexible printed circuit boards (FPC boards) or rigid-flexible circuit boards, in addition to rigid boards, the manufacturing of the display device 100 is facilitated, and the effects of movement of the tilt changing section 120 are reduced.

[0214] The main board 1002 has an RTC (Real-Time Clock) 1630 as a means of measuring time. The RTC 1630 has a battery for the RTC, which is provided separately from the supercapacitor 1070, and measures time by receiving power from this RTC battery. For example, if the display device 100 is configured to be connected to the internet, it is necessary to retain the time until the time is obtained from an NTP server (for example, from the time the display device 100 is powered on until it connects to the internet and the NTP server). The main board 1002 displays the time, for example, in a time notification function, so it has a battery for the RTC to retain time information even when offline.

[0215] This section describes the control performed by the main board 1002 using the supercapacitor 1070. The presence of the supercapacitor 1070 prevents malfunctions such as file corruption in the display device 100 even if there is a power outage where the external power supply via the power cable 910 is interrupted. The main board 1002 may have a function to receive power from the supercapacitor 1070 in the event of a power outage and perform processing to protect the file system. The supercapacitor 1070 is charged at predetermined timings. For example, the main board 1002 may perform power control to charge the supercapacitor 1070 when the power is turned on.

[0216] The main board 1002 may have a function to change the content of communication between the character and the user 200 according to the user 200's usage history of the display device 100, such as the history of functions related to communication between the character and the user 200. Changing the content of communication may, in particular, increase the variety of communication. For example, the main board 1002 may change the movements of the character displayed by animation or the phrases spoken by the character.

[0217] The main board 1002 may update a parameter indicating the relationship between user 200 and a character, for example, depending on whether user 200 has performed an action that increases or decreases the character's favorability. The relationship between user 200 and the character may include elements related to emotions such as human relationships, and may relate to the degree of favorability or intimacy, but it may be particularly good to include the character's mood towards the user. The main board 1002 may store such a parameter indicating the relationship between user 200 and the character (for example, intimacy) in the eMMC memory 1610 as flash memory (an example of non-volatile memory), and may have a function to update that parameter.

[0218] In addition to the above, it would be even better to include at least one of the following as conditions for updating the parameters: the time elapsed since user 200 started using the display device 100, the operations performed by user 200, and sensing of the surrounding conditions using sensors 330.

[0219] The main board 1002 should update the parameters stored in the eMMC memory 1610 based on the latest status in the event of a power outage. This prevents malfunctions that could occur due to data corruption caused by a power outage, such as the loss of the user's (user 200's) accumulated time with the character (for example, the time spent raising the character), including the history of communication between the character and the user 200.

[0220] The electrical components shown in Figure 16 are merely examples, and as mentioned in relation to Figure 9, the display device 100 may also have, or in place of, a humidity sensor, an acceleration sensor, a camera, etc., in addition to the temperature sensor 1631 and the illuminance sensor 1632. In embodiments where the display device 100 has a camera, the display device 100 can further use the camera to perform functions such as detecting the user 200's gaze, motion detection, and room monitoring, as well as further functions utilizing these functions.

[0221] The main board 1002 may also have connectors for wired connections during debugging, sockets for external storage devices such as microSD cards, and connection points for PCs, smartphones, etc.

[0222] [3. Example of software processing of the display device 100] Next, an example of software (program) processing performed by the display device 100 will be explained. In the following, we will explain the case where the parameter indicating the relationship between the user and the character is a parameter indicating the character's mood towards the user. In this case, actions that decrease the favorability rating are actions that make the character uncomfortable (in this embodiment, actions that make the character unhappy or angry), and actions that increase the favorability rating include apologizing to improve the character's mood (for example, calming their anger and returning them to their normal mood). In the following explanation, "user 200" may be used as appropriate when referring to the user of the display device 100.

[0223] <3-1. Examples of data to be stored in the display device 100> First, an example of data to be stored in the display device 100 will be described with reference to Figure 17. Figure 17 is a diagram showing an example of data stored in the storage unit 1670 of the display device 100. The storage unit 1670 includes, for example, an eMMC memory 1610, which is an example of flash memory, and a DDR memory 1620, which is an example of SDRAM, but may also include other storage media. Examples of storage media include optical recording media, magnetic recording media, semiconductor recording media, or other recording media.

[0224] The memory unit 1670 may store, for example, an OS (Operating System) 251 and an application program (hereinafter referred to as "application") 252 that runs on the OS 251. The OS 251 may be, for example, Android (registered trademark), but other operating systems may also be used. The application 252 is also called the main application and is an application program that enables the main board 1002 to realize various functions. The application 252 also has the function of Unity (Unity3D), a game engine with a built-in IDE (Integrated Development Environment). The main board 1002 realizes various functions, such as displaying an aerial image 130, by executing the application 252. The memory unit 1670 stores dictionary data for speech recognition (hereinafter referred to as "speech recognition dictionary") 253. The speech recognition dictionary 253 is a dictionary used for speech recognition and contains, for example, phrases for identifying what the user has said. User data 254 is data that is updated according to the user's use of the display device 100. The user data 254 may include data on the user's usage of the display device 100, such as parameters indicating the relationship between the user and the character, as well as data on various settings (for example, settings for each setting menu described later).

[0225] The performance content table is a table that stores the content of character-based performances performed on the display device 100 and the conditions for executing those performances. The display data is video data (e.g., video files) for displaying images on the display device 100, and includes data for displaying various screens and animation data to represent character movements. The audio data is data for outputting audio from the display device 100, and it is preferable that audio data (e.g., audio files) indicating pre-recorded audio is stored. The storage unit 1670 is also preferable to store a program for executing speech synthesis processing that generates audio based on text (documents). The speech synthesis processing should be text-to-speech (TTS). As will be described in detail later, the voice of character 131 is output based on pre-recorded audio data, and the voice of character 150 is output by speech synthesis processing. The main board 1002 also has a function to output character information related to the characters based on the performance content table, display data, and audio data. The character information should ideally consist of information that allows the user to recognize the existence of a character (such as character 131 or character 150). In this embodiment, this information is a combination of video information (which may also be called a video signal) for displaying the character and audio information (which may also be called an audio signal) for outputting the character's voice, but either one or the other may be used. The character's voice is typically the voice that the user recognizes as the content spoken by the character. In addition to this, the storage unit 1670 may also store programs and data necessary for executing each process of the display device 100.

[0226] <3-2. Processing when the display device 100 is started> The startup process of the display device 100 will be described below. When the power switch 340 of the display device 100 is turned ON, the main board 1002 starts application 252. While application 252 is starting up, the main board 1002 displays the message "Now Loading" as shown in Figure 18(a). At this stage, the OS 251 has already started up, so the main board 1002 may use the communication module 1650 to access a server (not shown) to obtain the latest user data and update the user data 254. In this embodiment, unless otherwise specified, user data is stored not only in the display device 100 but also on an external server for purposes such as backup. For example, each time a setting is made via the settings screen, the display device 100 sends data indicating the setting to the server. Alternatively, a configuration in which user data is stored only in the display device 100 may be adopted.

[0227] Once the application 252 has finished starting up, the main board 1002 displays an animation as shown in Figure 18(b), in which images resembling multiple bubbles float up from bottom to top, and the figure of character 131 gradually emerges. After that, a standby screen as described in Figures 1B(a) to (c) is displayed. Once the standby screen is displayed, the main board 1002 becomes ready to accept user input. In this embodiment, the display device 100 does not enter sleep mode after the application 252 has started up, but continues to display the standby screen. In this way, the user can recognize that character 131 is present whenever they look at the display device 100. However, the display device 100 may stop displaying the character if it determines that the user is not using the device, for example, if it determines that the user is not nearby. This reduces the power consumption of the display device 100.

[0228] The operation of the display device 100 when launching application 252 may be as follows, for example. When the power switch 340 is turned ON, the main board 1002 starts OS 251 and first runs an initial setup application to perform initial setup. The initial setup application is an application for performing predetermined initial setup, such as setting up the wireless LAN and verifying the ID used by the user, and may be stored in the memory unit 1670. The main board 1002 may, for example, grant login permission to users who have an ID and password to use a service using a website operated by the manufacturer and distributor of the display device 100, and then proceed with the subsequent processing. Once the initial setup is complete, the main board 1002 displays the logo of the manufacturer and distributor of the display device 100 and, using an updater, checks with the server for the availability of a predetermined software update. As a result of the initial setup, the display device 100 is in a state where it can communicate via wireless LAN. The main board 1002 may, for example, check for updates to OS 251, firmware, and application 252 through this communication. For example, the main board 1002 may perform the OS 251 update if one is available, and perform the firmware update if one is available. The main board 1002 should also check for updates to application 252 and update it as needed to ensure that application 252 is launched with the latest version. In this way, the display device 100 can use multiple software programs to ensure smooth startup processing while running each software program in its latest state.

[0229] The main board 1002 should perform processes that guide the user through the operations they should perform during startup, such as when the initial setup application is running (for example, guidance processing). If the guidance is provided by voice, the main board 1002 should use the natural voice of the same voice actor who voices character 131, or a different voice from the voice used for character 131, when performing the guidance process. This would attract the interest of users who are fans of that voice actor.

[0230] <3-3. Relationship between the display screen of the display device 100 and user operation> Figures 19A and 19B are explanatory diagrams of the aerial image 130 displayed on the display device 100, and the detection area and detection target of the non-contact sensor 1102. The non-contact sensor 1102 provided on the display device 100 has a detection area that is at least in the area where the aerial image 130 is displayed.

[0231] Figures 19A(a) and 19A(b) show examples of standby screens with character 131 displayed. First, a standby screen without icons 1710 and 1750 is displayed, as shown in Figure 19A(a). Figure 19A(a) shows an image 1700 that resembles a hand with an index finger raised, but image 1700 is an image that represents user operation and does not represent an image or icon displayed by the display device 100. When the standby screen shown in Figure 19A(a) is displayed, if user operation (for example, touch operation around character 131) is detected by the non-contact sensor 1102, the main board 1002 transitions to the standby screen shown in Figure 19A(b). This standby screen displays character 131, as well as icons (settings screen call icon) 1710 and (gallery call icon) 1750. When a touch operation on icon 1710 is detected by the non-contact sensor 1102, the main board 1002 displays the first setting screen 1720 shown in Figure 19A(c). While the first setting screen 1720 is displayed, when a touch operation on the page-turning icon Ar1 is detected, the main board 1002 displays the second setting screen 1730 shown in Figure 19B(a). While the second setting screen 1730 is displayed, when a touch operation on the page-turning icon Ar2 is detected, the main board 1002 displays the first setting screen 1720 shown in Figure 18A(c), and when a touch operation on the page-turning icon Ar3 is detected, the main board 1002 displays the third setting screen 1740 shown in Figure 19B(b). While the third setting screen 1740 is displayed, when a touch operation on the page-turning icon Ar4 is detected, the main board 1002 displays the second setting screen 1730 shown in Figure 19B(a). The operation of the display device 100 when the icon 1750 shown in Figure 19A(b) is touched will be explained later in the section <3-13. Gallery Function>.

[0232] In the screen shown in Figure 19A(a), the non-contact sensor 1102 detects touch operations and gesture operations (flick operations in this embodiment) performed by the user on the character. In the screen shown in Figure 19A(b), the non-contact sensor 1102 detects touch operations by the user on the character, as well as touch operations on the two icons 1710 and 1750 displayed at the top. In the screens shown in Figures 19A(c), 19B(a), and (b), the display device 100 detects touch operations for each setting menu displayed on the setting screen. When a touch operation is detected, the main board 1002 should clearly indicate to the user the location where the touch operation was detected. The main board 1002 may display an object, for example, a ripple-like object, at the location of the touch operation using animation. In this way, the user can understand that their touch operation has been recognized by the display device 100 and the location of that touch.

[0233] By designing the detection area of ​​the non-contact sensor 1102 to be located at or near the position of the aerial image 130, the user can perform aerial operations, such as gesture control, with the sensation of touching the aerial image 130. For example, if the aerial image 130 is a character, the user can get the sensation of touching the character.

[0234] More specifically, the non-contact sensor 1102 can detect user actions (e.g., fingers) in the air, particularly within the concave first space 300 of the display device 100, by assigning them to actions such as touching a character or selecting a menu item. The non-contact sensor 1102 can detect not only touches but also slides and shakes.

[0235] In a configuration where the detection area of ​​the non-contact sensor 1102 is positioned on the display unit 1000 side (i.e., behind the display device 100) of the aerial image 130, the user can access the detection area by touching or pressing the aerial image 130. Similarly, in a configuration where the detection area of ​​the non-contact sensor 1102 is positioned at or near the opening surface 700, which is close to the user in the internal space of the housing 110, the sense of depth of the aerial image 130 perceived in conjunction with the internal space of the housing 110 is enhanced, and aerial operations and gesture operations can be performed with the feeling of touching the aerial image 130.

[0236] Because the detection area of ​​the non-contact sensor 1102 is located behind the imaging surface 140 of the aerial image 130 (i.e., on the back side of the display device 100), the user can perform operations without consciously stopping their fingers in front of the aerial image 130. This is preferable because it makes it easier for the user to feel an awareness of interacting with the aerial image 130, especially the character, displayed by the display device 100.

[0237] <3-4. Details about the settings screen> Next, we will provide a detailed explanation of each setting menu displayed on settings screens 1720, 1730, and 1740, and the corresponding functions.

[0238] (3-4-1. Clock Mode (Time Notification Function)) In the first settings screen 1720 shown in Figure 19A(c), menu Op1, which is labeled "Clock Mode," is a menu for configuring settings related to the clock mode. The clock mode is a mode that enables a time notification function in which character 131 announces the time (more specifically, the current time). When a touch operation is detected on menu Op1, the main board 1002 displays the clock mode setting screen Op11 shown in Figure 20(a). The clock mode setting screen Op11 displays a description of the clock mode, along with a button B1 for setting the clock mode ON / OFF. Button B1 displays the current ON / OFF setting. When a touch operation is detected on button B1 which is labeled "ON," the main board 1002 turns off the clock mode and switches the display on button B1 to "OFF." When a touch operation is detected on button B1 which is labeled "OFF," the main board 1002 turns on the clock mode and switches the display on button B1 to "ON" (see Figure 20(a)).

[0239] Figure 20(b) is an example of a standby screen when clock mode is ON. As shown in Figure 20(b), the main board 1002 displays an object 132 that displays the time (in this example, an object resembling a board) associated with character 131. More specifically, character 131 is shown holding the board object 132 that displays the time in front of him (for example, holding it in his hand). The board object 132 displays the time (here, 10:26). To make the time easily visible to the user, the string representing the time may be decorated with an underline or other embellishments. The board object 132 is represented in a color that is, for example, blue and transparent, so that parts of character 131's body (here, his clothing) are visible behind the board object 132. The board object 132 may be in contact with character 131's hand, but by making it appear to be floating as in Figure 20(b), the user can get a sense of virtuality, or the feeling that character 131 is using magic. When the clock mode is ON, the main board 1002 displays an animation on the standby screen in which character 131 moves board object 132. This will be explained in detail later in section <3-6. Details of Clock Mode (Time Display Function)>.

[0240] (3-4-2. Zoom function) In the first settings screen 1720 shown in Figure 19A(c), menu Op2, which is labeled "Zoom," is a menu for configuring settings related to the zoom function. The zoom function is an enlargement display function that displays a portion of the standby screen (for example, a portion of the appearance of character 131 and a portion of the background) in an enlarged manner. When a touch operation is detected on menu Op2, the main board 1002 displays the zoom function setting screen Op21 shown in Figure 21(a). The zoom function setting screen Op21 displays a description of the zoom function, along with a button B2 for setting the zoom function ON / OFF. Button B2 displays the current ON / OFF setting. When a touch operation is detected on button B2 which is labeled "ON," the main board 1002 turns the zoom function OFF and switches the display of button B2 to "OFF" (see Figure 21(a)). When a touch operation is detected on button B2 which is labeled "OFF," the main board 1002 turns the zoom function ON and switches the display of button B2 to "ON."

[0241] Figure 21(b) shows an example of a standby screen when the zoom function is turned ON. As shown in Figure 21(b), the area around the face of character 131 is magnified on this standby screen. This allows the user to feel as if they are getting closer to Ray and facing him. The zoom display range may be different from that shown in Figure 21(b), or the display device 100 may be configured to allow the user to change the zoom display range through user operation (for example, a flick gesture).

[0242] (3-4-3. Voice Recognition Function) In the first settings screen 1720 shown in Figure 19A(c), menu Op3, which displays "Voice Recognition," is a menu for configuring settings related to the voice recognition function. The voice recognition function is a function used, for example, to interact with a character, and may be a function that converts the user's voice into text and outputs it. When a touch operation on menu Op3 is detected, the main board 1002 displays the voice recognition settings screen Op31 shown in Figure 22(a) or (b). This voice recognition settings screen Op31 displays a button B31 for switching the always-on voice recognition mode ON / OFF. Button B31 displays the current setting, and in Figure 22(a), which corresponds to the initial setting, it is "OFF". When the always-on voice recognition mode is "OFF", the display device 100 activates the voice recognition function when the main switch 310 is pressed. The mode in which the always-on voice recognition mode is OFF may be called the button start mode, etc.

[0243] When a touch operation is detected on button B31, which is displayed as "OFF", the main board 1002 turns on the permanent voice recognition mode and switches the display on button B31 to "ON" (see Figure 22(b)). Figure 22(b) shows the voice recognition setting screen Op31 after this switch. On this voice recognition setting screen Op31, button B31 displays "ON", which is the current setting. When the permanent voice recognition mode is "ON", the voice recognition function of the display device 100 is always operating and is in a state where it is waiting for user voice input based on that voice recognition function.

[0244] When a touch operation on the page turning icon Ar4 on the voice recognition setting screen Op31 is detected by the non-contact sensor 1102, the main board 1002 displays the voice recognition setting screen Op32 shown in Figure 22(c).

[0245] The voice recognition settings screen Op32 is used to adjust the sensitivity of the microphone 1040 in the voice recognition function. The microphone 1040 is the microphone that the user uses to transmit their voice to the character. When a touch operation is detected on button B32, which is labeled "+" on the voice recognition settings screen Op32, the main board 1002 increases the microphone sensitivity level by "1". When a touch operation is detected on button B33, which is labeled "-", the main board 1002 decreases the microphone sensitivity level by "1". In Figure 22(c), the number "4" indicates the current microphone sensitivity setting. A microphone test can be performed while the voice recognition settings screen Op32 is displayed. The main board 1002 detects the voice level (for example, the sound pressure level of a predetermined frequency band to which a human voice belongs) via the microphone 1040 and displays the result on the voice recognition settings screen Op32 using the bar Mt display. Furthermore, when a touch operation on the page turning icon Ar5 on the voice recognition setting screen Op32 is detected by the non-contact sensor 1102, the main board 1002 displays the voice recognition setting screen Op31 shown in Figure 22(a) or (b).

[0246] (3-4-4. Sound Settings Function) In the first setting screen 1720 shown in Figure 19A(c), when a touch operation is detected on menu Op4, which is labeled "Sound," the main board 1002 displays the sound setting screen Op41, which corresponds to the sound setting function shown in Figure 23. The sound setting function is a function that sets the volume output by the display device 100. In the sound setting screen Op41, when a touch operation is detected on button B41, which is labeled "+", the main board 1002 increases the volume level by "1", and when a touch operation is detected on button B42, which is labeled "-", the main board 1002 decreases the volume level by "1". In Figure 23, the number "7" indicates the current set volume. Button B43, which corresponds to "Operation Sound", displays the current setting of the operation sound output and is a button for switching the operation sound output ON / OFF. When a touch operation is detected on button B43, which is labeled "ON", the main board 1002 turns OFF the operation sound output and switches the display of button B43 to "OFF". When a touch operation is detected on button B43, which is displayed as "OFF", the main board 1002 turns on the output of the operation sound and switches the display of button B43 to "ON" (see Figure 23). The operation sound is a sound that is output when a user operation is detected by, for example, the non-contact sensor 1102, and may be an electronic sound.

[0247] By the way, regarding volume settings, it is best to use a common volume setting for multiple functions, but the main board 1002 may also be configured to allow different volume settings for each character. In this way, sound can be output at a volume that suits the user's preference for each character. For example, the user can set the volume according to which character's voice they particularly want to hear, and can, for example, increase the volume of their favorite character's voice.

[0248] Furthermore, regarding volume settings, the main board 1002 may be configured to allow different volume settings for each piece of information contained in the output audio. For example, it is preferable to set different volumes for the audio in the dialogue processing described later and the audio in the weather guidance function described later. In this way, the display device 100 can output sound at a volume that suits the user's preferences for each piece of information. For example, the user can set the volume according to which information they particularly want to hear, and may increase the volume of information that they want to listen to carefully.

[0249] (3-4-5. Brightness setting function) When a touch operation is detected on the menu Op2 labeled "Brightness" on the second setting screen 1730 shown in Figure 19B(a), the main board 1002 displays the brightness setting screen Op51, which corresponds to the brightness setting function shown in Figure 24(a) or (b). The brightness setting screen Op51 is for setting the brightness of the aerial image 130 display, for example, the brightness of the LCD 1003 display. The brightness setting screen Op51 corresponding to the initial settings is the screen shown in Figure 24(a), where automatic brightness setting is set to "ON". When automatic brightness setting is "ON", the main board 1002 adjusts the brightness of the aerial image 130 display according to the illuminance measured by the illuminance sensor 1632. Typically, when the surroundings become dark, that is, when the illuminance is low, the main board 1002 dims the display of the aerial image 130, and when the surroundings become bright, that is, when the illuminance is high, it increases the brightness of the display of the aerial image 130. Button B51 displays the current setting of the automatic brightness setting and is used to switch that setting ON / OFF. When a touch operation on button B51 is detected in the state shown in Figure 24(a), the main board 1002 turns off the automatic brightness setting and switches to "OFF," which fixes it to the brightness set by the user, and switches to the brightness setting screen Op52 shown in Figure 24(b).

[0250] When the brightness setting screen Op51 is displayed, if a touch operation is detected on button B52 labeled "+" and associated with "Brightness (when bright)", the main board 1002 increases the brightness level by "1", and if a touch operation is detected on button B53 labeled "-", the main board 1002 decreases the brightness level by "1". In Figure 24(a), the value "10" indicates the current brightness setting. The condition "Brightness (when bright)" is a setting adopted when the area around the display device 100 is relatively bright, for example, when the illuminance measured by the illuminance sensor 1632 is above a threshold. Examples of bright conditions include sunlight in the morning or at noon, as well as the area around the display device 100 being bright even at night or in a dark environment with the curtains closed, for example, if a desk lamp is on.

[0251] When the brightness setting screen Op51 is displayed, if a touch operation is detected on button B54, which is associated with "Brightness (when dark)" and labeled "+", the main board 1002 increases the brightness level by "1", and if a touch operation is detected on button B55, which is labeled "-", the main board 1002 decreases the brightness level by "1". In Figure 24(a), the number "5" indicates the current brightness setting. The condition "Brightness (when dark)" is a setting adopted when the area around the display device 100 is relatively dark, for example, when the illuminance measured by the illuminance sensor 1632 is below a threshold. Examples of dark conditions include nighttime, when the curtains are closed, and the room lights or desk lamp are turned off, making the area around the display device 100 dark.

[0252] Thus, the display device 100 has a configuration that allows the display brightness to be set individually depending on whether the surrounding environment of the display device 100 is bright or dark, so that the aerial image 130 can be displayed at a brightness that is easy for the user to see, and at a brightness that reflects the user's preference for each brightness level. Here, the brightness is divided into two levels, "bright" and "dark," but it may also be divided into three or more levels.

[0253] Furthermore, the main board 1002 also has a function to set the sensitivity of the illuminance sensor 1632. This sensitivity specifies the illuminance at which the brightness of the aerial image 130 display is switched. When a touch operation is detected on button B56, which is associated with the illuminance sensor sensitivity "Brightness (when bright)" on the brightness setting screen Op51, the main board 1002 increases the sensitivity level of the illuminance sensor 1632 by "1". When a touch operation is detected on button B57, which is indicated by "-", the main board 1002 decreases the sensitivity level of the illuminance sensor 1632 by "1". In Figure 24(a), the value "3" indicates the current sensitivity setting of the illuminance sensor 1632. In this way, the manner in which the display brightness changes can be adjusted according to the brightness around the display device 100, and the user can view the aerial image 130 after more precisely setting the brightness at which the aerial image 130 is easily visible.

[0254] Furthermore, in this embodiment, the sensitivity of the illuminance sensor 1632 is set to the same value for both "brightness (when bright)" and "brightness (when dark)". This ensures that the same sensitivity is used regardless of the environment, making it easy for the user to use. However, if the display device 100 is configured to allow the sensitivity of the illuminance sensor 1632 to be set individually for "brightness (when bright)" and "brightness (when dark)", the user's flexibility in adjusting the brightness setting will be improved.

[0255] When automatic brightness setting is "OFF", the main board 1002 displays the brightness setting screen Op52 shown in Figure 24(b). When a touch operation is detected on button B58, which is labeled "+", the main board 1002 increases the brightness level by "1", and when a touch operation is detected on button B59, which is labeled "-", the main board 1002 decreases the brightness level by "1". In Figure 24(b), the value "10" indicates the current sensitivity setting of the illuminance sensor 1632. When automatic brightness setting is "OFF", the main board 1002 does not adjust the brightness of the aerial image 130 display according to the illuminance measured by the illuminance sensor 1632, but fixes it to the display brightness set on the brightness setting screen Op52. In this way, the aerial image 130 can be displayed at the user's preferred display brightness regardless of the ambient brightness around the display device 100. When a touch operation is detected on button B51 while the brightness setting screen Op52 is displayed, the main board 1002 turns on automatic brightness setting and switches to the brightness setting screen Op51 shown in Figure 24(a).

[0256] (3-4-6. Alarm function) In the second settings screen 1730 shown in Figure 19B(a), menu Op6, labeled "Alarm," is a menu for configuring the alarm function. The alarm function outputs an alarm as a notification to the user at a predetermined time. The alarm is made by sound output and / or display, or by either one of these. The alarm output only needs to be a notification that the user can recognize; for example, it may be made by sound only.

[0257] When a touch operation is detected on menu Op6, which corresponds to the "alarm function," the main board 1002 displays the alarm setting screen Op61 shown in Figure 25(a). On the alarm setting screen Op61, buttons B61 to B67 corresponding to each day of the week from "Sunday" to "Saturday" are displayed in a vertical arrangement. In this way, the alarm function allows for individual alarm settings for each day of the week. Furthermore, multiple alarm settings (for example, 5) can be set for each day of the week. If a touch operation is detected on button B62, which corresponds to "Monday," the main board 1002 displays the alarm setting screen Op62 shown in Figure 25(b). The alarm setting screen Op62 displays the registered alarm settings for the day of the week that was touched, but since it is not registered in this case, the message "Not registered" is displayed. The alarm setting screen Op62 also displays a button B621 labeled "Add" for adding a new alarm setting. When a touch operation is detected on button B621, the main board 1002 displays the alarm setting screen Op63 shown in Figure 25(c). On this alarm setting screen Op63, buttons B622, B623, B624, B625, and B626 are displayed side by side. Buttons B622 to B625 each display the current setting and are buttons for changing that setting.

[0258] When a touch operation on button B622 is detected, the main board 1002 sets the set time according to the user's operation. The set time is the time when the alarm output will start. In Figure 25(c), it is set to "07:00" (7:00). When a touch operation on button B623 is detected, the main board 1002 sets the alarm output volume according to the user's operation. Here, the volume is set to "10". The alarm output volume is set independently of the volume set by the sound setting function, and the alarm output volume is preferentially selected in the alarm function. However, the alarm output volume may be the same as the volume set by the sound setting function. When a touch operation on button B625 is detected, the main board 1002 sets the ringing time, which is the duration for which the alarm will continue to output. Here, the ringing time is set to "1 minute". When a touch operation on button B625 is detected, the main board 1002 is preferably configured to at least set the snooze function to ON / OFF, and further to set the snooze interval, which is the interval between alarm outputs, and the number of outputs. The snooze interval is the length of time between the output of an alarm, the end of the sounding time, and the next time an alarm is output. In Figure 25(c), the snooze function is set to "ON".

[0259] When a touch operation on button B626 is detected, the main board 1002 sets (i.e., registers) an alarm based on the content displayed on the current alarm setting screen Op63. The same procedure is used to set a second or subsequent alarm for "Monday". The same procedure is also used to set alarms for days other than "Monday". In this way, the alarm function of the display device 100 has the day of the week at the highest level, and the conditions for outputting that alarm are associated with the lower levels of that day of the week. This makes it easy to set alarms appropriate for each day of the week, taking into account that the user's daily routine may differ from day to day.

[0260] In addition to or instead of setting alarms by day of the week, it may also be possible to set alarms based on seasonal conditions such as month, date, or season, or to set alarms without any such seasonal conditions (for example, by setting only the time).

[0261] Figure 26 shows an example of the standby screen when there are set alarms. As shown in Figure 26, if at least one alarm is set, alarm information 134, which is information about the set alarm, is displayed on the standby screen. In the example in Figure 26, alarm information 134 is displayed in the lower area of ​​the standby screen, more specifically, below the character 131, near the character's feet. Alarm information 134 includes a bell-shaped mark, the day of the week on which the alarm is set (in Figure 26, "MON" means Monday), and the set time of that alarm (in Figure 25, 07:00). Alarm information 134 is information about the next alarm to be output. Note that here, the setting for the next alarm to be output is displayed, but if two or more alarms are set, alarm information 134 for two or more alarm settings may be displayed. Alarm information 134 may also only contain information about alarms to be output on that day; for example, if the next alarm to be output is on Tuesday on Monday, the information about Tuesday's alarm may not be displayed.

[0262] Figure 27 illustrates the screen for changing or deleting alarm settings. In the alarm setting screen Op61 shown in Figure 25(a), when a touch operation is detected on any of the buttons B61 to B67, the main board 1002 displays the alarm setting screen, including the alarms currently set for that day of the week. Now, suppose a touch operation on button B62 is detected, and the alarm setting screen Op62 shown in Figure 27(a) is displayed. This alarm setting screen Op62 displays the information about the alarms mentioned above, and a button B627 that indicates whether or not to output the alarm ON / OFF. When a touch operation on button B627 is detected, the main board 1002 displays the settings related to the output of that alarm, as shown in Figure 27(b). The user can change the alarm settings in the same way as described in Figure 25(c). To save the changes, the user touches the button B628 labeled "Save". When the main board 1002 detects a touch operation, it saves (re-registers) the changed settings. To delete an alarm setting, the user touches button B629, which is labeled "Delete". When the main board 1002 detects this touch operation, it deletes the alarm setting. As a result, the alarm setting screen Op62 becomes the same screen as shown in Figure 25(b). The operation of the alarm function will be explained in detail in section <3-10. Details of the Alarm Function>.

[0263] (3-4-7. User Information Settings Function) In the second settings screen 1730 shown in Figure 19B(a), when a touch operation is detected on menu Op7, which displays "User Information," the main board 1002 displays the user information settings screen Op71, shown in Figure 28. The user information settings screen Op71 allows users to set (also called edit) information about themselves and displays the set content. When a touch operation is detected on button B71, which corresponds to the residential area, the main board 1002 displays a screen for setting the user's residential area and changes the setting according to the user's operation. In Figure 28, it is set to "Shizuoka Prefecture," which is displayed on button B71. The residential area is set at the prefecture level, but it may also be configured to be set at units other than prefectures, such as cities, towns, and villages. When a touch operation is detected on button B72, which corresponds to the birthday, the main board 1002 displays a screen for setting the user's birthday and changes the setting according to the user's operation. In Figure 28, it is set to "04 / 15" (April 15th), which is displayed on button B72.

[0264] (3-4-8. Wireless LAN Settings (Communication Settings)) In the second settings screen 1730 shown in Figure 19B(a), when a touch operation is detected on menu Op8, which displays "Wireless LAN Settings," the main board 1002 displays the wireless LAN settings screen shown in Figure 29(a). Since the display device 100 is configured to connect to the internet line via wireless LAN and communicate, it is configured to allow users to check and change settings related to that communication using the wireless LAN settings screen.

[0265] (3-4-9. Regarding the third settings screen 1740) In the third settings screen 1740 shown in Figure 19B(b), when a touch operation is detected on the menu Op91 labeled "Legal Information," the main board 1002 displays the screen related to legal information shown in Figure 29(b). This screen displays buttons for displaying legal matters related to the display device 100, such as "Authentication" and "Software License." When a touch operation is detected on the third settings screen 1740, specifically on menu Op92 which displays "Initialize Main Unit," the main board 1002 displays a button to initiate initialization, which is used to initialize the display device 100 shown in Figure 29(c) and return it to its factory settings. When a touch operation on the button is detected, the main board 1002 initializes the display device 100, deletes the user data 254 from within the device, and also erases the data corresponding to the user data 254 from the server. When a touch operation is detected on the menu Op93, which displays "Version Information," on the third settings screen 1740, the main board 1002 displays the version information shown in Figure 29(d).

[0266] <3-5. Features related to communication between users and characters> Next, the functions related to communication between the user and the character will be described. The main board 1002 outputs character information as video information to the LCD 1003 to display an aerial image 130 of the character, and also outputs character information as audio information to the speaker 1104 to output the character's voice from the speaker 1104. The main board 1002 further outputs character information according to user input and / or the relationship between the user and character 131, thereby performing various effects using the character. User input can be, for example, input using the non-contact sensor 1102, but is not limited to this, and can also be input by operating a physical button (for example, pressing the main switch 310) or by the user's voice (typically the user's voice). There are two characters, character 131 and character 150, but the display device 100 mainly displays character 131.

[0267] <3-5-1. Process to initiate dialogue with character 131> First, we will explain the process for initiating a dialogue, which is an example of communication with character 131. When the voice recognition mode is always OFF, the main board 1002 activates the voice recognition function in response to the pressing of the main switch 310, making it possible for the user to interact with character 131. When the voice recognition mode is always ON, the main board 1002 has the voice recognition function always active, and based on that voice recognition function, it makes it possible for the user to interact with character 131 in response to recognizing a predetermined trigger phrase. The trigger phrase is labeled as the "start phrase" in Figure 22(b).

[0268] The trigger phrase is a phrase that the user should utter to initiate a conversation with character 131. In other words, when the voice recognition mode is always ON, a conversation with character 131 will not begin even if the user utters a phrase other than the designated trigger phrase.

[0269] In this embodiment, "phrase" is an example of a human-understandable word (string of characters), and is not particularly limited in terms of syntax or length. However, a phrase uttered by a user and recognized by the display device 100 based on its speech recognition function may be referred to as a "recognized phrase" below.

[0270] The trigger phrase is preferably registered in the speech recognition dictionary 253 stored in the memory unit 1670. The trigger phrase may be a phrase that includes at least one of the first and last names of character 131, for example, "Rei-chan," "Rei-san," "Rei-tan," "Kirishima-san," "Kirishima-chan," etc. It is assumed that when a user addresses character 131, they will use honorifics such as "chan" or "san," but it is even better if the trigger phrase also includes addresses without honorifics. It is even better if the trigger phrase includes phrases that do not include the first and last names of character 131, for example, phrases that are expected to be used when a person speaks to or calls out to someone else, such as "Hey," "Excuse me," "Hey," "Excuse me," or "Hello."

[0271] When the voice recognition mode is OFF, the main board 1002 keeps the light around the main switch 310 green, and when the main switch 310 is pressed, it initiates a dialogue with the character 131. The process of initiating the dialogue includes activating the voice recognition function, changing the light around the main switch 310 to blue, and displaying an icon 1760 to indicate this, as shown in Figure 30. The icon 1760 here is an icon resembling a microphone, making it easy for the user to visually understand that dialogue with character 131 is possible and that the board is waiting for voice input from the user. When the voice recognition mode is ON, the main board 1002 pre-sets the light around the main switch 310 to blue. Then, when the main board 1002 recognizes a trigger phrase, it is preferable to display the icon 1760 to indicate this, as shown in Figure 30.

[0272] <3-5-2. Visual effects when a trigger phrase is recognized> Next, we will describe the effects performed by the display device 100 when the voice recognition mode is always ON and a trigger phrase is recognized, as described in section <3-5-1. Processing to start dialogue with a character>. When the main board 1002 recognizes a trigger phrase, it uses character 131 to output character information for performing effects using character 131 according to the effects content table shown in Figure 31.

[0273] Figure 31 is a diagram illustrating the presentation content table, which shows the presentation content when a trigger phrase is recognized. Note that Figure 31 and other diagrams illustrating the presentation content tables may not include all possible presentation content patterns, and may only illustrate representative examples.

[0274] In Figure 31 and other diagrams illustrating the performance content tables, the phrases listed in the "Phrase" column of each row are phrases that may be spoken by the character's voice when the conditions associated with the same row (record) are met. In Figure 31, these conditions include "Recognition Phrase," "State," and "Clock State." The "Recognition Phrase" indicates the type (or category) of the phrase recognized from the input user voice. The type of recognition phrase can be said to indicate the type (or theme) of the dialogue processing between the user and character 131. In Figure 31, the recognition phrase is labeled as "Trigger Phrase." The "State" indicates the state of character 131 when the recognition phrase is recognized. There are three states: "Normal" (an example of the second state), which indicates that character 131 is in a normal state; "Grumpy," which indicates that character 131 is in a bad mood, such as being angry (an example of the first state); and "Sleeping," which indicates that character 131 is asleep (an example of the third state). Note that states other than those listed for these characters may be included, and there may be even more states. "Clock State" indicates whether the clock mode is ON or OFF; if it is OFF, it is written as "Normal," and if it is ON, it is written as "Clock." When character 131's state is "Sleeping," it is left blank, meaning there is no ON / OFF state for the clock mode. Although omitted in Figure 31 and other diagrams explaining the performance content tables, it is preferable that the performance content tables also contain information that identifies the display data for showing animations, etc., when outputting phrases stored in the performance content tables.

[0275] Furthermore, if the performance content table has multiple patterns of phrases (which may be read as performances) associated with the same conditions, the main board 1002 outputs character information for performing a performance using one of the multiple phrase patterns (for example, the first pattern and the second pattern) randomly selected from among the multiple patterns. By making the selection random, it is expected that the performance will be interesting for the user. However, this is not limited to this, and the main board 1002 may select patterns in other ways, such as selecting patterns according to predetermined rules. Selection according to predetermined rules may include selecting so that the performance does not occur multiple times in a row when the same conditions are met, or selecting in a predetermined order. In such performances using characters, the main board 1002 may output the selected phrase using the character's voice and perform a performance in which the character performs predetermined movements. The predetermined movements may be movements related to the content of the phrase, for example, movements that allow the user to visually recognize that the user's trigger phrase has been recognized. Furthermore, in the effects shown in Figure 31 and other effects, it is preferable that the relationship between the animation display timing and the phrase output timing be pre-adjusted, for example, for each effect, each animation, or each phrase.

[0276] Furthermore, the main board 1002 should output character information so that different effects are displayed depending on the state of character 131. For example, the main board 1002 should have character 131 utter different phrases or change character 131's animations depending on the state of character 131. For example, when character 131 is in a normal state, the main board 1002 should have character 131 output phrases that express a cheerful mood, such as "Yes, what do you need?" or "Yes! Since you called, I'm sure you have something to tell me," and display animations such as raising a hand or saluting. For example, when character 131 is in a bad mood, the main board 1002 should have character 131 output phrases that express a bad mood, such as "Hmm, do you want to talk?", and display animations such as making a bad face (for example, a cold face). For example, when character 131 is asleep, the main board 1002 should output phrases in character 131's voice that express the appearance of waking up or being sleepy, such as "...Muuh? Yes..." or "...Zzz. What...?", and should also display animations such as the character making a sleepy or waking-up expression.

[0277] <3-5-3. Effects after the effect when the trigger phrase is recognized> Next, we will describe the effects performed by the display device 100 after the effects described in section <3-5-2. Effects when a trigger phrase is recognized>. After any of the effects described in section <3-5-2: Effects when a trigger phrase is recognized>, the main board 1002 outputs character information for performing an effect using character 131, based on the effect content table shown in Figure 32.

[0278] Figure 32 is a diagram illustrating an example of a performance content table showing the performance content after a performance occurs when a trigger phrase is recognized. Here, in addition to "state" and "clock state," there are "branch" and "procedure" as conditions for the performance to occur. "Branch" means that the process described in this column will be performed following the performance in each row. Here, there is "recognition impossible," which ends the interaction with the user because the system cannot recognize what the user said based on the speech recognition dictionary 253, and "request for clarification," which asks the user to repeat what the system said if the system cannot recognize what the user said. "Procedure" is the procedure that the user should take in order for the performance associated with the same row to be performed, and for example, it describes the phrase the user should say or the actions such as operations. The main board 1002 should display an animation showing character 131 tilting its head if recognition is impossible, and display an animation showing character 131 putting its hand to its ear to listen carefully if a request for clarification is made.

[0279] The main board 1002 outputs phrases such as "Sorry, I didn't understand," "I couldn't hear you," ..., "Sorry, I couldn't hear you," in the voice of character 131, for example, if a timeout occurs after the recognition of a trigger phrase or the pressing of the main switch 310, or if the user utters an unregistered phrase twice. A timeout means that no user voice input (for example, silence) has been received for a predetermined period of time since the trigger phrase was recognized or the main switch 310 was pressed. The predetermined period may be, for example, 30 seconds. Unrecognizable means that the system recognized that the user made some kind of utterance, but could not identify it as any of the phrases registered in the speech recognition dictionary 253. Phrases not registered in the speech recognition dictionary 253 may include, for example, words like "ah" or "uh," which do not have a clear linguistic meaning, or phrases that are not expected to be responses from the user. The second utterance of an unregistered phrase is an utterance made by the user after one "listen again" sequence.

[0280] In the repetition animation, the main board 1002, for example, after recognizing a trigger phrase or pressing the main switch 310, if the user utters a phrase not registered in the dictionary, will have character 131 utter phrases such as "Huh? Is something wrong? It's nothing," "Yes? What is it...?", ..., "Hmm. What do you want?", and then wait for voice input from the user again. In this way, even if the system recognizes that the user has said something but cannot identify it as one of the phrases registered in the speech recognition dictionary 253, the dialogue between character 131 and the user can continue, making it easier for the user to feel as if they are actually having a conversation with character 131. Note that, to avoid the dialogue becoming redundant, the repetition animation may only occur once, but it may be possible to have it occur two or more times. Also, in the animation shown in Figure 32, the diversity of expression is increased by making at least part of the animation content different depending on the state of character 131 (normal / unhappy / sleeping) and whether the clock mode is ON or OFF.

[0281] <3-5-4. Mischief> The display device 100 executes a process related to mischievous behavior as a process to change the state of character 131, more specifically, character 131's mood towards the user, from "normal" to "unhappy". Mischievous behavior can also be interpreted as harassment, etc., and is an example of user behavior that character 131 finds unpleasant. When the main board 1002 detects that the user has performed mischievous behavior towards character 131, it changes character 131's state to "unhappy" (if it is already unhappy, it remains unhappy) and outputs character information for performing a performance using character 131 based on the performance content table shown in Figures 33 and 34.

[0282] First, let me explain the nature of the prank. The main board 1002 accepts an operation to touch character 131 as input to the character using the non-contact sensor 1102 as a prank. In this embodiment, the operation to touch character 131 is an operation to touch a part of character 131's body. Figure 35 is a diagram illustrating the parts of character 131's body used for detecting pranks. Figure 35(a) shows character 131 displayed wearing a skirt. Figure 35(b) shows character 131 displayed wearing pajamas.

[0283] To explain Figure 35(a), the main board 1002 accepts operations to touch parts P11 and P12 of the character 131. These operations may include touching either part P11 or P12, as well as flicking (moving the fingers in a predetermined direction). Part P11 corresponds to the character 131's chest. The main board 1002 detects a user touching part P11 of the character 131 as a prank involving touching the chest. Part P12 corresponds to the skirt worn by the character 131. The main board 1002 detects a user touching part P12 of the character 131 as a prank involving touching the skirt. Furthermore, the main board 1002 detects a predetermined flicking operation on part P12 of the character 131 as a prank involving lifting the skirt. Here, the specified flick operation is best defined as a flick operation from bottom to top, and any flick operation that includes an upward component, or a flick operation that includes an upward component in addition to a predetermined amount in the horizontal direction, may also include a diagonal upward flick operation.

[0284] To explain Figure 35(b), the main board 1002 accepts operations that involve touching parts P21 and P22 of the character 131. Operations that involve touching parts P21 and P22 may include touching either part P21 or P22. Part P21 corresponds to the chest of the character 131. The main board 1002 detects a user's touch operation on part P21 of the character 131 as a prank involving touching the chest. Part P22 corresponds to the trousers worn by the character 131. The main board 1002 detects a user's touch operation on part P22 of the character 131 as a prank involving touching the trousers.

[0285] Furthermore, while parts P11, P12, P21, and P22 correspond to the respective body parts of character 131 (for example, the display positions of objects that display those body parts), which is suitable for more realistic detection of mischievous acts, they may be larger than the respective body parts, for example, from the standpoint of collision detection, or they may be identified, for example, by the parts of the bounding rectangle of each body part.

[0286] When the main board 1002 detects mischief, it sets the state of character 131 to "unhappy" and outputs character information for performing an action using character 131 based on the performance content table shown in Figure 33.

[0287] Figure 33 is a diagram illustrating a table of animation content, showing an example of the animation content when a mischievous act of touching a part of character 131's body is detected. Figure 33(a) corresponds to the case where clock mode is OFF, and Figure 33(b) corresponds to the case where clock mode is ON. Here, there are "state" and "procedure" as conditions for the animation to be performed. "Previous phrase" and "following phrase" mean that the voice of the character in the preceding phrase is output, followed immediately by the voice of the character in the following phrase. If there is no following phrase, it is blank. The preceding phrase and following phrase are each composed of separate audio files, but they may also be a single audio file.

[0288] Referring to Figure 33(a), when the clock mode is OFF, the main board 1002, for example, when character 131 is in a normal state, detects a predetermined flick operation in which the edge of the skirt (i.e., part P12) of the skirt costume is flicked from bottom to top, and outputs phrases such as "Ah! You were trying to lift my skirt!! Come on!" or "Hey! What are you doing! You were trying to peek!" in the voice of character 131. When character 131 outputs such voice, the main board 1002 may have character 131 perform actions such as being surprised (for example, jumping up) or avoiding the mischievous act (for example, moving part P12 that the user touched to avoid it, or swatting away the user's fingers). Furthermore, if the main board 1002 detects, for example, an operation to touch the chest of the skirt costume (i.e., a touch operation on body part P11) when character 131 is in a normal state, it will output phrases such as "What are you trying to do... That's awful..." in character 131's voice. When the main board 1002 outputs such voice from character 131, it is desirable for character 131 to perform actions such as being surprised (for example, jumping up) or avoiding the mischievous act (for example, moving body part P11 to avoid what the user touched, or swatting away the user's fingers). Also, if the main board 1002 detects, for example, an operation to touch the skirt costume (i.e., a touch operation on body part P12) when character 131 is in a normal state, it will output phrases such as "Ah! You really can't do that!" or "Ah! That's the worst..." in character 131's voice. The main board 1002 should, when outputting the voice of such character 131, allow character 131 to perform actions such as being surprised (for example, jumping up) or avoiding mischief (for example, moving part P12 that the user has touched to avoid it, or moving as if swatting away the user's fingers).In the presentation shown in Figure 33, the diversity of expression is increased by ensuring that at least part of the presentation content differs depending on the state of character 131 (normal / unhappy / sleeping) and whether the clock mode is ON or OFF (see Figure 33(b) if it is OFF). For example, it would be good to display animations that show the difference in emotion between when character 131 is normal and when character 131 is unhappy. In particular, when character 131 is unhappy, it would be good to use colder expressions than when character 131 is normal, for example, by remaining silent without uttering any phrases.

[0289] Let me explain another type of prank. The main board 1002 accepts user voice input via microphone 1040, indicating that the user intends to perform a prank on character 131. Such voice input may include phrases that evoke sexual harassment or phrases that indicate questions that character 131 would be reluctant or hesitant to answer. Such phrases are classified as "NG phrases."

[0290] If the main board 1002 detects a prank involving voice input containing a user's forbidden phrase, it sets the state of character 131 to "unhappy" and outputs character information for performing an action using character 131 based on the performance content table shown in Figure 34.

[0291] Figure 34 illustrates a table showing an example of the type of animation that can be triggered when prank activity via user voice input is detected. Here, too, the conditions for triggering the animation include "state" and "procedure." There are also "preceding phrase" and "following phrase."

[0292] For example, when character 131 is in a normal state and the main board 1002 receives voice input from the user that includes an inappropriate phrase, it will output phrases such as "I'm warning you," "Any more vulgar questions than this and my alert level will go up," "I'm getting angry," "I'm a little put off by you right now," ..., "Hey...", "Do you know what a dating sim is? Is that okay? If you keep doing things I don't like, it might end in a bad ending," in the voice of character 131. When character 131 outputs such voices, the main board 1002 may have character 131 make angry faces, cover their ears, or blush. In the effects shown in Figure 34, the diversity of expression is increased by making at least part of the effects differ depending on character 131's state (normal / unhappy) and whether the clock mode is ON or OFF. For example, it would be good to display animations that show the difference in emotions between when character 131 is normal and when character 131 is in a bad mood. In particular, when character 131 is in a bad mood, it would be good to use colder expressions than when character 131 is normal, such as "Are you asking me that now?", "I didn't know you were such a terrible person," or "Your question and timing are awful," or "I never want to see your face again." These expressions would convey stronger emotions such as anger.

[0293] The main board 1002 should disable detection of pranks based on user voice input and disable any changes to character 131's mood when character 131 is asleep. Disabling changes to character 131's mood means that even if an NG phrase is detected, no prank-related animations will be performed, and character 131 will not become unhappy. This is based on the idea that character 131 is asleep and therefore has not heard the NG phrase and is unaware of the prank.

[0294] Furthermore, the main board 1002 may always change character 131's state to "unhappy" when it detects mischief, or it may change it to "unhappy" with a predetermined probability. In this case, even if the main board 1002 detects mischief, it is preferable to keep character 131's state "normal". In this case, the main board 1002 may perform certain effects, such as outputting character 131's voice saying a phrase like "This is the last time, okay?".

[0295] The conditions under which character 131 becomes unhappy are not limited to the detection of mischievous behavior. For example, the main board 1002 may, for instance, perform a process to make character 131 unhappy if it meets a predetermined condition indicating that the user has not been using the display device 100 much recently (for example, that the trigger phrase has not been recognized for a predetermined period of time, or that there has been no interaction with character 131).

[0296] <3-5-5. Apology> In the case where character 131 becomes unhappy due to the prank described in section <3-5-4. Mischief>, one way to return character 131 to a normal state is for the user to express an apology to character 131. In this embodiment, the user's apology may be expressed by uttering a phrase that signifies an apology. The main board 1002 may detect this as an apology from the user when it receives an audio input of a phrase that signifies an apology to character 131 using the microphone 1040. Examples of phrases that signify an apology include "Sorry," "I'm sorry, Rei-chan," and "Please forgive me."

[0297] When the main board 1002 detects the user's act of apology, it changes the state of character 131 from "unhappy" to "normal" with a predetermined probability. The predetermined probability is preferably lower than 1 / 2, for example, 1 / 3. In this way, the user can get a sense of real interpersonal relationships, where apologizing doesn't always guarantee forgiveness, but sometimes it does.

[0298] When the main board 1002 detects a user's apology, it outputs character information for performing a performance using character 131, based on the performance content table shown in Figure 35.

[0299] Figure 36 is a diagram illustrating a table of presentation content, showing an example of the presentation content when a user's apology is detected. In Figure 36, there is a row called "Scenario," and for each apology, one of the scenarios will be presented. Here, the scenarios are "Forgive 1" and "Forgive 2" for when character 131 forgives the user, and "Do not forgive 1," "Do not forgive 2," and "Are you remorseful?" for when character 131 does not forgive the user. Here too, "State" and "Procedure" exist as conditions for the presentation to be performed.

[0300] Furthermore, there is a "branching" condition for the performance to occur. If the performance of the "Branch ○: Start" scenario is selected, the main board 1002 performs the performance corresponding to "Branch ○: Start" and then waits for a response from the user. The response is typically the user's answer (which can also be read as a response) to a phrase spoken by character 131, and there are "affirmative" responses, which mean positive responses, and "negative" responses, which mean negative responses. When the main board 1002 executes the "Branch ○: Start" scenario and recognizes an affirmative response from the user while waiting for a response, it performs the performance of the "Branch ○: Affirmative" scenario. An affirmative response means agreeing with or accepting the content of the character's phrase, for example, "Yes," "Uh-huh," or "That's right." On the other hand, when the main board 1002 recognizes a negative response from the user while waiting for a response, it performs the performance of the "Branch ○: Negative" scenario. A negative response means not agreeing with or not accepting the content of the character's phrase, for example, "No," "Wrong," or "I don't know." If the response is neither positive nor negative, the main board 1002 will execute one of two scenarios: "Branch ○: Silence," which means no response from the user, or "Branch ○: Ask Again," which means asking again. If the main board 1002 executes "Branch ○: Ask Again," it will output the voice of character 131 with a phrase such as "What did you say? Could you please repeat that, if possible," and enter the waiting period again. If a predetermined amount of time has passed since entering the waiting period after asking again without receiving a response (for example, 30 seconds have passed), the main board 1002 will time out and execute "Branch ○: Silence." The main board 1002 may also execute "Branch ○: Silence" if it detects an act of apology and the response is neither positive nor negative. Furthermore, if the user has responded even once, the main board 1002 will execute "Branch ○: Unrecognized."

[0301] For example, when the main board 1002 detects an apology from the user, if it decides to forgive the user, it outputs the voice of character 131 with phrases such as "...Okay, I understand, I'll forgive you now," "Let's go back to how things were before, shall we?" or "...I guess I have no choice...I'll forgive you as a special favor," or "I'll try my best to cheer up...Hmm!! Ha." For example, when the main board 1002 detects an apology from the user, if it decides not to forgive the user, it outputs the voice of character 131 with phrases such as "Isn't that just lip service?", "Doesn't sound like you're really apologizing...", "Are you really apologizing...?", or "I still can't forgive youI don't really feel like you're apologizing...", "Are you really apologizing...?", or "I still can't forgive you." When the main board 1002 detects the user's act of apology, it may perform the "Are you remorseful?" scenario. In this case, it outputs the voice of character 131 saying the phrase "Do you really mean it?" or "Do you truly think you did something wrong?" and then enters a state of waiting for a response from the user. Depending on the user's response during this waiting period, the main board 1002 performs one of the following scenarios shown in Figure 36: "Branch C: Affirmative," "Branch C: Negative," "Branch 0: Silence," and "Branch C: Ask again." The character 131's movements in response to the apology may display different animations depending on whether the user forgives or not, and the performance may express the conclusion of whether the user forgives or not, as well as the character 131's emotions. Furthermore, the diversity of expression may be increased by making at least part of the performance content different depending on whether the clock mode is ON or OFF.

[0302] Furthermore, the main board 1002 should disable detection of apologies based on user voice input and disable changes to character 131's mood when character 131 is asleep. Disabling changes to character 131's mood means that even if an apology is detected, no animation related to the apology will be performed, and character 131 will not be changed from an unhappy mood to a normal mood. This is based on the idea that character 131 is unaware of the apology because she is asleep.

[0303] Furthermore, this embodiment does not prevent the main board 1002 from necessarily returning the character 131 to "normal" state when it detects an act of apology. Also, acts of apology may be performed in other ways. For example, if the display device 100 is equipped with a camera, the main board 1002 may detect an act of apology based on the image captured by the camera, if it recognizes that the user is performing an act signifying apology, such as putting their hands together or bowing their head.

[0304] <3-5-6. Other examples of actions that forgive the user> Character 131 does not only forgive the user when an act of apology is detected. The main board 1002 should change character 131's state from "unhappy" to "normal" when character 131 enters a sleep state. This is to express the personality described above: "Ray has a personality where she might forget what happened yesterday and her mood improves after a good night's sleep." In this case, the main board 1002 should set character 131's state to normal at 5:00 when character 131 wakes up.

[0305] Other methods may be employed. When a user performs an operation such as giving a gift, the main board 1002 may always, or with a predetermined probability, perform a process to improve the mood of character 131. The main board 1002 may grant this as a reward for the user's use of the display device 100, such as the user's usage record of the display device 100 or clearing a predetermined game, or it may grant it on the user's birthday. In addition, when the main board 1002 detects an operation (flick operation) by the user to stroke a part of character 131's head, it may always, or with a predetermined probability, change character 131's state from unhappy to normal.

[0306] <3-5-7. Dialogue Processing> Next, we will describe the dialogue processing that simulates a conversation between the user and character 131. In the dialogue processing, the main board 1002 also outputs character information for performing a performance using character 131 based on the performance content table. In the dialogue processing of this embodiment, unless otherwise specified, the main board 1002 outputs a phrase in the voice of character 131 that corresponds to a phrase spoken by the user. In this dialogue processing as well, the main board 1002 may also speak one of the phrases in the voice of character 131 and perform a performance that causes character 131 to perform a predetermined movement. The content of the dialogue processing will be described below for each type of phrase spoken by the user (recognition phrase).

[0307] (3-5-7-1. Good morning) Figures 37A to 37F illustrate the presentation content table, which shows the presentation content when a user utters a phrase of the type "Good morning." In Figures 37A to 37F, "Recognition Phrase," "State," "Scenario Name," "Branch," "Procedure," "First Half Phrase," and "Second Half Phrase" are as previously explained. Here, "Time" is also included as a condition for performing the presentation. This "Time" is the condition for the time when the recognition phrase was recognized by the display device 100. A presentation with a blank "Time" column means that no time condition is set and the presentation can be performed regardless of the time.

[0308] As shown in Figure 37A(a), for example, if the user utters a phrase of the type "Good morning" between 5:00 and 6:30, the main board 1002 may output the voice of character 131 with a phrase indicating that the user is an early riser, such as "Good morning!" "That's pretty early. I'm still sleepy." or "Good morning." "You're up pretty early." Alternatively, the main board 1002 may output the voice of character 131 with a phrase asking about the reason for the early rise, such as "Good morning." "You're up early. Do you have plans?" In this case, as shown in Figure 37A(a), a branching process called "Branch A: Start" is executed, and the dialogue as shown in Figure 37A(a) continues.

[0309] Furthermore, as shown in Figure 37A(b), for example, if the user utters a phrase of the type "Good morning" between the times "5:00" and "6:30", the main board 1002 should output the voice of character 131 with the phrases "Hi, good morning." and "You're up early. By the way, do you want weather information?". In this case, the main board 1002 executes a branching process called "Branch B: Start" as shown in Figure 37A(b), and in addition to continuing the dialogue as shown in Figure 37A(b), it may also execute the "weather information function" which will be described later. Here, if the user gives an affirmative response to the phrase "You're up early. By the way, do you want weather information?", the main board 1002 executes the weather information function. The weather information function is a function that provides weather information to the user, which will be explained in detail in section <3-9: Weather Information Function>.

[0310] Figure 37B(a) also shows the animation content when the user utters a phrase of the "Good morning" type between the times "5:00" and "6:30". As shown in Figure 37B(a), a different animation pattern is shown compared to Figures 37A(a) and (b). Figure 37B(a) also shows the animation content when the user utters a phrase of the "Good morning" type between the times "6:30" and "10:00". As shown in Figure 37B(a), the main board 1002 should output the voice of character 131 with phrases such as "Good morning!", "Let's have a great day today!", or "Good morning.", "Take care of yourself and let's do our best today!". In this way, unlike the case of the times "5:00" to "6:30", the animation does not include phrases that indicate the user is an early riser, and the animation is performed assuming the user woke up at a normal time.

[0311] As shown in Figure 37B(b), for example, if the user utters a phrase of the type "Good morning" between the time "6:30" and "10:00", the main board 1002 executes a branching process called "Branch B: Start" as shown in Figure 37B(b), and in addition to continuing the dialogue as shown in Figure 37B(b), it may also execute the "Weather Information Function". This is the same as in Figure 37A(b).

[0312] As shown in Figure 37C(a), for example, if the user utters a phrase of the type "Good morning" between 10:00 and 12:00, the main board 1002 may output the voice of character 131 with a phrase that expresses that the user is late getting up, such as "Good morning!" "You're late! I've been waiting for you!" or "Good morning." "Were you tired? You're up quite late..." Alternatively, the main board 1002 may output the voice of character 131 with a phrase that asks about the user's plans for the day, such as "Good morning~." "It's already noon, do you have any plans for today?" In this case, as shown in Figure 37C(a), a branching process called "Branch A: Start" is executed, and the dialogue as shown in Figure 37C(a) continues.

[0313] As shown in Figure 37C(b), for example, if the user utters a phrase of the type "Good morning" between the time "10:00" and "12:00", the main board 1002 will execute a branching process called "Branch B: Start" as shown in Figure 37C(b), and in addition to continuing the dialogue as shown in Figure 37C(b), it may also execute the "Weather Information Function". This is the same as in Figure 37A(b), etc.

[0314] Figure 37D shows the effects when the user utters a "good morning" phrase at an even later time. If the phrase is uttered between 12:00 and 16:00, the main board 1002 outputs the voice of character 131, who uses phrases that express the user's lateness, such as "You're up this late because you stayed up late, huh?" or "Unhealthy lifestyle is a keyword that describes you, isn't it?". If the phrase is uttered between 18:00 and 22:00, and the user utters "...", "It's not time for good morning anymore," or "Oh, yeah, good morning," or "What's wrong? What on earth...", the main board 1002 outputs the voice of character 131, who uses phrases that express the user's surprise or disbelief, such as a retort from character 131 if the greeting is inappropriate for the current time of day.

[0315] Figures 37E(a) and (b) show other examples of branching processes when the scenario described above as "Branch A: Start" or "Branch B: Start" is executed. In this way, multiple patterns of execution are available for branching processes as well, increasing the diversity of the presentation.

[0316] Figure 37F shows the animation when character 131 is in a "bad mood." As shown in Figure 37F, when character 131 is in a bad mood, the animation differs from the normal state, and in particular, character 131 is portrayed with a somewhat colder emotion. In this way, multiple animation patterns are available, increasing the diversity of the animations. Phrases related to recognition failure and asking for repetition are also provided. In the morning, users are more likely to say "good morning" than at other times of the day, so the number of phrases is increased compared to other times of the day.

[0317] (3-5-7-2. Hello) Figure 38 is a diagram illustrating the presentation content table, which shows the presentation content when a user utters a phrase of the type "hello". Figure 38(a) shows the presentation content when character 131 is in its normal state. As shown in Figure 38(a), for example, if a user utters a phrase of the type "hello" between the time "5:00" and "6:30", the main board 1002 will output character 131's voice with phrases that express discomfort because the time and the content of the greeting do not match, such as "Yes, hello." or "But shouldn't it be more like good morning at this time?". Between the time "10:00" and "18:00", in response to the greeting "Hello", the main board 1002 will output character 131's voice with phrases that express formality or confusion, or phrases that express discomfort because the time and the content of the greeting do not match. If the greeting is made between 18:00 and 22:00, the main board 1002 will output the voice of character 131 along with a phrase that expresses the sense of incongruity between the time and the content of the greeting, such as "Hello." or "I replied because you asked me to, but shouldn't it be more like good evening?"

[0318] Figure 38(b) shows the animation when character 131 is in a "bad mood." As shown in Figure 37(b), when character 131 is in a bad mood, the animation differs from the normal state, and in particular, character 131 is portrayed with a somewhat colder emotion. In this way, multiple animation patterns are available, increasing the diversity of the animations. During the daytime, users are more likely to use phrases of the "hello" type compared to other times of the day, so the number of phrases is increased during this time.

[0319] (3-5-7-3. Good evening.) Figure 39 is a diagram illustrating the presentation content table that shows the presentation content when a user utters a phrase of the type "Good evening". Figure 39(a) shows the presentation content when character 131 is in its normal state. As shown in Figure 39(a), for example, if a user utters a phrase of the type "Good evening" between the time "5:00" and "6:30", the main board 1002 will output the voice of character 131 with a phrase that expresses the sense of incongruity between the time and the content of the greeting, such as "Good evening?" or "Good morning, right? No matter how you look at it." If spoken between 18:00 and 22:00, the main board 1002 outputs the voice of character 131 with phrases that express being formal or confused by the greeting "Good evening," such as "Good evening is..." or "Isn't it a little, or rather, quite early?", or with phrases that express discomfort because the time and the content of the greeting do not match. If spoken between 22:00 and 5:00, the main board 1002 outputs the voice of character 131 with phrases that express concern for the user, such as "Good evening...", "Is there still something you need to do? You're busy," or "Good evening, you say," or "Are you still awake? You'll ruin your health, you should rest."

[0320] Figure 39(b) shows the presentation when character 131 is in a "bad mood." As shown in Figure 39(b), when character 131 is in a bad mood, the presentation differs from the normal state, and in particular, character 131 is portrayed with a somewhat colder emotion. In this way, multiple presentation patterns are prepared, increasing the diversity of the presentation. During nighttime hours, users are more likely to use phrases of the "good evening" type compared to other times of the day, so the number of phrases is increased compared to other times of the day.

[0321] (3-5-7-5. Goodnight) Figures 40A and 40B illustrate a table of animation content that shows what happens when a user utters a phrase of the "goodnight" type. Figure 40A shows the animation content when character 131 is in its normal state. As shown in Figure 40A, for example, if a user utters a phrase of the "goodnight" type between the time "5:00" and "10:00", the main board 1002 will output the voice of character 131 with phrases such as "What, you're going to sleep now? You're a night owl, aren't you? Goodnight," which expresses discomfort because the time and the content of the greeting don't match, or "Goodnight at this time?" "Take care of your health. Goodnight." For example, if spoken between 12:00 and 16:00, the main board 1002 will output the voice of character 131 with phrases such as "Good night," "Time for a nap," "Good night," and "Did I wake up too early?", which imply taking a nap or express concern about waking up too early. If spoken between 19:00 and 22:00, the main board 1002 will output the voice of character 131 with phrases such as "Okay," and "Good night! I think I'll go to sleep soon too." If spoken between 22:00 and 5:00, the main board 1002 will output the voice of character 131 with phrases such as "It's already this late... I think I'll go to sleep too. Good night." For the nighttime hours, the voice output of character 131 will be a phrase that expresses a natural feeling about going to sleep, rather than a phrase that expresses discomfort because the time and the content of the greeting do not match.

[0322] Figure 40B shows the animation when character 131 is in a "bad mood." As shown in Figure 40B, when character 131 is in a bad mood, the animation differs from the normal state, and in particular, character 131 is portrayed with a somewhat colder emotion. In this way, multiple animation patterns are available, increasing the diversity of the animations. During nighttime hours, users are more likely to use phrases of the "goodnight" type compared to other times of the day, so the number of phrases is increased during nighttime hours.

[0323] When the main board 1002 recognizes a phrase of this type, such as "goodnight," it may stop the display of the image on the display device 100 (for example, by hiding the display) and prevent the character 131 from being displayed. In this way, the user can get the feeling that the character 131 has turned off the lights and gone to sleep. In this case, the main board 1002 should spontaneously start displaying the standby screen when it is 5:00, which is the character 131's wake-up time.

[0324] (3-5-7-6. Call to Action) Figure 41(a) is a diagram illustrating the performance content table that shows the performance content when a user utters a phrase of the "calling out" type. The calling phrase may be the same as the trigger phrase, for example, but it may be different. When the main board 1002 recognizes the calling phrase, it outputs character information for performing a performance using character 131 based on the performance content table shown in Figure 41(a). Phrases output by character 131's voice include phrases such as "Hmm?", "Did you call me?", "Yes?", and "What do you need?". Due to the nature of the calling phrases, the response phrases do not change depending on the time of day. The character 131's movements should be such as raising its hand, so that the user can recognize that the trigger phrase has been recognized. Also, as shown in Figure 41(a), it is good to output phrases and display animations that show the differences in emotions and appearance when character 131 is normal, when character 131 is in a bad mood, and when character 131 is sleeping.

[0325] (3-5-7-7. Thank you) Figure 41(b) is a diagram illustrating the performance content table that shows the performance content when a user utters an utterance of the type "thank you". When a user utters a phrase of the type "thank you", the main board 1002 outputs character information for performing a performance using character 131 based on the performance content table shown in Figure 41(b). Phrases output by character 131's voice include "You're welcome!", "Let me know if you need anything else!", "That was pretty good, wasn't it?", and "Was I able to help? Let me know if you need anything else!". Due to the nature of the phrase "thank you", the response phrase does not change depending on the time of day.

[0326] If the phrase "thank you" is spoken by the main board 1002 after it has requested Ray to perform a function (in other words, after it has made a request), for example, within a predetermined time after the request or the function has been performed, or if it is the first phrase spoken after the request has been made, the main board 1002 may perform an action based on the action content table shown in Figure 41(b). Otherwise, the main board 1002 may perform an action of character 131 expressing that it does not understand why it is being thanked, such as "What are you talking about?". In addition, although only the action content for character 131 in a normal state is described here, the variety of expressions would increase if an action for when it is in a bad mood were also prepared.

[0327] (3-5-7-8. I'm off.) Figures 42A and 42B illustrate a table of animation content showing what happens when a user says an utterance of the type "I'm leaving." Figure 42 shows the animation content when character 131 is in the "normal" state. As shown in Figure 42A, for example, if a user says a phrase of the type "I'm leaving" between 5:00 and 12:00, the main board 1002 outputs voice from character 131 that expresses that the user's actions are natural and do not seem out of place in their daily routine, such as "Have a good day!", "Take care!", or "Have a good day!", "No detours, okay?". Between 12:00 and 16:00, if the user utters a phrase of the type "I'm leaving," the main board 1002 outputs the voice of character 131 with phrases that express that the user is leaving a little late compared to their usual routine, such as "Are you going out now?", "Have a good day!", "You're going out!", and "Have a good day, be careful." Between 19:00 and 5:00, if the user utters a phrase of the type "I'm leaving," the main board 1002 outputs the voice of character 131 with phrases that express surprise at the user leaving a little late compared to their usual routine, or a warning, such as "What? You're going out? Now?", "Be careful.", "It's dark, are you sure?", and "Be careful on the road at night. Have a good day."

[0328] Figure 42B shows the animation when character 131 is in a "bad mood." As shown in Figure 42B, when character 131 is in a bad mood, the animation differs from the normal state, and in particular, character 131 is portrayed with a somewhat colder emotion. In this way, having multiple animation patterns increases the diversity of the animation.

[0329] (3-5-7-9. I'm back.) Figure 43(a) is a diagram illustrating the presentation content table that shows the presentation content when a user utters an utterance of the type "I'm home". When the main board 1002 recognizes a phrase of the type "I'm home", it outputs character information for performing a presentation using character 131 based on the presentation content table shown in Figure 43(a). Phrases output by character 131's voice include phrases such as "Welcome back," "I've been waiting for you," and "Welcome back!" and "If you're finished with your business, let's talk next." Due to the nature of the greeting and the phrases, the response phrase does not change depending on the time of day. It would also be desirable to output phrases or display animations that show the differences in emotions and appearance when character 131 is normal, when character 131 is in a bad mood, and when character 131 is sleeping. Furthermore, if presentation content for the sleeping state is prepared, the diversity of presentations will increase.

[0330] (3-5-7-10. Temperature) Figure 43(b) is a diagram illustrating the performance content table that shows the performance content when a user makes an utterance of the type "temperature". The phrases of the type of temperature should be related to the temperature the user feels, such as "It's hot." or "It's cold." When the main board 1002 recognizes a phrase of temperature, it outputs character information for performing a performance using character 131 based on the performance content table shown in Figure 43. Phrases output by character 131's voice include "Hmm...", "Isn't it a little hot?", "It's cold.", and "It's kind of cold. Are you okay?". Due to the nature of the calls and phrases, the response phrases do not change depending on the time of day. The movements of character 131 should express feeling hot or cold depending on the temperature.

[0331] The main board 1002 may further output character information for performing temperature-related effects using character 131, under the following conditions: When 90 minutes have passed since the display device 100 was started and the user touches character 131, the main board 1002 may perform an effect corresponding to "hot" if the temperature measured by the temperature sensor 1631 is higher than a first temperature (e.g., 47°C), and an effect corresponding to "cold" if it is lower than a second temperature (e.g., 29°C). These temperatures are just examples determined considering the internal temperature of the display device 100, and other temperatures may be used. After this effect is performed, the main board 1002 may refrain from performing temperature-related effects for a predetermined time (e.g., 30 minutes), and then resume performing temperature-related effects after the predetermined time has elapsed.

[0332] (3-5-7-11. So cute!) Figure 44 is a diagram illustrating the performance content table that shows the performance content when a user makes an utterance of the type "cute". The phrase of the type "cute" should be a phrase in which the user praises character 131, such as "You're cute." or "You're lovely." When the main board 1002 recognizes a phrase of the type "cute", it outputs character information for performing a performance using character 131 based on the performance content table shown in Figure 44. A phrase that character 131 outputs in voice should be a phrase like "Hehe. You can say it more and more, you know?" Due to the nature of the call and the phrase, this phrase does not change depending on the time of day. Character 131's movements include "blushing" and "looking down and then smiling". If character 131 is in a bad mood, there are "turning away" and "glaring and then turning away". This is to express that even if praised while in a bad mood, character 131 does not accept it in a way that seems like an attempt to appease the praise.

[0333] <3-6. Details about Clock Mode (Time Display Function)> This section explains the details of the clock mode (time display function; an example of the first function). Here, we will explain the display of the standby screen when the clock mode is ON.

[0334] Figure 20(b), described earlier, shows the standby screen when the clock mode is ON, specifically the standby screen in the home position. The main board 1002 moves character 131 by randomly selecting and playing multiple animations or according to predetermined rules. In the home position, character 131 is shown holding a board object 132 that displays the time in front of it (for example, holding it in its hand). The home position is a common display across multiple animations (except for the time displayed on the board object 132). That is, the home position is common to all of the animation patterns (for example, the third pattern and the fourth pattern), and for example, the display of character 131 and the position of the board object 132 at the start and end of each animation are the same. The inventors have found that by doing so, even when different animation patterns are played consecutively in time, the boundaries between different animation patterns are less likely to be recognized by the user, resulting in a more natural-looking movement. Furthermore, it is preferable for the main board 1002 to leave a predetermined time interval between finishing the playback of one animation pattern and starting the next. The inventors have found that this way, the character 131 remains stationary in the home position for a while, preventing it from moving the board object 132. As a result, even when different animation patterns are played consecutively, the user is less likely to perceive the animation boundaries between the different patterns, resulting in a more natural-looking movement.

[0335] The main board 1002 further outputs character information for the animation in which character 131 moves board object 132. The details of this animation will be explained with reference to the screen examples shown in Figures 45A to 45D. In each of Figures 45A to 45D, time progresses in alphabetical order from (a) → (b) → ... This is also the case in Figures 47A to 48B, which will be described later. Note that these diagrams show images of animations actually displayed on the display device, so they may not strictly match the display on the actual display device 100.

[0336] Immediately after the clock mode is turned ON, as shown in Figure 45A(a), the board object 132 appears near the base of character 131's right foot (i.e., the board object 132 appears). Next, character 131 leans slightly forward to pick up the board object 132 with both hands (Figure 45A(b)). Next, character 131 makes a motion to lift the board object 132. The board object 132 is located near character 131's hands and moves upward with the position of the hands, but it is slightly away from the hands, giving the impression of floating and as if character 131 is using magic (Figure 45A(c)). Finally, character 131 holds the board object 132 in front of them (Figure 45A(d)). Figure 45A(d) is the same display as Figure 21(b), and this is the home position.

[0337] Next, the main board 1002 plays an animation of one of the patterns. This causes character 131 to move the board object 132 slightly downward from its home position (Figures 45A(e), (f), and 45B(a)). In this way, the user can be given the feeling that the board object 132 has weight and that character 131 is holding it with all its might. After a predetermined time (for example, 2 minutes), character 131 raises the board object 132 to its home position (Figure 45B(b)).

[0338] Next, the main board 1002 plays an animation of one of the patterns. As a result, character 131 raises board object 132 to its right front (Figures 45B(c), (d)). After a predetermined time has elapsed, character 131 returns to its home position and raises board object 132 again (Figures 45B(e), (f), Figure 45C(a)).

[0339] Next, the main board 1002 plays an animation of one of the patterns. As a result, character 131 raises board object 132 to its left front (Figure 45C(b), (c)). After a predetermined time has elapsed, character 131 raises board object 132 in the home position (Figure 45C, (d), (e), (f)).

[0340] By starting each animation pattern from the home position in this way, even if multiple animations are played in a random order, for example, the movement of character 131 moving board object 132 can be displayed with minimal user discomfort. Three animation patterns have been explained here, but there may be other animation patterns as well.

[0341] To explain the animation when the clock mode is turned from ON to OFF, the main board 1002 plays an animation for turning it OFF. Character 131 leans slightly forward as if to place the board object 132 on the ground (Figure 45D(a)). The board object 132 leaves Character 131's hand and moves downward (Figure 45D(b)). Then, the board object 132 disappears. If the position where the board object 132 disappears is the same as the position where the board object 132 appeared as described in Figure 45A(a), it will appear as if Character 131 is returning the board object 132 to its original position.

[0342] The main board 1002 may perform such time announcements when it receives input from the user requesting the execution of the time display function, such as "Tell me the time." In this case, the main board 1002 may terminate the animation related to the time display function by displaying animations for a certain period of time or a certain number of times, or by receiving input from the user requesting termination, such as "I know the time now." In this case as well, the animation described in Figure 45D may be performed.

[0343] Figure 46 shows the standby screen displayed when character 131 is wearing pajamas and clock mode is ON. Figure 46 shows the standby screen with clock mode ON and the home position displayed. When character 131 is wearing pajamas, the same effects as those described in Figures 45A to 45D should be performed, except that the costume is different.

[0344] Furthermore, the time may be displayed on objects other than board object 132.

[0345] <3-7. Character 131's presentation during sleep> Figures 47A and 47B show the animations performed when character 131 is sleeping. Based on the animation content tables shown in Figures 47A and 47B, the main board 1002 outputs character information for performing the sleep animation using character 131. This section describes the operation when the clock mode is ON. When it is time for character 131 to sleep (in this case, when it is 0:00), the main board 1002 plays the sleep animation. If the standby screen shown in Figure 47A(a) was displayed immediately before the sleep animation was played, the board object 132 moves upward from character 131's hand and then disappears (Figures 47A(b), (c), (d), (e)). In addition, character 131 stretches and yawns, and floats in the air with its legs bent (Figures 47A(e), (f), Figure 47B(a)). When character 131 lies down while floating and moves downwards (Figures 47B(a), (b)), a bed object 133, which is an object resembling a bed, is displayed, and the scene of character 131 going to sleep on the bed object 133 is displayed (Figure 47B(c))). During the time character 131 is sleeping, from 0:00 to 5:00, this standby screen is displayed. When clock mode is OFF, the animation displayed by board object 132 should not be performed, or the animation should start from the one described in Figure 47A(d). When clock mode is OFF, the animation displaying board object 132 should not be performed.

[0346] Furthermore, if touch input is detected on character 131 while the character is sleeping, the main board 1002 may, if the number of touch inputs is less than a predetermined number per unit time, show character 131 performing a motion of discomfort while sleeping, and then depicting the character sleeping normally. If the number of touch inputs is the predetermined number per unit time, the main board 1002 may show the character falling from the bed object 133, waking up, and then returning to a sleeping state after 2-3 minutes.

[0347] <3-8. Scenes upon waking up> Figures 48A and 48B show the animation content when character 131 wakes up. The main board 1002 outputs character information for performing the wake-up animation using character 131, based on the animation content table shown in Figures 48A and 48B. Here, we will explain the operation when the clock mode is ON. When it is time for character 131 to wake up (in this embodiment, when it is 5:00 AM), the main board 1002 plays the wake-up animation. First, character 131 gets up on the bed object 133 (Figure 48A(a)), and when character 131 gets out of the bed object 133 and stands up, the bed object 133 disappears (Figures 48A(b), (c), (d)). The standing character 131 then walks towards the front (user side), and the board object 132 appears above character 131 (Figures 48A(e), (f)). This appearance position should be the same as the disappearance position explained in Figure 47A(e). Then, when character 131 makes an action to pick up board object 132 (Figure 48B(a)), the board object 132 moves down to the position of character 131's hand, and the operation when time mode is ON, as described above, begins (Figure 48B(b)). When clock mode is OFF, it is preferable that the animation for displaying board object 132 is not performed, and that after the animation explained in Figure 47A(e), the standby screen shown in Figure 1B(b) is displayed.

[0348] <3-9. Details of the Timer Function> Figures 49A to 49E illustrate the presentation content table showing the presentation content related to the timer function (examples of the first and second functions). Based on the presentation content tables shown in Figures 49A to 49E, the main board 1002 outputs character information for performing presentations related to the timer function using characters 131 and 150. Figures 49A to 49D describe the presentation content when character 131 is in a normal state, and Figure 49E(a) describes the presentation content when character 131 is in a bad mood. Figures 49E(b) and (c) describe the presentation content when character 131 is sleeping.

[0349] First, the user speaks a request to character 131 to execute the timer function. Possible phrases include, "Rei-chan, set the timer," or "Rei-chan, please set the timer." When character 131 is in its normal state and recognizes such a phrase, the main board 1002 performs one of several scenarios (performance content) associated with "Pattern 1," as shown in Figures 49A to 49C. For example, the main board 1002 outputs character 131's voice, which includes a phrase indicating that the execution of the timer function has been acknowledged, such as, "Okay! How many minutes later?" or "Okay! How many minutes later?", along with a phrase asking the user for the time (duration) to set the timer. Along with this output of character 131's voice, the main board 1002 displays an animation, as shown in Figure 50, in which character 131 appears to be listening intently to the user. This makes it easier for the user to visually recognize that they need to tell character 131 the time to set. The user speaks the time they want to set the timer for, for example, "3 minutes." When the main board 1002 recognizes such a speech, it executes the scenario of pattern 2. If the user is silent, denies it, or is unrecognizable, the main board 1002 should execute the corresponding scenario. In this case, if the main board 1002 needs to ask the user for clarification, it should output the voice of character 131 with a phrase such as, "Sorry, I didn't understand. How many minutes later do you want to set the timer?" and prompt the user to speak the time they want to set the timer for again.

[0350] If the scenario in Pattern 2 is executed, the main board 1002 will output a phrase in the voice of character 131 to confirm with the user whether the time spoken by the user was correctly recognized. Such a phrase might be, "I'll set it for XX minutes from now, is that okay?" (where "XX minutes" is the time recognized as the time the timer was set). If the user responds with an affirmative utterance (answer) such as "Yes" or "Uh-huh," and the main board 1002 recognizes this, the main board 1002 will execute the scenario in Pattern 3 and set the timer for that time. Phrases used in this execution include phrases that express understanding, such as "Okay! Timer start!" or "Timer countdown, start!", and phrases that indicate the start of the timer countdown. On the other hand, if the user is silent, denies it, or is not recognized, the main board 1002 should execute the corresponding scenarios shown in Figures 49A to 49C. If the main board 1002 needs to ask the user for clarification, it outputs the voice of character 131 with a phrase such as, "Sorry, I didn't quite understand. I'll set it for XX minutes from now, is that okay?" to prompt the user to speak the time to set the timer again. Once the timer is set, the main board 1002 determines, for example, whether the set time has elapsed based on the time measured by the RTC 1630. When that time has elapsed, it flashes the main switch 310 yellow and notifies the user with the effects shown in Figure 49D. The main board 1002 may select one of several patterns of effects, such as "Time's up!", and output the voice of character 131.

[0351] If character 131 is in a bad mood, as shown in Figure 49E(a), the main board 1002 outputs a voice message from character 131 saying, "The timer is unavailable right now," and prevents the function from being executed. The main board 1002 may choose not to execute the function every time, or it may execute it with a predetermined probability, and at the very least, the execution of the timer function should be restricted. For example, if character 131 was in a normal state when the timer was set, but became unhappy as time passed, the notification may be given in an unhappy state.

[0352] Furthermore, while character 131 is awake when the timer is set, character 131 may be asleep after the set time has elapsed. In such cases, the effect shown in Figure 49E(c) is performed. Specifically, when the set time has elapsed, the main board 1002 makes character 150 appear on foot from the side of the screen and displays the information shown in Figure 51. The main board 1002 then makes the main switch 310 flash yellow and outputs a phrase such as "It's time for the timer. Wake up!" through the voice of character 150. In Figure 49E, the phrase written as "Teru(...)" means that the phrase in the "..." part is spoken by the voice of character 150.

[0353] Additionally, character 131 may be asleep when the timer is set. In such cases, a sequence is performed in which character 150 appears from the moment the timer is set. First, the main board 1002 performs one of the scenarios shown in patterns 1 to 3 in Figure 49E(b), outputting character information for the sequence in which character 131 calls character 150. For example, character 131's voice is output with the phrase, "Munya... Teru, good luck with the rest." Then, character 150 walks in from the side of the screen, and the display shown in Figure 52 appears. The main board 1002 outputs a voice from character 150 asking, "How many minutes later do you want to set the timer?", and interacts with the user regarding the timer setting. Once the timer is set, character 150 walks to the side of the screen and exits. The sequence after the set time has elapsed is as already explained using Figure 49E(c). If character 131 is awake after the set time has elapsed, the sequence shown in Figure 49D may be used to notify the user with character 131's voice.

[0354] The main board 1002 should initiate the animation using character 150 not at the end of character 131's animation, but when the output of character 131's voice in the animation using character 131 ends. This way, a smooth execution can be achieved between the animation in which character 131 recognizes that character 150 will accept the request, and the animation in which character 150's request is fulfilled.

[0355] If the main board 1002 detects a user touch operation during the animation in which character 150 walks in from the side of the screen, it may skip that animation, cancel the timer function itself, and perform an animation in which character 150 turns back.

[0356] In this embodiment, only one timer set can be set using the timer function, but it may be possible to set multiple timers. If a timer is set again within the specified time frame, the previously set timer should be overwritten. It is preferable that the timer can be set only in minute increments (for example, settable time: 1 minute to 99 minutes), but it may also be possible to set it in finer increments of seconds or hours. It is also preferable to prevent the setting of two or more timers at the same time on the same day of the week.

[0357] The main board 1002 may stop the timer function notification when the main switch 310 is pressed. However, instead of or in addition to such user operation, the notification may also be stopped when a person is no longer detected using a human detection sensor (e.g., a motion sensor) such as an infrared sensor, microwave sensor, or camera. Furthermore, the timer function may be configured so that it does not notify even after the set time has elapsed if the display device 100 is not connected to the server via communication.

[0358] The main board 1002 should cancel the timer if the user instructs it to be canceled, for example, if there is a voice input such as "Cancel the timer, Rei-chan."

[0359] <3-10. Details of the alarm function> Figure 53 is a diagram illustrating a presentation content table showing the presentation content related to the alarm function (examples of the first and second functions). When the set time set according to the alarm setting screen arrives, the main board 1002 outputs character information for performing presentations using characters 131 and 151 related to the alarm function, according to Figure 53. This alarm output differs depending on the mood of character 131, as well as the time of day. If it is during the morning hours of "6:30 to 10:00", the main board 1002 outputs the voice of character 131 with phrases such as "Good morning! It's morning! Wake up! Hey, wake up!" or "Wake up! Hey, wake up! It's morning!" If it is during the afternoon hours of "12:00 to 16:00", the main board 1002 outputs the voice of character 131 with phrases such as "It's time for the appointment!" or "It's time! Are you okay?" When character 131 is asleep, the main board 1002 should bring out character 150 in the same manner as the timer function (see Figure 52) and output character 150's voice saying, "It's time! Put it down!". When the user presses the main switch 310 while the alarm is sounding, the main board 1002 will output a phrase indicating that the user has understood, using the voice of character 131 or character 150, and stop the alarm.

[0360] The alarm output should be combined with an electronic sound or the like, and should continue until the user notices it. The main board 1002 flashes the voice recognition button yellow at the set date and time to output an alarm to the user. If the snooze function is ON, the main board 1002 does not stop the snooze function even if it detects a user touch operation, and outputs the alarm again after the time set by the snooze function has elapsed. If it detects that the main switch 310 is pressed, the main board 1002 also stops the snooze function. If the snooze function is OFF, the main board 1002 stops outputting the alarm in both cases: when a user touch operation is detected and when the main switch 310 is pressed. In addition to or instead of such user operation, the main board 1002 may also stop outputting the alarm when a person is no longer detected using a human detection sensor such as an infrared sensor, microwave sensor, or camera. Furthermore, the alarm function may be configured not to notify even at the set date and time if the display device 100 is not connected to the server via communication.

[0361] <3-11. Weather Information Function> Figures 54A and 54B are diagrams illustrating the presentation content table showing the presentation content related to the weather guidance function (an example of the second function). Based on the presentation content table shown in Figures 54A and 54B, the main board 1002 outputs character information for performing the weather guidance function using characters 131 and 150. The weather guidance function is a function in which characters provide weather information to the user. The flow of the weather guidance function is generally as follows. (Step 1) Character 131 responds to the user's request (utterance). (Step 2) Character 150 appears and reads out the weather information. (Step 3) Character 131 speaks the closing phrase.

[0362] To explain in more detail, a user who wants to know the weather will utter a phrase requesting the weather information function to be executed. This phrase should be a request to character 131 for weather information, such as "Rei-chan, what's the weather like tomorrow?" or "Hey, hey, tell me the weather in Shizuoka." When character 131 is in a normal state, if the main board 1002 recognizes such a phrase, it will perform one of the Pattern 4 scenarios based on the performance content table shown in Figure 54A(a). This performance includes a scene in which character 131 requests weather information from character 150. Phrases in this performance include "It's the weather forecast, leave it to me!" or "So, Teru, please!"

[0363] Next, the main board 1002 displays character 150 using the display described in Figure 51 or Figure 52, and performs a weather reading using character 150's voice. The main board 1002 connects to the internet via wireless LAN or the like using the communication module 1650, communicates with a predetermined server device (for example, a weather forecast server), and obtains weather information. Then, the main board 1002 performs a weather reading using character 150's voice. After this reading is finished, character 150 walks to the side of the screen and exits. Once the weather information guidance is complete in this way, the main board 1002 performs a scene associated with one of the scenario names "End 1" to "End 3," with an ending phrase such as "Thanks, Teru."

[0364] Regarding the weather, if the user's utterance is not specific in location, such as "Rei-chan, what's the weather like tomorrow?", the weather for the residential area (more specifically, the prefectural capital) set in the user's information will be notified. If the location is not specified, such as "Hey, hey, tell me the weather in Shizuoka," it would be good to be informed of the weather for that location. Also, if the user specifies when they want to know the weather, such as "Rei-chan, what's the weather like tomorrow?", it would be good to be informed of the weather at that time. If the weather is not specified, such as "Hey, hey, tell me the weather in Shizuoka," it would be good to be informed of the weather for a predetermined period. For example, if the request is before 6 AM, the weather for the morning could be provided; if it's after 6 AM, the weather for the afternoon could be provided; and if it's at night, the weather for tomorrow could be provided.

[0365] If character 131 is in a bad mood, the main board 1002 will output character 131's voice saying phrases such as "Weather?" or "No. I don't want to do it now," as shown in Figure 54A(b), and will not provide weather information. This may not be done every time, but it may be done with a predetermined probability, and at the very least, the execution of the weather information function should be limited.

[0366] If character 131 is asleep, the animation shown in Figure 54B is performed. In this case as well, when the main board 1002 recognizes a phrase requesting the execution of the weather guidance function, it performs one of the Pattern 4 scenarios according to Figure 54B. This scenario also includes a character animation in which character 131 requests character 150 to give the weather guidance. When the weather guidance is finished, a closing phrase is spoken in character 131's voice, and character 150 exits. Phrases requesting the weather guidance include, "It's the weather forecast, leave it to me!" and "So, Teru, please!". Once the weather guidance is complete, the main board 1002 may perform an animation associated with one of the scenario names "End 1" to "End 28". Closing phrases include, as shown in Figure 54B, "...that's what they say. It'll be sunny, goodnight..." and "...that's what they say. Watch out for the rain...". In this way, the expression is as if it were secondhand information from character 150, and the part with "..." before "because" should include a general description of the weather, such as sunny or rainy. This gives the user the impression that character 131 is telling them about the weather. This effect can also be used when character 131 is awake.

[0367] Incidentally, the reason why character 150 is made to read out the weather is that character 131 uses pre-recorded voice data (pre-recorded voice data) and outputs predetermined voices based on the voice data stored in the memory unit 1670, whereas character 150's voice is output through voice synthesis processing. The main board 1002 provides weather information by converting the text data of weather information acquired from the server and outputting it as voice. This makes it possible to provide voice guidance even for weather information that changes moment by moment. If weather guidance is possible with character 131's voice, the main board 1002 may provide weather guidance using character 131's voice. Alternatively, the main board 1002 may display a speech bubble for weather guidance and provide weather information using display elements such as text and images within it.

[0368] Furthermore, if the same weather information is to be announced two or more times under the same conditions, the main board 1002 may, from the second time onward, refrain from showing the animation of character 150 walking in, and instead immediately read out the weather information using character 150's voice. This is because, from the second time onward, the user may want to check the weather again, and it may be desirable to have it read out quickly.

[0369] Furthermore, if the user requests the weather information function from near the display device 100, the main board 1002 may read out the weather information in the manner described above. If the user requests the weather information function from far away from the display device 100, the main board 1002 may read out the weather information without displaying the animation related to the weather information function. In this case, if the user is far away from the display device 100, the user cannot see the display on the display device 100, which is not a problem, and it is also advantageous in that the user can quickly learn the weather information. Whether the user is near the display device 100, far away from the display device 100, or near the display device 100 can be detected by methods such as using the human detection sensor described above.

[0370] Furthermore, it was explained that if a phrase of the type "Good morning" is recognized in the dialogue processing described above, the main board 1002 may execute the weather information function. In this case as well, the main board 1002 may provide weather information using the presentation described in Figure 54A(a). Although weather information has been described here, other current events information such as news, or other information, may be provided instead of or in addition to weather information.

[0371] <3-12. Anniversary notification function> Figure 55 is a diagram illustrating a presentation content table showing the presentation content related to the anniversary notification function (an example of the first function). When a user makes a predetermined input to the display device 100 on a predetermined birthday, the main board 1002 performs a birthday-related presentation based on the presentation content table shown in Figure 55. In addition to birthdays, other anniversaries include purchase anniversaries. If a user makes a predetermined input after a predetermined period has elapsed since the purchase date (for example, the day the display device 100 was first started), the main board 1002 may perform a presentation to celebrate the purchase anniversary. For example, in the case of the first anniversary from the purchase date, the presentation may only be performed on the same day one year after the purchase date, but if it is performed thereafter, for example until the next anniversary, the possibility of the user missing the anniversary presentation is reduced. Other anniversaries may include the birthday of character 131. Predetermined inputs may include touch operations on the head of character 131 or other inputs. The head of character 131 may include an area to display character 131's hair, but if it is not included, the realism in terms of detecting touches on the head will be increased.

[0372] <3-13. Gallery Function> Figure 56 shows a screen related to the gallery function displayed by the main board 1002 when the icon 1750 shown in Figure 19A(b) is operated. The main board 1002 is configured to display a list of phrases from character 131 that the user has heard (also called unlocked or acquired) so far on this screen Op9, and to play those phrases using character 131's voice. On the other hand, the content of phrases that the user has not yet heard is displayed as "???", and the content of the phrase is not displayed, nor can the voice be heard. The completion rate is the ratio of the number of phrases the user has heard to the total number of phrases. The total number of phrases is hidden to attract the user's interest and to encourage conversation among users. In Figure 56, buttons Ct1 to Ct5, which are displayed as "???", display the type of phrase the user has uttered, such as trigger phrases, good morning, good afternoon, good evening, good night, etc. When a touch operation is detected on any of the buttons Ct1 to Ct5, the main board 1002 has the function of checking and playing back previously released (acquired) phrases or animations (motions) for the corresponding phrase type.

[0373] In the gallery function, the main board 1002 may display not only phrases that have already been unlocked, but also information that identifies when those phrases were unlocked (for example, how many days ago or the date), or it may display them in a way that allows users to identify the first phrase unlocked or the most recently unlocked phrase. In this way, users can reminisce about what they did and when.

[0374] In the gallery function, the main board 1002 may display phrases that were not released by the anniversary notification function, even if they have not been released by the user. This way, even if the user misses an anniversary phrase, they can listen to it later. In the anniversary notification function, for example, if there is another anniversary after a purchase anniversary, the user may not be able to hear the phrase for that anniversary, but the user can listen to it by using the gallery function. Alternatively, the main board 1002 may not display anniversary phrases that occur at predetermined intervals in the gallery function unless they are released, while displaying anniversary phrases that occur only once in the gallery function even if they are not released.

[0375] Furthermore, the main board 1002 may display the unlock status of phrases in a timeline with respect to the gallery function. For example, the main board 1002 may display a graph where the horizontal axis represents time (e.g., date) and the vertical axis represents the number of phrases unlocked during that period, the total number of phrases unlocked up to that point, or the completion rate. In this way, the user can recall how they have interacted with character 131 based on the unlock status of phrases in the past.

[0376] <3-14. External Device Control Function> Figures 57A and 57B illustrate a presentation content table showing the presentation content related to the external device control function. Based on the presentation content tables shown in Figures 57A and 57B, the main board 1002 outputs character information for performing presentations related to the external device control function using character 131. The external device control function is a function that controls external devices in response to input from the user requesting the execution of a predetermined function. Figure 58 is an example of the overall configuration of a system for performing external device control functions, and illustrates a display device 100 connected to a network 1810 according to one embodiment. In the embodiment shown in Figure 58, the display device 100 is connected to a smart home appliance remote control 1800, a television 1820, a user terminal 1830, an audio system 1840, an air conditioner 1850, lighting 1860, and a vehicle 1870 via a network 1810. The standard of the network 1810 is not particularly limited, but includes, for example, wireless communication LANs of Wi-Fi® or other standards, and public communication lines of LTE, 4G, 5G or other standards. The network 1810 may be a wireless communication path, but may also include a wired communication path. In the embodiment shown in Figure 58, the display device 100 has an input unit that accepts input from the user, particularly voice input. By having the display device 100 work in cooperation with the smart home appliance remote control 1800, the user can use the display device 100 to make input to the smart home appliance remote control 1800.

[0377] For example, a user can input commands and instructions to the display device 100, such as a television 1820, user terminal 1830, audio system 1840, air conditioner 1850, lighting 1860, and vehicle 1870, which are designed and configured as smart home appliances, by voice or other means. Input may also be made by other means, such as tapping. The display device 100 then receives these commands and instructions as requests from the user and controls the external devices to perform functions corresponding to those requests.

[0378] Furthermore, the phrase "to the display device 100" in this explanation can be reinterpreted as "to the character displayed by the display device 100." This allows users to naturally instruct, command, or request IoT control from a familiar character they frequently see on a daily basis. This reduces the awkwardness associated with talking to a machine, as is often the case with conventional systems.

[0379] When the main board 1002 receives a request to execute an external device control function, it outputs character information for performing a performance using character 131 based on the performance content table related to external device control shown in Figures 57A and 57B. If character 131 is in a normal state, the main board 1002 performs the performance shown in Figures 57A and 57B and executes the external device control. If character 131 is in a bad mood, it outputs the voice of character 131 of the phrase described in the row labeled "Angry" (which can be read as "bad mood") in Figure 57A(b), and prevents the execution of that function. It is possible to prevent its use every time, or to execute it with a predetermined probability, and at the very least, it is preferable to limit the execution of the external control function.

[0380] For example, in addition to the display set on the display device 100, the display device 100 can also display additional screens for operating connected devices or equipment. For example, if the display device 100 has a carbon dioxide sensor, and the carbon dioxide concentration detected by the sensor exceeds a predetermined value, the display device 100 can activate a ventilation fan or improve ventilation efficiency. Furthermore, it can also output messages such as "Please open the windows and ventilate" in the voice of the displayed character.

[0381] This configuration makes it easier for users to accept requests and warnings from characters than they would from typical machine-generated notifications. In the embodiment shown in Figure 58, the display device 100 can also output notifications in response to information from sensors and other devices connected to it. In other words, the display device 100 can extend the use of sensors and other devices connected to it.

[0382] For example, based on information from sensors installed in the vehicle 1870 and sensors installed in on-board equipment such as camera equipment, it is possible to output notifications about the shaking of the vehicle 1870, the approach of suspicious persons, the presence or absence of passengers inside the vehicle, the temperature inside the vehicle, etc. Furthermore, for example, the video captured by the camera on vehicle 1870 can be displayed as an aerial image 130.

[0383] Furthermore, the display device 100 can also connect to a server of the company that sells the display device 100 via the network 1810. This allows the display device 100 to obtain information necessary for updates, backups, etc., from the server.

[0384] [4. Functions of User Terminal 1830] The user terminal 1830 is preferably configured to communicate with the display device 100 to implement various functions. The user terminal 1830 is a communication terminal that communicates with the display device 100 via the network 1810. The user terminal 1830 displays various information based on the information acquired from the display device 100. The user terminal 1830 may communicate directly with the display device 100, or it may communicate via a server. In the latter case, it is desirable for the information transmitted and received between the display device 100 and the user terminal 1830 to be registered on the server, as this serves as a backup. In this embodiment, the user terminal 1830 is a smartphone, but it may be a communication terminal other than a smartphone. For example, the user terminal 1830 may be a portable communication terminal such as a tablet, feature phone, or wearable device. The user terminal 1830 may also be a stationary communication terminal such as a desktop personal computer.

[0385] <4-1. Electrical configuration of user terminal 1830> Figure 59 is a block diagram showing the electrical configuration of the user terminal 1830. As shown in Figure 59, the user terminal 1830 comprises a control unit 1831, an audio input / output unit 1832, a communication unit 1833, a UI (User Interface) unit 1834, a storage unit 1835, and an imaging unit 1837.

[0386] The control unit 1831 includes a computer with a CPU, ROM, and RAM. The CPU controls various parts of the user terminal 1830 by reading programs stored in the ROM and memory unit 1835 into the RAM and executing them. The audio input / output unit 1832 has a microphone and speaker and realizes functions related to audio input and output in the user terminal 1830. The communication unit 1833 includes, for example, a wireless communication circuit and antenna and is an interface for connecting to a communication line NW. The UI unit 1834 includes, for example, a touch panel and accepts user operations and notifies information by displaying images. The memory unit 1835 includes, for example, an EEPROM (Electrically Erasable Programmable ROM) and stores the OS and applications 1836 that run on this OS as programs executed by the control unit 1831. Application 1836 is an application program that communicates with the user terminal 1830 to realize various functions. The functions of application 1836 may be realized by firmware or programs other than application programs. The imaging unit 1837 captures images and generates the captured images. The imaging unit 1837 may be equipped with an image sensor and an imaging lens, such as a CCD or CMOS.

[0387] <4-2. Functions of User Terminal 1830> The functions performed by the control unit 1831 of the user terminal 1830 based on the application 1836 will be described. (4-2-1. Displaying the Home Screen) Figure 60 shows the home screen SC1 displayed in the UI unit 1834. The home screen SC1 displays the message "Status" along with information indicating the current state of character 131 and the relationship between the user and character 131. The home screen SC1 displays an image of character 131 and character 131's mood as a state of being. When character 131 is in a normal state, as shown in Figure 60(a), the text "Ray's mood: Normal" is displayed along with image IM1, which shows character 131 with a calm expression. When character 131 is in a bad mood, as shown in Figure 60(b), the text "Ray's mood: Angry" (or "Ray's mood: Bad mood") is displayed along with image IM2, which shows character 131 with an angry expression. Images IM1 and IM2 are still images here, but they may also be moving images (e.g., animations). Additionally, the home screen SC1 displays the number of days elapsed since the purchase date (i.e., the initial startup of the display device 100) along with the text "Since I met you."

[0388] The control unit 1831 of the user terminal 1830 may obtain information necessary for displaying the home screen SC1 from the display device 100 or the server at a timing instructed by the user, or at a predetermined timing without user instruction, and display the home screen SC1 based on the obtained information. Other information may also be displayed on the home screen SC1.

[0389] This home screen SC1 is displayed in a way that makes it appear as if it is showing the inner thoughts of character 131. On the other hand, the display device 100 displays character 131 as if it were a real person, so the relationship between the user and character 131, such as character 131's mood, is not displayed in text. Furthermore, if character 131's animation is displayed only on the display device 100 and not on the user terminal 1830, it is possible to express to the user the worldview that Ray is the only one of his kind in the world. The control unit 1831 may also display an image in the area where images IM1 and IM2 are displayed that depicts the appearance of the display device 100 and the same image as the aerial image 130. For example, it would be good if the same image as the aerial image 130, including the housing 110 as shown in Figure 1A(b), is displayed on the UI unit 1834. In this case, the control unit 1831 may display an image that is a composite of an image that mimics the housing 110 of the display device 100 and the same image as the aerial image 130.

[0390] (4-2-2. Settings for display device 100) The application 1836 should be configured so that all or part of the settings that were previously configured via the settings menu on the display device 100 can now be configured using the user terminal 1830. This is because if the user is familiar with operating the user terminal 1830, it may be easier for them to configure the settings this way.

[0391] Figure 61(a) shows the main settings screen C2. The main settings screen C2 displays setting menus for "Residential Area," "Birthday," and "Wireless LAN." When a touch operation is detected on the "Residential Area" menu, the control unit 1831 displays the settings screen shown in Figure 61(b). The main board 1002 sets the user's residential area via this screen. When a touch operation is detected on the "Birthday" menu, the control unit 1831 displays the settings screen shown in Figure 61(c). The main board 1002 sets the user's birthday via this screen. When a touch operation is detected on the "Wireless LAN" menu, the control unit 1831 displays the settings screen shown in Figure 61(d). Since the display device 100 is configured to communicate by connecting to the internet line via wireless LAN, settings related to that communication can be made using this settings screen. When settings are made via these screens, the control unit 1831 transmits data indicating the settings to the display device 100, either directly or via the server. In the display device 100, the main board 1002 performs settings based on this data.

[0392] Furthermore, application 1836 is configured to allow alarm settings. Figure 62(a) shows the main alarm setting screen C3. As shown in Figure 62(a), the top of the setting screen C31 displays items corresponding to each day of the week from "Sunday" to "Saturday". When any day of the week is touched and a touch operation is detected on the button marked with a "+" inside a circle, the control unit 1831 displays screen C32 for setting alarms for that day of the week, as shown in Figure 62(b). The setting items shown in Figure 62(b) are the same as the setting items described in alarm setting screen Op63, so their explanation is omitted. For example, if the "Time" item is selected, the control unit 1831 displays screen C33 as shown in Figure 62(c). The user sets the time at which the alarm will be output by operating this screen C33. When settings are made via these screens, the control unit 1831 transmits data indicating the settings to the display device 100, either directly or via the server. In the display device 100, the main board 1002 performs settings based on this data.

[0393] Figure 63(a) shows the screen for displaying version information, displaying licenses, and logging out. Figure 63(b) shows the screen displayed when version information is selected in Figure 63(a).

[0394] (4-2-3. Other examples of functions performed in Application 1836) The functions of user terminal 1830 in the following description may be implemented by the functions of application 1836. (4-2-3-1) Application 1836 may allow further settings to be made, or it may display other information that is displayed on the display device 100. For example, the control unit 1831 may display a screen based on the gallery function described above, based on application 1836.

[0395] For example, the control unit 1831 may have a function to upload images of a screen based on the gallery function (for example, the same screen as the drawing shown in Figure 56) to a predetermined SNS (Social Networking Service). The control unit 1831 may also have a function to upload animations displayed as aerial images 130 to the SNS in order to share the content of communication between the user and character 131 with other users. Furthermore, the control unit 1831 may be configured to upload the history of communication between character 131 and the user on the display device 100.

[0396] (4-2-3-2) Voice input may be performed via the user terminal 1830. For example, the user inputs a botrigger phrase into the user terminal 1830. The control unit 1831 in the user terminal 1830 may perform voice recognition based on the voice recognition function within the device, and send the recognized phrase as text data to the server or display device 100 so that it is input to the display device 100. In this way, it may be possible to utilize the high-quality voice recognition function of the user terminal 1830, and it is possible to express a world view in which the user terminal 1830 is used as an item for communicating with character 131.

[0397] In addition to the display device 100, the user terminal 1830 may also be configured to output character information for performing character-based effects based on the application 1836. For example, character information for performing effects as described in (4-2-3-3) to (4-2-3-8) may be output as follows.

[0398] (4-2-3-3) The control unit 1831 of the user terminal 1830 may display the character 150 on the UI unit 1834. This display is even better if it is done during a period when the character 150 is not displayed on the display device 100. Furthermore, the control unit 1831 may have the character 150 provide guidance on how to use the display device 100 or the application 1836 (for example, by giving hints on how to use it). For example, if character 131 is in a bad mood, the control unit 1831 may have the character 150 provide guidance on how to cheer up character 131.

[0399] (4-2-3-4) When a user is out and speaks to the user terminal 1830, expressing their intention to return home, such as "I'm coming home now," the control unit 1831 may display a message from character 131, such as "Come home soon," on the UI unit 1834. Furthermore, when the display device 100 detects that the user has returned home, the main board 1002 may display character 131 and output character 131's voice saying a phrase such as "Welcome home." The detection of the user's return home may be determined by the main board 1002 based on one or more of the following: location information obtained from the user terminal 1830, an increase in illuminance measured by the illuminance sensor 1632 (meaning the lights in the house have been turned on), or the detection of a person nearby by the person detection sensor. Such functions of the display device 100 may be implemented as functions that do not interact with the application 1836.

[0400] (4-2-3-5) In the display device 100, when the user says the phrase "goodnight", the main board 1002 may perform a function to notify the user of information regarding the status of the user terminal 1830. For example, the main board 1002 may obtain battery level information from the user terminal 1830 and, if it is below a threshold, output a phrase such as "Battery is dead!" in the voice of character 131. In this way, the user can be prevented from forgetting to charge the user terminal 1830.

[0401] (4-2-3-6) The display device 100 may have a function to cooperate with other applications running on the user terminal 1830. For example, the main board 1002 may select the type of dialogue (e.g., theme) in the dialogue processing based on the music playback history or video sharing site browsing history on the user terminal 1830. Also, if the main board 1002 determines that there is a search history related to Rei or a search history of things that Rei likes, it may perform a process to change the state of character 131 from unhappy to normal. Furthermore, cooperation with other applications is desirable, such as applications in which Rei appears, or other applications that have a predetermined relationship with character 131.

[0402] (4-2-3-7. Outing function) The application 1836 should have a function that allows the user to go out (go on an outing) with the character 131 in conjunction with the display device 100. Hereinafter, this function will be referred to as the "outing function". The outing function is a function that allows the user to take the character 131 from the display device 100 to a user terminal 1830 that the user can carry around, and allows the user to walk around with the character, for example, to feel as if they are on a date with the character 131. Specific implementation methods include, for example, the following methods.

[0403] (4-2-3-7-1. Occurrence of an outing event) Initially, the outing function is locked, and the outing event, which allows the user to take character 131 to the user terminal 1830, is unlocked when certain conditions are met. For example, the outing event may occur with a predetermined probability (for example, randomly) in response to an invitation phrase uttered by the user to character 131 displayed on the display device 100. The outing event may occur with a predetermined probability when character 131 is in a normal state, but may not occur if character 131 is in a bad mood or sleeping. The outing event may also occur when the user and character 131 become close, for example, when the intimacy level exceeds a threshold. Under this configuration, since the outing event occurs when the user goes out, the display device 100 may restrict the days on which the outing event occurs to certain days such as Saturdays, Sundays, and public holidays. This is done, for example, to prevent the outing event from occurring when the user goes to work on weekdays. The invitation phrases that trigger the outing event should be predetermined phrases such as "I'm leaving" or "Let's go for a walk" when spoken to the character 131 displayed on the display device 100.

[0404] When an outing event occurs, the main board 1002 outputs the voice of character 131 as a response phrase to the invitation phrase, such as "Take Rei with you" or "I want to go with you." The first time an outing event occurs, a special phrase may be output. For example, the voice of character 131 may be output as a special phrase such as "We can go out for the first time!" On the other hand, if an outing event does not occur in response to the invitation phrase (i.e., the event extraction fails), the main board 1002 may output the voice of character 131 as a response phrase such as "Have a good trip" or "I'm not going."

[0405] In the user terminal 1830, the control unit 1831 may display character 131 on the UI unit 1834 when an outing event occurs. Except when an outing event occurs, the image of character 131 may not be displayed except for images IM1 and IM2 described above on the home screen SC1, and only when an outing event occurs may character 131 for the outing function be displayed on the home screen SC1. During the period when character 131 is displayed on the user terminal 1830, the main board 1002 may prevent character 131 from being displayed on the display device 100. In this way, it can be expressed that character 131 has moved to the user terminal 1830, and the user can feel the worldview that Rei is the only one in the world.

[0406] The conditions for an outing event to occur may be any of the following methods (A) to (C).

[0407] (A) Use local communication via Bluetooth / WLAN. (Step 1) Connect the display device 100 and the user terminal 1830 via Bluetooth / WLAN. (Step 2) The control unit 1831 launches the application 1836 and displays a screen that includes the "Go Out" button. The user taps (touches) this button. (Step 3) When the user speaks an invitation phrase to the display device 100, character 131 responds with a phrase. When an outing event occurs, the display device 100 displays an animation in which character 131 disappears. (Step 4) Character 131 appears on user terminal 1830, and the outing begins. This way, you can go out even without using an internet connection.

[0408] (B) Communication is performed via the server. (Step 1) The display device 100 and user information (e.g., ID / password) are linked on the server in advance. (Step 2) The control unit 1831 launches the application 1836 and accepts the user's login operation. Once login is permitted, it displays a screen including an "Go Out" button. The user taps this button. The control unit 1831 sends a "Ready to Go Out" notification to the display device 100 via the server. (Step 3) When the user speaks an invitation phrase to the display device 100, character 131 responds with a phrase. When an outing event occurs, the display device 100 displays an animation in which character 131 disappears. (Step 4) Character 131 is displayed on the home screen of user terminal 1830, and the outing begins. In this way, the user only needs to log in, press a button, and then speak to character 131 on display device 100, making the operation easy for the user.

[0409] (C) Using the positioning function of user terminal 1830 (for example, GPS function), character 131 follows the user when they leave home. (Step 1) The display device 100 and user information (e.g., ID / password) are linked on the server in advance. (Step 2) In the settings of application 1836, register the home location information in advance using the positioning function of the user terminal 1830. At this time, it is best to keep the home location information locally rather than sending it to the server. (Step 3) The control unit 1831 launches the application 1836 and accepts the user's login operation. Once login is permitted, it displays a screen including an "Go Out" button. The user taps this button. The control unit 1831 sends a "Ready to Go Out" notification to the display device 100 via the server. (Step 4) The user leaves their home. If they move more than a predetermined distance (for example, 100m), the display device 100 will show an animation in which character 131 disappears. (Step 5) Character 131 appears on the home screen of user terminal 1830, and the outing begins. This approach makes it easy for users, as all they have to do is log in, press a button, and leave their home.

[0410] (4-2-3-7-2. Outing event in progress (outside the home)) During the outing event, if a user asks a question to the user terminal 1830, the character displayed on the user terminal 1830 may respond. Hereinafter, the character corresponding to character 131 displayed on the user terminal 1830 will be referred to as "character 131A" to distinguish it from character 131 when displayed on the display device 100 (see Figures 64 and 65). Character 131A wears different clothes from character 131, meaning it has changed clothes for going out, but it may also wear the same skirt outfit as character 131. In this case, character 150 may respond to normal questions, and character 131A may respond if the question is asked using a predetermined registered phrase. The predetermined registered phrase may be a phrase related to going out, such as "Are you having fun going out?".

[0411] For example, the following may be used for dialogue processing during such outing events. The control unit 1831 in the user terminal 1830 should acquire surrounding information such as the current location, time of day, and weather, and output a phrase appropriate to the situation in the voice of character 131A. For example, if the time of day is "11:00 to 14:00" and the current location is a "restaurant", a phrase such as "I'm hungry! What should we eat today?" may be used. For example, if the time of day is "8:00 to 15:00", the weather is "sunny", and the current location is a "park", a phrase such as "It's a nice day! Shall we play catch?" may be used. If the user's place of residence (current location set on the display device 100) is "outside Shizuoka Prefecture" and the current location is "within Shizuoka Prefecture", a phrase such as "Shizuoka is a nice place, isn't it? Have you seen Mt. Fuji yet?" may be used.

[0412] Furthermore, as a function for when the user is out and about, the user terminal 1830 may perform functions such as taking photos using AR and physical interaction functions using touch operations on the UI unit 1834.

[0413] Figure 64 shows an example of the main screen of the outing function. On the main screen, the control unit 1831 switches the AR function ON / OFF setting in response to the operation of the button labeled "AR ON / OFF". Figure 64(a) shows the main screen when the AR function is OFF, and Figure 64(b) shows the main screen when the AR function is ON. The button labeled "Take Photo" is for taking a photo. The button labeled "Location" is for displaying location information, such as displaying a list of spots around the current location. The button labeled "Settings" is for making various settings. For example, if the current location (in this case, "○○ Hall") is displayed on the main screen shown in Figure 64, and this is different from the actual current location, it may be possible to change it from this setting.

[0414] (Photo shooting function) When the control unit 1831 detects a touch operation on the button labeled "Take Photo" as shown in Figure 64(a) or (b), it executes the photo-taking function and displays the screen shown in Figure 65(a). Figure 65(a) is a screen in which an object of character 131A, Ray, is placed on a plane detected by the AR Core function, for example. The image taken by the shooting unit 1837 is displayed in the background of character 13. At this time, the control unit 1831 may disable the shooting button if the user attempts to take a photo from a low angle, or it may perform an effect such as having character 131A say a scolding phrase.

[0415] Figure 65(a) also shows multiple buttons labeled "Pose 1" to "Pose 5". When the control unit 1831 detects a touch operation on any of the buttons, it changes the pose (e.g., posture) of the character 131A. In this way, multiple poses can be selected, and it would be good to make it so that more poses become available when certain conditions are met. For example, the user could purchase them. When the control unit 1831 detects a touch operation on the button labeled with a camera mark in Figure 65(a), it takes a picture with the shooting unit 1837, generates a captured image, and stores it in the storage unit 1835. The control unit 1831 may also display the screen shown in Figure 65(b). Figure 65(b) shows buttons SN1 to SN3 for posting the captured image to a predetermined SNS. Buttons SN1 to SN3 each correspond to a different SNS and are buttons for launching the SNS application or displaying a web page for posting information to the SNS. When the control unit 1831 detects a touch operation on any of the buttons SN1 to SN3, it is preferable to open a screen for creating SNS posting information as shown in Figure 65(c), and attach the captured image to it. Location information may also be inserted. By operating this screen, the user enters a message, tags, etc., to create the posting information. When the control unit 1831 detects a touch operation on the button labeled "Post," it is preferable to perform the process of posting (sending) this posting information to SNS. In this way, by having the user post to SNS, the existence of the display device 100 and application 1836 can be made known to more people. Furthermore, the user who posts may also receive some kind of benefit, for example, an increase in intimacy level, the unlocking of limited phrases, or some other reward as a thank you for the promotion.

[0416] (Physical contact function) The skinship function allows the user to interact with the character 131A displayed on the UI unit 1834 by touching the screen. Similar to the function of the display device 100 described above, it is desirable that the character 131A's reaction changes depending on the part of the body touched. In addition, if a part other than the user's body is touched, a generic phrase (for example, the content of which may change depending on the current location, time of day, and weather) may be output by the character 131A's voice.

[0417] Furthermore, the control unit 1831 may rotate the character 131A around an axis corresponding to its direction (for example, the Y-axis, where the Y-axis is the axis extending vertically from the display surface of the UI unit 1834) in response to the user's swipe operation.

[0418] Furthermore, the control unit 1831 may generate an event that causes the character 131A to fall over (hereinafter referred to as a "falling event") in accordance with the posture detected by the posture detection sensor (for example, an acceleration sensor (G sensor) or a gyro sensor) of the display device 100. Examples of the falling event include (C) and (D) below. (C) A fall event is detected only when character 131A is displayed on the screen of user terminal 1830. In this case, a fall event does not occur when user terminal 1830 is locked. For example, a fall event occurs when the value measured by the G sensor exceeds a threshold or when the amount of change in posture is large. The threshold and amount of change that trigger a fall event may be changed from the settings screen.

[0419] (D) Always detect fall events. In this case, the control unit 1831 keeps the application 1836 running in the background and constantly detects fall events. For example, a fall event occurs when the value measured by the G sensor exceeds a threshold or when the amount of change in posture is large. The threshold and amount of change that trigger a fall event may be changed from the settings screen. If fall events occur multiple times in a short period of time, the next time the application 1836 is launched, a scene may be triggered in which character 131A scolds the user, for example, by saying a phrase such as, "Handle it a little more carefully!" Although this was described as a fall-over event, it could also be described as an event where character 131A gets angry, etc.

[0420] (4-2-3-7-3. Return-home event) When the user returns home, the control unit 1831 of the user terminal 1830 should terminate the display of character 131A in the UI unit 1834. The detection of the user's return home should be determined by the main board 1002 based on one or more of the following: location information acquired from the user terminal 1830, an increase in illuminance measured by the illuminance sensor 1632, or the detection of a person nearby by the person detection sensor. Once the user's return home is detected, the display device 100 should resume displaying character 131. When the user returns home in this way, the main board 1002 in the display device 100 should output special phrases, such as "It was fun going out~" or "Let's go out again~", which are different from the usual phrases, using the voice of character 131.

[0421] As a condition for the return-home event to occur, a method corresponding to any of the above methods (A) to (C) may be adopted. In this case, the above explanation of (A) to (C) should be read as follows: "Go out button" should be read as "End out button," and the "Ready to go out" notification should be read as "End out" notification, etc. Also, when there is no processing related to the invitation phrase and the outing event ends, an animation of character 131A disappearing should be displayed on the user terminal 1830, character 131 should be displayed (appear) on the standby screen of the display device 100, and the outing should end.

[0422] Furthermore, during the return-to-home event, the display device 100 or user terminal 1830 may determine the satisfaction level of character 131 at the end of the outing. For example, the satisfaction level of character 131 at the end of the outing can be determined from parameters such as the length of the outing, the amount of physical contact during the outing, and the number of steps taken. The satisfaction level can be, for example, "fun," "boring," or "unhappy." If it is "fun," the intimacy level may increase, or the state of character 131 upon returning to the display device 100 may be set to "normal." If it is "boring," the intimacy level may decrease, or the state of character 131 upon returning to the display device 100 may be set to "unhappy." For example, if the outing time and number of steps are above a threshold, and physical contact or commemorative photos are taken, it should result in "fun." Otherwise, it should result in "boring." If a fall event occurs (for example, if a fall event occurs 10 or more times within 5 minutes and is repeated), if the user detects an action similar to mischief using the physical contact function, or if the user takes an action that makes character 131A uncomfortable, such as peeking from a low angle using the commemorative photo function, character 131's state should be changed to "unhappy". If characte...

Claims

1. The control unit has a function to change the relationship between the user and a predetermined character in response to input from the user and to output character information about the character according to the relationship, The control unit has a function to output the character information for creating a scene where the character is sleeping. The aforementioned characters include a first character and a second character. The control unit, upon receiving a predetermined input from the user, outputs character information relating to the first character in accordance with the predetermined input, while also having a function to output character information relating to the second character in accordance with the predetermined input when the first character is performing a sleeping animation.

2. The control unit has a function to change the relationship between the user and a predetermined character in response to input from the user and to output character information about the character according to the relationship, The aforementioned characters include a first character and a second character. The control unit is a system that includes a function to output character information for the first character for outputting voice based on pre-recorded voice data, and a function to output character information for the second character for outputting voice after performing voice synthesis processing.

3. The control unit has a function to change the relationship between the user and a predetermined character in response to input from the user and to output character information about the character according to the relationship, The aforementioned characters include a first character and a second character. When the control unit receives input from the user requesting the first character to perform a predetermined function, When the request is for the execution of the first function, the first function is executed by outputting the voice of the first character. When the request is for the execution of a second function, the system is equipped with a function to execute the second function by outputting the voice of the second character. The control unit, upon receiving an input requesting the execution of the second function, outputs character information relating to the first character for performing a visual effect that indicates the execution of the second function using the second character, and then the system has a function to execute the second function by outputting the voice of the second character.

4. The control unit includes a function to output character information relating to the second character, so that the performance using the second character is initiated when the audio output for the performance using the first character has finished. The system according to claim 3.

5. The control unit has a function to change the relationship between the user and a predetermined character in response to input from the user and to output character information about the character according to the relationship, The control unit is a system that includes a function to output character information for the character to explain the malfunction when it detects any of the following signs of malfunction: the temperature measured by the temperature sensor is above a threshold, the clock speed is reduced, or the memory usage is high and the operation is slow.

6. A program for a computer to implement the functions of the control unit of the system described in any one of claims 1 to 5.

Images