System and the like

Abstract

To provide a technique which provides information to a user.SOLUTION: A system (display device 1) comprises control means which includes an acquisition function of acquiring information on a plurality of times as externally provided external information, a summary generation function of generating a summary on the basis of the information on the plurality of times acquired by the acquisition function, and an output function of outputting at least the summary generated by the summary generation function.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a system. [Background technology]

[0002] Patent Document 1 describes a stereoscopic image display device that does not require special glasses. This stereoscopic image display device includes a light source, a liquid crystal display panel, and a parallax barrier disposed on the viewer's side of the liquid crystal display panel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-295113 A Summary of the Invention [Problem to be solved by the invention]

[0004] One object of the present invention is to provide a technique for providing information to a user.

[0005] The purpose of the present invention is not limited to this, and the applicant intends to obtain rights for configurations that aim to obtain effects from parts of the configurations disclosed in this specification and drawings, etc., by filing a divisional application, amendment, etc. For example, this specification discloses a problem in which the part described as "can be" in this specification is read as "the problem is to be". The problems are described as being independent, and the applicant intends to obtain rights for the configurations for solving each problem separately by filing a divisional application, amendment, etc. Even if the problems are implicitly understood from the description in the specification, the applicant intends to include part of the configurations described in this specification in the scope of the patent claim by amendment or divisional application. In addition, the applicant has disclosed a configuration that solves a problem that combines these independent problems, and intends to obtain rights for it. [Means for solving the problem]

[0006] (1) It is preferable to provide a control means having an acquisition function for acquiring information at multiple times as external information provided from the outside, a summary creation function for creating a summary based on the information at multiple times acquired by the acquisition function, and an output function for at least outputting the summary created by the summary creation function. The output is, for example, an audio output, but it may also be output by display, communication, printing, or other methods. The same applies to the case where "output" is described below.

[0007] In this way, it is possible to output a summary created based on information at multiple times acquired from outside.

[0008] (2) The control means may have a function of generating additional information in addition to the summary based on information at multiple times acquired by the acquisition function.

[0009] In this way, additional information can be created in addition to the summary.

[0010] (3) The control means may be capable of creating the summary as a hearsay sentence, and may have a function of outputting the hearsay sentence in association with a character. The output associated with a character may be, for example, an audio output by the character's speech, but it may also be an output by displaying the summary and additional information as text in a speech bubble displayed around the character, a display or audio output on another device through communication with the other device, or a print output on paper or other medium. In the following, the same applies to "output associated with a character", "output associated with a character", "outputting the summary associated with a character", "outputting the summary associated with a character", "outputting the weather information summary associated with a character", and "outputting the news information summary associated with a character".

[0011] In this way, a summary created in a hearsay format can be output in association with a character.

[0012] (4) The control means preferably has a function of creating the summary and the additional information in different contexts, and a function of outputting both the summary and the additional information in association with characters.

[0013] In this way, the summary and additional information can be distinguished and output in association with the characters.

[0014] (5) The control means may have a function of outputting the summary in association with a character in response to a request from a user.

[0015] In this way, a summary can be output in association with the character in response to a request from the user, which makes it possible to give the user a sense of closeness to the character.

[0016] (6) The control means may have a function of restricting output of the summary associated with a character even if requested by the user.

[0017] In this way, even if there is a request from the user, the output associated with the character may be restricted. In this way, if the output associated with the character is not restricted, the user can feel a sense of closeness to the character.

[0018] (7) The control means may have a function of outputting the output information created by the summary creation function in association with a character even if there is no request from the user.

[0019] In this way, the output information created by the summary creation function may be output in association with the character. For example, the character may output the output information created by the summary creation function without a request from the user. By doing so, it is possible to give the impression that the character actively output the information of its own volition, and it is possible to give the user a sense of familiarity with the character by incorporating the character into daily life.

[0020] (8) The control means may be capable of acquiring discrete weather information at a plurality of times as the external information acquired by the acquisition function.

[0021] In this way, discrete weather information for multiple times can be provided to the user.

[0022] (9) The discrete weather information at multiple times acquired by the acquisition function may have a priority order, and the control means may be capable of creating a summary of the weather information by the summary creation function based on the priority order.

[0023] In this way, it is possible to provide the user with weather information with a high priority among discrete weather information at multiple times.

[0024] (10) The control means may be capable of creating a summary of the weather information using the summary creation function based on the weather information that is acquired most frequently among the discrete weather information for the multiple times acquired by the acquisition function.

[0025] In this way, it is possible to provide the user with the weather information that has been acquired the most frequently, that is, the weather information determined by majority vote, among the discrete weather information for multiple times.

[0026] (11) The control means may acquire weather information for a specific time and a specific area as the weather information acquired by the acquisition function.

[0027] In this way, weather information for a particular time and a particular area can be provided to a user.

[0028] (12) The control means may have a function of outputting, in response to a user request, a summary of weather information created by the summary creation function based on weather information for the specific time and the specific area, in association with the character.

[0029] In this way, weather information for a time and area desired by the user can be provided in association with the character.

[0030] (13) The control means may have a memory function for storing at least words relating to time, words relating to location, and keywords relating to weather, and a function for outputting a summary of the weather information in association with a character when a request is made by a user and the request includes a word stored in the memory function.

[0031] In this way, if a user request contains a word stored in the memory function, a summary of the weather information is output in association with the character, making it possible to respond to the user request while communicating between the user and the character.

[0032] (14) The weather information includes at least one of information on weather conditions, information on temperature, and information on precipitation.

[0033] In this way, it is possible to provide the user with useful information for determining what to wear that day, whether an umbrella is necessary, and so on.

[0034] (15) The control means may be configured to be able to acquire daily weather information as the external information acquired by the acquisition function, in addition to the discrete weather information at the multiple times.

[0035] In this way, in addition to discrete weather information at multiple times, weather information for each day can also be obtained, improving user convenience.

[0036] (16) The control means may have a function of increasing the intimacy level of the character by providing food to the character.

[0037] This allows the user to feel closer to the character.

[0038] (17) The control means may have a function of causing the character to ignore a request from the user if the degree of intimacy is less than a predetermined specified value, even if such a request is made by the user.

[0039] In this way, the user can be encouraged to feed the character, and when an output associated with the character is produced, the user can be given a sense of closeness to the character.

[0040] (18) The control means may have a function of causing the character to actively output the summary by voice when the degree of intimacy is greater than a predetermined value.

[0041] In this way, when the user feeds the character, the intimacy level increases and the character actively outputs voice. In this way, the character can be integrated into the user's daily life, giving the user a sense of closeness to the character while providing the user with useful information.

[0042] (19) The control means may be capable of acquiring news information as the external information acquired by the acquisition function.

[0043] In this way, news information can be provided to the user, improving user convenience.

[0044] (20) The control means may have a memory function for storing at least words relating to time periods and keywords relating to news information, and a function for outputting a summary of the news information in association with a character when a request is made by a user and the request includes a word stored in the memory function.

[0045] In this way, if a user request contains a word stored in the memory function, a summary of the news information is output in association with the character, making it possible to respond to the user's request while promoting communication between the user and the character.

[0046] (A1) It is preferable to provide a display device having a housing having a light-shielded image display space therein and an image observation area on a first direction side of the image display space, and an image display unit that focuses an aerial image in the image display space.

[0047] In this way, the floating image combined with the image display space is displayed in a relatively dark space as seen from the observer's side, which enhances the three-dimensional effect of the floating image given to the observer, thereby improving the visual effect of the display device.

[0048] (A2) It is preferable to provide a display device having a light-reducing member that reduces external light that enters the image display space from the observation area.

[0049] In this way, the light-reducing member reduces the amount of external light entering the image display space, so that the image display space can be kept dark, thereby enhancing the three-dimensional effect of the aerial image that can be presented to the observer.

[0050] (A3) The image display unit may include a first display unit that emits a first image light, and an optical system that forms the aerial image based on the first image light, and may be a display device having a shading member between the shading member and the optical system on the optical path of reflected light of the first image light that is reflected by the shading member.

[0051] In this way, it is possible to prevent the image displayed in the image display space due to the first image light being reflected by the light reducing member from being observed by the viewer.

[0052] (A4) The display device may be such that the light blocking member is provided at a position that does not overlap the observation area when viewed from the first direction.

[0053] In this way, even if an observer looks into the image display space from the observation area, the image displayed in the image display space can be prevented from being observed by the observer due to the first image light being reflected by the dimming member.

[0054] (A5) The image display unit may include a first display unit that emits a first image light, and an optical system that forms the aerial image based on the first image light, and the optical system may be a display device that is positioned to avoid an area on the optical path of light of the first image light that is reflected by the dimming member.

[0055] In this way, it is possible to prevent the image displayed in the image display space due to the first image light being reflected by the light reducing member from being observed by the viewer.

[0056] (A6) The optical system may include an optical element having a first surface onto which the first image light is incident and a second surface opposite to the first surface, and the optical element may be a display device that reflects the first image light toward the first direction and transmits light incident on the second surface toward the first surface.

[0057] In this way, the second surface of the optical element can be superimposed on the aerial image combined using the optical element, allowing the observer to observe the second surface of the optical element, thereby further enhancing the three-dimensional effect of the aerial image that is imparted to the observer.

[0058] (A7) The display device may have a second display unit that emits second image light that overlaps with the aerial image when viewed from the first direction.

[0059] This increases the freedom to select the image that the viewer can view by overlaying it on the aerial image.

[0060] (A8) It is preferable that the display device has a light emission luminance of the first display section that is higher than a light emission luminance of the second display section.

[0061] In this way, the difference in emission luminance between the first display section and the second display section can enhance the three-dimensional effect of the aerial image that can be imparted to the observer.

[0062] (A9) The first display unit may be a display device that is positioned above the observation area with its display surface facing downward and is inclined so that a second direction side opposite the first direction is lower than the first direction side.

[0063] In this way, an aerial image can be formed using the optical member by the first display portion disposed above the observation area.

[0064] (A10) The display surface of the first display unit may be a display device that is inclined at an angle of approximately 20 degrees with respect to the horizontal direction.

[0065] In this way, the three-dimensional effect of the aerial image that can be given to the observer can be enhanced, and the image displayed in the image display space can be made more difficult to observe due to the first image light being reflected by the dimming member.

[0066] (A11) The second display unit may be a display device that is disposed below the observation area with its display surface facing upward.

[0067] In this way, the image displayed on the second display screen appears to be located below the aerial image, thereby further enhancing the three-dimensional effect of the aerial image that can be imparted to the viewer.

[0068] (A12) The first display unit may be a display device that emits the first image light so that the aerial image can be observed on the second display surface.

[0069] In this way, the floating image is observed as being positioned above the image displayed on the second display screen, so that the three-dimensional effect of the floating image that can be imparted to the viewer can be further enhanced.

[0070] (A13) The housing may be a display device having a first housing space above the first display unit, and a processing board disposed on the second direction side of the first housing space.

[0071] In this way, the processing substrate can be placed in an area of ​​the first housing space that is secured above the first display unit and that has a relatively large margin in the height direction.

[0072] (A14) The processing board may have a mounting section to which an external device can be detachably attached, and the mounting section may be a display device exposed to the outside of the housing from the second direction side of the housing.

[0073] In this way, the portion of the mounting section to which the external device can be detachably attached that is exposed to the outside of the housing can be made less visible to the observer.

[0074] (A15) It is preferable that the display device has a sensor provided below the observation area, and the processing substrate executes processing according to information measured by the sensor.

[0075] This prevents the sensor configuration from interfering with the observation of the aerial images.

[0076] (A16) The display device may include a sensor substrate on which the sensor is mounted, the sensor substrate being provided in a portion of the housing on the first direction side.

[0077] In this way, it is possible to prevent the sensor configuration from interfering with the observation of the aerial images.

[0078] (A17) The sensor may be a display device including a sensor that detects a gesture.

[0079] In this way, when the observer makes a gesture, it is possible to prevent parts of the observer's body from interfering with the observation of the aerial image.

[0080] (A18) The housing may have a second housing space below the image display space, and the display device may have a control board provided in the second housing space for supplying a signal from the sensor to the processing board.

[0081] This makes it possible to prevent the dimensions of the housing above the observation area from becoming enlarged, compared to when the control board is accommodated in the first accommodation space above the first display unit.

[0082] (A19) The display device may preferably have a wiring section that electrically connects the control board and the processing board, and a partition member that separates the wiring section so that it cannot be observed from the image display space side.

[0083] In this way, the wiring portion electrically connecting the control board and the processing board can be prevented from being seen.

[0084] (A20) The processing substrate may be a display device that controls display of the first display unit in response to information measured by the sensor.

[0085] In this way, the aerial image can be changed according to the information measured by the sensor.

[0086] (A21) The display device may preferably have a viewing angle control member that is provided on a display surface of the first display portion and that narrows the viewing angle in the first direction.

[0087] This makes it possible to prevent the image displayed on the first display surface from being directly viewed by the viewer.

[0088] (A22) The display device may be such that the first image light is image light of an image that is recognized as a two-dimensional image.

[0089] In this way, even when using the first display unit that displays an image that is recognized as a two-dimensional image, it is possible to give the observer a three-dimensional effect of the aerial image.

[0090] (A23) The housing may have a configuration in which a plurality of parts are joined by fasteners, and the fasteners may be a display device that is not exposed at least from the first direction side.

[0091] This can prevent the fixing member from being seen by a viewer, thereby improving the aesthetic appearance of the display device.

[0092] (A24) It is preferable that at least a part of the portion facing the image display space is treated to reduce light reflection.

[0093] This makes it easier to maintain the image display space in a dark state, thereby enhancing the three-dimensional effect of the aerial image that can be given to the observer.

[0094] (A25) The housing may be a display device having a portion that surrounds the observation area when viewed from the first direction side.

[0095] In this way, the floating image displayed in the image display space, combined with the housing positioned in front of it, can further enhance the three-dimensional effect of the floating image that can be imparted to the viewer.

[0096] (A26) The image display unit may be a display device that displays an image of a character representing a cat.

[0097] In this way, a three-dimensional image of a cat character can be displayed.

[0098] (A27) The image display unit may be a display device that changes the image based on a communication function that reproduces communication between the character and an observer who observes the character.

[0099] In this way, the image of the character representing the cat can be changed based on the communication function.

[0100] (A28) It is preferable to further include an odor sensor that detects an odor, and an operation control means that executes at least one of a display output operation of displaying on the video display unit an image of a character performing a movement corresponding to the detection result by the odor sensor, and an audio output operation of outputting audio corresponding to the detection result by the odor sensor as audio uttered by the character displayed on the video display unit.

[0101] In this way, communication regarding odors can be reproduced between the character displayed on the video display unit and the user. For example, when an odor is detected by the odor sensor, the character can be controlled to output a voice such as "something smells" or to display an image of the character sniffing an odor. This allows for more diverse communication between the character and the user.

[0102] (A29) The display output operation may be an operation of displaying on the video display unit an image of a character displayed on the video display unit performing an action corresponding to the type of odor detected, and the audio output operation may be an operation of outputting an audio corresponding to the type of odor detected as an audio uttered by the character displayed on the video display unit.

[0103] In this way, communication regarding the type of odor can be reproduced between the character displayed on the video display unit and the user. For example, when the odor of tobacco is detected, a display output operation of displaying an image of the character making a smoky motion and a voice output operation of uttering words expressing concern for the user's health can be executed. Also, when the odor of a fragrance such as aromatic oil is detected, a voice output operation of uttering a question regarding the fragrance can be executed. By performing such control, the range of communication between the character displayed on the video display unit and the user can be expanded, and the user's attachment to the character can be increased.

[0104] (A30) The device may further include a voice detection means capable of detecting voice, and a detection means capable of detecting the presence of a person in a detection target area, and when the voice detection means detects the presence of a person in the detection target area, it is preferable that the voice detection means activates voice recognition for interacting with a character displayed on the video display unit.

[0105] In this way, when the user interacts with a character displayed on the video display unit, the user does not need to utter a trigger word to activate voice recognition, for example, and can communicate more naturally.

[0106] (A31) It is preferable to further include a wind detection means for detecting wind, and an operation control means for executing at least one of a display output operation for displaying on the video display unit an image of a character moving in accordance with the detection result by the wind detection means, and an audio output operation for outputting audio in accordance with the detection result by the wind detection sensor as audio uttered by the character displayed on the video display unit.

[0107] In this way, it is possible to reproduce communication between the character displayed on the video display unit and the user through wind applied to the display device. The wind may be generated artificially by the user, for example, by moving a body part such as waving a hand toward the display device, or by blowing on the display device.

[0108] (A32) The operation control means may perform the display output operation or the audio output operation in response to a breath blown by the user, based on a detection result by the wind detection means.

[0109] In this way, communication can be reproduced between the character displayed on the video display unit and the user in response to breathing onto the display device.

[0110] The inventions shown in (1) to (32) above and the inventions shown in (A1) to (A32) above can be arbitrarily combined. For example, it is preferable to add at least a part of the configuration of at least one of the inventions from (2) onwards to all or a part of the configuration of the invention shown in (1). In particular, it is preferable to add at least a part of the configuration of at least one of the inventions from (2) onwards to the invention shown in (1). In addition, it is also possible to extract any configuration from the inventions shown in (1) to (32) and combine the extracted configurations. The applicant of this application intends to obtain rights to the inventions including these configurations. In addition, even if there is a description such as "in the case of" or "when", it is not intended to describe the configuration as being limited to that case or time. These are examples of better configurations, and the applicant intends to obtain rights to configurations other than these cases or times. In addition, the parts described in order are not limited to this order. Configurations in which some parts have been deleted or the order has been changed are also disclosed, and the applicant intends to obtain rights. Effect of the Invention

[0111] According to the present invention, a technique for providing information to a user can be provided.

[0112] The effects of the invention of this application are not limited to this, and effects obtained from the configuration disclosed in the specification and drawings, etc. are also disclosed, and the intention is to obtain rights to the configuration that achieves the effects by divisional applications, amendments, etc. For example, in this specification, the phrase "can" is a description that clearly indicates the effect that is achieved, and there are also parts that show the effect even without the phrase "can." Also, there are effects that can be understood from the configuration even without such a description. [Brief description of the drawings]

[0113] [Figure 1A] FIG. 1 is a diagram illustrating an overview of a display device according to an embodiment. [Figure 1B] FIG. 1 is a diagram illustrating an overview of a display device according to an embodiment. [Figure 1C] FIG. 2 is a diagram illustrating a character according to an embodiment. [Diagram 2] 1 is a diagram showing an external configuration of a display device according to an embodiment; [Diagram 3] 1 is a diagram showing an external configuration of a display device according to an embodiment; [Figure 4] 1 is a diagram showing an external configuration of a display device according to an embodiment; [Diagram 5] 1 is a front view of a display device according to an embodiment; [Figure 6] FIG. 2 is a right side view of the display device according to the embodiment. [Figure 7] FIG. 2 is a left side view of the display device according to the embodiment. [Figure 8] FIG. 1 is a plan view (top view) of a display device according to an embodiment. [Figure 9] FIG. 2 is a rear view of the display device according to the embodiment. [Figure 10] FIG. 2 is a bottom view of the display device according to the embodiment. [Figure 11] 1 is a rear view of a display device according to an embodiment with a rear cover removed; [Figure 12] FIG. 2 is a diagram illustrating an example of how a display device according to an embodiment is used. [Figure 13]FIG. 1 is a diagram illustrating a principle regarding display of a display device according to an embodiment. [Figure 14] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 15] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 16] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 17] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 18] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 19] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 20] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 21] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 22] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 23] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 24] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Diagram 25] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 26] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 27] FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. [Figure 28] 1A and 1B are diagrams illustrating a cause for a virtual image being displayed on a display device according to an embodiment. [Figure 29] 1A and 1B are diagrams illustrating a cause of occurrence of reflection in a display device according to an embodiment. [Diagram 30] 1A to 1C are diagrams illustrating the principle of displaying a virtual image and suppressing reflection in a display device according to an embodiment. [Diagram 31]1A and 1B are diagrams for explaining a cause of a virtual image being displayed on a display device. [Diagram 32] 1 is a front view of a display device according to an embodiment; [Diagram 33] FIG. 2 is a control block diagram of a display device according to an embodiment. [Diagram 34] FIG. 1A is a flowchart for reproducing communication related to odors, and FIG. 1B is a diagram showing a voice management table related to odors. [Diagram 35] 13 is a flowchart for reproducing communication regarding frequency of stay. [Diagram 36] FIG. 13A is a flowchart for reproducing communication related to a place of stay, and FIG. 13B is a diagram showing a voice management table related to a place of stay. [Figure 37] FIG. 13A is a flowchart for reproducing communication related to the number of visits, and FIG. 13B is a diagram showing a voice management table related to the number of visits. [Figure 38] FIG. 13A is a flowchart for reproducing communication related to stay time, and FIG. 13B is a diagram showing a voice management table related to stay time. [Figure 39] FIG. 1A is a flowchart for reproducing communication related to behavioral patterns, and FIG. 1B is a diagram showing a voice management table related to behavioral patterns. [Diagram 40] 13 is a flowchart for reproducing an operation relating to detection of a suspicious person. [Diagram 41] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Diagram 42] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Diagram 43] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Diagram 44] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Diagram 45] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 46] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 47] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 48] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 49] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 50] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 51] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 52] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 53] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 54] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 55] FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. [Figure 56] FIG. 1 is a diagram for simply explaining an example of a development concept of a system according to an embodiment. [Figure 57] FIG. 11 is a diagram showing an example of a display form in a video display space of the system according to an embodiment. [Figure 58A] 13 is a table showing an example of a formula for calculating the probability that a character will move to areas 1 and 2 depending on the number of times a user has fed the character in a system according to one embodiment. [Figure 58B] 13 is a table showing an example of a formula for calculating the probability that a character will move to areas 3 to 5 depending on the number of times a user has fed the character in a system according to one embodiment. [Figure 59] 13 is a table showing an equation for calculating the probability that a character will move to each area depending on the number of times a user has fed the character in a system according to one embodiment. [Figure 60] (A) is a table showing an example of values ​​to be substituted for coefficients X (X1, X2, X3) included in the formula for calculating coefficients a, b, c in a system according to one embodiment, and used to classify the degree of intimacy into low, medium, and high. (B) is an example of a table in which the upper and lower limits of the probability of moving to each area at each degree of intimacy are set in a system according to one embodiment. [Figure 61] In a system according to one embodiment, the tables show coefficients included in a basic formula showing the probability of moving to area 1 depending on intimacy, where (A) is an example of a table showing coefficient a, (B) is an example of a table showing coefficient b, and (C) is an example of a table showing coefficient c. [Figure 62] In a system according to one embodiment, the tables show coefficients included in a basic formula showing the probability of moving from area 2 to area 4 depending on the degree of intimacy, where (A) is an example of a table showing coefficient A, and (B) is an example of a table showing coefficient B. [Figure 63] 13 is a graph showing the relationship between the number of days a user has fed a character and the probability of the character moving to each area in a system according to one embodiment. [Figure 64] 64 is an example of a table showing the data of the graph shown in FIG. 63 by degree of intimacy in a system according to one embodiment. [Figure 65A] FIG. 13 is an example of a diagram for explaining a character's state, behavior pattern, etc. when the intimacy level is low in a system according to an embodiment. [Figure 65B] 13 is an example of an image displayed in the image display space of the system according to an embodiment when the intimacy level is low. [Figure 66A] FIG. 13 is an example of an explanatory diagram for moving to another area when the intimacy level is low in the system according to an embodiment. [Figure 66B] 13 is an example image for explaining movement to another area when the intimacy level is low in a system according to an embodiment. [Figure 67A]FIG. 13 is an example of an explanatory diagram for moving to another area when a feeding event is detected and the intimacy level is low in a system according to an embodiment. [Figure 67B] 13 is an example image showing movement from area 5 to area 1 when a feeding event is detected when intimacy level is low in a system according to one embodiment. [Figure 67C] 13 is an example image showing movement from area 1 or 2 to area 5 when a feeding event is detected when intimacy level is low in a system according to one embodiment. [Figure 68A] FIG. 13 is an example of an explanatory diagram for explaining movement to another area when a scratching event is detected when the intimacy level is low in a system according to an embodiment. [Figure 68B] 13 is an example image showing movement from area 5 to area 3 when a scratching event is detected when intimacy level is low in a system according to one embodiment. [Figure 68C] 13 is an example image showing movement from area 5 to area 3 when a scratching event is detected when intimacy level is low in a system according to one embodiment. [Figure 69A] FIG. 13 is an example of an explanatory diagram of movement to another area when a human detection event is detected when the intimacy level is low in a system according to an embodiment. [Figure 69B] 13 is an example image for explaining movement from area 1 or area 3 to area 5 when a human detection event is detected when the intimacy level is low in a system 1 according to an embodiment. [Figure 70A] FIG. 13 is an example of a diagram for explaining a character's communication function when the intimacy level is low in a system according to an embodiment. [Figure 70B] FIG. 13 is an example of a diagram for explaining a character's communication function when the intimacy level is low in a system according to an embodiment. [Figure 71] FIG. 13 is a diagram showing an example of a motion reproduced when a character runs away when an event is detected in a system according to an embodiment. [Figure 72] FIG. 13 is a diagram showing an example of a playback motion when a character turns and escapes while moving through an area in a system according to an embodiment. [Figure 73] FIG. 13 is a diagram showing an example of a playback motion when a character turns and escapes while waiting in a system according to an embodiment. [Figure 74] FIG. 13 is a diagram showing an example of a motion reproduced when an event is detected while a character is eating food in a system according to an embodiment. [Figure 75] FIG. 13 is a diagram showing an example of a motion reproduced when an event is detected while a character is scratching its claws in a system according to an embodiment. [Figure 76] FIG. 13 is a diagram showing an example of a motion reproduced when JUNO runs away after an event is detected while sleeping in the system according to an embodiment. [Figure 77A] FIG. 13 is an example of a diagram for explaining a character's state, behavior pattern, etc. when the intimacy level is medium in a system according to an embodiment. [Fig. 77B] 13 is an example of an image displayed in the image display space of the system according to an embodiment when the intimacy level is medium. [Fig. 78A] FIG. 13 is an example of an explanatory diagram for moving to another area when the intimacy level is at a medium level in the system according to an embodiment. [Fig. 78B] 13 is an example image for explaining movement to another area when the intimacy level is at medium level in the system 1 according to an embodiment. [Figure 79A] FIG. 13 is an example of an explanatory diagram for moving to another area when a feeding event is detected while the intimacy level is at a medium level in a system according to an embodiment. [Fig. 79B] 13 is an example image showing movement from area 5 to area 1 when a feeding event is detected while intimacy is at an intermediate level in a system according to one embodiment. [Figure 80A]FIG. 13 is an example of an explanatory diagram for explaining movement to another area when a claw scratching event is detected during intimacy in a system according to an embodiment. [Figure 80B] 13 is an example image showing movement from area 5 to area 3 when a scratching event is detected while intimacy is at a medium level in a system according to one embodiment. [Fig. 80C] FIG. 13 is an example of an explanatory diagram of movement to another area when a human detection event is detected during intimacy in a system according to an embodiment. [Figure 81A] FIG. 13 is an example of a diagram for explaining a communication function of a character when the intimacy level is medium in a system according to an embodiment. [Fig. 81B] FIG. 13 is an example of a diagram for explaining a communication function of a character when the intimacy level is medium in a system according to an embodiment. [Figure 82] FIG. 13 is an example of a diagram for explaining other states and actions of a character when the intimacy level is at medium level in a system according to an embodiment. [Figure 83A] FIG. 13 is an example of a diagram for explaining a character's state, behavior pattern, etc. when the intimacy level is high in a system according to an embodiment. [Figure 83B] 13 is an example of an image displayed in the image display space of the system according to an embodiment when the intimacy level is high. [Fig. 84A] FIG. 13 is an example of an explanatory diagram for moving to another area when the intimacy level is high in the system according to an embodiment. [Fig. 84B] 13 is an example image for explaining movement to another area when the intimacy level is high in a system according to an embodiment. [Fig. 85A] FIG. 13 is an example of an explanatory diagram for moving to another area when a feeding event is detected and the intimacy level is high in a system according to an embodiment. [Fig. 85B]13 is an example image showing an example of a case where a feeding event is detected when the intimacy level is high in the system according to an embodiment. [Figure 86A] FIG. 13 is an example of an explanatory diagram for explaining movement to another area when a scratching event is detected when the intimacy level is high in a system according to an embodiment. [Figure 86B] 13 is an example image showing movement from area 1 to area 3 when a scratching event is detected when intimacy is high in a system according to one embodiment. [Fig. 87A] FIG. 13 is an example of an explanatory diagram of movement to another area when a human detection event is detected when the intimacy level is high in a system according to an embodiment. [Fig. 87B] 13 is an example image illustrating movement from an area other than area 1 to area 1 when a human detection event is detected when the intimacy level is high in a system according to an embodiment. [Fig. 88A] FIG. 13 is an example of a diagram for explaining call-and-response, which is an example of a character's communication function when the intimacy level is high, in a system according to an embodiment. [Fig. 88B] 13 is an example image for explaining area movement when a call is made when the intimacy level is high in a system according to an embodiment. [Figure 89] FIG. 13 is an example diagram for explaining a summary of character actions when the intimacy level is high in the system according to an embodiment. [Figure 90A] FIG. 13 is an example of a diagram for explaining functions specific to high intimacy levels in a system according to an embodiment. [Figure 90B] FIG. 13 is an example of a diagram for explaining a function specific to a high intimacy level in a system according to an embodiment, showing an example of a group of call words and a character response word for a timer function. [Figure 90C]FIG. 13 is an example of a diagram for explaining a function specific to a high intimacy level in a system according to an embodiment, showing an example of a group of call words and a character response word for an alarm function. [Figure 90D] FIG. 13 is an example of a diagram for explaining a function specific to a high intimacy level in a system according to an embodiment, showing an example of a group of call words and a character response word for a home appliance control function. [Figure 91] FIG. 11 is an example of a diagram for explaining a timer function and an alarm function in a system according to an embodiment. [Figure 92A] FIG. 13 is a diagram for explaining an example of a specification for a character to learn Japanese in a system according to an embodiment. [Fig. 92B] FIG. 13 is a diagram for explaining an example of a specification for a character to learn Japanese in a system according to an embodiment. [Figure 93A] FIG. 13 is an example of a diagram for explaining a response of a character when the intimacy level is low in the system according to an embodiment. [Figure 93B] FIG. 13 is an example of a diagram for explaining a character's response during intimacy in a system according to an embodiment. [Figure 93C] FIG. 13 is an example of a diagram for explaining JUNO's response when the intimacy level is high in the system according to an embodiment. [Figure 94] FIG. 11 is an example of a diagram for explaining recognition words in a system according to an embodiment. [Figure 95] FIG. 13 is an example of a diagram for explaining a group of dialogue-related words in the system according to an embodiment. [Figure 96A] FIG. 97 is a diagram showing an example of word group names and call word groups in the dialogue word group table shown in FIG. 95. [Figure 96B] FIG. 97 is a diagram showing an example of word group names and call word groups in the dialogue word group table shown in FIG. 95. [Figure 97A]FIG. 97 is a diagram showing an example of word group names and character response words in the first stage, second stage and third stage in the dialogue word group table shown in FIG. 95. [Figure 97B] FIG. 97 is a diagram showing an example of word group names and character response words in the first stage, second stage and third stage in the dialogue word group table shown in FIG. 95. [Figure 97C] FIG. 97 is a diagram showing an example of word group names and character response words in the fourth stage in the dialogue word group table shown in FIG. 95. [Fig. 97D] FIG. 97 is a diagram showing an example of word group names and character response words in the fourth stage in the dialogue word group table shown in FIG. 95. [Figure 97E] FIG. 97 is a diagram showing an example of word group names and character response words in the fourth stage in the dialogue word group table shown in FIG. 95. [Figure 98A] FIG. 13 is an example of a diagram for explaining a state in which JUNO is trying to learn Japanese in a system according to an embodiment. [Figure 98B] FIG. 13 is an example of a diagram for explaining a state in which JUNO is trying to learn Japanese in a system according to an embodiment. [Figure 98C] FIG. 13 is an example of a diagram for explaining a state in which JUNO is trying to learn Japanese in a system according to an embodiment. [Figure 99A] FIG. 11 is a diagram showing an example of a response flow in a system according to an embodiment. [Figure 99B] FIG. 11 is a diagram showing an example of a response flow in a system according to an embodiment. [Figure 100] FIG. 13 is an example of a diagram for explaining a case where a response is made in Japanese in a system according to an embodiment. [Figure 101A] FIG. 11 is an example of a diagram for explaining special call words in a system according to an embodiment. [Figure 101B]FIG. 11 is an example of a diagram for explaining special call words in a system according to an embodiment. [Figure 101C] FIG. 11 is an example of a diagram for explaining special call words in a system according to an embodiment. [Figure 101D] FIG. 11 is an example of a diagram for explaining special call words in a system according to an embodiment. [Figure 102A] FIG. 13 is a diagram for explaining an example of an overview of priority of response words in a system according to an embodiment. [Figure 102B] FIG. 13 is a diagram for explaining an example of an overview of priority of response words in a system according to an embodiment. [Figure 102C] FIG. 13 is a diagram for explaining an example of an overview of priority of response words in a system according to an embodiment. [Figure 103] FIG. 1 is an example of a diagram for explaining dialogue words in a system according to an embodiment. [Figure 104A] FIG. 13 is a diagram illustrating an example of priority of response words in a system according to an embodiment. [Figure 104B] FIG. 13 is a diagram illustrating an example of priority of response words in a system according to an embodiment. [Figure 105A] FIG. 13 is a diagram illustrating an example of priority of response words in a system according to an embodiment. [Figure 105B] FIG. 13 is a diagram illustrating an example of priority of response words in a system according to an embodiment. [Figure 105C] FIG. 13 is a diagram illustrating an example of priority of response words in a system according to an embodiment. [Fig. 106] 13 is an example of character-specific stored information in a system according to an embodiment. [Figure 107A] FIG. 13 is an example of a diagram for explaining feeding in a system according to an embodiment. [Figure 107B]FIG. 13 is an example of a diagram for explaining feeding in a system according to an embodiment. [Figure 107C] FIG. 13 is an example of a diagram for explaining feeding in a system according to an embodiment. [Figure 108A] FIG. 13 is an example diagram for explaining a behavior pattern of a character during a sleep transition in a system according to an embodiment. [Figure 108B] FIG. 13 is an example diagram for explaining a behavior pattern of a character during a sleep transition in a system according to an embodiment. [Figure 108C] FIG. 13 is an example diagram for explaining a behavior pattern of a character during a sleep transition in a system according to an embodiment. [Fig. 108D] FIG. 13 is an example diagram for explaining a behavior pattern of a character during a sleep transition in a system according to an embodiment. [Fig. 109] FIG. 13 is an example of a diagram for explaining a Japanese response during sleep in a system according to an embodiment. [Figure 110] FIG. 13 is an example of a diagram for explaining a behavior pattern of a character when waking up in a system according to an embodiment. [Fig. 111A] FIG. 13 is an example of a diagram illustrating a zoom display of a character in a system according to an embodiment. [Fig. 111B] FIG. 13 is an example of a diagram illustrating a zoom display of a character in a system according to an embodiment. [Figure 111C] FIG. 13 is an example of a diagram illustrating a zoom display of a character in a system according to an embodiment. [Fig. 112A] 11A to 11C are diagrams illustrating an example of a screen transition displayed on an Android (registered trademark) smartphone that can be linked with a system according to an embodiment. [Fig. 112B] 11A to 11C are diagrams illustrating an example of a screen transition displayed on an Android (registered trademark) smartphone that can be linked with a system according to an embodiment. [Fig. 113A]FIG. 11 is a diagram showing an example of a screen transition displayed on an iOS (registered trademark) smartphone that can be linked with a system according to an embodiment. [Fig. 113B] FIG. 11 is a diagram showing an example of a screen transition displayed on an iOS (registered trademark) smartphone that can be linked with a system according to an embodiment. [Figure 113C] FIG. 11 is a diagram showing an example of a screen transition displayed on an iOS (registered trademark) smartphone that can be linked with a system according to an embodiment. [Fig. 113D] 1A and 1B are examples of enlarged views of screens displayed on a smartphone screen, in which (A) is an enlarged view of a login screen, and (B) is an enlarged view of a menu screen. [Figure 113E] 1A and 1B are examples of enlarged views of screens displayed on a smartphone, in which (A) is an enlarged view of a setup screen, and (B) is an enlarged view of a connection standby screen. [Fig. 114] 13 is a flowchart showing an example of an area movement process executed by a processing substrate in a system according to an embodiment. [Figure 115] 13 is a flowchart showing an example of an area movement lottery process executed by a processing board in a system according to an embodiment. [Fig. 116] 13 is a flowchart showing an example of a feeding event process executed by a processing board in a system according to an embodiment. [Figure 117] 13 is a flowchart showing an example of a feeding event process executed by a processing board in a system according to an embodiment. [Figure 118] 13 is a flowchart showing an example of a cat scratching event process executed by a processing board in a system according to an embodiment. [Figure 119] 13 is a flowchart illustrating an example of a human detection event process executed by a processing board in a system according to an embodiment. [Figure 120] 11 is a flowchart showing an example of a system update process executed by a processing board in a system according to an embodiment. [Figure 121]10 is a flowchart illustrating an example of a server process executed in a cloud server according to an embodiment. [Figure 122] 11 is a flowchart showing an example of an external connection process in a system according to an embodiment. [Figure 123] 1 is a flowchart showing an example of an alarm process in a system according to an embodiment. [Figure 124] 1 is a flowchart showing an example of a greeting memorization process in a system according to an embodiment. [Fig. 125] 11 is a flowchart showing an example of a greeting response process in the system according to an embodiment. [Fig. 126] 1 is a flowchart showing an example of a dialogue memorization process in a system according to an embodiment. [Figure 127] 1 is a flowchart showing an example of a dialogue response process in a system according to an embodiment. [Fig. 128A] 1 is an example of an outline for explaining assumed voice input information from a user in a system according to an embodiment. [Fig. 128B] FIG. 11 is a diagram showing an example of names of defined time periods in a system according to an embodiment. [Fig. 129A] 13 is an example of an outline for explaining a response sentence created based on acquired weather information in a system according to an embodiment. [Fig. 129B] 13 is an example of an outline for explaining a response sentence created based on acquired weather information in a system according to an embodiment. [Figure 129C] 13 is an example of an outline for explaining a response sentence created based on acquired weather information in a system according to an embodiment. [Fig. 130] 13 is an example of weather event processing executed by a processing board in a system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0114] Hereinafter, the embodiments will be described in detail with reference to the drawings. The embodiments shown below are examples of the embodiments of the present disclosure, and the present disclosure is not limited to these embodiments. In the drawings referred to in the present embodiment, the same parts or parts having similar functions are given the same or similar symbols (symbols with A, B, etc. added after the numbers), and repeated explanations may be omitted. In addition, in each of the drawings referred to in the following description, the scale may be different from the actual scale in order to make each member, each region, etc. recognizable.

[0115] <1. Overview of Display Device 1> 1A and 1B are diagrams for explaining an overview of a display device according to an embodiment of the present invention. The display device 1 displays an image so that an observer can observe the image through an observation area SC. In this embodiment, observation may be interpreted as meaning to see or visually inspect. The observer is a user who uses the display device 1, for example, a person who observes an image displayed on the display device 1. The observation area SC is an area provided on the front side of the display device 1, and is a rectangular area here. The display device 1 displays an image by aerial imaging. Aerial imaging refers to forming an image in the air, for example, by using light reflection. Aerial imaging is sometimes adopted to give an observer the sensation that the image is floating in a specified space. An image displayed by aerial imaging is called an aerial image.

[0116] In the case shown in FIG. 1A, the display device 1 displays the character 1100 as an aerial image. The character 1100 is a female character. The display device 1 further displays a stage image 2100, which means a stage, below the character 1100. The stage image 2100 shows a magic circle here. An observer who sees such a display can get the impression that the character 1100 is standing on a stage. The colors of the character 1100 and the stage image 2100 are not particularly important. For example, the skin of the character 1100 is flesh color. The hair color and the costume of the upper body of the character 1100 are a relatively light bluish color (e.g., light blue), the accessories placed on the knees are a darker bluish color (e.g., blue), and the skirt and shoes are white. The stage image 2100 is a relatively dark bluish color (e.g., dark light blue).

[0117] In the case shown in Fig. 1B(a), the display device 1 displays a character 1200 as an aerial image. The character 1200 is a cat character. In the case shown in Fig. 1B(b), the display device 1 displays a stage image 2200, which means a stage, below the character 1200. In this case, the stage image 2200 shows the character string "Yupiteru" and the character string "Juno" (pronounced "Yuno"), which is the name of the character 1200, written alternately along the perimeter of a circular stage.

[0118] The colors of the character 1200 and the stage image 2200 do not matter. Here, the character 1200 and the stage image 2200 will be described in more detail with reference to FIG. 1C. The character 1200 is roughly divided into a character body 1210 and a collar 1220. The character body 1210 is a cat character named "Juno" here. The character body 1210 includes body parts shown in white, such as the area between the eyebrows, around the nose and mouth, the abdomen, and the front side of the legs. Among the body parts of the character body 1210, the parts shown in light gray are a relatively light brown color (e.g., ocher), such as the areas above and beside the eyes of the face, the upper part of the torso, and the parts other than the front side of the legs. Among the body parts of the character main body 1210, the parts shown in darker gray are an even darker brown color (for example, brown), and for example, the face, head, torso, and legs with stripes drawn with thin lines correspond to these parts. The collar 1220 means a collar displayed at the neck position of the character main body 1210. The collar 1220 is a reddish color (for example, red). A tag 1221 is attached to the collar 1220 at a position below the face of the character main body 1210. The tag 1221 is circular and has a predetermined mark written on it. The mark is a mark in which a "●" (a circle with a filled-in inside) is placed below a "V"-shaped symbol, and each of the marks is written in a reddish color (for example, red). The mark written on the tag 1221 may function as a trademark indicating the source of the display device 1. The stage image 2200 as a whole is a brownish color, with the circle and the text being a relatively dark brown color (e.g., brown), and the image resembling smoke inside the circle being a lighter brown color (e.g., ochre).

[0119] In FIGS. 1A, 1B, and 1C, the background of the characters 1100, 1200 is shown as black, but is not limited to pure black, and may be a relatively dark background (for example, black or other dark colors).

[0120] The characters shown in FIG. 1A, FIG. 1B, and FIG. 1C are examples. When displaying a humanoid character, the display device 1 is not limited to a female character, and may display a male character. The display device 1 may display any of real people (e.g., family members), real people who existed in the past (e.g., historical figures), and fictional characters (e.g., characters appearing in fictional works such as manga and anime). When displaying a non-humanoid animal character, the display device 1 is not limited to a cat character, and may display any of animal characters that can be kept in a general household, such as dogs and hamsters, other animal characters such as horses and cows, and fictional animal characters (e.g., characters appearing in fictional works such as manga and anime). The display device 1 may also display other characters having a collar 1220.

[0121] Based on the communication function, the display device 1 moves the body parts of the character or outputs a sound that imitates the character's speech. The communication function is a function that reproduces communication between an observer and a character, or a function that reproduces pseudo-communication. In the communication function, the character moves in response to the behavior of the observer (for example, speech or body movement). For example, the character 1100 performs a motion that reproduces a motion that a human normally performs, such as moving the whole body, such as dancing, or changing facial expressions (for example, expressing emotions such as joy, anger, sadness, and happiness). For example, the character 1200 performs a motion that reproduces a motion that an animal normally performs (for example, yawning, moving the tail, walking, etc.). In addition, the display device 1 outputs, for example, a singing sound as if the character 1100 is singing, or outputs a sound related to a dialogue with the observer, in accordance with the motion of the character 1100. For example, the display device 1 outputs a sound that means a cat's meow, or outputs a sound related to a dialogue with the observer, in accordance with the motion of the character 1200. Such output of video and audio is not limited to the communication function, and may be performed by various functions of the display device 1.

[0122] The dialogue with the user in the communication function may be realized, for example, by cooperation between a voice recognition application (an application having a voice recognition engine function) incorporated in the display device 1, an external voice recognition server (not shown), and an external dialogue server (not shown). For example, when an operation unit 2214 described later is pressed, the voice recognition application is started and voice recognition for dialogue with the character is enabled. In this embodiment, enabling the voice recognition is to operate the voice recognition function, for example, by turning on the voice recognition engine and putting the device into a state of waiting for voice input. Then, the voice uttered by the user (also referred to as user voice) is transmitted from a communication control circuit 706 of a processing board 224 described later to the voice recognition server, and the voice recognition server executes voice recognition processing on the user voice and outputs the voice recognition result. For example, in the dialogue server, for each character string that can be input, the voice uttered by the character (for example, the character 1100) (also referred to as character voice) is registered in advance in association with each other. Then, the voice recognition result by the voice recognition server (for example, the voice recognition result shown as a character string) is input to the dialogue server, and the character voice corresponding to the voice recognition result is output from the dialogue server. The character voice output by the dialogue server is then output from the display device 1. In this manner, a response process to the voice by the user is executed, and a dialogue between the user and the character is reproduced.

[0123] The display device 1 is used by general users, for example, as a display device for entertainment. In this case, the display device 1 is used, for example, in a residence such as the user's home. The display device 1 may be used for commercial purposes (for example, advertising) or other purposes. For example, in the case of commercial use, the display device 1 is installed in a public place such as a store or other commercial facility. The display device 1 may display a video in which a character introduces or advertises a product or service at a venue such as an exhibition of the product or service. The display device 1 may be used for purposes other than these.

[0124] <2. External configuration of display device 1> 2, 3 and 4 are perspective views showing the external configuration of the display device 1. FIG. 2 is a view of the display device 1 as viewed from the diagonal front right. FIG. 3 is a view of the display device 1 as viewed from slightly above diagonally front left. FIG. 4 is a view of the display device 1 as viewed from slightly below diagonally front right. FIG. 5 is a front view of the display device 1. FIG. 6 is a right side view of the display device 1. FIG. 7 is a front side view of the display device 1. FIG. 8 is a plan view (top view) of the display device 1. FIG. 9 is a bottom view of the display device 1. FIG. 10 is a rear view of the display device 1.

[0125] The display device 1 has a rectangular parallelepiped appearance. The display device 1 has a housing 100. The housing 100 is a box-shaped member that forms the outer shape of the display device 1. The housing 100 is rectangular parallelepiped. The housing 100 is longer in the height direction (also referred to as the up-down direction) than in the width direction (also referred to as the left-right direction). The width direction and depth direction (also referred to as the front-rear direction) of the housing 100 are approximately the same length. The ratio of the width direction, height direction, and depth direction (also referred to as the front-rear direction) of the display device 1 is approximately 2:3:2. In the following description, the side of the depth direction of the display device 1 where the observer is located is referred to as the "front side" (corresponding to the first direction side), and the opposite side is referred to as the "rear side" (corresponding to the second direction side).

[0126] The housing 100 has R-shaped corners. As shown in FIG. 5, when the display device 1 is viewed from the front side, each of the four corners of the housing 100 is rounded. The surface of the housing 100 may be processed to give it a matte finish to give it a textured look. In this way, a rectangular parallelepiped display device 1 with excellent design is provided. The housing 100 may be formed of a resin material, but may also be formed of metal or other materials. The housing 100 has a light-shielded image display space inside. The image display space is a space inside the housing 100 where an aerial image is displayed (formed). The image display space is entirely or in large part formed of a light-shielding material. Therefore, the material constituting the housing 100 prevents external light from entering the image display space. For example, the entire housing 100 may be formed of a material that blocks external light, or the outer surface or the surface facing the image display space may be covered with a material that blocks light. The color of the outer surface of the housing 100 is not important, but it is preferable to use, for example, white or black, which gives off a sense of luxury or simplicity. The display device 1 is preferable to have a size and weight that allows a person such as an observer to carry it alone.

[0127] The portions of the housing 100 exposed to the outside are divided into a right panel 101, a front panel 102, a left panel 103, a rear panel 104, an upper cover portion 105, and a lower cover portion 106. The right panel 101, the front panel 102, the left panel 103, the rear panel 104, the upper cover portion 105, and the lower cover portion 106 are plate-shaped members. The right panel 101 is a panel located on the right side of the housing 100 when viewed from the front side. The left panel 103 is a panel located on the left side of the housing 100 when viewed from the front side. The right panel 101 and the left panel 103 have the same shape and the same dimensions. In addition, the right panel 101 and the left panel 103 are symmetrical in the height direction, and a common mold can be used to form the right panel 101 and the left panel 103. Therefore, the right panel 101 and the left panel 103 can use the same material, which is expected to reduce costs and the effort required for assembly.

[0128] The front panel 102 is located on the front side of the display device 1. The front panel 102 has an opening 110 into which the dimming member 200 can be attached. The observation area SC is provided in the front panel 102. The observation area SC is an area observed by an observer who views an image displayed on the display device 1. The observation area SC is also an area in which the opening 110 formed in the housing 100 exists. The dimming member 200 is provided in the observation area. The dimming member 200 is provided upright in the vertical direction so as to cover the opening 110. The dimming member 200 and the opening 110 are provided above the center of the height direction of the housing 100 when viewed from the front side of the display device 1. The dimming member 200 and the opening 110 are square or rectangular.

[0129] The light reducing member 200 includes a rectangular (or square) first region 210 used for observing an image, and a second region 220 surrounding the periphery of the first region 210. The second region 220 is an edge portion that is not used for observing an image, and may be a region that does not transmit light. Therefore, the observation region SC may be said to be a region in which the first region 210 exists. In the following description, when light reducing member 200 is mentioned, it may be thought of as the first region 210.

[0130] The light-reducing member 200 is an optical member that reduces external light that enters the image display space from the observation area SC. The light-reducing member 200 is provided to maintain the image display space in a dark state. The light-reducing member 200 is provided to suppress the occurrence of reflection in which the observer himself is reflected in the observation area SC, and to make it difficult for the observer to observe the internal components of the image display space. The light-reducing member 200 is a plate-shaped (including panel-shaped) member, but may be a member called a sheet-shaped or film-shaped member. The light-reducing member 200 may be a member called a smoke panel or smoke film. The light-reducing member 200 has, for example, a light transmittance of 24% and a thickness of about 2 mm. The light-reducing member 200 has a translucency that does not interfere with the observation of the image displayed in the image display space of the display device 1. The light-reducing member 200 is a semi-transparent member of a dark color such as black, but may be a blue color or other color.

[0131] The information area ST is provided below the observation area SC (the dimming member 200 and the opening 110) in the front panel 102. The information area ST is located in an opening opened in the front panel 102. The information area ST is an area where a sensor is used to collect predetermined information and where the predetermined information is presented to the observer. The electronic components provided in the information area ST include a first sensor 2331, a second sensor 2332, a microphone 2333, and a light-emitting unit 2334. The first sensor 2331 is a gesture sensor that detects gestures made by the observer. The second sensor 2332 includes one or more sensors other than the gesture sensor. The sensor of the second sensor 2332 is, for example, a gas sensor (odor sensor), a temperature sensor, and an illuminance sensor. The microphone 2333 collects sound and converts the collected sound into an audio signal. The light-emitting unit 2334 presents predetermined information by emitting light. The light emitting unit 2334 includes, for example, a light emitting body (for example, a light emitting diode) that emits light in a predetermined color, and a light guiding member that guides the light emitted by the light emitting body. In this embodiment, the light guiding member of the light emitting unit 2334 is arranged so as to surround the area where the first sensor 2331, the second sensor 2332, and the microphone 2333 are provided. The light emitting unit 2334 emits light in a color of, for example, a bluish color (for example, light blue), but may be configured to emit light in one or more colors of a greenish color, a reddish color, or other colors. The light emitting unit 2334 is not limited to a configuration having a light guiding member, and may emit light in a predetermined area by a configuration having, for example, a plurality of light emitting bodies.

[0132] An operation area OT is provided on the top cover 105. The operation area OT is an area that accepts an operation performed by a viewer. The operation area OT is an area in which operation units 2211, 2212, 2213, and 2214 are provided. The operation units 2211, 2212, 2213, and 2214 are switches that accept a pressing operation. The operation unit 2211 is operated when instructing to turn on or off the power of the display device 1. The operation unit 2212 is operated when instructing to increase the volume. The operation unit 2213 is operated when instructing to decrease the volume. The operation unit 2214 is operated when instructing to execute a predetermined function. The operation unit 2214 may be, for example, an operation unit called a special key. The operation unit 2214 may be operated, for example, when instructing the display device 1 to start voice recognition. Openings are provided in the top cover 105 at positions corresponding to the positions of the operation units 2211, 2212, 2213, and 2214. The operation units 2211, 2212, 2213, and 2214 are exposed to the outside by being positioned in each opening. Note that the number of operation units, the functions assigned to the operation units, and the acceptable operation methods described here are merely examples.

[0133] A human presence sensor 234 is provided on the lower cover 106. The human presence sensor 234 is exposed to the outside by being positioned at the opening of the lower cover 106. The human presence sensor 234 is a detection unit that detects the presence of a person (moving object) in the detection target space. In this embodiment, the human presence sensor 234 is a microwave Doppler sensor that emits microwaves and detects the presence of a person in the detection target space based on the reception intensity of the microwaves, which are the reflected waves. For example, the human presence sensor 234 detects whether a person is present in front of the display device 1.

[0134] The rear panel 104 of the display device 1 is provided with a power jack 2311 and a back cover 1041. The power jack 2311 is a jack to which a power cord is connected. The power jack 2311 receives power from an external power source or the like via the power cord. The back cover 1041 is a removable cover. When the back cover 1041 is removed, a slot 222 and a USB (Universal Serial Bus) connector 223 are exposed as shown in FIG. 11. The slot 222 is an attachment section to which an external storage medium is attached. The external storage medium corresponds to, for example, the microSD (registered trademark) standard, but may also correspond to microSDHC or other SD card standards or other standards. The USB connector 223 is an attachment section to which a USB cable or a USB port of an external device corresponding to the USB standard is attached. In the display device 1, data can be input / output between the display device 1 and the external device via the slot 222 and the USB connector 223. The number of attachment sections may be even greater. The portions of the slot 222 and the USB connector 223 that are exposed to the outside of the housing are on the rear side of the display device 1, so that they can be made difficult to be observed by an observer. In addition, the number of attachment parts to which external devices are attached and the corresponding standards are not particularly important.

[0135] The housing 100 is configured by joining a right panel 101, a front panel 102, a left panel 103, a rear panel 104, an upper cover 105, and a lower cover 106. In this embodiment, the joining method is screwing using screws, which are an example of a fastener. However, there are no screwed parts on the front panel 102 and the upper cover 105.

[0136] The right panel 101 is fixed with screws 710 and 720 at two locations, top and bottom, on the front side. The left panel 103 of the housing 100 is fixed with screws 730 and 740 at two locations, top and bottom, on the front side. The rear panel 104 is fixed with screws 751, 752, and 753 at three locations on the upper side, screws 754 and 755 at two locations near the center in the height direction, and screws 756, 757, and 758 at three locations on the lower side. The lower cover 106 is fixed with screws 761 and 762 at two locations on the front side. The front panel 102 and the upper cover 105 are relatively easily observed by an observer using the display device 1. On the other hand, the other parts are relatively difficult to observe by an observer using the display device 1. Therefore, according to the display device 1, it is possible to prevent the screw fastening points from being seen by the viewer, and maintain the aesthetic appearance of the display device 1. This can also contribute to giving the viewer a sense of immersion in the image.

[0137] FIG. 12 is a diagram for explaining an example of how the display device 1 is used by an observer. FIG. 12 shows the positional relationship between the display device 1 and an observer U using the display device, as viewed from the left side of the display device 1. The display device 1 is placed on a desk 3000. A chair 3100 is placed in front of (on the front side of) the display device 1. The observer U is seated on the chair 3100 facing the display device 1. In this example, the position of the eyes of the observer U is higher than the top surface of the display device 1. Therefore, the observer U observes the observation area SC by looking down at an angle θ. The angle θ is the angle between the horizontal direction and the line of sight of the observer U. The angle θ is within a range of angles θ1 to θ2. The angle θ is, for example, 15 degrees. The horizontal distance between the observer U and the observation area SC is approximately 50 cm.

[0138] <3. Principle of display on the display device 1> Fig. 13 is a diagram for explaining the principle of display of display device 1. Fig. 13 shows a schematic diagram of the inside of display device 1 as viewed from the left side of display device 1. A first display unit 310, an optical member 320, and a second display unit 330 are provided in an image display space S1 inside housing 100. First display unit 310 and optical member 320 work together to form an image display unit that focuses an aerial image in image display space S1.

[0139] The first display unit 310 and the second display unit 330 are display units that display images. The first display unit 310 and the second display unit 330 are display units that have a light source, for example, a liquid crystal display. The light source is a backlight. The first display unit 310 and the second display unit 330 display color (multi-color) images based on the three primary colors of red (R), green (G), and blue (B). The first display unit 310 and the second display unit 330 display images that are recognized as two-dimensional images in this embodiment.

[0140] The first display unit 310 is an 8-inch liquid crystal display. For example, in this case, the first display unit 310 includes a backlight, which is an example of a light source, and a display body in which liquid crystal elements, which are an example of a display element, are arranged two-dimensionally. The light source is, for example, a white light source. The liquid crystal element modulates light from the backlight in units of pixels. The liquid crystal element is configured, for example, by sandwiching liquid crystal molecules between a pixel electrode provided in units of pixels and a common electrode shared by a plurality of pixels. The first display unit 310 has a first display surface 311 that emits a first image light L1. The first display unit 310 emits image light for displaying characters 1100 and 1200 in terms of Figs. 1A to 1C. The first display unit 310 is arranged so that one end on the observation area SC side is located above the observation area SC and the back side is inclined so that it is lower. This inclination is equal to the inclination of the first display surface 311, and is approximately 20 degrees with respect to the horizontal plane. The inventors have found that by setting the inclination at approximately 20 degrees in this manner, the three-dimensional effect of the aerial image that can be given to the observer can be enhanced, and the virtual image formed in the image display space by the first image light L1 being reflected by the dimming member 200 can be made less likely to be observed. The basis for this will be described later. However, this angle is only one example. In the width direction of the display device 1, the height of each position on the first display surface 311 is the same.

[0141] The optical member 320 is an example of an optical system that forms an aerial image based on the first image light L1 from the first display unit 310. The optical member 320 has optical transparency and reflectivity. The optical member 320 is plate-shaped. The optical member 320 is a half mirror. The optical member 320 is made of an acrylic mitre and has an optical transmittance of about 30%. The thickness of the optical member 320 is about 2 mm. The optical transmittance is merely an example and may be, for example, 6%, and it is sufficient that the optical transmittance is sufficient to allow the observer to observe the aerial image displayed in the image display space S1. The optical member 320 is disposed in the image display space S1 at an incline with respect to the opening 110 and the first display surface 311. The optical member 320 is inclined so that the tip on the observation area SC side is located below the observation area SC and is higher toward the back side. This inclination is 35 degrees with respect to the horizontal plane. However, this angle is an example. In the width direction of the display device 1, the height of each position of the optical member 320 is the same.

[0142] The second display unit 330 is a 7-inch liquid crystal display. The second display unit 330 has a second display surface 331 that emits a second image light L2 of a displayed image. The second display unit 330 displays an image V2 that corresponds to the stage images 2100 and 2200 in terms of FIG. 1A and FIG. 1B. The second display unit 330 is arranged so that the normal direction of the second display surface 331 faces upward. The second display surface 331 is located on the rear side of one end of the lower side of the optical member 320. A pedestal unit 400 is provided on the front side of the second display surface 331. The pedestal unit 400 forms a part of the member that constitutes the image display space S1 of the housing 100. The upper surface of the pedestal unit 400 is located at approximately the same height as the second display surface 331, but the second display surface 331 may be slightly lower.

[0143] With the above configuration, first image light L1 emitted by first display unit 310 is incident on first surface 321 of optical member 320 at a predetermined angle. The incident first image light L1 is reflected by optical member 320 toward the front side of display device 1 at the predetermined angle. This reflected light L1r causes aerial image V1 to be recognized in image display space S1 of housing 100. Aerial image V1 is a virtual image recognized below the tip of the rear side of first display surface 311, for example, at a position below the intersection of first display surface 311 and optical member 320.

[0144] The second image light L2 emitted by the second display unit 330 is incident on the second surface 332 of the optical member 320 and is transmitted to the first surface 321 side. The second image light L2 directly enters the observer's eyes. The image V2 on the second display surface 331 is a real image that is recognized as being located below the aerial image V1. For example, the aerial image V1 is recognized at the center in the front-back direction of the second display surface 331. In this way, the aerial image V1 is recognized as the characters 1100 and 1200 in FIGS. 1A to 1C, and the image V2 is recognized as the stage images 2100 and 2200 in FIGS. 1A to 1C. According to the display device 1, the observer is given a three-dimensional effect of the image by observing the aerial image V1 combined using the optical member 320, and the three-dimensional effect can be further enhanced by having the observer observe the image V2 below it. Furthermore, by using the second display unit 330, it is possible to increase the freedom of selection of the image that the observer can view along with the aerial image, compared to a case in which a light-emitting body and a light-guiding member are combined to emit light in a predetermined pattern.

[0145] It is desirable to provide a viewing angle control member 312 that narrows the viewing angle to the front side on the first display surface 311 of the first display unit 310. In this way, even if the observer approaches the observation area SC and looks up, it is possible to prevent the image displayed on the first display surface 311 from being directly observed by the observer. No viewing angle control member that narrows the viewing angle to the front side is provided on the second display surface 331 so that the observer can easily observe the image V2.

[0146] The above is the basic display principle of the display device 1. Furthermore, the display device 1 has a light reducing member 200 and a light blocking member 500 as components for enhancing the visual effect. The action and effect of these will be described later.

[0147] <4. Internal configuration of display device 1> Next, the internal configuration of the display device 1 will be described. FIGS. 14, 15, 16, 17, and 18 are diagrams showing a state in which the right panel 101, the front panel 102, the left panel 103, and the rear panel 104 of the housing 100 have been removed. As shown in FIG. 14, the right inner panel 107 is exposed by removing the right panel 101. As shown in FIG. 15, the front inner panel 108 is exposed by removing the front panel 102. As shown in FIG. 16, the left inner panel 109 is exposed by removing the left panel 103. As shown in FIG. 16, the wiring section 800 and the partition member 900 are exposed by removing the rear panel 104.

[0148] The right inner panel 107 and the left inner panel 109 hold the first display unit 310 and the optical member 320 described in FIG. 13 by sandwiching them from both the left and right sides.

[0149] The screws 710 and 720 shown in Figs. 2, 4 and 6 pass through holes H1 and H2 that pass through the right panel 101, the front inner panel 108 and the right inner panel 107 in order to connect them. The screws 730 and 740 shown in Figs. 3 and 7 pass through holes H3 and H4 that pass through the left panel 103, the front inner panel 108 and the left inner panel 109 in order to connect them. The screws 751, 752 and 753 shown in Fig. 10 fasten the rear panel 104 and the top cover 105. The screw 754 fastens the rear panel 104 and the right panel 101. The screw 755 fastens the rear panel 104 and the left panel 103. The screws 765, 765 and 757 fasten the rear panel 104 and the bottom cover 106.

[0150] 19, 20, 21, 22, 23, and 24 are diagrams showing a state in which the right inner panel 107 and the left inner panel 109 are further removed. FIG. 19 is a diagram showing the display device 1 as viewed from the right side. FIG. 20 is a diagram showing the display device 1 as viewed from the left side. FIG. 21 is a diagram showing the display device 1 as viewed from diagonally below the right side. FIG. 22 is a diagram showing the display device 1 as viewed from diagonally above the right side. FIG. 23 is a diagram showing the display device 1 as viewed from diagonally below the left side. FIG. 24 is a diagram showing the display device 1 as viewed from diagonally above the left side.

[0151] Provided inside the housing 100 are a first display unit 310, an optical member 320, a second display unit 330, a base unit 400, and a light blocking member 500. The first display unit 310, the optical member 320, the second display unit 330, and the base unit 400 are configured as described with reference to FIG.

[0152] The light shielding member 500 is provided on the optical path of the first image light L1 from the first display unit 310, reflected by the light reducing member 200. The light shielding member 500 covers one end of the optical member 320 on the light reducing member 200 (observation area SC) side, in other words, one end farther from the first display unit 310, so as to prevent the reflected light from entering the optical member 320. The light shielding member 500 is, for example, a member that absorbs visible light, but may be any member that prevents the first image light L1 from passing through. The light shielding member 500 may be, for example, black or other dark colors, but may be any other color. The reason for this will be described later, but it is to prevent the image (virtual image) formed in the image display space S1 by the first image light L1 being reflected by the light reducing member 200 from being observed by the observer. In particular, it prevents the observation of images that may be observed when an observer observes an image by peering into the image display space S1 from the observation area SC. In addition to these images, it also prevents the observation of external objects such as the ceiling. The upper surface of the light blocking member 500 is approximately horizontal, but is not limited to this.

[0153] A first plate 601 is provided above the first display unit 310, and a first storage space S2 (see FIGS. 19 and 20) for storing various components is formed above the first plate 601. A second plate 602 is provided below the second display unit 330, and a second storage space S3 (see FIGS. 25 to 27) for storing various components is formed below the base unit 400 and the second plate 602.

[0154] As shown in Figs. 17 to 24, the first housing space S2 is a space separated from the image display space S1 by a first plate 601 provided on the first display unit 310. The first plate 601 is connected to the right inner panel 107 using screws 791, 792, and 793, and is connected to the left inner panel 109 using screws 794, 795, and 796. An operation board 221, a processing board 224, speakers 2251 and 2252, and a holder 226 are provided in the first housing space S2. The operation board 221 is a board on which operation units 2211, 2212, 2213, and 2214 are mounted on the surface. The operation board 221 supplies signals corresponding to the operations of the operation units 2211, 2212, 2213, and 2214 to the processing board 224. The processing board 224 functions as a processing board that controls the display device 1. The processing board 224 may be called a main board. The processing board 224 is electrically connected to the operation board 221 and the speakers 225A and 225B via, for example, a printed wiring board (for example, FPC). The processing board 224 is located on the rear side of the first housing space S2 in the depth direction of the display device 1. The processing board 224 is a so-called SoC board, and operates a processor and an OS (Operating System). The processing board 224 reads signals from the sensors mounted on the sensor board 233 and audio signals from the microphone 2333 by the OS, and obtains signals according to the operations of the operation units 2211, 2212, and 2213 from the operation board 221. The processing board 224 performs control for displaying images on the first display unit 310 and the second display unit 330. The processing board 224 is mounted with a slot 222 and a USB connector 223, and is exposed from the rear side of the display device 1.

[0155] The speakers 2251, 2252 are provided on both the left and right sides when viewed from the front side, and emit sound to the outside through openings 1051, 1052 opened in the top cover portion 105. It is desirable that the speakers 2251, 2252 are not too far away from the processing board 224. It is desirable that the speakers 2251, 2252 are spaced apart so that the sounds from each other do not interfere with each other as much as possible.

[0156] The holding part 226 is provided at a position overlapping the processing substrate 224, and holds the processing substrate 224 on its lower surface. A heat sink 2261 for heat dissipation or absorption is provided on the upper surface of the holding part 226 opposite to the side holding the processing substrate 224. The first display part 310 and the first plate 601 are inclined downward from the front side to the rear side. Therefore, the first housing space S2 gradually increases in height from the front side to the rear side, and there is more room in the height dimension toward the rear side. By utilizing this, members having a thickness such as the processing substrate 224 and the holding part 226 are housed. In this way, the housing part above the observation area SC is prevented from becoming longer, which contributes to the weight reduction and size reduction of the display device 1 and is also desirable in terms of design.

[0157] 25, 26, and 27 are diagrams showing the internal configuration of the display device 1 when the lower cover portion 106 is removed. The second plate 602 is provided below the base portion 400 and the second display portion 330. A second housing space S3 is formed between the second plate 602 and the lower cover portion 106. The second housing space S3 is provided with the control board 231, the wiring 232, the sensor board 233, and the human sensor 234. The front side inner panel 108 is coupled to the second plate 602 using screws 771, 772, 773, and 774. The front side inner panel 108 is coupled to the lower cover portion 106 using screws 775, 776, and 777. The front side inner panel 108 is fixed using the screws 771 to 777, but since it is covered by the front side panel 102 under the circumstances in which the display device 1 is used, the screws 771 to 777 are not exposed to the front side.

[0158] The control board 231 controls the supply of power and signals to the display device 1. The control board 231 is electrically connected to the processing board 224 via the wiring section 800, and transmits and receives various signals and supplies power to the processing board 224. The control board 231 is electrically connected to the sensor board 233 and the human sensor 234 via wiring 232. The wiring 232 is, for example, a printed wiring board (for example, an FPC).

[0159] The sensor board 233 is a board on which a first sensor 2331, a second sensor 2332, and a microphone 2333 are mounted. The sensor board 233 is provided in a region below the dimming member 200 (observation area SC) on the rear surface of the front side inner panel 108. The sensor board 233 is vertically erected so that its board surface extends in the height and width directions. The sensor of the sensor board 233 may be one that detects predetermined information. The first sensor 2331 is an optical sensor that emits light (e.g., infrared light) to the front side and detects the gesture of the observer by receiving the reflected light. The microphone 2333 collects sound and outputs a sound signal indicating the collected sound. The gas sensor as the second sensor 2332 detects odors such as gas odor and burnt odor. The temperature sensor detects temperature. The illuminance sensor detects illuminance. The first sensor 2331 is located approximately at the center in the width direction of the information area ST. The sensor board 233 outputs signals from the first sensor 2331, the second sensor 2332, and the microphone 2333 to the control board 231 via the wiring 232. The processing board 224 performs control, via the wiring unit 800 and the control board 231, to cause the light emitting unit 2334 to emit light when voice recognition is possible.

[0160] The human presence sensor 234 is disposed at a predetermined distance, such as about 30 cm, behind the front side internal panel 108 so as not to come into contact with the front side internal panel 108. This is desirable in order to ensure the detection sensitivity of the human presence sensor 234. The human presence sensor 234 is disposed slightly forward from the center in the depth direction. The human presence sensor 234 outputs a signal indicating the detection result to the control board 231.

[0161] The wiring section 800 includes a power line 810, a signal line 820, and a signal line 830. The wiring section 800 is routed from the lower side to the upper side of the housing 100. The wiring section 800 passes between the rear panel 104 and the partition member 900. The control board 231 supplies power to the processing board 224 via the power line 810. The control board 231 transmits and receives signals to and from the processing board 224 via the signal line 820. The processing board 224 supplies signals (e.g., driving signals and video signals) for driving the second display unit 330 via the signal line 830. The power line 810, the signal line 820, and the signal line 830 are provided so as to pass through the spatial region between the partition member 900 and the rear panel 104. A background plate 910 is provided on the front side of the partition member 900. The background plate 910 is, for example, a dark color such as black. The partition member 900 is formed in a plate shape using, for example, a resin material, and is a member that does not transmit light, so that the wiring section 800 cannot be observed from the image display space S1 side.

[0162] The processing board 224 functions as an operation control means for controlling the display of the first display unit 310 and the second display unit 330 and controlling the audio output from the speakers 2251 and 2252 based on the communication function. For example, the processing board 224 controls the display of the first display unit 310 and the second display unit 330 and controls the audio output from the speakers 2251 and 2252 according to information measured by the sensor board 233 and the human sensor 234. For example, when the processing board 224 detects the presence of a person based on the human sensor 234, it may cause the displayed character to perform an action such as a greeting or output the voice of the character. When the first sensor 2331 detects a gesture such as a hand wave, such as moving left and right, up and down, bye-bye, or back and forth, the processing board 224 controls the display of the first display unit 310 and controls the audio output from the speakers 2251 and 2252. For example, the processing board 224 rotates the character in the direction according to the hand wave left and right or up and down. When the processing board 224 detects a bye-bye gesture, it makes the character say bye-bye. When the processing board 224 detects a gesture of moving the hand back and forth, it returns the display of the character to the default (for example, FIG. 1A to FIG. 1C). The processing board 224 changes the display of at least one of the first display unit 310 and the second display unit 330 according to the illuminance measured by the illuminance sensor. For example, the processing board 224 may darken the display when the surroundings of the display device 1 become dark. The processing board 224 may perform control such as moving the eyes of the character based on the illuminance sensor. The processing board 224 controls the display of the first display unit 310 according to the temperature detected by the temperature sensor, controls the output of the sound from the speakers 2251 and 2252, and changes the content of the character's speech according to whether the surroundings of the display device 1 are hot or cold. Furthermore, the processing board 224 controls the light emission of the light emitting unit 2334. The processing board 224 changes the volume of sound in response to the operation of the operation units 2211, 2212, 2213, and 2214. The voice engine may be realized by the function of the processing board 224, or may be realized by an external recognition server.In the latter case, the processing board may have a communication module for performing wireless LAN such as Wi-Fi or network communication, and may communicate with the voice recognition server via the communication module.

[0163] The processing board 224 makes the emission luminance of the light source of the first display unit 310 brighter than that of the light source of the second display unit 330. This is because the inventors have found that by doing so, the difference between the emission luminance of the first display unit 310 and the emission luminance of the second display unit 330 can further enhance the three-dimensional effect of the aerial image that can be given to the observer. If the image display space S1 is bright, the three-dimensional effect may be lost, so the emission luminance of the second display unit 330 is lowered.

[0164] The first display unit 310 is inclined at an angle of approximately 20 degrees from the horizontal. This is to prevent the observer from sitting in a chair and peeking into it, which may cause display problems in combination with the display of the second display unit 330. In this way, the observer can see the entire display 1 even when looking from slightly above it. Therefore, the observer does not need to peek into it.

[0165] In this way, the ceiling is not reflected by the optical member 320 and observed by the observer, and the aerial image is not overlapped with the image of the second display surface 331, making it difficult to see clearly. The operation will be described. As shown in FIG. 28, it is assumed that the first image light L31, L32 is incident on the dimming member 200. In this case, the reflected waves L31r, L32r of the first image light L31, L32 are emitted from the dimming member 200 and further reflected by the optical member 320, and the observer observes the image V3, which is a virtual image. For example, when the first display unit 310 displays an image of a character, the image of the lower part of the character, for example, the vicinity of the feet, may be observed. Also, as shown in FIG. 29, light L4 is incident on the dimming member 200 from the ceiling, lighting, etc., passes through it, and is further reflected by the optical member 320, so that the ceiling is reflected near the lower end of the optical member 320. In this way, an image other than the original aerial image V1 may be observed, which may result in a deterioration in the display quality of display device 1.

[0166] Therefore, by disposing the light shielding member 500 on the optical path of the light reflected by the light reducing member 200 and on the optical path of the light L4 from the ceiling, lighting, etc., the display of the image V3 and the reflection of the ceiling are suppressed. As shown in FIG. 30, the light shielding member 500 suppresses the reflection at the optical member 320, so that the display of the image V3 and the reflection of the ceiling are suppressed. Since the light shielding member 500 is located below the observation area SC and the light reducing member 200 is also present, the presence of the light shielding member 500 is difficult for the observer to recognize. For example, assuming that the horizontal distance between the observer and the observation area SC is 50 cm and the angle θ at which the observer's line of sight is incident is 35 degrees, the light shielding member 500 is provided on the optical path of the light related to the image V3 and the reflection. As shown in FIG. 31, if the inclination angle of the first display unit 310 is 10 degrees, the position of the image V3 is displayed further back, making it easier to observe the image V3. To prevent this, it is necessary to extend the light blocking member 500 further to the rear side, which is undesirable. Therefore, it is preferable to set the inclination angle of the first display portion 310 to 20 degrees.

[0167] According to this purpose, even if the light blocking member 500 is not used, the optical member 320 may be arranged so as to avoid the region on the optical path of the reflected light of the first image light L1 reflected by the light reducing member 200. In this way, it is possible to prevent a virtual image formed in the video display space by the reflection of the first image light L1 by the light reducing member from being observed by the viewer.

[0168] In addition, the viewer makes a gesture near the first sensor 2331, which is a gesture sensor. In this case, if the first sensor 2331 is located above the observation area SC, the hand making the gesture may get in the way, which may hinder the observation of the image through the observation area SC. In this embodiment, since sensors such as the first sensor 2331 are provided below the observation area SC, even if the viewer makes a gesture while observing the observation area SC, the observation is prevented from being hindered.

[0169] The video display space S1 is darkened in order to make the floating image appear more three-dimensional. For example, when a specific character is displayed, the surroundings are dark, which gives the character a sense of floating. Also, when viewed from the front, the observation area SC is surrounded by the housing 100 (front panel 102). In this way, the floating image displayed in the video display space, combined with the housing located in front of it, can further enhance the three-dimensional feeling of the floating image that can be given to the observer.

[0170] The portions facing the image display space S1 are treated (for example, matte finished) to reduce light reflection. In this way, even if external light or other light enters the image display space S1, reflection is suppressed, making it easier to maintain a dark state. The internal space can be prevented from becoming bright. Such treatment may be applied to some or all of the frames of the background board 910, the base unit 400, the first display unit 310, and the second display unit 330.

[0171] Furthermore, when a fastening method using fasteners such as screws is adopted, the front panel 102 and the front inner panel 108 can be removed more easily than when they are attached with adhesives, etc., so dust that has entered the image display space can be removed with an air blower or a soft cloth. On the other hand, because the front panel 102 is not fixed with fasteners such as screws, it is possible to prevent the viewer from noticing this and disappointing the viewer.

[0172] 5. Detailed configuration of the processing board 224 FIG. 33 is a control block diagram of the processing board 224 included in the display device 1 according to an embodiment. As shown in FIG. 33, the processing board 224 is a controller having a CPU 701 functioning as an arithmetic processing device that executes arithmetic processing and the like, and a memory including a RAM 703 used as a work area and a ROM 702 storing a control program and the like. The processing board 224 may be read as, for example, a control unit (an example of an operation control means). Various programs are stored in the ROM 702 of the processing board 224, and the processing board 224 realizes various functions by executing these programs. For example, a voice recognition application for reproducing a dialogue between a user and a character is stored in the ROM 702, and the processing board 224 reproduces a dialogue between a user and a character by executing the voice recognition application based on input signals from various sensors 234, 2331 to 2332 or the GPS receiver 60. The processing board 224 also has a display control circuit 704 that controls images and the like displayed on the display device 1 (more specifically, the video display unit), and a voice control circuit 705 that controls voices output from the speakers 2551 and 2552. The processing board 224 also has a communication control circuit 706 that controls communication with other devices (for example, an external voice recognition server (also called a voice recognition engine) not shown, and an external dialogue server (also called a dialogue engine) not shown, etc.).

[0173] <6. About the communication function for Character 1100> Various controls for reproducing communication between the user and the character 1100 displayed on the display device 1 (for example, a video display unit) will be described below in order with reference to FIGS.

[0174] <About the smell-related communication function> For example, an odor sensor that detects an odor may be provided as the second sensor 2332 in the display device 1, and the processing board 224 may execute a voice output operation to output a voice corresponding to the detection result by the odor sensor as the character voice of the character 1100. The odor sensor may output a signal that changes according to the intensity of an odor, or output a signal that changes according to the components (odor components) contained in the odor. The odor sensor may output a signal that changes according to the type of odor. The odor sensor may be configured to detect the concentration of a specific odor component (tar, ammonia, alcohol, carbon dioxide, etc.) using, for example, a deodorizing filter. In this way, the processing board 224 can reproduce communication related to odors between the character displayed on the video display unit of the display device 1 and the user, and more diverse communication between the character and the user can be reproduced.

[0175] The processing board 224 may, for example, execute a voice output operation to output a voice corresponding to the type of detected odor as the character voice of the character 1100. Note that a single sensor capable of identifying multiple types of odor components may be configured as the odor sensor, or multiple sensors capable of identifying different types of odor components may be configured as the odor sensor.

[0176] FIG. 34(A) is a flowchart for reproducing communication related to odors, and FIG. 34(B) is a diagram showing a voice management table (also referred to as a voice management table) related to odors. The voice management table is stored, for example, in the processing board 224 of the display device 1 or in the dialogue server. As shown in FIG. 34(B), the voice management table preregisters, for each type of odor component (also referred to as an odor substance) detectable by the odor sensor, the type of odor that mainly contains that odor component and the voice (referred to as a character voice) emitted by the character 1100. For example, in the voice management table, a tobacco odor that mainly contains tar is registered in association with tar, and words of concern for the user's health, such as "Smoking too much is not good," are registered as the character voice when a tobacco odor is detected.

[0177] Then, first, in step S11, the processing board 224 (which may be read as the control unit) judges whether or not an odor has been detected. For example, when the concentration of an odor component detected by the odor sensor exceeds a predetermined threshold (reference value) for a specific odor, the processing board 224 judges that the odor component has been detected. In step S12, the processing board 224 judges the type of odor registered in association with the detected odor component in the voice management table. For example, when a concentration of tar exceeding a predetermined threshold is detected, it is judged that a tobacco odor has been detected. Then, the processing board 224 outputs a character voice corresponding to the type of odor detected from the speakers 2551 and 2552. For example, when it is judged that a tobacco odor has been detected, the processing board 224 executes a voice output operation that outputs a voice of words concerned about the user's health (here, "Smoking too much is not good"). Note that the following is an example of another example. For example, when the concentration of a specific main component contained in a fragrance such as aromatic oil or food exceeds a predetermined reference value, the processing board 224 determines that the odor of the fragrance or food has been detected, and outputs a voice as the character voice asking a question about the source of the odor. When alcohol exceeding a predetermined reference value is detected, the processing board 224 determines that an alcohol odor has been detected, and outputs a voice such as "You smell like alcohol" or "You've been drinking again, stop it" as the character voice of the character 1100. When a breath odor component (e.g., hydrogen sulfide, methyl mercaptan, dimethyl sulfide, etc.) exceeding a predetermined reference value is detected, the processing board 224 determines that bad breath has been detected, and outputs a voice such as "Your breath stinks a little, are you brushing your teeth?" or "Please take care of your periodontitis" as the character voice of the character 1100. In addition, when odor components contained in gas fuel, such as tertiary butyl mercaptan, dimethyl sulfide, or tetrahydrothiophene, are detected, the processing board 224 determines that the odor of gas fuel has been detected and outputs a voice such as "It smells like gas!" as the character voice of the character 1100.

[0178] In this way, the processing board 224 may execute control to output a sound corresponding to the type of odor detected as the character voice of the character 1100. By executing such control, it is possible to widen the range of communication between the character displayed on the video display unit and the user, and to increase the user's attachment to the character.

[0179] The display device 1 is not limited to a device that specifies the voice to be output based on a voice management table. The display device 1 may specify the voice to be output according to a scenario in which the content of the character's utterance changes based on the content of past dialogues (dialogue history), for example. This scenario may specify a branching scenario in which data specifying the voice to be output by the character is managed in a tree structure.

[0180] Here, an audio output operation of a sound corresponding to the detection result by the odor sensor is exemplified, but the present invention is not limited thereto, and the processing board 224 may execute a display output operation of displaying an image of a character performing a movement corresponding to the detection result by the odor sensor on the image display unit. For example, a display management table (not shown) may be provided in advance, similar to the audio management table, and the processing board 224 may execute a display output operation of displaying an image of the character 1100 performing a movement corresponding to the type of odor detected on the image display unit. The display management table may be, for example, a table in which data that defines the character 1100 in association with an odor component and a type of odor is registered. The data that defines the character 1100 may be data showing an image of the character 1100 to be displayed, or may be data that defines the content of the movement of the character 1100. For example, when some odor component is detected, the processing board 224 may display an image of the character 1100 performing a movement of sniffing an odor on the image display unit. Furthermore, when the processing board 224 determines that, for example, a cigarette odor has been detected, it may display on the video display unit an image of the character 1100 making a movement as if it were smoky. At this time, the processing board 224 may display an image imitating smoke on the video display unit. Furthermore, the processing board 224 may execute both the audio output operation and the display output operation as described above. For example, when some odor component is detected by the odor sensor, the processing board 224 may display on the video display unit an image of the character 1100 making a movement to sniff an odor, while outputting a sound such as "I smell something" from the speakers 2551, 2552 as the character voice of the character 1100.

[0181] Furthermore, the processing board 224 may control the character 1100 to change its reaction according to the concentration (also called detection intensity) of the odor component detected by the odor sensor. For example, the processing board 224 may selectively display one of a number of stepwise movement images (for example, "grimacing (making a disgusted expression)", "breaking into a cold sweat", "turning pale", "fainting", etc.) according to the concentration of the odor component.

[0182] The display device 1 may be capable of detecting that a user is smoking by combining, for example, a smoke sensor that detects smoke, a camera image (e.g., an image captured by the imaging unit 227), an image from a thermal camera, or another sensor as a method of distinguishing between tobacco odor and odors (odors) other than tobacco odor.

[0183] When the processing board 224 detects that the user is smoking, the processing board 224 may display an image simulating smoke (a video reproducing the appearance of smoke billowing out) superimposed on the character 1100 (in front of the character 1100). At this time, the processing board 224 may push the face of the character 1100 forward, close up the face, and make the character 1100 look displeased. The processing board 224 may display the degree of dislike of the character 1100 changed according to the intimacy, which is the degree of intimacy between the user and the character 1100. For example, when the intimacy is equal to or greater than a predetermined value, the processing board 224 makes the character 1100 look as if he or she forgives the user because he or she loves the user, and when the intimacy is less than the predetermined value, the processing board 224 makes the character 1100 look unpleasant. For example, the processing board 224 may specify the degree of intimacy according to the user's usage record of the display device 1 (for example, the past usage time or the content of past conversations). The algorithm for specifying the degree of intimacy is not limited to this.

[0184] <About the communication function based on user location information> Also, for example, the display device 1 may have a GPS (Global Positioning System) receiver 60 (see FIG. 33), and the following control may be executed based on the user's position information. Then, for example, the processing board 224 may acquire position information of the display device 1 held (carried) by the user as the user's own position information based on a signal from the GPS receiver 60, and execute the following control based on the acquired position information.

[0185] For example, it is preferable to control so that a voice corresponding to the frequency of stay (also called visit frequency) of the user staying at a place specified based on the user's location information is output as the character voice. Fig. 35 is a flowchart for reproducing communication related to the frequency of stay.

[0186] For example, in step S21, the processing board 224 judges whether or not the user's location information is received from the GPS receiver 60. When the user's location information is received, in step S22, the processing board 224 judges whether or not the number of stays (also referred to as the number of visits, etc.) in which the user stayed in a place specified based on the user's location information is equal to or greater than a predetermined value (for example, "2"). The frequency of stay is the number of times the user stayed in one place during a predetermined period (for example, one day or one week, etc.). For example, the processing board 224 searches for and specifies the user's stay location using a network, etc., based on the location information (for example, the user's current location information) acquired from the GPS receiver 60. Then, the processing board 224 calculates the user's stay frequency based on the number of stays in the specified place by the user, and judges whether or not the user's stay frequency is equal to or greater than a predetermined value. For example, if the user's visit frequency is equal to or greater than a predetermined value, in step S23, the processing board 224 controls so that a predetermined character voice registered in advance for the user's visit frequency is output from the speakers 2551, 2552. For example, if the user goes to the same place twice on the same day, the processing board 224 determines that the user's visit frequency is equal to or greater than a predetermined value, and outputs a character voice such as "Oh, I was there earlier too."

[0187] Moreover, the processing board 224 may perform control so as to output a voice corresponding to a location specified based on the acquired position information as the character voice of the character 1100. Fig. 36(A) is a flowchart for reproducing communication related to a place of stay, and Fig. 36(B) is a diagram showing a voice management table of the character voice related to the place of stay.

[0188] For example, in step S31, the processing board 224 judges whether or not the user's location information is received from the GPS receiver 60. When the user's location information is received, in step S32, the processing board 224 searches for and identifies the user's stay location using a network or the like based on the user's location information. Then, in step S33, the processing board 224 outputs a character voice corresponding to the identified location from the speakers 2551 and 2552 based on the voice management table of FIG. 36(B). For example, when the user goes to a supermarket, the processing board 224 outputs a character voice such as "What should I make for dinner today?", or when the user goes to a hospital, the processing board 224 outputs a character voice such as "Are you OK? Are you feeling unwell?" as the character voice of the character 1100. In addition, when the user goes to an amusement park, the processing board 224 outputs a character voice such as "You had a lot of fun."

[0189] The processing board 224 may also be controlled to output a voice corresponding to the number of stays (also called the number of visits) that the user has stayed in a place specified based on the acquired position information as the character voice of the character 1100. Fig. 37(A) is a flowchart for reproducing communication related to the number of stays, and Fig. 37(B) is a diagram showing a voice management table of the character voice related to the number of stays.

[0190] Steps S41 and S42 in FIG. 37(A) are the same as steps S31 and S32 in FIG. 36(A), and therefore description thereof will be omitted. In step S43, the processing board 224 acquires the number of times the user has stayed at the location specified in step S42. Then, in step S44, the processing board 224 outputs character voices according to the number of times the user has stayed from the speakers 2551 and 2552 based on the voice management table in FIG. 37(B). For example, when the user goes to a certain game center for the first time, the processing board 224 outputs character voices such as "The game was fun." Then, when the user goes to the certain game center again, the processing board 224 outputs character voices such as "I wanted to go again." Furthermore, when the user goes to the certain game center a predetermined number of times or more (for example, 10 times or more), the processing board 224 outputs character voices such as "Let's play games again."

[0191] Also, the processing board 224 may perform control so as to output a voice corresponding to the user's staying time at a location specified based on the acquired position information as the character voice of the character 1100. Fig. 38(A) is a flowchart for reproducing communication related to the staying time, and Fig. 38(B) is a diagram showing a voice management table of the character voice related to the staying time.

[0192] Steps S51 and S52 in FIG. 38(A) are the same as steps S31 and S32 in FIG. 36(A), and therefore description thereof will be omitted. In step S53, the processing board 224 acquires the user's stay time at the location specified in step S52. Then, in step S54, the processing board 224 outputs character voices according to the user's stay time from the speakers 2551 and 2552 based on the voice management table in FIG. 38(B). For example, when the user's stay time at the convenience store is within 5 minutes, the processing board 224 outputs character voices such as "You were quick. I was waiting, so I'm happy." Also, when the user's stay time at the convenience store is 5 minutes or more and less than 10 minutes (5 to 10 minutes), the processing board 224 outputs character voices such as "What did you buy?" Furthermore, when the user's stay time at the convenience store exceeds 10 minutes, the processing board 224 outputs character voices such as "You were late. I was worried."

[0193] The processing board 224 may perform the above-mentioned control based on the user's position information acquired using the GPS receiver 60, etc. In this way, the user can get the feeling that the character is responding to the user's actions, and the user's attachment to the character increases.

[0194] The display device 1 may be installed, for example, in the user's home, and the display device 1 (specifically, the communication control circuit 706) may communicate with the user's smartphone (a smartphone equipped with a GPS receiver) or the like to obtain location information of the user while out (location information of the smartphone carried by the user). Then, when the user communicates with the character 1100 after returning home, etc., the processing board 224 may perform control to execute the above-mentioned operations. The content of the character's voice and movement at this time may be changed as appropriate.

[0195] <Communication features related to user behavior patterns> Moreover, when it is determined that the user's behavior pattern matches a predetermined behavior pattern, it is preferable to control so that a voice corresponding to the predetermined behavior pattern is output as the character voice of the character 1100. Fig. 39(A) is a flowchart for reproducing communication related to a behavior pattern, and Fig. 39(B) is a diagram showing a voice management table of character voices related to behavior patterns.

[0196] For example, the day of the week (e.g., Monday to Friday) on which the user goes to work, the time of going to work (e.g., 8:30), and the specific route used when going to work are registered in advance by the user as the "action pattern when going to work" in the processing board 224. Then, for example, when the position information of the user is received from the GPS receiver 60 (step S61), the processing board 224 judges whether the user's action pattern matches a predetermined action pattern based on the user's position information (step S62). For example, if the user passes through a specific route at 8:30 on Tuesday, the processing board 224 judges that the user's action pattern matches the "action pattern when going to work". Then, in step S63, the processing board 224 outputs the voice (e.g., "Do your best at work") registered in association with the "action pattern when going to work" as the character voice of the character 1100 from the speakers 2551 and 2552 based on the voice management table of FIG. 39(B). In this way, the character reacts to the user's actions, which increases the user's attachment to the character. It is not essential that the user pre-registers a predetermined behavior pattern (e.g., a behavior pattern when going to work). For example, the processing board 224 may have a learning function and register a predetermined behavior pattern by learning the user's behavior pattern.

[0197] <Communication function regarding detection of suspicious individuals> Also, when a person detected in the detection target space is determined to be a suspicious person, a predetermined operation may be executed. Then, when a user of the display device 1 is detected after the predetermined operation is executed, a sound related to the detection of the suspicious person may be output as a character sound of the character 1100. Fig. 40 is a flowchart for realizing the operation related to the detection of the suspicious person.

[0198] For example, in the display device 1, as shown in FIG. 32, a photographing unit 227 (e.g., a camera) for photographing an observer (such as a user) is provided on the front side of the display device 1, and the photographing unit 227 (more specifically, a sensor included in the photographing unit 227) may be used as a detection means for detecting the presence of a person in a detection target space. The "detection target space" may be, for example, the field of view range (photographable range) of the photographing unit 227 in a space in which the display device 1 is placed (e.g., the user's home). The photographing unit 227 includes, for example, a lens and an image sensor (e.g., a CCD or CMOS), and photographs a multi-color image. The lens of the photographing unit 227 is provided at a position where the observer (particularly the face) can be photographed. In the example shown in FIG. 32, the lens of the photographing unit 227 is provided above the observation area SC of the front panel 102 arranged on the front side of the housing 100, near the center in the width direction of the display device 1. The photographing unit 227 outputs, for example, image signals of R, G, and B color components to the processing board 224. The processing board 224 processes this image signal to generate a captured image.

[0199] The processing board 224 monitors the detection target space using the photographing unit 227 (step S71). When the processing board 224 detects the presence of a person in the detection target space, it performs user authentication such as face recognition in step S72, and judges whether the detected person is a registered user of the display device 1 in step S73. When the processing board 224 judges that the person photographed by the photographing unit 227 is a registered user of the display device 1, it judges that the person is not a suspicious person. On the other hand, when it is judged that the person photographed by the photographing unit 227 is an unregistered user, the processing board 224 judges that the person is a suspicious person in step S74 and executes a predetermined operation. Examples of the "predetermined operation" include an operation of starting recording by the photographing unit 227, an operation of sounding an alarm at a relatively large volume, and an operation of reporting to a security company registered in advance. Alternatively, the "predetermined operation" may be an operation of the character 1100 calling out to the suspicious person (for example, "Who!? Tell me the password!"). Then, if there is no reply from the person determined to be a suspicious person, or if the password is incorrect, the above-mentioned recording start operation, etc. may be executed. If the password is incorrect, the processing board 224 may output a voice such as "That's not right! Say it again" as the character voice of the character 1100, and when it is determined again that the password is incorrect, the above-mentioned recording start operation, etc. may be executed. By doing so, for example, it is possible to make the suspicious person retreat from the detection target space, and crime prevention in the detection target space can be improved.

[0200] Then, when the user of the display device 1 is detected after the execution of the predetermined operation, the processing board 224 may output a voice regarding the detection of a suspicious person as the character voice of the character 1100. An example of the "voice regarding the detection of a suspicious person" is "That was scary." In this way, the attachment of the user to the character displayed on the video display unit can be increased.

[0201] Furthermore, when a detection target space is being monitored using the display device 1, the following operations may be performed in addition to the detection of suspicious individuals.

[0202] For example, the display device 1 may be placed in a reception space of a company, and the character 1100 may be displayed as a so-called receptionist performing reception duties. Specifically, when a microwave Doppler sensor (an example of a detection means) detects the approach of a person in a detection target space (e.g., a reception space), the processing board 224 may output a voice such as "Welcome. Which department would you like to serve?" as the character voice of the character 1100. Then, based on a response from the detected person (visitor), the processing board 224 may perform control so as to make a call to the department desired by the visitor.

[0203] <Communication function regarding the startup time of display device 1> Furthermore, the processing board 224 may control the language used in the character voice of the character 1100 so that the content of communication changes according to the startup time (e.g., cumulative startup time) of the display device 1 from a predetermined point in time (e.g., the time of the first startup). For example, as described below, the processing board 224 may control the character 1100 to speak to the user in more familiar language as the startup time of the display device 1 becomes longer.

[0204] For example, for the various character voices as described above, three patterns of character voices, a polite tone, a tone of speech that sounds like a friend to the user, and a tone of speech that sounds like a lover to the user, may be registered in advance in the voice management table. Then, for example, when the startup time of the display device 1 from a predetermined time is less than 100 hours, the processing board 224 outputs the character voice of the polite tone among the three patterns of character voices registered in the voice management table. Also, when the startup time is more than 100 hours and less than 1000 hours, the processing board 224 outputs the character voice of the friendly tone among the three patterns of character voices registered in the voice management table. Furthermore, when the startup time exceeds 1000 hours, the processing board 224 outputs the character voice of the lover to the user among the three patterns of character voices registered in the voice management table.

[0205] In this way, the user can sense a change in the relationship (eg, intimacy) with the character due to a change in the way the character speaks to the user.

[0206] The processing board 224 may control the character 1100 to decrease its intimacy with the user depending on the time that has passed since the previous startup. For example, even if the cumulative startup time of the display device 1 exceeds 100 hours, when one week has passed since the previous startup, the processing board 224 may output the character's voice in a polite tone rather than in an equal tone with the user. In this way, the user will interact with the character displayed on the video display unit as quickly as possible in order to maintain the intimacy with the character, and the attachment to the character can be further increased.

[0207] Also, the processing board 224 may output a greeting voice such as "Nice to meet you" as the character voice when it is first started, and when a predetermined period of time (for example, a convenient period such as one month) has passed since the first start, output a voice indicating an anniversary (for example, "It's been one month since we met") as the character voice. In this way, the attachment to the character 1100 can be further increased.

[0208] <About communication functions that link with other devices> Also, for example, when the display device 1 is installed in a user's home or the like, the following operation may be performed by cooperation with other devices. For example, when a user visits a place where a device capable of communicating with the display device 1 (hereinafter referred to as the other device) is installed, the other device may identify the user as described below and automatically guide the user with information suitable for the user (for example, gourmet information or event information). For example, a face image of a user who uses a cooperation system between a display device and other devices and a display device owned by the user (for example, an IP address, etc.) are associated and registered in an external server or the like in advance. When a person is detected by a sensor of a photographing unit provided in the other device, the control unit of the other device compares the face image of the person detected in the photographed image with the face image of the user registered in the external server to identify the user of the display device 1. Then, the control unit of the other device specifies the display device 1 (for example, the IP address, etc. of the display device 1) registered in association with the user in the external server, and transmits information indicating that the user of the display device 1 has been detected to the display device 1. The processing board 224 of the display device 1 has "information suitable for the user" (e.g., gourmet information and event information) registered in advance, and the processing board 224 transmits the registered "information suitable for the user" to the other device in response to receiving a signal from the other device. Then, the control unit of the other device outputs (e.g., outputs audio) the "information suitable for the user" received from the display device 1. When providing the "information suitable for the user", the other device may display the character 1100 on its display screen. Also, the information provided to the user may be transmitted from the other device to the display device 1 and may be accumulated as user information in the processing board 224 of the display device 1. Then, for example, when the display device 1 is installed at the user's home, when the user's return is detected, the processing board 224 may output a voice such as "You went to **, how was it?" as the character voice of the character 1100. In this way, the user's attachment to the character displayed on the video display unit can be increased.

[0209] <Other communication features> Furthermore, the following control may be performed based on the detection results from various sensors.

[0210] For example, a wind detection means (e.g., a wind sensor) capable of detecting wind (e.g., wind speed) may be provided on the front panel 102 of the display device 1, and the following control may be performed based on the detection result by the wind sensor. The wind may be, for example, artificially generated by the user, for example, wind generated by moving a body part such as waving a hand toward the display device 1, or wind generated by blowing breath (exhalation). For example, when the wind sensor detects wind with a wind speed equal to or higher than a predetermined value, the processing board 224 may display and output an image of fluttering hair (e.g., hair) or clothes (e.g., skirt) of a character (e.g., character 1100) displayed on the video display unit. Also, for example, when an image of a lit candle is displayed on the video display unit, if the wind sensor detects wind, the processing board 224 may switch to an image of the candle going out. Furthermore, the processing board 224 may switch the image depending on the strength of the detected wind (e.g., wind speed), such as by causing the clothing of the character 1100 to flutter more when the detected wind is stronger than a predetermined threshold (wind speed is greater than a predetermined threshold).

[0211] Also, for example, an object detection means (e.g., an ultrasonic sensor) capable of detecting the presence or absence (existence) of an object and the distance to the object may be installed in the display device 1, and the following control may be performed based on the detection result by the ultrasonic sensor. For example, when the processing board 224 determines that the observer's hand is touching the character 1100 recognized as a three-dimensional image based on the detection result by the ultrasonic sensor, the processing board 224 may display and output an image in which the character 1100 shows a reaction. Also, the processing board 224 may perform display control so that the character 1100 shows a different reaction depending on the part of the character 1100 touched by the observer. The part of the character 1100 touched by the observer may be a body part exemplified by the head, chest, abdomen, buttocks, or legs. Also, the part of the character 1100 touched by the observer may be the clothing (e.g., western clothes, skirt) or decoration (e.g., accessories) of the character 1100.

[0212] Also, for example, an earthquake detection means (for example, a seismic sensor) capable of sensing an earthquake and measuring the seismic intensity may be provided in the display device 1, and the processing board 224 may issue an earthquake alarm when the seismic sensor detects an earthquake.

[0213] Also, for example, a weather sensor capable of measuring the weather may be provided in the display device 1, and the processing board 224 may notify the user of weather changes, weather forecasts, etc. based on the measurement results of the weather sensor. The weather sensor may have, for example, an air pressure sensor, a temperature sensor, and a humidity sensor, and may be configured to be able to measure the weather based on the detection results of each sensor.

[0214] In addition, the processing board 224 may display an image in which the facial expression and movement (also called motion) of a character displayed on the image display unit changes depending on the temperature detected by a temperature sensor and / or the humidity detected by a humidity sensor.

[0215] In addition, the processing board 224 may display an image in which the facial expression and movement of a character displayed on the image display unit changes according to the detection results (e.g., the distance between the display device 1 and the observer) of a human presence sensor such as a microwave Doppler sensor.

[0216] The processing board 224 may also use a humidity sensor to determine whether or not the user has blown on the display device 1. The processing board 224 may also use a temperature sensor to determine the type of breath blown on the display device 1. These sensors may be provided in the display device 1 as the second sensor 2332, for example.

[0217] Here, there are two types of breath blown by the user: a breath blown with a "huff" sound, and a breath blown with a "ha" sound. Generally, when blowing cold breath, one blows quickly with a "huff," so the breath blown with a "huff" sound is referred to here as a "cold breath." Generally, when blowing warm breath, one blows slowly with a "ha" sound, so the breath blown with a "ha" sound is referred to here as a "warm breath."

[0218] Regarding these two types of breath, the inventor of the present application has obtained the following findings through experiments and the like. In detail, first, regarding humidity, a measurement result was obtained that when a user blows "blow" toward a temperature and humidity sensor (for example, a sensor in which a temperature sensor and a humidity sensor are integrated) at a position a certain distance away from the temperature and humidity sensor, the humidity rises sharply from the steady state immediately after the blowing. In addition, a measurement result was obtained that the humidity increases monotonically while the user is blowing, and when the user stops blowing, it takes several tens of seconds to return to the original steady state. A similar measurement result was obtained when the user blew "blow" toward the temperature and humidity sensor under the same conditions. Meanwhile, regarding temperature, a measurement result was obtained that when the user blew "blow", the temperature slightly drops from the steady state or remains almost the same, and when the user blew "blow", the temperature rises. The breath blown by the user toward the display device 1 is an example of wind that hits the display device 1.

[0219] Taking such knowledge into consideration, for example, in a display device 1 provided with a humidity sensor, when the humidity sensor detects that the humidity has risen sharply from a steady state, the processing board 224 may determine (also called "decision") that the observer has blown on the display device 1. Furthermore, when the monotonous increase in humidity is subsequently not detected, the processing board 224 may determine that the blowing on the display device 1 has stopped. Alternatively, when a predetermined time (for example, the time it takes to normally exhale (for example, about 3 seconds)) has elapsed since the sudden rise in humidity was detected, the processing board 224 may determine that the blowing on the display device 1 has stopped, regardless of the humidity detection result at that time.

[0220] Furthermore, when a sudden increase in humidity is detected by the humidity sensor, if the temperature sensor detects that the temperature has increased from the steady state (for example, at approximately the same time), the processing board 224 may determine that the type of breath blown by the observer is "warm breath." Conversely, when a sudden increase in humidity is detected by the humidity sensor, if the temperature sensor detects that the temperature has not changed from the steady state or has decreased from the steady state, the processing board 224 may determine that the type of breath blown by the observer is "cold breath."

[0221] The processing board 224 may then output video and audio corresponding to the detection of the blowing of breath as the movement and voice of the character displayed on the video display unit. The processing board 224 may also output video and audio corresponding to the type of breath blown as the movement and voice of the character displayed on the video display unit. For example, if the type of breath blown by the observer is "cold breath", the processing board 224 may display and output a video of the character's clothes fluttering, and if the type of breath blown by the observer is "warm breath", the processing board 224 may display and output a video of the character making a movement to avoid the breath.

[0222] The inventor of the present application also obtained the following findings using various gas sensors. In detail, the measurement results showed that when a person blows a breath toward a gas sensor at a position a certain distance (e.g., about 10 cm) from the gas sensor, the output value of the gas sensor drops immediately thereafter, and when the person stops blowing, the output value of the gas sensor starts to rise immediately thereafter. Conversely, the measurement results showed that when a person blows a breath toward the gas sensor, the output value of the gas sensor rises immediately thereafter, and when the person stops blowing, the output value of the gas sensor starts to drop immediately thereafter. The measurement results showed that when the wind sent from a blower (e.g., an electric fan) hits the gas sensor, the output value of the gas sensor drops, and when the wind from the electric fan no longer hits the gas sensor, the output value of the gas sensor rises.

[0223] Taking such knowledge into consideration, for example, in a display device 1 provided with a gas sensor, when the output value of the gas sensor drops sharply from a steady state, the type of breath blown by the observer may be determined to be "cold breath." Conversely, when the output value of the gas sensor rises sharply from the steady state, the type of breath blown by the observer may be determined to be "warm breath."

[0224] A wind sensor can also be used to detect breath. The inventors of the present application have confirmed that the wind sensor can detect a relatively weak wind, such as that caused by blowing on a piece of paper, and that the wind reaches its peak immediately when you blow on it, and that the response is good when the wind starts to blow. However, if you continue to blow, the wind will drop from its peak immediately. It is thought that a strong wind, such as a quick breath, will cool the heater and take time to recover.

[0225] In relation to the configuration of detecting breath using a wind sensor, the following configuration may be adopted.

[0226] When the wind sensor continuously detects a predetermined amount of wind, the processing board 224 may process the direction in which breathing is not being done. The processing of the direction in which breathing is not being done is, for example, a process of determining that breathing is not being done. In this case, the processing board 224 may process the direction in which breathing is being done when the time from when the wind sensor starts to detect the wind until it ends is within the time in which breathing is possible.

[0227] In this case, the wind sensor may be a sensor that detects wind by electrically measuring the heat taken away by receiving the wind. In this sensor, the wind received by the sensor corresponds to the amount of wind blown by the breath. This sensor is preferably set so that when the wind received by the sensor continues for a period of time equivalent to the time the breath is blown, the amount of wind drops to a value smaller than the actual amount of wind during the period of time, and is preferably set so that this drop is unlikely to occur when the wind received by the sensor is smaller than the amount of wind blown by the breath.

[0228] In the display device 1, a flow path for introducing wind into a plurality of wind sensors, which share a common breath inflow path, may be provided as the wind sensor. The flow path may be configured so that wind enters the different wind sensors at a predetermined ratio that is not 1:1. The processing board 224 may determine the magnitude and duration of the breath based on the ratio and differences in characteristics of the plurality of wind sensors. The display device 1 may include a plurality of wind sensors that have different timings at which the volume of the wind drops to a value less than the actual volume of air during the duration, and the processing board 224 may determine the magnitude and duration of the breath based on the outputs of the plurality of wind sensors. The display device 1 may include at least a wind sensor that is not a gas sensor and a gas sensor as the plurality of wind sensors.

[0229] Also, a gas sensor for detecting a predetermined gas may be used as the wind sensor. A gas sensor for detecting a predetermined gas, which is equipped with a heater, may be used as the wind sensor. In this case, the gas sensor equipped with a heater may have a characteristic of outputting a larger amount of detected gas when the degree of blowing is relatively small (as a result of detecting the gas contained in the breath), and outputting a smaller amount of detected gas when the degree of blowing is relatively large (due to cooling of the heater by the wind of the breath).

[0230] When the processing board 224 determines that breathing is being done, if the strength of the blowing is relatively strong (for example, if the air volume is above a threshold), an animation that moves the clothes, hair, etc. of the character 1100 is displayed, whereas if the strength of the blowing is relatively weak, a different animation from the animation that moves the clothes, hair, etc. of the character 1100 is displayed.

[0231] Incidentally, when a gas sensor is used to detect breath, the gas sensor outputs the above-mentioned output value in response to components contained in human breath (e.g., carbon dioxide).The reason why the output value of the gas sensor differs between when a person blows a "puff" and when a person blows a "breath" is thought to be that when a person blows a "puff," the speed of the breath is high and it is likely to diffuse to the surroundings immediately after hitting the gas sensor, whereas when a person blows a "breath," the speed of the breath is low and the components of the breath are likely to remain in place even after hitting the gas sensor.

[0232] Furthermore, the processing board 224 may also use the result of humidity detection by the humidity sensor, for example, to determine whether the detected wind is due to the viewer's breath or the wind from a fan. For example, in the display device 1, when the humidity sensor detects that the output value of the gas sensor drops sharply from the steady state and the humidity rises sharply from the steady state, the processing board 224 may determine that the user has blown on the display. On the other hand, when the output value of the gas sensor drops sharply from the steady state but the humidity remains at the steady state, the processing board 224 may determine that the wind is being blown by the fan. Furthermore, the processing board 224 may also use the result of temperature detection by the temperature sensor to determine the type of the breath when it is determined that the user has blown on the display. Also, when the output value of the gas sensor rises and falls repeatedly within a relatively short fixed period, the processing board 224 may determine that the wind from a swinging fan is blowing on the display.

[0233] The wind sensor, humidity sensor, temperature sensor, temperature and humidity sensor, and gas sensor used to detect wind are examples of wind detection means. The display device 1 may have an air pressure sensor as a sensor for detecting wind. This is because air pressure can change depending on the occurrence and strength of wind.

[0234] The above-mentioned various controls can be executed. Note that the above-mentioned controls are not limited to the character 1100, and may be executed for the communication functions related to other characters (for example, the character 1200).

[0235] <7. Specific examples of images displayed by the display device 1> Next, specific examples of images displayed by the display device 1 will be described with reference to Figs. 41 to 55. Figs. 41 to 55 are diagrams showing examples of characters (images including the character 1200 described in Figs. 1B and 1C) displayed on the display device 1. Each image shown in Figs. 41 to 55 is an image that is displayed on the display device 1 and can be observed by the observer through the observation area SC. Although stage image 2200 is displayed in Figs. 41 to 55, character 1200 may be displayed without displaying this.

[0236] 41 to 47 show displays that the display device 1 performs during a period when it waits for an observer's action based on the communication function. The display device 1 (specifically, the processing board 224) performs the display described below by switching between regular and random displays. For example, as shown in FIG. 41(a), the character 1200 is displayed in a standing posture facing the front side (observation side) of the display device 1. The display device 1 may use this posture as a basic posture and display during a period when communication based on the communication function is not performed. For example, as shown in FIG. 41(b), the character 1200 performs a motion of licking its front paws. For example, as shown in FIG. 41(c) and FIG. 42(a), the character 1200 performs a motion of looking alternately to the left and right. For example, as shown in FIG. 42(b), the character 1200 performs a motion of facing the ground. For example, as shown in FIG. 42(c), the character 1200 performs a motion of stretching while sitting. For example, as shown in Fig. 43(a), character 1200 performs an action of peering into the ground. Furthermore, as shown in Fig. 43(b) and Fig. 43(c), character 1200 performs an action of walking forward. When character 1200 performs a walking action, display device 1 may not display a stage image, but may display an image in which character 1200 steps in place and the overall position of character 1200 does not move much. When character 1200 performs an action other than walking, display device 1 may display a stage image at the feet of character 1200.

[0237] For example, as shown in FIG. 44(a), the character 1200 performs a motion of peering into its own abdomen. For example, as shown in FIG. 44(b), the character 1200 performs a motion of tilting its head. For example, as shown in FIG. 44(c), the character 1200 performs a motion of rubbing its own eyes. For example, as shown in FIG. 45(a), the character 1200 performs a motion of yawning. For example, as shown in FIG. 45(b), the character 1200 performs a motion of licking its own front paws while sitting. For example, as shown in FIG. 45(c), the character 1200 performs a motion of looking diagonally forward to the left. As shown in FIGS. 46(a) and (b), the character 1200 performs a motion of stretching. For example, as shown in FIG. 46(c), the character 1200 performs a motion of peering into the ground. As shown in FIG. 47(a), the character 1200 faces a direction diagonally forward to the right of the display device 1, and then faces forward as shown in FIG. 47(b). As shown in FIG. 47(c), the character 1200 makes the motion of licking its front paws.

[0238] 48 to 55 are examples of display of character 1200 performed by display device 1 (more specifically, processing board 224) during dialogue processing in the communication function. The character strings shown in Figs. 48 to 55 are character strings that can be recognized by voice recognition, and are displayed above character 1200. This is preferably such that a phrase (sentence) recognized by voice recognition is displayed for a predetermined time after voice recognition, and is made invisible after the predetermined time has elapsed. Then, it is preferably controlled so that character 1200 begins speaking after the phrase (voice recognition result) is made invisible.

[0239] FIG. 48(a) is displayed when the display device 1 recognizes the observer's utterance of "Come here, come here", and in response, a voice is output. FIG. 48(b) to FIG. 49(b) are displayed when the display device 1 recognizes the observer's utterance of "Sit", and in response, a voice is output. FIG. 49(c) is displayed when the display device 1 recognizes the observer's utterance of "I wonder if I'm hungry", and in response, a voice is output. FIG. 50(a) is displayed when the display device 1 recognizes the observer's utterance of "Policeman", and in response, a voice is output. FIG. 50(b) and (c) are displayed when the display device 1 recognizes the observer's utterance of "It's snack time", and in response, a voice is output. FIG. 51(a) is displayed when the display device 1 recognizes the observer's utterance of "Shake your hand", and in response, a voice is output. 51(b) to 52(b) are displayed when the display device 1 recognizes an observer's utterance of "cute," and in response, a sound is output. When the display device 1 displays the character 1200 at a position other than the center of the stage image (for example, a position toward the front), when rotating the character 1200, it is preferable that the display device 1 also rotates the stage image 2200 in the same manner as the character 1200. Also, it is preferable that the display device 1 displays the stage image 2200 having a shape with a comparatively large distance in the depth direction, and displays an image in which the character 1200 moves along the outer edge of the stage image 2200 inside the stage image 2200 (for example, an image in which a cat character walks along the outer edge of a circular stage).

[0240] FIG. 52(c) is displayed when the display device 1 recognizes the observer's utterance of "Hello," and in response, a voice is output. FIG. 53(a) is displayed when the display device 1 recognizes the observer's utterance of "Eat dinner," and in response, a voice is output. FIG. 53(b) and (c) are displayed when the display device 1 recognizes the observer's utterance of "Maybe it's a girl," and in response, a voice is output. FIG. 54(a) is displayed when the display device 1 recognizes the observer's utterance of "Wait," and in response, a voice is output. FIG. 54(b) is displayed when the display device 1 recognizes the observer's utterance of "Good boy," and in response, a voice is output. FIG. 54(c) and FIG. 55 are displayed when the display device 1 recognizes the observer's utterance of "Yunho," and in response, a voice is output.

[0241] <8. More detailed explanation of character 1200> The character 1200 described in FIG. 1B, FIG. 1C, and FIG. 41 to FIG. 55 will be described in more detail. In the following description, the description in which the display device 1 is the processing subject may be regarded as processing performed by the processing board 224. It is particularly preferable that the character 1200 is drawn with approximately 3:1 ratio. It has been found that doing so particularly emphasizes the sense of depth. The display device 1 may draw the character 1200 so that the head occupies approximately one-third of the height, the torso one-third, and the legs one-third in the vertical direction when viewed from the side (side). The display device 1 may draw the character 1200 so that the head occupies approximately one-third of the height, and the torso two-thirds in the horizontal direction when viewed from the side, except for the tail.

[0242] The display device 1 may have a function for drawing a tail that alternates between an invisible tail state and a visible tail state when the face is in front of the display device, and the time when the tail is visible may be shorter than the time when the tail is invisible. The tail is drawn behind the head, but the occasional visibility of the tail provides a greater sense of three-dimensionality, and the constant sense of three-dimensionality prevents eye and head fatigue. In particular, the display device 1 may perform such drawing processing in a standby state for voice recognition. The display device 1 may have a function for drawing the tail so that it appears from the top of the head when the tail is visible. In this way, the viewer's line of sight is usually on the face's eyes, and the tail suddenly pops out above the head from the center top, providing a sense of surprise and interest.

[0243] The display device 1 may provide a flat portion on the top of the head of the character 1200 when viewed from the front, and in particular, may draw the head portion between the left and right ears as flat when viewed from the front. In this way, a three-dimensional tail suddenly appears on top of the head from the flat top of the head, which has a weak three-dimensional effect, and this can provide more surprise and interest. The display device 1 may also draw the tail so that it appears from above the head, and then draw the part near the tip of the tail moving. In this way, the viewpoint can be focused on the tail part behind the face, and a stronger sense of three-dimensionality can be achieved. The display device 1 may also draw the tail so that it appears from the left and right sides of the body in the same way. When the tail appears from the left and right sides, the display device 1 may shorten the length and / or time for which the tail appears compared to when the tail appears from above the head. When the tail appears from the left and right sides, the display device 1 may draw the tail so that it appears briefly. The display device 1 may also draw the tail so that the viewpoint is moved to the back side from time to time, and a stronger sense of three-dimensionality can be achieved. The display device 1 may render the tail appear from above the head during voice recognition, and may render the tail appear from either the left or right side when the cat outputs a voice. The display device 1 may provide portions of the tail at predetermined intervals along its length that have a rendering pattern different from other portions. For example, a ring-shaped or arc-shaped pattern with a predetermined width may be provided at regular intervals on the cylindrical surface of the tail. The inventors have found that by doing this, the movement of the tail appears to move more dynamically in three-dimensional space.

[0244] The display device 1 may perform drawing to change the current posture to the same posture when there is a change from a state not undergoing voice recognition to a state undergoing voice recognition, so that the posture remains the same during voice recognition. The display device 1 may utter "Let's talk" before starting voice recognition. The display device 1 may have a function to detect the motion of an observer (e.g., a sensor or switch), and may have a function to utter "What's that?" when motion from the observer is detected. Both functions may be executed consecutively in the form of "What's that?" and "Let's talk", and then processing may be performed to enter the voice recognition state.

[0245] The display device 1 is provided with information on words that are praising, such as "cute," as words for speech recognition, and when words that are praising are recognized, the display device 1 may change its body posture. It is even better to move the cat's position, such as by walking. In this way, the observer, wanting to see the change in posture and movement of the cat, will be motivated to use words and actions that praise the cat, and excellent therapeutic effects such as improvement of behavior and speech can be obtained.

[0246] The display device 1 may be provided with a voice recognition phrase corresponding to feeding, and when the phrase is recognized, the tail may be drawn to wag faster than in other cases. In this way, the viewer can feel as if the cat is actually alive.

[0247] The display device 1 may draw the outer contour of the cat's eye as a roughly circular shape, and the iris as a vertically long ellipse. The display device 1 may draw the iris with a vertical gradation of light and dark, and may draw it in a lighter color as it goes downward. The display device 1 may provide a roughly circular or elliptical part in the center of the iris that is darker than the periphery. The display device 1 may draw a crescent or half-moon shaped white area along the outer edge of the iris in the part of the face above the iris and toward the outside. In contrast, the display device 1 may draw a circular white area below the iris and toward the center of the face. The display device 1 may continue drawing this white area even if there is a change in the position of the character 1200 (such as a change in posture) or a change in the position of the camera or light. Usually, highlights are often added by calculation as a reflection of light, etc. However, by doing so, the individuality of the cat can be emphasized in the part of the eyes that is most visible when speaking with voice recognition, and the impression of the cat as a unique cat can be made in the short time that voice recognition is performed.

[0248] The display device 1 can be in a state where the face is facing forward and a state where the face is facing downward, and when the face is facing forward, the outline of the eyes is drawn in a roughly circular shape, whereas when the face is facing downward, the eyes are drawn in a horizontally elongated oval shape.

[0249] The display device 1 may take a state in which the upper end of the face is above the upper end of the torso and a state in which the upper end of the face is below the upper end of the torso, and when the upper end of the face is above the upper end of the torso, the outline of the eyes may be drawn in a roughly circular shape, whereas when the upper end of the face is below the upper end of the torso, the eyes may be drawn in a horizontally elongated oval shape. The display device 1 has a function for drawing a state in which the mouth is open and yawning, and when drawing this, the eyes may be closed.

[0250] The display device 1 may prepare a plurality of standby motion patterns for the character 1200, and repeat the process of selecting one of these patterns and drawing it. The standby motions include a motion to lick a foot, a motion to stroke a face with a hand, a motion to lick a buttocks, a motion to face forward, a motion to look left and right, a motion to lick the ground, a motion to stretch, a motion to yawn, a motion to walk, and the like.

[0251] The display device 1 may be drawn so that the ears move after the face. For example, the ears may be drawn so that they remain in position for a while even after the face starts moving. The display device 1 may be provided with a function of simultaneously performing a tail-swaying motion from side to side when performing a licking motion. The display device 1 may be designed so that the pattern on the body of the character 1200 is asymmetrical when viewed from the front. This makes it easier to remember the character as a unique cat, and makes it easier to distinguish whether the right side or the left side is being seen. This is particularly useful when providing a scene that is so enlarged that the entire image of the character 1200 cannot be seen. In particular, the pattern may be a calico cat pattern, and more particularly, a tabby cat pattern. The inventors have found that this makes it particularly easy to grasp the three-dimensional effect. In particular, the pattern on the head may be provided in the front-back direction, while the pattern on the body may be provided in the up-down direction, so that the directions of the patterns are different. This makes it easier to recognize the change in the position of the camera or the cat, regardless of the direction in which they move.

[0252] The display device 1 may have a function of emitting the cry "Meow", and may have multiple intonations for the cry "Meow", with different motions corresponding to each cry. The intonations may include a questioning intonation, and the motion may include a head tilting motion.

[0253] <9. About the communication function for Character 1200> A supplementary explanation will be given regarding the communication function regarding the character 1200. By doing the following, a higher quality communication function with the character 1200 can be provided to the observer. Here, an outline of the action story between the person and the character 1200 until the character is tamed will be explained. In this embodiment, as shown in FIG. 32, a photographing unit 227 for photographing the observer is provided on the front side of the display device 1.

[0254] (9-1) First step: Taming the animal Usually, there are various types of cats, such as stray cats, that run away just by seeing a person, or run away when a person approaches. In such a case, the cat's behavior appears to be running away while repeatedly looking back to see the person. Here, the processing board 224 recognizes the presence of an observer based on the captured image captured by the imaging unit 227, and when the strength (received strength) of the microwave measured by the human sensor 234 exceeds a threshold, it displays an image in which the character 1200 behaves as if it is running away. This is to consider that a person has approached the cat. The imaging unit 227 recognizes the observer in order to show the observer a scene in which the character 1200 runs away. In other words, the character 1200 does not run away unless the observer approaches the display device 1 from the front side.

[0255] Next, the person is likely to call out to the cat, such as "Squat down (lowering the position of the eyes)" or "Come on, come on". Some cats stop at this time and watch the situation. However, when the person approaches, the cat runs away. This happens several times. It is considered that the cat will not get used to such human behavior. Therefore, when the microwave intensity measured by the human sensor 234 during the period when the image of the character 1200 running away is displayed is below a certain threshold, and when the processing board 224 recognizes a word, for example, "Come on", which indicates that the character 1200 is being called, the processing board 224 displays an image of the character 1200 performing an action of stopping. Here, the reason why the condition of the microwave intensity being below the threshold is set is because it is assumed that the human will stop.

[0256] Next, it is considered that a person feeds a cat that has stopped. However, if a person approaches a cat, the cat will run away, so it is considered that the only option is to place food in a place where the cat often comes. Therefore, the display device 1 has a function of displaying an image in which food is placed in the image display space. For example, when the processing board 224 receives a predetermined operation using the operation area OT (for example, an operation of the operation unit 2214), or when a predetermined gesture instructing to place food is detected by the first sensor 2231, it displays an image showing food in the image, assuming that food has been placed. This image may be displayed using either the first display unit 310 or the second display unit 330. In this way, the observer will place food, for example, every day. At this time, the processing board 224 may display the character 1200 as if it is watching the situation from a distance. When the processing board 224 does not recognize the presence of the observer using the human sensor 234 and the observer is not photographed by the photographing unit 227, it displays an image of the character 1200 eating food. Here, a situation is reproduced in which a cat does not eat food in front of a person. From the observer's perspective, a situation is created in which the food is gone before the observer notices it. The processing board 224 counts the number of times the observer has given food and stores it. This state continues for a while, but the processing board 224 may display the distance between the observer, the character 1200, and the person (the distance felt by the observer) gradually decreasing depending on the number of times the food is given. For example, the processing board 224 may start to feed the cat and display the character 1200 larger along with the number of times the food is given, recreating the distance decreasing (i.e., getting used to it a little). Furthermore, when the number of times the food is given increases and reaches a predetermined threshold, it is determined that the cat has gotten used to it, and even if the observer is looking at the character, that is, even if the processing board 224 recognizes the presence of the observer using the human sensor 234 and the observer is being photographed by the photographing unit 227, it displays an image of the character 1200 eating food. Only at this time can the observer observe the character 1200 eating food.

[0257] However, it is often the case that people do not know when a cat will eat. Therefore, the processing board 224 may determine the time period (or a single point in time) when the character 1200 will eat the food, and may make the character 1200 eat the food during that time period. This is to make the observer realize that the character seems to eat the food during that time period. When the processing board 224 detects that the observer is about to touch the character 1200, it causes the character 1200 to output a predetermined voice content such as "squeak" or "squeak" and makes the character 1200 move to run away. The observer's attempt to touch the character 1200 may be identified by the intensity of the microwaves detected by the human sensor 234 exceeding a threshold value when the character 1200 is making the action of eating food, or by the first sensor 2331 detecting a predetermined gesture.

[0258] This situation continues for a while, and the processing board 224 may gradually reduce the distance at which the observer's hand can approach the character 1200. When the character approaches a predetermined distance, the processing board 224 may display an image of the character 1200 grooming itself or relaxing in public, assuming that the character 1200 has become accustomed to the observer. Specifically, the processing board 224 may increase the threshold for the microwave intensity, or increase the threshold for the distance from the display device 1 when a gesture is made. In this way, the processing board 224 decreases the detection sensitivity, slowing down the reaction of the character 1200 to the observer's movements and preventing the character 1200 from making utterances such as "shrimp" or "shrimp shrimp."

[0259] Although the cat is always on guard, it will take on new actions such as grooming, lying down, scratching its claws, and wiping its face. As the cat gradually gets used to people and their hands, people will be able to touch the cat with their hands. Therefore, the processing board 224 may not reflect the microwave detection result by the human sensor 234 on the behavior of the character 1200. The processing board 224 may cause the first sensor 2331 to function as a motion sensor, and may display an image of the character 1200 feeling good when stroked according to the detection result of the observer's gestures in the up, down, left, right, front and back directions. The processing board 224 may further recognize the observer's voice using the microphone 2333, and display an image of the character 1200 responding to the voice. For example, the processing board 224 may be configured to emit the meow "nya" as the spoken content regardless of what words it recognizes.

[0260] The processing board 224 may further display an image of the character 1200 begging for food as time passes. The processing board 224 may recognize the presence of the observer by the image capturing unit 227 and may display an image of the character 1200 emitting a cry of "meow meow" and rubbing its face against the observer during a set time period (e.g., morning, afternoon, or evening) when the presence of the observer is detected by the human sensor 234. Furthermore, the processing board 224 may reproduce a state in which the character 1200 learns its own name as the observer calls out to the character 1200. For example, the processing board 224 may only respond to the word "Yuno" by emitting a cry of "meow," or may display an image of the character acting in a spoiled manner or rubbing its face against the observer in response to calls such as "Yuno" and "Here's food."

[0261] (9-2) Second step: learning the language (Japanese) When the processing board 224 recognizes the observer's call of "Yuno" at a certain time, it outputs a voice indicating a response of "What?". From this point on, the character 1200 will begin to learn words. The processing board 224 counts the number of times it is called out to, calculates the cumulative average per day, and responds according to a value calculated by multiplying the cumulative average of the number of times food is given per day by the cumulative average of the number of times it is called out to by a random coefficient (hereinafter referred to as the "cumulative average value"). This call is, for example, a predetermined call such as "Good morning," "Hello," "Good evening," "How are you?", or "Do you want food?". Depending on the cumulative average value, the processing board 224 may stop responding (i.e., reproduce the character 1200's forgetting of words) or increase the number of words it responds with. For example, the processing board 224 may reduce the number of words it responds with as the cumulative average value becomes smaller, and increase the number of words it responds with as the cumulative average value becomes larger. The cumulative average value is calculated based on the number of times the character 1200 is called out from the start of use of the display device 1 to the present time, but may also be calculated based on the number of times the character 1200 is called out from a predetermined period of time back in the past to the present time. The predetermined period may be a fixed period such as one month, or if the display device 1 has not been used for a certain period of time, it may be the period from the time when the character 1200 started to be used after that period. In the beginning of use of the display device 1, the cumulative average value may vary greatly, so that, for example, if the observer does not call out to the character 1200 even once a day, the character 1200 may not respond, but as the number of times the character 1200 is called out to increases, the cumulative average value does not change greatly. In this way, it is possible to reproduce the state in which it takes time for the character 1200 to forget words. The random coefficient used in calculating the cumulative average value is a value for reproducing the whimsical nature of a cat, and may be called a whimsical coefficient. The random coefficient may be varied depending on the actions of the character 1200 and the observer, such as giving or not giving food to the character 21200 when it begs for food, or waking up the character 1200 by talking to or touching it when it is sleeping.

[0262] It is preferable that the processing board 224 further makes the character 1200 talk to the observer or respond to questions about predetermined information such as the weather or the news when the cumulative average value exceeds a certain threshold value. Of course, if the observer reduces the number of times the character feeds, the cumulative average value decreases, so the processing board 224 reduces the types of responses or stops responding. In other words, if the observer is careless in feeding the character or does not call out to the character 1200 appropriately, the character 1200 may not maintain its response function, or even if it does, it may not respond due to its "whims." In this way, a higher quality communication function with the character 1200 can be provided to the observer.

[0263] [Other embodiments] The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit and scope of the present disclosure.

[0264] The display device 1 may have an operation receiving means for receiving an operation by the observer, such as a touch sensor, a proximity sensor, or other means, in the observation area SC. In this way, the observer can perform operations using the observation area SC. For example, if such an operation method is adopted in the communication function, it is expected to have the effect of improving the quality of communication with the character that can be observed through the observation area SC.

[0265] The first display unit 310 may display an image that is recognized as a three-dimensional image (stereoscopic image). In this way, it is expected that the effect of further improving the stereoscopic effect given to the observer can be achieved. As a display method, a naked-eye stereoscopic method such as a lenticular lens method or a parallax barrier method, or a display method that allows the image to be recognized as a three-dimensional image by using glasses, etc. may be used.

[0266] The display device is not limited to a rectangular parallelepiped shape, and may be a cube, a cylinder, other polyhedrons, a sphere, or other shapes. The shapes, structures, dimensions, colors, patterns, and the like of each component of the display device 1 described above are merely examples. Some of the components of the display device 1 described above may be omitted, or other components may be added. For example, the second display unit 330 may be omitted, or the second display unit 330 that emits the second image light L2 on the first direction side may be arranged in an upright state instead of the background plate 910. Aerial imaging may be realized using an optical system other than that described in FIG. 13. The configuration of the present invention may be applied to a display device of a display type or a projector type, rather than an aerial imaging type. In the case of a display type, a liquid crystal display, an organic EL display, or other display type display is used as the image display unit. For example, a display unit provided inside the image display space and having a display surface facing the direction of the observation area SC (for example, a display surface parallel to the observation area SC) may be used as the image display unit. This display unit may have the same configuration as the first display unit 310 or the second display unit 330, and may be provided on the rear side of the image display space. In the case of a projector type, a projection surface and a projection means for projecting an image onto the projection surface may be used. The light-reducing member 200 may be disposed on the front side or rear side of the opening 110 to reduce light entering the image display space from the outside. If the image display space S1 is to be maintained in a relatively dark state, a configuration without the light-reducing member 200 may be considered.

[0267] In the above embodiment, voice recognition for interacting with a character displayed on the video display unit is enabled in response to a pressing operation of the operation unit 2214, but the present invention is not limited to this. For example, when the presence of a person is detected in a detection target area of ​​a human presence sensor (one example of a detection means) such as a full-circle microwave Doppler sensor, the processing board 224 may enable the voice recognition. In this way, when the user interacts with a character displayed on the video display unit, for example, there is no need to perform an operation for enabling voice recognition or to utter a trigger word, and communication can be reproduced more naturally.

[0268] At this time, for example, it is preferable that both the voice recognition in the voice recognition application in the display device 1 and the voice recognition in the voice recognition server are enabled. However, as described below, the enabling of the voice recognition in the voice recognition application in the display device 1 and the enabling of the voice recognition in the voice recognition server may be executed in stages. For example, when the presence of a person is detected in a detection target area of ​​the human presence sensor or the like, the voice recognition in the voice recognition application in the display device 1 may be enabled, and thereafter, when the voice recognition application detects a call or greeting from an observer, the voice recognition in the voice recognition server may be enabled.

[0269] Furthermore, when the display device 1 is equipped with an imaging unit (an example of a detection means) such as a camera, a detection target space may be provided on the front side of the display device 1, and the processing board 224 may enable voice recognition when it detects the presence of a person on the front side of the display device 1. In this way, it is possible to enable voice recognition after more reliably determining that the observer is about to speak to the character displayed on the video display unit.

[0270] Furthermore, after a person is detected in the detection target area, the processing board 224 may be configured to activate the voice recognition on condition that the user authentication is successful. The authentication method for the user authentication may be, for example, an authentication method that does not require manual operation by the user (face authentication, vocal cord authentication, etc.). In this way, when user authentication is performed on a person other than a registered user registered in the display device 1 and the user authentication fails, the voice recognition is not activated and the user cannot interact with the character displayed on the video display unit. In other words, the character displayed on the video display unit does not react to people other than the registered user. This can increase the user's attachment to the character.

[0271] Furthermore, when a voice containing a phrase other than the phrases registered in advance in a dialogue server installed in the display device 1 or an external server is detected, the voice recognition may not be enabled.

[0272] Also, an odor sensor may be used as a means for activating the voice recognition. For example, the odor of a registered user of the display device 1 may be registered in advance in the display device 1, and when the odor of the registered user is detected by the odor sensor, the voice recognition may be activated.

[0273] Furthermore, even if voice recognition is enabled in this manner, it is preferable that the voice of a character (e.g., character 1100) displayed on the video display unit is not output immediately, but that the voice of the character is not output unless the user speaks to it. For example, the processing board 224 does not immediately output the character's voice saying "Welcome back" in response to voice recognition being enabled, but rather outputs the character's voice saying "Welcome back" on the condition that the user's voice saying "I'm home" is detected while voice recognition is enabled. Furthermore, for example, in the dialogue system, a branching scenario is created to reproduce the following dialogue between an observer and a character (e.g., character 1100). For example, on a weekday, User: Right now Character: Welcome home~ User:Achi Character: I'm tired in this heat. The following dialogue is reproduced, and if it is a holiday, User: Right now Character: Ah, welcome back User:Achi Character: I went out of my way in the heat so I won't complain User: Work, work. Character: Sorry if that's true, thank you for your hard work. It is advisable to create a branching scenario that reproduces such a dialogue.

[0274] In addition, a branching scenario may be created according to the detection result by the odor sensor. For example, User: The meal was delicious. Character: "Well, let me smell it. Sniff sniff. It smells good!" or "Hmmmm? Isn't that a lot of salt? Take care of yourself!" A dialogue such as the following may be reproduced.

[0275] As another example, when an odor is detected by an odor sensor, Character: Sniff sniff, what's that smell? User: I lit some aroma. Character: Ah, that's why I thought it would be relaxing. In addition, if the reply of the observer to the question about the type of odor is not registered in the branching scenario, a voice such as "Oh, is that so!" may be uniformly output as the character's voice.

[0276] Also, the processing board 224 may constantly record the voice for a recent predetermined time (for example, about 15 seconds), and when the voice of a trigger word for activating the voice recognition is detected, the voice recognition may be activated. For example, the voice recognition application of the display device 1 may be constantly activated (or, in response to a pressing operation of the operation unit 41, etc.), and the voice recognition by the voice recognition server may be activated by the detection of a trigger word. At this time, the processing board 224 may transmit to the voice recognition server voice data of the voice detected by the microphone 2333 (an example of a voice detection means) (for example, a voice detected between the start and end of detection) excluding the voice of the trigger word. For example, it is assumed that a user voice saying "I'm going to Tokyo tomorrow, so can you tell me the weather, Rei-tan?" is detected. In this case, "Rei-tan", which is the nickname of the character 1100, is the trigger word. In this case, the processing board 224 transmits the detected voice "I'm going to Tokyo tomorrow, so tell me the weather, Rei-tan" minus the trigger word voice "Rei-tan" to the voice recognition server.

[0277] Here, for example, if the detected voice as described above, "I'm going to Tokyo tomorrow, so can you tell me the weather, Rei-tan?", is sent as is to the voice recognition server, the voice of the character corresponding to the voice recognition result may not be accurately extracted in the dialogue server due to the inclusion of "Rei-tan" in the voice recognition result, and the dialogue with the user may not be able to proceed smoothly. In contrast, if the above-described method is adopted, even if the trigger word is uttered in the middle of a sentence, the voice data of the voice excluding the trigger word is sent to the voice recognition server. Therefore, it is possible to prevent the dialogue with the user from becoming disjointed due to the voice recognition being performed with the trigger word included in the middle of a sentence.

[0278] In addition, when realizing a dialogue between a user and a character displayed on the video display unit through cooperation between a voice recognition server and a dialogue server, the following operations may be executed depending on the configuration of the voice recognition server.

[0279] For example, if the voice recognition server is configured to start voice recognition processing after completing reception of the voice data file, the processing board 224 may individually transmit to the voice recognition server each voice data file of the voice before and after the voice of the trigger word among the voices detected by the microphone 2333. For example, among the voices detected between the detection start time and the detection end time, the voice data file of the user voice detected before the voice of the trigger word (also referred to as the user voice of the first half) may be transmitted to the voice recognition server prior to the voice data file of the user voice detected after the voice of the trigger word (also referred to as the user voice of the second half).

[0280] In this way, the voice recognition server starts the voice recognition process for the first half of the user voice without waiting for the completion of reception of the voice data file of the second half of the user voice. Therefore, the voice recognition process can be performed more efficiently than when a voice data file of a voice that combines the first half and second half of the user voice is transmitted to the voice recognition server. As a result, the display device 1 can receive the character voice from the dialogue server earlier, and the user's waiting time until a response to the user's voice can be shortened.

[0281] Also, for example, when the voice recognition server is configured to perform voice recognition processing in real time by playing back the voice data from the display device 1 in a stream, it is preferable that the playback speed T1 of the user voice in the first half is controlled to be faster than the standard playback speed T0 (T1>T0) when the voice data of the first half is transmitted to the voice recognition server in real time. For example, the processing board 224 changes the setting so that the user voice in the first half is played back 25% faster than the standard playback speed T0 in the voice recognition server, and transmits the voice data of the first half. In this way, the voice recognition processing of the user voice in the first half can be completed earlier and the voice recognition processing of the user voice in the second half can be started earlier in the voice recognition server, compared to the case where the user voice in the first half is played back at the standard playback speed T0. As a result, the display device 1 can receive the character voice from the dialogue server earlier, and the waiting time of the user until a response to the user voice can be shortened.

[0282] Furthermore, when the voice data of the latter half of the user voice is transmitted to the voice recognition server in real time, it is preferably controlled such that the playback speed T2 of the voice in the latter half is faster than the standard playback speed T0 and slower than the playback speed T1 of the user voice in the first half (that is, the relationship of T0 < T2 < T1 is satisfied). For example, the processing board 224 changes the setting so that the user voice in the latter half is played back 10% faster than the standard playback speed T0 in the voice recognition server and transmits the voice data of the latter half. In this way, both the user voice in the first half and the user voice in the latter half are streamed and played back faster than the standard playback speed T0, and voice recognition processing is executed in the voice recognition server. As a result, the waiting time of the user until a response is made to the user's voice can be further shortened. On the other hand, the user voice in the latter half is played back at a speed that is faster than the standard playback speed T0 but slower than the playback speed T1 of the user voice in the first half. Therefore, a decrease in voice recognition accuracy in the voice recognition system can be suppressed. Therefore, it is possible to further shorten the waiting time of the user while suppressing a decrease in voice recognition accuracy.

[0283] In addition, when voice recognition processing by streaming playback of voice data is executed in real time in the voice recognition server, for example, although the user voice detected by the microphone 2333 is transmitted to the voice recognition server in real time, it is preferably controlled such that voice recognition in the voice recognition server is not started until the voice of the trigger word is detected.

[0284] <10. Details of the communication function related to character 1200> Next, the details of the communication function related to the above-described character 1200 will be described below. In the following, in particular, the stages up to taming in the above-described first step and the stage of learning the language in the second step will be described. In addition, the function of the character 1200 that has learned the language in the second step to transmit or respond to predetermined information such as weather and news will also be described in detail.

[0285] In addition, since the communication function, for example, has been improved compared to the above-mentioned display device 1 and therefore goes beyond the functional category of a "display device," hereinafter, the system will be referred to as system 1 instead of the above-mentioned display device 1. However, since the external configuration, display principle, internal configuration (structure), etc. of system 1 are similar to those of the above-mentioned display device 1, explanations of these will be omitted.

[0286] In addition, in the explanation of the display device 1, the person observing the image displayed on the display device 1 is referred to as an "observer." However, as mentioned above, the system 1 goes beyond the scope of a display device, and therefore, hereinafter, the term "user" will be used instead of "observer."

[0287] (System 1 Concepts) First, the concept of the system 1 will be briefly described with reference to Fig. 56. Fig. 56 is a diagram briefly explaining an example of the development concept of the system 1 according to an embodiment.

[0288] As with the above-described display device 1, characters such as animals are virtually displayed in the video display space S1 of the system 1. In this embodiment, a cat is displayed as an example of a character virtually displayed in the video display space S1 of the system 1. As described above, the name of this cat is Juno (also called JUNO).

[0289] As shown in FIG. 56, JUNO is a character who wants to become friendly with, for example, the user who is the owner of the system 1, but is a little nervous at first and cannot communicate well, and his behavior is realized by software control. For example, JUNO is virtually and pseudo-reproduced as having instincts such as nature and characteristics that are said to be innate, like a real life form such as an animal. By such software control, JUNO has an animal-like nature and performs animal-specific behaviors due to changes in the environment, etc. In the following description, when it is expressed that JUNO has a virtual instinct function, it means that JUNO is realized by software control so that it behaves as if it has an instinct. The same is to be understood for various other actions performed by JUNO (for example, speech and other behaviors). The software control may or may not use artificial intelligence technology, and the specific realization method is not important.

[0290] For example, in the initial setting, JUNO is set to be highly wary (i.e., low intimacy with the user) and unable or difficult to communicate with the user, and in some cases, JUNO may behave threateningly toward the user. Then, when the user takes an action such as feeding JUNO every day, JUNO's wariness may begin to ease, and the intimacy between the user and JUNO may increase. As the intimacy between the user and JUNO increases, JUNO's behavioral patterns may change, and JUNO may behave in a way that surprises the user. As the intimacy between the user and JUNO increases, it may become possible for JUNO to have everyday conversations with the user, for example, and the user may be able to spend a relaxing and healing time with JUNO.

[0291] In this way, for example, by setting the initial setting to make the character more wary, unable to communicate well with the user, or threatening the user, it is possible to realistically express the instinctive functions of a living thing. Then, as the intimacy between the user and JUNO increases, it becomes possible to communicate with JUNO as if it had a virtual instinctive function. JUNO is a character imitating a living thing (for example, a living organism such as an animal, and in the embodiment of this system, a cat). In this way, it is possible to realize enjoying daily life with a character 1200 (virtual animal) that is as realistic as if it actually exists. Note that the character 1200 can be, for example, a human, a non-human animal, a fantasy creature, a non-existent character, etc., as long as it can be expressed as realistic as if it actually exists.

[0292] (Character behavior patterns change depending on intimacy) First, an example of JUNO's behavioral patterns according to the intimacy between the user and JUNO will be briefly described. The intimacy between the user and JUNO can be expressed, for example, by the sense of distance between the user and JUNO. If the intimacy between the user and JUNO is low, JUNO will not approach the user as a virtual instinct function, and will likely keep a certain distance between itself and the user. On the other hand, if the intimacy between the user and JUNO increases, JUNO will become less wary of the user and may approach the user. Therefore, in the present system 1, JUNO's behavioral patterns are configured to be behavioral patterns in which the distance between the user and JUNO can change depending on the intimacy between them.

[0293] The distance between the user and JUNO can be changed, for example, by changing the display position of JUNO in the stereoscopic image display space S1. The display position of JUNO in the image display space S1 will be described below with reference to FIG. 57. FIG. 57 is a diagram showing an example of a display form in the image display space S1 of the system 1 according to an embodiment. If JUNO can move freely in the stereoscopic image display space S1, it is possible to create a sense of presence as if the virtually displayed JUNO actually exists in reality.

[0294] As shown in FIG. 57, in the image display space S1 of the system 1, a plurality of areas are displayed, for example, in the depth direction, as a three-dimensional virtual space image. The plurality of areas are displayed so that the sense of distance in a front view differs depending on the area. In this embodiment, for example, as a plurality of areas capable of displaying JUNO, the area is divided into area 1 on the near side, which is the side closest to the user, area 5 on the far side, which is the side furthest from the user, and areas 2 to 4 which are intermediate between area 1 and area 5. Among areas 2 to 4, the order of areas closest to the user is area 2, area 3, and area 4. In addition, it is also possible to display area 0 closer to the user than area 1, as an area that is not displayed normally but is used, for example, when performing a zoom display, which will be described later. However, in this embodiment, area 0 is not an area capable of displaying JUNO, but JUNO may be displayed in area 0.

[0295] Among the multiple areas displayed in the depth direction of the video display space S1 of the system 1, in area 1 which is the side closest to the user (the lower side of the display screen), an image of, for example, a character is displayed largest, and in area 5 which is the side farthest from the user (the upper side of the display screen), an image of, for example, a character is displayed smallest. Then, the images of the character are displayed so as to become gradually smaller from area 1 to area 5. In this way, it is possible to further emphasize the sense of depth.

[0296] The processing board 224 is configured to be able to control JUNO to move among areas 1 to 5 in the image display space S1 of the system 1 according to the intimacy between the user and JUNO. In this embodiment, the intimacy is abstract and conceptual, and an experience value is used as a parameter that directly or indirectly specifies the intimacy. The intimacy may be used as a parameter instead of the experience value. The experience value is an example of experience information of the present invention. The intimacy is an index of the intimacy between the user and JUNO, and the larger the experience value, the higher the intimacy, and the smaller the experience value, the lower the intimacy. The experience value is an index value of how much JUNO has experienced the action (behavior) performed by the user on JUNO, and changes, for example, according to the performance of the action. The performance of the action is specified, for example, by the number of times or frequency of the action, but it is even better if the specific content of the action is also taken into account. Typically, the more actions JUNO experiences from the user on JUNO, the higher the experience value becomes, and the intimacy changes accordingly in a higher direction. In general human relationships and between humans and animals, the more active the communication, the closer the relationship between the two will be. This can be said to be a pseudo-representation of this. The experience value may not only change in an upward direction, but also in a downward direction, in which case the intimacy level will change in a downward direction. For example, the less JUNO experiences the action from the user toward JUNO, for example, the fewer the number or frequency of the action, the lower the intimacy level. The specific content of the action may be taken into account, and for example, even if the user takes an action toward JUNO, the intimacy level may be lowered if the action is not one that JUNO likes. In this case, it can be understood that there is a lack of experience of the action that JUNO likes, and the intimacy level will be lowered. In general human relationships and between humans and animals, the less good communication is experienced, the lower the intimacy level between the two will be. This can be said to be a pseudo-representation of this. In this embodiment, the action toward JUNO is realized by the user inputting information to the system 1.Next, the relationship between the user's action and the experience value or the degree of intimacy grasped therefrom, and the operation of the system 1 related thereto will be described.

[0297] For example, if the intimacy between the user and JUNO is low, the ratio (i.e. probability) of JUNO moving to an area where the distance to the user is relatively far is high, and as the intimacy between the user and JUNO increases, the probability of JUNO moving to an area where the distance to the user is relatively close increases. By doing this, JUNO can be displayed as if it is actively moving away from or towards the user based on the virtual instinct function, making JUNO even more realistic.

[0298] (About intimacy) The processing board 224 adds a predetermined value to the experience value when, for example, JUNO has a specific experience, etc., as a preset condition. When this experience value increases, the processing board 224 controls JUNO to perform a behavior pattern as if the intimacy between the user and JUNO, which is an abstract concept, has increased. In this embodiment, for example, the number of times (number of days) that the user has fed JUNO is used as a parameter of the experience value. When the number of times (number of days) that the user has fed JUNO is used as a parameter of the experience value, the experience value is added by 1 when the user feeds JUNO. In this embodiment, the upper limit of the experience value added per day is 1. In other words, even if the user feeds JUNO multiple times in a day, the experience value added is only 1. In this way, it is possible to have the user use the system 1 for a long period of time. However, the upper limit of the experience value added by the user feeding JUNO is not limited to 1 as long as it is within a range of values ​​that allow the user to use the system 1 for a long period of time, and may be, for example, 2 or 3, or the experience value may be added by the number of times the user has fed JUNO. Also, if the number of days that the user does not feed JUNO exceeds a preset number of days, the experience value is subtracted by a predetermined value (for example, 1). The process of adding and subtracting the experience value is performed by a cloud server described later, but may be performed by the processing board 224. The method of feeding JUNO by the user will be described later. Note that food is an example of an item given to JUNO, and may be replaced with or combined with a ball, a toy, or other items. Also, using the number of times (number of days) that the user fed JUNO as a parameter of the experience value is one example, and the number of times the user took action, such as the time the user used the system, the number of times it was used, the number of times he / she called out to JUNO, etc., may be used as a parameter of the experience value, or these may be combined. Also, if an unfriendly parameter, for example, is used as a parameter of the experience value, the intimacy level decreases when the unfriendly parameter is added.Furthermore, while the experience value is an example of experience information of the present invention as described above, it can be a parameter related to the usage record of the communication function between the user and JUNO, as a higher level concept than experience information.

[0299] In consideration of the fact that there are users who wish to increase experience points in a short period of time depending on their preferences, the upper limit of experience points that are added by users feeding JUNO may be set to a relatively large value, such as 5 or 10, or there may be no upper limit and the experience points may increase according to the number of times food is given. Furthermore, if the user is allowed to set the upper limit of experience points that are added by feeding JUNO, it is possible to provide a system 1 that allows the user to set the speed at which experience points increase according to the user's preferences.

[0300] In this embodiment, for example, when the number of times (number of days) that the user has fed JUNO is between 0 and less than 30, the level of intimacy is defined as low, when the number of times (number of days) that the user has fed JUNO is between 30 and less than 60, the level of intimacy is defined as medium, and when the number of times (number of days) that the user has fed JUNO is 60 or more, the level of intimacy is defined as high.

[0301] (Character area movement) As described above, if the intimacy between the user and JUNO is low, there is a high probability that JUNO will move to an area that is relatively far from the user, and as the intimacy between the user and JUNO increases, there is a high probability that JUNO will move to an area that is relatively close to the user. Therefore, the probability that JUNO will move to each area depending on the intimacy will be explained with reference to Figures 58A to 62.

[0302] FIG. 58A is a table showing an example of a formula for calculating the probability that JUNO (character) moves to areas 1 and 2 depending on the number of times (number of days) that a user feeds JUNO in a system 1 according to an embodiment. FIG. 58B is a table showing an example of a formula for calculating the probability that JUNO moves to areas 3 to 5 depending on the number of times (number of days) that a user feeds JUNO in a system 1 according to an embodiment. FIG. 59 is a table showing a formula for calculating the probability that JUNO moves to each area (areas 1 to 5) depending on the number of times (number of days) that a user feeds JUNO in a system 1 according to an embodiment. The formulas shown in FIG. 58A, FIG. 58B, and FIG. 59 are tables showing the same basic formula in different ways. Specifically, FIG. 58A and FIG. 58B are tables showing the basic formula for calculating the probability of moving to each area and the formula for calculating the coefficients a, b, and c included in the basic formula for each degree of intimacy. On the other hand, FIG. 59 is a table showing a basic formula for determining the probability of moving to each area, and a formula obtained by substituting the formula for determining the coefficients a, b, and c for each degree of intimacy into the basic formula.

[0303] Fig. 60(A) is a table showing an example of values ​​substituted for coefficients X (X1, X2, X3) included in the formula for calculating coefficients a, b, c in the system 1 according to an embodiment, and used to classify the intimacy level into low, medium, and high. In this embodiment, as described above, the intimacy level is defined as low when the number of times (number of days) the user has fed JUNO is 0 or more and less than 30, the intimacy level is medium when the number of times (number of days) the user has fed JUNO is 30 or more and less than 60, and the intimacy level is high when the number of times (number of days) the user has fed JUNO is 60 or more. Therefore, as shown in Fig. 60(A), in any of the cases of low, medium, and high intimacy, the upper limit of the number of times (number of days) the user has fed JUNO is set to, for example, 30 days. Therefore, for example, if the number of times (days) the user has fed JUNO is 0 or more and less than 45 days, the number of times (days) the user has fed JUNO is 45 or more and less than 90 days, the number of times (days) the user has fed JUNO is 45 or more and less than 90 days, and the number of times (days) the user has fed JUNO is 90 days or more, then X1=45, X2=45, and X3=45 as shown in Fig. 60(A). Note that, for the sake of convenience in calculation, in the case of high intimacy, the upper limit of the number of times (days) the user has fed JUNO is set to, for example, 30 days, but even if the number of times (days) the user has fed JUNO exceeds 30 days, the probability of movement to each area is determined using the same formula as in the case of high intimacy.

[0304] 60(B) is an example of a table in which upper and lower limits are set for the probability of moving to each area at each level of intimacy (low intimacy, medium intimacy, high intimacy) in the system 1 according to an embodiment. For example, at medium intimacy, the upper limit of the probability of moving to area 1 is, for example, 0.3 (30%) and the lower limit is, for example, 0.05 (5%).

[0305] 61 is a table showing coefficients included in a basic formula showing the probability of moving to area 1 according to intimacy (low intimacy, medium intimacy, high intimacy) in system 1 according to one embodiment, and shows (A) an example of a table showing coefficient a, (B) an example of a table showing coefficient b, and (C) an example of a table showing coefficient c. For example, as shown in FIG. 61(A), the value of coefficient a in the case of high intimacy is set to 10.9091.

[0306] FIG. 62 shows tables indicating coefficients included in a basic formula indicating the probability of moving to areas 2 to 4 according to intimacy (low intimacy, medium intimacy, high intimacy) in system 1 according to one embodiment, where (A) is an example of a table indicating coefficient A, and (B) is an example of a table indicating coefficient B. For example, as shown in FIG. 62(A), the value of coefficient A included in the basic formula indicating the probability of moving to area 3 in the case of medium intimacy is set to 0.20. Also, as shown in FIG. 62(B), the value of coefficient B included in the basic formula indicating the probability of moving to area 2 in the case of high intimacy is set to 0.05.

[0307] In this way, as the number of times (number of days) that the user feeds Juno increases, the probability that Juno will move to Area 1 (including the probability of staying in Area 1) increases, and Juno will be displayed as approaching the user. However, if the user behaves in a way that Juno does not like, it is also possible to increase the probability that Juno will move to Area 5, for example, or decrease the probability that Juno will move to Area 1, for example (including the probability of staying in Area 1).

[0308] Next, the concept of the probability of moving to Area 1 will be explained. For example, when keeping an animal that is not used to humans at all, such as a stray cat, it is thought that the first step is to get the animal used to humans. Even if an animal starts to receive food, it is thought that it probably does not get used to humans linearly, but gets used to humans little by little. Therefore, in this embodiment, in the case of low intimacy, the rate of change of JUNO as it gets used to humans is not constant, and the more time passes, the higher the probability of moving to Area 1 becomes. That is, in the case where the number of times (number of days) that a user feeds JUNO is used as a parameter of experience value, for example, in the case of low intimacy where the number of times (number of days) that JUNO has been fed is less than 30 days, the amount of change in the probability of moving to Area 1 relative to the increase in the number of times (number of days) that JUNO has been fed becomes larger as the number of times (number of days) that JUNO has been fed increases. In this way, not only can the user feel a sense of closeness to JUNO, but the user can embody the process of JUNO's accustoming to humans accelerating through the rate of JUNO moving to Area 1. The formula for low intimacy shown in the Area 1 column of Fig. 59 (hereinafter referred to as "first formula") shows that the amount of change in the probability of moving to Area 1 with respect to the increase in the number of times (number of days) of feeding increases as the number of times (number of days) of feeding increases. Note that the amount of change in the probability of moving to Area 1 with respect to the increase in the number of times (number of days) of feeding increases as the number of times (number of days) of feeding increases, but this is only one example, and if an aloofness parameter is used as the parameter of experience value, the amount of change in the probability of moving to Area 1 with respect to the decrease in the value of experience value can also be increased.

[0309] However, for example, animals do not get used to something forever, but rather hit a plateau at some point and the rate of change as they get used to it is thought to approach zero (approaching the asymptote of y=1). In other words, at a high level of intimacy, where the number of times (number of days) the user has fed JUNO is, for example, 60 days or more, the amount of change in the probability of moving to Area 1 relative to the increase in the number of times (number of days) the user has fed JUNO becomes smaller as the number of times (number of days) the user has fed JUNO increases. The formula that expresses this is the formula for high intimacy shown in the Area 1 column in Figure 58A (hereinafter, this formula will be referred to as the "third formula"):

[0310] By the way, for example, when animals start to get used to something, it is thought that the change due to the change becomes larger, but at some point the rate of change reaches a maximum, after which the rate of change becomes smaller, leading to the third formula. In other words, it is necessary to smoothly connect the first formula and the third formula.

[0311] In this embodiment, as shown in Figures 58A, 59B, and 59, the probability of moving to area 1 at medium intimacy is the same as the first formula indicating the probability of moving to area 1 at low intimacy. However, in the following, for convenience of explanation, the formula indicating the probability of moving to area 1 at medium intimacy is referred to as the second formula.

[0312] In this embodiment, we will explain how to easily, that is, without putting a load on the software, use the above first, second, and third formulas to vary the probability that JUNO will move to Area 1 depending on the intimacy level. In terms of not putting a load on the software, it is preferable not to use trigonometric functions, exponential functions, and logarithmic functions, and it is preferable to use only the four arithmetic operations. Furthermore, instead of changing the formulas for the above first, second, and third formulas for the three cases, they are expressed as one formula, and the value of the differential coefficient is made equal at "x = any specified point", so that each formula can be smoothly connected. In this way, it is possible to simplify the program. If the coefficients are determined by the number of times (number of days) that the user has fed JUNO in each of the three cases of low intimacy, medium intimacy, and high intimacy, all the coefficients can be determined by simply inputting three parameters, for example, 30 / 30 / 30, such as 30 days / 30 days / 30 days. This is to make debugging easier.

[0313] Next, the probability that JUNO will move to each area according to the intimacy level, which is calculated by the four arithmetic operations described with reference to Figs. 58A to 62, is graphed and tabulated as shown in Figs. 63 and 64. Fig. 63 is a graph showing the relationship between the passage of time, and thus the number of times (number of days) that a user has fed JUNO, and the probability of movement to each area, in a system 1 according to an embodiment. Fig. 64 is an example of a table showing the data of the graph shown in Fig. 63, that is, the number of times (number of days) that a user has fed JUNO and the data of the probability of movement to each area, for each level of intimacy (low intimacy, medium intimacy, high intimacy) in a system 1 according to an embodiment. Note that 0 to 30 shown in the vertical column of "number of days" for each level of intimacy in Fig. 64 is not the total number of days that a user has fed JUNO, but the number of times (number of days) that a user has fed JUNO at each level of intimacy. For example, if the "number of days" in the intimacy level is "20", the number of times (number of days) the user has fed Juno during the intimacy level is 20 days, and the total number of times (total number of days) the user has fed Juno is 50 days. Below, the amount of change in the probability of movement to each area depending on the number of times (number of days) the user has fed Juno will be explained with reference to Figures 63 and 64.

[0314] (Change in probability of moving to each area when intimacy is low) As shown in FIG. 63 and FIG. 64, in the low intimacy level where the number of times (number of days) the user has fed JUNO is 0 to 30 days, the change in the probability of moving to areas 1 to 4 relative to the number of times (number of days) the user has fed JUNO increases in the positive direction as the number of times (number of days) the user has fed JUNO increases (hereinafter, the change in the positive direction is referred to as the "positive change"). However, the positive change in the probability of moving to areas 1 to 4 relative to the number of times (number of days) the user has fed JUNO is largest in the order of area 4, area 3, area 2, and area 1 among areas 1 to 4. In other words, among the positive change in the probability of moving to areas 1 to 4 relative to the number of times (number of days) the user has fed JUNO, the positive change in the probability of moving to area 1 is the smallest. On the other hand, the change in the probability of moving to area 5 relative to the number of times (number of days) the user has fed JUNO increases in the negative direction as the number of times (number of days) the user has fed JUNO increases (hereinafter, the change in the positive direction is referred to as the "negative change"). In this way, at low intimacy, as the number of times (number of days) the user feeds JUNO increases, the positive change increases in the order of area 4, area 3, area 2, and area 1, while the negative change increases in the probability of moving to area 5. The probability of moving to each area at low intimacy is highest in the order of area 5, followed by area 4, area 3, area 2, and area 1. By doing this, the position at which JUNO is displayed allows the user to embody the fact that JUNO is not used to the user in the initial state and is not attached to the user. Then, the character is not used to the user for a while from the initial state, and as the number of times (number of days) the user feeds JUNO increases, the user can embody the process of the character becoming less wary and becoming used to the user through the ratio, i.e., the proportion, of the positions at which the character stays.

[0315] (Change in probability of moving to each area when intimacy is high) Next, when the number of times (number of days) the user has fed JUNO is between 30 and 60 days, the change in the probability of moving to areas 1 to 3 relative to the number of times (number of days) the user has fed JUNO increases in the positive direction as the number of times (number of days) the user has fed JUNO increases. However, unlike the case of low intimacy, the positive change in the probability of moving to areas 1 to 3 relative to the number of times (number of days) the user has fed JUNO is larger in the order of area 1, area 2, and area 3 among areas 1 to 3. In other words, among the positive changes in the probability of moving to areas 1 to 3 relative to the number of times (number of days) the user has fed JUNO, the positive change in the probability of moving to area 1 is the largest. Also, the change in the probability of moving to area 4 relative to the number of times (number of days) the user has fed JUNO is 0. Furthermore, the change in the probability of moving to area 5 relative to the number of times (number of days) the user has fed JUNO is still larger in the negative direction. The probability of moving to each area during the intimacy level is highest in the following order from the first half to the middle of the intimacy level: the probability of moving to area 5, the probability of moving to area 4, the probability of moving to area 3, the probability of moving to area 2, and the probability of moving to area 1, and highest in the second half of the intimacy level: the probability of moving to area 1, the probability of moving to area 4, the probability of moving to area 3, the probability of moving to area 2, and the probability of moving to area 5. As the number of times (number of days) that the user feeds JUNO increases, the difference between the probability of moving to area 2, the probability of moving to area 3, and the probability of moving to area 4 becomes smaller, and at the boundary between the medium intimacy level and the high intimacy level (for example, the number of days that the user has fed JUNO is 60 days), the probability of moving to area 2, the probability of moving to area 3, and the probability of moving to area 4 are the same or almost the same (for example, 0.20 in this embodiment).

[0316] (Change in probability of moving to each area when intimacy is high) Next, at a high level of intimacy where the number of times (number of days) the user has fed JUNO is 60 days or more, the change in the probability of moving to areas 1 and 2 relative to the number of times (number of days) the user has fed JUNO increases in the positive direction as the number of times (number of days) the user has fed JUNO increases. However, the positive change in the probability of moving to areas 1 and 2 relative to the number of times (number of days) the user has fed JUNO decreases as the number of times (number of days) the user has fed JUNO increases. Also, the change in the probability of moving to areas 3 to 5 relative to the number of times (number of days) the user has fed JUNO increases in the negative direction as the number of times (number of days) the user has fed JUNO increases, and the negative change in the probability of moving to areas 3 to 5 relative to the number of times (number of days) the user has fed JUNO decreases as the number of times (number of days) the user has fed JUNO increases. In other words, at a high level of intimacy, the rate of change in the probability of moving to any of areas 1 to 5 slows down as the number of times (number of days) the user has fed JUNO increases. In particular, the graph showing the probability of moving to Area 5 approaches the asymptote of y=1. By doing this, Juno's movements can be made more realistic. The probability of moving to each area at high intimacy is, in descending order, Area 1, Area 2, Area 3, Area 4, and Area 5.

[0317] In this way, by changing the amount of change in the probability of moving to each of areas 1 to 5 depending on the number of times (number of days) the user has fed JUNO, it is possible to control JUNO's behavioral patterns to vary depending on the level of intimacy: low, medium, and high. In this way, it is possible to form a virtual instinct function in which JUNO as character 1200 acts as if it were acting instinctively.

[0318] <11. Functions of this system> Next, the functions (eg, state, behavior pattern, etc.) of the character 1200 (in this embodiment, JUNO modeled after a cat) according to the intimacy level (low intimacy level, medium intimacy level, high intimacy level) will be described.

[0319] In the system 1 according to the present embodiment, JUNO as the character 1200 is displayed in a behavior pattern as if he has feelings according to the intimacy level described above. For example, at the initial setting, JUNO stays in the position farthest from the user (for example, area 5). Also, when the intimacy level is low, including at the initial setting, the character 1200 is wary (hereinafter referred to as "virtual wariness"). Therefore, at the low intimacy level, as described above, the processing board 224 controls the behavior pattern so that JUNO does not approach the user (for example, does not approach area 1) as if he does not trust the user, and JUNO often stays in, for example, area 5 away from the user. Also, the processing board 224 controls the behavior pattern so that, for example, JUNO recognizes area 5 as its territory is displayed. Also, even when JUNO is displayed in areas 1 to 4, the processing board 224 controls the behavior pattern so that JUNO has a virtual wariness is displayed.

[0320] (11-1. Low intimacy level) Fig. 65A is an example of a diagram for explaining JUNO's state, behavior pattern, etc. when the intimacy level is low in the system 1 according to an embodiment. Also, Fig. 65B is an example of an image displayed in the video display space S1 of the system 1 when the intimacy level is low in the system 1 according to an embodiment. Note that hereinafter in this specification, the expression of the character 1200 will be unified into the description of JUNO.

[0321] 65A, a fixed location is set as JUNO's course of action. When the intimacy level is low, JUNO's fixed location is, for example, area 5, which is the farthest from the user, and JUNO is mainly in area 5. Furthermore, JUNO is in area 5 regardless of whether or not a person is detected, which will be described later. JUNO is mainly located in area 5, and is on guard when moving to or in other areas.

[0322] As shown in FIG. 65B, a food bowl is displayed in area 1. A claw-sharpening board is displayed in area 3. Furthermore, JUNO is displayed in area 5. Area 5 is an area that is not subject to external influences. External influences include human detection, which will be described later. The food bowl and claw-sharpening board will also be described later.

[0323] Fig. 66A is an example of an explanatory diagram for moving to another area when the intimacy level is low in the system 1 according to an embodiment. Fig. 66B is an example of an image for explaining moving to another area when the intimacy level is low in the system 1 according to an embodiment.

[0324] The processing board 224, for example, performs a movement lottery at a predetermined cycle, and JUNO may move to another area based on the result of this movement lottery, or JUNO may move to another area based on event detection.

[0325] When JUNO moves to another area based on the result of the movement lottery performed at a predetermined cycle as shown in FIG. 66A and FIG. 66B, the movement lottery performed at the predetermined cycle is performed, for example, at the movement probability shown in FIG. 63 and FIG. 64 described above. The above-mentioned predetermined cycle is performed, for example, at a cycle of about 1 to 10 minutes. Also, when JUNO who won the movement lottery moves to another area, the processing board 224 determines the cycle for performing the movement lottery after the movement by lottery. In this case, the area to which JUNO can move is, for example, any of areas 1 to 5. In this way, JUNO can be moved to an area based on the result of the movement lottery performed at a predetermined cycle or event detection. In other words, JUNO can be moved to an area at random timing, and JUNO can be displayed as if he is moving of his own volition.

[0326] 66B, when intimacy is low, the probability that JUNO will move to Area 1 based on the result of the movement lottery is 0 to 5%. Similarly, the probability that JUNO will move to Area 2 is, for example, 0 to 10%, the probability that JUNO will move to Area 3 is, for example, 0 to 15%, the probability that JUNO will move to Area 4 is, for example, 0 to 20%, and the probability that JUNO will move to Area 5 is, for example, 100 to 50%.

[0327] Next, movement to another area based on event detection will be described with reference to Figures 67A to 69B. Event detection includes, for example, feeding event detection, randomly occurring claw scratching event detection, human detection event detection, and the like.

[0328] (If a feeding event is detected) Fig. 67A is an example of an explanatory diagram of movement to another area when a feeding event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 67B is an example image showing movement from area 5 to area 1 when a feeding event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 67C is an example image showing movement from area 1 or 2 to area 5 when a feeding event is detected when the intimacy level is low in a system 1 according to an embodiment.

[0329] In Fig. 67A, for example, the conditions for starting movement, the conditions for escaping to Area 5, Juno's behavior after escaping to Area 5, Juno's behavior after finishing eating the food, etc. are described. In particular, when Juno escapes, by having it escape from Area 1, which is closest to the user, to Area 5, which is farthest from the user, it becomes possible to clearly grasp that Juno has moved away from the user.

[0330] When the user feeds (i.e., when food is generated), a feeding event is detected. When a feeding event is detected, the processing board 224 calculates the probability that JUNO will eat the food. Although FIG. 67A states that "the eating probability is calculated from the intimacy sheet," the intimacy sheet is, for example, the probability of moving to area 1 shown in FIG. 64. For example, if the number of times (number of days) that the user has fed JUNO is the 50th day, the processing board 224 determines whether or not to establish the movement start condition with a probability of 0.1500 (15%), which is the probability listed in the "20" column in the intimacy corresponding to the 50th day in the table shown in FIG. 64.

[0331] When the movement start condition is met, the processing board 224 decides that JUNO will start to take action to eat food. Then, when it is decided that JUNO will start to take action to eat food, the processing board 224 displays a behavior pattern in which JUNO starts to move vigilantly in order to eat food, and for example, as shown in Fig. 67B, JUNO moves from Area 5 to Area 1 where a food bowl is placed.

[0332] A condition for JUNO to run back to Area 5 from an area other than Area 5 (for example, Area 1 or Area 2) may be met, for example, when an external detection (for example, a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or an external detection such as a loud sound of a predetermined decibel or more is detected) is detected while moving to Area 1 or while JUNO is eating food. For example, when the above-mentioned proximity detection is detected while moving to Area 1 or while JUNO is eating food, the processing board 224 displays a behavior pattern in which JUNO moves to run back from an area other than Area 5 to, for example, Area 5, as shown in FIG. 67C. In this way, when a proximity detection is detected, JUNO moves to run back to an area relatively far from the user, such as Area 5, and thus JUNO's movement can be made more realistic.

[0333] After JUNO escapes to Area 5, the processing board 224 plays a fear motion that shows that JUNO is afraid, and then displays a behavior pattern as if it is wary of its surroundings, while starting a timer. After completing its movement to Area 5, the processing board 224 determines the period for conducting the above-mentioned movement lottery. The period for conducting the movement lottery may be, for example, about 1 to 10 minutes. Then, when the period for conducting the movement lottery arrives, the processing board 224 conducts the movement lottery.

[0334] On the other hand, when JUNO has finished eating the food, the processing board 224 displays a behavior pattern in which JUNO is alert to its surroundings, and also performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern in which JUNO moves to another area. The destination area is determined based on the movement probabilities shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern in which JUNO stays in area 1.

[0335] (If a scratching event is detected) Fig. 68A is an example of an explanatory diagram of movement to another area when a scratching event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 68B is an example image showing movement from area 5 to area 3 when a scratching event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 68C is an example image showing movement from area 5 to area 3 when a scratching event is detected when the intimacy level is low in a system 1 according to an embodiment.

[0336] FIG. 68A shows, for example, the conditions for starting movement, the conditions for Juno to escape to Area 5, Juno's behavior after escaping to Area 5, and Juno's behavior after scratching the claw post.

[0337] The scratching event occurs, for example, randomly. When the scratching event is detected, the movement start condition is established. The scratching event can be generated randomly, for example, by starting a timer based on an external detection (for example, when an external detection such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more is detected), and generating the event when no proximity detection is detected for a predetermined time (for example, 5 minutes). Note that the scratching event may be generated randomly, for example, by a lottery or the like.

[0338] When the movement start condition is met, the processing board 224 determines that JUNO will start the action of sharpening its claws. Then, when it is determined that JUNO will start the action of sharpening its claws, the processing board 224 displays a behavior pattern in which JUNO starts moving to sharpen its claws while remaining alert. For example, as shown in Fig. 68B, JUNO moves from area 5 to area 3 where the scratching board is placed.

[0339] A condition for JUNO to run back to Area 5 may be met, for example, when an external detection (such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more) is detected while moving to Area 3 or while scratching the claws as shown in Fig. 68C. For example, when the above-mentioned proximity detection occurs while moving to Area 3 or while scratching the claws, the processing board 224 displays a behavior pattern of JUNO moving to run back to Area 5.

[0340] After JUNO escapes to area 5, the processing board 224 plays a motion of JUNO showing anger, turning away, etc. Also, it is recommended that JUNO not use the scratching post on that day.

[0341] On the other hand, when JUNO finishes scratching the claws, the processing board 224 displays a behavior pattern of being alert to the surroundings and performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The area to which the user moves is determined based on the movement probability shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in area 3.

[0342] (If a human detection event is detected) Fig. 69A is an example of an explanatory diagram of movement to another area when a human detection event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 69B is an example image for explaining movement from area 1 or area 3 to area 5 when a human detection event is detected when the intimacy level is low in a system 1 according to an embodiment.

[0343] Figure 69A describes, for example, the conditions for starting movement, Juno's actions after returning to Area 5, and his actions after escaping to Area 5.

[0344] A human detection event occurs, for example, when a person is detected while JUNO is in one of areas 1 to 4. When a human detection event is detected, a movement start condition is established. When the movement start condition is established, the processing board 224 displays a behavior pattern in which JUNO moves to area 5 (runs away). Human detection corresponds to, for example, external detection (for example, proximity detection (i.e., when a person is detected because a user approaches JUNO), gesture detection, calling detection, or external detection such as detection of a loud sound above a certain decibel), etc.).

[0345] As shown in Fig. 69B, after JUNO returns to Area 5, the processing board 224 plays back motions that show actions such as getting angry and looking away, and also performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in Figs. 63 and 64 above. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in Area 5.

[0346] (Communication functions) Next, the communication function of JUNO when the intimacy level is low will be described. Figures 70A and 70B are examples of diagrams for explaining the communication function of JUNO when the intimacy level is low in system 1 according to an embodiment.

[0347] As shown in Figure 70A, JUNO does not respond to calls from the user. Also, when the user calls out to JUNO and JUNO is in areas 1 to 4, JUNO moves (runs away) to area 5. When JUNO moves to area 5, the motion played back differs depending on the situation, etc.

[0348] Also, as shown in Fig. 70B, when the intimacy level is low, JUNO does not spontaneously speak. Area movement is performed according to the result of a movement lottery (period judgment) performed at a predetermined period. In addition, the home appliance control function, alarm function, and timer function described later cannot be used.

[0349] An example of JUNO's motion that is played when JUNO moves from any of areas 1 to 4 to area 5 when the intimacy level is low will be briefly described below with reference to FIG. 70A.

[0350] If an event (e.g., a call) is detected while JUNO is waiting in any of Areas 1 to 4 (i.e., while waiting motion is being played), the processing board 224 displays JUNO's behavior pattern in the following order: interrupt the waiting motion, play a motion in which the ears react, play a 180-degree turn motion, and move (run, walk, etc.) toward Area 5. Furthermore, when JUNO reaches Area 5, the processing board 224 makes a 180-degree turn and starts the waiting motion, and as a behavioral derivative, performs a behavior in which only the head is turned while walking, and moves to Area 5. Then, the processing board 224 displays a behavior pattern in which JUNO makes a 180-degree turn and starts the waiting motion when it reaches Area 5.

[0351] Also, if, for example, a movement start condition is met while JUNO is in one of areas 1 to 4, and an event (for example, a call) is detected during playback of a short motion, such as immediately before moving to another area, the processing board 22 displays JUNO's behavior pattern in the following order: completion of playback of the short motion, playback of a motion in which the ears react, playback of a 180-degree turn motion, and movement (running, walking, etc.) toward area 5. Also, when JUNO reaches area 5, the processing board 224 makes a 180-degree turn and starts a standby motion, and as a behavioral derivative, performs a behavior in which only the head is turned while walking, and moves to area 5. Then, the processing board 224 displays a behavior pattern in which, when JUNO reaches area 5, it makes a 180-degree turn and starts a standby motion.

[0352] Also, for example, if a movement start condition is met while JUNO is in one of areas 1 to 4 and an event (e.g., a call) is detected during area movement, the processing board 224 stops the area movement, plays a motion in which the ears react, plays a 180-degree turn motion, and displays JUNO's behavior pattern in the following order: movement (running, walking, etc.) toward area 5. Also, the processing board 224 makes JUNO turn 180 degrees when it reaches area 5 and starts a standby motion, and as a behavioral derivative, it turns its head while walking and moves to area 5. Then, the processing board 22 displays a behavior pattern in which it makes a 180-degree turn when it reaches area 5 and starts a standby motion.

[0353] Even if an event (for example, a call) occurs while JUNO is in Area 5, the processing board 224 displays a behavior pattern indicating that JUNO is unresponsive.

[0354] At low intimacy, JUNO does not speak spontaneously. Processing board 224 also performs a move lottery at a predetermined cycle, and displays JUNO's area movement based on the result of this move lottery. Processing board 224 can also display a behavior pattern of moving to another area when the feeding event described above is detected (when the user feeds), when a random scratching event is detected, or when a human detection event is detected (when a human is detected), in addition to the move lottery performed at a predetermined cycle. At low intimacy, other functions (home appliance control function, alarm function, timer function) described later cannot be used.

[0355] In this way, the processing board 224 is able to display a behavior pattern as if JUNO had a virtual instinct function when the intimacy level is low.

[0356] (Other examples of motion in low density areas) The motion of JUNO is as described above, but the motion of JUNO is not limited to the above example, and various motions may be played back. Examples of the playback motion of JUNO are shown in Figs. 71 to 76.

[0357] Fig. 71 is a diagram showing an example of a playback motion in the case where JUNO runs away when an event is detected in the system 1 according to an embodiment. The specific operation of JUNO is as shown in Fig. 71, and therefore a detailed description thereof will be omitted.

[0358] 72 is a diagram showing an example of a playback motion when JUNO turns and escapes while moving between areas in the system 1 according to one embodiment. The specific movements of JUNO are as shown in FIG. 72, and therefore a detailed description thereof will be omitted.

[0359] Fig. 73 is a diagram showing an example of a playback motion when JUNO turns and escapes during standby in the system 1 according to an embodiment. The specific movements of JUNO are as shown in Fig. 73, and therefore detailed description thereof will be omitted.

[0360] 74 is a diagram showing an example of a motion reproduced when an event is detected while JUNO is eating food in the system 1 according to an embodiment. The specific operation of JUNO is as shown in FIG. 74, and therefore a detailed description thereof will be omitted.

[0361] Fig. 75 is a diagram showing an example of a motion reproduced when an event is detected while JUNO is scratching its claws in the system 1 according to an embodiment. The specific operation of JUNO is as shown in Fig. 75, and therefore a detailed description thereof will be omitted.

[0362] Fig. 76 is a diagram showing an example of a motion played back when JUNO runs away after an event is detected while sleeping in the system 1 according to an embodiment. The specific operation of JUNO is as shown in Fig. 76, and therefore a detailed description thereof will be omitted.

[0363] It should be noted that the examples of playback motions shown in Figs. 71 to 76 may be used not only when the degree of intimacy is low, but also when the degree of intimacy is medium or high.

[0364] (11-2. In the case of intimacy level) Fig. 77A is an example of a diagram for explaining JUNO's state, behavior pattern, etc. when the intimacy level is medium in the system 1 according to an embodiment. Fig. 77B is an example of an image displayed in the video display space S1 of the system 1 according to an embodiment when the intimacy level is medium.

[0365] As shown in FIG. 77A, JUNO's state and feelings at medium intimacy are as follows. That is, JUNO is starting to get used to the environment and understand where things are. Furthermore, the processing board 224 displays JUNO's behavior pattern as if it can move freely within the area regardless of whether or not a person is detected. That is, unlike the case of low intimacy, there is no fixed position as if JUNO recognizes Area 5 as its territory. Furthermore, at medium intimacy, JUNO does not run away based on the occurrence of an external event. However, since JUNO is still not used to people, the processing board 224 controls JUNO so that it is often seen to ignore calls from the user.

[0366] As shown in Fig. 77A, JUNO's behavioral policy is set so that it does not remain in a fixed position. During the intimacy level, JUNO moves freely between areas 1 to 5 regardless of whether or not a person is detected, which is a behavioral pattern that indicates that JUNO is gradually getting used to the area.

[0367] As shown in Figure 77B, a food bowl is displayed in area 1, and a scratching board is displayed in area 3. During intimacy, JUNO can move freely between areas 1 and 5 without being detected by humans. Area 5 is an area that is not subject to outside influences.

[0368] Fig. 78A is an example of an explanatory diagram for moving to another area when the intimacy level is high in the system 1 according to an embodiment. Fig. 78B is an example of an image for explaining moving to another area when the intimacy level is high in the system 1 according to an embodiment.

[0369] Also, as in the case of low intimacy, the processing board 224, for example, conducts a movement lottery at a predetermined period, and JUNO may move to another area based on the results of this movement lottery, or JUNO may move to another area based on event detection.

[0370] Also, as shown in Figures 78A and 78B, when JUNO moves to another area based on the result of a movement lottery performed at a predetermined cycle, the movement lottery performed at the predetermined cycle is performed, for example, at the movement probability shown in Figures 63 and 64, as in the case of low intimacy. The above-mentioned predetermined cycle is performed, for example, at a cycle of about 1 to 10 minutes. Also, when JUNO who won the movement lottery moves to another area, the processing board 224 determines by lottery the cycle for performing the movement lottery after the movement. In this case, the area to which JUNO can move is, for example, any of areas 1 to 5.

[0371] 78B, when the intimacy level is medium, the probability that JUNO will move to Area 1 based on the result of the movement lottery is, for example, 5 to 30%. Similarly, the probability that JUNO will move to Area 2 is, for example, 10 to 20%, the probability that JUNO will move to Area 3 is, for example, 15 to 20%, the probability that JUNO will move to Area 4 is, for example, 20%, and the probability that JUNO will move to Area 5 is, for example, 50 to 10%.

[0372] Next, movement to another area based on event detection will be described with reference to Fig. 79A to Fig. 80C. Event detection includes, for example, a feeding event that may occur when the user feeds the robot, a scratching event that may occur randomly, and the like, as in the case of low intimacy. Note that, during intimacy, it is preferable not to move to another area even if a human detection event that may occur when a human is detected occurs, as in the case of low intimacy.

[0373] (If a feeding event is detected) Fig. 79A is an example of an explanatory diagram of a movement to another area when a feeding event is detected during intimacy in a system 1 according to an embodiment. Fig. 79B is an example of an image showing a movement from area 5 to area 1 when a feeding event is detected during intimacy in a system 1 according to an embodiment.

[0374] Figure 79A describes, for example, the conditions for starting movement, the conditions for escaping to Area 5, and Juno's behavior after finishing eating the food.

[0375] When the user gives food (i.e., when food is generated), a feeding event is detected. When a feeding event is detected, the processing board 224 calculates the probability that JUNO will eat the food. Although Fig. 79A states that "the eating probability is calculated from the intimacy sheet", the intimacy sheet is the same as in the case of low intimacy, for example, the probability of moving to Area 1 shown in Fig. 64 is used.

[0376] When the movement start condition is met, the processing board 224 decides that JUNO will start to take action to eat food. Then, when it is decided that JUNO will start to take action to eat food, the processing board 224 displays a behavior pattern in which JUNO starts to move vigilantly in order to eat food, and for example, as shown in FIG. 79B, JUNO moves from Area 5 to Area 1 where a food bowl is placed. Note that, during the intimacy level, no conditions are set for escaping to Area 5. Also, JUNO will not escape even if an external event is detected.

[0377] When JUNO has finished eating the food, the processing board 224 displays a behavior pattern in which JUNO is alert to its surroundings, and also performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern in which JUNO moves to another area. The destination area is determined based on the movement probabilities shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern in which JUNO stays in Area 1.

[0378] (If a scratching event is detected) Fig. 80A is an example of an explanatory diagram of a movement to another area when a scratching event is detected during intimacy in a system 1 according to an embodiment. Fig. 80B is an example of an image showing a movement from area 5 to area 3 when a scratching event is detected during intimacy in a system 1 according to an embodiment.

[0379] FIG. 80A describes, for example, the conditions for starting movement, the conditions for Juno to escape to Area 5, Juno's behavior after scratching the post, etc.

[0380] As in the case of low intimacy, the scratching event occurs, for example, randomly. When the scratching event is detected, the movement start condition is established. The scratching event can be generated randomly, for example, by starting a timer based on an external detection (for example, when an external detection such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more is detected), and generating the event when no external detection is detected for a predetermined time (for example, 5 minutes). Note that the scratching event may be generated randomly, for example, by a lottery. Note that, during intimacy, the condition for escaping to Area 5 is not set. Also, even if an external event is detected, JUNO will not escape.

[0381] When the movement start condition is met, the processing board 224 determines that JUNO will start the action of sharpening its claws. Then, when it is determined that JUNO will start the action of sharpening its claws, the processing board 224 displays a behavior pattern in which JUNO starts moving to sharpen its claws while remaining alert. For example, as shown in Fig. 80B, JUNO moves from area 5 to area 3 where the scratching board is placed.

[0382] When JUNO finishes sharpening its claws, the processing board 224 performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probability shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in area 3.

[0383] (If a human detection event is detected) FIG. 80C is an example of an explanatory diagram illustrating movement to another area when a human detection event is detected when the intimacy level is medium in the system 1 according to an embodiment.

[0384] As shown in FIG. 80C, even if a human detection event is detected when the intimacy level is medium, JUNO does not move to Area 5 (does not escape), unlike when the intimacy level is low.

[0385] (Communication functions) Next, the communication function of JUNO when the intimacy level is medium will be described with reference to Fig. 81A and Fig. 81B. Fig. 81A and Fig. 81B are examples of diagrams for explaining the communication function of JUNO when the intimacy level is medium in the system 1 according to one embodiment.

[0386] During the intimacy level, JUNO can respond to calls from the user. However, as shown in Figure 81A, the response differs depending on the area JUNO is in.

[0387] For example, when JUNO is in area 1, he will respond to the user's Japanese. When the user calls out to him, JUNO will respond with motions only, without speaking. When JUNO is in areas 2 to 4, JUN will respond with motions, for example.

[0388] Also, when the user calls out, the processing board 224 performs a response lottery. The probability that JUNO will respond to the user's call is set according to the area in which JUNO is staying, for example, as shown in FIG. 81B. When JUNO is in area 1, for example, the probability of being in area 1 (i.e., the probability of moving to area 1 shown in FIG. 64, which is a 5 to 30% probability) is used, and when JUNO is in area 2, for example, the probability of being in area 2 (i.e., the probability of moving to area 2 shown in FIG. 64, which is a 10 to 20% probability) is used. Note that if the response lottery is lost, the processing board 224 displays a behavior pattern in which JUNO ignores the user's call.

[0389] When the above-mentioned motion response is performed, the processing board 224 displays a behavior pattern in which JUNO responds, for example, with its ears. For example, as shown in Fig. 81B, when JUNO is in areas 1 to 4, the processing board 224 displays a behavior pattern in which JUNO responds to a call from the user by twitching its ears.

[0390] Even if an event (for example, a call) occurs while JUNO is in Area 5, the processing board 224 displays a behavior pattern indicating that JUNO is unresponsive.

[0391] (Other, Summary) Next, other overviews of intimacy will be described with reference to Fig. 82. Fig. 82 is an example of a diagram for explaining other states and actions of JUNO when intimacy is in progress in the system 1 according to one embodiment.

[0392] As shown in FIG. 82, JUNO does not spontaneously speak during intimacy. The processing board 224 also performs a move lottery at a predetermined cycle and displays JUNO's area movement based on the result of this move lottery. In addition to the move lottery performed at a predetermined cycle, the processing board 224 can also display a behavior pattern of moving to another area when the above-mentioned feeding event is detected (when the user feeds) or when a randomly occurring scratching event is detected. During intimacy, other functions (home appliance control function, alarm function, timer function) described later cannot be used.

[0393] In this way, the processing board 224 is able to display behavioral patterns as if JUNO had a virtual instinct function during the intimacy level.

[0394] (11-3. High Intimacy) Fig. 83A is an example of a diagram for explaining JUNO's state, behavior pattern, etc. when the intimacy level is high in the system 1 according to one embodiment. Fig. 83B is an example of an image displayed in the video display space S1 of the system 1 according to one embodiment when the intimacy level is high.

[0395] As shown in FIG. 83A, JUNO's state and feelings when the intimacy level is high are as follows. That is, JUNO trusts the owner (user) and is completely accustomed to the surrounding environment, and is no longer wild. The processing board 224 displays JUNO's behavior pattern as being not wary and mostly staying in the front direction of the display area. In this embodiment, as shown in FIG. 64 above, in the case of high intimacy, in the period from 60 to 90 days (shown as 30 days from 0 to 30 in FIG. 64 for convenience), the ratio of JUNO being in area 1 or area 2 (i.e., the combined probability of moving to area 1 and the probability of moving to area 2) is, for example, 50% or more, and when the number of days the user has fed JUNO exceeds 90 days, the ratio of JUNO being in area 1 (i.e., the probability of moving to area 1) exceeds, for example, 50%. Also, in the case of high intimacy, JUNO trusts the owner (user) and begins to act proactively. However, the processing board 224 may set, for example, a whimsical coefficient or the like so that JUNO's reaction may be displayed as, for example, when the intimacy level is low or when the intimacy level is low.

[0396] Also, as shown in FIG. 83A, a fixed position is set as JUNO's course of action. JUNO's fixed position at low intimacy level is, for example, area 1, which is closest to the user, and JUNO is most likely to be in area 1 among areas 1 to 5. JUNO is mainly in area 1, but even if it is in an area other than area 1, it moves to area 1 in response to human detection. In this way, by moving JUNO to area 1, which is closest to the user, it is possible to embody the process of JUNO getting used to the user and becoming attached to the user. In particular, when moving from area 5, which is farthest from the user, to area 1, which is farthest from the user, the moving distance of JUNO is large, and it is possible to clearly grasp that JUNO has approached the user, that is, that JUNO has become attached to the user.

[0397] As shown in Fig. 83B, a food bowl and JUNO are displayed in area 1, and a scratching board is displayed in area 3. Area 5 is an area that is not subject to external influences.

[0398] Fig. 84A is an example of an explanatory diagram for moving to another area when the intimacy level is high in the system 1 according to an embodiment. Fig. 84B is an example of an image for explaining moving to another area when the intimacy level is high in the system 1 according to an embodiment.

[0399] Also, as in the cases of low and medium intimacy, the processing board 224, for example, conducts a movement lottery at a predetermined period, and JUNO may move to another area based on the results of this movement lottery, or JUNO may move to another area based on event detection.

[0400] Also, as shown in Figures 84A and 84B, when JUNO moves to another area based on the result of a movement lottery held at a predetermined cycle, the movement lottery held at the predetermined cycle is held at the movement probability shown in Figures 63 and 64, for example, as in the cases of low and medium intimacy. The above-mentioned predetermined cycle is held at a cycle of about 1 to 10 minutes, for example. Also, when JUNO who won the movement lottery moves to another area, the processing board 224 decides by lottery the cycle for holding the movement lottery after the movement. In this case, the area to which JUNO can move is, for example, any of areas 1 to 5.

[0401] 84B, when intimacy is high, the probability that JUNO will move to Area 1 based on the result of the movement lottery is, for example, 30 to 50%. Similarly, the probability that JUNO will move to Area 2 is, for example, 20 to 30%, the probability that JUNO will move to Area 3 is, for example, 20 to 10%, the probability that JUNO will move to Area 4 is, for example, 20 to 7%, and the probability that JUNO will move to Area 5 is, for example, 10 to 3%.

[0402] Next, we will explain movement to another area based on event detection. Event detection includes, for example, feeding event detection that may occur when a user feeds a robot, scratching event detection that may occur randomly, human detection event detection that may occur when a human is detected, etc.

[0403] (If a feeding event is detected) Fig. 85A is an example of an explanatory diagram of a movement to another area when a feeding event is detected when the intimacy level is high in the system 1 according to an embodiment. Fig. 85B is an example of an image showing an example of a case where a feeding event is detected when the intimacy level is high in the system 1 according to an embodiment.

[0404] Figure 85A describes, for example, the conditions for starting movement, the conditions for escaping to Area 5, and Juno's behavior after finishing eating the food.

[0405] When the user gives food (i.e., when food is generated), a feeding event is detected. When a feeding event is detected, the processing board 224 calculates the probability that JUNO will eat the food. Although Fig. 85A states that "the eating probability is calculated from the intimacy sheet", the intimacy sheet is the same as in the cases of low intimacy and medium intimacy, for example, the probability of moving to Area 1 shown in Fig. 64 is used.

[0406] When the movement start condition is met, the processing board 224 decides that JUNO will start to take action to eat food. Then, when it is decided that JUNO will start to take action to eat food, the processing board 224 displays a behavior pattern in which JUNO starts to move vigilantly in order to eat food, and for example, as shown in FIG. 85B, JUNO moves from the area where it is currently located to Area 1 where a food bowl is placed. However, at high intimacy, JUNO is often in Area 1, and if a feeding event occurs while in Area 1, it starts to eat food in Area 1. At high intimacy, there is no condition set for escaping to Area 5. Also, JUNO will not escape even if an external event is detected.

[0407] After JUNO has finished eating the food, the processing board 224 performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probability shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in area 1.

[0408] (If a scratching event is detected) Fig. 86A is an example of an explanatory diagram of movement to another area when a scratching event is detected in a system 1 according to an embodiment when the intimacy level is high. Fig. 86B is an example of an image showing movement from area 1 to area 3 in a system 1 according to an embodiment when a scratching event is detected in a system 1 according to an embodiment when the intimacy level is high.

[0409] Figure 86A describes, for example, the conditions for starting movement, the conditions for Juno to escape to Area 5, Juno's behavior after scratching the post, etc.

[0410] As in the case of low intimacy and medium intimacy, the scratching event occurs, for example, randomly. When the scratching event is detected, the movement start condition is established. The scratching event can be generated randomly, for example, by starting a timer based on an external detection (for example, when an external detection such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more is detected), and generating the event when no external detection is detected for a predetermined time (for example, 5 minutes). Note that the scratching event may be generated randomly, for example, by a lottery. Note that in the case of high intimacy, the condition for escaping to Area 5 is not set. Also, even if an external event is detected, JUNO will not escape.

[0411] When the movement start condition is met, the processing board 224 determines that JUNO will start the action of sharpening its claws. Then, when it is determined that JUNO will start the action of sharpening its claws, the processing board 224 displays a behavior pattern in which JUNO starts moving to sharpen its claws while remaining alert, and for example, as shown in Fig. 86B, JUNO moves from area 1 to area 3 where a scratching board is placed.

[0412] When JUNO finishes sharpening its claws, the processing board 224 performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The area to which the user moves is determined based on the movement probabilities shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in area 3.

[0413] (If a human detection event is detected) Fig. 87A is an example of an explanatory diagram of movement to another area when a human detection event is detected when the intimacy level is high in a system 1 according to an embodiment. Fig. 87B is an example image for explaining movement from an area other than area 1 to area 1 when a human detection event is detected when the intimacy level is high in a system 1 according to an embodiment.

[0414] FIG. 87A shows, for example, the conditions for starting movement, JUNO's actions after moving to Area 1, etc.

[0415] As shown in FIG. 87A and FIG. 87B, a human detection event occurs when JUNO is in any of areas 1 to 4, for example, when an external detection occurs. When a human detection event is detected, a movement start condition is established. When the movement start condition is established, the processing board 224 displays a behavior pattern in which JUNO moves (approaches) to area 1. External detection corresponds to proximity detection (i.e., when a person is detected because a user approaches JUNO), gesture detection, calling detection, or external detection such as detection of a loud sound above a predetermined decibel is detected), etc.

[0416] After JUNO moves to area 1, the processing board 224 performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in the above-mentioned Figures 63 and 64. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in area 1.

[0417] (Communication functions) Next, the communication function of JUNO when the intimacy level is high will be described with reference to Fig. 88A and Fig. 88B. Fig. 88A is an example of a diagram for explaining a call response, which is an example of the communication function of JUNO when the intimacy level is high, in the system 1 according to one embodiment. Fig. 88B is an example of an image for explaining area movement when a call is made in the case of a high intimacy level, in the system 1 according to one embodiment.

[0418] As shown in Fig. 88A, when the response condition is met, the processing board 224 can control JUNO to respond in Japanese to a call from a user, for example. The processing board 224 can also control the content of the response to differ depending on the area JUNO is in. The response condition corresponds to, for example, when a call from a user is detected.

[0419] 88A, the human detection event is such that, when JUNO is in any of areas 1 to 4, the processing board 224 can control JUNO to respond to a call from a user, for example, in Japanese or with a motion. Also, when JUNO is in areas 2 to 4, the processing board 224 can control JUNO to respond to a call from a user, for example, with a motion.

[0420] If JUNO is in area 1 and has not yet mastered Japanese, the processing board 224 performs the following control: For example, if an event such as a call from the user is detected, the processing board 224 controls JUNO to respond with a motion in response to the event detection.

[0421] Furthermore, if JUNO is in area 1 and has learned Japanese, the processing board 224 exercises control as follows. That is, for example, when an event such as a call from a user is detected, if JUNO remembers the Japanese that was called out to him (i.e., if he has already learned the Japanese that was called out to him), the processing board 224 controls JUNO to respond in Japanese. On the other hand, if JUNO is in area 1 and has learned Japanese, but does not remember the Japanese that was called out to him (i.e., if he has not already learned the Japanese that was called out to him), the processing board 224 controls JUNO to respond with a motion.

[0422] Furthermore, when JUNO is in areas 2 to 4, if an event such as a call from a user is detected, the processing board 224 controls JUNO to respond with a motion in response to the event detection.

[0423] Furthermore, when JUNO is in Area 5, even if an event such as a call is detected, the processing board 224 controls JUNO so that it does not respond. As described above, Area 5 is an area that is not subject to external influences.

[0424] In this embodiment, even if JUNO learns Japanese and becomes able to speak, it is controlled not to speak spontaneously. Therefore, unless an event such as a user calling out is detected, JUNO is controlled not to speak to the user. However, this is not limited to this, and even if an event such as a user calling out is not detected, for example, when an event such as human detection occurs, the processing board 224 may control JUNO to speak spontaneously.

[0425] Also, as shown in Figure 88B, when Juno is called and the intimacy level is high, there is a 30-50% probability that he will move to Area 1, a 20-30% probability that he will move to Area 2, a 20-10% probability that he will move to Area 3, a 20-7% probability that he will move to Area 4, and a 10-3% probability that he will move to Area 5. Note that the above probabilities include not only the probability of moving, but also the probability of staying in the same place as before.

[0426] In this embodiment, the language that JUNO (character) learns is assumed to be Japanese, but the language to be learned is not limited to Japanese and may be other languages ​​such as English, Chinese, French, German, Korean, etc., or multiple languages ​​may be learnable. It is also preferable to have JUNO learn the language spoken by the user.

[0427] (summary) Next, a summary of JUNO's actions when the intimacy level is high will be described with reference to Fig. 89. Fig. 89 is an example of a diagram for explaining a summary of JUNO's actions when the intimacy level is high in the system 1 according to one embodiment.

[0428] As described above, the processing board 224 performs a movement lottery at a predetermined cycle and displays JUNO's area movement based on the result of this movement lottery. In addition to the movement lottery performed at a predetermined cycle, the processing board 224 can also display a behavior pattern of moving to another area when the feeding event, scratching event, or human detection event described above is detected.

[0429] In this way, the processing board 224 is able to display a behavior pattern as if JUNO had a virtual instinct function when the intimacy level is high.

[0430] <12. Features specific to high intimacy> By the way, there are functions unique to the high intimacy level that are not available to the low intimacy level or the medium intimacy level. The functions unique to the high intimacy level will be described below with reference to Figures 90A, 90B, 90C, 90D, and 91.

[0431] FIG. 90A is an example of a diagram for explaining a function specific to a high intimacy level in the system 1 according to an embodiment.

[0432] As shown in FIG. 90A, functions specific to a high intimacy level include, for example, a home appliance control function, an alarm function, and a timer function.

[0433] Although the alarm function and the timer function are similar functions, in this embodiment, from the viewpoint of reliably informing the user of the passage of time, a function that notifies the user of the passage of time multiple times (e.g., five times) is defined as the alarm function, and a function that notifies the user of the passage of time only once is defined as the timer function.

[0434] 90A, a plurality of call word groups in which a plurality of call words having the same function are stored, and JUNO response words corresponding to the call word groups are stored in the ROM 702 of the processing board 224. The JUNO response words include a content confirmation word for confirming the content, and an implementation word for notifying that implementation (or time has passed).

[0435] In this embodiment, as a plurality of call word groups having the same function, for example, a timer call word group, an alarm call word group, a home appliance control call word group, etc. are stored, but the present invention is not limited to this.

[0436] Figure 90B is an example of a diagram for explaining a function specific to a high intimacy level in system 1 according to one embodiment, and shows an example of a group of call-out words and JUNO response words (character response words) for the timer function.

[0437] 90B, for example, a plurality of call words such as "timer for ** minutes," "set timer for ** minutes," and "set timer in ** minutes" are stored in the call word group for the timer function. In addition, for example, a content confirmation word such as "I'll set the timer for ** minutes" and an implementation word such as "The time's up" are stored as JUNO response words corresponding to the call word group for the timer function.

[0438] Figure 90C is an example of a diagram for explaining a function specific to a high intimacy level in system 1 according to one embodiment, and shows an example of a group of call words and JUNO response words (character response words) for the alarm function.

[0439] 90C, for example, a plurality of call words such as "alarm at ** minutes", "set alarm at ** minutes", and "set alarm at ** minutes" are stored in the call word group of the alarm function. In addition, for example, a content confirmation word such as "I'll set the alarm in ** hours and ** minutes" and an implementation word that repeats "It's time" five times are stored as JUNO response words corresponding to the call word group of the alarm function.

[0440] Figure 90D is an example of a diagram for explaining functions specific to a high intimacy level in system 1 according to one embodiment, and shows an example of a group of call words and JUNO response words (character response words) for home appliance control functions.

[0441] 90D, for example, a plurality of call words such as "Turn on the TV," "Turn on the TV," and "TV on" are stored in the call word group of the home appliance control function. In addition, for example, a content confirmation word such as "I'll turn on XX / I'll ​​turn off XX" and an implementation word such as "I tried it" are stored as JUNO response words corresponding to the call word group of the home appliance control function.

[0442] Returning to Fig. 90A, when an external event such as a call from a user is detected, the processing board 224 judges whether the called word is stored in any call word group. If the called word is stored in any call word group, the processing board 224 controls JUNO to issue a JUNO response word (content confirmation word) corresponding to the call word group in which the called word is stored. Then, the processing board 224 performs the content corresponding to the call word and controls JUNO to issue a response word (execution word).

[0443] For example, when a call from a user is detected and the called word is a setting word for a home appliance control function, the processing board 224 determines whether the called word has been stored, and if the called word has been stored, controls JUNO to utter a content confirmation word such as "I'll turn on ○○" or "I'll remove ○○", and then carries out the content, and then controls JUNO to say "I tried it."

[0444] Fig. 91 is an example of a diagram for explaining the timer function and the alarm function in the system 1 according to an embodiment. As shown in Fig. 91, if a timer or alarm has been set, the information can be cancelled by voice. For example, when a call word such as "cancel alarm" is detected, the processing board 224 controls JUNO to say "I'll cancel the alarm" and also cancels the content so that it is not carried out.

[0445] Also, if a timer or alarm is already set and you set an additional timer or alarm, it will be overwritten. For example, if the user says words such as "Alarm at 3:15," JUNO will say "I'll set the alarm at 3:15," overwriting the previous setting. Although only one timer and one alarm can be set in total, this is not limited to the above, and it may be possible to set one timer and one alarm, or to set multiple timers and alarms at least.

[0446] <13. Japanese language learning function> The Japanese language learning function of JUNO will be described with reference to Figures 92A to 101D. The Japanese language learning function is a function that is unlocked during intimacy (that is, activated during intimacy), for example.

[0447] 92A and 92B are diagrams for explaining an example of a specification in which a character learns Japanese in the system 1 according to an embodiment.

[0448] As shown in FIG. 92A, JUNO can learn Japanese by persistently speaking to it in Japanese. It is advisable to set a daily upper limit (described later) for when it is spoken to in Japanese, and furthermore, to reduce the daily increase upper limit for learning Japanese on days when it is not spoken to in Japanese. In this way, JUNO will not be able to learn Japanese unless it studies Japanese every day. Also, even if it does learn Japanese once, if days go by without speaking to JUNO, JUNO will no longer respond in Japanese.

[0449] When JUNO is called by a user while in Area 1, the processing board 224 counts the number of times the user is called. However, there is a limit set on how much the count can increase per day (the above-mentioned upper limit of increase).

[0450] As shown in Fig. 92B, before learning Japanese, the robot passes through several states, such as not understanding Japanese, trying to learn Japanese, and speaking Japanese. In the state where the robot does not understand Japanese or is trying to learn Japanese, the robot will only respond with motions even if the user calls out to it. In the state where the robot speaks Japanese, the robot will, for example, speak Japanese.

[0451] (13-1. Not understanding Japanese) When the user does not understand Japanese, if there is a registered word (described later) preregistered in the ROM 702 of the processing board 224 among the words spoken by the user, the number of times that group, described later, is counted up. For example, if the user says "Good morning," the number of times the "Good morning group" is counted up by one. This number of times spoken is stored in the character-specific storage information, described later.

[0452] Note that JUNO will not react when a user speaks a word that has not been preregistered in the ROM 702 of the processing board 224. For example, even if a user speaks to JUNO "Good morning," JUNO will not react if "Good morning" is not a registered word. In addition, the number of times it has been spoken to will not be counted.

[0453] For example, it is not necessary to set the number of times a registered word (described later) is spoken by a user and the upper limit of increase per day, etc. Also, depending on the setting situation, the number of times until Japanese is memorized may be changed, etc.

[0454] Next, as mentioned above, JUNO's responses to calls at each level of intimacy will be explained again. Fig. 93A is an example of a diagram for explaining the response of JUNO (character 1200) at a low level of intimacy in system 1 according to one embodiment. Fig. 93B is an example of a diagram for explaining the response of JUNO (character) at a medium level of intimacy in system 1 according to one embodiment. Fig. 93C is an example of a diagram for explaining the response of JUNO (character) at a high level of intimacy in system 1 according to one embodiment.

[0455] As shown in FIG. 93A, when the intimacy level is low, JUNO makes a motion of running away to Area 5, for example, when called by the user.

[0456] As shown in FIG. 93B, in the case of medium intimacy, if JUNO is in Area 1 to Area 4 and is called out to by the user, he responds with a motion.

[0457] As shown in FIG. 93C, in the case of high intimacy, if JUNO is called by the user while in Areas 2 to 4, he will respond with motion. On the other hand, if the user calls out to JUNO while he is in Area 1, he will respond in Japanese, such as "Good morning." However, he will only respond in Japanese if the user calls out to him using a registered word that he has memorized. If the user calls out to him using a registered word that he has not memorized, he will only respond with motion.

[0458] Next, the recognition words will be described. Fig. 94 is an example of a diagram for explaining the recognition words, particularly the echo words, in the system 1 according to an embodiment.

[0459] As shown in Fig. 94, the recognition words include, for example, a customary greeting group, a dialogue word group, and a parrot-echoing word group, etc. Note that Fig. 94 shows a parrot-echoing word group table, and this parrot-echoing word group table is stored in the ROM 702 of the processing board 224.

[0460] As shown in FIG. 94, the echo word group table includes JUNO's response words that return the same message as the user's call, such as "Good morning" or "Good evening." For example, if the user says "Good morning" to JUNO, JUNO will echo (i.e., repeat) "Good morning." In this way, JUNO echoing the user's call can give the user a sense of closeness to JUNO. In particular, by JUNO echoing back what the user says in Area 1, which is close to the user, the user can feel an even greater sense of closeness to JUNO. Moreover, because JUNO only repeats back what the user says when the intimacy level is high, the bond between the user and JUNO can be further deepened.

[0461] Furthermore, the call word group in the echo word group table includes a number of word groups, such as a "good morning group," a "good afternoon group," a "good evening group," and a "good night group." The "good morning group" includes call words (registered words), such as "good morning." The "hello group" includes registered words, such as "hello." The "good evening group" includes call words (registered words), such as "good evening." The "good night group" includes call words (registered words), such as "good night."

[0462] Fig. 95 is an example of a diagram for explaining a dialogue-related word group in the system 1 according to an embodiment. Fig. 95 shows a dialogue-related word group table consisting of word group names, call word groups, and JUNO response words. The dialogue-related word group table is stored in, for example, the ROM 702 of the processing board 224. Note that Fig. 95 shows only a part of the dialogue-related word group table, and details of the dialogue-related word group table are shown in Figs. 96A to 97E.

[0463] Figures 96A and 96B are diagrams showing an example of a word group name and a call word group in the dialogue-related word group table shown in Figure 95. Note that the word group name and the call word group are linked to each other.

[0464] 97A and 97B are diagrams showing examples of word group names and JUNO response words (character response words) in the first, second, and third stages in the dialogue word group table shown in Fig. 95. The word group names and JUNO response words are linked to each other.

[0465] Also, Figures 97C to 97E are diagrams showing examples of word group names ...

Claims

1. A system that displays characters on a video display unit, A wind detection means for detecting wind, Equipped with control means, Based on the detection results from the wind detection means, the control means displays on the video display unit an image of the character performing movements corresponding to the user's breath. A system characterized by the following features.

2. The wind detection means includes a humidity sensor for detecting humidity and a temperature sensor for detecting temperature. The control means determines the type of breath blown by the user based on the humidity detection result from the humidity sensor and the temperature detection result from the temperature sensor. The system according to feature 1.

3. When the control means detects that the humidity has risen sharply from a steady state using the humidity sensor, it determines that the user has breathed on the surface. The system according to claim 2.

4. When the humidity sensor detects a rapid increase in humidity from a steady state, and the temperature sensor detects an increase in temperature from a steady state, the control means determines that the type of breath blown by the user is warm breath. The system according to claim 2 or 3, characterized in that it is the same as described above.

5. When the humidity sensor detects that the humidity has risen sharply from a steady state, and the temperature sensor detects that the temperature has not changed from a steady state or has decreased from a steady state, the control means determines that the type of breath blown by the user is cold breath. The system according to any one of claims 2 to 4.

6. The control means, when the type of breath blown by the user is cold breath, displays an image of the character's clothes fluttering on the image display unit. The system according to any one of claims 2 to 5, characterized by the following:

7. If the type of breath blown by the user is warm, the control means displays on the video display unit an image of the character moving to avoid the breath. The system according to any one of claims 2 to 6, characterized by the following:

8. The control means determines that the blowing of breath has stopped when a predetermined time has elapsed since the humidity sensor detected a rapid increase in humidity from a steady state. The system according to any one of claims 3 to 7.

9. The wind detection means includes a gas sensor, The control means determines that the type of breath blown by the user is cold when the output value of the gas sensor drops sharply from a steady state, and determines that the type of breath blown by the user is warm when the output value of the gas sensor drops sharply from a steady state. The system according to any one of claims 1 to 8.

10. The wind detection means includes a humidity sensor for detecting humidity and a gas sensor. The control means determines that the user has breathed on the gas sensor when the output value of the gas sensor drops sharply from a steady state and the humidity sensor detects a sharp increase in humidity from a steady state. The system according to any one of claims 1 to 9.

11. The wind detection means includes a humidity sensor for detecting humidity and a gas sensor. The control means determines that air is being blown by the fan when the output value of the gas sensor drops sharply from a steady state, but the humidity detected by the humidity sensor remains at a steady state. The system according to any one of claims 1 to 10.

12. The control means outputs sound corresponding to the detection result by the wind detection means as sound emitted by the character displayed on the video display unit. The system according to any one of claims 1 to 11.

13. A program for a computer to implement the functions of the control means of the system described in any one of claims 1 to 12.

Images