Computer program product and system

Abstract

The present disclosure provides computer program products and systems. A computer executes: a process for sequentially determining whether or not an object is in a state having a high priority order from among a plurality of states in which the type of the object is different from each other, in accordance with the priority order set for each of the plurality of states, on the basis of state information relating to the state of the object; when it is initially determined that the object is in a certain state, specifying the certain state as the state of the object; selecting a state image corresponding to the specified state from a plurality of state images representing a plurality of states and each including an animation video; and displaying the selected state image on a display unit.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority and benefit of Japanese Patent Application No. 2024-177104, filed October 9, 2024. The specification, claims, and drawings of Japanese Patent Application No. 2024-177104 are hereby incorporated by reference in their entirety into the present specification. TECHNICAL FIELD

[0003] The present disclosure relates to a computer program product and a system. BACKGROUND

[0004] In the past, a robot is known that is capable of virtual communication with a user by performing various actions in accordance with its state (for example, JP Laid-Open No. 2002-59389).

[0005] However, the state of an object such as a robot is maintained as an internal parameter of the object, and thus there is a problem in that it is not necessarily easy to correctly grasp the state of the object from the appearance of the object. SUMMARY

[0006] An object of the present disclosure is to enable the state of an object to be easily grasped.

[0007] In a display control method according to the present disclosure, one or more processors perform the following processing: based on state information related to a state of an object, determine whether the object is in each of a plurality of states that are respectively set in accordance with a priority order that is different for each type of the object, from a state with a high priority order in the plurality of states; in a case where it is initially determined that the object is in a certain state, determine the certain state as the state of the object; select a state image corresponding to the determined state from a plurality of state images that respectively include animation videos and represent the plurality of states; and cause the selected state image to be displayed on a display.

[0008] EFFECT OF THE INVENTION

[0009] According to the present disclosure, the state of an object can be easily grasped. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a diagram showing the appearance of a robot and a smartphone.

[0011] Figure 2 is a schematic diagram showing the structure of the main body of a robot.

[0012] Figure 3 is a block diagram showing the functional structure of a robot.

[0013] Figure 4 It is a block diagram representing the functional structure of a smartphone.

[0014] Figure 5 It is an image representing the main screen.

[0015] Figure 6 It is a diagram representing the content of status information.

[0016] Figure 7 It is a diagram representing an emotion map.

[0017] Figure 8 It is a diagram that represents the priority of the robot's state and the timing of the display of the state image.

[0018] Figure 9 It is a diagram showing the status images corresponding to "Power off / Communication off", "Sleep mode", and "Sleep mode".

[0019] Figure 10 It is a graph that represents the state image corresponding to the external stimuli received by the robot.

[0020] Figure 11 It is a diagram representing the content of the previously displayed information.

[0021] Figure 12 It is a diagram representing the state image corresponding to the robot's emotions.

[0022] Figure 13 It is a diagram that represents the state image corresponding to the robot's personality.

[0023] Figure 14 This is an image representing a detailed information screen.

[0024] Figure 15 This is an image representing the settings screen.

[0025] Figure 16 This is a flowchart representing the control steps of the main screen display process.

[0026] Figure 17 This is a flowchart representing the control steps of the main screen display process. Detailed Implementation

[0027] The embodiments of this disclosure will now be described with reference to the accompanying drawings. Figure 1As illustrated, a robot management system 1 (system) is provided with a robot 10, a smartphone 20 (terminal device), and a server 60. The robot 10 is provided with a main body 100, and an exterior 110 that covers the entire surface of the main body 100. The robot 10 is a pet robot that imitates a small living creature. The robot 10 is capable of performing a plurality of actions that are mutually different and imitate the behavior of the living creature. The exterior 110 is composed of a material having flexibility, and deforms in accordance with the movement of the main body 100. The exterior 110 has, for example, a fur formed of a plush fabric, a decorative member that imitates an eye, and the like. The smartphone 20 is communicably connected to the robot 10 through close proximity wireless communication. In the present embodiment, BLE (Bluetooth (registered trademark) Low Energy) is used as the close proximity wireless communication. The close proximity wireless communication using a means other than BLE can also be used. The robot 10 and the smartphone 20 are communicably connected through BLE, and perform the transmission and reception of data, thereby acting in cooperation. For example, the smartphone 20 acquires, from the robot 10, state information 232 related to the state of the robot 10 (refer to Figure 4 and Figure 6 ). The smartphone 20 causes the display portion 24 to display a home screen that contains various information related to the state of the robot 10, on a management application 231 (program) used for managing the robot 10 as an object based on the state information 232 (refer to Figure 5 ). It is also possible to cause two or more robots 10 to act in cooperation with respect to one smartphone 20. Instead of the smartphone 20, other types of devices such as a tablet terminal, a smart watch, a notebook PC, or a management server can be used. The smartphone 20 is communicably connected to the server 60 via a network N such as the Internet. The smartphone 20 forwards data (a log and the like containing information related to the history of the robot 10, and the like) acquired from the robot 10 to the server 60. The data accumulated in the server 60 is referenced as a backup, for example.

[0028] As Figure 2As shown, the main body 100 of the robot 10 has a head 101, a torso 103, and a connecting part 102 connecting the head 101 and the torso 103. The main body 100 has a drive unit 16 for moving the head 101 relative to the torso 103. The drive unit 16 has a torsion motor 161 and a vertical movement motor 162. The torsion motor 161 is a servo motor that rotates the head 101 and the connecting part 102 within a given angular range about a first rotation axis 161a extending in the extending direction of the connecting part 102. By actuating the torsion motor 161, the robot 10 achieves the activity of torsion of the head 101. The vertical movement motor 162 is a servo motor that rotates the head 101 within a given angular range about a second rotation axis 162a perpendicular to the first rotation axis 161a. By actuating the vertical movement motor 162, the robot 10 achieves the activity of vertical movement of the head 101. The direction of the up-and-down movement of the head 101 also becomes an inclined direction relative to the vertical direction based on the angle of the torsion of the head 101 based on the torsion motor 161. The robot 10 achieves the shaking or vibration of the head 101 by making the torsion motor 161 and / or the up-and-down movement motor 162 move in small, periodic motions. By appropriately changing and combining the timing, magnitude, and speed of the movements of the torsion motor 161 and the up-and-down movement motor 162, the robot 10 can perform various actions, such as joyful actions, surprised actions, and breathing actions that mimic biological breathing. Among these, the breathing action is a type of spontaneous movement based on the robot 10.

[0029] like Figure 2 As shown, the main body 100 includes a touch sensor 171, an accelerometer 172, a gyroscope 173, an illuminance sensor 174, a microphone 175, a sound output unit 15, and a power receiving coil 193. The touch sensor 171 is located on the upper part of the head 101, the upper part of the torso 103, and the side. The accelerometer 172, the gyroscope 173, and the power receiving coil 193 are located near the lower surface of the torso 103. The illuminance sensor 174 and the sound output unit 15 are located on the upper part of the torso 103. The microphone 175 is located on the upper part near the base of the head 101.

[0030] like Figure 3 As shown, robot 10 includes a CPU 11 (Central Processing Unit), RAM 12 (Random Access Memory), storage unit 13, operation unit 14, sound output unit 15, drive unit 16, sensor unit 17, communication unit 18, and power supply unit 19. All parts of robot 10 are connected via data transmission paths such as buses. Figure 3 All the functional structures shown are located in the main body 100.

[0031] The CPU 11 is a processor that controls the operation of the robot 10 by reading out and executing the program 131 stored in the storage 13 and performing various arithmetic processes. The robot 10 can have a plurality of processors (for example, a plurality of CPUs), and the plurality of processors can perform the plurality of processes performed by the CPU 11 of the present embodiment. In this case, the plurality of processors can participate in common processes, or the plurality of processors can independently perform different processes in parallel. The RAM 12 provides a memory space for the CPU 11 to use, and stores temporary data. The storage 13 is a non-transitory recording medium that can be read by the CPU 11 as a computer, and stores the program 131 and various data. Thus, the storage 13 includes a computer program product including the program 131. The storage 13 includes, for example, a non-volatile memory such as a flash memory. The program 131 is stored in the storage 13 in the form of a program code readable by a computer. The program 131 includes firmware for controlling each hardware of the robot 10. As data stored in the storage 13, there are action setting data 132 and the like. In the action setting data 132, the action content of the following actions is set: an interaction action performed by the robot 10 in accordance with the state of the robot 10, the content of a stimulus from the outside, and the like; a self-generation action performed by the robot 10 autonomously without depending on a stimulus from the outside; and a breathing action. The self-generation action can be seen as a quirky behavior of the robot 10, and thus can also be a quirky action. The settings involved in the action content include, for example, the setting of the action timing and the action amount of the twist motor 161 and the up-and-down motor 162 of the drive unit 16, and the setting of the pitch (highness or lowness), the duration, and the volume of the sound output by the sound output unit 15.

[0032] The operation unit 14 has an operation button and an operation knob or the like for turning the power on and off, adjusting the volume of the output sound of the sound output unit 15, and the like. The operation unit 14 outputs operation information corresponding to the input operation of the operation button and the operation knob or the like to the CPU 11. The sound output unit 15 has a speaker that outputs sound at a pitch (highness or lowness), a duration, and a volume corresponding to a control signal and sound data transmitted from the CPU 11. The sound can be a sound that imitates the call of an animal. The drive unit 16 causes the twist motor 161 and the up-and-down motor 162 described above to operate in accordance with a control signal transmitted from the CPU 11.

[0033] The sensor unit 17 includes the aforementioned touch sensor 171, accelerometer 172, gyroscope sensor 173, illuminance sensor 174, and microphone 175, and outputs the detection results based on each sensor and microphone 175 to CPU 11. Touch sensor 171 detects contact between the user or other objects and the robot 10. Touch sensor 171 may include, for example, a pressure sensor or a capacitive sensor, and outputs detection data regarding whether or not contact with the robot 10 has occurred to CPU 11. Accelerometer 172 detects acceleration in each of the three orthogonal axes and outputs the detection data to CPU 11. Gyroscope sensor 173 detects angular velocity in each of the three orthogonal axes and outputs the detection data to CPU 11. Illuminance sensor 174 detects the brightness around the robot 10 and outputs the detection data to CPU 11. Microphone 175 detects sound around the robot 10 and outputs the detected sound data to CPU 11.

[0034] The communication unit 18 is a communication module including an antenna, modem circuit, signal processing circuit, etc., and performs wireless data communication with the smartphone 20 according to the BLE communication standard. The power supply unit 19 includes a battery 191, a remaining power detection unit 192, and a receiving coil 193. The battery 191 supplies power to various parts of the robot 10. In this embodiment, the battery 191 is a rechargeable battery that can be recharged repeatedly via contactless charging. The remaining power detection unit 192 detects the remaining power of the battery 191 according to a control signal sent from the CPU 11 and outputs the detection result to the CPU 11. The charging operation of the battery 191 is performed with the robot 10 stored (set) inside a dedicated power supply unit (storage unit, charging dock) shown in the figure. The power supply unit includes a power supply coil, which charges the battery 191 by electromagnetic induction at a position opposite to the receiving coil 193 when the robot 10 is stored.

[0035] like Figure 4 As shown, the smartphone 20 includes a CPU 21 (processing unit), RAM 22, storage unit 23, display unit 24, operation unit 25, and communication unit 26. The various components of the smartphone 20 are connected via data transmission paths such as a bus. A display control device 200, which controls the display operation of the display unit 24, is configured using the CPU 21, RAM 22, and storage unit 23.

[0036] The CPU 21 is a processor that controls the operation of the smartphone 20 by reading out and executing programs such as the management application 231 stored in the storage section 23, and performs various arithmetic processing. The CPU 21 is an example of one or more processing sections. In addition, the smartphone 20 can have a plurality of processors (for example, a plurality of CPUs), and the plurality of processors can perform the plurality of processes performed by the CPU 21 of the present embodiment. In this case, one or more processing sections are configured by the plurality of processors. In this case, the plurality of processors can participate in common processing, or the plurality of processors can independently and in parallel perform different processing. The RAM 22 provides a memory space for the CPU 21 to use, and stores temporary data. The storage section 23 is a non-transitory recording medium that is readable by the CPU 21 of the computer, and stores programs such as the management application 231 and various data. Thus, the storage section 23 includes a computer program product including a program. The management of the robot 10 performed by the management application 231 refers to causing information related to the state of the robot 10 to be displayed on a given display section. The storage section 23 has, for example, a non-volatile memory such as a flash memory. As data stored in the storage section 23, there are state information 232, previous display information 233, and the like. The contents of the state information 232 and the previous display information 233 will be described later.

[0037] The display section 24 includes a display panel such as a liquid crystal panel capable of performing display based on a dot matrix method, and a drive circuit for the display panel. The display section 24 displays various menus, screens of the management application 231, and the like in accordance with a control signal transmitted from the CPU 21. The operation section 25 has an operation unit such as a touch panel provided so as to overlap the display panel of the display section 24, and an operation button, and outputs an operation signal corresponding to an operation on the operation unit to the CPU 21. The communication section 26 is a communication module having an antenna, a modulation / demodulation circuit, a signal processing circuit, and the like, and performs wireless data communication with the robot 10 in accordance with the communication standard of BLE. In addition, the communication section 26 transmits and receives voice data for telephone communication, packet data related to Internet connection, and the like with a base station.

[0038] Next, the operation of the robot management system 1 will be described. If the user performs an operation of instructing the startup of the management application 231 on the operation section 25 of the smartphone 20, the CPU 21 executes the management application 231 to cause it to start up. The display operation of the display section 24 described below is performed by the CPU 21 controlling the display section 24 in accordance with a given process performed by the management application 231. The management application 231 corresponds to a given application program for causing the state image 31 to be displayed on the display section 24. The CPU 21 acquires the state information 232 from the robot 10 in a case where the management application 231 is started up, and causes the display section 24 to display the state image 31 based on the state information 232. Figure 5The main screen 30 shown is displayed on the display unit 24. Alternatively, a given startup screen or welcome screen may be displayed before the main screen 30. On the main screen 30, the status image 31, growth days image 32, personality image 33, information icon 34, remaining battery power image 35, setting button 36, menu icon 37, and tab bar 38 are displayed in a given configuration. Details regarding these images, icons, etc., will be described later. Figure 5 The letter "A" in the name is the name given to robot 10 by the user on management application 231. Status image 31, growth days image 32, personality image 33, and remaining battery power image 35 represent the status of robot 10. Among these, growth days image 32 and personality image 33 are two ways of "updating information based on the robot's historical records." Thus, the main screen 30 contains various information related to the status of robot 10. By viewing the main screen 30, the user can obtain real-time information about the status of robot 10.

[0039] refer to Figure 6 This section explains the contents of the state information 232 referenced by the CPU 21 when the main screen 30 is displayed, and the elements E1 to E6 representing the state of the robot 10. The state information 232 contains data related to each of the elements E1 to E6 representing the state of the robot 10. Specifically, the state information 232 contains data representing the contents of each of the elements E1 to E6, and information about the time when each data is generated in the robot 10 (or the time when the smartphone 20 receives the data). Element E1 is the robot's "action mode." The robot 10 in this embodiment has action modes such as "normal mode," "deep sleep mode," and "sleep mode." "Normal mode" is a mode in which the robot 10 performs interactive actions based on external stimuli, or automatically generates actions when given conditions are met. "Deep sleep mode" is a mode in which the movement of the robot 10's head 101 and the output of sound from the sound output unit 15 are stopped. The "deep sleep mode" is described later in the setting screen 50 (see reference). Figure 15 The "sleep mode" is executed when the toggle switch 52 is switched on. The "sleep mode" is a mode of biological sleep that suppresses the robot 10's response to external stimuli. The "sleep mode" is executed, for example, when external stimuli (ambient light levels, etc.) meet given conditions. Both the "deep sleep mode" and the "sleep mode" are forms of "function suppression modes" in which the robot 10's functions are suppressed. Therefore, element E1 indicates whether the robot 10 is operating within a given function suppression mode. The aforementioned types of "operation modes" are just one example and are not limited to this.

[0040] The element E2 is "external stimulus" indicating a kind of stimulus received by the robot 10 from the outside. As the kind of external stimulus, there are, for example, "a loud voice", "a conversation", "a caress of the body", "a touch of the head", "a lift", "a turn upside down", "a swing", and the like, and the like, but are not limited thereto. The external stimulus related to a voice such as "a loud voice", "a conversation" is detected based on the detection data of the microphone 175. The external stimulus related to a contact such as "a caress of the body", "a touch of the head" is detected based on the detection data of the touch sensor 171. The external stimulus accompanied by a change in posture such as "a lift", "a turn upside down", and "a swing" is detected based on the detection data of the acceleration sensor 172 and the gyro sensor 173. The element E3 is "battery remaining capacity" of the battery 191. The "battery remaining capacity" is expressed in percentage with 100% set at the time of full charge. The "battery remaining capacity" is detected by the remaining capacity detection section 192.

[0041] The element E4 is an emotion parameter indicating a virtual emotion of the robot 10. The element E4 is constituted of "emotion value (X)" and "emotion value (Y)" (hereinafter, also collectively referred to as "emotion value"). The emotion value is calculated by the emotion value calculation section 193 based on the elements E1 to E3. The emotion value is calculated by the emotion value calculation section 193 based on the elements E1 to E3. Figure 7The position of the plotted point in the emotion map of the XY coordinate plane shown indicates the virtual emotion of the robot 10. The "emotion value (X)" is the position of the plotted point in the X-axis direction, and the greater the value, the higher the degree of comfort, and the smaller the value, the higher the degree of unease. The "emotion value (Y)" is the position of the plotted point in the Y-axis direction, and the greater the value, the higher the degree of excitement, and the smaller the value, the higher the degree of listlessness. The maximum value of the "emotion value (X)" is "200", and the minimum value is "-200". The maximum value of the "emotion value (Y)" is "200", and the minimum value is "-200". Thus, the emotion value is the coordinate of any point within the emotion region R, which is a square with a length of 400 on one side. The emotion region R is divided into nine square regions R1 to R9 arranged in a matrix of 3 x 3. The regions R1 to R9 each indicate a certain emotion of the robot 10. The region R1 that satisfies -200 ≤ X ≤ -67, 67 ≤ Y ≤ 200 indicates the emotion of "frustration". The region R2 that satisfies -66 ≤ X ≤ 66, 67 ≤ Y ≤ 200 indicates the emotion of "excitement". The region R3 that satisfies 67 ≤ X ≤ 200, 67 ≤ Y ≤ 200 indicates the emotion of "joy". The region R4 that satisfies -200 ≤ X ≤ -67, -66 ≤ Y ≤ 66 indicates the emotion of "unease". The region R5 that satisfies -66 ≤ X ≤ 66, -66 ≤ Y ≤ 66 indicates the emotion of "normal". The region R6 that satisfies 67 ≤ X ≤ 200, -66 ≤ Y ≤ 66 indicates the emotion of "comfort". The region R7 that satisfies -200 ≤ X ≤ -67, -200 ≤ Y ≤ -67 indicates the emotion of "sadness". The region R8 that satisfies -66 ≤ X ≤ 66, -200 ≤ Y ≤ -67 indicates the emotion of "listlessness". The region R9 that satisfies 67 ≤ X ≤ 200, -200 ≤ Y ≤ -67 indicates the emotion of "calm". The regions R1 to R4, R6 to R9 corresponding to the eight types of emotions other than "normal" further divide the respective interiors into ten level regions ("Lv1" to "Lv10") indicating ten stages of levels. In each of the regions R1 to R4, R6 to R9, the level regions of lower levels are arranged closer to the region R5 of "normal", and the level regions of higher levels are arranged farther from the region R5 of "normal". Hereinafter, sometimes the state of the emotion of the robot 10 is noted as "comfort Lv10" or the like by combining the type and the level of the emotion. In addition, the length of one side of the emotion region R and the regions R1 to R9 can also increase within a certain range corresponding to the growth of the robot 10. For example, the emotion region R can be set as a region of -100 ≤ X ≤ 100, -100 ≤ Y ≤ 100 in the initial state, and increase to a region of -200 ≤ X ≤ 200, -200 ≤ Y ≤ 200 according to the growth of the robot 10. The emotion value changes each time according to external stimuli or the like received by the robot 10. The amount of change in the emotion value at one time is selected from the following variables DXP, DXM, DYP, and DYM.

[0042] DXP: change amount in +X direction

[0043] DXM: change amount in -X direction

[0044] DYP: change amount in +Y direction

[0045] DYM: change amount in -Y direction

[0046] The variable DXP also represents ease of calmness, the variable DXM represents ease of anxiety, the variable DYP represents ease of excitement, and the variable DYM represents ease of listlessness. In the present embodiment, the initial values of the variables DXP, DXM, DYP, and DYM are "10". Further, in a case where the mood values reach the maximum values in the +X axis direction, -X axis direction, +Y axis direction, and -Y axis direction, respectively, the variables DXP, DXM, DYP, and DYM are increased by a given amount. In the present embodiment, the maximum values of the variables DXP, DXM, DYP, and DYM are "20".

[0047] Figure 6 The element E5 illustrated is a personality parameter representing a virtual personality of the robot 10. The element E5 is composed of a "personality value (cheerful)", a "personality value (shy)", a "personality value (lively)", and a "personality value (spoiled)" (hereinafter collectively referred to as "personality value"). The "personality value (cheerful)" is a value obtained by subtracting "10" from the variable DXP, and represents ease of change in the +X axis direction in the mood map, that is, ease of calmness. The "personality value (shy)" is a value obtained by subtracting "10" from the variable DXM, and represents ease of change in the -X axis direction in the mood map, that is, ease of anxiety. The "personality value (lively)" is a value obtained by subtracting "10" from the variable DYP, and represents ease of change in the +Y axis direction in the mood map, that is, ease of excitement. The "personality value (spoiled)" is a value obtained by subtracting "10" from the variable DYM, and represents ease of change in the -Y axis direction in the mood map, that is, ease of listlessness. Thus, each personality value changes in accordance with changes in the variables DXP, DXM, DYP, and DYM, and has an initial value of "0" and a maximum value of "10". The personality corresponding to the largest personality value among the four personality values is set as the personality of the robot 10 at the time point. For example, in the example illustrated, the "personality value (spoiled)" is the largest at "7", and thus the personality of the robot 10 at the time point is set to "spoiled". In addition, in a case where two or more personality values are the same and the largest, one personality is decided in accordance with a given priority order. In the present embodiment, the priority order of the personalities is set in the order of "cheerful", "lively", "shy", and "spoiled" from high to low. Figure 6

[0048] ​The element E6 is "days of cultivation" indicating the number of days (accumulated operation period) from the initial start day of the robot 10. The "days of cultivation" is counted to 5 digits in the inside of the robot 10. The elements E1, E2, E4 to E6 among the elements E1 to E6 are one way of information updated according to the history of the robot 10.

[0049] Each data of the elements E1 to E6 is generated by the CPU 11 of the robot 10 in sequence according to the operation condition of the robot 10, and stored to the storage section 13 of the robot 10 together with the time of generation. The CPU 21 of the smartphone 20 repeatedly acquires the elements E1 to E6 of the status information 232 from the robot 10 at a given frequency in the case of being connected by BLE communication with the robot 10, and updates the status information 232. In detail, the CPU 21 acquires and updates the data of the elements E1 to E4 among the status information 232 from the robot 10 at a frequency of 1 second. Further, the CPU 21 acquires and updates the data of the elements E5 and E6 among the status information 232 from the robot 10 at a frequency of 1 minute. The updating of the status information 232 in this way corresponds to the acquisition of the status information 232. In addition, the form of the status information 232 is not limited to the form shown. For example, the status information 232 can also be in the form of a queue in which the elements E acquired from the robot 10 are sequentially accumulated in time series. Figure 6

[0050] The CPU 21 of the smartphone 20 displays the main screen 30 of Figure 5 based on the latest status information 232 on the display section 24, or updates the main screen 30. For example, the CPU 21 displays the main screen 30 of Figure 5 ​As shown, a status image 31 is displayed in the substantially center of the main screen 30. The status image 31 contains an animation video that clearly indicates the status of the robot 10. In detail, the status image 31 contains an appearance image 311 that indicates a certain element of the status of the robot 10 by the appearance of the robot 10. The appearance of the actual robot 10, such as the color of the exterior 110, is reflected in the appearance image 311. Further, the status image 31 contains a virtual image 312 that indicates the appearance of the virtual image of the user. The appearance image 311 and the virtual image 312 are animation videos of a given length. The animation videos of the appearance image 311 and the virtual image 312 can be repeatedly displayed until the status image 31 is switched to another image, or can be repeatedly displayed a given number of times. Further, the status image 31 contains a text 313 that indicates a certain element of the status of the robot 10. The text 313 is displayed, for example, on the upper portion of the appearance image 311 and the virtual image 312. The certain element of the status of the robot 10 indicated by the status image 31 includes any one of whether the power of the robot 10 is on, whether the robot 10 is in communication connection with the smartphone 20, whether the robot 10 is operating in a function-suppressed mode (a dozing mode or a sleep mode), whether the robot 10 is in a state of being given a stimulus from the outside, the virtual emotion of the robot 10, and the virtual personality of the robot 10. The area of the display region of the status image 31 is larger than the area of the display region of each of the other growth day image 32, the personality image 33, and the battery remaining capacity image 35 (the area of the display region of information corresponding to an element other than the status image 31). The area of the display region of the status image 31 is the area of the smallest rectangle that encloses the appearance image 311, the virtual image 312, and the text 313, and is the area of a rectangle whose sides are parallel to the outline of the main screen 30.

[0051] The CPU 21 determines the status of the robot 10 from the status information 232, and selects and causes to display, on the display portion 24, the status image 31 corresponding to the determined status from among a plurality of status images 31 that indicate a plurality of statuses of the robot 10 that differ from each other in terms of the type (category, kind) of the robot 10. The plurality of status images 31 are generated in advance and stored in the storage portion 23 of the smartphone 20. In detail, the CPU 21 determines whether the robot 10 is in a certain status in order from the status with the highest priority among the plurality of statuses in accordance with the priority order set for the plurality of statuses. Then, the CPU 21 determines the certain status as the status of the robot 10 in a case where the robot 10 is determined to be in the certain status at the beginning, and causes the status image 31 corresponding to the certain status to be displayed on the display portion 24. In the present embodiment, the priority order of the status of the robot 10 is as shown in Table 1 below. Figure 8the state information 232, the CPU 21 displays the state image 31 corresponding to the state of the priority order "2" at the time of acquisition (update) of the element E2 of the state information 232. In the case where the robot 10 is not in the state of any one of the priority orders "1" and "2", the CPU 21 displays the state image 31 corresponding to the state of the priority order "3" at all times except at the start of the management application. At the start of the management application, the CPU 21 displays the state image 31 corresponding to the state of the priority order "4" in the case where the robot 10 is not in the state of any one of the priority orders "1" and "2", and displays the state image 31 corresponding to the state of the priority order "4" at all times except at the start of the management application in the case where the robot 10 is not in the state of any one of the priority orders "1" to "3". In the present embodiment, the case where the robot 10 is not in the state of the priority order "3" means the case where the coordinates of the emotion value in the element E4 of the state information 232 are not located inside any one of the regions R1 to R4 and R6 to R9, that is, means the case where the coordinates are located inside the region R5 of "normal". In the case where the robot 10 is not in the state of any one of the priority orders "1" to "4", the CPU 21 displays the state image 31 corresponding to the state of the priority order "5".

[0052] In the case of displaying or updating the state image 31, the CPU 21 first performs discrimination relating to the state of the priority order "1". In the case where the state information 232 discriminates that the robot 10 is any one of "power off / communication off", "deep sleep mode", and "sleep mode", the CPU 21 selects the state image 31 corresponding to the discriminated state among "power off / communication off", "deep sleep mode", and "sleep mode" from among the plurality of state images, and causes it to be displayed on the display section 24. In this case, discrimination of "deep sleep mode" and "sleep mode" is performed on the basis of the element E1 of the state information 232. As described above, the state image 31 corresponding to the state of the priority order "1" is displayed at all times in the case where the robot 10 is in the state of the priority order "1". Figure 9As shown, the three states of the priority order "1" are further decided on each other. "Power off / communication off" is assigned to the highest priority order "1-1", "deep sleep mode" is assigned to the next priority order "1-2", and "sleep mode" is assigned to the further next priority order "1-3". In the case where the robot 10 is in the state of "power off / communication off", the CPU 21 causes the state image 31 including the avatar image 312 shown on the left side of Fig. 12, and the text 313 to be displayed. That is, the state image 31 of this case does not contain the appearance image 311, and contains the avatar image 312 of the animation video of the virtual figure that is searching for the robot 10. Thereby, it is represented that the robot 10 is in the state of "power off / communication off". The text 313 contains the words indicating that the robot 10 cannot be found. In the case where the robot 10 is not in the state of "power off / communication off" and is in the state of "deep sleep mode", the CPU 21 causes the state image 31 including the appearance image 311 shown in the center of Fig. 13, and the avatar image 312, and the text 313 to be displayed. The appearance image 311 contained in this state image 31 contains the animation video of the robot 10 in deep sleep, and the avatar image 312 contains the animation video of the virtual figure that is taking care of the robot 10. The text 313 contains the words indicating that the robot 10 is in deep sleep. In the case where the robot 10 is not in either of the states of "power off / communication off" and "deep sleep mode" and is in the state of "sleep mode", the CPU 21 causes the state image 31 including the appearance image 311 shown on the right side of Fig. 14, and the avatar image 312, and the text 313 to be displayed. The appearance image 311 contained in this state image 31 contains the animation video of the robot 10 in sleep, and the avatar image 312 contains the animation video of the virtual figure that is sleeping together with the robot 10. The text 313 contains the words indicating that the robot 10 is in sleep. Figure 9 Figure 9 Figure 9

[0053] In the case where it is determined that the robot 10 is not in any of the states of "power off / communication off", "deep sleep mode", and "sleep mode", the CPU 21 determines whether the robot 10 is in the state of "being stimulated by an external stimulus" on the basis of the element E2 of the state information 232. In the case where it is determined that the robot 10 is in the state of "being stimulated by an external stimulus", the CPU 21 selects the state image 31 corresponding to the external stimulus to which the robot 10 is subjected from among the plurality of state images 31 and causes it to be displayed on the display section 24. For example, in the case where the external stimulus to which the robot 10 is subjected is "a loud sound", the CPU 21 causes the state image 31 including the appearance image 311 shown on the left side of Fig. 15, and the avatar image 312, and the text 313 to be displayed. The appearance image 311 contained in this state image 31 contains the animation video of the robot 10 that is being stimulated by a loud sound, and the avatar image 312 contains the animation video of the virtual figure that is trying to calm down the robot 10. The text 313 contains the words indicating that the robot 10 is being stimulated by a loud sound. In the case where the external stimulus to which the robot 10 is subjected is "a touch", the CPU 21 causes the state image 31 including the appearance image 311 shown on the right side of Fig. 15, and the avatar image 312, and the text 313 to be displayed. The appearance image 311 contained in this state image 31 contains the animation video of the robot 10 that is being stimulated by a touch, and the avatar image 312 contains the animation video of the virtual figure that is trying to calm down the robot 10. The text 313 contains the words indicating that the robot 10 is being stimulated by a touch. Figure 10 ​​​The appearance image 311 and the virtual avatar image 312 shown on the left are displayed in a status image 31. The appearance image 311 contains an animated video of a surprised robot 10, and the virtual avatar image 312 contains an animated video of a virtual avatar making a loud sound. Furthermore, when the external stimulus received by the robot 10 is "touching its body," the CPU 21 causes the... Figure 10 The centrally displayed appearance image 311 and virtual avatar image 312 are both state images. The appearance image 311 contains an animated video of the robot 10 enjoying being touched, and the virtual avatar image 312 contains an animated video of a virtual avatar touching the robot 10's body. Furthermore, in cases where the external stimulus received by the robot 10 is "reversed," the CPU 21 enables the... Figure 10 The appearance image 311 and the virtual avatar image 312 shown on the right are displayed as state images 31. The appearance image 311 contains an animated video of the robot 10 being upside down, and the virtual avatar image 312 contains an animated video of a virtual avatar holding the upside-down robot 10. Each state image 31 corresponding to an external stimulus does not contain text 313. The state images 31 corresponding to external stimuli are not limited to... Figure 10 The images shown are prepared for each external stimulus. When displaying state image 31 corresponding to the "state of being stimulated by an external stimulus," as shown... Figure 11 As shown, the CPU 21 records element E2 of the state information 232 referenced in the display in the previous display information 233. When the CPU 21 determines that the robot 10 is in a "state of being stimulated by an external stimulus" in the next determination, it will display the state image 31 corresponding to the external stimulus in the current case only if the element E2 of the state information 232 used in the determination is different from the element E2 of the previous case recorded in the previous display information 233.

[0054] If the CPU 21 determines that the robot 10 is not in a state of "being stimulated by external stimuli," it determines the "emotional state" of the robot 10 based on element E4 of the state information 232. Then, the CPU 21 selects the state image 31 corresponding to the determined "emotional state" from a plurality of state images 31 and displays it on the display unit 24. For example, if the robot 10's emotional value belongs to the "Reassurance Lv10" region, the CPU 21 displays the state image 31 containing the state image 31. Figure 12 The appearance image 311 and the status image 31 with text 313 shown on the left are displayed. The appearance image 311 of the status image 31 contains an animated video of the robot 10 feeling at ease. In addition, the text 313 contains words indicating that the robot 10 is at ease. Furthermore, when the robot 10's emotion value is in the "Joy Lv10" region, the CPU 21 enables the display of the status image 311. Figure 12the appearance image 311 and the text 313 shown in the right side of FIG. 10. The appearance image 311 of the state image 31 contains an animation video of the robot 10 that is happy. Further, the text 313 contains a word that indicates that the robot 10 is happy. In Figure 12 the virtual image 312 is omitted in FIG. 10, but the state image 31 corresponding to the "state of emotion" can further contain the virtual image 312. The state image 31 corresponding to the "state of emotion" is not limited to the image shown in Figure 12 FIG. 10, and is prepared for each combination of the kind and the level of emotion. Two or more texts 313 that are different from each other for each combination of the kind and the level of emotion can be prepared, and any one of these texts 313 is selected at random or according to a given rule and displayed. In the case where the state image 31 corresponding to the "state of emotion" is displayed, as shown in Figure 11 FIG. 10, the CPU 21 records the element E4 of the state information 232 to be referred to in the display in the previous display information 233. In the case where the CPU 21 displays the state image 31 corresponding to the "state of emotion" of the robot 10 next time, the CPU 21 displays the state image 31 corresponding to the state of emotion this time only in the case where the element E4 of the state information 232 at this point of time is different from the previous element E4 recorded in the previous display information 233.

[0055] In the case where the CPU 21 determines that the robot 10 is not in the "state of being stimulated from the outside" and does not display the state image 31 in the display section 24 after the start of the management application 231, the CPU 21 determines the "state of personality" of the robot 10 on the basis of the element E5 contained in the state information 232. Then, the CPU 21 selects the state image 31 corresponding to the determined "state of personality" from among a plurality of state images 31 and displays it in the display section 24. That is, in the case where the state image 31 displayed first after the start of the management application 231 is not any one of the "power off / communication off", the "deep sleep mode", the "sleep mode", and the "state of being stimulated from the outside", the CPU 21 displays the state image 31 corresponding to the "state of personality". For example, in the case where the personality of the robot 10 is "cheerful", the CPU 21 displays the state image 31 containing the appearance image 311 and the text 313 shown in the leftmost side of FIG. 11. The appearance image 311 of the state image 31 contains an animation video of the robot 10 that is cheerful. The text 313 contains a word that indicates that the robot 10 is cheerful. Figure 13 Further, in the case where the personality of the robot 10 is "lively", the CPU 21 displays the state image 31 containing the appearance image 311 and the text 313 shown in the leftmost side of FIG. 12. The appearance image 311 of the state image 31 contains an animation video of the robot 10 that is lively. The text 313 contains a word that indicates that the robot 10 is lively. Figure 13the appearance image 311 shown on the 2nd from the left and the state image 31 of the text 313 are displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a lively manner. The text 313 contains words indicating that the robot 10 is lively. Further, in a case where the personality of the robot 10 is "shy", the CPU 21 causes the state image 31 of the appearance image 311 shown on the 2nd from the right and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a shy manner. The text 313 contains words indicating that the robot 10 is shy. Further, in a case where the personality of the robot 10 is "spoiled", the CPU 21 causes the state image 31 of the appearance image 311 shown on the rightmost side and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a spoiled manner. The text 313 contains words indicating that the robot 10 is spoiled. In Figure 13 the appearance image 311 shown on the 2nd from the left and the state image 31 of the text 313 are displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a lively manner. The text 313 contains words indicating that the robot 10 is lively. Further, in a case where the personality of the robot 10 is "shy", the CPU 21 causes the state image 31 of the appearance image 311 shown on the 2nd from the right and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a shy manner. The text 313 contains words indicating that the robot 10 is shy. Further, in a case where the personality of the robot 10 is "spoiled", the CPU 21 causes the state image 31 of the appearance image 311 shown on the rightmost side and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a spoiled manner. The text 313 contains words indicating that the robot 10 is spoiled. In Figure 13 the appearance image 311 shown on the 2nd from the left and the state image 31 of the text 313 are displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a lively manner. The text 313 contains words indicating that the robot 10 is lively. Further, in a case where the personality of the robot 10 is "shy", the CPU 21 causes the state image 31 of the appearance image 311 shown on the 2nd from the right and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a shy manner. The text 313 contains words indicating that the robot 10 is shy. Further, in a case where the personality of the robot 10 is "spoiled", the CPU 21 causes the state image 31 of the appearance image 311 shown on the rightmost side and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a spoiled manner. The text 313 contains words indicating that the robot 10 is spoiled. In Figure 13 the appearance image 311 shown on the 2nd from the left and the state image 31 of the text 313 are displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a lively manner. The text 313 contains words indicating that the robot 10 is lively. Further, in a case where the personality of the robot 10 is "shy", the CPU 21 causes the state image 31 of the appearance image 311 shown on the 2nd from the right and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a shy manner. The text 313 contains words indicating that the robot 10 is shy. Further, in a case where the personality of the robot 10 is "spoiled", the CPU 21 causes the state image 31 of the appearance image 311 shown on the rightmost side and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a spoiled manner. The text 313 contains words indicating that the robot 10 is spoiled. In

[0056] As shown in Figure 5 the appearance image 311 shown on the 2nd from the left and the state image 31 of the text 313 are displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a lively manner. The text 313 contains words indicating that the robot 10 is lively. Further, in a case where the personality of the robot 10 is "shy", the CPU 21 causes the state image 31 of the appearance image 311 shown on the 2nd from the right and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a shy manner. The text 313 contains words indicating that the robot 10 is shy. Further, in a case where the personality of the robot 10 is "spoiled", the CPU 21 causes the state image 31 of the appearance image 311 shown on the rightmost side and the text 313 to be displayed. The appearance image 311 of the state image 31 contains an animation video of the robot 10 in a spoiled manner. The text 313 contains words indicating that the robot 10 is spoiled. In Figure 14The detailed information screen 40 shown is displayed on the display section 24. The detailed information screen 40 contains detailed information relating to a certain element of the state of the robot 10 (here, the personality). On the detailed information screen 40, the personality 41 of the robot 10 at this time point, a chart 42 that represents the personality value of each of the four personalities in 11 stages from "0" to "10", and a button 43 for closing the detailed information screen 40 are displayed. In addition, the personality of the robot 10 can be set to be updated once a day, for example, at a timing at which the date changes. Figure 5 The battery remaining capacity image 35 shown is an image that represents the battery remaining capacity of the storage battery 191 in three stages. In the case where the storage battery 191 is in the middle of charging, a given charging-in-progress mark can also be further displayed on the battery remaining capacity image 35. Further, in the case where the non-contact charging of the storage battery 191 is not properly performed in a state where the robot 10 is stored in the power supply, a given charging error mark can also be further displayed on the battery remaining capacity image 35. In the case where the operation of the selection setting button 36 is performed, the CPU 21 causes the display section 24 to display the setting screen 50 shown in Fig. 6. Figure 15 The setting screen 50 shown is displayed on the display section 24. On the setting screen 50, a slider 51 for adjusting the volume of the sound (meowing sound) output from the sound output section 15 of the robot 10, a toggle switch 52 that is switched on in the case where the robot 10 is moved to the deep sleep mode, an update button 53 for updating the firmware of the robot 10, and a list button 54 for displaying a list of the robots 10 that cooperate with the smartphone 20 are displayed.

[0057] In Figure 5 The menu mark 37 is displayed in the upper left corner of the home screen 30 shown. If the operation of selecting the menu mark 37 is performed, the CPU 21 causes the menu screen, which is not shown in the drawing, to be displayed on the display section 24. From the menu screen, a screen for editing the user's profile, a list screen of the robots 10 that cooperate, a screen for newly registering a robot 10 (that cooperates), a screen for managing the version of the application 231, and the like can be further caused to be displayed. The tab bar 38 is displayed in the lowermost portion of the home screen 30. The tab bar 38 contains a home icon 381 and an interaction record icon 382. If the interaction record icon 382 is selected in a state where the home screen 30 is being displayed, the CPU 21 causes the display in the display section 24 to be transitioned from the home screen 30 to the interaction record screen, which is not shown in the drawing. In the interaction record screen, information relating to the history of the interaction of the robot 10 with the user is displayed. If the home icon 381 is selected in a state where the interaction record screen is being displayed, the CPU 21 causes the display in the display section 24 to be transitioned from the interaction record screen to the home screen 30.

[0058] Next, with reference to Figure 16 and Figure 17, to explain the main screen display processing performed by the CPU 21 in order to realize the above-described operation. The main screen display processing is started in a case where an operation of causing the operation section 25 to start the management application 231 is performed. If the main screen display processing is started, the CPU 21 sets a start display completion flag to "invalid" (step S1). The start display completion flag is one-bit data stored in the RAM 22, and "0" indicates "invalid" and "1" indicates "valid". A state in which the start display completion flag is "invalid" indicates that a state image 31 other than the "standby state" has not been displayed after the start of the management application 231, and a state in which the start display completion flag is "valid" indicates that a state image 31 other than the "standby state" has been displayed after the start of the management application 231. The CPU 21 causes the main screen 30 to be displayed on the display section 24 (step S2). At this stage, the state image 31, the growth day image 32, the personality image 33, and the battery remaining capacity image 35 can be set to a state in which they are not displayed. The CPU 21 determines whether or not the robot 10 is in a communication connection by BLE communication (step S3). In a case where it is determined that the robot 10 is not in the communication connection (NO in step S3), the CPU 21 causes the state image 31 corresponding to the "power off / communication off" to be displayed on the main screen 30 (step S4). Here, the CPU 21 selects the state image 31 corresponding to the "power off / communication off" from among the plurality of state images 31 stored in the storage section 13, and acquires image data of the state image 31 from the storage section 13. Then, the CPU 21 causes the selected state image 31 to be displayed on the display section 24 by transmitting the image data to the display section 24 together with a control signal. The processing in the display of the state image 31 described later is also the same except for the kind of the selected state image 31. In addition, in a case where the power of the robot 10 is turned off, since the communication connection with the smartphone 20 is not performed either, the branch to "NO" in step S3 is made. If step S4 ends, the CPU 21 causes the processing to proceed to step S27.

[0059] In a case where it is determined that the robot 10 is in the communication connection (YES in step S3), the CPU 21 acquires given elements of the state information 232 from the robot 10 (step S5). Here, in a case where it is a timing at which the elements E1 to E4 are acquired once every one second, the CPU 21 acquires the elements E1 to E4 from the robot 10. Further, in a case where it is a timing at which the elements E5 and E6 are acquired once every one minute, the CPU 21 acquires the elements E5 and E6 (and the elements E1 to E4) from the robot 10. The CPU 21 updates the growth day image 32, the personality image 33, and the battery remaining capacity image 35 of the main screen 30 based on the latest state information 232 (display is performed in a case where it is not displayed) (step S6).

[0060] The CPU 21 determines whether the robot 10 is in the doze mode based on the element El of the state information 232 (step S7). In the case where the robot 10 is determined to be in the doze mode (YES in step S7), the CPU 21 causes the state image 31 corresponding to the doze mode to be displayed on the display portion 24 (step S8). Thereafter, the CPU 21 rewrites the start display completion flag to "in effect" (step S9), and causes the processing to proceed to step S27. In the case where the start display completion flag has already been rewritten to "in effect", the CPU 21 omits step S9 (the same also applies to steps S12, S16, S20 hereinafter). In the case where the robot 10 is determined not to be in the doze mode (NO in step S7), the CPU 21 determines whether the robot 10 is in the sleep mode based on the element El of the state information 232 (step S10). In the case where the robot 10 is determined to be in the sleep mode (YES in step S10), the CPU 21 causes the state image 31 corresponding to the sleep mode to be displayed on the display portion 24 (step Sll). Thereafter, the CPU 21 rewrites the start display completion flag to "in effect" (step S12), and causes the processing to proceed to step S27.

[0061] In the case where the robot 10 is determined not to be in the sleep mode (NO in step S10), the CPU 21 determines whether the detection time of the external stimulus received by the robot 10 is within a given time based on the element E2 of the state information 232 and the time information thereof (step S13). The given time is, for example, 30 seconds. In the case where the detection time of the external stimulus is determined to be within the given time (YES in step S13), the CPU 21 refers to the previous display information 233, and determines whether the present external stimulus is different from the external stimulus recorded in the state information 232 (step S14). Here, the CPU 21 determines that the present external stimulus is different from the external stimulus recorded in the previous display information 233 in the case where at least one of the content of the external stimulus and the time thereof in the latest state information 232 is different from the external stimulus recorded in the previous display information 233, or in the case where the previous display information 233 does not record the element E2. In the case where the present external stimulus is determined to be different from the external stimulus recorded in the previous display information 233 (YES in step S14), the CPU 21 causes the state image 31 corresponding to the detected external stimulus to be displayed on the display portion 24 (step S15). Thereafter, the CPU 21 rewrites the start display completion flag to "in effect" (step S16), and causes the processing to proceed to step S27.

[0062] If it is determined that the time since the detection of the external stimulus is not within a given time period (including the case where no external stimulus is detected) ("No" in step S13), or if it is determined that the external stimulus this time is the same as the external stimulus recorded in the previous display information 233 ("No" in step S14), the CPU 21 determines whether the start display completion mark is "effective" ( Figure 17 (Step S17). If the startup display completion flag is determined to be "failed", that is, if the status image 31 of "power off / communication off", "deep sleep mode", "sleep mode" or "state under external stimulation" is not displayed after the management application 231 is started ("No" in step S17), the CPU 21 determines whether the robot 10's personality has been acquired (step S18). Here, if any of the personality values ​​of element E5 in the status information 232 is "1" or above, the CPU 21 determines that the robot 10's personality has been acquired; if all personality values ​​are "0", the CPU 21 determines that the robot 10's personality has not been acquired. If it is determined that the robot 10's personality has been acquired ("Yes" in step S18), the CPU 21 displays the status image 31 corresponding to the robot 10's personality on the display unit 24 (step S19). Afterwards, the CPU 21 rewrites the startup display completion flag to "effective" (step S20) and moves the process to step S27.

[0063] In the case where it is determined in step S17 that the display completion flag is started ("Yes" in step S17), the CPU 21 determines whether or not there is an update of the coordinates of the emotion based on the element E4 of the state information 232 (step S21). Here, the CPU 21 determines that there is an update of the coordinates of the emotion in the case where the difference between the time of the element E4 of the state information 232 and the current time is within a given time (for example, within 12 seconds) and the coordinates of the element E4 of the state information 232 are different from the coordinates of the element E4 recorded in the previous display information 233. In the case where it is determined that there is an update of the coordinates of the emotion ("Yes" in step S21), the CPU 21 determines whether or not the coordinates of the emotion in the element E4 of the state information 232 are in a region other than "normal", that is, whether or not they are inside any one of the regions R1 to R4 and R6 to R9 (step S22). In the case where it is determined that the coordinates of the emotion are in a region other than "normal" ("Yes" in step S22), the CPU 21 causes the state image 31 corresponding to the region to which the coordinate value of the element E4 of the state information 232 belongs to be displayed on the display section (step S23). On the other hand, in the case where it is determined that the coordinates of the emotion are in the region R5 of "normal" ("No" in step S22), the CPU 21 determines whether or not the personality of the robot 10 is acquired (step S24). In the case where it is determined that the personality is acquired ("Yes" in step S24), the CPU 21 causes the state image 31 corresponding to the personality of the robot 10 to be displayed on the display section 24 (step S25). In the case where it is determined that there is no update of the coordinates of the emotion ("No" in step S21), or in the case where it is determined that the personality is not acquired ("No" in steps S18 or S24), the CPU 21 causes the state image 31 corresponding to the standby state to be displayed on the display section (step S26). In the case where any one of steps S23, S25, and S26 ends, the CPU 21 causes the process to proceed to step S27.

[0064] In step S27, the CPU 21 repeatedly determines whether or not the animation video of the appearance image 311 and / or the virtual image 312 of the state image 31 ends. In the case where it is determined that the animation video ends ("Yes" in step S27), the CPU 21 determines whether or not the operation of the end management application 231 is performed (step S28). The CPU 21 causes the process to return to step S3 in the case where it is determined that the operation is not performed ("No" in step S28), and causes the main screen display process and the management application 231 to end in the case where it is determined that the operation is performed ("Yes" in step S28). Figure 16

[0065] ​As above, in the display control method according to the present embodiment, the CPU 21 executes processing of sequentially determining whether the robot 10 is in a state from among a plurality of states respectively set in accordance with priority orders that are different from each other with respect to types of the robot 10, in the order of the priority orders from a state with a high priority order among the plurality of states. The CPU 21 executes processing of determining the state of the robot 10 to be the state in which the robot 10 is determined to be in at the first time. The CPU 21 executes processing of selecting a state image 31 corresponding to the determined state from among a plurality of state images 31 that represent the plurality of states and respectively include animation videos. The CPU 21 executes processing of causing the selected state image 31 to be displayed on the display section 24.

[0066] According to the present disclosure, the state of the robot 10 can be visually and intuitively and easily understood by the state image 31 including the animation video. Further, in a case where the state of the robot 10 is important for the user, the user can be timely notified. Further, since the state image 31 corresponding to a state with a relatively low degree of importance is also displayed in accordance with the state of the robot 10, various state images 31 can be displayed to attract the interest of the user.

[0067] Further, the CPU 21 executes processing of selecting a state image 31 corresponding to the state determined to be any one of a power-off state in which the power source of the robot 10 is turned off, a communication-off state in which the robot 10 is not communicatively connected to the smartphone 20, and a function-suppressed state in which the robot 10 is operating in a deep sleep mode or a sleep mode (function-suppressed mode) from among the plurality of state images 31 on the basis of the state information 232, and causing the selected state image 31 to be displayed on the display section 24. Thereby, in a case where the state of the robot 10 is any one of the power-off state, the communication-off state, the deep sleep mode, and the sleep mode that are important for the user, the user can be timely notified.

[0068] Further, the CPU 21 executes processing of determining whether the robot 10 is in a state in which the robot 10 is subjected to a given stimulus from the outside on the basis of the state information 232 in a case where it is determined that the state is not any one of the power-off state, the communication-off state, and the function-suppressed state. The CPU 21 executes processing of selecting a state image 31 corresponding to the stimulus to which the robot 10 is subjected from among the plurality of state images 31 in a case where it is determined that the robot 10 is in the state in which the robot 10 is subjected to the stimulus. The CPU 21 executes processing of causing the selected state image 31 to be displayed on the display section 24. Thereby, the user can be timely shown the state of the robot 10 corresponding to the communication with the user, and a change thereof.

[0069] Further, the state information 232 includes an emotion value (element E4, emotion parameter) indicating a virtual emotion of the robot 10. The CPU 21 executes processing of determining a state of the emotion of the robot 10 based on the emotion value included in the state information 232 in a case where it is determined that the robot 10 is not in a state of being stimulated, selecting a state image 31 corresponding to the determined state of the emotion from among the plurality of state images 31, and causing the selected state image 31 to be displayed on the display section 24. Thereby, it is possible to easily grasp the state of the emotion that is difficult to confirm from the appearance of the robot 10 according to the state image 31.

[0070] Further, the state information 232 includes a personality value (element E5, personality parameter) indicating a virtual personality of the robot 10. The CPU 21 executes processing of determining a state of the personality of the robot 10 based on the personality value included in the state information 232 in a case where it is determined that the robot 10 is not in a state of being stimulated and a specific state image 31 is not caused to be displayed on the display section 24 after the start of the management application. The CPU 21 executes processing of selecting a state image 31 corresponding to the determined state of the personality from among the plurality of state images 31, and causing the selected state image 31 to be displayed on the display section 24. Thereby, it is possible to easily grasp the state of the personality that is difficult to confirm from the appearance of the robot 10 according to the state image 31.

[0071] Further, the animation video of the state image 31 includes an appearance image 311 indicating an appearance of the robot 10. Thereby, it is possible to visually and intuitively and easily understand the state of the robot 10.

[0072] Further, the animation video of the state image 31 includes a virtual image 312 indicating an appearance of a virtual avatar of the user. Thereby, it is possible to visually and intuitively and easily understand the state of the communication of the robot 10 with the user.

[0073] Further, at least a part of the plurality of state images 31 includes a text 313 indicating a state of the robot 10. Thereby, it is possible to further explicitly show the state of the robot 10.

[0074] Further, the CPU 21 executes processing of causing the state image 31, and a growth day image 32 and a personality image 33 which are information related to the robot 10 other than the state image 31 and which are information updated according to a history of the robot 10 to be displayed in a given arrangement in the display section 24 based on the state information 232. By causing these state image 31, growth day image 32, and personality image 33 each reflecting a plurality of elements related to the state of the robot 10 to be displayed on the home screen 30, it is possible to easily and from multiple angles grasp the state of the robot 10. Further, by the state image 31 including an animation video, it is possible to visually and intuitively and easily understand the state of the robot 10.

[0075] Further, the area of the display region of the state image 31 is larger than the area of each of the display regions of the growth day image 32 and the personality image 33. Thereby, the state image 31 including the moving image can be made more easily seen by the user.

[0076] Further, the home screen 30 includes at least one of the growth day image 32 indicating the length of the cumulative moving period of the robot 10 and the personality image 33 indicating the virtual personality of the robot 10. Thereby, the growth day and / or the personality which are difficult to confirm from the appearance of the robot 10 can be grasped from the image separate from the state image 31 at all times.

[0077] Further, the CPU 21 performs processing of causing the information mark 34 and the state image 31, the growth day image 32, and the personality image 33 to be displayed on the display section 24, and causing the detailed information screen 40 related to the personality (certain element) of the robot 10 to be displayed on the display section 24 in a case where an operation of selecting the information mark 34 is performed. Thereby, the user can grasp the detailed information related to the personality of the robot 10 in a timely manner by the simple operation of selecting the information mark 34.

[0078] Further, the CPU 21 performs processing of causing the setting button 36 and the state image 31, the growth day image 32, and the personality image 33 to be displayed on the display section 24, and causing the setting screen 50 in which the moving of the robot 10 is set to be displayed on the display section 24 in a case where an operation of selecting the setting button 36 is performed. Thereby, the user can cause the setting screen 50 to be displayed by the simple operation of selecting the setting button 36, and thereby set the moving of the robot 10.

[0079] Further, the CPU 21 performs processing of repeatedly acquiring each of the elements of the state information 232 at a given frequency, and updating the home screen 30 based on the latest state information 232 acquired. Thereby, the real-time state of the robot 10 can be reflected in the home screen 30.

[0080] Further, the robot management system 1 according to the present embodiment includes the robot 10 and the display control device 200 described above. Further, the robot management system 1 according to the present embodiment includes the server 60 and the display control device 200 having the CPU 21 that performs the processing described above. Thereby, the state of the robot 10 can be easily grasped.

[0081] In addition, the present disclosure is not limited to the above-described embodiments, and various modifications can be made. For example, in the above-described embodiments, a manner in which the smartphone 20 causes the home screen 30 to be displayed by performing various processing in accordance with the management application 231 is exemplified, but is not limited thereto. For example, the server (not illustrated) provided outside the smartphone 20 can cause the home screen 30 to be displayed. Figure 1The server 60 and the like transmits data for displaying the main screen 30 on the display section 24 to the smartphone 20, and controls the display section 24 of the smartphone 20. In this case, the computer of the server executes an information processing method of data for causing the CPU 21 of the other computer to execute the following processing. The above data is "data for causing the CPU 21 (the other computer) to execute the following processing: based on the state information 232 related to the state of the robot 10, determining whether the robot 10 is in each of a plurality of states respectively set in priority order different from each other for types of the robot 10, from a state with high priority among the plurality of states in the order of priority; in a case where it is initially determined that the robot 10 is in a certain state, determining the certain state as the state of the robot 10; selecting a state image 31 corresponding to the determined state from a plurality of state images 31 each representing the plurality of states and including an animation video; and causing the selected state image 31 to be displayed on the display section 24". The data can include data such as image data that specifies the content and configuration of the main screen 30, HTML (HyperText Markup Language) data, and the like. Further, the data can include control information for controlling the action of the display section 24. Further, the data can be a program for causing the main screen 30 to be displayed on the display section 24. The computer of the server can also execute at least a part of the processing executed by the CPU 21 in the above embodiment. In this case, the computer of the server can correspond to "one or more processing sections". Further, the "one or more processing sections" can be constituted by the CPU 21 of the display control device 200 and the computer of the server.

[0082] Further, the elements indicating the state of the robot 10 are not limited to Figure 6 The example elements E1 to E6 can also be set to arbitrary elements according to the use of the robot 10 and the like. For example, as elements updated according to the history of the robot 10, elements such as fatigue, drowsiness, and physical condition can be set, and state images 31 and other images indicating these elements can be displayed on the main screen 30.

[0083] Further, the priority order of the state of the robot 10 is not limited to Figure 8 As illustrated, it can be appropriately changed according to the use of the robot 10 and the like. That is, the CPU 21 can determine whether the robot 10 is in any of a plurality of states in the order of priority from high to low according to the priority order of the plurality of states arbitrarily determined, and cause the state image 31 corresponding to a certain state to be displayed in a case where it is determined that the robot 10 is in the certain state. The priority order of the plurality of states can be changed according to the operation of the user. Further, the CPU 21 can also decide the state image 31 displayed on the display section 24 regardless of the priority order. For example, the CPU 21 can also cause the state image 31 corresponding to a certain state to be displayed in a case where it is determined that the robot 10 is in the certain state, regardless of the priority order. Figure 8The state image 31 displayed in the last state among the states shown is displayed.

[0084] Further, the animation video of the avatar image 312 of the state image 31 can be omitted, and further, the text 313 in the state image 31 can be omitted, and the state image 31 can be constituted only by the animation video of the appearance image 311.

[0085] Further, the manner in which the state image 31 is displayed on the display 24 of the smartphone 20 is exemplified, but is not limited thereto. For example, in a case where the robot 10 has a display, the home screen 30 (or a portion including the state image 31 among the home screen 30) can be displayed on the display. In this case, the control involved in the display of the home screen 30 can be executed by the CPU 11 of the robot 10, or can be executed remotely by the processing section of the external device such as the CPU 21 of the smartphone 20.

[0086] Further, the structure of the robot 10 is not limited to Figures 1-3 the exemplification. For example, the robot can be a robot that imitates a living being that actually exists, such as a human, an animal, a bird, a fish, or the like, a robot that imitates a living being that does not actually exist, such as a dinosaur, or the like, or a robot that imitates a living being that is not grounded in reality, or the like.

[0087] Further, in the above-described embodiment, the robot 10 is exemplified as the "object", but is not limited thereto. The "object" can be arbitrary as long as it is a management object of the management application 231. For example, the "object" can be an arbitrary thing whose state-indicating parameter is changed. Further, the "object" can be an avatar that acts in place of a user in a virtual space such as a meta universe.

[0088] Further, in the above description, an example in which the flash memory of the storage 13, 23 is used as a computer-readable medium of the program related to the present disclosure is disclosed, but is not limited to this example. As other computer-readable media, an information recording medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a CD-ROM, or the like can be used. Further, as a medium that provides data of the program related to the present disclosure via a communication line, a carrier wave is also used in the present disclosure. Further, the details of the structure and the details of the operation of each of the constituent elements of the robot 10 and the smartphone 20 in the above-described embodiment can be appropriately changed without departing from the scope of the present disclosure. The embodiments of the present disclosure are described, but the scope of the present disclosure is not limited to the above-described embodiments, and includes the scope of the invention recited in the claims and the equivalent scope thereof.

Claims

1. A computer program product that causes a computer's processing unit to perform the following processing: Based on the state information involved in the object's state, according to the priority order set for multiple states with different types of the object, the object is sequentially determined to be in the state with the highest priority among the multiple states. If the object is initially determined to be in a certain state, then that certain state is determined to be the state of the object. Select the state image corresponding to the determined state from a plurality of state images that represent the plurality of states and each contains an animated video; and The selected state image is displayed on the display unit.

2. The computer program product according to claim 1, wherein, The computer program product causes the computer's processing unit to perform the following processing: If, based on the state information, it is determined that the object is in a power-off state (power supply is disconnected), a communication disconnect state (the object and the given terminal device are not connected), or a function-suppressed state (the object is operating in a given function-suppressed mode), the state image corresponding to the determined state among the power-off state, the communication disconnect state, and the function-suppressed state is selected from the plurality of state images. and The selected status image is displayed on the display unit.

3. The computer program product according to claim 2, wherein, The computer program product causes the computer's processing unit to perform the following processing: If it is determined that the state is not the power-off state, the communication-disconnected state, or the function-suppressed state, the state information is used to determine whether the object is in a state where it is subjected to a given stimulus from the outside. If it is determined that the object is in a state of being stimulated, select the state image corresponding to the stimulus received by the object from the plurality of state images; The selected status image is displayed on the display unit.

4. The computer program product according to claim 3, wherein, The state information includes emotion parameters that represent the virtual emotions of the object. The computer program product causes the computer's processing unit to perform the following processing: If it is determined that the object is not in a state where it has been stimulated, the emotional state of the object is determined based on the emotional parameters contained in the state information; Select the state image corresponding to the determined state of the emotion from the plurality of state images; and The selected status image is displayed on the display unit.

5. The computer program product according to claim 4, wherein, The status information includes personality parameters that represent the virtual personality of the object. The computer program product causes the computer's processing unit to perform the following processing: If it is determined that the object is not in a state of being stimulated and the specific state image is not displayed on the display after the program for displaying the state image on the display is started, the state of the object's personality is determined based on the personality parameters contained in the state information; Select the state image corresponding to the determined state of the personality from the plurality of state images; and The selected status image is displayed on the display unit.

6. The computer program product according to claim 1, wherein, The computer program product causes the computer's processing unit to perform the following processing: If, based on the state information, it is determined that the object is in a state where it has received a given stimulus from the outside, a state image corresponding to the stimulus received by the object is selected from the plurality of state images; and The selected status image is displayed on the display unit.

7. The computer program product according to claim 1, wherein, The state information includes emotion parameters that represent the virtual emotions of the object. The computer program product causes the computer's processing unit to perform the following processing: The emotional state of the object is determined based on the emotional parameters contained in the state information; Select the state image corresponding to the determined state of the emotion from the plurality of state images; and The selected status image is displayed on the display unit.

8. The computer program product according to claim 1, wherein, The status information includes personality parameters that represent the virtual personality of the object. The computer program product causes the computer's processing unit to perform the following processing: The personality state of the object is determined based on the personality parameters contained in the status information; and Select a state image from the plurality of state images that corresponds to the determined state of the personality, and display the selected state image on the display unit.

9. The computer program product according to claim 1, wherein, The animated video contains appearance images that represent the appearance of the object.

10. The computer program product according to claim 9, wherein, The animated video contains virtual avatar images representing the appearance of the user's virtual avatar.

11. The computer program product according to claim 1, wherein, At least a portion of the plurality of state images contain text representing the state of the object.

12. The computer program product according to any one of claims 1 to 11, wherein, The computer program product causes the computer's processing unit to perform the following processing: Based on the state information, the state image and information related to the object that differs from the state image, i.e., information updated according to the object's historical records, are displayed on the display unit in a given configuration.

13. The computer program product according to claim 12, wherein, The area of ​​the display area of ​​the status image is larger than the area of ​​the display area of ​​the information.

14. The computer program product according to claim 12, wherein, The information includes at least one of information representing the length of the cumulative action period of the object and information representing the virtual personality of the object.

15. The computer program product according to claim 12, wherein, The computer program product causes the computer's processing unit to perform the following processing: The first identification image, the status image, and the information are displayed together on the display unit; and When the operation of selecting the first identifier image is performed, detailed information related to a certain element of the object's state is displayed on the display unit.

16. The computer program product according to claim 12, wherein, The computer program product causes the computer's processing unit to perform the following processing: The second identification image, the status image, and the information are displayed together on the display unit; When the operation of selecting the second identifier image is performed, a setting screen for setting the action of the object is displayed on the display unit.

17. The computer program product according to claim 12, wherein, The computer program product causes the computer's processing unit to perform the following processing: The state information is repeatedly acquired at a given frequency, and the screen containing the state image and the information is updated based on the latest acquired state information.

18. A system comprising an object and a display control device, The system's processing unit: Based on the state information involved in the state of the object, according to the priority order set for multiple states with different types of the object, the object is sequentially determined to be in the state with the highest priority among the multiple states. If the object is initially determined to be in a certain state, then that certain state is determined to be the state of the object. Select the state image corresponding to the determined state from a plurality of state images that represent the plurality of states and each contains an animated video; and The selected state image is displayed on the display unit.

19. A system comprising a server and a display control device, The system's processing unit: Based on the state information involved in the object's state, according to the priority order set for multiple states with different types of the object, the object is sequentially determined to be in the state with the highest priority among the multiple states. If the object is initially determined to be in a certain state, then that certain state is determined to be the state of the object. Select the state image corresponding to the determined state from a plurality of state images that represent the plurality of states and each contains an animated video; and The selected state image is displayed on the display unit.

Images