Computer program product and system

Abstract

The present disclosure provides computer program products and systems. A computer program product causes a processing unit of a computer to execute: a process for determining a change timing at which a state of an object changes from a certain state to a predetermined state on the basis of history information relating to a history of the state of the object; and causing a display unit to display a coordinate axis indicating the elapse of time and an index disposed at a position corresponding to the change timing in the coordinate axis and indicating that the state of the object changes to the predetermined state.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to and the benefit of Japanese Patent Application No. 2024-177112, filed October 9, 2024. The specification of Japanese Patent Application No. 2024-177112 is hereby incorporated by reference in its entirety into this specification. TECHNICAL FIELD

[0003] The present disclosure relates to a computer program product and a system. BACKGROUND

[0004] Conventionally, a robot that is capable of having a virtual exchange with a user by performing various actions according to its state is known (for example, Japanese Patent Application Publication No. 2002-59389).

[0005] However, the state of an object such as a robot is maintained as an internal parameter of the object, and thus there is a problem that it is not necessarily easy to accurately grasp a history of the state from the appearance of the object. SUMMARY

[0006] The present disclosure is directed to improving and solving such a situation, and aims to enable easy grasping of a history of a state of an object.

[0007] The computer program product according to the present disclosure causes a processing section of a computer to execute the following processing: determining a change timing at which a state of an object changes from a certain state to a given state, based on history information related to a history of the state of the object; and causing a display section to display a coordinate axis that represents the passage of time and an index that is arranged at a position in the coordinate axis that corresponds to the change timing, indicating that the state of the object changed to the given state. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a diagram that represents the structure of a robot management system.

[0009] Figure 2 is a schematic diagram that represents the structure of a main body of a robot.

[0010] Figure 3 is a block diagram that represents the functional structure of a robot.

[0011] Figure 4 is a diagram that represents the content of a log.

[0012] Figure 5 is a diagram that represents an emotion map.

[0013] Figure 6 is a block diagram that represents the functional structure of a smartphone.

[0014] Figure 7 is a view showing the contents of the state information.

[0015] Figure 8 is a view showing the home screen.

[0016] Figure 9 is a view showing the detailed information screen.

[0017] Figure 10 is a view showing the interaction record screen.

[0018] Figure 11 is a flowchart showing the control steps of the acquisition information display processing.

[0019] Figure 12A is a view showing the change of the mood in the case where the mood icon is displayed.

[0020] Figure 12B is a view showing the mood icon.

[0021] Figure 13 is a view showing the determination method of the displayed mood icon.

[0022] Figure 14 is a flowchart showing the control steps of the mood icon display processing.

[0023] Figure 15 is a view showing the display action of the timeline information related to the sleep state.

[0024] Figure 16 is a view showing the display action of the timeline information related to the sleepless state.

[0025] Figure 17 is a flowchart showing the control steps of the timeline display processing.

[0026] Figure 18 is a view showing the state where the evaluation screen is displayed in the home screen.

[0027] Figure 19 is a flowchart showing the control steps of the evaluation screen display processing. DETAILED DESCRIPTION

[0028] Hereinafter, the embodiments of the present disclosure will be described based on the drawings. As used herein, the following terms have the following meanings. Figure 1As shown, the robot management system 1 (display system, system) is provided with a robot 10 (object), a smartphone 20 (terminal device), and a server 60. The robot 10 is provided with a main body 100 and an exterior 110 that covers the entire surface of the main body 100. The robot 10 is a pet robot that imitates a small living creature. The robot 10 is capable of performing a plurality of actions that imitate the behavior of a living creature. The exterior 110 is composed of a material having flexibility and deforms according to the movement of the main body 100. The exterior 110 has, for example, a fur formed of a plush fabric, a decorative member that imitates an eye, and the like. The smartphone 20 is communicably connected to the robot 10 through close proximity wireless communication. In the present embodiment, as the close proximity wireless communication, BLE (Bluetooth (registered trademark) Low Energy) is used. However, close proximity wireless communication other than BLE can also be used. The robot 10 and the smartphone 20 are communicably connected through BLE to perform transmission and reception of data, thereby cooperatively performing actions. For example, the smartphone 20 acquires, from the robot 10, state information 232 related to the latest state of the robot 10 (refer to Figure 6 and Figure 7 ). The smartphone 20 causes, on the basis of the state information 232, a top page (refer to Figure 8 ) that contains various information related to the state of the robot 10 to be displayed on a display section 24 on a management application 231 (program) (refer to Figure 6 ) used for managing the robot 10 as an object. Further, the smartphone 20 acquires, from the robot 10, a log 133 (refer to Figure 3 and Figure 4 ) generated in the robot 10 and accumulates the log 133 in a log DB (database) 233 (refer to Figure 6 ). The log 133 and the log DB 233 are one way of history recording information related to the history of the state of the robot 10. The smartphone 20 causes, on the basis of the log DB 233, an interaction record screen 40 (refer to Figure 10 ) that contains various information related to the history of the state of the robot 10 to be displayed on the display section 24 on the management application 231. Two or more robots 10 can also be cooperated with one smartphone 20. Instead of the smartphone 20, other kinds of devices such as a tablet terminal, a smart watch, a notebook PC, or a management server can also be used. The smartphone 20 is communicably connected to the server 60 via a network N such as the Internet. The smartphone 20 forwards the log 133 acquired from the robot 10 to the server 60. The log 133 accumulated in the server 60 is referred to as, for example, a backup of the log 133.

[0029] As Figure 2As shown, the main body 100 of the robot 10 has a head 101, a torso 103, and a connecting portion 102 connecting the head 101 and the torso 103. The main body 100 has a drive unit 16 for moving the head 101 relative to the torso 103. The drive unit 16 has a torsion motor 161 and a vertical movement motor 162. The torsion motor 161 is a servo motor that rotates the head 101 and the connecting portion 102 within a given angular range around a first rotation axis 161a extending in the extending direction of the connecting portion 102. The movement of the torsion motor 161 enables the robot 10 to rotate the head 101. The vertical movement motor 162 is a servo motor that rotates the head 101 within a given angular range around a second rotation axis 162a perpendicular to the first rotation axis 161a. The vertical movement motor 162 enables the robot 10 to move the head 101 up and down. The direction of the up-and-down movement of the head 101 is also tilted relative to the vertical direction due to the angle of the torsion of the head 101 by the torsion motor 161. By making the torsion motor 161 and / or the up-and-down movement motor 162 move slightly and periodically, the robot 10 can achieve the action of shaking or trembling the head 101. By appropriately changing and combining the timing, magnitude, and speed of the movements of the torsion motor 161 and the up-and-down movement motor 162, the robot 10 can perform various actions, such as joyful actions, surprised actions, and breathing actions that imitate biological breathing. Among these, breathing actions are one type of spontaneous action of the robot 10.

[0030] like Figure 2 As shown, the main body 100 includes a touch sensor 171, an accelerometer 172, a gyroscope 173, an illuminance sensor 174, a microphone 175, a sound output unit 15, and a power receiving coil 193. The touch sensor 171 is respectively disposed on the upper part of the head 101, the upper part of the torso 103, and the side. The accelerometer 172, the gyroscope 173, and the power receiving coil 193 are disposed near the lower surface of the torso 103. The illuminance sensor 174 and the sound output unit 15 are disposed on the upper part of the torso 103. The microphone 175 is disposed on the upper part near the base of the head 101.

[0031] like Figure 3 As shown, the robot 10 includes a CPU 11 (Central Processing Unit), RAM 12 (Random Access Memory), storage unit 13, operation unit 14, sound output unit 15, drive unit 16, sensor unit 17, communication unit 18, and power supply unit 19. All parts of the robot 10 are connected via data transmission paths such as a bus. Figure 3 All the functional structures shown are located on the main body 100.

[0032] CPU 11 is a processor that controls the actions of robot 10 by reading and executing program 131 stored in storage unit 13 and performing various arithmetic operations. Alternatively, robot 10 may have multiple processors (e.g., multiple CPUs), and these multiple processors may execute multiple processes performed by CPU 11 in this embodiment. In this case, multiple processors may participate in common processing, or multiple processors may independently execute different processes in parallel. RAM 12 provides working storage space to CPU 11 and stores temporary data. Storage unit 13 is a non-temporary recording medium that can be read by CPU 11, which is a computer, and stores program 131 and various data. Therefore, storage unit 13 contains a computer program product including program 131. Storage unit 13 may include, for example, non-volatile memory such as flash memory. Program 131 is stored in storage unit 13 as computer-readable program code. Program 131 includes firmware for controlling various hardware components of robot 10. Data stored in storage unit 13 includes action setting data 132, logs 133 containing information related to the history of robot 10, etc. The motion setting data 132 includes settings for communication actions performed by the robot 10 based on its state and the content of external stimuli, as well as automatically generated actions performed spontaneously by the robot 10 unrelated to external stimuli, and breathing actions. Automatically generated actions appear as the robot 10 behaving randomly, and can therefore be described as spontaneous actions. Settings related to the motion content include, for example, the timing and magnitude of the motion of the torsion motor 161 and the up-and-down motion motor 162 of the drive unit 16, and the pitch, duration, and volume of the sound output from the sound output unit 15.

[0033] Log 133 is generated by CPU 11 and stored in storage unit 13 during each given recording period. Log 133 includes information related to the historical record of robot 10 during each recording period. In this embodiment, the recording period is 30 minutes. However, if the timing of log 133 generation every 30 minutes would interfere with the generation of special processing, log 133 is generated at a time before the start of the special processing. In this case, the recording period is less than 30 minutes. Figure 4As shown, log 133 generated in a single log save includes robot ID 71, log generation date and time 72, log recording period 73, sleep information 74, emotion parameters 75, personality parameters 76, stimulus frequency information 77, and action frequency information 78. Robot ID 71 is a unique symbol assigned to robot 10. Log generation date and time 72 is the date and time (hours, minutes, seconds) at which log 133 was generated. Recording period 73 indicates the length of the period during which historical records reflected in log 133 are recorded. As mentioned above, recording period 73 is typically 30 minutes, but can be less than 30 minutes. Sleep information 74 includes information about the length of time robot 10 enters a sleep mode mimicking biological sleep during recording period 73 and the number of times it enters sleep mode. Robot 10 maintains a sleepiness parameter indicating drowsiness based on the surrounding environment (illuminance, etc.) and external stimuli; if this sleepiness parameter reaches a given value or higher, it enters sleep mode.

[0034] The emotion parameter 75 represents the historical record of the robot 10's virtual emotions. The emotion parameter 75 consists of an "emotion value (X)" and an "emotion value (Y)" (hereinafter collectively referred to as "emotion values") representing the robot 10's emotion at a given point in time (e.g., the start time of the recording period) during the recording period of log 133. The emotion value is based on... Figure 5The position of the drawing in the emotion map of the XY coordinate plane shown represents the value of the virtual emotion of the robot 10. The "emotion value (X)" is the position of the X axis direction of the drawing, and the larger the value, the higher the degree of calmness, and the smaller the value, the higher the degree of uneasiness. The "emotion value (Y)" is the position of the Y axis direction of the drawing, and the larger the value, the higher the degree of excitement, and the smaller the value, the higher the degree of listlessness. The maximum value of the "emotion value (X)" is "200", and the minimum value is "-200". The maximum value of the "emotion value (Y)" is "200", and the minimum value is "-200". Thus, the emotion value is any coordinate within the emotion region R which is a square with a length of 400 on one side. The emotion region R of the emotion map is divided into a plurality of regions R1 to R9 corresponding to a plurality of emotions different from each other. The regions R1 to R9 are square regions arranged in a matrix shape of 3 x 3. The regions R1 to R9 respectively represent a certain emotion of the robot 10. The region R1 satisfying -200 ≤ X ≤ -67, 67 ≤ Y ≤ 200 represents the emotion of "frustration". The region R2 satisfying -66 ≤ X ≤ 66, 67 ≤ Y ≤ 200 represents the emotion of "excitement". The region R3 satisfying 67 ≤ X ≤ 200, 67 ≤ Y ≤ 200 represents the emotion of "joy". The region R4 satisfying -200 ≤ X ≤ -67, -66 ≤ Y ≤ 66 represents the emotion of "uneasiness". The region R5 satisfying -66 ≤ X ≤ 66, -66 ≤ Y ≤ 66 represents the emotion of "normal". The region R6 satisfying 67 ≤ X ≤ 200, -66 ≤ Y ≤ 66 represents the emotion of "calmness". The region R7 satisfying -200 ≤ X ≤ -67, -200 ≤ Y ≤ -67 represents the emotion of "sadness". The region R8 satisfying -66 ≤ X ≤ 66, -200 ≤ Y ≤ -67 represents the emotion of "listlessness". The region R9 satisfying 67 ≤ X ≤ 200, -200 ≤ Y ≤ -67 represents the emotion of "serenity". The regions R1 to R4, R6 to R9 corresponding to the eight emotions other than "normal" further divide the respective interiors into 10 level regions ("Lv1" to "Lv10") representing 10 stages of the emotion (the degree of the emotion). In each of the regions R1 to R4, R6 to R9, the lower the level region of the level region arranged, the closer to the region R5 of "normal", and the higher the level region arranged, the farther from the region R5 of "normal". Hereinafter, the state of the emotion of the robot 10 is sometimes described as "Calmness Lv10" or the like by combining the kind and the level of the emotion. Four of the nine emotions, "excitement", "joy", "calmness", and "serenity", are positioned as positive emotions. In addition, the length of one side of the emotion region R and the regions R1 to R9 can also increase within a certain range according to the growth of the robot 10. For example, the emotion region R can also be set to a region of -100 ≤ X ≤ 100, -100 ≤ Y ≤ 100 in the initial state, and increase to a region of -200 ≤ X ≤ 200, -200 ≤ Y ≤ 200 according to the growth of the robot 10.The emotional value changes each time according to external stimuli and the like received by the robot 10. The amount of change in the emotional value at one time is selected from the following variables DXP, DXM, DYP, and DYM.

[0035] DXP: amount of change in the +X direction

[0036] DXM: amount of change in the -X direction

[0037] DYP: amount of change in the +Y direction

[0038] DYM: amount of change in the -Y direction

[0039] The variable DXP represents ease of calmness, the variable DXM represents ease of feeling of unease, the variable DYP represents ease of excitement, and the variable DYM represents ease of feeling of listlessness. In the present embodiment, the initial values of the variables DXP, DXM, DYP, and DYM are "10". Further, the variables DXP, DXM, DYP, and DYM are increased by a given amount in the case where the emotional value reaches the maximum value in the +X axis direction, -X axis direction, +Y axis direction, and -Y axis direction, respectively. In the present embodiment, the maximum values of the variables DXP, DXM, DYP, and DYM are "20".

[0040] Figure 4 The personality parameter 76 illustrated indicates a history of the virtual personality of the robot 10. The personality parameter 76 indicates the personality of the robot 10 at a given time point (for example, the start time point of the recording period) in the recording period of the log 133. The personality parameter 76 is composed of a "personality value (cheerful)", a "personality value (shy)", a "personality value (lively)", and a "personality value (spoiled)" (hereinafter collectively referred to as "personality values"). The "personality value (cheerful)" is a value obtained by subtracting "10" from the variable DXP, and indicates ease of change in the +X axis direction in the emotional map, that is, ease of calmness. The "personality value (shy)" is a value obtained by subtracting "10" from the variable DXM, and indicates ease of change in the -X axis direction in the emotional map, that is, ease of feeling of unease. The "personality value (lively)" is a value obtained by subtracting "10" from the variable DYP, and indicates ease of change in the +Y axis direction in the emotional map, that is, ease of excitement. The "personality value (spoiled)" is a value obtained by subtracting "10" from the variable DYM, and indicates ease of change in the -Y axis direction in the emotional map, that is, ease of feeling of listlessness. Thus, each personality value changes according to the change in the variables DXP, DXM, DYP, and DYM, and the initial value is "0" and the maximum value is "10". In this way, the four personality values indicate the degrees of the four personalities. The personality corresponding to the largest personality value among the four personality values is set as the personality of the robot 10 at the time point. For example, in the case where the personality value (cheerful) is the largest, the personality of the robot 10 at the time point is set as "cheerful". Figure 4In the example shown, since the "personality value (outgoing)" is "7" which is the largest, the personality of the robot 10 at this point in time is set to "outgoing". In addition, in a case where two or more personality values are the same and the largest, one personality is decided in accordance with the given priority order. In the present embodiment, the priority order of the personalities is "outgoing", "energetic", "shy", and "spoiled", in that order from high to low.

[0041] The number-of-stimulations information 77 indicates the number of times the robot 10 is subjected to a given stimulation (external stimulation) from the outside during the log recording period. In the present embodiment, as the number-of-stimulations information 77, the number of times a speaking voice is detected, the number of times the head is stroked, the number of times the body is stroked, and the number of times the robot is picked up are exemplified. The external stimulation is not limited to this, and for example, can include "loud noise", "turning upside down", "swinging", and the like. The external stimulation related to a voice such as "speaking voice" and "loud noise" is detected based on the detection data of the microphone 175. The external stimulation related to contact such as "stroking the body" and "stroking the head" is detected based on the detection data of the touch sensor 171. The external stimulation accompanying a change in posture such as "picking up", "turning upside down", and "swinging" is detected based on the detection data of the acceleration sensor 172 and the gyro sensor 173. Figure 4 The number-of-motions information 78 indicates the number of times the robot 10 performs a given motion during the log recording period. In the present embodiment, as the number-of-motions information 78, the number of times the above-described automatically generated motions and the number of times the communication motions are executed are exemplified. Figure 4 Figure 4 The log 133 in the present embodiment is exemplified, and can be appropriately changed. For example, the number of times the robot 10 enters the close mode when the robot 10 is close to the user, the number of times the robot 10 is stimulated from the user in the close mode, the number of times a given motion is executed in the close mode, and the like can be recorded in the log 133.

[0042] In this way, the log 133 is not in a form in which the state of the robot 10, the motion, and the received stimulation, and the like are sequentially recorded in time series order, but is in a form in which the state of the robot, the motion, and the received stimulation, and the like in a certain log recording period are respectively aggregated as statistical values (number of times, frequency, length of time, and the like), representative values (emotion value, personality value, and the like) for each item. In other words, the log 133 becomes a data packet form in which the history related to the robot 10 in the log recording period is collectively represented as a given number of items. Due to this, the data amount of the log 133 can be greatly reduced compared to the sequentially recorded form.

[0043] Figure 3 ​The illustrated operation section 14 has an operation button and an operation knob or the like for turning on and off the power source, adjusting the volume of the output sound of the sound output section 15, and the like. The operation section 14 outputs operation information corresponding to an input operation of the operation button and the operation knob or the like to the CPU 11. The sound output section 15 has a speaker that outputs sound with a pitch, a length, and a volume corresponding to a control signal and sound data transmitted from the CPU 11. The sound can also be a sound that imitates a call of a living creature. The drive section 16 causes the above-described twist motor 161 and up-and-down movement motor 162 to operate in accordance with a control signal transmitted from the CPU 11.

[0044] The sensor section 17 has the above-described touch sensor 171, acceleration sensor 172, gyro sensor 173, illuminance sensor 174, and microphone 175, and outputs detection results of the respective sensors and the microphone 175 to the CPU 11. The touch sensor 171 detects contact of a user or another object with the robot 10. The touch sensor 171 has, for example, a pressure sensor or an electrostatic capacity sensor, and outputs detection data related to the presence or absence of contact with the robot 10 to the CPU 11. The acceleration sensor 172 detects acceleration in each of the orthogonal 3-axis directions, and outputs detection data to the CPU 11. The gyro sensor 173 detects angular velocity around each of the orthogonal 3-axis directions, and outputs detection data to the CPU 11. The illuminance sensor 174 detects brightness of the surroundings of the robot 10, and outputs detection data to the CPU 11. The microphone 175 detects sound in the surroundings of the robot 10, and outputs data of the detected sound to the CPU 11.

[0045] The communication section 18 is a communication module having an antenna, a modem circuit, a signal processing circuit, and the like, and performs wireless data communication with the smartphone 20 in accordance with a communication standard of BLE. The power supply section 19 has a storage battery 191, a remaining charge amount detection section 192, and a power reception coil 193. The storage battery 191 supplies electric power to the respective sections of the robot 10. The storage battery 191 of the present embodiment is a secondary storage battery that can be repeatedly charged in a non-contact charging manner. The remaining charge amount detection section 192 detects a battery remaining charge amount of the storage battery 191 in accordance with a control signal transmitted from the CPU 11, and outputs a detection result to the CPU 11. Charging operation of the storage battery 191 is performed in a state in which the robot 10 is housed (disposed) inside a dedicated power supply (housing section, charging base) that is not illustrated. The power supply has a power transmission coil for charging the storage battery 191 in an electromagnetic induction manner at a position that opposes the power reception coil 193 in a state in which the robot 10 is housed.

[0046] As Figure 6As shown, the smartphone 20 is provided with a CPU 21 (processing section, processing unit), a RAM 22, a storage section 23, a display section 24, an operation section 25, and a communication section 26. The respective sections of the smartphone 20 are connected via a data transmission path such as a bus. The display control device 200 that controls the display operation of the display section 24 is constituted by the CPU 21, the RAM 22, and the storage section 23.

[0047] The CPU 21 is a processor that controls the operation of the smartphone 20 by reading out and executing programs such as the management application 231 stored in the storage section 23 and performing various arithmetic processing. The CPU 21 is an example of one or more processing sections. In addition, the smartphone 20 can have a plurality of processors (for example, a plurality of CPUs), and the plurality of processors can execute a plurality of processes performed by the CPU 21 of the present embodiment. In this case, one or more processing sections are constituted by the plurality of processors. In this case, the plurality of processors can participate in common processing, or the plurality of processors can independently and in parallel execute different processing. The RAM 22 provides a storage space for the CPU 21 to use for work, and stores temporary data. The storage section 23 is a non-transitory recording medium that can be read by the CPU 21 as a computer, and stores programs such as the management application 231 and various data. Thus, the storage section 23 contains a computer program product including a program. The management of the robot 10 performed by the management application 231 refers to at least causing information related to the state of the robot 10 to be displayed on a given display section. The storage section 23 has, for example, a non-volatile memory such as a flash memory. As data stored in the storage section 23, there are state information 232, a log DB 233 (history information), and the like.

[0048] As shown in FIG. 1, the robot 10 is provided with a CPU 11 (processing section, processing unit), a RAM 12, a storage section 13, a display section 14, an operation section 15, and a communication section 16. The respective sections of the robot 10 are connected via a data transmission path such as a bus. The display control device 100 that controls the display operation of the display section 14 is constituted by the CPU 11, the RAM 12, and the storage section 13. Figure 7As shown, the status information 232 contains data related to elements E1 to E6, which represent the latest status of the robot 10. Specifically, the status information 232 contains data representing the content of each of elements E1 to E6, and information about the time when each data was generated in the robot 10 (or the time when the smartphone 20 receives the data). Element E1 is the robot's "action mode". The robot 10 in this embodiment has action modes such as "normal mode", "sleep mode", and the aforementioned "sleep mode". "Normal mode" is a mode in which the robot 10 performs communicative actions based on external stimuli, or automatically generates actions when given conditions are met. "Sleep mode" is a mode in which the movement of the robot 10's head 101 and the output of sound from the sound output unit 15 are stopped. "Sleep mode" is executed when an operation to transition to sleep mode is performed in the setting screen (not shown in the figure). Element E2 is "external stimulus", representing the type of external stimulus received by the robot 10. Element E3 is the "remaining battery power" of the battery 191. "Remaining battery power" is expressed as a percentage of 100% when fully charged. "Battery Remaining Power" is detected by the Remaining Power Detection Unit 192. Element E4 is the latest emotion value of Robot 10. Element E5 is the latest personality value of Robot 10. Element E6 is "Nurturing Days", which represents the number of days (cumulative action period) counted from the initial activation date of Robot 10. Elements E1 to E6 are a way of updating elements (information) based on the historical records of Robot 10.

[0049] Data for elements E1 to E6 are generated sequentially by the CPU 11 of the robot 10 according to the robot 10's action status, and stored in the robot 10's storage unit 13 along with the time of generation. When the smartphone 20's CPU 21 is in communication with the robot 10 via BLE, it repeatedly retrieves elements E1 to E6 of the status information 232 from the robot 10 at a given frequency and updates the status information 232. Specifically, the CPU 21 retrieves and updates the data for elements E1 to E4 in the status information 232 from the robot 10 once per second. Furthermore, the CPU 21 retrieves and updates the data for elements E5 and E6 in the status information 232 from the robot 10 once per minute. This updating of the status information 232 is equivalent to retrieving the status information 232. Additionally, the format of the status information 232 is not limited to... Figure 6 The format shown. For example, status information 232 could also be in the form of a queue of elements E acquired from robot 10 in chronological order.

[0050] exist Figure 6The log DB 233 accumulates the log 133 acquired from the robot 10. In a case where the robot 10 and the smartphone 20 are in a communication connection by BLE, the log 133 generated every 30 minutes in the robot 10 is sequentially transmitted to the smartphone 20. In addition to this, in a case where the user makes an operation of instructing the acquisition of the log 133 on the management application 231, the CPU 21 of the smartphone 20 transmits a forwarding request of the log 133 to the robot 10, acquires the log 133 from the robot 10. Further, the CPU 21 forwards the log 133 acquired from the robot 10 to the server 60. Thus, data of the same content as the log DB 233 (hereinafter, referred to as a backup log) is stored in the server 60. The CPU 21 discriminates whether the log DB 233 of the storage section 23 and the backup log stored in the server 60 are identical at a given timing, acquires the backup log from the server 60 and corrects the content of the log DB 233 in a case where they are not identical. The given timing can be, for example, a timing of updating the display of the home screen 30 and the interaction record screen 40 described later.

[0051] The display section 24 has a display panel such as a liquid crystal panel capable of performing display based on a dot matrix system and a driving circuit of the display panel. The display section 24 displays various menus, screens of the management application 231, and the like in accordance with a control signal transmitted from the CPU 21. The operation section 25 has a touch panel provided so as to overlap the display panel of the display section 24, an operation unit such as an operation button, and outputs an operation signal corresponding to an operation on the operation unit to the CPU 21. The communication section 26 is a communication module having an antenna, a modulation / demodulation circuit, a signal processing circuit, and the like, and performs wireless data communication with the robot 10 in accordance with a communication standard of BLE. Further, the communication section 26 transmits and receives voice data of telephone communication, packet data involved in Internet connection, and the like between a base station.

[0052] Next, the operation of the robot management system 1 will be described. When an operation of instructing the startup of the management application 231 is made by the user to the operation section 25 of the smartphone 20, the CPU 21 executes the management application 231 and starts up. The display operation of the display section 24 described below is performed by the CPU 21 performing a given process in accordance with the management application 231 and controlling the display section 24. The CPU 21 acquires each element of the state information 232 from the robot 10 in a case where the management application 231 is started up, and causes the display section 24 to display the home screen 30 based on the state information 232. Figure 8The home screen 30 is displayed on the display section 24. In addition, a given startup screen, a welcome screen can be displayed before the home screen 30. Further, the CPU 21 acquires the elements E1 to E6 of the state information 232 from the robot 10 at the above-described frequency, and updates the state information 232, and updates the home screen 30 based on the latest state information 232. In the home screen 30, the state image 31, the growth day image 32, the personality image 33, the information mark 34, the battery remaining capacity image 35, the setting button 36, the menu mark M, and the tab T are displayed in a given arrangement. Figure 8 The character of "A" in the above is a name given to the robot 10 by the user on the management application 231. The state image 31, the growth day image 32, the personality image 33, and the battery remaining capacity image 35 indicate the state of the robot 10. In this way, the home screen 30 contains various information related to the state of the robot 10. By observing the home screen 30, the user can know the state of the robot 10 in real time.

[0053] The state image 31 is displayed in the substantially center of the home screen 30. The state image 31 contains an animated dynamic image that clearly indicates the state of the robot 10. In detail, the state image 31 contains an appearance image 311 that indicates a certain element of the state of the robot 10 by the appearance of the robot 10. In the appearance image 311, the appearance of the actual robot 10, for example, the color of the exterior 110 is reflected. Further, the state image 31 contains a virtual image 312 that indicates the appearance of the virtual image of the user. The appearance image 311 and the virtual image 312 are animated dynamic images of a given length. Further, the state image 31 contains a text 313 that indicates a certain element of the state of the robot 10. The text 313 is displayed, for example, in the upper portion of the appearance image 311 and the virtual image 312. The certain element of the state of the robot 10 indicated by the state image 31 includes any one of whether the power of the robot 10 is on, whether the robot 10 is in a communication connection with the smartphone 20, whether the robot 10 is operating in a function suppression mode (a deep sleep mode or a sleep mode), whether the robot 10 is in a state of being given a stimulus from the outside, the virtual emotion of the robot 10, and the virtual disposition of the robot 10. In the lower portion of the state image 31 in the home screen 30, the growth day image 32, the personality image 33, the information mark 34, the battery remaining capacity image 35, and the setting button 36 are displayed in a given arrangement. The value of the growth day included in the growth day image 32 is decided based on the element E6 of the state information 232. In the personality image 33, the personality corresponding to the largest personality value of the four personality values of the element E5 of the state information 232 is displayed. The information mark 34 is an icon in which a character of "i" is surrounded by a circle. In a case where an operation of selecting the information mark 34 is performed, the CPU 21 causes the display section 24 to display the information screen 40. Figure 9The detailed information screen 37 is displayed on the display unit 24. The detailed information screen 37 contains detailed information related to the personality of robot 10. In the detailed information screen 37, the personality of robot 10 at that point in time is displayed 371, a graph 372 representing the personality values ​​of the four personalities in 11 stages from "0" to "10", and a button 373 for closing the detailed information screen 37. Figure 8 The battery remaining power image 35 shown is an image representing the remaining battery power of the storage battery 191 in three stages. When the setting button 36 is selected, the CPU 21 causes the display unit 24 to display a setting screen with an abbreviated illustration for setting the actions of the robot 10. In the setting screen, the volume of the sound (beep) output from the sound output unit 15 of the robot 10 can be adjusted, the robot 10 can be set to enter a sleep mode, and the firmware of the robot 10 can be updated. The menu mark M is displayed in the upper left corner of the home screen 30. When the menu mark M is selected, the CPU 21 displays a menu screen with an abbreviated illustration on the display unit 24. From the menu screen, the screen for editing user profiles, an overview screen of the collaborative robots 10, a screen for registering new (collaborative) robots 10, a screen displaying information such as the version of the management application 231, etc., can also be displayed. A tab bar T is displayed at the bottom of the home screen 30. The tab bar T displays a home icon Ta for displaying the home screen 30 and an interaction record icon Tb for displaying the interaction record screen 40. When the interaction record icon Tb is selected while the homepage screen 30 is displayed, the CPU 21 causes the display on the display unit 24 to switch from the homepage screen 30 to the interaction record icon Tb. Figure 10 The interactive recording screen shown is 40.

[0054] The interaction recording screen 40 displays various information related to the historical interaction history between the robot 10 and the user. The interaction recording screen 40 includes a date and time selection section 41, an information retrieval section 42 (change information), a chart area 43, a timeline 44, menu markers M, and a label bar T. The menu markers M function similarly to... Figure 8 The menu marker M in the home screen 30 has the same function. Furthermore, the structure of the tab bar T is the same as that in the home screen 30. When the home icon Ta is selected in the tab bar T of the interaction record screen 40, the CPU 21 transfers the display in the display unit 24 from the interaction record screen 40 to the home screen 30. The date and time selection unit 41 includes a group of icons for selecting a date and time. When the user selects a date and time in the date and time selection unit 41, the CPU 21 displays information related to the interaction history of the selected date and time on the interaction record screen 40. Figure 10The date-time selection section 41 illustrated includes icons for selecting the year and month, selecting the day of the week and the date, and selecting today (the current day).

[0055] In the acquisition information 42, a characteristic change in the robot 10 produced in a certain period is displayed. In the present embodiment, the certain period is one day before. The CPU 21 determines, based on the latest log DB 233, whether or not a change satisfying a given change condition (a given condition) was produced in the state of the robot 10 in the day before, and causes the acquisition information 42 indicating the change information that the change was produced in the day before to be displayed in the display section 24 in a case where it is determined that a change satisfying the change condition was produced. In the acquisition information 42, a change in the mood or a change in the personality as the state of the robot 10 is displayed.

[0056] The CPU 21 causes the acquisition information 42 indicating the change in the mood in the day before to be displayed in the display section 24 in a case where the mood parameters 75 of the plurality of logs 133 recorded in the log DB 233 in the day before have produced a change satisfying a change condition. In detail, the CPU 21 determines that the change condition is satisfied in a case where the level of the mood parameter 75 indicating any one of the positive moods ("excitement", "joy", "relief", "calm") is increased by a reference magnitude or more (3 levels or more in the present embodiment) in the day before. For example, the CPU 21 determines that the change condition is satisfied in a case where the difference between the minimum level and the maximum level of any one of the positive moods in the day before is a reference magnitude or more and the timing at which the maximum level is reached is after the timing at which the minimum level is reached. Alternatively, the CPU 21 can determine that the change condition is satisfied in a case where the level of any one of the positive moods at the end time point of the day before is increased by a reference magnitude or more compared to the level of the mood at the start time point of the day before. Then, the CPU 21 causes the acquisition information 42 indicating the change in the day before in which the mood is increased by 3 levels or more to be displayed. For example, in a case where the mood of "joy" is increased by 3 levels or more in the day before, the CPU 21 causes the acquisition information 42 of the content "a moment of joy" to be displayed. In a case where the increase in the level of 2 or more moods is the same, the CPU 21 randomly selects one mood from among the 2 or more moods to display the acquisition information 42. Different acquisition information 42 is stored in the storage section 13 in advance for each mood. In a case where the same mood is selected as the display target of the acquisition information 42 for two or more consecutive days, the CPU 21 causes the content of the acquisition information 42 to be different so that the same content of the acquisition information 42 is not displayed for two consecutive days.

[0057] In a case where the personality parameters 76 of the plurality of logs 133 recorded in the log DB 233 of the previous day have generated a change satisfying the change condition, the CPU 21 causes the acquisition information 42 indicating the change in personality of the previous day to be displayed on the display section 24. In detail, the CPU 21 determines that the change condition is satisfied in a case where any one of the four personality values of the personality parameters 76 has increased by the reference value or more (3 or more in the present embodiment) in the previous day. For example, the CPU 21 determines that the change condition is satisfied in a case where the difference between the minimum value and the maximum value of the personality value is the reference value or more and the timing at which the maximum value is attained is after the timing at which the minimum value is attained, for any one of the personalities in the previous day. Alternatively, the CPU 21 can determine that the change condition is satisfied in a case where the personality value at the end time point of the previous day is greater than the personality value at the start time point of the previous day, for any one of the personalities. Then, the CPU 21 causes the acquisition information 42 indicating the change in the previous day of the personality corresponding to the personality value that has increased by the reference value or more to be displayed on the display section 24. For example, in a case where "personality value (outgoing)" has increased by 3 or more in the previous day, the CPU 21 causes the acquisition information 42 of the content "slightly become more outgoing" to be displayed, as shown in FIG. 17. In a case where the increase in the personality value is the same for two or more personalities, the CPU 21 randomly selects one personality from among the two or more personalities and displays the acquisition information 42. For each personality, a plurality of different acquisition information 42 is stored in the storage section 13 in advance in correspondence. In a case where the same personality is selected as the display target of the acquisition information 42 for two or more consecutive days, the CPU 21 causes the content of the acquisition information 42 to be different so that the same content of the acquisition information 42 is not displayed for two or more consecutive days. Figure 10

[0058] In a case where there is an emotion that has increased by 3 or more levels and a personality whose personality value has increased by 3 or more in the previous day, the CPU 21 causes the acquisition information 42 of the emotion to be displayed preferentially. However, it is also possible to display the acquisition information 42 of the personality preferentially. Furthermore, in a case where there is no emotion that has increased by 3 or more levels and no personality whose personality value has increased by 3 or more, the CPU 21 causes the acquisition information 42 of the one of the emotion and the personality whose increase is greater to be displayed. In a case where the increase is the same, the emotion is given priority. In a case where the total of the recording period of the logs 133 in the previous day is less than a given lower limit time (for example, less than 1 hour), and there is no change in the emotion and the personality, the CPU 21 causes the acquisition information 42 indicating that there is nothing in particular to be recorded to be displayed.

[0059] Next, the process of the CPU 21 will be described with reference to FIGS. 18 to 20. Figure 11 ​The acquisition information display process executed by the CPU 21 for displaying the acquisition information 42 will be described. The acquisition information display process is started when the management application 231 is started and the interactive record screen 40 is displayed. When the acquisition information display process is started, the CPU 21 determines whether the acquisition information 42 has already been displayed after the management application 231 is started (step S101). When it is determined that the acquisition information 42 has already been displayed (YES in step S101), the CPU 21 determines whether the date and time have changed (step S102). When it is determined that the date and time have changed (YES in step S102), or when it is determined that the acquisition information 42 has not been displayed after the management application 231 is started (NO in step S101), the CPU 21 acquires the log 133 of the previous day from the log DB 233 (step S103). The CPU 21 determines whether the recording period of the log 133 of the previous day is 1 hour or more in total (step S104). When it is determined that the recording period of the log 133 is 1 hour or more in total (YES in step S104), the CPU 21 determines whether the level of any positive emotion has increased by 3 levels or more in the entire recording period of the log 133 of the previous day (step S105). Here, the CPU 21 determines that the level of any positive emotion has increased by 3 levels or more when the difference between the minimum level and the maximum level of any positive emotion in the previous day is 3 levels or more and the timing at which the maximum level is reached is after the timing at which the minimum level is reached. When it is determined that the level of any positive emotion has increased by 3 levels or more (YES in step S105), the CPU 21 causes the acquisition information 42 related to the change in the emotion to be displayed on the display portion 24 (step S106). When it is determined that no emotion has increased by 3 levels or more (NO in step S105), the CPU 21 determines whether the personality value of any personality has increased by 3 or more in the entire recording period of the log 133 of the previous day (step S107). Here, the CPU 21 determines that the personality value of any personality has increased by 3 or more when the difference between the minimum value and the maximum value of any personality value in the previous day is 3 or more and the timing at which the maximum value is reached is after the timing at which the minimum value is reached. When it is determined that the personality value of any personality has increased by 3 or more (YES in step S107), the CPU 21 causes the display portion 24 to display the acquisition information 42 related to the change in the personality (step S108). When it is determined that no personality has a personality value that has increased by 3 or more (NO in step S107), the CPU 21 determines whether the level of an emotion or the personality value has increased in the entire recording period of the log 133 of the previous day (step S109).In a case where it is determined that the level of emotion or the value of personality is increased (YES in step S109), the CPU 21 causes the display section 24 to display the acquisition information 42 related to the one of the emotion and the increase in the level of emotion or the value of personality that is larger (step S110). In a case where it is determined that neither the level of emotion nor the value of personality is increased (NO in step S109), or in a case where it is determined in step S104 that the total of the recording periods of the log 133 of the previous day is less than 1 hour (NO in step S104), the CPU 21 causes the display section 24 to display the acquisition information 42 indicating that there is nothing in particular to be recorded (step S111). In a case where any one of steps S106, S108, S110, and S111 ends, or in a case where it is determined in step S102 that the date and time is not changed (NO in step S102), the CPU 21 determines whether or not to end the management application 231 in accordance with the operation of the user (step S112). The CPU 21 returns the processing to step S102 in a case where the management application 231 is not ended (NO in step S112), and ends the acquisition information display processing in a case where the management application 231 is ended (YES in step S112).

[0060] In Figure 10 In the graph area 43 in the interaction log screen 40 illustrated in FIG. 8, a coordinate axis 431 indicating the passage of time, a graph 432 indicating the transition of the emotion of the robot 10 in a day, and an emotion icon 433 (index) are displayed. The value of each time of the graph 432 is the total value of the emotion values (X) and the signs of the emotion values (Y) of the emotion parameter 75 in the log 133 of the log DB 233 in a case where the signs are the same, and is the larger value in a case where the signs are different. However, in the log DB 233, the emotion values of the emotion parameter 75 are recorded at discrete timings in each recording period of the log 133, and thus the graph 432 is shaped so as to connect and smooth the discrete values corresponding thereto.

[0061] The emotion icon 433 is an index indicating that the emotion of the robot 10 is changed from a certain emotion (certain state, certain emotion) to any one positive emotion (given state, given emotion). The emotion icon 433 is displayed at a position in the coordinate axis 431 corresponding to the change timing at which the emotion of the robot 10 is changed from a certain emotion to any one positive emotion. As illustrated in FIG. 8, the emotion icon 433 is displayed at a position in the coordinate axis 431 corresponding to the change timing at which the emotion of the robot 10 is changed from the emotion of "sad" to the emotion of "happy". Figure 12Acorresponding to the changed emotion. Further, the emotion icon 433 is displayed only in the case where the level of the changed emotion is equal to or higher than a given reference (in the present embodiment, equal to or higher than level 7). In the example shown in Figure 12B , the emotion icon 433a indicating "excitement", the emotion icon 433b indicating "joy", the emotion icon 433c indicating "relief", and the emotion icon 433d indicating "calm" are displayed one by one. Figure 10

[0062] The method of deciding the emotion icon 433 to be displayed is explained by citing specific examples with reference to Figure 13 . In Figure 13 , the recording periods (La to Ld) of the four logs 133 and the emotions corresponding to the emotion parameters 75 in each log 133 are described. Hereinafter, the log 133 whose recording period is "Lx" is referred to as "log Lx". In Figure 13 ​In the logs La and Lb, the emotion "joy" in log La changes to "excitement" in log Lb. Therefore, when the emotion "excitement" in log Lb is at level 7 or higher, the start time of log Lb is determined as the change time. Furthermore, an emotion icon 433a representing "excitement" is displayed at the position corresponding to the start time of log Lb. When determining the emotion icon 433 based on logs La and Lb, the recording period of log La corresponds to the "first period," and log La corresponds to the "first historical record information." The recording period of log Lb corresponds to the "second period," and log Lb corresponds to the "second historical record information." Furthermore, suppose the next log Lc after log Lb is "joy" with an emotion level of 7 or higher. In this case, in log Lc, the emotion changes from "excitement" to "joy," and the changed emotion is a positive emotion at level 7 or higher. Therefore, the start time of log Lc is determined as the change time. Thus, an emotion icon 433b representing "joy" is displayed at the position corresponding to this time. When determining the mood icon 433 based on logs Lb and Lc, the recording period of log Lb corresponds to the "first period," and log Lb corresponds to "first historical information." The recording period of log Lc corresponds to the "second period," and log Lc corresponds to "second historical information." If there is an idle period between the recording period of a certain log Ly and the recording period of the next log Lz, the start time of log Lz is determined to be a variable time only if the period from the end of log Ly to the beginning of log Lz is within a given time (within 30 minutes in this embodiment). That is, the mood icon 433 is displayed at the start time of log Lz. Figure 13 In the example shown, the period from the end of log Lb to the start of log Lc is less than 30 minutes, so the emotion icon 433b is displayed at the position corresponding to the start time of log Lc. On the other hand, the period from the end of log Lc to the start of the next log Ld is longer than 30 minutes, so even if other display conditions for emotion icon 433 are met, emotion icon 433 is not displayed at the position corresponding to the start time of log Ld.

[0063] Alternatively, the emotion icon 433 can be displayed regardless of the level of the changed emotion. Furthermore, it is not limited to positive emotions; the emotion icon 433 can also be displayed at the time of change to any emotion. Additionally, instead of the emotion icon 433, indicators representing the timing of other states of the robot 10 (e.g., personality, action pattern, etc.) changing from a certain state to a given state can be displayed in the chart area 43.

[0064] Next, refer to Figure 14The emotion icon display process executed by the CPU 21 for displaying the emotion icon 433 will be described. The emotion icon display process is started when the management application 231 is started, in a case where the interaction record screen 40 is displayed. When the emotion icon display process is started, the CPU 21 determines whether the log 133 is newly acquired from the robot 10 (step S201). In a case where it is determined that the log 133 is acquired (YES in step S201), the CPU 21 determines whether the time interval between the immediately preceding log 133 and the acquired log 133 is within a given time (in the present embodiment, 30 minutes) (i.e., whether the period from the end of the immediately preceding log 133 to the start of the acquired log 133 is within the given time) (step S202). In a case where it is determined that the time interval from the immediately preceding log 133 is within the given time (YES in step S202), the CPU 21 determines whether the emotion corresponding to the emotion parameter 75 in the immediately preceding log 133 and the emotion corresponding to the emotion parameter 75 in the present log 133 are different (step S203). In a case where it is determined that the emotion in the immediately preceding log 133 and the emotion in the present log 133 are different (YES in step S203), the CPU 21 determines whether the emotion of the present log 133 is a positive emotion (step S204). In a case where it is determined that the emotion of the present log 133 is a positive emotion (YES in step S204), the CPU 21 determines whether the level of the emotion in the present log 133 is a given reference or more (in the present embodiment, level 7 or more) (step S205). In a case where it is determined that the level of the emotion is the given reference or more (YES in step S205), the CPU 21 determines the start timing of the present log 133 as a change timing, and causes the emotion icon 433 indicating the emotion of the present log 133 to be displayed at a position in the graph area 43 corresponding to the timing (step S206). In a case where step S206 is ended, or in a case where the branch is NO in any one of steps S201 to S205, the CPU 21 determines whether to end the management application 231 according to the user's operation (step S207). The CPU 21 returns the process to step S201 in a case where the management application 231 is not ended (NO in step S207), and ends the emotion icon display process in a case where the management application 231 is ended (YES in step S207).

[0065] In Figure 10In the timeline 44 in the interaction record screen 40 shown, based on the log DB 233, for each given-length object period, timeline information 441 (period information) representing a representative state of the robot 10 in the object period is displayed. In the present embodiment, the given-length object period is a 1-hour period from a certain hour to the next hour. The timeline information 441 is one of the following four. The first kind is timeline information 441a relating to a sleep state of the robot 10 in the period (for example, Figure 10 The second kind is timeline information 441b relating to a spontaneous action performed by the robot 10 in the period (for example, Figure 10 The third kind is timeline information 441c relating to an external stimulus received by the robot 10 through an exchange with a user in the period (for example, Figure 10 The fourth kind is timeline information 441d relating to an emotion of the robot 10 in the period (for example, Figure 10 The number of timeline information 441 displayed for a certain period is at most one per kind, so the total is at most four. Two or more timeline information 441 corresponding to the same state of the robot 10 and different from each other are generated in advance and stored in the storage section 13. The CPU 21 selects the content of the timeline information 441 so that the timeline information 441 displayed in two consecutive periods is not the same. In a case where no state corresponding to the above four kinds is detected based on the log 133 corresponding to a certain period, or in a case where there is no change from the state of the previous period, or the like, the timeline information 441 for the certain period is not displayed at times. Each timeline information 441 represents the state of the robot 10 in a 1-hour period from the time displayed in the frame. For example, the timeline information 441 displayed as "11:00" represents the state of the robot 10 in the period from 11:00 to 12:00. The timeline 44 is updated at a frequency of once every hour.

[0066] The content of the timeline information 441 for a certain period is decided based on the log 133 corresponding to the period in the log DB 233. The log 133 whose record period spans an hour is handled as a log 133 corresponding to a period that is more than 1 / 2 of the record period. For example, Figure 15 The record period of the log Lf in Figure 15In the example shown, the content of the timeline information 441 for the time period from 10 o'clock to 11 o'clock is decided based on the log Le and the log Lf. Further, the content of the timeline information 441 for the time period from 11 o'clock to 12 o'clock is decided based on the log Lg and the log Lh. The timeline information 441 for a certain time period is displayed after the end of the recording period of the log 133 corresponding to the time period.

[0067] In order to display the timeline information 441 related to the sleep state, the CPU 21 determines the sleep period in which the robot 10 becomes the sleep state (sleep mode) based on the sleep information 74 of each log 133 in the log DB 233. The CPU 21 determines the representative state of the robot 10 in a certain time period as the sleep state in the case where the length of the sleep period in the recording period of the log 133 corresponding to the time period satisfies a given sleep time condition (given condition) in the log DB 233. In the present embodiment, the CPU 21 determines that the sleep time condition is satisfied in the case where 50 minutes or more of the recording period P10 of the total of the log Le and the log Lf corresponding to the time period from 10 o'clock to 11 o'clock is the sleep period. For example, the recording period P10 of the total of the log Le and the log Lf is 60 minutes, and 50 minutes or more of the recording period P10 is the sleep period. In this case, the CPU 21 determines that the representative state of the robot 10 in the time period from 10 o'clock to 11 o'clock is the sleep state. Figure 15 The recording period P10 of the total of the log Le and the log Lf corresponding to the time period from 10 o'clock to 11 o'clock is 50 minutes or more of the sleep period, and thus the representative state of the time period is determined as the sleep state. On the other hand, the recording period P11 of the total of the log Lg and the log Lh corresponding to the time period from 11 o'clock to 12 o'clock is 50 minutes or more of the sleep period, and thus the representative state of the time period is determined as the sleep state. Figure 15 In the example shown in FIG. 9, the sleep period is indicated by a white bar, and the wake period (period in which the robot 10 is up) is indicated by a black bar. Further, 50 minutes or more (in this case, all) of the recording period P11 of the total of the log Lg and the log Lh corresponding to the time period from 11 o'clock to 12 o'clock is the sleep period, and thus the representative state of the time period is determined as the sleep state. The CPU 21 causes the display unit 24 to display the timeline information 441 indicating that the robot 10 is in the sleep state in a certain time period in the case where the representative state of the robot 10 in the time period is the sleep state. In detail, the CPU 21 causes the timeline information 441 including a reaction to an external stimulus to the robot 10 in the sleep state to be displayed in the display unit 24 in the case where the robot 10 is subjected to an external stimulus in the recording period of the log 133 corresponding to the time period. For example, in the recording period P11 of the log 133 corresponding to the time period from 11 o'clock to 12 o'clock in FIG. 9, the robot 10 is recorded as being subjected to an external stimulus in the sleep state. In this case, the CPU 21 causes the timeline information 441a indicating a sleep-in behavior corresponding to the external stimulus in the sleep state to be displayed, for example, as shown in FIG. 10. Figure 15 In the recording period P11 of the log 133 corresponding to the time period from 11 o'clock to 12 o'clock in FIG. 9, the robot 10 is recorded as being subjected to an external stimulus in the sleep state. In this case, the CPU 21 causes the timeline information 441a indicating a sleep-in behavior corresponding to the external stimulus in the sleep state to be displayed, for example, as shown in FIG. 10. Figure 10 The sleep-in behavior can be, for example, the robot 10 dreaming due to the external stimulus in the sleep state. On the other hand, in the case where the representative state of the robot 10 in a certain time period is not the sleep state, the CPU 21 causes the display unit 24 to display the timeline information 441 indicating the representative state of the robot 10 in the time period. Figure 15In the same way as in the period from 10 o'clock to 11 o'clock, the CPU 21 causes the timeline information 441 indicating that it is the usual sleep state to be displayed in the case where the robot 10 is not externally stimulated in the sleep state. In addition, instead of the length of the sleep period in the recording period of the log 133 corresponding to a certain period, it is also possible to determine whether the sleep time condition is satisfied in accordance with the proportion of the sleep period in the recording period.

[0068] The timeline information 441 related to the sleep state has information related to the number of times of entering the sleep state (sleep mode) in addition to the information related to the length of the sleep period. The CPU 21 determines the representative state of the robot 10 in a certain period to be the "sleepless state" in the case where the number of times of entering the sleep state of the robot 10 in the recording period of the log 133 corresponding to the period is equal to or more than a given number of times (4 times or more in this embodiment) in the log DB 233. Then, the CPU 21 causes the display unit 24 to display the timeline information 441 indicating that the robot 10 is in the sleepless state in the period (for example, "seems to be in a daze"). In the case where the number of times of entering the sleep state is less than the given number of times, the CPU 21 determines the representative state of the robot 10 in the period to be the "sleep state". Then, the CPU 21 causes the display unit 24 to display the timeline information 441 indicating that the robot 10 is in the sleep state in the period (for example, "seems to be in a deep sleep"). Figure 16 In the example shown in FIG. 12, the robot 10 entered the sleep state 4 times in the recording periods P12 of the logs Li and Lj corresponding to the period from 12 o'clock to 13 o'clock, and therefore the CPU 21 determines the representative state of the robot 10 in the period to be the "sleepless state". On the other hand, in the recording periods P13 of the logs Lk and Li corresponding to the period from 13 o'clock to 14 o'clock, the robot 10 entered the sleep state 3 times, and in addition, the sleep period in the recording period P13 is less than 50 minutes, and therefore the CPU 21 determines the representative state of the robot 10 in the period to be neither the "sleepless state" nor the "sleep state".

[0069] In order to display the timeline information 441 relating to external stimuli received through the exchange with the user, the CPU 21 refers to the number of stimuli information 77 of each log 133 in the log DB 233. The CPU 21 causes the display portion 24 to display the timeline information 441 indicating that the robot 10 received an external stimulus during the recording period of the log 133 corresponding to a certain time period, in a case where the robot 10 received a given external stimulus during the recording period. Further, the CPU 21 selects a certain external stimulus from among a plurality of external stimuli in accordance with a given priority order relating to the external stimuli, in a case where there is a history of receiving a different plurality of external stimuli during the recording period, and displays the timeline information 441 indicating that the selected certain external stimulus was received. For example, the CPU 21 causes the timeline information 441 indicating that the robot 10 received a certain external stimulus from among a plurality of external stimuli to be displayed, in a case where there is a history of receiving the external stimulus a given number of times or more. Further, the CPU 21 causes the timeline information 441 indicating the external stimulus that was received the most number of times from among a plurality of external stimuli to be displayed. Further, the priority order of all external stimuli that can be detected can be determined in advance, and the timeline information 441 indicating the external stimulus that is highest in the priority order from among a plurality of external stimuli received during the recording period of the log 133 corresponding to a certain time period can be displayed. Further, the CPU 21 can cause the timeline information 441 indicating that the robot 10 accepted a given positive exchange from among a plurality of different exchanges to be displayed, in a case where the external stimulus resulting from the positive exchange is included in the plurality of external stimuli. The positive exchange can include, for example, stroking the head, stroking the body, swinging left and right, and picking up, etc. As for a given exchange, an exchange of 2 or more given times can be counted as 1. For example, for "stroking", which is prone to be repeated, 3 strokes can be counted as 1. In Figure 10 In the example shown, the timeline information 441c indicating stroking of the head is displayed, reflecting the fact that the number of times of exchange of stroking the head during the time period from 14:00 to 15:00 was the most.

[0070] In a case where the mood satisfaction of the robot 10 during the recording period of the log 133 corresponding to a certain time period displays a given display condition, the CPU 21 causes the timeline information 441 relating to the mood of the robot 10 to be displayed for the time period. For example, as shown in Fig. 12, the timeline information 441 indicating that the mood of the robot 10 was good is displayed, in a case where the mood satisfaction of the robot 10 during the recording period of the log 133 corresponding to a certain time period was 80% or more. Figure 13As shown, if the display conditions for emotion icon 433 are met, CPU 21 determines that the display conditions for timeline information 441 are also met, and displays the timeline information 441 related to the emotion of robot 10 for that time period. If there are multiple changes in emotion within a certain time period, CPU 21 can also display the timeline information 441 representing the subsequent emotions within the initial emotion change. Furthermore, even without emotion changes, if robot 10 receives a positive external stimulus during a certain time period and its emotion is positive, CPU 21 also determines that the display conditions for timeline information 441 are met, and displays the timeline information 441 related to that emotion. Figure 10 In the example shown, during the recording of log 133 corresponding to the time period from 3 PM to 4 PM, CPU 21 displays timeline information 441d representing the emotion of "joy".

[0071] Next, refer to Figure 17 The timeline display processing performed by CPU 21 for displaying timeline 44 will be described. The timeline display processing begins after the management application 231 starts, while the interactive recording screen 40 is displayed. When the timeline display processing begins, CPU 21 determines whether the acquisition of log 133 corresponding to the previously just-concluded time period is complete (step S301). If it is determined that the acquisition of log 133 is complete (yes in step S301), CPU 21 determines whether the length of the sleep period during the recording period of log 133 meets the sleep time condition (step S302). In this embodiment, if the length of the sleep period during the recording period of log 133 corresponding to the previously just-concluded time period is 50 minutes or more, CPU 21 determines that the sleep time condition is met. If it is determined that the sleep time condition is met (yes in step S302), CPU 21 determines whether there is a positive external stimulus during sleep (step S303). If it is determined that there is a positive external stimulus during sleep (yes in step S303), CPU 21 displays the timeline information 441 of sleep behavior (step S304). On the other hand, if it is determined that there is no positive external stimulus during sleep (no in step S30), CPU 21 displays the timeline information 441 of normal sleep (step S305). In step S302, if it is determined that the sleep time condition is not met (no in step S302), CPU 21 determines whether the number of times the user entered a sleep state during the recording period of log 133 corresponding to the previous time period is more than 4 times (step S306). If it is determined that the number of times the user entered a sleep state is more than 4 times (yes in step S306), CPU 21 displays the timeline information 441 of the insomnia state (step S307).

[0072] In a case where any one of steps S304, S305, S307 ends, or in a case where it is determined that the number of times of becoming the sleep state is less than 4 times (NO in step S306), the CPU 21 determines whether or not the robot 10 performed a given spontaneous action in the recording period of the log 133 corresponding to the time period immediately before (step S308). In a case where it is determined that the given spontaneous action was performed (YES in step S308), the CPU 21 causes the timeline information 441 of the spontaneous action to be displayed (step S309). In a case where step S309 ends, or in a case where it is determined that the given spontaneous action was not performed (NO in step S308), the CPU 21 determines whether or not the robot 10 was subjected to an external stimulus caused by an exchange with the user in the recording period of the log 133 corresponding to the time period immediately before (step S310). In a case where it is determined that the robot 10 was subjected to the external stimulus caused by the exchange (YES in step S310), the CPU 21 decides one external stimulus in accordance with the given priority order as described above, causes the timeline information 441 of the decided external stimulus to be displayed (step S311). In a case where step S311 ends, or in a case where it is determined that the robot 10 was not subjected to the external stimulus caused by the exchange (NO in step S310), the CPU 21 determines whether or not the emotion in the recording period of the log 133 corresponding to the time period immediately before satisfies the display condition of the timeline information 441 described above (step S312). In a case where it is determined that the emotion in the recording period satisfies the display condition of the timeline information 441 (YES in step S312), the CPU 21 causes the timeline information 441 related to the emotion to be displayed (step S313). In a case where step S313 ends, or in a case where the branch is NO in any one of steps S301, S312, the CPU 21 determines whether or not to end the management application 231 in accordance with the operation of the user (step S314). The CPU 21 returns the process to step S301 in a case where the management application 231 is not ended (NO in step S314), and ends the timeline display process in a case where the management application 231 is ended (YES in step S314).

[0073] In a case where the execution of the management application 231 initially shifts to the home screen 30 after a certain condition is satisfied, Figure 18The illustrated evaluation screen 50 is displayed overlapping the home screen 30. The evaluation screen 50 is a screen for the user to input an evaluation of the management application 231. In the evaluation screen 50, there are displayed a character of "Please click the stars to send the evaluation of the 'application'", five star marks 51 for inputting an evaluation of five levels, a send button 52, and a cancel button 53. The "application" included in the character is actually the application name of the management application 231. When the user selects the Nth (N is any one of 1-5) star mark 51 from the left by clicking or the like, the Nth star mark 51 from the left is colored. Thus, as the evaluation of the management application 231, an evaluation of the Nth stage out of five stages is input. When the user performs an operation of selecting the send button 52 in a state where the evaluation is input, the CPU 21 sends information of the input evaluation to an application store server, which is omitted from illustration. The sent information of the evaluation is reflected in the display of the evaluation of the management application 231 in the application store provided by the application store server.

[0074] The CPU 21 causes the evaluation screen 50 to be displayed in a case where any one of first to third conditions below is satisfied. The first condition is satisfied in a case where the user opens the setting screen of the robot 10 a given number of times in total (6 times or more in the present embodiment). As described above, by displaying the evaluation screen 50 at a timing where the first condition is satisfied, it is expected that a higher evaluation is input in a state where the user is somewhat satisfied with the robot 10 and the management application 231. Figure 8 The setting screen is displayed by selecting the setting button 36 in the illustrated home screen 30. By the user changing the setting of the robot 10 in the setting screen, it can be considered that the robot 10 is customized to conform to the user's preference, environment. Therefore, by displaying the evaluation screen 50 at a timing where the first condition is satisfied, it is expected that a higher evaluation is input in a state where the user is somewhat satisfied with the robot 10 and the management application 231.

[0075] The second condition and the third condition are satisfied in a case where it is determined based on the log DB 233 that the history of the state of the robot 10 satisfies a given condition. Among them, the second condition is satisfied in a case where the number of times the emotion of the robot 10 changes to a given emotion satisfies a certain condition. For example, the second condition can be satisfied in a case where the number of days on which the change of the emotion of the robot 10 to a given emotion occurs a given number of times or more is a given number of days or more. Here, the change of the emotion of the robot 10 to a given emotion can be a change of the emotion that satisfies the display condition of the emotion icon 433 described above. For example, the second condition can be satisfied in a case where the number of days on which the emotion icon 433 is displayed two or more times is three or more days. The lower limit values of the number of times the emotion icon 433 is displayed in a day and the number of days in total can be appropriately changed.

[0076] The third condition is satisfied in a case where the history of the emotion of the robot 10 and the level thereof based on the log DB 233 satisfies a certain condition. For example, the third condition can also be satisfied in a case where the number of times that the level of a given emotion is equal to or higher than a given reference is equal to or higher than a reference number. For example, the third condition can also be satisfied in a case where, for each of two or more of the positive emotions ("excited", "joy", "relaxed", and "calm"), the number of times that the level reaches the maximum level 10 is two or more. In a case where a certain emotion reaches the level 10, the state image 31 corresponding to the level 10 of the emotion can also be displayed in the top page screen 30. The level of the emotion for which the third condition is satisfied, the lower limit value of the number of times of achievement thereof can also be changed as appropriate.

[0077] In addition, in a case where the evaluation screen 50 is displayed in a case where any one of the first to third conditions is satisfied, the count related to the satisfied condition (in the case of the first condition, the number of times of display of the setting screen, in the case of the second condition, the number of times of display of the emotion icon 433, and in the case of the third condition, the number of times that the positive emotion reaches the maximum level 10) is reset. Further, in a case where a plurality of robots 10 cooperate, the first to third conditions can also be determined based on the total count of the plurality of robots 10. Further, a case where a given period (for example, one week) has elapsed from the day on which the management application 231 is installed in the smartphone 20 and initially started can also be set as a condition for displaying the evaluation screen 50. Further, in a case where the CPU 21 performs the process of causing the evaluation screen 50 to be displayed in the display section 24 by the management application 231, whether or not the evaluation screen 50 is actually displayed can also be limited by the OS (Operating System) of the smartphone 20. For example, the total number of times of display in a year, the frequency of display in the most recent given period (for example, one month) can be limited by the OS. In this case, the process of causing the evaluation screen 50 to be displayed in the display section 24 by the CPU 21 according to the management application 231 includes a process of outputting a display request of the evaluation screen 50 to the OS.

[0078] Next, the evaluation screen 50 will be described with reference to FIG. 6. Figure 19The evaluation screen display process performed by the CPU 21 for displaying the evaluation screen 50 will be described. The evaluation screen display process is started when the management application 231 is launched. When the evaluation screen display process is started, the CPU 21 determines whether to shift the display in the display section 24 from other screens to the home screen 30 (step S401). In the case of shifting to the home screen 30 (YES in step S401), the CPU 21 determines whether the number of times of display of the setting screen is equal to or more than a given number of times (6 or more in the present embodiment) (step S402). In the case of determining that the number of times of display of the setting screen is equal to or more than the given number of times (YES in step S402), the CPU 21 causes the evaluation screen 50 to be displayed so as to overlap the home screen 30 (step S405). In the case of determining that the number of times of display of the setting screen is less than the given number of times (NO in step S402), the CPU 21 determines whether the days on which the mood icon 433 has been displayed two or more times are equal to or more than three days in total (step S403). In the case of determining that the days on which the mood icon 433 has been displayed two or more times are equal to or more than three days in total (YES in step S403), the CPU 21 causes the evaluation screen 50 to be displayed so as to overlap the home screen 30 (step S405). In the case of determining that the days on which the mood icon 433 has been displayed two or more times are less than three days in total (NO in step S403), the CPU 21 determines whether the number of times of reaching the level 10 is two or more for each of two or more positive moods (step S404). In the case of determining that the number of times of reaching the level 10 is two or more for each of two or more positive moods (YES in step S404), the CPU 21 causes the evaluation screen 50 to be displayed so as to overlap the home screen 30 (step S405). After the evaluation screen 50 is displayed in step S405, the CPU 21 determines whether the input of the evaluation using the star mark 51 and the transmission operation (operation of selecting the transmission button 52) have been performed (step S406). In the case of determining that the input of the evaluation and the transmission operation have been performed (YES in step S406), the CPU 21 transmits the information of the input evaluation to the application store server (step S407). In the case where step S407 ends or in the case where the branch is NO in any one of steps S401, S404, and S406, the CPU 21 determines whether to end the management application 231 according to the operation of the user (step S408). The CPU 21 returns the process to step S401 without ending the management application 231 (NO in step S408), and ends the evaluation screen display process when the management application 231 is ended (YES in step S408). The order of steps S402 to S404 can also be changed.

[0079] As described above, in the display control method of the present embodiment, the CPU 21 performs processing of determining a change timing at which the state of the robot 10 changes from a certain state to a given state, on the basis of the log DB 233 to which the history of the state of the robot 10 is related. The CPU 21 performs processing of causing the display section 24 to display a coordinate axis 431 indicating the passage of time, and an emotion icon 433 indicating that the state of the robot 10 changes to the given state as an index, which is arranged at a position in the coordinate axis 431 corresponding to the change timing.

[0080] Conventionally, the state of an object such as a robot is maintained as an internal parameter of the object, and therefore it is not necessarily easy to accurately grasp the history of the state from the appearance of the object.

[0081] According to the present disclosure, it is possible to easily grasp the history of the state of an object.

[0082] With the display control method according to the present embodiment, it is possible to easily grasp the change in the state of the robot 10, which does not easily occur from the appearance of the robot 10, and the timing at which the change occurs, with an index. Therefore, it is possible to easily grasp the history of the state of the robot.

[0083] Further, the log DB 233 contains a history of an emotion parameter 75 indicating a virtual emotion that is the state of the robot 10. The CPU 21 performs processing of determining a change timing at which the emotion of the robot 10 changes from a certain emotion to a given emotion, on the basis of the log DB 233. The CPU 21 performs processing of causing the display section 24 to display a coordinate axis 431 and an emotion icon 433 indicating that the emotion of the robot 10 changes to the given emotion. Thereby, it is possible to easily grasp the change in the emotion of the robot 10, which does not easily occur from the appearance of the robot 10, and the timing at which the change occurs.

[0084] Further, the emotion parameter 75 indicates a coordinate in a coordinate plane that is divided into a plurality of regions R1 to R9 corresponding to a plurality of emotions that are different from each other. The CPU 21 performs processing of causing the display section 24 to display the emotion icon 433 indicating that the emotion of the robot 10 changes to the given emotion, in a case where the emotion parameter 75 changes from a coordinate within a region corresponding to a certain emotion to a coordinate within a region corresponding to a given emotion, among the plurality of regions R1 to R9. Thereby, it is possible to determine the emotion of the robot 10 and the change thereof by simple processing of determining the coordinate of the emotion parameter 75 in the emotion map.

[0085] Further, the emotion parameter 75 indicates any one of a plurality of emotions that are different from each other, and the given emotion is any one of positive emotions, i.e., "excitement", "joy", "relief", and "calm", among the plurality of emotions. Thereby, it is possible to make the user recognize that there is a positive change in the emotion of the robot 10. Thus, it is possible to easily generate attachment to the robot 10.

[0086] Further, the emotion parameter 75 indicates a level of emotion. The CPU 21 executes the process of causing the emotion icon 433 to be displayed on the display section 24 in a case where the level of emotion at the change timing is above a given reference. Thereby, the user can recognize a state in which the emotion of the robot 10 becomes rich. Thus, it is possible to easily cause the robot 10 to be attached.

[0087] Further, the CPU 21 executes the process of determining the start timing of the second period as the change timing in a case where the state of the robot 10 indicated by the log 133 as the first history record information is a certain state and the state of the robot 10 indicated by the log 133 as the second history record information is a given state, based on the log 133 as the first history record information related to the state of the robot 10 in the first period and the log 133 as the second history record information related to the state of the robot 10 in the second period after the first period. Thereby, it is possible to determine the change of the emotion of the robot 10 and the change timing thereof by a simple process of comparing the emotions in the two logs 133.

[0088] Further, the CPU 21 executes the process of determining the start timing of the second period as the change timing only in a case where the period from the end of the first period to the start of the second period is within a given time. Thereby, it is possible to display the emotion icon 433 only in a case where it is possible to determine the change timing of the emotion to some extent.

[0089] Further, the robot management system 1 according to the present embodiment includes the server 60 and the display control device 200 having the CPU 21 that executes the above-described processes. Further, the robot management system 1 according to the present embodiment includes the robot 10 and the display control device 200 having the CPU 21 that executes the above-described processes. Thereby, it is possible to easily grasp the change of the state of the robot 10 in which the appearance of the robot 10 is difficult to appear and the timing at which the change occurs using the index.

[0090] In addition, the present disclosure is not limited to the above-described embodiments, and various modifications can be made. For example, in the above-described embodiments, a manner in which the home screen 30 and the interaction record screen 40 are caused to be displayed by the smartphone 20 executing various processes according to the management application 231 is exemplified, but is not limited thereto. For example, it is also possible to cause the home screen 30 and the interaction record screen 40 to be displayed by a server (not illustrated) provided outside the smartphone 20 Figure 1The server 60 and the like illustrated transmits data for displaying the top page 30 and / or the interaction record page 40 in the display section 24 to the smartphone 20 to control the display section 24 of the smartphone 20. In this case, the computer of the server executes an information processing method that generates data for causing the CPU 21 of another computer to execute the following processing. The above-described data is "data for causing the CPU 21 (another computer) to execute the following processing: determining a change timing at which the state of the robot 10 changes from a certain state to a given state based on the log DB 233 involved in the history of the state of the robot 10; causing the display section 24 to display a coordinate axis 431 indicating the passage of time and an emotion icon 433 that is arranged at a position in the coordinate axis 431 corresponding to the change timing as an index and indicates that the state of the robot 10 changes to the given state". Alternatively, the processing of "determining a change timing at which the state of the robot 10 changes from a certain state to a given state based on the log DB 233 involved in the history of the state of the robot 10" can also be executed by the computer of the server. In this case, the above-described data is "data for causing the CPU 21 (another computer) to execute processing of causing the display section 24 to display a coordinate axis 431 indicating the passage of time and an emotion icon 433 that is arranged at a position in the coordinate axis 431 corresponding to the change timing as an index and indicates that the state of the robot 10 changes to the given state". This data can also include data specifying the content and configuration of the interaction record page 40, such as image data, HTML (Hyper Text Markup Language) data, and the like. Furthermore, this data can also include control information for controlling the action of the display section 24. Furthermore, this data can also be a program for displaying the interaction record page 40 in the display section 24. In the above-described embodiment, at least a part of the processing executed by the CPU 21 can also be executed by the computer of the server. In this case, the computer of the server can correspond to "one or more processing sections". Furthermore, the CPU 21 of the display control device 200 and the computer of the server can constitute "one or more processing sections".

[0091] Furthermore, in Figure 13 , Figure 15 and Figure 16 , a manner in which the object period of the timeline information 441 (a time period from the hour to the next hour) does not coincide with the recording period of the log 133 is exemplified, but is not limited thereto, and the recording period of the log 133 can coincide with the object period of the timeline information 441.

[0092] Furthermore, the content of the log 133 is not limited to Figure 4The content shown can also include other elements that represent the state of the robot 10. For example, as elements of the state of the robot 10, elements such as fatigue, sleep, physical condition, and the like can be listed.

[0093] Further, the manner in which the top page 30 and the interaction record page 40 are displayed on the display portion 24 of the smartphone 20 is exemplified, but is not limited thereto. For example, in the case where the robot 10 has a display portion, the top page 30 and the interaction record page 40 can also be displayed on the display portion. In this case, the control involved in the display of the top page 30 and the interaction record page 40 can be executed by the CPU 11 of the robot 10, or can be executed remotely by the processing portion of an external device such as the CPU 21 of the smartphone 20.

[0094] Further, the structure of the robot 10 is not limited to Figures 1-3 the exemplified structure. For example, it can be a robot that imitates a living being that actually exists, such as a human, an animal, a bird, a fish, or the like, a robot that imitates a living being that does not exist, such as a dinosaur, or a robot that imitates a fictional living being, or the like.

[0095] Further, in the above-described embodiment, the robot 10 is exemplified as the "object", but is not limited thereto. The "object" can be an object that becomes a management target of the management application 231. For example, the "object" can also be an arbitrary object that represents a change in a parameter of the state. Further, the "object" can also be a virtual avatar that acts in place of a user in a virtual space such as a meta universe.

[0096] Further, in the above description, as the computer-readable medium of the program related to the present disclosure, an example in which a flash memory of the storage portion 13, 23 is used is disclosed, but is not limited to this example. As other computer-readable media, an information recording medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a CD-ROM, or the like can be applied. Further, as a medium that provides data of the program related to the present disclosure via a communication line, a carrier wave is also applicable to the present disclosure. Further, the detailed configuration and the detailed action of each structural element of the robot 10 and the smartphone 20 in the above-described embodiment can of course be appropriately changed within a range that does not depart from the gist of the present disclosure. The embodiments of the present disclosure have been described, but the scope of the present disclosure is not limited to the above-described embodiments, and includes the scope of the invention recited in the claims and the equivalent range thereof.

Claims

1. A computer program product, characterized in that, The computer program product causes the computer's processing unit to perform the following processes: Based on the historical information involved in the object's state history, the timing of the change of the object's state from a certain state to a given state is determined. and The display unit shows a coordinate axis representing the elapsed time and an indicator, the indicator being positioned on the coordinate axis corresponding to the change timing, indicating that the state of the object has changed to the given state.

2. The computer program product according to claim 1, wherein, The historical record information includes a history of emotional parameters representing the virtual emotion of the state of the object. The computer program product causes the computer's processing unit to perform the following processing: Based on the historical data, determine the timing of the change in the object's emotion from a certain emotion to a given emotion; and The display unit displays the coordinate axis and the index representing the emotional change of the object as the given emotion.

3. The computer program product according to claim 2, wherein, The emotion parameter represents the coordinates in a coordinate plane that is divided into multiple regions corresponding to multiple distinct emotions. The computer program product causes the computer's processing unit to perform the following processing: When the coordinates of the emotion parameter change from the coordinates in the region corresponding to the certain emotion among the plurality of regions to the coordinates in the region corresponding to the given emotion, the display unit displays the index indicating that the object's emotion has changed to the given emotion.

4. The computer program product according to claim 2, wherein, The emotion parameter represents any one of a plurality of distinct emotions. The given emotion is at least one given positive emotion among the plurality of emotions.

5. The computer program product according to claim 2, wherein, The emotion parameter represents the degree of the emotion. The computer program product causes the computer's processing unit to perform the following processing: When the level of the given emotion at the time of the change is above a given benchmark, the display unit displays the indicator.

6. The computer program product according to claim 1, wherein, The computer program product causes the computer's processing unit to perform the following processing: Based on the first historical information related to the state of the object in the first period and the second historical information related to the state of the object in the second period after the first period, when the state of the object represented by the first historical information is the certain state and the state of the object represented by the second historical information is the given state, the start timing of the second period is determined as the change timing.

7. The computer program product according to claim 6, wherein, The computer program product causes the computer's processing unit to perform the following processing: The start timing of the second period is determined as the change timing only if the period from the end of the first period to the start of the second period is within a given time.

8. A system, characterized in that, The processing unit of the system, including the server and the display control device, performs the following processing: Based on the historical information related to the object's state history, the timing of the change from a certain state to a given state is determined. The display unit shows a coordinate axis and an indicator representing the elapsed time. The indicator is positioned on the coordinate axis at a position corresponding to the change timing, indicating that the state of the object has changed to the given state.

9. A system, characterized in that, The processing unit of the system, which includes the object and the display control device, performs the following processing: Based on the historical information related to the object's state history, the timing of the change from a certain state to a given state is determined. The display unit shows a coordinate axis and an indicator representing the elapsed time. The indicator is positioned on the coordinate axis at a position corresponding to the change timing, indicating that the state of the object has changed to the given state.

Images