Information processing device, information processing method, and program

Abstract

[Problem] For enhancing communication between an autonomous operating body and a user. [Solution] The present disclosure provides an information processing device having an application control unit and a reflection unit, the application control unit controlling the operation of an application related to communication between an autonomous operating body and a user; the reflection unit causing the autonomous operating body to reflect an operation performed by the user in the application. The application control unit controls the operation of an avatar simulating the autonomous operating body. The reflection unit reflects a reward obtained by the avatar based on the operation performed by the user on the autonomous operating body.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. Background Art

[0002] In recent years, various devices with learning functions have been developed. Such devices include autonomous operating bodies such as robots that perform autonomous operations based on estimated states. For example, Patent Document 1 discloses a robot device that performs information acquisition behavior as one of the behaviors in autonomous operations.

[0003] Reference List

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2002-219677 Summary of the invention

[0006] Problems to be solved by the present invention

[0007] At the same time, even when the user is absent, the autonomous operating body as described in Patent Document 1 can perform various autonomous behaviors. However, if the situation in which the autonomous operating body and the user can communicate with each other is limited to the situation in which both share the same physical space, the communication becomes intermittent, and the user may not be able to fully enjoy the advantages of having the autonomous operating body.

[0008] Therefore, the present disclosure proposes a new and improved information processing device, information processing method, and program, which can further activate the communication between the autonomous operating body and the user.

[0009] Solution to the problem

[0010] The present disclosure provides an information processing device, comprising: an application control unit that controls the operation of an application related to communication between an autonomous operating body and a user; and a reflection unit that causes the autonomous operating body to reflect the operation performed by the user in the application. In the information processing device, the application control unit controls the operation of an avatar that imitates the autonomous operating body, and the reflection unit causes the autonomous operating body to reflect a reward obtained by the avatar based on the operation performed by the user.

[0011] The present disclosure also provides an information processing method implemented by a processor, the information processing method comprising: controlling the operation of an application related to communication between an autonomous operating body and a user; and causing the autonomous operating body to reflect the operation performed by the user in the application. In the information processing method, the controlling further comprises controlling the operation of an avatar that mimics the autonomous operating body, and the causing further comprises causing the autonomous operating body to reflect a reward obtained by the avatar based on the operation performed by the user.

[0012] The present disclosure also provides a program for causing a computer to function as an information processing device, comprising: an application control unit that controls the operation of an application related to communication between an autonomous operating body and a user; and a reflection unit that causes the autonomous operating body to reflect an operation performed by the user in the application. The application control unit controls the operation of an avatar that imitates the autonomous operating body, and the reflection unit causes the autonomous operating body to reflect a reward obtained by the avatar based on the operation performed by the user.

[0013] Effects of the Invention

[0014] As described above, according to the present disclosure, it is possible to make the communication between the autonomous operating body and the user more active.

[0015] Note that the above-described effects are not necessarily limitative, and any of the effects described in this specification may be achieved together with or instead of the above-described effects, or other effects that are obvious from the present specification may be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] [ Figure 1 ] is a diagram showing an exemplary hardware configuration of an autonomous operating body according to an embodiment of the present disclosure.

[0017] [ Figure 2 ] is an exemplary configuration of an actuator included in an autonomous operating body according to an embodiment of the present disclosure.

[0018] [ Figure 3 ] is a diagram for explaining the operation of an actuator included in an autonomous operating body according to an embodiment of the present disclosure.

[0019] [ Figure 4 ] is a diagram for explaining the operation of an actuator included in an autonomous operating body according to an embodiment of the present disclosure.

[0020] [ Figure 5 ] is a diagram for explaining the function of a display included in an autonomous operating body according to an embodiment of the present disclosure.

[0021] [ Figure 6 ] is a diagram illustrating exemplary operations of an autonomous operating body according to an embodiment of the present disclosure.

[0022] [ Figure 7 ] is a diagram showing an exemplary system configuration according to an embodiment of the present disclosure.

[0023] [ Figure 8 ] is a diagram illustrating an exemplary functional configuration of an autonomous operating body according to an embodiment of the present disclosure.

[0024] [ Fig. 9] is a diagram showing an exemplary functional configuration of an information processing server according to an embodiment of the present disclosure.

[0025] [ Fig.10 ] is a diagram showing an exemplary functional configuration of an information processing terminal according to an embodiment of the present disclosure.

[0026] [ Fig.11 ] is a diagram illustrating an example of a home screen of an application according to an embodiment of the present disclosure.

[0027] [ Fig.12 ] is a diagram showing an example of a home screen of an application according to the present embodiment.

[0028] [ Fig.13 ] is an example of colors showing emotions of the autonomous operating body according to the present embodiment.

[0029] [ Fig.14 ] is an example of an application menu screen according to the present embodiment.

[0030] [ Fig.15 ] is an example of a notification screen according to the present embodiment.

[0031] [ Fig.16 ] is an example of a content list screen according to the present embodiment.

[0032] [ Fig.17 ] is an example of a setting contents screen according to the present embodiment.

[0033] [ Fig.18 ] is an example of an appearance change content screen according to the present embodiment.

[0034] [ Fig.19 ] is an example of an appearance change screen for color representation of an eyeball according to the present embodiment.

[0035] [ Fig. 20 ] is a diagram for explaining reflection of settings caused by a reflection unit according to the present embodiment.

[0036] [ Fig.21 ] is an example of a photo content screen according to the present embodiment.

[0037] [ Fig. 22 ] is an example of a physical condition check content screen according to the present embodiment.

[0038] [ Fig.23 ] is an example of a game content screen according to the present embodiment.

[0039] [ Fig.24 ] is an example of a game content description screen according to the present embodiment.

[0040] [ Fig.25 ] is a diagram for explaining the interaction between the autonomous operating body and the user via the spherical toy according to the present embodiment.

[0041] [ Fig.26 ] is a diagram for explaining the interaction between the autonomous operating body and the user via the bone-shaped toy according to the present embodiment.

[0042] [ Fig. 27 ] is an example of a reward acquisition pop-up window according to the present embodiment.

[0043] [ Fig.28 ] is a flowchart showing the flow of appearance change setting control to be performed by the information processing server according to the present embodiment.

[0044] [ Fig.29 ] is a flowchart showing the process of game content control to be performed by the information processing server according to the present embodiment.

[0045] [ Fig.30 ] is a diagram showing an exemplary hardware configuration according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] Preferred embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. Note that in this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and their description will not be repeated.

[0047] Note that the description will be given in the following order.

[0048] 1. Configuration

[0049] 1.1. Overview of Autonomous Operation Object 10

[0050] 1.2. Exemplary Hardware Configuration of Autonomous Operating Entity 10

[0051] 1.3. Example System Configuration

[0052] 1.4. Exemplary Functional Configuration of Autonomous Operating Entity 10

[0053] 1.5. Exemplary Functional Configuration of Information Processing Server 20

[0054] 1.6. Exemplary Functional Configuration of Information Processing Terminal 30

[0055] 2. First embodiment

[0056] 2.1. Overview

[0057] 2.2. Application details

[0058] 2.3. Control Flow

[0059] 3. Example Hardware Configuration

[0060] 4. Summary

[0061] <1. Configuration>

[0062] <<1.1. Overview of Autonomous Operation Object 10>>

[0063] First, an overview of the autonomous operating body 10 according to an embodiment of the present disclosure is described. The autonomous operating body 10 according to an embodiment of the present disclosure is an information processing device that estimates a situation based on collected sensor information and autonomously selects and performs various operations according to the state. Unlike a robot that performs operations only according to instruction commands from a user, the autonomous operating body 10 autonomously performs operations estimated to be optimal in each situation.

[0064] Therefore, the autonomous operating body 10 according to the embodiment of the present disclosure does not perform an operation corresponding to the user's instruction, or performs a behavior different from the operation in some cases, depending on the situation. The above situation corresponds to, for example, a situation where an operation corresponding to the user's instruction is performed; a situation where the safety of the user or the autonomous operating body 10 or the surrounding environment is compromised; a situation where the autonomous operating body 10 takes precedence over other expectations (instincts) such as a charging process, etc.

[0065] Furthermore, there are cases where the autonomous operating body 10 does not intentionally follow the user's instructions to attract the user's attention or notify the user of its own emotions or the state of hardware.

[0066] On the other hand, the autonomous operating body 10 has a strong desire (instinct) to be liked by the user. Therefore, the autonomous operating body 10 repeatedly performs operations corresponding to user instructions to please the user even without instructions, or learns operations that the user likes and spontaneously performs the operations.

[0067] As described above, the autonomous operating body 10 according to the embodiment of the present disclosure comprehensively judges its desires and emotions, or the surrounding environment, etc., and determines and performs autonomous operations, such as animals, including humans. In the above aspects, the autonomous operating body 10 is obviously different from a passive device that performs corresponding operations according to instructions.

[0068] The autonomous operating body 10 according to an embodiment of the present disclosure may be an autonomous mobile robot that moves autonomously in space and performs various operations. The autonomous operating body 10 may be an autonomous mobile robot in the shape of an animal such as a human or a dog and has, for example, operating capabilities. Alternatively, the autonomous operating body 10 may be, for example, a vehicle or other device that has the ability to communicate with a user. The shape, capabilities, expectations, etc. of the autonomous operating body 10 according to an embodiment of the present disclosure may be appropriately designed according to the purpose and role.

[0069] <<1.2. Exemplary Hardware Configuration of Autonomous Operating Entity 10>>

[0070] Next, an exemplary hardware configuration of the autonomous operating body 10 according to the embodiment of the present disclosure is described. Note that in the example case described below, the autonomous operating body 10 is a dog-shaped quadruped walking robot.

[0071] Figure 1 1 is a diagram showing an exemplary hardware configuration of the autonomous operating body 10 according to an embodiment of the present disclosure. Figure 1 As shown, the autonomous operating body 10 is a dog-shaped quadruped walking robot having a head, a trunk, four legs and a tail. The autonomous operating body 10 also includes two displays 510 on the head.

[0072] The autonomous operating body 10 also includes various sensors. For example, the autonomous operating body 10 includes a microphone 515, a camera 520, a time-of-flight (ToF) sensor 525, a human sensing sensor 530, a distance measurement sensor 535, a touch sensor 540, an illumination sensor 545, a single button 550, and an inertial sensor 555.

[0073] (Microphone 515)

[0074] The microphone 515 has the function of collecting surrounding sounds. For example, the above-mentioned sounds include the user's speech and the surrounding environment sounds. For example, the autonomous operating body 10 may include four microphones on the head. Since a plurality of microphones 515 are included, the sounds generated in the surrounding environment can be collected with high sensitivity, and the location of the sound source can be achieved.

[0075] (Camera 520)

[0076] The camera 520 has a function of imaging the user and the surrounding environment. For example, the autonomous operating body 10 may include two wide-angle cameras at the tip of the nose and at the waist. In this case, the wide-angle camera disposed at the tip of the nose captures an image corresponding to the front field of view of the autonomous operating body (i.e., the field of view of the dog), and the wide-angle camera at the waist captures an image of the surrounding area centered on the above space. The autonomous operating body 10, for example, can perform simultaneous localization and mapping (SLAM) by extracting feature points of the ceiling, etc. based on the image captured by the wide-angle camera disposed at the waist.

[0077] (ToF sensor 525)

[0078] The ToF sensor 525 has a function of detecting the distance of an object present in front of the head. The ToF sensor 525 is provided at the nose tip of the head. With the ToF sensor 525, the distance to various objects can be detected with high accuracy, and operations can be performed according to the relative position relative to objects including the user, obstacles, etc.

[0079] (Human sensor 530)

[0080] The human sensing sensor 530 has a function of detecting whether a user, a pet kept by the user, etc. exists. For example, the human sensing sensor 530 is set on the chest. Using the human sensing sensor 530, a moving object existing in front can be detected, and various operations such as operations corresponding to emotions including interest, fear, and surprise can be performed on the moving object.

[0081] (Distance measurement sensor 535)

[0082] The distance measuring sensor 535 has a function of acquiring the state of the floor in front of the autonomous operating body 10. The distance measuring sensor 535 is provided, for example, on the chest. Through the distance measuring sensor 535, the distance to an object existing on the floor in front of the autonomous operating body 10 can be detected with high accuracy, and an operation can be performed according to the position relative to the object.

[0083] (Touch sensor 540)

[0084] The touch sensor 540 has a function of detecting user contact. The touch sensor 540 is configured at a position where the user easily touches the autonomous operating body 10, such as the top of the head, the chin, the back, etc. For example, the touch sensor 540 can be a capacitive or pressure-sensitive touch sensor. With the touch sensor 540, the user's contact actions such as touching, clicking, tapping, and pressing can be detected, and an operation corresponding to the contact action can be performed.

[0085] (illuminance sensor 545)

[0086] The illuminance sensor 545 detects the illuminance of the space where the autonomous operating body 10 exists. The illuminance sensor 545 may be provided at, for example, the base of the tail on the back of the head, etc. With the illuminance sensor 545, the surrounding brightness can be detected, and an operation corresponding to the brightness can be performed.

[0087] (Single button 550)

[0088] The individual buttons 550 have a function of detecting whether the bottom surface of the legs of the autonomous operating body 10 is in contact with the floor. Therefore, the individual buttons 550 are provided at the portions corresponding to the claws of the four legs. Through the individual buttons 550, it is possible to detect contact or non-contact between the autonomous operating body 10 and the floor surface, for example, to recognize that the autonomous operating body 10 is lifted by the user, etc.

[0089] (Inertial sensor 555)

[0090] The inertial sensor 555 is a six-axis sensor that detects physical quantities such as speed, acceleration, and rotation of the head and trunk. That is, the inertial sensor 555 detects acceleration and angular velocity of the X-axis, Y-axis, and Z-axis. The inertial sensor 555 is provided on the head and trunk. Using the inertial sensor 555, the movement of the head and trunk of the autonomous operating body 10 can be detected with high precision, and movement control depending on the situation can be performed.

[0091] Examples of sensors included in the autonomous operating body 10 according to an embodiment of the present disclosure have been described above. Figure 1 The described configuration is only an example, and the configuration of the sensors included in the autonomous operating body 10 is not limited to this example. For example, in addition to the above configuration, the autonomous operating body 10 may also include a temperature sensor, a geomagnetic sensor, various communication devices including a global navigation satellite system (GNSS) signal receiver, etc. The configuration of the sensors included in the autonomous operating body 10 can be flexibly changed according to specifications and operations.

[0092] Next, an exemplary configuration of joints of the autonomous operating body 10 according to an embodiment of the present disclosure is described. Figure 2 is an exemplary configuration of the actuator 570 included in the autonomous operating body 10 according to the embodiment of the present disclosure. The autonomous operating body 10 according to the embodiment of the present disclosure has Figure 2 In addition to the respective rotating parts shown, there are a total of 22 rotational degrees of freedom: two each in the ears and tail; and one in the mouth.

[0093] For example, the autonomous operating body 10 has three degrees of freedom in the head, so that the autonomous operating body 10 can nod and tilt its head. The autonomous operating body 10 also reproduces the waist swinging action through the actuator 570 in the waist, so that the autonomous operating body 10 can perform natural and flexible behaviors similar to a real dog.

[0094] Note that the autonomous operating body 10 according to the embodiment of the present disclosure can achieve the above 22 rotational degrees of freedom by, for example, combining a uniaxial actuator and a biaxial actuator. For example, a uniaxial actuator can be used in the elbow and knee of the leg; a biaxial actuator can be used in the shoulder and base of the thigh.

[0095] Figure 3 and Figure 4 1 is a diagram for explaining the operation of the actuator 570 included in the autonomous operating body 10 according to the embodiment of the present disclosure. Figure 3As shown, by rotating the output gear with the motor 575, the actuator 570 can drive the movable arm 590 at a desired rotational position and a desired rotational speed.

[0096] like Figure 4 As shown, the actuator 570 according to an embodiment of the present disclosure includes a rear cover 571 , a gear box cover 572 , a control board 573 , a gear box base 574 , a motor 575 , a first gear 576 , a second gear 577 , an output gear 578 , a detection magnet 579 and two bearings 580 .

[0097] The actuator 570 according to an embodiment of the present disclosure may be, for example, a magnetic spin valve giant magnetoresistance (svGMR). Under the control of the main processor, the control board 573 rotates the motor 575, transmits power to the output gear 578 via the first gear 576 and the second gear 577, and drives the moving arm 590.

[0098] In addition, the position sensor included in the control board 573 detects the rotation angle of the detection magnet 579 that rotates synchronously with the output gear 578, thereby being able to detect the rotation angle of the moving arm 590 as the rotation position with high accuracy.

[0099] Note that the magnetic svGMR is a non-contact type and has excellent durability. In addition, the magnetic svGMR is used in the GMR saturation region to advantageously reduce the influence of signal changes caused by changes in the distance between the detection magnet 579 and the position sensor.

[0100] The above is a description of an exemplary configuration of the actuator 570 included in the autonomous operating body 10 according to an embodiment of the present disclosure. With the above configuration, the bending / extension motion of the joint included in the autonomous operating body 10 can be controlled with high accuracy, and the rotational position of the joint can be correctly detected.

[0101] Next, see Figure 5 The function of the display 510 included in the autonomous operating body 10 according to the embodiment of the present disclosure is described. Figure 5 5 is a diagram for explaining the function of the display 510 included in the autonomous operating object 10 according to the embodiment of the present disclosure.

[0102] (Display 510)

[0103] The display 510 has a function of visually presenting the eye movements and emotions of the autonomous operating body 10. Figure 5 As shown, the display 510 can express the movement of the eyeball, pupil and eyelid according to emotions and actions. The display 510 does not display characters, symbols, images, etc. that are not related to eye movement to produce natural movements similar to those of actual animals such as dogs.

[0104] like Figure 5 As shown, the autonomous operating body 10 also includes two displays 510r and 510l corresponding to the right eye and the left eye, respectively. For example, the displays 510r and 510l are formed using two independent organic light emitting diodes (OLEDs). OLEDs enable the reproduction of the curved surface of the eyeball and achieve a more natural appearance than in the case where a pair of eyeballs are formed by a single flat panel display or in the case where two eyes are formed by two independent flat panel displays.

[0105] As described above, the displays 510r and 510l can accurately and flexibly display the Figure 5 The line of sight and emotion of the autonomous operating object 10 are shown. In addition, the user can intuitively understand the state of the autonomous operating object 10 based on the eye movement displayed on the display 510.

[0106] The above is a description of an exemplary hardware configuration of the autonomous operating body 10 according to an embodiment of the present disclosure. Through the above configuration, the movement of the joints and eyeballs of the autonomous operating body 10 can be controlled with high precision and flexibility, such as Figure 6 As shown, it is possible to obtain movement and emotional expressions similar to those of actual animals. Figure 6 is a diagram illustrating an exemplary operation of the autonomous operating object 10 according to an embodiment of the present disclosure, Figure 6 A simplified view of the external structure of the autonomous operating body 10 is shown to mainly explain the movements of the joints and eyeballs of the autonomous operating body 10. Also, in the following description, the external structure of the autonomous operating body 10 will be simplified in some cases, but the hardware configuration and appearance of the autonomous operating body 10 according to an embodiment of the present disclosure may be appropriately designed without being limited to the examples shown in the drawings.

[0107] <<1.3. Exemplary System Configuration>>

[0108] Next, an exemplary system configuration according to an embodiment of the present disclosure is described. Figure 7 is a diagram showing an exemplary system configuration according to an embodiment of the present disclosure. Figure 7 As shown, the information processing system according to the embodiment of the present disclosure includes a plurality of autonomous operating bodies 10, an information processing server 20, and an information processing terminal 30. Note that the respective components included in the information processing system are connected via a network 40 so that they can communicate with each other.

[0109] (Autonomous operating body 10)

[0110] The autonomous operating body 10 according to an embodiment of the present disclosure is an information processing device that estimates a state based on collected sensor information and autonomously selects and performs various operations according to the state. As described above, the autonomous operating body 10 according to an embodiment of the present disclosure may be, for example, an autonomous mobile robot having the shape of an animal such as a human or a dog and having operational capabilities.

[0111] (Information Processing Server 20)

[0112] The information processing server 20 according to an embodiment of the present disclosure is an information processing device that is connected to a plurality of autonomous operating bodies 10 and has a function of collecting various information from the autonomous operating bodies 10. The information processing server 20 can analyze, for example, the status of the hardware of the autonomous operating body 10, the degree of user enthusiasm for the autonomous operating body 10, etc. based on sensor information collected by the autonomous operating body 10.

[0113] The information processing server 20 also has a function of presenting a recommended action to be performed by the autonomous operating body 10 under the circumstances estimated by the autonomous operating body 10. In this case, the information processing server 20 may transmit to the autonomous operating body 10 control sequence data for causing the autonomous operating body 10 to perform the recommended action.

[0114] The information processing server 20 according to an embodiment of the present disclosure also controls the operation of an application related to the communication between the autonomous operating body 10 and the user. In this application, the information processing server 20 can, for example, dynamically control the output representation of an avatar that reflects the state of the autonomous operating body. The information processing server 20 also has a function of causing the autonomous operating body 10 to reflect the user operations performed on the above-mentioned avatar. Utilizing the above-mentioned function of the information processing server 20, communication between the autonomous operating body 10 and the user can be achieved without relying on the positions of the autonomous operating body 10 and the user.

[0115] (Information processing terminal 30)

[0116] The information processing terminal 30 according to an embodiment of the present disclosure is an information processing device that provides a user with a user interface related to the above-mentioned application. The information processing terminal 30 according to an embodiment of the present disclosure can be, for example, a mobile phone, a smart phone, a tablet, any of various wearable devices, a general-purpose computer, etc. owned by the user.

[0117] (Network 40)

[0118] The network 40 has a function of connecting various components included in the information processing system. The network 40 may include a public line network such as the Internet; a telephone line network; and a satellite communication network; various local area networks (LANs) including Ethernet (registered trademark), wide area networks (WANs), etc. The network 40 may also include a dedicated line network such as an Internet Protocol-Virtual Private Network (IP-VPN). The network 40 may also include a wireless communication network such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0119] The above is a description of an exemplary system configuration according to an embodiment of the present disclosure. Figure 7 The configuration described is merely an example, and the configuration of the information processing system according to the embodiment of the present disclosure is not limited to the example. For example, the autonomous operating body 10 can also communicate information with various external devices other than the information processing server 20. The above-mentioned external devices may include, for example, a server that sends weather, news, and other service information; various information processing terminals owned by users, and home appliances. The system configuration according to the embodiment of the present disclosure can be flexibly modified according to specifications and operations.

[0120] <<1.4. Exemplary Functional Configuration of Autonomous Operating Entity 10>>

[0121] Next, an exemplary functional configuration of the autonomous operating object 10 according to an embodiment of the present disclosure is described. Figure 8 1 is a diagram showing an exemplary functional configuration of the autonomous operating body 10 according to an embodiment of the present disclosure. Figure 8 As shown, the autonomous operating body 10 according to an embodiment of the present disclosure includes an input unit 110 , a recognition unit 120 , a learning unit 130 , an action planning unit 140 , an operation control unit 150 , a driving unit 160 , an output unit 170 and a communication unit 180 .

[0122] (Input unit 110)

[0123] The input unit 110 has a function of collecting various information related to the user and the surrounding environment. The input unit 110 collects, for example, the user's speech and the ambient sound generated around the user, image information related to the user and the surrounding environment, and various sensor information. To this end, the input unit 110 includes Figure 1 Various sensors are shown.

[0124] (Identification Unit 120)

[0125] The recognition unit 120 has a function of performing various recognitions related to the user, the surrounding environment, and the state of the autonomous operating body 10 based on various information collected by the input unit 110. In one example, the recognition unit 120 can perform human recognition, facial expression and line of sight recognition, object recognition, color recognition, shape recognition, mark recognition, obstacle recognition, step recognition, brightness recognition, etc.

[0126] The recognition unit 120 also performs emotion recognition related to the user's voice, word recognition, sound source localization, etc. The recognition unit 120 can also recognize contact by a user or the like, surrounding temperature, the presence of a moving object, a posture of the autonomous operating body 10, and the like.

[0127] The recognition unit 120 also has a function of estimating and understanding the surrounding environment and the state that the autonomous operating body 10 is currently in based on the information recognized as above. In this way, the recognition unit 120 can comprehensively estimate the state using the environmental knowledge pre-stored therein.

[0128] (Study Unit 130)

[0129] The learning unit 130 has a function of learning the environment (state) and the action, and the influence of the action on the environment. The learning unit 130 performs the above learning using, for example, a machine learning algorithm such as deep learning. Note that the learning algorithm adopted by the learning unit 130 is not limited to the above example and can be appropriately designed.

[0130] (Action planning unit 140)

[0131] The action planning unit 140 has a function of planning an action performed by the autonomous operating body 10 based on the state estimated by the recognition unit 120 and the knowledge learned by the learning unit 130. The function of the action planning unit 140 according to an embodiment of the present disclosure will be described in detail later.

[0132] (Operation Control Unit 150)

[0133] The operation control unit 150 has a function of controlling the operation of the driving unit 160 and the output unit 170 based on the motion planned by the motion planning unit 140. The operation control unit 150 performs rotation control on the actuator 570, display control on the display 510, sound output control on the speaker, etc., based on the above-mentioned motion planning, for example. The function of the operation control unit 150 according to an embodiment of the present disclosure will be described in detail later.

[0134] (Drive unit 160)

[0135] The driving unit 160 has a function of bending and extending a plurality of joints of the autonomous operating body 10 under the control of the operation control unit 150. More specifically, the driving unit 160 drives the actuators 570 provided at the respective joints under the control of the operation control unit 150.

[0136] (Output unit 170)

[0137] The output unit 170 has a function of outputting visual information and audio information under the control of the operation control unit 150. Therefore, the output unit 170 includes a display 510 and a speaker.

[0138] (Communication unit 180)

[0139] The communication unit 180 has a function of performing information communication with the information processing server 20, the information processing terminal 30, and another autonomous operating body 10. For example, the communication unit 180 transmits information related to the state recognized by the recognition unit 120, etc. to the information processing server 20. The communication unit 180 also receives, from the information processing server 20, for example, a recommended action, control sequence data related to the recommended action, and data corresponding to a reward described later.

[0140] The above is an exemplary functional configuration of the autonomous operating body 10 according to an embodiment of the present disclosure. Figure 8 The described configuration is merely an example, and the functional configuration of the autonomous operating body 10 according to the embodiment of the present disclosure is not limited to this example. The functional configuration of the autonomous operating body 10 according to the embodiment of the present disclosure can be flexibly modified according to specifications and operations.

[0141] <<1.5. Exemplary Functional Configuration of Information Processing Server 20>>

[0142] Next, an exemplary functional configuration of the information processing server 20 according to the embodiment of the present disclosure is described. Fig. 9 2 is a diagram showing an exemplary functional configuration of the information processing server 20 according to an embodiment of the present disclosure. Fig. 9 As shown, the information processing server 20 according to an embodiment of the present disclosure includes a learning unit 210, an action recommendation unit 220, an analysis unit 230, a storage unit 240, an application control unit 250, a reflection unit 260 and a terminal communication unit 270.

[0143] (Study Unit 210)

[0144] The learning unit 130 has the function of learning the environment (state) and the action, and the influence of the action on the environment. In this case, the learning unit 210 generally performs learning based on the action history collected from multiple autonomous operating bodies 10. In other words, the learning unit 210 can be regarded as a collective intelligence shared by multiple autonomous operating bodies 10.

[0145] (Action Recommendation Unit 220)

[0146] The action recommendation unit 220 has a function of determining an action recommended to the autonomous operating body 10 based on the information related to the estimated state received from the autonomous operating body 10 and the knowledge maintained by the learning unit 210 as collective intelligence. In addition, one of the features of the action recommendation unit 220 is to transmit the recommended action and control sequence data for causing the autonomous operating body 10 to perform the recommended action to the autonomous operating body via the terminal communication unit 270.

[0147] Here, the control sequence data is information including control signals related to temporal changes in rotational positions of joints of autonomous operating body 10, eyeball expressions, and sound outputs. That is, the control sequence data can be regarded as setting data for causing autonomous operating body 10 to perform desired operations (actions).

[0148] By using the above-described functions of the action recommendation unit 220 according to an embodiment of the present disclosure, new actions that can be performed by the autonomous operating body 10 can be added at any time, the user's interest in the autonomous operating body 10 can be continuously maintained, and the like.

[0149] (Analysis Unit 230)

[0150] The analysis unit 230 has a function of performing various analyses based on information received from the autonomous operating body 10. For example, the analysis unit 230 can analyze the state of the actuator 570, etc., based on the action history and operation state received from the autonomous operating body 10. The analysis unit 230 can also analyze the user's interest (enthusiasm) in the autonomous operating body 10, etc., based on information received from the autonomous operating body 10 about the user's contact and reaction, etc.

[0151] (Storage unit 240)

[0152] The storage unit 240 has a function of accumulating information to be used by various components of the information processing server 20. The storage unit 240 stores control sequence data associated with, for example, states and user reactions received from the autonomous operating body 10. The storage unit 240 also stores information to be used by the analysis unit 230 when performing analysis and analysis results. The storage unit 240 also stores various data related to applications and rewards described later.

[0153] (Application Control Unit 250)

[0154] The application control unit 250 controls the operation of the application related to the communication between the autonomous operating body 10 and the user. For example, the application control unit 250 controls the operation and output expression of the avatar imitating the autonomous operating body 10 in the application. In this way, the application control unit 250 can perform output control reflecting the operating conditions and emotions of the autonomous operating body 10. The functions of the application control unit 250 will be described in detail below.

[0155] (Reflection unit 260)

[0156] The reflection unit 260 has a function of causing the autonomous operating body 10 to reflect the user operation in the application. For example, the reflection unit 260 can cause the autonomous operating body 10 to reflect the reward obtained by the avatar based on the operation performed by the user. The function of the reflection unit 260 will be described in detail below.

[0157] (Terminal Communication Unit 270)

[0158] The terminal communication unit 270 has a function of performing information communication with a plurality of autonomous operating bodies 10 via the network 40. The terminal communication unit 270 receives, for example, information related to the estimated state from the autonomous operating body 10. The terminal communication unit 270 also transmits, for example, information related to the recommended action determined by the action recommendation unit 220 and control sequence data to the autonomous operating body 10.

[0159] Under the control of the reflection unit 260 , the terminal communication unit 270 also transmits to the autonomous operating body 10 various control signals for causing the autonomous operating body 10 to reflect the settings made by the user in the application and the rewards acquired by the avatar.

[0160] The above is a description of an exemplary functional configuration of the information processing server 20 according to an embodiment of the present disclosure. Fig. 9 The described configuration is merely an example, and the functional configuration of the information processing server 20 according to the embodiment of the present disclosure is not limited to this example. For example, various functions of the information processing server 20 may be dispersed over a plurality of devices. The functional configuration of the information processing server 20 according to the embodiment of the present disclosure may be flexibly modified according to specifications and operations.

[0161] <<1.6. Exemplary functional configuration of information processing terminal 30>>

[0162] Next, an exemplary functional configuration of the information processing terminal 30 according to the embodiment of the present disclosure is described. Fig.10 is a diagram showing an exemplary functional configuration of the information processing terminal 30 according to an embodiment of the present disclosure. Fig.10As shown, the information processing terminal 30 according to the embodiment of the present disclosure includes an input unit 310 , a display unit 320 , a sound output unit 330 , a control unit 340 , and a communication unit 350 .

[0163] (Input unit 310)

[0164] The input unit 310 has a function of detecting an input operation performed by a user. Therefore, the input unit 310 includes a keyboard, a touch panel, various buttons, etc. The input unit 310 may also include a microphone for detecting a user's voice input, etc. The input unit 310 may also include an imaging device for capturing an image of a user, etc.

[0165] (Display unit 320)

[0166] The display unit 320 has a function of presenting various visual information to the user. For example, the display unit 320 displays a user interface related to the above-mentioned application under the control of the information processing server 20. To this end, the display unit 320 includes various display devices.

[0167] (Sound Output Unit 330)

[0168] The sound output unit 330 has a function of outputting various sounds. For example, the sound output unit 330 outputs various sounds related to the above-mentioned application under the control of the information processing server 20. To this end, the sound output unit 330 includes a speaker, an amplifier, and the like.

[0169] (Control Unit 340)

[0170] The control unit 340 comprehensively controls the various components included in the information processing terminal 30. For example, the control unit 340 can control the activation and suspension of each component. The control unit 340 also has a function of transmitting various control signals generated by the information processing server 20 to the display unit 320 and the sound output unit 330. The control unit 340 may also have functions equivalent to the application control unit 250 and the reflection unit 260 of the information processing server 20.

[0171] (Communication unit 350)

[0172] The communication unit 350 performs information communication with the information processing server 20 and the autonomous operating body 10 via the network 40. For example, the communication unit 350 receives a control signal related to an application and data related to a reward described below from the information processing server 20. For example, the communication unit 350 also transmits information related to a user operation detected by the input unit 310 to the information processing server 20.

[0173] The above is a description of an exemplary functional configuration of the information processing terminal 30 according to an embodiment of the present disclosure. Fig.10 The functional configuration described is merely an example, and the functional configuration of the information processing terminal 30 according to the embodiment of the present disclosure is not limited to the example. For example, as described above, the information processing terminal 30 may have functions equivalent to the application control unit 250 and the reflection unit 260 of the information processing server 20. The functional configuration of the information processing terminal 30 according to the embodiment of the present disclosure may be flexibly modified according to the specifications and operations.

[0174] <2. First Embodiment>

[0175] <<2.1. Overview>>

[0176] Next, the first embodiment of the present disclosure is described. As described above, the autonomous operating body 10 according to the embodiment of the present disclosure includes a display 510 that indicates the movement of multiple joints and eyeballs, so that various operations can be performed. Even when the user is absent, such autonomous operations as described above can be performed.

[0177] However, generally, only when both share the same physical space can the user sense the operation of the autonomous operating body 10 and communicate with the autonomous operating body 10. For this reason, for example, when the user tends to leave home for work or school, the user can neither know the actions of the autonomous operating body 10 for many hours every day nor communicate with the autonomous operating body 10.

[0178] In this case, it is assumed that the user cannot fully enjoy the advantage of having the autonomous operating object 10 , and the user may gradually lose interest in the autonomous operating object 10 as a result of intermittent communication due to the user's absence.

[0179] To overcome the above problems, the user may remotely monitor the state of the autonomous operating body 10 , for example, from a place other than home, etc. However, in this case, the user hardly enjoys interactive communication with the autonomous operating body 10 while being able to know the state of the autonomous operating body 10 .

[0180] Therefore, the information processing server 20 according to the first embodiment of the present disclosure provides the user with an application that enables interactive communication between the autonomous operating body 10 and the user in order to solve the above-mentioned problem.

[0181] For example, the information processing server 20 according to the present embodiment can control the operation of an avatar that imitates the autonomous operating body 10 that the user has in the above-mentioned application. In this way, the information processing server 20 according to the present embodiment can make the output representation of the avatar reflect the current state of the autonomous operating body 10. The above-mentioned state includes, for example, the autonomous operation performed by the autonomous operating body 10, the emotion of the autonomous operating body 10, etc.

[0182] With the above functions of the information processing server 20 according to the present embodiment, the user can know the current state of the autonomous operating body 10 even from outside the home, and can get a feeling of being with the autonomous operating body 10. Therefore, the effect of maintaining the user's interest in the autonomous operating body 10 can be expected.

[0183] Furthermore, one of the features of the information processing server 20 according to the present embodiment is a function of causing the autonomous operating body 10 to reflect the operation performed by the user on the avatar. The information processing server 20 can provide game content for realizing, for example, communication between the avatar and the user. In this way, the information processing server 20 can give a reward based on the operation performed by the user on the avatar in the game content, and cause the autonomous operating body 10 to reflect the reward.

[0184] For example, in the above-mentioned game content, the user can perform various communication-related operations on the avatar, such as touching, clicking, and playing with toys such as balls. In this case, the information processing server 20 according to the present embodiment gives a reward based on the fact that the communication strength has exceeded a predetermined value, and makes the avatar and the autonomous operating body 10 reflect the reward. Here, the above-mentioned reward can be, for example, a change related to the appearance or sound of the autonomous operating body 10, a new operation that the autonomous operating body 10 can perform, etc.

[0185] That is, by deepening the communication with the avatar in the application, the user can enjoy the communication result as the function enhancement of the autonomous operating body 10. With the above-mentioned function of the information processing server 20 according to the present embodiment, the user can continue to perform interactive communication with the autonomous operating body 10 even from outside the home, etc. Therefore, the user's interest in the autonomous operating body 10 can be maintained, and the user's expectation for the autonomous operating body 10 can be always met.

[0186] <<2.2. Application details>>

[0187] In the following description, the functions of the application according to the present embodiment are described in detail with reference to specific examples. As described above, the application according to the present embodiment includes game content that enables communication between an avatar imitating an autonomous operating body 10 and a user.

[0188] On the other hand, the functions of the application according to the present embodiment are not limited to the examples. The application according to the present embodiment widely covers a function for allowing the user to know the state of the autonomous operating body 10 and a function for causing the autonomous operating body 10 to reflect settings made by the user.

[0189] The application according to the present embodiment may also have a function for presenting notifications from the dispenser of the autonomous operating body 10 to the user and a function for activating communication between users. The application according to the present embodiment may be a comprehensive application including various functions for enhancing the convenience of the user who owns the autonomous operating body 10.

[0190] First, a home screen of an application according to the present embodiment is described. Fig.11 and Fig.12 is a diagram showing an example of a home screen of an application according to the present embodiment. Fig.11 and Fig.12 As shown, according to the present embodiment, an avatar AV imitating an autonomous operating body 10 possessed by a user is displayed on the main screen.

[0191] The application control unit 250 can display an avatar AV of a 3D model that imitates the appearance, voice (barking), etc. of the autonomous operating body 10 owned by the user. Here, the appearance includes, for example, the color of the eyeball (pupil, iris, etc.) and the body color of the autonomous operating body 10.

[0192] The application control unit 250 according to the present embodiment may also control the output representation related to the avatar AV based on the state of the autonomous operating body 10. The above-mentioned output representation may include operation display, color representation, audio output, etc. For example, the application control unit 250 may make the operation display of the avatar AV reflect the action performed by the autonomous operating body 10.

[0193] For example, in Fig.11 In the exemplary case shown, the application control unit 250 can cause the screen to display the avatar AV lying down based on the fact that the autonomous operating body 10 is lying down. Fig.12 In the illustrated exemplary case, the application control unit 250 may cause the screen to display the avatar AV being charged on the charging station CS, for example, based on the fact that the autonomous operating body 10 is being charged.

[0194] In addition to the above-described examples, the application control unit 250 may make the avatar AV in the application reflect various operations performed by the autonomous operating body 10 , such as sitting, walking, and lying down.

[0195] The application control unit 250 according to the present embodiment can also make the output representation of the avatar reflect the emotion of the autonomous operating body 10. Fig.11 and Fig.12 In the illustrated exemplary case, the application control unit 250 sets the color of the background BG on the home screen based on the current mood of the autonomous operating object 10 .

[0196] Fig.13 is an example of a color representing the emotion of the autonomous operating object 10 according to the present embodiment. Fig.13As shown, the emotions of the autonomous operating body 10 according to the present embodiment can be represented by various predetermined colors, such as pink for liking and yellow for happiness. In addition, for example, the intensity of each emotion can be represented by the hue of the color. In the case where two or more emotions coexist, the emotions are represented by gradients of two or more corresponding colors, etc.

[0197] As described above, with the application control unit 250 according to the present embodiment, not only the operation of the autonomous operating body 10 but also the output expression of the avatar AV reflecting various states including emotions can be controlled. With the above-mentioned function of the application control unit 250 according to the present embodiment, the user can know the state of the autonomous operating body 10 even from outside the home and can always have the feeling of being with the autonomous operating body 10.

[0198] Return to reference Fig.11 and Fig.12 , continue to describe the screen configuration of the application. Fig.11 and Fig.12 As shown, according to this embodiment, buttons b11 to b13 may be displayed on the main screen of the application.

[0199] For example, button b11 may be a link to an application menu. Fig.14 is an example of an application menu screen according to this embodiment. Fig.14 As shown, the application menu according to the present embodiment includes pet (autonomous operating body 10) registration, tutorial, help guide, support, owner (user) information, store, fan page, logout and application overview information, etc.

[0200] The above-mentioned pet registration can be a function for linking the autonomous operating body 10 owned by the user to the application. By setting the information of the user's autonomous operating body 10 through pet registration, the user can link the application to the autonomous operating body 10. Note that the user can link multiple autonomous operating bodies 10 to the application. In this case, for example, multiple avatar AVs are displayed on the main screen. The user selects the desired avatar AV to use the function linked to the corresponding autonomous operating body 10.

[0201] Meanwhile, the above-mentioned store may include an online store from which options related to functional enhancement of the autonomous operating body 10 can be purchased. The user can enhance the function of the autonomous operating body 10 by, for example, purchasing options such as the appearance, voice (barking), and new operations of the autonomous operating body 10 from the store.

[0202] In addition, the above-mentioned fan page may include a site for mutual communication between users who own the autonomous operating body 10. Through the fan page, users can exchange information related to the autonomous operating body 10, or exchange actions that they have taught the autonomous operating body 10.

[0203] Meanwhile, the button b12 on the home screen may be a link to a notification screen displaying a dispenser notification from the autonomous operating body 10 . Fig.15 is an example of a notification screen according to this embodiment. Fig.15 As shown, the notification screen displays, for example, information on system updates, various activity information, etc. of the autonomous operating body 10. The user can obtain detailed information by clicking on each displayed information or the like.

[0204] Furthermore, the button b13 on the home screen may be a link to a content list screen provided by the application. Fig.16 is an example of a content list screen according to this embodiment. Fig.16 As shown, the top of the content list screen according to the present embodiment displays basic information of the autonomous operating object 10 such as an icon, a name, a gender, a date of birth, a personality, and the like.

[0205] As shown in the figure, icons I1 and I2 indicating the network connection status and the remaining battery level of the autonomous operating body 10 may also be displayed on the content list screen. By visually checking the icons I1 and I2, the user can intuitively know the network connection status and the remaining battery level of the autonomous operating body 10.

[0206] In addition, according to the present embodiment, a plurality of buttons linking to the content provided by the application are displayed on the content list screen. For example, button b21 is a link to the setting content, and button b22 is a link to the appearance change content. In addition, button b23 is a link to the photo content, and button b24 is a link to the body state check content. Meanwhile, according to the present embodiment, button b25 is a link to the game content. In addition, button b26 is a link to the home screen.

[0207] Fig.17 is an example of a setting content screen according to this embodiment. Fig.17 As shown, for example, the volume of the voice (barking) of the autonomous operating body 10 can be set in the setting content according to the present embodiment. The user can change the volume of the voice of the autonomous operating body 10 or mute the autonomous operating body 10 according to preference and situation.

[0208] It is also possible to make network settings for the autonomous operating body 10 in the setting contents according to the present embodiment. On the screen, the user sets information of the Wi-Fi or the like to which the autonomous operating body 10 is to be connected.

[0209] In addition, whether to allow the autonomous operating body 10 to take a photo can be set in the setting content according to the present embodiment. The autonomous operating body 10 according to the present embodiment can automatically take a photo of a person, object, etc. that the autonomous operating body 10 is interested in, or take a photo according to an instruction from the user. On the screen, the user can set whether to allow the autonomous operating body 10 to perform shooting as described above.

[0210] It is also possible to perform settings related to system updates in the settings according to this embodiment. For example, the user can set whether to perform system updates automatically or manually.

[0211] The name and gender of the autonomous operating body 10 can also be set in the setting content according to the present embodiment. The autonomous operating body 10 can sense the user's call to itself by performing voice recognition of the set name. At the same time, gender affects the marking behavior that the autonomous operating body 10 can perform, etc.

[0212] According to the present embodiment, the language used in the voice recognition by the autonomous operating body 10 can also be set in the setting contents, etc. The autonomous operating body 10 can select a dictionary to be used for the voice recognition according to the set language.

[0213] Initialization of the autonomous operating body 10 and the like can also be performed in the setting contents according to the present embodiment.

[0214] Next, the contents of the changes in the appearance according to the present embodiment are described. Fig.18 is an example of a content screen with appearance changes according to this embodiment. Fig.18 As shown, according to the present embodiment, the settings related to the appearance or voice (barking) of the autonomous operating body 10 can be changed in the appearance change content. Here, the above appearance can include color representations related to eyeballs, such as pupils or irises. The user can select the color representation or barking voice of the autonomous operating body 10 from the changes that can be set, and make the autonomous operating body 10 reflect the selected color representation or barking voice.

[0215] The mode of autonomous operation that can be performed by the autonomous operating body 10 can also be newly added to the appearance change content according to the present embodiment, etc. For example, the changes that can be set and the new operations that can be added to the appearance change content increase with purchases in stores, current distribution, reward acquisition described later, etc.

[0216] Fig.19 1 is an example of an appearance change screen for the color representation of the eyeball according to the present embodiment. On the screen, the settings of the color representation of the current eyeball (pupil, iris, etc.) of the autonomous operating body 10 and the changes in the color representation that can be changed can be displayed as a list, as shown in the figure.

[0217] The user can check the displayed change list and select a desired color to change the color representation of the eyeball of the autonomous operating body 10. At this stage, the reflection unit 260 causes the autonomous operating body 10 to reflect the selected color representation based on the user's operation.

[0218] Fig. 20 2 is a diagram for explaining reflection of settings by the reflection unit 260 according to the present embodiment. Fig. 20 The left side of FIG. 1 shows the head of the autonomous operating body 10 before the user performs appearance change settings. Fig. 20 The right side of shows the head of the autonomous operating body 10 after the user performs appearance change settings.

[0219] Here, as can be seen from the comparison between the right image and the left image, the color representation of the eyeballs and the barking voice displayed on the display 510 are changed by the user performing the appearance change setting. After the appearance change setting, new operations of tilting the head and raising one ear are also added. In this way, the reflection unit 260 according to the present embodiment can make the autonomous operating body 10 reflect the corresponding appearance, voice and new operation based on the appearance change setting made by the user. The application control unit 250 can also make the output representation of the avatar AV reflect the appearance change setting as described above.

[0220] In this way, using the application according to the present embodiment, the user can easily change the appearance and voice of the autonomous operating body 10 and add new operations according to preferences, emotions, etc. Therefore, the autonomous operating body 10 that keeps changing is obtained, and the effect of maintaining the user's interest in the autonomous operating body 10 without boring the user can be expected.

[0221] Next, the photograph content according to the present embodiment is described. Fig.21 is an example of a photo content screen according to the present embodiment. As described above, the autonomous operating body 10 according to the present embodiment can take photos autonomously according to its own interests, or take photos according to instructions from the user. In the photo content according to the present embodiment, images captured by the autonomous operating body 10 as described above can be checked.

[0222] like Fig.21 As shown on the left, the photo content screen according to the present embodiment can display a list of images captured by the autonomous operating body 10 in chronological order, etc. In addition, the user can select a desired photo to switch the screen to a photo detail screen, such as Fig.21 as shown on the right.

[0223] In addition to the enlarged image of the photo, the detail screen of the photo also displays basic information such as the shooting date. Note that the image captured autonomously by the autonomous operating body 10 can be associated with the emotion of the autonomous operating body 10 at the time of image capture, etc., and then displayed. The user can also click each button shown on the detail screen, etc., to delete the photo, share the photo in a fan page or other application, download the photo, etc.

[0224] Note that Fig.21 , an image captured by the autonomous operating body 10 is shown, which is the same as an image captured using a lens with a wide viewing angle. However, the camera included in the autonomous operating body 10 according to the present embodiment is a wide-angle camera, and therefore, the image captured by the autonomous operating body 10 is actually an image with a wide viewing angle and distortion.

[0225] The user can also download the image with the distortion as described above without any changes, or can correct the distortion of the image and process the image into a panoramic image (corresponding to Fig.21 removable photos in ).

[0226] As described above, with the application according to the present embodiment, the user can easily refer to the image captured by the autonomous operating body 10, and the user can know the internal information such as what the autonomous operating body 10 is interested in at the time of shooting.

[0227] Next, the contents of the physical condition check according to the present embodiment are described. Fig. 22 is an example of a body state check content screen according to the present embodiment. The body state check content according to the present embodiment is content for checking the operation state of the autonomous operating object 10.

[0228] like Fig. 22 As shown, for example, the battery status, network status, error information, operation time, etc. can be checked in the physical status check content according to the present embodiment. The above-mentioned battery status may include, for example, details of the remaining battery level, time required for full charging, information about battery degradation and details of replacement, etc.

[0229] Meanwhile, the above-mentioned network status may include detailed information and communication history, etc., about the Wi-Fi to which the autonomous operating body 10 is connected.

[0230] In addition, the above-mentioned error information may include various error information and log information related to components included in the autonomous operating body 10 (such as the actuator 570).

[0231] Furthermore, the above-mentioned operation time may include the total operation time of the autonomous operating body 10 and the history of the operation time for each day, and the like.

[0232] Furthermore, although not shown in the drawings, the emotional transition of the autonomous operating body 10, the change of emotion toward the user, and the like may be checked in the body state check content according to the present embodiment.

[0233] As described above, using the body state check content according to the present embodiment, the user can easily check the internal information about the autonomous operating body 10. This function is very effective for a device that does not have any communication means using language, such as the autonomous operating body 10 according to the present embodiment.

[0234] Next, the game content according to the present embodiment is described. As described above, in the game content according to the present embodiment, the user can perform various communications with the avatar AV imitating the autonomous operating body 10 and obtain a reward corresponding to the communication. Fig.23 is an example of a game content screen according to the present embodiment.

[0235] like Fig.23 As shown, in the game content according to the present embodiment, an avatar AV simulating the autonomous operating body 10 is displayed on the main screen. The application control unit 250 causes the screen to display the avatar AV reflecting the appearance, voice, etc. of the autonomous operating body 10, for example, in a virtual living room VH displaying virtual objects VO such as furniture. Note that, without the user performing any operation, the application control unit 250 may control the output representation of the avatar AV linked to the actual movement of the autonomous operating body 10, or may cause the avatar AV to perform an action selected from modes such as standing on four legs, sitting, lying down, and walking.

[0236] Furthermore, a button b31 for returning to the home screen, a button b32 as a link to an explanation screen for explaining how to play the game contents, and the like are displayed on the game contents screen according to the present embodiment. Fig.24 is an example of a game content description screen according to this embodiment. Fig.24 As shown, the user can deepen the communication with the avatar AV by performing operations such as touching, clicking, or throwing a ball or a bone-shaped toy on the displayed avatar AV.

[0237] like Fig.23 As shown, the lower part of the game content screen displays virtual objects corresponding to the ball toy T1 and the bone toy T2, and the upper part of the screen displays meters M corresponding to various communications, such as touch, click, and games using the ball toy T1 or the bone toy T2. The meter M indicates the intensity corresponding to the communication type, and becomes higher as the number of tap (touch) operations performed by the user increases. Here, when the meter M reaches the upper limit value, the application control unit 250 gives a reward corresponding to the communication type, and the reflection unit 260 causes the autonomous operating body 10 to reflect the content of the reward.

[0238] The application control unit 250 according to the present embodiment can also make the avatar AV perform various reactions corresponding to the user operation. For example, in the case of performing a touch (tap) operation, the application control unit 250 can make the avatar AV perform actions such as barking, wagging its tail, curling its back, scratching its head, clapping its hands, or begging.

[0239] Furthermore, in the case where a click (flick) operation is performed, the application control unit 250 can cause the avatar AV to perform actions such as raising the head and shaking the body, kissing, and turning over (exposing the belly).

[0240] The same applies to playing a game using a virtual object such as a ball-shaped toy T1 or a bone-shaped toy T2. Fig.25 is a diagram for explaining interaction between the autonomous operating body 10 and the user via the spherical toy T1 according to the present embodiment.

[0241] like Fig.25 As shown, in the game content according to the present embodiment, the user can throw the spherical toy T1 displayed at the lower part of the screen toward the avatar AV by flicking the spherical toy T1.

[0242] At this stage, the application control unit 250 controls the operation display of the ball toy T1 according to, for example, the laws of physics, and causes the avatar AV to react to the ball toy T1. The application control unit 250 can cause the avatar AV to perform actions such as smelling; pushing the ball toy T1 to the right, left, or forward with the tip of the nose; putting one front foot on the ball toy T1; holding the ball toy T1 with both front feet and biting it; or, for example, kicking the ball toy T1 to the right, left, or forward. The application control unit 250 according to the present embodiment can also cause the screen to display an effect E1 related to an emotion expression as a reaction, such as Fig.25 shown.

[0243] also, Fig.26 1 is a diagram for explaining the interaction between the autonomous operating body 10 and the user via the bone-shaped toy T2 according to the present embodiment. As with the ball-shaped toy T1, the user can throw the bone-shaped toy T2 toward the avatar AV by flicking the bone-shaped toy T2 displayed at the lower part of the screen.

[0244] At this stage, the application control unit 250 controls the operation display of the bone-shaped toy T2 according to, for example, the law of physics, and makes the avatar AV react to the bone-shaped toy T2.

[0245] The application control unit 250 can cause the avatar AV to perform actions such as smelling the bone-shaped toy T2; pushing the bone-shaped toy T2 to the right, left or forward with the tip of the nose; standing up the bone-shaped toy T2; knocking down the bone-shaped toy T2; holding the bone-shaped toy T2 in the mouth, for example, shaking the bone-shaped toy T2 while holding the bone-shaped toy T2 in the mouth; or lifting the bone-shaped toy T2 up and down.

[0246] As described above, in the game content according to the present embodiment, the user can perform various communications with the avatar AV that imitates the autonomous operating body 10. In addition, as described above, in the case where the intensity of one of the various communications (for example, the number of times the user has performed an operation, or the number of times the avatar AV has performed a reaction) has reached a predetermined value, the application control unit 250 can give a reward corresponding to the type of communication.

[0247] Fig. 27 is an example of a reward acquisition pop-up window according to the present embodiment. In the case where the strength of one of the various communications has reached a predetermined value due to an operation performed by the user, that is, in the case where the meter M has reached its maximum value, the application control unit 250 causes the game content to display a pop-up screen therein, such as Fig. 27 As shown. Fig. 27 In the illustrated exemplary case, the application control unit 250 gives a new behavior, which is a new operation that the autonomous operating body 10 can perform, as a reward, and causes the screen to display a notification to the effect that the reward has been obtained.

[0248] As described above, the reward according to the present embodiment may be, for example, a reward related to the enhancement of the function of the autonomous operating body 10. The above-mentioned enhancement of the function may be an increase in the number of functions that the autonomous operating body 10 can perform, such as Fig. 27 The new operation shown may also be to add a change in appearance such as the color of the eyeball, to add a change in the voice (barking) of the autonomous operating body 10, and the like.

[0249] In addition, the application control unit 250 according to the present embodiment can make the screen display as follows Fig. 27 The pop-up screen shown in the figure is displayed, and the output representation of the avatar AV is made to reflect the given reward. The application control unit 250 can make the avatar AV perform a new operation given as a reward, or can make the output representation of the avatar AV reflect a new change in appearance or voice given as a reward. With the above-mentioned function of the application control unit 250 according to the present embodiment, the user can visually recognize each newly acquired reward, and make appearance change settings according to the user's preference.

[0250] At the same time, the reflection unit 260 according to the present embodiment makes the autonomous operating body 10 reflect the newly acquired reward, which is the increase of the new operation or the increase of the appearance or voice change. In the case where the reward is a new operation, the reflection unit 260 according to the present embodiment also sends the control sequence data for making the autonomous operating body 10 perform the new operation to the autonomous operating body 10 via the terminal communication unit 270.

[0251] As described above, with the game content according to the present embodiment, the user can communicate with the avatar AV imitating the autonomous operating body 10 even from outside the home to obtain a reward related to function enhancement and have the autonomous operating body 10 reflect the reward.

[0252] The above is a description of an example of the game content according to the present embodiment. Note that the above description is merely an example, and the game content according to the present embodiment is not limited to this example.

[0253] For example, the user operation is not necessarily an input operation performed by the user on the input unit 310. The user operation according to the present embodiment may be, for example, a voice input. In other words, the user can communicate with the avatar AV through voice.

[0254] Furthermore, in the above description, the application control unit 250 causes the avatar AV to perform various operations corresponding to the operations performed by the user. However, such an operation may be an unacquired operation that the autonomous operating body 10 still cannot perform. In this case, the application control unit 250 according to the present embodiment counts the number of times the avatar AV has performed each acquired operation, etc. In the case where the number exceeds a threshold, the application control unit 250 may give the unacquired operation as a reward.

[0255] In addition, the application control unit 250 according to the present embodiment can select a reward based on feedback from the user. For example, the application control unit 250 can cause the avatar AV to randomly perform a plurality of unobtained operations, and select the unobtained operations to which the user shows a good reaction as a reward. Note that the user's reaction can be analyzed based on, for example, the user's speech detected by the input unit 310, the user's facial image, etc.

[0256] The user can also cause the avatar AV and the autonomous operating body 10 to perform learning related to recognition of a new object via the input unit 310. For example, the user speaks a name corresponding to an object, etc., while showing the object to the avatar AV via the imaging device included in the input unit 310. At this stage, the information processing server 20 can associate the image of the object with the voice recognition result, and additionally register the image in the object recognition dictionary included in the autonomous operating body 10. With the information processing server 20 according to the present embodiment, continuous communication can be performed so that after the user returns home, the autonomous operating body 10 remembers, for example, the content that the user taught the avatar AV from outside the home.

[0257] In addition, in the above exemplary case, the application control unit 250 causes the display device or the like included in the information processing terminal 30 to display the user interface. However, the display control according to the present embodiment is not limited to this example. The application control unit 250 according to the present embodiment can also perform display control on the avatar AV through, for example, augmented reality (AR) technology, virtual reality (VR) technology, mixed reality (MR) technology, etc.

[0258] <<2.3. Control Flow>>

[0259] Next, a control flow to be executed by the information processing server according to the present embodiment will be described in detail. First, an appearance change setting control flow to be executed by the information processing server 20 according to the present embodiment will be described. Fig.28 : is a flowchart showing the flow of appearance change setting control to be executed by information processing server 20 according to the present embodiment.

[0260] like Fig.28 As shown, first, the terminal communication unit 270 acquires information on the state of the autonomous operating object 10 (S1101).

[0261] Then, the application control unit 250 continuously performs operation control of the application including control of the output representation of the avatar according to the state acquired in step S1101 ( S1102 ).

[0262] Here, if the user has made an appearance change setting ( S1103 : Yes), the reflection unit 260 performs a process of causing the autonomous operating body 10 to reflect the selected setting.

[0263] Here, if the appearance change setting is to add behavior, which is a new operation ( S1104 : Yes), the reflection unit 260 transmits control sequence data for causing the autonomous operating body 10 to perform the new operation to the autonomous operating body 10 via the terminal communication unit 270 ( S1105 ).

[0264] On the other hand, in a case where the appearance change setting is a setting related to appearance or voice change (S1104: No), the reflecting unit 260 causes the autonomous operating object 10 to reflect the selected setting (S1106).

[0265] The information processing server 20 repeatedly executes the process starting from step S1101 until the application end operation is performed (S1107: Yes).

[0266] Next, the game content control process according to this embodiment is described in detail. Fig.29 : is a flowchart showing the flow of game content control to be performed by the information processing server 20 according to the present embodiment.

[0267] like Fig. 20 As shown, first, the terminal communication unit 270 acquires information on the state of the autonomous operating object 10 (S1201).

[0268] Then, the application control unit 250 continuously performs operation control on the game content based on the information acquired in step S1201 ( S1202 ).

[0269] Here, in the case where the user performs an operation on the avatar (S1203: Yes), the application control unit 250 adds one point to the communication type corresponding to the operation (S1204).

[0270] Here, if the above point exceeds a predetermined value (S1205: Yes), the application control unit 250 causes the screen to display a notification related to reward acquisition according to the communication type (S1206), and also causes the avatar to reflect the content of the reward (S1207).

[0271] The reflecting unit 260 also causes the autonomous operating body 10 to reflect the function enhancement setting corresponding to the acquired reward ( S1208 ).

[0272] The information processing server 20 repeatedly executes the process starting from step S1201 until the game content ending operation is performed (S1209: Yes).

[0273] <3. Exemplary Hardware Configuration>

[0274] Next, an exemplary hardware configuration common to the information processing server 20 and the information processing terminal 30 according to the embodiment of the present disclosure is described.

[0275] Fig.30 1 is a block diagram showing an example of an exemplary hardware configuration of the information processing server 20 and the information processing terminal 30 according to an embodiment of the present disclosure. Fig.30 As shown, the information processing server 20 and the information processing terminal 30 include a CPU 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a memory 880, a driver 881, a connection port 882, and a communication device 883. For example. Note that the hardware configuration shown here is an example, and some components may be omitted. In addition, components other than those shown herein may also be included.

[0276] (CPU 871)

[0277] The CPU 871 functions as, for example, an arithmetic processing unit or a control device, and controls the overall operation of each component or a portion thereof based on various programs recorded in the ROM 872 , the RAM 873 , the storage 880 , or the removable recording medium 901 .

[0278] (ROM 872 and RAM 873)

[0279] The ROM 872 is a device that stores programs to be read by the CPU 871, data to be used for calculation, etc. The RAM 873 stores, for example, temporarily or permanently programs to be read by the CPU 871, various parameters that are appropriately changed when executing the programs, etc.

[0280] (Host bus 874, bridge 875, external bus 876 and interface 877)

[0281] The CPU 871, ROM 872 and RAM 873 are connected to each other via, for example, a host bus 874 capable of high-speed data transfer. Meanwhile, the host bus 874 is connected to an external bus 876 having a relatively low data transfer rate via, for example, a bridge 875. The external bus 876 is also connected to various components via an interface 877.

[0282] (Input device 878)

[0283] The input device 878 is formed by, for example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, etc. In addition, the input device 878 may be formed with a remote controller capable of transmitting control signals using infrared rays or other radio waves. The input device 878 also includes a sound input device such as a microphone.

[0284] (Output device 879)

[0285] The output device 879 is, for example, a display device such as a cathode ray tube (CRT), LCD or organic EL; an audio output device such as a speaker or earphone; or a device capable of visually or audibly notifying a user of acquired information, such as a printer, mobile phone or fax machine. The output device 879 according to the present disclosure also includes any of various vibration devices capable of outputting tactile stimulation.

[0286] (Memory 880)

[0287] The memory 880 is a device that stores various data. The memory 880 may be, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0288] (Drive 881)

[0289] The drive 881 is a device that reads information recorded on the removable recording medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information on the removable recording medium 901 .

[0290] (Removable Recording Medium 901)

[0291] For example, the removable recording medium 901 is a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, any of various semiconductor storage media, etc. The removable recording medium 901 may of course be, for example, an IC card on which a non-contact IC chip is mounted, an electronic device, or the like.

[0292] (Connection port 882)

[0293] For example, the connection port 882 is a port for connecting the external connection device 902, such as a universal serial bus (USB) port, an IEEE1394 port, a small computer system interface (SCSI), an RS-232C port, or an optical audio terminal.

[0294] (External connection device 902)

[0295] For example, the externally connected device 902 is a printer, a portable music game player, a digital camera, a digital video camera, an IC recorder, or the like.

[0296] (communication device 883)

[0297] For example, the communication device 883 is a communication device for connecting to a network, such as a wired or wireless LAN, Bluetooth (registered trademark), a communication card for wireless USB (WUSB), a router for optical communication, a router for asymmetric digital subscriber line (ADSL), a modem for various communications, and the like.

[0298] <4. Conclusion>

[0299] As described above, the information processing server 20 according to an embodiment of the present disclosure includes: an application control unit 250 that controls the operation of an application related to the communication between the autonomous operating body 10 and the user; and a reflection unit 260 that makes the autonomous operating body reflect the operation performed by the user in the application. In addition, one of the features of the application control unit 250 according to an embodiment of the present disclosure is to control the operation of the avatar that imitates the autonomous operating body 10, and the reflection unit 260 makes the autonomous operating body reflect the reward obtained by the avatar according to the operation of the user. Through such a configuration, the communication between the autonomous operating body and the user can be made more active.

[0300] Although the preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to these examples. Obviously, a person skilled in the art of the present disclosure can make various changes or modifications within the scope of the technical spirit claimed herein, and it should be understood that those changes or modifications are within the technical scope of the present disclosure.

[0301] In addition, the effects disclosed in this specification are merely illustrative or exemplary, and not restrictive. That is, in addition to or instead of the above effects, the technology according to the present disclosure can achieve other effects that are obvious to those skilled in the art from the description of this specification.

[0302] It is also possible to create a program for causing hardware such as CPU, ROM, and RAM in a computer to have functions equivalent to those of the components of the autonomous operating body 10 and the information processing server 20, and provide a computer-readable non-transitory recording medium storing the program.

[0303] In addition, the various steps related to the process to be performed by the autonomous operating body 10 and the information processing server 20 in the present disclosure do not have to be performed in chronological order in the order shown in the flowchart. For example, the various steps related to the process performed by the autonomous operating body 10 and the information processing server 20 may be performed in an order different from the order shown in the flowchart, or may be performed in parallel.

[0304] Note that the configuration described below is also within the technical scope of the present disclosure.

[0305] (1) An information processing device comprising:

[0306] an application control unit that controls the operation of an application related to communication between the autonomous operating body and a user; and

[0307] a reflection unit that causes the autonomous operating body to reflect an operation performed by a user in the application, wherein

[0308] The application control unit controls the operation of the avatar imitating the autonomous operating body, and

[0309] A reflection unit that causes the autonomous operating body to reflect a reward acquired by the avatar based on an operation performed by the user.

[0310] (2) The information processing device according to (1), wherein

[0311] Rewards include enhanced functionality of autonomous operators.

[0312] (3) The information processing device according to (2), wherein

[0313] Functionality enhancements include adding new operations that the autonomous agent can perform.

[0314] (4) The information processing device according to (3), wherein

[0315] The reflection unit causes control sequence data to be transmitted to the autonomous operating entity, and the control sequence data is used to cause the autonomous operating entity to perform a new operation.

[0316] (5) The information processing device according to any one of (2) to (4), wherein

[0317] Functionality enhancements include adding changes related to the appearance of the autonomous operating body.

[0318] (6) The information processing device according to (5), wherein

[0319] Appearance-related changes include at least color representation.

[0320] (7) The information processing device according to any one of (2) to (6), wherein

[0321] Functionality enhancements include adding variations related to the speech of autonomous agents.

[0322] (8) The information processing device according to any one of (1) to (7), wherein

[0323] The application control unit controls an output representation associated with the avatar based on the state of the autonomous operating body.

[0324] (9) The information processing device according to any one of (1) to (8), wherein

[0325] The application control unit causes the avatar's output representation to reflect the operation being performed by the autonomous operating body.

[0326] (10) The information processing device according to any one of (1) to (9), wherein

[0327] The application control unit causes the output representation of the avatar to reflect the emotion of the autonomous operating body.

[0328] (11) The information processing device according to (9) or (10), wherein

[0329] The output representation includes at least one of an operation display, a color representation, or a voice output.

[0330] (12) The information processing device according to any one of (1) to (11), wherein

[0331] The application control unit causes the avatar's output representation to reflect the reward earned by the avatar.

[0332] (13) The information processing device according to any one of (1) to (12), wherein

[0333] The application control unit controls the operation of the game content related to the communication between the avatar and the user, and

[0334] The application control unit awards a reward based on the intensity of communication with the user in the game.

[0335] (14) The information processing device according to (13), wherein

[0336] In-game communication includes contact with the avatar, or interaction with the avatar via virtual objects.

[0337] (15) An information processing method implemented by a processor,

[0338] The information processing method includes:

[0339] controlling the operation of applications related to communications between autonomous agents and users; and

[0340] The autonomous operating body reflects the operation performed by the user in the application, wherein

[0341] The control also includes controlling the operation of an avatar that mimics the autonomous operator, and

[0342] The causing also includes causing the autonomous operator to reflect a reward obtained by the avatar based on an operation performed by the user.

[0343] (16) A program for causing a computer to function as an information processing device, comprising:

[0344] an application control unit that controls the operation of an application related to communication between the autonomous operating body and a user; and

[0345] a reflection unit that causes the autonomous operating body to reflect an operation performed by a user in the application, wherein

[0346] The application control unit controls the operation of the avatar imitating the autonomous operating body, and

[0347] A reflection unit that causes the autonomous operating body to reflect a reward acquired by the avatar based on an operation performed by the user.

[0348] Reference Symbols List

[0349] 10 Autonomous Operation

[0350] 110 Input Units

[0351] 120 recognition units

[0352] 130 study units

[0353] 140 Action Planning Unit

[0354] 150 Operation control unit

[0355] 160 driver unit

[0356] 170 output units

[0357] 510 Display

[0358] 570 Actuator

[0359] 20Information processing server

[0360] 210 study units

[0361] 220 Action Recommendation Unit

[0362] 230 Analysis Units

[0363] 240 storage units

[0364] 250 Application Control Unit

[0365] 260 reflection unit

[0366] 270 Terminal Communication Unit

[0367] 30 Information processing terminal.

Claims

1. An information processing device, comprising: an application control unit that controls the operation of an application related to communication between the autonomous operating body and a user; as well as a reflection unit that causes the autonomous operating body to reflect an operation performed by the user in the application, wherein The application control unit controls the operation of the avatar that imitates the autonomous operating body, and the reflecting unit causes the autonomous operating body to reflect a reward acquired by the avatar based on the operation performed by the user; wherein, based on the coexistence of two or more emotions of the autonomous operating body, the application control unit sets the color of the background of the screen of the application to a gradient of two or more colors among a plurality of colors; wherein each color of the plurality of colors is associated with a different emotion, and The two or more colors correspond to the two or more emotions of the autonomous operating body.

2. The information processing device according to claim 1, wherein The reward includes an enhancement of the functionality of the autonomous operating body.

3. The information processing device according to claim 2, wherein The functional enhancement includes adding new operations that the autonomous operating body can perform.

4. The information processing device according to claim 3, wherein The reflection unit causes control sequence data to be transmitted to the autonomous operating body, the control sequence data being used to cause the autonomous operating body to execute the new operation.

5. The information processing device according to claim 2, wherein The functional enhancement includes an increase in changes related to the appearance of the autonomous operating body.

6. The information processing device according to claim 5, wherein The changes associated with the appearance include at least a color representation.

7. The information processing device according to claim 2, wherein The functional enhancements include an increase in variations associated with the voice of the autonomous operating body.

8. The information processing device according to claim 1, wherein The application control unit controls an output expression associated with the avatar based on a state of the autonomous operating body.

9. The information processing device according to claim 1, wherein The application control unit causes an output representation of the avatar to reflect the operation being performed by the autonomous operating body.

10. The information processing device according to claim 1, wherein The application control unit causes the output expression of the avatar to reflect the emotion of the autonomous operating body.

11. The information processing device according to claim 9, wherein The output representation includes at least one of the following: operation display, color representation and voice output.

12. The information processing device according to claim 1, wherein The application control unit causes an output representation of the avatar to reflect the reward earned by the avatar.

13. The information processing device according to claim 1, wherein The application control unit controls the operation of game content related to communication between the avatar and the user, and The application control unit gives the reward based on the intensity of communication with the user in the game.

14. The information processing device according to claim 13, wherein The communication in the game includes contact with or interaction with the avatar via a virtual object.

15. An information processing method implemented by a processor, The information processing method comprises: controlling the operation of applications related to communications between autonomous agents and users; and causing the autonomous operating body to reflect an operation performed by the user in the application, wherein The controlling also includes controlling the operation of an avatar that mimics the autonomous operating body, and The controlling further includes causing the autonomous operating body to reflect a reward acquired by the avatar based on the operation performed by the user; wherein, based on the coexistence of two or more emotions of the autonomous operating body, the color of the background of the screen of the application is set to a gradient of two or more colors among a plurality of colors; wherein each color of the plurality of colors is associated with a different emotion, and The two or more colors correspond to the two or more emotions of the autonomous operating body.

16. A computer-readable storage medium having stored thereon instructions, which, when executed by a computer, cause the computer to perform the information processing method according to claim 15.

Images