Information processing device, information processing method, and program

Abstract

This technology relates to an information processing device, an information processing method, and a program that make it possible to enhance the entertainment of an autonomous moving object. An information processing device according to the present invention is provided with: an image processing unit that generates a first video in which a virtual second autonomous moving object virtually interacting with a first autonomous moving object is superimposed in a captured video capturing the first autonomous moving object; and a display control unit that controls the display of the first video. This technology is applicable to, for example, an information processing terminal that performs an operation of an autonomous moving object, etc.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that improve the entertainment value of an autonomous moving body. [Background technology]

[0002] In recent years, with the spread of SNS (Social Networking Service), each user can easily increase the number of friends by becoming friends with other users on the SNS (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-3635 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a demand for improving the entertainment value of pet robots, which have become increasingly popular in recent years. For example, it is expected that the entertainment value of pet robots will be improved if they can make friends with other pet robots and increase their number of companions, just like users on social networking sites.

[0005] The present technology has been made in view of such circumstances, and aims to improve the entertainment value of autonomous moving bodies such as pet-type robots. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present technology includes an image processing unit that generates a first video in which a virtual second autonomous moving body that virtually interacts with a first autonomous moving body is superimposed on a captured video of the first autonomous moving body, and a display control unit that controls the display of the first video.

[0007] In the information processing method of one aspect of the present technology, an information processing apparatus generates a video in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured video of the first autonomous mobile body, and controls the display of the video.

[0008] A program of one aspect of the present technology causes a computer to execute a process of generating a video in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured video of the first autonomous mobile body, and controlling the display of the video.

[0009] In one aspect of the present technology, a video in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured video of the first autonomous mobile body is generated, and the display of the video is controlled.

Brief Description of Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 2a

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Figure 56

Figure 57

Figure 58

Figure 59

Figure 60

[0011] Hereinafter, embodiments of the present technology will be described in the following order. 1. Embodiment 2. Variations 3.Other

[0012] <<1. Embodiment>> An embodiment of the present technology will be described with reference to FIGS.

[0013] <Configuration example of information processing system 1> FIG. 1 is a block diagram showing an embodiment of an information processing system 1 to which the present technology is applied.

[0014] The information processing system 1 includes autonomous moving bodies 11-1 to 11-n, information processing terminals 12-1 to 12-n, and an information processing server 13.

[0015] Hereinafter, when there is no need to distinguish between the autonomous mobile bodies 11-1 to 11-n, they will simply be referred to as the autonomous mobile body 11. Hereinafter, when there is no need to distinguish between the information processing terminals 12-1 to 12-n, they will simply be referred to as the information processing terminals 12.

[0016] Communication via the network 21 is possible between each autonomous mobile body 11 and the information processing server 13, between each information processing terminal 12 and the information processing server 13, between each autonomous mobile body 11 and each information processing terminal 12, between autonomous mobile bodies 11, and between information processing terminals 12. Also, direct communication without using the network 21 is possible between each autonomous mobile body 11 and each information processing terminal 12, between autonomous mobile bodies 11, and between information processing terminals 12.

[0017] The autonomous mobile body 11 is an information processing device that recognizes its own and surrounding situations based on collected sensor data and the like, and autonomously selects and executes various actions according to the situation. The autonomous mobile body 11 is characterized in that, unlike a robot that simply performs actions according to a user's instructions, it autonomously executes appropriate actions according to the situation.

[0018] The autonomous mobile body 11 can execute, for example, user recognition based on a captured image, object recognition, etc., and perform various autonomous actions according to the recognized user, object, etc. Also, the autonomous mobile body 11 can execute, for example, speech recognition based on a user's utterance and perform actions based on a user's instructions and the like.

[0019] Furthermore, the autonomous mobile body 11 performs pattern recognition learning in order to acquire the ability of user recognition and object recognition. At this time, the autonomous mobile body 11 can not only perform supervised learning based on given learning data, but also dynamically collect learning data based on teachings by a user or the like and perform pattern recognition learning related to objects and the like.

[0020] Also, the autonomous mobile body 11 can be disciplined by a user. Here, the discipline of the autonomous mobile body 11 means, for example, something broader than general discipline that teaches and makes it remember rules and prohibitions, and refers to the appearance of changes that a user feels in the autonomous mobile body 11 when the user interacts with the autonomous mobile body 11.

[0021] The levels of the shape, capabilities, desires, etc. of the autonomous mobile body 11 can be appropriately designed according to the purpose and role. For example, the autonomous mobile body 11 is composed of an autonomous mobile robot that autonomously moves within a space and executes various operations. Specifically, for example, the autonomous mobile body 11 is composed of an autonomous mobile robot having a shape and motion capabilities imitating animals such as humans and dogs. Also, for example, the autonomous mobile body 11 is composed of a vehicle or other device having communication capabilities with the user.

[0022] The information processing terminal 12 consists of, for example, a smartphone, a tablet terminal, a PC (personal computer), etc., and is used by the user of the autonomous mobile body 11. The information processing terminal 12 realizes various functions by executing a predetermined application program (hereinafter simply referred to as an application). For example, the information processing terminal 12 communicates with the information processing server 13 via the network 21 or directly communicates with the autonomous mobile body 11 to collect various data related to the autonomous mobile body 11, present it to the user, or give instructions to the autonomous mobile body 11.

[0023] The information processing server 13 collects various data from each autonomous mobile body 11 and each information processing terminal 12, provides various data to each autonomous mobile body 11 and each information processing terminal 12, and controls the operations of each autonomous mobile body 11. Also, for example, the information processing server 13 can perform pattern recognition learning and processing corresponding to the training of the user, similar to the autonomous mobile body 11, based on the data collected from each autonomous mobile body 11 and each information processing terminal 12. Further, for example, the information processing server 13 supplies the above-described applications and various data related to each autonomous mobile body 11 to each information processing terminal 12.

[0024] The network 21 is composed of, for example, some of public line networks such as the Internet, telephone line networks, satellite communication networks, and various LANs (Local Area Networks) including Ethernet (registered trademark), WANs (Wide Area Networks), etc. Further, the network 21 may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network). Also, the network 21 may include a wireless communication network such as Wi-Fi (registered trademark), Bluetooth (registered trademark).

[0025] Note that the configuration of the information processing system 1 can be flexibly changed according to specifications, operations, etc. For example, in addition to the autonomous mobile body 11, the information processing terminal 12, and the information processing server 13, the autonomous mobile body 11 may further perform information communication with various external devices. The above external devices may include, for example, servers that transmit weather, news, and other service information, and various household electrical appliances owned by users.

[0026] Also, for example, the relationship between the autonomous mobile body 11 and the information processing terminal 12 does not necessarily have to be a one-to-one relationship, and may be, for example, a many-to-many, many-to-one, or one-to-many relationship. For example, it is possible for one user to check data regarding a plurality of autonomous mobile bodies 11 using one information processing terminal 12, or to check data regarding one autonomous mobile body 11 using a plurality of information processing terminals.

[0027] <Example of the hardware configuration of the autonomous mobile body 11> Next, an example of the hardware configuration of the autonomous mobile body 11 will be described. Hereinafter, the case where the autonomous mobile body 11 is an inu-type quadruped walking robot will be described as an example.

[0028] FIG. 2 is a diagram showing an example of the hardware configuration of the autonomous mobile body 11. The autonomous mobile body 11 is an inu-type quadruped walking robot including a head, a body, four legs, and a tail.

[0029] The autonomous mobile body 11 is equipped with two displays 51L and 51R on its head. Hereinafter, when there is no need to distinguish between the display 51L and the display 51R individually, they are simply referred to as the display 51.

[0030] In addition, the autonomous mobile body 11 is equipped with various sensors. The autonomous mobile body 11 includes, for example, a microphone 52, a camera 53, a ToF (Time Of Flight) sensor 525, a human presence sensor 55, a distance measuring sensor 56, a touch sensor 57, an illuminance sensor 58, a sole button 59, and an inertial sensor 60.

[0031] The autonomous mobile body 11 is equipped with, for example, four microphones 52 on its head. Each microphone 52 collects ambient sounds including, for example, the user's speech and surrounding environmental sounds. Also, by having a plurality of microphones 52, sounds generated in the surroundings can be collected with high sensitivity, and sound source localization becomes possible.

[0032] The autonomous mobile body 11 is equipped with, for example, two wide-angle cameras 53 at the tip of its nose and the waist, and photographs the surroundings of the autonomous mobile body 11. For example, the camera 53 placed at the tip of the nose photographs within the forward field of view of the autonomous mobile body 11 (i.e., the field of view of a dog). The camera 53 placed at the waist photographs the surroundings centered on the upper part of the autonomous mobile body 11. The autonomous mobile body 11 can extract, for example, characteristic points on the ceiling based on the image photographed by the camera 53 placed at the waist, and realize SLAM (Simultaneous Localization and Mapping).

[0033] The ToF sensor 54 is provided, for example, at the tip of the nose and detects the distance to an object existing in front of the head. The autonomous mobile body 11 can accurately detect the distance to various objects with the ToF sensor 54, and realize operations according to the relative positions of objects including the user and obstacles.

[0034] The human presence sensor 55 is disposed, for example, on the chest and detects the presence of a user or a pet raised by the user. By detecting a moving object existing in front with the human presence sensor 55, the autonomous mobile body 11 can realize various actions with respect to the moving object, for example, actions corresponding to emotions such as interest, fear, and surprise.

[0035] The distance measuring sensor 56 is disposed, for example, on the chest and detects the situation of the floor surface in front of the autonomous mobile body 11. The autonomous mobile body 11 can accurately detect the distance to an object existing on the floor surface in front with the distance measuring sensor 56 and can realize an action according to the relative position with the object.

[0036] The touch sensor 57 is disposed, for example, at a site where there is a high possibility that the user touches the autonomous mobile body 11, such as the top of the head, under the chin, and the back, and detects contact by the user. The touch sensor 57 is constituted by, for example, a capacitance type or pressure sensitive type touch sensor. The autonomous mobile body 11 can detect contact actions such as touching, stroking, knocking, and pressing by the user with the touch sensor 57 and can perform an action according to the contact action.

[0037] The illuminance sensor 58 is disposed, for example, at the base of the tail on the back of the head and detects the illuminance of the space where the autonomous mobile body 11 is located. The autonomous mobile body 11 can detect the ambient brightness with the illuminance sensor 58 and can execute an action according to the brightness.

[0038] The sole button 59 is disposed, for example, at a site corresponding to the pads of the four legs respectively and detects whether or not the bottom surface of the leg of the autonomous mobile body 11 is in contact with the floor. The autonomous mobile body 11 can detect contact or non-contact with the floor surface with the sole button 59 and can grasp, for example, that it has been lifted up by the user.

[0039] The inertial sensors 60 are arranged, for example, on the head and torso, respectively, and detect physical quantities such as the speed, acceleration, and rotation of the head and torso. For example, the inertial sensor 60 is configured with a six-axis sensor that detects acceleration and angular velocity along the X-axis, Y-axis, and Z-axis. The autonomous mobile body 11 can accurately detect the movement of the head and torso using the inertial sensor 60, and realize operation control according to the situation.

[0040] The configuration of the sensors included in the autonomous moving body 11 can be flexibly changed depending on the specifications, operation, etc. For example, in addition to the above configuration, the autonomous moving body 11 may further include various communication devices including a temperature sensor, a geomagnetic sensor, and a GNSS (Global Navigation Satellite System) signal receiver.

[0041] Next, a configuration example of the joints of the autonomous mobile body 11 will be described with reference to Fig. 3. Fig. 3 shows a configuration example of the actuator 71 provided in the autonomous mobile body 11. In addition to the rotation points shown in Fig. 3, the autonomous mobile body 11 has two degrees of freedom of rotation each in the ears and tail, and one in the mouth, for a total of 22 degrees of freedom of rotation.

[0042] For example, the autonomous mobile body 11 has three degrees of freedom in its head, allowing it to nod and tilt its head at the same time. In addition, the autonomous mobile body 11 can reproduce the swinging motion of its waist with the actuator 71 provided in its waist, allowing it to achieve natural and flexible movements that are closer to those of a real dog.

[0043] The autonomous moving body 11 may achieve the above 22 degrees of rotational freedom by combining, for example, a single-axis actuator and a two-axis actuator. For example, single-axis actuators may be used in the elbows and knees of the legs, and two-axis actuators may be used in the shoulders and thighs.

[0044] Next, the function of the display 51 provided in the autonomous moving body 11 will be described with reference to FIG.

[0045] The autonomous mobile body 11 is provided with two displays 51R and 51L corresponding to the right eye and the left eye respectively. Each display 51 has a function of visually expressing the eye movements and emotions of the autonomous mobile body 11. For example, each display 51 can produce natural movements similar to those of real animals such as dogs by expressing the movements of the eyeballs, pupils, and eyelids according to emotions and actions, and can express the line of sight and emotions of the autonomous mobile body 11 with high precision and flexibility. Also, the user can intuitively grasp the state of the autonomous mobile body 11 from the movements of the eyeballs displayed on the display 51.

[0046] Also, each display 51 is realized by, for example, two independent OLEDs (Organic Light Emitting Diodes). By using OLEDs, it becomes possible to reproduce the curved surface of the eyeball. As a result, a more natural exterior can be realized compared to the case where a pair of eyeballs are expressed by a single flat display or the case where two eyeballs are expressed by two independent flat displays respectively.

[0047] With the above configuration, as shown in FIG. 5, the autonomous mobile body 11 can reproduce movements and emotional expressions closer to those of real living organisms by accurately and flexibly controlling the movements of the joints and eyeballs.

[0048] Note that FIG. 5 is a diagram showing an operation example of the autonomous mobile body 11. In FIG. 5, for the purpose of focusing on and explaining the movements of the joints and eyeballs of the autonomous mobile body 11, the external structure of the autonomous mobile body 11 is shown in a simplified manner.

[0049] <Functional configuration example of the autonomous mobile body 11> Next, with reference to FIG. 6, a functional configuration example of the autonomous mobile body 11 will be described. The autonomous mobile body 11 includes an input unit 101, a communication unit 102, an information processing unit 103, a drive unit 104, an output unit 105, and a storage unit 106.

[0050] The input unit 101 includes various sensors shown in FIG. 2 and has a function of collecting various sensor data related to the user and the surrounding situation. Further, the input unit 101 includes input devices such as switches and buttons, for example. The input unit 101 supplies the collected sensor data and the input data input via the input device to the information processing unit 103.

[0051] The communication unit 102 communicates with other autonomous mobile bodies 11, information processing terminals 12, and information processing servers 13 via the network 21 or without using the network 21, and performs transmission and reception of various data. The communication unit 102 supplies the received data to the information processing unit 103 and acquires the data to be transmitted from the information processing unit 103.

[0052] Note that the communication method of the communication unit 102 is not particularly limited and can be flexibly changed according to the specifications and operations.

[0053] The information processing unit 103 includes a processor such as a CPU (Central Processing Unit), for example, and performs various information processes and controls each part of the autonomous mobile body 11. The information processing unit 103 includes a recognition unit 121, a learning unit 122, an action planning unit 123, and an operation control unit 124.

[0054] The recognition unit 121 recognizes the situation in which the autonomous mobile body 11 is located, based on the sensor data and input data supplied from the input unit 101 and the received data supplied from the communication unit 102. The situation in which the autonomous mobile body 11 is located includes, for example, its own situation and its surroundings. Its own situation includes, for example, the state and movement of the autonomous mobile body 11. The surrounding situation includes, for example, the state, movement, and instructions of surrounding people such as a user, the state and movement of surrounding living things such as pets, the state and movement of surrounding objects, time, place, and the surrounding environment. The surrounding objects include, for example, other autonomous mobile bodies. In addition, in order to recognize the situation, the recognition unit 121 performs, for example, person identification, facial expression and gaze recognition, emotion recognition, object recognition, action recognition, spatial region recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, temperature recognition, voice recognition, word understanding, position estimation, posture estimation, etc.

[0055] The recognition unit 121 also has a function of estimating and understanding a situation based on various recognized information. In this case, the recognition unit 121 may comprehensively estimate the situation using knowledge stored in advance.

[0056] The recognition unit 121 supplies data indicating the recognition result or estimation result of the situation (hereinafter referred to as situation data) to the learning unit 122 and the behavior planning unit 123. In addition, the recognition unit 121 registers the data indicating the recognition result or estimation result of the situation in the behavior history data stored in the storage unit 106.

[0057] The behavior history data is data that indicates the history of behavior of the autonomous moving body 11. The behavior history data includes, for example, items such as the date and time when the behavior started, the date and time when the behavior ended, the trigger for performing the behavior, the location where the behavior was instructed (if a location was instructed), the situation when the behavior was performed, and whether the behavior was completed (whether the behavior was performed to the end).

[0058] For example, if an action is triggered by a user instruction, the content of the instruction is registered as the trigger for the execution of the action. Also, for example, if an action is triggered by a specific situation, the content of the situation is registered. Furthermore, for example, if an action is triggered by an object pointed to or recognized by the user, the type of the object is registered.

[0059] The learning unit 122 learns the situation, the behavior, and the effect of the behavior on the environment based on the sensor data and input data supplied from the input unit 101, the received data supplied from the communication unit 102, the situation data supplied from the recognition unit 121, the data related to the behavior of the autonomous mobile body 11 supplied from the behavior planning unit 123, and the behavior history data stored in the memory unit 106. For example, the learning unit 122 performs the above-mentioned pattern recognition learning and learns behavior patterns corresponding to the user's discipline.

[0060] For example, the learning unit 122 realizes the above learning by using a machine learning algorithm such as deep learning. Note that the learning algorithm employed by the learning unit 122 is not limited to the above example and can be designed as appropriate.

[0061] The learning unit 122 supplies data indicating the learning results (hereinafter referred to as learning result data) to the action planning unit 123 and stores the data in the storage unit .

[0062] The behavior planning unit 123 plans an action to be taken by the autonomous mobile body 11 based on the recognized or estimated situation and the learning result data. The behavior planning unit 123 supplies data indicating the planned action (hereinafter referred to as behavior plan data) to the operation control unit 124. In addition, the behavior planning unit 123 supplies data regarding the action of the autonomous mobile body 11 to the learning unit 122, and registers the data in the action history data stored in the memory unit 106.

[0063] The motion control unit 124 controls the operation of the autonomous mobile body 11 to execute the planned action by controlling the drive unit 104 and the output unit 105 based on the action plan data. For example, the motion control unit 124 performs rotation control of the actuator 71, display control of the display 51, voice output control by the speaker, etc. based on the action plan.

[0064] Based on the control by the motion control unit 124, the drive unit 104 flexes and extends a plurality of joint parts of the autonomous mobile body 11. More specifically, based on the control by the motion control unit 124, the drive unit 104 drives the actuator 71 provided in each joint part.

[0065] The output unit 105 includes, for example, a display 51, a speaker, a haptics device, etc., and outputs visual information, auditory information, tactile information, etc. based on the control by the motion control unit 124.

[0066] The storage unit 106 includes, for example, a non-volatile memory and a volatile memory, and stores various programs and data.

[0067] Hereinafter, when each part of the autonomous mobile body 11 communicates with the information processing server 13 etc. via the communication unit 102 and the network 21, the description of "via the communication unit 102 and the network 21" will be appropriately omitted. For example, when the recognition unit 121 communicates with the information processing server 13 via the communication unit 102 and the network 21, it is simply described that the recognition unit 121 communicates with the information processing server 13.

[0068] <Functional configuration example of the information processing terminal 12> Next, with reference to FIG. 7, a functional configuration example of the information processing terminal 12 will be described. The information processing terminal 12 includes an input unit 201, a communication unit 202, an information processing unit 203, an output unit 204, and a storage unit 205.

[0069] The input unit 201 includes various sensors such as a camera 211, a microphone (not shown), and an inertial sensor (not shown). The input unit 201 also includes input devices such as a switch (not shown) and a button (not shown). The input unit 201 supplies input data input via the input devices and sensor data output from the various sensors to the information processing unit 203.

[0070] The communication unit 202 communicates with the autonomous mobile body 11, other information processing terminals 12, and the information processing server 13 via the network 21 or without the network 21, and transmits and receives various types of data. The communication unit 202 supplies received data to the information processing unit 203 and obtains data to be transmitted from the information processing unit 203.

[0071] The communication method of the communication unit 202 is not particularly limited, and can be flexibly changed according to the specifications and operation.

[0072] The information processing unit 203 includes, for example, a processor such as a CPU, and performs various types of information processing and controls each unit of the information processing terminal 12. The information processing unit 203 includes a recognition unit 221, an image processing unit 222, a UI (user interface) control unit 223, and a friend setting unit 224.

[0073] The recognition unit 221 performs object recognition within the video captured by the camera 211.

[0074] The image processing unit 222 generates moving images, still images, operation screens, and the like to be displayed on the display unit 241 included in the output unit 204 .

[0075] The UI control unit 223 controls the output of visual information, auditory information, tactile information, etc. by the output unit 204, thereby controlling the user interface of the information processing terminal 12. For example, the display control unit 231 included in the UI control unit 223 controls the output of visual information by the display unit 241 included in the output unit 204. In other words, the display control unit 231 controls the display of moving images, still images, operation screens, etc. by the display unit 241.

[0076] As will be described later, the friend setting unit 224 sets friends between an autonomous moving body 11 (hereinafter referred to as the user's own device) owned by a user using the information processing terminal 12 and an autonomous moving body 11 (hereinafter referred to as the other device) owned by another user.

[0077] The output unit 204 includes, for example, a display unit 241 consisting of a display, a speaker (not shown), a haptic device (not shown), etc., and outputs visual information, auditory information, tactile information, etc. based on the control of the UI control unit 223.

[0078] The storage unit 205 includes, for example, a non-volatile memory and a volatile memory, and stores various programs and data.

[0079] The functional configuration of the information processing terminal 12 can be flexibly changed according to the specifications and operation.

[0080] Furthermore, hereinafter, the expression "via the communication unit 202 and the network 21" will be omitted as appropriate when each unit of the information processing terminal 12 communicates with the information processing server 13, etc. via the communication unit 202 and the network 21. For example, when the friend setting unit 224 communicates with the information processing server 13 via the communication unit 202 and the network 21, it will simply be described that the friend setting unit 224 communicates with the information processing server 13.

[0081] Furthermore, in the following description, when the image processing unit 222 generates an image such as a setting screen and the display control unit 231 causes the generated image to be displayed on the display unit 241, the description of the processing of the image processing unit 222 and the display control unit 231 will be omitted as appropriate. For example, it will be simply stated that "the display unit 241 displays the setting screen" or "the setting screen is displayed on the display unit 241."

[0082] <Example of functional configuration of information processing server 13> Next, an example of the functional configuration of the information processing server 13 will be described with reference to Fig. 8. The information processing server 13 includes a communication unit 301, an information processing unit 302, and a storage unit 303.

[0083] The communication unit 301 communicates with each autonomous mobile body 11 and each information processing terminal 12 via the network 21 and transmits and receives various data. The communication unit 301 supplies the received data to the information processing unit 302 and obtains the data to be transmitted from the information processing unit 302.

[0084] Note that the communication method of the communication unit 301 is not particularly limited and can be flexibly changed according to the specifications and operations.

[0085] The information processing unit 302 includes, for example, a processor such as a CPU and performs various information processing and controls each part of the information processing terminal 12. The information processing unit 302 includes an autonomous movement control unit 321 and an application control unit 322.

[0086] The autonomous movement control unit 321 has the same configuration as the information processing unit 103 of the autonomous mobile body 11. Specifically, the information processing unit 103 includes a recognition unit 331, a learning unit 332, an action planning unit 333, and an operation control unit 334.

[0087] And the autonomous movement control unit 321 has the same functions as the information processing unit 103 of the autonomous mobile body 11. For example, the autonomous movement control unit 321 receives sensor data, input data, etc., action history data, etc. from the autonomous mobile body 11 and recognizes the situation of the autonomous mobile body 11 and its surroundings. For example, the autonomous movement control unit 321 generates control data for controlling the operation of the autonomous mobile body 11 based on the situation of the autonomous mobile body 11 and its surroundings and transmits it to the autonomous mobile body 11 to control the operation of the autonomous mobile body 11. For example, like the autonomous mobile body 11, the autonomous movement control unit 321 performs pattern recognition learning and learning of action patterns corresponding to the training of the user.

[0088] Note that the learning unit 332 of the autonomous movement control unit 321 can also perform learning of collective knowledge common to a plurality of autonomous mobile bodies 11 by performing pattern recognition learning and learning of action patterns corresponding to the training of the user based on data collected from a plurality of autonomous mobile bodies 11.

[0089] The application control unit 322 communicates with the autonomous moving body 11 and the information processing terminal 12 via the communication unit 301 and controls the application executed by the information processing terminal 12 .

[0090] For example, the application control unit 322 collects various types of data related to the autonomous mobile body 11 from the autonomous mobile body 11 via the communication unit 301. Then, the application control unit 322 transmits the collected data to the information processing terminal 12 via the communication unit 301, thereby causing the data related to the autonomous mobile body 11 to be displayed in an application executed by the information processing terminal 12.

[0091] For example, the application control unit 322 receives data indicating instructions to the autonomous mobile body 11 input via an application from the information processing terminal 12 via the communication unit 301. Then, the application control unit 322 transmits the received data to the autonomous mobile body 11 via the communication unit 301, thereby giving instructions from the user to the autonomous mobile body 11.

[0092] The storage unit 303 includes, for example, a non-volatile memory and a volatile memory, and stores various programs and data. For example, the storage unit 303 stores data related to each autonomous mobile body 11. The data related to each autonomous mobile body 11 includes, for example, a thumbnail image representing the face or appearance, a profile, specifications (for example, color, shape, type, etc.), possessions (for example, food, accessories, etc.), companions, etc. A companion of an autonomous mobile body 11 refers to another autonomous mobile body 11 that has a companion relationship with the autonomous mobile body 11.

[0093] The functional configuration of the information processing server 13 can be flexibly changed according to the specifications and operation.

[0094] Furthermore, hereinafter, the expression "via the communication unit 301 and the network 21" will be omitted as appropriate when each unit of the information processing server 13 communicates with the information processing terminal 12, etc. via the communication unit 301 and the network 21. For example, when the application control unit 322 communicates with the information processing terminal 12 via the communication unit 301 and the network 21, it will simply be stated that the application control unit 322 communicates with the information processing terminal 12.

[0095] <Processing of Information Processing System 1> Next, the processing of the information processing system 1 will be described with reference to FIGS.

[0096] <Linkage function setting change process> First, the link function setting process executed by the information processing system 1 will be described with reference to the flowchart of FIG.

[0097] The link function is a function that allows the player's device to link with other devices. When the link function is turned on, the player's device can, for example, virtually interact with and exchange information with other devices.

[0098] In step S1, the information processing terminal 12 displays a link function setting screen and accepts settings for link functions.

[0099] For example, when the information processing unit 203 of the information processing terminal 12 executes an application for the autonomous moving body 11, the home screen of Fig. 10 is displayed on the display unit 241. The home screen displays the name of the autonomous moving body 11 (Kotaro), buttons 401 to 404, etc.

[0100] A button 401 is a button for displaying a setting screen.

[0101] The button 402 is a button for displaying a notice sent from the information processing server 13 .

[0102] Button 403 is a button for displaying information about the own device and for operating the own device.

[0103] Button 404 is a button for feeding the player's character food.

[0104] For example, when button 403 is selected, the menu screen in FIG. 11 is displayed on display unit 241.

[0105] Note that the method of selecting button 403 is not particularly limited. For example, it may be selected using a cursor or the like displayed within the home screen, by touching or bringing a finger close to the display position of button 404. Note that the other buttons on each screen described later can also be selected by the same method.

[0106] The menu screen displays a profile column 421, buttons 422-1 to 422-7, and button 423, etc.

[0107] The profile column 421 displays a thumbnail image of the player's character and the profile, etc. The profile of the player's character includes the name, gender, birthday, and personality.

[0108] Buttons 422-1 to 422-7 are buttons for selecting various functions. When any one of buttons 422-1 to 422-7 is selected, a screen or the like for executing the function corresponding to the selected button 422 is displayed on display unit 241.

[0109] Button 423 is a button for closing the menu screen and displaying the previous screen. Hereinafter, the reference numerals and descriptions for buttons having the same design and function as button 423 are omitted.

[0110] For example, when button 422-4 is selected, the setting screen in FIG. 12 is displayed on display unit 241. The setting screen displays button 441 and buttons 442-1 to 442-7, etc.

[0111] Button 441 is a button for returning to the home screen in FIG. 10. When button 441 is selected, the home screen in FIG. 10 is displayed on the display unit 241. Hereinafter, the reference signs and explanations for buttons having the same design and function as button 441 are omitted.

[0112] For buttons 442-1 to 442-8, each setting item and setting value are displayed. When any one of buttons 442-1 to 442-8 is pressed, a setting screen related to the selected setting item is displayed.

[0113] For example, when button 442-7 is selected, the linkage function setting screen in FIG. 13 is displayed on the display unit 241.

[0114] On the linkage function setting screen, button 461, switch 462, button 463, etc. are displayed.

[0115] Button 461 is a button for returning to the previous screen. When button 461 is selected, the previous screen is displayed on the display unit 241. Hereinafter, the reference signs and explanations for buttons having the same design and function as button 461 are omitted.

[0116] Switch 462 is a switch for setting on / off of the linkage function.

[0117] Button 463 is a button for confirming the setting content of the linkage function by switch 462. When button 463 is selected, the UI control unit 223 stores the setting content of the linkage function in the storage unit 205.

[0118] On the other hand, for example, when button 404 is selected on the home screen in FIG. 10 described above and the linkage function is not on, the warning screen in FIG. 14 is displayed on the display unit 241. On the warning screen, tabs 481-1 to 481-3, button 482, etc. are displayed.

[0119] Tab 481-1 is a tab for displaying information about the user's friends.

[0120] Tab 481-2 is a tab for displaying and editing the profile of the own device, etc.

[0121] Tab 481-3 is a tab for performing an operation to restore the display of the hidden friends among the friends of the own device.

[0122] Button 482 is a button for setting the cooperation function.

[0123] In this example, although tab 481-1 is selected, the cooperation function is not set to on and cannot be used, so a warning screen is displayed to warn that the cooperation function should be set to on.

[0124] Then, when button 482 is selected, the cooperation function setting screen shown in FIG. 13 described above is displayed, and the cooperation function can be set to on.

[0125] In step S2, the UI control unit 223 notifies the information processing server 13 of the setting content of the cooperation function.

[0126] After that, the processing of the information processing terminal 12 ends.

[0127] On the other hand, in step S51, the application control unit 322 of the information processing server 13 updates the setting content of the cooperation function. Specifically, the application control unit 322 updates the setting content of the cooperation function in the data related to the autonomous mobile body 11 (own device) stored in the storage unit 303.

[0128] After that, the processing of the information processing server 13 ends.

[0129] <First Embodiment of Friend Acquisition Processing> Next, with reference to FIGS. 15 to 32, a first embodiment of the friend acquisition processing executed by the information processing system 1 will be described.

[0130] <Processing of Information Processing Terminal 12> First, with reference to the flowchart of FIG. 15, the processing of the information processing terminal 12 will be described.

[0131] In step S101, the information processing terminal 12 requests a search for recommended candidates. A recommended candidate is another device recommended by the information processing system 1 as a candidate for a new friend of the own device (hereinafter referred to as a friend candidate).

[0132] For example, when button 422-1 is selected on the menu screen of FIG. 11 described above and the cooperation function is set to on, the friend list of FIG. 16 is displayed on the display unit 241.

[0133] Or, for example, when button 482 is selected on the warning screen of FIG. 14 described above, the cooperation function setting screen of FIG. 13 is displayed, and after the cooperation function is set on the cooperation function setting screen, the friend list of FIG. 16 is displayed on the display unit 241.

[0134] Note that the details of the display method of the friend list will be described later. Also, in the friend list of FIG. 16, the parts corresponding to the warning screen of FIG. 14 are marked with the same reference numerals, and the description thereof will be omitted as appropriate.

[0135] In the friend list, tabs 481-1 to 481-3, buttons 501-1 to 501-5, pull-down menu 502, check box 503, and button 504, etc. are displayed.

[0136] Note that hereinafter, when it is not necessary to distinguish buttons 501-1 to 501-5 individually, they are simply referred to as button 501.

[0137] Each button 501 indicates information about the user's friends. Specifically, each button 501 displays a thumbnail image and name of a friend. Also, on the button 501 of a new friend who became a friend within a recent predetermined period, the character "NEW!!" is displayed. As will be described later, when any one of the buttons 501 is selected, information about the friend corresponding to the selected button 501 is displayed on the display unit 241.

[0138] The pull-down menu 502 is a menu for setting the display order of friends. In this example, the display order is set to the new order, and the user's friends are displayed in the order of the most recent time they became friends.

[0139] The check box 503 is used to set whether to limit the friends displayed in the friend list to those present near the user's device. When the check box 503 is checked, for example, only friends within the communication range of the short-range wireless communication device provided in the communication unit 102 of the user's device among the user's friends are displayed. On the other hand, when the check box 503 is not checked, all of the user's friends are displayed.

[0140] Note that FIG. 17 shows an example of the display of the friend list when the check box 503 is checked.

[0141] The button 504 is a button for adding a new friend. When the button 504 is selected, the menu screen of FIG. 18 is displayed on the display unit 241.

[0142] This menu screen is a screen for selecting a method for adding (acquiring) a friend. The methods for adding a friend include a method of eating food together with another device and a method of reading a QR code indicating information about another device (hereinafter simply referred to as the QR code of another device).

[0143] The button 521 is a button for selecting a method of eating food together with another device.

[0144] Button 522 is a button for selecting a method for reading the QR code of another device.

[0145] Button 523 is a button for displaying a QR code showing information about the own device (hereinafter simply referred to as the own device's QR code).

[0146] When the button 521 is selected, the UI control unit 223 requests the information processing server 13 to search for recommendation candidates.

[0147] Note that the information processing server 13 can also be requested to search for recommendation candidates using the following method.

[0148] Specifically, for example, when button 404 is selected on the home screen of Fig. 10 described above, the meal home screen of Fig. 19 is displayed on the display unit 241. On the meal home screen, a player's aircraft 541, a food dish 542, a food dish 543, a button 544, a button 545, and the like are displayed.

[0149] The player's own aircraft 541 is an image that virtually represents the player's own aircraft. For example, the design, accessories, etc. of the player's own aircraft are reflected in the player's own aircraft 541.

[0150] The food dish 542 is an image that virtually represents the food dish that the player's character eats from alone.

[0151] The food tray 543 is an image that virtually represents a food tray when the player's own character eats food together with other characters (companions or potential companions).

[0152] Button 544 is a button for displaying the possessions of the player's aircraft.

[0153] Button 545 is a button for displaying the Mogu Mogu stamp, which will be described later.

[0154] For example, when the food plate 543 is selected on the meal home screen of FIG. 19, the meal home screen of FIG. 20 is displayed on the display unit 241. That is, on the meal home screen of FIG. 20, compared with the meal home screen of FIG. 19, the positions of the food plates 542 and 543 are swapped. Then, on the meal home screen of FIG. 20, that is, when the food plate 543 is displayed in front of the own device 541, if the food plate 543 is selected, the window 561 of FIG. 21 is displayed on the display unit 241.

[0155] The window 561 is a window for the own device to select a partner to eat food together. Buttons 562 to 564 and the like are displayed on the window 561.

[0156] The button 562 is a button for the own device to select a new friend as a partner to eat food together.

[0157] The button 563 is a button for the own device to select an existing friend as a partner to eat food together.

[0158] The button 564 is a button for the own device to cancel the selection of a partner to eat food together. When the button 564 is selected, the window 561 closes, and the previous screen is displayed on the display unit 241.

[0159] Then, when the button 562 is selected, the UI control unit 223 requests the information processing server 13 to search for recommended candidates.

[0160] In response to this, as will be described later, the information processing server 13 searches for recommended candidates and transmits data indicating the search results of the recommended candidates to the information processing terminal 12. The data indicating the search results of the recommended candidates includes, for example, the profile and thumbnail image of each recommended candidate.

[0161] In step S102, the information processing terminal 12 receives and displays the search results of recommended candidates. Specifically, the UI control unit 223 receives data indicating the search results of recommended candidates from the information processing server 13. Then, based on the received data, the recommended candidate list in FIG. 22 is displayed on the display unit 241.

[0162] The recommended candidate list is a screen showing the search results of recommended candidates. In this example, an example is shown where other devices with birthdays close to the own device are searched as recommended candidates. Buttons 581-1 to 581-3 and button 582 etc. are displayed in the recommended candidate list.

[0163] Hereinafter, when it is not necessary to distinguish buttons 581-1 to 581-3 individually, they are simply referred to as button 581.

[0164] Each button 581 shows information regarding each recommended candidate. Specifically, a thumbnail image and name of each recommended candidate are displayed on each button 581. As will be described later, when any one of the buttons 581 is selected, information regarding the recommended candidate corresponding to the selected button 581 is displayed.

[0165] Button 582 is a button for setting the search conditions for friend candidates. When button 582 is selected, the search condition setting screen in FIG. 23 is displayed on the display unit 241.

[0166] The search condition setting screen is a screen for setting the search conditions for friend candidates. A text box 601, a pull-down menu 602, a button 603, a pull-down menu 604, a button 605, and a button 606 etc. are displayed on the search condition setting screen.

[0167] The text box 601 is used to input the name of another device as a search condition for friend candidates.

[0168] The pull-down menu 602 is used to set the gender of the other device as a search condition for friend candidates. When the pull-down menu 602 is selected, gender options are displayed, allowing the gender to be selected.

[0169] Button 603 is selected when setting the birthday of another device as a search condition for a companion candidate. When button 603 is selected, a birthday input screen (not shown) is displayed, allowing the birthday to be input.

[0170] The pull-down menu 604 is used to set the address of another device as a search condition for a companion candidate. When the pull-down menu 604 is selected, address options (for example, prefectures) are displayed, allowing the user to select an address.

[0171] Button 605 is a button for canceling the search for fellow user candidates. When button 605 is selected, the search for fellow user candidates is not performed and the previous screen is displayed on display unit 241.

[0172] Button 606 is a button for searching for fellow user candidates using the set search conditions.

[0173] In step S103, the UI control unit 223 determines whether or not search conditions for fellow users have been set. If button 606 is selected after at least one search condition has been set on the setting screen in Figure 23, the UI control unit 223 determines that search conditions for fellow users have been set, and the process proceeds to step S104.

[0174] In step S104, the UI control unit 223 requests a search for fellow user candidates. Specifically, the UI control unit 223 notifies the information processing server 13 of the set search conditions and requests a search for fellow user candidates.

[0175] In response to this, the information processing server 13 searches for fellow user candidates based on the set search conditions, as will be described later, and transmits data indicating the search results of the fellow user candidates to the information processing terminal 12.

[0176] In step S105, the information processing terminal 12 receives and displays the search results of potential friends. Specifically, the UI control unit 223 receives data indicating the search results of potential friends from the information processing server 13. The data indicating the search results of potential friends includes, for example, the profiles and thumbnail images of each potential friend. Then, based on the received data, a potential friend list indicating the search results of potential friends is displayed on the display unit 241.

[0177] Note that the potential friend list has the same screen as the recommended candidate list in FIG. 22 described above. That is, in the potential friend list, information regarding the searched potential friends is displayed in each button 581 in FIG. 22.

[0178] After that, the process proceeds to step S106.

[0179] On the other hand, in step S103, if it is determined that the search conditions for potential friends are not set, the processes of steps S104 and S105 are skipped, and the process proceeds to step S106.

[0180] In step S106, the UI control unit 223 determines whether a potential friend is selected. If it is determined that no potential friend is selected, the process returns to step S103.

[0181] After that, in step S106, the processes of steps S103 to S106 are repeatedly executed until it is determined that a potential friend is selected.

[0182] On the other hand, for example, if any button 581 is selected in the recommended candidate list or the potential friend list in FIG. 22, the window 621 in FIG. 24 is displayed on the display unit 241. The window 621 displays a profile column 622, a button 623, a button 624, and the like.

[0183] In the profile column 622, the thumbnail image, name, gender, birthday, owner's nickname, and address of the selected friend candidate are displayed.

[0184] Button 623 is a button for canceling the selection of a friend candidate. When button 623 is selected, window 621 closes, and the previous screen is displayed on display unit 241.

[0185] Button 624 is a button for inviting the selected friend to a meal. When button 624 is selected, in step S106, it is determined that a friend candidate has been selected, and the process proceeds to step S107.

[0186] In step S107, UI control unit 223 notifies information processing server 13 of the selection result of the friend candidate.

[0187] In contrast, as will be described later, information processing server 13 receives the notification of the selection result of the friend candidate and transmits data regarding the bait for the own vehicle owned by the user. The data regarding the bait for the own vehicle includes, for example, the type and quantity of the bait for the own vehicle.

[0188] In step S108, information processing terminal 12 receives and displays the data regarding the bait. Specifically, UI control unit 223 receives the data regarding the bait for the own vehicle from information processing server 13. Then, based on the received data, the bait list in FIG. 25 is displayed on display unit 241.

[0189] The bait list is a screen that displays a list of the bait for the own vehicle owned by the user. The bait can be obtained, for example, when the user purchases it from the service for autonomous mobile body 11 provided by information processing server 13 or acquires it as a prize. Icons 641-1 to 641-7, etc. are displayed in the bait list.

[0190] In the following, when it is not necessary to individually distinguish between icon 641-1 to icon 641-7, they are simply referred to as icon 641.

[0191] Each icon 641 includes an image indicating the type of bait owned by the user. Also, below each icon 641, the name of the bait is displayed, and in the upper right corner, the number of baits owned by the user indicated by each icon 641 is displayed.

[0192] Note that a + sign is displayed on icon 641-4 indicating a bait not owned by the user. For example, when the + sign is selected, a screen for purchasing the bait corresponding to icon 641-4 is displayed on the display unit 241.

[0193] In step S109, the information processing terminal 12 notifies the selection result of the bait.

[0194] Specifically, in the bait list of FIG. 25, when an icon 641 other than icon 641-4 that the user does not own is selected, window 661 of FIG. 26 is displayed on the display unit 241.

[0195] In window 661, the name, image, and description of the selected bait are displayed. Also, button 662 and button 663 are displayed in window 661.

[0196] Button 662 is a button for canceling the selection of the bait. When button 662 is selected, window 661 closes, and the previous screen is displayed on the display unit 241.

[0197] Button 663 is a button for confirming the selection of the bait. When button 663 is selected, the UI control unit 223 notifies the information processing server 13 of the selection result of the bait.

[0198] On the other hand, as will be described later, the information processing server 13 receives the notification of the selection result of the bait and controls the autonomous mobile body 11 to perform an operation of eating the bait.

[0199] In step S110, the information processing terminal 12 displays a meal between the autonomous moving body 11 (own device) and the fellow candidate. For example, the meal screen of FIG.

[0200] 27 displays an animation (moving image) virtually showing the player's character and the selected companion candidate eating food together. This meal screen displays a switch 681, the player's character 682, a companion candidate 683, a label 684, a food dish 685, and food 686.

[0201] Switch 681 is a switch for selecting the display method of the meal screen. The user can use switch 681 to set the display method of the meal screen to "normal" or "AR (Augmented Reality)." When switch 681 is set to "normal," the meal screen of FIG. 27 is displayed. On the other hand, when switch 681 is set to "AR," the meal screen of FIG. 28, which will be described later, is displayed. In other words, switch 681 makes it possible to switch between the meal screen of FIG. 27 and the meal screen of FIG. 28.

[0202] The player's own aircraft 682 is an image that virtually represents the player's own aircraft. For example, the design and accessories of the actual player's aircraft are reflected in the player's own aircraft 682.

[0203] The companion candidate 683 is an image that virtually represents the selected companion candidate. For example, the companion candidate 683 reflects the design, accessories, etc. of the actual companion candidate.

[0204] The label 684 indicates the name of the companion candidate 683 and is displayed near the companion candidate 683 .

[0205] The food dish 685 is an image that virtually represents a food dish.

[0206] The food 686 is an image that virtually represents the food selected in the process of step S109, and is displayed on the food dish 685.

[0207] Then, an animation is displayed in which the player's character 682 and the potential companion 683 eat food 686 on a food dish 685 together.

[0208] 28, a video is displayed in which an image of the selected friend candidate is virtually superimposed on a video (through image) captured by the player's own device using AR. More specifically, a video is displayed in which the player's real device 701 and the virtual friend candidate 683 eat food together.

[0209] In the figure, parts corresponding to those in the meal screen of FIG. 27 are given the same reference numerals, and their explanation will be omitted as appropriate.

[0210] 28 displays a switch 681, a potential companion 683, a label 684, the player's aircraft 701, a food dish 702, food 703, and a button 704. More specifically, the switch 681, the potential companion 683, food 703, and button 704 are superimposed on a video of the player's aircraft 701 and food dish 702 in reality.

[0211] The food 703 is an image that virtually represents the food selected in the processing of step S109, and is displayed superimposed on the actual food dish 702.

[0212] Button 704 is a button for capturing a still image of one scene from the video (through image) displayed on the meal screen. For example, when button 704 is pressed, image processing unit 222 generates an image in which switch 681 and button 704 are removed from the image displayed on the meal screen at the moment button 704 was pressed, and stores the image in storage unit 205.

[0213] For example, the image processing unit 222 generates a video in which the real player's device 682 and the virtual friend candidate 683 eat the virtual food 703 together by superimposing the virtual friend candidate 683 and the food 703 on a captured image of the player's real device 682 eating food (which does not actually exist) from the food dish 702. The display unit 241, under the control of the display control unit 231, displays the video generated by the image processing unit 222, thereby displaying the eating screen of Fig. 28.

[0214] At this time, the recognition unit 221 recognizes the food dish 702 in the captured video. The image processing unit 222 superimposes virtual food 703 on the recognized food dish 702. The image processing unit 222 also superimposes potential companions 683 around the food dish 702, and moves the potential companions 683 to represent the appearance of the companions eating the food 703.

[0215] The images of the friend candidate 683 and the food 703 may be either two-dimensional or three-dimensional.

[0216] In response to this, for example, the user adjusts the angle of view of the camera 211 as necessary so that the positional relationship between the user's device 701 and the fellow player candidate 683 becomes appropriate.

[0217] After the meal is over, the friend setting unit 224 notifies the information processing server 13 that the player's machine has acquired a new friend, thereby adding the friend candidate who ate the food together to the player's machine's friends.

[0218] In step S111, the information processing terminal 12 receives and displays the image of the stamp.

[0219] Specifically, as will be described later, the information processing server 13 generates a stamp image to which a stamp serving as a reward for the current meal has been added, and transmits the stamp image to the information processing terminal 12.

[0220] On the other hand, the UI control unit 223 receives the stamp image from the information processing server 13. Then, based on the received stamp image, the stamp screen (mogu mogu stamp) in FIG. 29 is displayed. On this stamp screen, a mount 721 and stamps 722 to 724, etc. are displayed.

[0221] The mount 721 is an image imitating, for example, a point card. On the mount 721, the pressing positions of stamps from 1 to 10 are displayed.

[0222] The stamp 722 is a stamp given by feeding the own aircraft. The stamp 723 is a stamp given by feeding a friend of the own aircraft. The stamp 724 is a stamp given by the own aircraft newly acquiring a friend. For the stamp 724, for example, a stamp created by a new friend user is used. In this example, the character "Mai", which is the name of the new friend, is shown on the stamp 724.

[0223] That is, in this example, a total of three stamps are given by the own aircraft eating food with the new friend "Mai". The stamps are accumulated until a predetermined number (10 in this example), and when the predetermined number is reached, a prize (for example, a snack) is given.

[0224] In step S112, the information processing terminal 12 receives and displays the image of the commemorative gift.

[0225] Specifically, as will be described later, the information processing server 13 generates a commemorative gift image showing a commemorative gift that is a reward for the own aircraft newly acquiring a friend, and transmits it to the information processing terminal 12.

[0226] On the other hand, the UI control unit 223 receives the commemorative gift image from the information processing server 13. Then, based on the commemorative gift image, the window 741 in FIG. 30 is displayed on the display unit 241. In the window 741, an image and a message representing the commemorative gift given as a commemoration of newly acquiring a friend are displayed.

[0227] In step S113, the information processing terminal 12 displays information about the new friend. For example, after the window 741 in Fig. 30 is closed, the window 761 in Fig. 31 is displayed on the display unit 241.

[0228] Window 761 is a window for notifying that a new friend has been acquired. Window 761 displays a profile field 762, a button 763, a button 764, and the like.

[0229] A thumbnail image and profile of the new friend are displayed in the profile field 421. The profile of the new friend includes the name, gender, birthday, owner's nickname, and address.

[0230] The button 763 is a button for closing the window 761 .

[0231] Button 764 is a button for displaying a buddy list. When button 764 is pressed, for example, the buddy list shown in FIG.

[0232] Thereafter, the processing of the information processing terminal 12 ends.

[0233] <Processing of information processing server 13> Next, with reference to a flowchart in FIG. 32, a process executed by the information processing server 13 in response to the process of the information processing terminal 12 in FIG. 15 will be described.

[0234] In step S151, the information processing server 13 searches for recommendation candidates and notifies the search results.

[0235] Specifically, when a search for recommended candidates is requested by the information processing terminal 12 in the processing of step S101 in FIG. 15 described above, the application control unit 322 searches for recommended candidates from data related to each autonomous moving body 11 stored in the storage unit 303 in accordance with predetermined conditions. For example, the application control unit 322 searches for recommended candidates from other devices that have the link function turned on and that are not yet friends with the autonomous moving body 11, and whose birthdays differ from the autonomous moving body 11 by a predetermined range. The application control unit 322 transmits data indicating the search results for recommended candidates to the information processing terminal 12.

[0236] In step S152, the application control unit 322 determines whether a search for fellow user candidates has been requested. If a search for fellow user candidates has been requested by the information processing terminal 12 in step S104 of Fig. 15 described above, the process proceeds to step S153.

[0237] In step S153, the information processing server 13 searches for fellow candidate members and notifies the information processing terminal 12 of the search results. For example, the application control unit 322 searches for fellow candidate members based on the search criteria specified by the information processing terminal 12 from other devices that have the cooperation function set to on and that have not yet become friends with the autonomous moving body 11 in the data related to each autonomous moving body 11 stored in the storage unit 303. The application control unit 322 transmits data indicating the search results for fellow candidate members to the information processing terminal 12.

[0238] Thereafter, the process proceeds to step S154.

[0239] On the other hand, if it is determined in step S152 that a search for fellow user candidates has not been requested, the process of step S153 is skipped and the process proceeds to step S154.

[0240] In step S154, the application control unit 322 determines whether or not the selection result of the fellow user candidate has been notified. If it is determined that the selection result of the fellow user candidate has not been notified, the process proceeds to step S152.

[0241] After that, in step S154, the processes of steps S152 to S154 are repeatedly executed until it is determined that the search result of the friend candidate is notified.

[0242] On the other hand, in step S154, when the search result of the friend candidate is notified from the information processing terminal 12 in step S107 of FIG. 15 described above, the process proceeds to step S155.

[0243] In step S155, the information processing server 13 transmits data related to the bait and receives the selection result of the bait. Specifically, the application control unit 322 reads out the data related to the bait for the own machine from the storage unit 303 and transmits the read data to the information processing terminal 12.

[0244] In addition, the application control unit 322 receives the notification of the selection result of the bait from the information processing terminal 12 in step S109 of FIG. 15 described above.

[0245] In step S156, the information processing server 13 controls the autonomous mobile body 11 (own machine) to perform an operation of eating the bait. Specifically, the application control unit 322 instructs the autonomous movement control unit 321 to cause the own machine to perform an operation of eating the bait. The operation control unit 334 of the autonomous movement control unit 321 controls the autonomous mobile body 11, which is the own machine, to perform an operation of eating the bait.

[0246] In step S157, the application control unit 322 updates the data related to the friends of the autonomous mobile body 11 (own machine). Specifically, the application control unit 322 receives a notification indicating that the own machine has acquired a new friend transmitted from the information processing terminal 12 in step S110 of FIG. 15 described above. The application control unit 322 adds the other machines that ate the bait together with the own machine this time as new friends in the data related to the own machine stored in the storage unit 303.

[0247] In step S158, the information processing server 13 transmits an image of a stamp. Specifically, the application control unit 322 generates a stamp image indicating the stamp given as a privilege for the own vehicle that ate food with a new friend this time, and transmits it to the information processing terminal 12.

[0248] In step S159, the information processing server 13 transmits an image of a commemorative gift. Specifically, the application control unit 322 generates a commemorative gift image indicating the commemorative gift that is a reward for the own vehicle having acquired a new friend, and transmits it to the information processing terminal 12.

[0249] After that, the processing of the information processing server 13 ends.

[0250] <Second Embodiment of Friend Acquisition Process> Next, with reference to the flowchart of FIG. 33, a second embodiment of the friend acquisition process executed by the information processing system 1 will be described.

[0251] In step S201, the information processing terminal 12 displays a QR code (registered trademark) reading screen.

[0252] For example, when the button 522 is selected on the menu screen of FIG. 18 described above, the reading screen of FIG. 34 is displayed on the display unit 241.

[0253] This reading screen is a screen for reading the QR code of another autonomous mobile body 11 (other vehicle) that the own vehicle wants to make a friend with. On the reading screen, a frame 781 etc. are displayed.

[0254] The frame 781 indicates the range for reading the QR code of another vehicle that the own vehicle wants to make a friend with.

[0255] In step S202, the information processing terminal 12 reads and transmits the QR code of another autonomous mobile body 11 (other vehicle) that it wants to make a friend with.

[0256] For example, after a user of another device displays the menu screen of FIG. 18 on the owned information processing terminal 12, the user selects the button 523 on the menu screen. As a result, as shown in FIG. 35, a QR code 801 representing information about another device is displayed on the information processing terminal 12 of the user of the other device.

[0257] On the other hand, the user of the own device adjusts the position of the camera 211 of the information processing terminal 12 so that the QR code 801 of the other device in FIG. 35 is within the frame 781 of the reading screen in FIG. 34. Then, the user of the own device takes a picture of the QR code 801 with the camera 211.

[0258] The friend setting unit 224 adds the other device corresponding to the QR code 801 as a friend of the own device by transmitting the image of the photographed QR code 801 to the information processing server 13.

[0259] On the other hand, in step S251, the information processing server 13 receives the QR code and searches for another autonomous mobile body 11 (another device) corresponding to the QR code.

[0260] Specifically, the application control unit 322 receives the image of the QR code transmitted from the information processing terminal 12. The application control unit 322 analyzes the received QR code to obtain the identification information (for example, the manufacturing number) of the autonomous mobile body 11. The application control unit 322 searches for the autonomous mobile body 11 having the identification information that matches the obtained identification information in the data related to each autonomous mobile body 11 stored in the storage unit 303.

[0261] In step S252, the application control unit 322 registers the searched other autonomous mobile body 11 as a new friend. Specifically, the application control unit 322 adds the other device searched in the process of step S251 as a new friend in the data related to the own device stored in the storage unit 303.

[0262] In step S253, the information processing server 13 transmits data related to fellow members of the autonomous mobile body 11 (own device). Specifically, the application control unit 322 reads out data related to fellow members of the own device from the storage unit 303. The data related to fellow members of the own device includes data related to newly added fellow members, and includes, for example, a profile and thumbnail image of each fellow member.

[0263] Thereafter, the processing of the information processing server 13 ends.

[0264] In response to this, in step S203, the information processing terminal 12 receives and displays data related to the companions of the autonomous moving body 11 (own device). Specifically, the UI control unit 223 receives the data related to the companions of the own device transmitted from the information processing server 13.

[0265] Then, based on the received data, first, a window 821 shown in FIG.

[0266] Window 821 is a window for notifying the player that their own character has acquired a new friend. Window 821 displays the name of the newly acquired friend, a button 822, and the like.

[0267] When the button 822 is selected, the window 821 closes and the companion list of FIG.

[0268] The companion list in Fig. 37 has the same content as the companion list in Fig. 16. However, in the example in Fig. 37, only new companions who have joined the player's aircraft within the most recent predetermined period are displayed.

[0269] In addition, all of the companions of the player's aircraft may be displayed in the companion list of FIG.

[0270] Thereafter, the processing of the information processing terminal 12 ends.

[0271] <Processing for the autonomous mobile unit 11 to eat food with existing companions> Next, a process in which the autonomous moving body 11 eats food together with existing companions will be described with reference to FIGS.

[0272] <Processing of information processing terminal 12> First, the processing of the information processing terminal 12 will be described with reference to the flowchart of FIG.

[0273] In step S301, the information processing terminal 12 requests data relating to the companions of the autonomous moving body 11 (own device).

[0274] For example, when button 422-1 is selected on the menu screen of FIG. 11 described above, if the collaboration function is set to on, or when button 563 is selected on window 561 of FIG. 21 described above, the UI control unit 223 requests data regarding the companions of the player's device from the information processing server 13.

[0275] In response to this, the information processing server 13 transmits data relating to the fellow autonomous moving body 11, similar to the processing in step S253 of FIG.

[0276] In step S302, the information processing terminal 12 receives and displays data related to fellow users of the autonomous moving body 11 (own device). Specifically, the UI control unit 223 receives the data related to fellow users of the own device transmitted from the information processing server 13. Then, based on the received data, the fellow user list of FIG. 16 described above is displayed on the display unit 241.

[0277] In step S303, the information processing terminal 12 accepts the selection of the companion to eat food with and notifies the selection result.

[0278] Specifically, when any of the buttons 501 is selected in the companion list of FIG. 16, that is, when any of the companions of the player's device is selected, the profile screen of FIG. 39 is displayed on the display unit 241.

[0279] The profile screen is a screen that displays the profiles of selected friends, etc. On the profile screen, the thumbnail image and profile of the selected friend, button 861, and button 862, etc. are displayed.

[0280] The friend's profile includes name, gender, birthday, the owner's nickname, and address. Also, the friend's profile includes the date when they became a friend (buddy) with the own device, the number of times they have met the own device, and the number of times they have eaten food together with the own device.

[0281] Note that as the number of times of meeting the own device, for example, the number of times a friend has approached within a predetermined range from the own device is used. As the range that satisfies this condition, for example, the communication range of the short-range wireless communication device provided in the communication unit 102 of the own device is used.

[0282] Button 861 is a button for the own device to select the displayed friend as the partner to eat food with.

[0283] Button 862 is a button for performing other operations on the displayed friend other than selecting the partner to eat food with.

[0284] For example, when button 861 is selected on the profile screen of FIG. 39, window 881 of FIG. 40 is displayed on display unit 241. In window 881, profile column 882, button 883, and button 884, etc. are displayed.

[0285] In profile column 882, the thumbnail image and profile of the selected friend are displayed. The profile includes name, gender, birthday, the owner's nickname, and address.

[0286] Button 883 is a button for canceling the selection of the displayed friend.

[0287] Button 884 is a button for confirming the selection of the displayed companion. By selecting button 884, the companion whose profile is displayed in window 881 is selected as the companion to eat food with the player's character.

[0288] Then, the UI control unit 223 notifies the information processing server 13 of the selection result of the companion that will eat the food together with the player's own machine.

[0289] In step S304, data relating to the bait is received and displayed, similar to the process in step S108 of FIG. 15 described above.

[0290] In step S305, the result of the bait selection is notified in the same manner as in the process of step S109 in FIG. 15 described above.

[0291] In step S306, the state of the autonomous moving body 11 (own device) and the companion having a meal is displayed, similar to the process of step S110 in Fig. 15. Note that, whereas in the process of step S110, the companion having a meal with the own device is a new companion candidate, in this process, the companion having a meal with the own device is an existing companion selected in the process of step S303, which is different.

[0292] In step S307, an image of a stamp is received and displayed, similar to the process in step S111 of Fig. 15. In this process, two types of stamps are given: a stamp given when the player's own machine feeds food, and a stamp given when the player's machine feeds a companion.

[0293] Thereafter, the processing of the information processing terminal 12 ends.

[0294] <Processing of information processing server 13> Next, with reference to the flowchart of FIG. 41, a process executed by the information processing server 13 in response to the process of the information processing terminal 12 of FIG. 38 will be described.

[0295] In step S351, the information processing server 13 reads out data related to fellow members of the autonomous mobile body 11 (own device). Specifically, in response to the request from the information processing terminal 12 in step S301 of Fig. 38 described above, the application control unit 322 reads out data related to fellow members of the own device from the storage unit 303. The data related to fellow members of the own device includes, for example, a profile and thumbnail image of each fellow member.

[0296] In step S352, the information processing server 13 transmits data related to the companions of the autonomous moving body 11 (own device). Specifically, the application control unit 322 transmits to the information processing terminal 12 the data read in the process of step S351.

[0297] In step S353, the information processing server 13 receives the selection result of the companions to eat food together. Specifically, the application control unit 322 receives the notification of the selection result of the companions to eat food together with the own device, which was transmitted from the information processing terminal 12 in step S303 of Fig. 38 described above.

[0298] In step S354, data regarding the food is transmitted and the selection result is received, similar to the process in step S155 of FIG. 32 described above.

[0299] In step S355, similar to the processing in step S156 in FIG. 32 described above, the autonomous moving body 11 (own body) is controlled to perform an action of eating food.

[0300] In step S356, the application control unit 322 updates data related to fellow autonomous moving body 11 (own device). Specifically, the application control unit 322 updates the number of times that fellow autonomous moving body 11 has eaten food together this time in the data related to the own device stored in the storage unit 303.

[0301] In step S357, the image of the stamp is transmitted in the same manner as in step S158 of FIG.

[0302] After that, the processing of the information processing server 13 ends.

[0303] <Processing to Hide Friends> Next, with reference to the flowchart of FIG. 42, the processing to hide friends, which is executed by the information processing system 1, will be described.

[0304] In step S401, the information processing terminal 12 requests the information processing server 13 for data regarding the friends of the autonomous mobile body 11 (own vehicle) in the same manner as the processing in step S301 of FIG. 38 described above.

[0305] In response, in step S451, the information processing server 13 reads out data regarding the friends of the autonomous mobile body 11 in the same manner as the processing in step S351 of FIG. 41 described above, and in step S452, transmits the read data to the information processing terminal 12 in the same manner as the processing in step S352 of FIG. 41 described above.

[0306] In step S402, the information processing terminal 12 receives and displays data regarding the friends of the autonomous mobile body 11 (own vehicle) in the same manner as the processing in step S302 of FIG. 38 described above. As a result, the friend list shown in FIG. 16 above is displayed on the display unit 241.

[0307] In step S403, the information processing terminal 12 displays data regarding the selected friend. Specifically, when any button 501 is selected in the friend list of FIG. 16, that is, when any one of the friends of the own vehicle is selected, the profile screen of FIG. 39 described above is displayed on the display unit 241.

[0308] In step S404, the information processing terminal 12 accepts the setting to hide the selected friend and notifies the setting content.

[0309] Specifically, when button 862 is selected on the profile screen of Fig. 39, window 901 of Fig. 43 is displayed on display unit 241. In window 901, button 902 and the like are displayed.

[0310] The button 902 is a button for hiding the selected companion. When the button 902 is selected, the window 901 closes and the window 921 of FIG.

[0311] A window 921 is a window for confirming whether or not to hide the selected friend. In the window 921, a button 922, a button 923, and the like are displayed.

[0312] Button 922 is a button for canceling the non-display of the selected fellow user. When button 922 is selected, window 921 closes and the non-display of the selected fellow user is canceled.

[0313] Button 922 is a button for confirming that the selected friend should not be displayed. When button 922 is selected, window 921 closes and window 941 in Fig. 45 is displayed on display unit 241 for a predetermined time.

[0314] A window 941 is a window that notifies the user that the selected companion has been hidden.

[0315] For example, a friend set to be hidden will not be displayed in the friend list on the information processing terminal 12. Furthermore, information about a friend set to be hidden will not be transmitted from the information processing server 13 to the information processing terminal 12. This allows, for example, communication with a specific friend among the friends of the player's device to be stopped.

[0316] The UI control unit 223 transmits data indicating the hidden companions to the information processing server 13.

[0317] Thereafter, the processing of the information processing terminal 12 ends.

[0318] In response to this, in step S453, the information processing server 13 sets the selected friend to be hidden. Specifically, the application control unit 322 receives data indicating the hidden friend from the information processing terminal 12. The application control unit 322 sets the friend selected by the user to be hidden in the data related to the information processing terminal 12 stored in the storage unit 303.

[0319] Thereafter, the processing of the information processing server 13 ends.

[0320] <Process to unhide companions> Next, with reference to the flowchart of FIG. 46, a process for canceling the hiding of a friend, which is executed by the information processing system 1, will be described.

[0321] In step S501, the information processing terminal 12 requests data relating to a hidden companion.

[0322] For example, when tab 481-3 is selected in the companion list of Fig. 16 described above, the setting screen of Fig. 47 is displayed. A button 961 is displayed on the setting screen.

[0323] The button 961 is a button for checking the hidden companions. When the button 961 is selected, the UI control unit 223 requests the information processing server 13 for data related to the hidden companions.

[0324] In response to this, in step S551, the information processing server 13 reads out data related to the hidden fellow users. Specifically, the application control unit 322 reads out data related to the hidden fellow users of the own device from the storage unit 303. The data related to the hidden fellow users includes, for example, a profile and a thumbnail image of each fellow user.

[0325] In step S552 , the information processing server 13 transmits data relating to the hidden companion to the information processing terminal 12 .

[0326] In contrast, in step S502, the information processing terminal 12 receives and displays data regarding the friends that are hidden. Specifically, the UI control unit 223 receives the data regarding the friends that are hidden and transmitted from the information processing server 13. Then, based on the received data, the hidden friends list in FIG. 48 is displayed on the display unit 241.

[0327] The hidden friends list is a list that shows the hidden friends in a list form. Buttons 981-1 and 981-2 etc. are displayed in the hidden friends list.

[0328] Hereinafter, when it is not necessary to distinguish buttons 981-1 and 981-2 individually, they are simply referred to as button 981.

[0329] Each button 981 indicates information regarding the hidden friends. Specifically, a thumbnail image and name of the hidden friend are displayed on each button 981.

[0330] In step S503, the information processing terminal 12 accepts the selection of the friend to unhide and notifies the selection result.

[0331] Specifically, in the hidden friends list in FIG. 48, when any button 981 is selected, that is, when any of the friends set to be hidden is selected, the profile screen in FIG. 49 is displayed on the display unit 241.

[0332] The profile screen is a screen that displays the profile etc. of the selected friend. The profile of the selected friend and button 1001 etc. are displayed on the profile screen.

[0333] The companion's profile includes the name, gender, birthday, owner's nickname, and address. The companion's profile also includes the date on which the companion became a companion with the player's character, the number of times the companion has met the player's character, and the number of times the companion has eaten food with the player's character.

[0334] A button 1001 is a button for canceling the hiding of a displayed companion. When the button 1001 is selected, a window 1021 is displayed on the display unit 241 for a predetermined period of time.

[0335] A window 1021 is a window that notifies the user that the selected friend has been unhidden.

[0336] The UI control unit 223 notifies the information processing server 13 of the selection result of the companions to be unhidden.

[0337] Thereafter, the processing of the information processing terminal 12 ends.

[0338] In response to this, in step S553, the information processing server 13 cancels the hiding of the selected friend. Specifically, the application control unit 322 cancels the hiding of the friend selected by the user in the data related to the own device stored in the storage unit 303.

[0339] Thereafter, the processing of the information processing server 13 ends.

[0340] <Profile editing process> Next, the profile editing process executed by the information processing system 1 will be described with reference to the flowchart of FIG.

[0341] In step S601, the information processing terminal 12 requests transmission of data related to the autonomous moving body 11 (own device). Specifically, when the tab 481-2 is selected in the companion list in Fig. 16 described above, the UI control unit 223 requests data related to the autonomous moving body 11 from the information processing server 13.

[0342] In contrast, in step S651, the application control unit 322 reads data related to the autonomous mobile body 11 (own vehicle) from the storage unit 303.

[0343] In step S652, the application control unit 322 transmits the read data related to the autonomous mobile body 11 (own vehicle) to the information processing terminal 12.

[0344] In contrast, in step S602, the information processing terminal 12 receives and displays data related to the autonomous mobile body 11 (own vehicle). Specifically, the UI control unit 223 receives data related to the own vehicle from the information processing server 13. Then, based on the received data, the profile screen of FIG. 52 is displayed on the display unit 241.

[0345] On the profile screen of FIG. 52, the profile of the own vehicle, the thumbnail image 1041, the stamp 1042, and the buttons 1043 to 1045, etc. are displayed.

[0346] The profile of the own vehicle includes the name, gender, birthday, owner's nickname, and address.

[0347] For the thumbnail image 1041, for example, an image of the own vehicle taken is used.

[0348] The stamp 1042 is a stamp generated based on the name of the own vehicle, as will be described later.

[0349] The button 1043 is a button for displaying the QR code of the own vehicle. For example, when the button 1043 is selected, the QR code screen of FIG. 35 described above is displayed.

[0350] The button 1044 is a button for changing the appearance of the own vehicle.

[0351] The button 1045 is a button for editing the profile of the own vehicle.

[0352] In step S603, the information processing terminal 12 accepts the editing of the profile of the autonomous mobile body 11 (own vehicle) and notifies the editing content. Specifically, when the button 1045 is selected on the profile screen of FIG. 52, the editing screen of FIG. 53 is displayed on the display unit 241.

[0353] The editing screen of FIG. 53 is a screen for editing the profile of the own vehicle. The editing screen displays the profile of the own vehicle, a thumbnail image 1061, a button 1062, a stamp 1063, a button 1064, a text box 1065, a pull-down menu 1066, and a button 1067, etc.

[0354] The button 1062 is a button for editing the thumbnail image 1061.

[0355] The button 1064 is a button for editing the stamp 1063.

[0356] The text box 1065 is used for editing the nickname of the owner of the own vehicle. That is, the user can input an arbitrary nickname as his / her own nickname into the text box 1065.

[0357] The pull-down menu 1066 is used for setting the address of the own vehicle. When the pull-down menu 1066 is selected, options for the address (for example, prefectures) are displayed, and the address can be selected.

[0358] The button 1067 is a button for determining the content of the changed profile of the own vehicle.

[0359] For example, when the button 1062 is selected, the window 1081 of FIG. 54 is displayed on the display unit 241. The window 1081 is a window for selecting a method for adding a photo to be used for the thumbnail image 1061. Buttons 1082 to 1084, etc. are displayed on the window 1081.

[0360] When button 1082 is selected, camera 211 is activated. This makes it possible to take an image to be used for thumbnail image 1061.

[0361] When button 1083 is selected, for example, a list of images stored in storage unit 205 is displayed on display unit 241. This makes it possible to select an image to be used for thumbnail image 1061 from the displayed list.

[0362] When button 1084 is selected, the selection of thumbnail image 1061 is cancelled and window 1081 is closed.

[0363] For example, when button 1064 is selected on the profile editing screen of FIG. 53, the stamp editing screen of FIG. 55 is displayed on display unit 241. The stamp editing screen is a screen for editing the stamps of the own device. On the stamp editing screen, stamp 1101, text box 1102, pull-down menu 1103, pull-down menu 1104, button 1105, etc. are displayed.

[0364] Stamp 1101 indicates the state of the stamp being edited.

[0365] Text box 1102 is used for inputting the name to be displayed on stamp 1101. Then, the name input in text box 1102 is displayed on stamp 1101.

[0366] Pull-down menu 1103 is used for setting the color of stamp 1101. When pull-down menu 1103 is selected, color options are displayed and the color of stamp 1101 can be selected.

[0367] Pull-down menu 1104 is used for setting the design of stamp 1101 (for example, the outer shape of stamp 1101, etc.). When pull-down menu 1104 is selected, design options are displayed and the design of stamp 1101 can be selected.

[0368] A button 1105 is used to confirm the edited content of the stamp 1101. When the button 1105 is selected, the stamp of the player's device is changed to the stamp 1101 that is being displayed.

[0369] For example, when button 1044 is selected on the profile screen of Fig. 52, the dress-up screen of Fig. 56 is displayed. The dress-up screen is a screen for changing the dress-up of the image of the user's own device that is virtually displayed on each information processing terminal 12.

[0370] The dress-up screen displays the player's own device 1121, buttons 1122 to 1125, and the like.

[0371] The player's own device 1121 is an image that virtually shows the state of the dress-up items attached to the player's own device.

[0372] Button 1122 is a button for selecting a collar to be attached to the player's character. The currently selected collar is displayed on button 1122. If no collar is selected, the word "none" is displayed on button 1122.

[0373] Button 1123 is a button for selecting a name sticker to be attached to the player's character. The currently selected name sticker is displayed on button 1123. If no name sticker is selected, the word "none" is displayed on button 1123.

[0374] Button 1124 is a button for selecting a dog tag to be attached to the player's aircraft. The currently selected dog tag is displayed on button 1124. If no dog tag is selected, the word "none" is displayed on button 1124.

[0375] Button 1125 is a button for confirming the dress-up item displayed on player's device 1121 .

[0376] For example, when button 1122 is selected, window 1141 in FIG. 57 is displayed on display unit 241. Window 1141 is a window for selecting a collar to be worn on the own vehicle. A list of collars, cursor 1142, button 1143, etc. are displayed in window 1141.

[0377] Cursor 1142 is used to select a collar to be worn on the own vehicle. Then, as shown in FIG. 58, the collar 1144 selected by cursor 1142 is superimposed on the own vehicle 1121. Thereby, the state of wearing the selected collar on the own vehicle can be confirmed.

[0378] Button 1143 is a button for finalizing the selected collar. That is, when button 1143 is selected, the collar selected by cursor 1142 is finalized. Then, as shown in FIG. 59, window 1141 closes, and the selected collar is displayed on button 1122. Further, when button 1125 is selected, the dressing item worn on the own vehicle 1121 is finally finalized.

[0379] UI control unit 223 receives the editing of the profile, thumbnail image, stamp, and dressing item of the own vehicle as described above, and generates data indicating the editing content. UI control unit 223 transmits the generated data to information processing server 13.

[0380] After that, the processing of information processing terminal 12 ends.

[0381] On the other hand, in step S653, information processing server 13 updates the profile etc. of autonomous mobile body 11 (own vehicle). Specifically, application control unit 322 receives data indicating the editing content of the profile etc. from information processing terminal 12. Application control unit 322 updates the data related to the own vehicle stored in storage unit 303 based on the received data.

[0382] After that, the processing of information processing server 13 ends.

[0383] In this way, the entertainment of the autonomous mobile body 11 can be improved.

[0384] For example, the user can enjoy the pleasure of increasing the number of friends of the autonomous mobile body 11 or communicating with other autonomous mobile bodies 11. Also, the user can communicate with the users of other autonomous mobile bodies 11 through the friends of the autonomous mobile body 11, for example.

[0385] The user can enjoy the state where the autonomous mobile body 11 eats food with other autonomous mobile bodies 11 on the meal screens of FIGS. 27 and 28.

[0386] The user can bring out the personality of the autonomous mobile body 11 by editing the profile of the autonomous mobile body 11 or changing its clothes.

[0387] [[2. Modification Example]] Hereinafter, a modification example of the above-described embodiment of the present technology will be described.

[0388] In the above description, an example where two autonomous mobile bodies 11 eat food together is shown as an action that makes them become friends. However, the autonomous mobile bodies 11 may be virtually communicated with each other by other methods so that they become friends. For example, it is possible to make two autonomous mobile bodies 11 become friends by virtually performing actions other than eating food together. Specifically, for example, it is possible to make two autonomous mobile bodies 11 become friends by virtually playing together, taking a walk together, or sharing items (such as toys, etc.).

[0389] In the above description, an example where the side invited to eat food together becomes a friend of the inviting side is shown. However, for example, at the same time, the inviting side may also become a friend of the invited side.

[0390] Also, for example, it is also possible to make three or more autonomous mobile bodies 11 become friends by virtually acting together.

[0391] For example, as an action that satisfies a predetermined condition other than the condition for becoming a friend, two or more autonomous mobile bodies 11 may be made to virtually perform a predetermined action jointly.

[0392] For example, information about another device may be acquired by a method other than the information processing terminal 12 reading a QR code, and the other device may be added as a friend of the own device. For example, the other device may be added as a friend of the own device when the information processing terminal 12 reads an image of the other device.

[0393] In the above description, in the meal screen of FIG. 28, an example was shown in which the position where the friend candidate 683 (or friend) is superimposed is determined based on the food dish 702, but it may be determined based on other items. For example, the recognition unit 221 may recognize the position of the own device 701 in addition to the food dish 702 in the captured video, and the image processing unit 222 may control the position where the friend candidate 683 is superimposed based on the positions of the own device 701 and the food dish 702. Further, for example, the recognition unit 221 may further recognize the movement of the own device 701, and the image processing unit 222 may move the friend candidate 683 in accordance with the movement of the own device 701.

[0394] For example, when another autonomous mobile body 11 that is a friend approaches near the autonomous mobile body 11, the autonomous mobile body 11 may react. For example, when the autonomous mobile body 11 detects a friend within the communication range of the short-range wireless communication device provided in the communication unit 102, the autonomous mobile body 11 may perform actions such as greeting, being happy, or approaching the friend.

[0395] For example, the way of eating food may be changed between when the autonomous mobile body 11 eats food alone and when it eats food with another autonomous mobile body 11.

[0396] The division of processing between the above-described information processing terminal 12 and the information processing server 13 is an example, and can be changed as appropriate. For example, the information processing terminal 12 may hold data regarding its own profile and friends, etc., and display information regarding its own profile and friends, etc. based on the held data. For example, the information processing terminal 12 may control the autonomous mobile body 11 to perform an operation of eating food.

[0397] <<3. Others>> <Configuration Example of Computer> The above-described series of processes can be executed by hardware or by software. When the series of processes is executed by software, the program constituting the software is installed in a computer. Here, the computer includes a computer incorporated in dedicated hardware, and, for example, a general-purpose personal computer that can execute various functions by installing various programs.

[0398] FIG. 60 is a block diagram showing a configuration example of the hardware of a computer that executes the above-described series of processes by a program.

[0399] In a computer 2000, a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, and a RAM (Random Access Memory) 2003 are interconnected by a bus 2004.

[0400] Further connected to the bus 2004 is an input / output interface 2005. Connected to the input / output interface 2005 are an input unit 2006, an output unit 2007, a recording unit 2008, a communication unit 2009, and a drive 2010.

[0401] The input unit 2006 includes an input switch, buttons, a microphone, an imaging device, etc. The output unit 2007 includes a display, a speaker, etc. The recording unit 2008 includes a hard disk, a non-volatile memory, etc. The communication unit 2009 includes a network interface, etc. The drive 2010 drives a removable medium 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0402] In the computer 2000 configured as described above, the CPU 2001 loads and executes, for example, a program recorded in the recording unit 2008 via the input / output interface 2005 and the bus 2004 into the RAM 2003, whereby the above-described series of processes are performed.

[0403] The program executed by the computer 2000 (CPU 2001) can be recorded and provided, for example, on a removable medium 2011 as a package medium or the like. Further, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0404] In the computer 2000, the program can be installed in the recording unit 2008 via the input / output interface 2005 by attaching the removable medium 2011 to the drive 2010. Further, the program can be received by the communication unit 2009 via a wired or wireless transmission medium and installed in the recording unit 2008. Alternatively, the program can be installed in advance in the ROM 2002 or the recording unit 2008.

[0405] Note that the program executed by the computer may be a program in which processing is performed in time series in the order described in this specification, or may be a program in which processing is performed in parallel or at a necessary timing such as when a call is made.

[0406] In addition, in this specification, the term "system" refers to a collection of a plurality of components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device in which a plurality of modules are housed in one housing are both systems.

[0407] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

[0408] For example, the present technology can adopt a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.

[0409] In addition, each step described in the above flowchart can be executed by one device or can be shared and executed by a plurality of devices.

[0410] Furthermore, when a plurality of processes are included in one step, the plurality of processes included in that one step can be executed by one device or can be shared and executed by a plurality of devices.

[0411] <Example of configuration combination> The present technology can also adopt the following configuration.

[0412] (1) An image processing unit that generates a first video in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured video of the first autonomous mobile body, A display control unit that controls the display of the first video An information processing apparatus comprising: (2) The virtual second autonomous mobile body virtually communicates with the first autonomous mobile body by performing an action that meets a predetermined condition with the first autonomous mobile body The information processing apparatus according to (1) above. (3) The predetermined condition is a condition for the first autonomous moving body to become friends with the second autonomous moving body. The information processing device according to (2) above. (4) The behavior that serves as a condition for becoming a companion is an behavior that the virtual second autonomous moving body performs in cooperation with the first autonomous moving body. The information processing device according to (3) above. (5) The behavior that serves as a condition for becoming a companion is the behavior of the virtual second autonomous moving body eating food together with the first autonomous moving body. The information processing device according to (4) above. (6) The image processing unit superimposes the virtual food on the plate in the captured video. The information processing device according to (5) above. (7) The behavior that serves as a condition for becoming a companion is an behavior in which the virtual second autonomous moving body shares an item with the first autonomous moving body. The information processing device according to any one of (4) to (6). (8) a companion setting unit that adds the second autonomous moving body that has performed the behavior that satisfies the condition for becoming a companion to the first autonomous moving body; The information processing device according to any one of (3) to (7) further comprises: (9) The friend setting unit adds the third autonomous moving body to friends of the first autonomous moving body based on information about the third autonomous moving body acquired from another information processing device. The information processing device according to (8). (10) The friend setting unit adds the third autonomous moving body to friends of the first autonomous moving body based on an image representing information related to the third autonomous moving body displayed on the other information processing device. The information processing device according to (9) above. (11) The display control unit controls the display of a list of companions excluding the companions set to be hidden from among the companions of the first autonomous mobile body. The information processing apparatus according to any one of (3) to (10) above. (12) The image processing unit determines a position to superimpose the virtual second autonomous mobile body based on a predetermined item in the captured video. The information processing apparatus according to any one of (1) to (11) above. (13) The image processing unit superimposes a label indicating the name of the second autonomous mobile body in the vicinity of the virtual second autonomous mobile body in the captured video. The information processing apparatus according to any one of (1) to (12) above. (14) The image processing unit moves the virtual second autonomous mobile body in accordance with the movement of the first autonomous mobile body in the first video. The information processing apparatus according to any one of (1) to (13) above. (15) The image processing unit generates a second video in which the virtual first autonomous mobile body and the virtual second autonomous mobile body virtually communicate with each other, The display control unit controls the display of the second video. The information processing apparatus according to any one of (1) to (14) above. (16) The display control unit switches the display between the first video and the second video. The information processing apparatus according to (15) above. (17) The display control unit controls the display of the number of times the second autonomous mobile body has virtually communicated with the first autonomous mobile body. The information processing apparatus according to any one of (1) to (16) above. (18) An information processing apparatus generates a video in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured video of the first autonomous mobile body, and controls the display of the video. Information processing methods. (19) generating a moving image in which a virtual second autonomous moving body that virtually interacts with a first autonomous moving body is superimposed on a captured moving image of the first autonomous moving body; Control the display of the video A program that causes a computer to execute a process.

[0413] The effects described in this specification are merely examples and are not limiting, and other effects may also be present. [Explanation of symbols]

[0414] 1 Information processing system, 11-1 to 11-n Autonomous mobile body, 12-1 to 12-n Information processing terminal, 13 Information processing server, 201 Input unit, 203 Information processing unit, 204 Output unit, 211 Camera, 221 Recognition unit, 222 Image processing unit, 223 UI control unit, 224 Partner setting unit, 231 Display control unit, 241 Display unit, 302 Information processing unit, 321 Autonomous mobile body control unit, 322 Application control unit, 331 Recognition unit, 332 Learning unit, 333 Action planning unit, 334 Operation control unit

Claims

1. An image processing unit that generates a first video in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured video of the first autonomous mobile body, A display control unit that controls the display of the first video Comprising The virtual second autonomous mobile body is an action that satisfies the condition for the first autonomous mobile body to become a partner with the second autonomous mobile body, and virtually communicates with the first autonomous mobile body by performing an action that is performed jointly with the first autonomous mobile body An information processing apparatus.

2. The action that satisfies the condition for becoming a partner is an action in which the virtual second autonomous mobile body eats food together with the first autonomous mobile body The information processing apparatus according to claim 1.

3. The image processing unit superimposes the virtual food on a plate in the captured video The information processing apparatus according to claim 2.

4. The action that satisfies the condition for becoming a partner is an action in which the virtual second autonomous mobile body shares an item with the first autonomous mobile body The information processing apparatus according to claim 1.

5. A friend setting unit that adds the second autonomous mobile body that has performed the action that satisfies the condition for becoming a friend to the friends of the first autonomous mobile body The information processing apparatus according to claim 1, further comprising.

6. The friend setting unit adds the third autonomous mobile body to the friends of the first autonomous mobile body based on information about the third autonomous mobile body acquired from another information processing apparatus The information processing apparatus according to claim 5.

7. The friend setting unit adds the third autonomous mobile body to the friends of the first autonomous mobile body based on an image representing information about the third autonomous mobile body displayed on the other information processing apparatus The information processing apparatus according to claim 6.

8. The display control unit controls the display of a list of friends excluding the friends that are set to be non-displayed from the friends of the first autonomous mobile body The information processing apparatus according to claim 1.

9. The image processing unit determines the position to superimpose the virtual second autonomous mobile body based on a predetermined item in the captured video The information processing apparatus according to claim 1.

10. The image processing unit superimposes a label indicating the name of the virtual second autonomous mobile body near the virtual second autonomous mobile body in the captured video The information processing apparatus according to claim 1.

11. The image processing unit moves the virtual second autonomous mobile body in accordance with the movement of the first autonomous mobile body in the first video The information processing apparatus according to claim 1.

12. The image processing unit generates a second moving image in which the virtual first autonomous mobile body and the virtual second autonomous mobile body virtually communicate with each other. The display control unit controls the display of the second moving image. The information processing apparatus according to claim 1.

13. The display control unit switches the display between the first moving image and the second moving image. The information processing apparatus according to claim 12.

14. The display control unit controls the display of the number of times the second autonomous mobile body has virtually communicated with the first autonomous mobile body. The information processing apparatus according to claim 1.

15. An information processing apparatus generates a moving image in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured moving image of the first autonomous mobile body, controls the display of the moving image, and the virtual second autonomous mobile body performs an action that is a condition for the first autonomous mobile body to become a companion with the second autonomous mobile body, and virtually communicates with the first autonomous mobile body by performing an action jointly with the first autonomous mobile body. An information processing method.

16. Generate a moving image in which a virtual second autonomous mobile body that virtually communicates with the first autonomous mobile body is superimposed on a captured moving image of the first autonomous mobile body, Control the display of the moving image, and the virtual second autonomous mobile body performs an action that is a condition for the first autonomous mobile body to become a companion with the second autonomous mobile body, and virtually communicates with the first autonomous mobile body by performing an action jointly with the first autonomous mobile body. A program for causing a computer to execute the processing.

Images