Autonomous mobile vehicle, information processing device, information processing method, and program

Abstract

The present invention pertains to an autonomous moving body, an information processing device, an information processing method, and a program, configured so as to make it possible to cause the autonomous moving body to perform a desired behavior quickly and reliably. The autonomous moving body comprises: a recognition unit that recognizes a marker; a behavior plan unit that plans behavior of the autonomous moving body in relation to the marker; and an action control unit that controls the action of the autonomous moving body so as to perform the planned behavior. The present invention is applicable to, for example, an autonomous mobile robot having a shape or action capability simulating an animal.

Description

Technical Field

[0001] The present technology relates to an autonomous mobile body, an information processing apparatus, an information processing method, and a program, and more particularly to an autonomous mobile body, an information processing apparatus, an information processing method, and a program that can cause an autonomous mobile body to execute a desired action quickly or surely.

Background Art

[0002] Conventionally, it has been proposed to cause an autonomous mobile body to perform learning related to pattern recognition to increase recognizable objects and diversify actions (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the invention described in Patent Document 1, it takes a certain amount of time until the autonomous mobile body executes a desired action. In addition, the training of the user may fail, and the autonomous mobile body may not act as the user desires.

[0005] The present technology has been made in view of such a situation, and enables an autonomous mobile body to execute a desired action quickly or surely.

Means for Solving the Problems

[0006] The autonomous mobile body according to the first aspect of the present technology is an autonomous mobile body that operates autonomously, and includes a recognition unit that recognizes a marker, an action planning unit that plans an action of the autonomous mobile body with respect to the recognized marker, and an operation control unit that controls the operation of the autonomous mobile body so as to perform the planned action.

[0007] In the first aspect of this technology, a marker is recognized, an action of the autonomous mobile body in relation to the recognized marker is planned, and the operation of the autonomous mobile body is controlled to perform the planned action.

[0008] The second aspect of this technology is an autonomous mobile unit comprising a recognition unit that recognizes a marker and an action planning unit that plans the actions of the autonomous mobile unit in relation to the recognized marker.

[0009] The second aspect of this technology involves an information processing method that recognizes a marker and plans the actions of an autonomous mobile unit in relation to the recognized marker.

[0010] The second aspect of this technology involves a program that causes a computer to perform a process of recognizing markers and planning the actions of an autonomous mobile unit in relation to the recognized markers.

[0011] In the second aspect of this technology, markers are recognized, and the actions of the autonomous mobile entity in relation to the recognized markers are planned. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing one embodiment of an information processing system to which this technology is applied. [Figure 2] This figure shows an example of the hardware configuration of an autonomous mobile device. [Figure 3] This is an example of the configuration of actuators used in an autonomous mobile vehicle. [Figure 4] This is a diagram illustrating the functions of the display on an autonomous mobile device. [Figure 5] This figure shows an example of the operation of an autonomous mobile device. [Figure 6] This is a block diagram showing an example of the functional configuration of an autonomous mobile device. [Figure 7] This is a block diagram showing an example of the functional configuration of an information processing terminal. [Figure 8] This is a block diagram showing an example of the functional configuration of an information processing server. [Figure 9] It is a flowchart for explaining marker corresponding processing. [Figure 10] It is a flowchart for explaining the details of individual value setting processing. [Figure 11] It is a diagram for explaining a calculation example of individual values. [Figure 12] It is a diagram showing an installation example of a proximity prohibition marker. [Figure 13] It is a diagram showing an installation example of a proximity prohibition marker. [Figure 14] It is a diagram showing an installation example of a proximity prohibition marker. [Figure 15] It is a diagram showing a configuration example of a computer.

Mode for Carrying Out the Invention

[0013] Hereinafter, the mode for carrying out the present technology will be described. The description will be made in the following order. 1. Embodiment 2. Variation 3. Others

[0014] <<1. Embodiment>> Referring to FIGS. 1 to 14, the embodiment of the present technology will be described.

[0015] <Configuration Example of Information Processing System 1> FIG. 1 is a block diagram showing an embodiment of an information processing system 1 to which the present technology is applied.

[0016] The information processing system 1 includes autonomous mobile bodies 11-1 to 11-n, information processing terminals 12-1 to 12-n, and an information processing server 13.

[0017] In the following, when it is not necessary to individually distinguish the autonomous mobile bodies 11-1 to 11-n, they are simply referred to as autonomous mobile body 11. In the following, when it is not necessary to individually distinguish the information processing terminals 12-1 to 12-n, they are simply referred to as information processing terminal 12.

[0018] Communication is possible via the network 21 between each autonomous mobile unit 11 and the information processing server 13, between each information processing terminal 12 and the information processing server 13, between each autonomous mobile unit 11 and each information processing terminal 12, between each autonomous mobile unit 11, and between each information processing terminal 12. Furthermore, direct communication is also possible between each autonomous mobile unit 11 and each information processing terminal 12, between each autonomous mobile unit 11, and between each information processing terminal 12 without using the network 21.

[0019] The autonomous mobile unit 11 is an information processing device that recognizes its own and its surroundings based on collected sensor data, and autonomously selects and executes various actions according to the situation. Unlike robots that simply perform actions according to user instructions, one of the features of the autonomous mobile unit 11 is that it autonomously performs appropriate actions according to the situation.

[0020] The autonomous mobile device 11 can, for example, perform user recognition or object recognition based on captured images, and perform various autonomous actions according to the recognized user or object. Furthermore, the autonomous mobile device 11 can, for example, perform speech recognition based on user utterances and take actions based on user instructions.

[0021] Furthermore, the autonomous mobile device 11 performs pattern recognition learning in order to acquire the ability to recognize users and objects. In this process, the autonomous mobile device 11 can not only perform supervised learning based on given learning data, but also dynamically collect learning data based on instructions from users, etc., and perform pattern recognition learning related to objects, etc.

[0022] Furthermore, the autonomous mobile device 11 can be trained by the user. Here, training the autonomous mobile device 11 is broader than general training, such as teaching and having it memorize rules and prohibitions, and refers to the changes that the user can perceive in the autonomous mobile device 11 as a result of the user's interaction with the autonomous mobile device 11.

[0023] The shape, capabilities, and needs of the autonomous mobile unit 11 can be appropriately designed according to its purpose and role. For example, the autonomous mobile unit 11 may consist of an autonomous mobile robot that moves autonomously within space and performs various actions. Specifically, for example, the autonomous mobile unit 11 may consist of an autonomous mobile robot that has a shape and movement capabilities that mimic a human or an animal such as a dog. Alternatively, for example, the autonomous mobile unit 11 may consist of a vehicle or other device that has the ability to communicate with a user.

[0024] The information processing terminal 12 consists of, for example, a smartphone, tablet, or PC (personal computer), and is used by the user of the autonomous mobile device 11. The information processing terminal 12 performs various functions by executing a predetermined application program (hereinafter simply referred to as "application"). For example, the information processing terminal 12 communicates with the information processing server 13 via the network 21 or communicates directly with the autonomous mobile device 11 to collect various data related to the autonomous mobile device 11, present it to the user, or give instructions to the autonomous mobile device 11.

[0025] The information processing server 13, for example, collects various data from each autonomous mobile unit 11 and each information processing terminal 12, provides various data to each autonomous mobile unit 11 and each information processing terminal 12, and controls the operation of each autonomous mobile unit 11. Furthermore, for example, based on the data collected from each autonomous mobile unit 11 and each information processing terminal 12, the information processing server 13 can perform pattern recognition learning and processing corresponding to user training, similar to the autonomous mobile units 11. In addition, for example, the information processing server 13 supplies various data related to the aforementioned applications and each autonomous mobile unit 11 to each information processing terminal 12.

[0026] Network 21 consists of several public network networks, such as the Internet, telephone networks, and satellite communication networks, as well as various LANs (Local Area Networks) and WANs (Wide Area Networks), including Ethernet®. Network 21 may also include dedicated network networks such as IP-VPN (Internet Protocol-Virtual Private Network). Network 21 may also include wireless communication networks such as Wi-Fi® and Bluetooth®.

[0027] The configuration of the information processing system 1 can be flexibly changed depending on the specifications and operation. For example, the autonomous mobile unit 11 may communicate with various external devices in addition to the information processing terminal 12 and the information processing server 13. These external devices may include, for example, servers that transmit weather, news, and other service information, as well as various home appliances owned by the user.

[0028] Furthermore, for example, the relationship between the autonomous mobile unit 11 and the information processing terminal 12 does not necessarily have to be one-to-one; for example, it may be many-to-many, many-to-one, or one-to-many. For example, one user can use one information processing terminal 12 to check data related to multiple autonomous mobile units 11, or use multiple information processing terminals to check data related to one autonomous mobile unit 11.

[0029] <Example hardware configuration of autonomous mobile unit 11> Next, we will describe an example of the hardware configuration of the autonomous mobile unit 11. In the following explanation, we will use the case where the autonomous mobile unit 11 is a dog-type quadruped robot as an example.

[0030] Figure 2 shows an example of the hardware configuration of the autonomous mobile robot 11. The autonomous mobile robot 11 is a dog-type quadrupedal robot equipped with a head, torso, four legs, and a tail.

[0031] The autonomous mobile unit 11 is equipped with two displays 51L and 51R on its head. Hereafter, when it is not necessary to distinguish between displays 51L and 51R individually, they will simply be referred to as "display 51".

[0032] Furthermore, the autonomous mobile device 11 is equipped with various sensors. For example, the autonomous mobile device 11 includes a microphone 52, a camera 53, a ToF (Time Of Flight) sensor 525, a human presence sensor 55, a distance measuring sensor 56, a touch sensor 57, an illuminance sensor 58, a foot button 59, and an inertial sensor 60.

[0033] The autonomous mobile device 11 is equipped with, for example, four microphones 52 on its head. Each microphone 52 collects ambient sounds, such as the user's speech and surrounding environmental sounds. By having multiple microphones 52, it is possible to collect sounds occurring in the surroundings with high sensitivity and to localize sound sources.

[0034] The autonomous mobile robot 11 is equipped with two wide-angle cameras 53, for example, on its nose and waist, to capture images of its surroundings. For example, the camera 53 positioned on its nose captures images within the autonomous mobile robot 11's forward field of view (i.e., the dog's field of view). The camera 53 positioned on its waist captures images of the area around the autonomous mobile robot 11, primarily above it. Based on the images captured by the camera 53 positioned on its waist, the autonomous mobile robot 11 can extract feature points from the ceiling and other elements to achieve SLAM (Simultaneous Localization and Mapping).

[0035] The ToF sensor 54 is, for example, located at the tip of the nose and detects the distance to an object located in front of the head. The autonomous mobile body 11 can accurately detect the distance to various objects using the ToF sensor 54, and can perform actions according to its relative position to objects such as the user and obstacles.

[0036] The human presence sensor 55 is positioned, for example, on the chest to detect the location of the user or a pet owned by the user. The autonomous mobile device 11 can detect an animal in front of it using the human presence sensor 55, and can perform various actions toward that animal, such as actions corresponding to emotions like interest, fear, or surprise.

[0037] The distance measuring sensor 56 is positioned, for example, on the chest, and detects the condition of the floor surface in front of the autonomous mobile body 11. The autonomous mobile body 11 can accurately detect the distance to an object on the floor surface in front of it using the distance measuring sensor 56, and can perform actions according to its relative position to the object.

[0038] The touch sensors 57 are positioned in areas where the user is likely to touch the autonomous mobile device 11, such as the top of the head, under the chin, or on the back, to detect user contact. The touch sensors 57 are composed of, for example, capacitive or pressure-sensitive touch sensors. The autonomous mobile device 11 can detect user contact actions such as touching, stroking, tapping, or pressing using the touch sensors 57, and can perform actions corresponding to those contact actions.

[0039] The illuminance sensor 58 is positioned, for example, on the back of the head, at the base of the tail, and detects the illuminance of the space in which the autonomous mobile body 11 is located. The autonomous mobile body 11 can detect the ambient brightness using the illuminance sensor 58 and perform actions according to that brightness.

[0040] The foot buttons 59 are positioned, for example, on the parts corresponding to the paw pads of each of the four legs, and detect whether or not the bottom surface of the legs of the autonomous mobile unit 11 is in contact with the floor. The autonomous mobile unit 11 can detect contact or non-contact with the floor surface using the foot buttons 59, and can, for example, understand that it has been picked up by a user.

[0041] The inertial sensors 60 are, for example, positioned in the head and torso, respectively, and detect physical quantities such as velocity, acceleration, and rotation of the head and torso. For example, the inertial sensors 60 are composed of a 6-axis sensor that detects acceleration and angular velocity in the X, Y, and Z axes. The autonomous mobile body 11 can accurately detect the movement of its head and torso using the inertial sensors 60, enabling situation-appropriate motion control.

[0042] The configuration of sensors equipped in the autonomous mobile unit 11 can be flexibly changed depending on the specifications and operation. For example, in addition to the above configuration, the autonomous mobile unit 11 may be further equipped with various communication devices, such as a temperature sensor, a geomagnetic sensor, and a GNSS (Global Navigation Satellite System) signal receiver.

[0043] Next, with reference to Figure 3, an example of the configuration of the joints of the autonomous mobile body 11 will be described. Figure 3 shows an example of the configuration of the actuators 71 provided by the autonomous mobile body 11. In addition to the rotation points shown in Figure 3, the autonomous mobile body 11 has a total of 22 degrees of freedom of rotation: two each in the ears and tail, and one in the mouth.

[0044] For example, the autonomous mobile robot 11 has three degrees of freedom in its head, enabling it to perform actions such as nodding and tilting its head. Furthermore, the autonomous mobile robot 11 can replicate the swinging motion of its waist using an actuator 71 located in its waist, thereby achieving more natural and flexible movements that are closer to those of a real dog.

[0045] The autonomous mobile body 11 may achieve the 22 rotational degrees of freedom described above by, for example, combining a single-axis actuator and a two-axis actuator. For example, single-axis actuators may be used in the elbow and knee areas of the legs, and two-axis actuators may be used in the shoulders and thigh joints.

[0046] Next, referring to Figure 4, the functions of the display 51 provided by the autonomous mobile unit 11 will be explained.

[0047] The autonomous mobile robot 11 is equipped with two displays 51R and 51L, corresponding to the right and left eyes, respectively. Each display 51 has the function of visually representing the eye movements and emotions of the autonomous mobile robot 11. For example, each display 51 can represent the movements of the eyeballs, pupils, and eyelids in accordance with emotions and actions, thereby creating natural movements similar to those of real animals such as dogs, and expressing the gaze and emotions of the autonomous mobile robot 11 with high precision and flexibility. In addition, users can intuitively grasp the state of the autonomous mobile robot 11 from the eye movements displayed on the displays 51.

[0048] Furthermore, each display 51 is realized, for example, by two independent OLEDs (Organic Light Emitting Diodes). By using OLEDs, it becomes possible to reproduce the curved surface of the eyeball. As a result, a more natural appearance can be achieved compared to representing a pair of eyeballs with a single flat display or representing two eyeballs with two independent flat displays.

[0049] With the above configuration, the autonomous mobile unit 11 can reproduce movements and emotional expressions that are closer to those of real living organisms by precisely and flexibly controlling the movements of its joints and eyeballs, as shown in Figure 5.

[0050] Figure 5 shows an example of the operation of the autonomous mobile unit 11. In Figure 5, the external structure of the autonomous mobile unit 11 is simplified in order to focus on the operation of the joints and eyeballs of the autonomous mobile unit 11.

[0051] <Example of functional configuration of autonomous mobile unit 11> Next, with reference to Figure 6, an example of the functional configuration of the autonomous mobile unit 11 will be described. The autonomous mobile unit 11 comprises an input unit 101, a communication unit 102, an information processing unit 103, a drive unit 104, an output unit 105, and a storage unit 106.

[0052] The input unit 101 is equipped with various sensors as shown in Figure 2 and has the function of collecting various sensor data related to the user and the surrounding environment. The input unit 101 also includes input devices such as switches and buttons. The input unit 101 supplies the collected sensor data and input data input via the input devices to the information processing unit 103.

[0053] The communication unit 102 communicates with other autonomous mobile units 11, information processing terminals 12, and information processing servers 13 via or without the network 21, and sends and receives various types of data. The communication unit 102 supplies received data to the information processing unit 103 and obtains data to be transmitted from the information processing unit 103.

[0054] Furthermore, the communication method of the communication unit 102 is not particularly limited and can be flexibly changed according to the specifications and operation.

[0055] The information processing unit 103 includes, for example, a processor such as a CPU (Central Processing Unit), and performs various information processing and controls the various parts of the autonomous mobile unit 11. The information processing unit 103 includes a recognition unit 121, a learning unit 122, an action planning unit 123, and an operation control unit 124.

[0056] The recognition unit 121 recognizes the situation in which the autonomous mobile unit 11 is located, based on sensor data and input data supplied from the input unit 101, and received data supplied from the communication unit 102. The situation in which the autonomous mobile unit 11 is located includes, for example, the situation of itself and its surroundings. The situation of itself includes, for example, the state and movement of the autonomous mobile unit 11. The surrounding situation includes, for example, the state, movement and instructions of people in the surroundings such as users, the state and movement of living things in the surroundings such as pets, the state and movement of objects in the surroundings, time, place, and the surrounding environment. Objects in the surroundings include, for example, other autonomous mobile units. In addition, the recognition unit 121 performs, for example, person identification, facial expression and gaze recognition, emotion recognition, object recognition, motion recognition, spatial area recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, temperature recognition, speech recognition, word comprehension, position estimation, posture estimation, etc., in order to recognize the situation.

[0057] For example, the recognition unit 121 performs marker recognition, which recognizes markers placed in real space, as will be described later.

[0058] Here, a marker is a component that represents a predetermined two-dimensional or three-dimensional pattern. The marker pattern can be represented, for example, by an image, letters, design, color, or shape, or a combination of two or more of these. The marker design can also be represented by a code such as a QR code (registered trademark), a symbol, a mark, etc.

[0059] For example, a sheet-like component with a predetermined image or pattern can be used as a marker. For example, a component with a predetermined two-dimensional shape (e.g., a star) or three-dimensional shape (e.g., a sphere) can be used as a marker.

[0060] Furthermore, marker types are distinguished by differences in patterns. For example, marker types are distinguished by differences in the patterns attached to them. For example, marker types are distinguished by differences in the shape of the markers. For example, marker types are distinguished by differences in the color of the markers.

[0061] Furthermore, the pattern does not necessarily need to be displayed on the entire marker; it is sufficient if the pattern is displayed on at least a portion of the marker. For example, it is sufficient if a predetermined pattern is applied to only a portion of the marker. For example, it is sufficient if a portion of the marker has a predetermined shape.

[0062] Furthermore, the recognition unit 121 has the function of estimating and understanding the situation based on the various pieces of information it has recognized. In this case, the recognition unit 121 may comprehensively estimate the situation using knowledge that has been stored in advance.

[0063] The recognition unit 121 supplies data indicating the recognition or estimation result of the situation (hereinafter referred to as situation data) to the learning unit 122 and the action planning unit 123. The recognition unit 121 also registers the data indicating the recognition or estimation result of the situation into the action history data stored in the storage unit 106.

[0064] The behavioral history data is data that shows the history of the actions of the autonomous mobile unit 11. The behavioral history data includes items such as the date and time when the action started, the date and time when the action ended, the trigger for performing the action, the location where the action was instructed (if a location was specified), the circumstances when the action was performed, and whether or not the action was completed (whether the action was performed to the end).

[0065] The triggers for an action include, for example, if the action is triggered by a user instruction, the content of that instruction is registered. Also, for example, if the action is triggered by a predetermined situation, the details of that situation are registered. Furthermore, for example, if the action is triggered by an object indicated by the user or an object recognized, the type of that object is registered. This includes the markers mentioned above.

[0066] The learning unit 122 learns the situation and actions, and the effects of those actions on the environment, based on sensor data and input data supplied from the input unit 101, received data supplied from the communication unit 102, situation data supplied from the recognition unit 121, data on the actions of the autonomous mobile unit 11 supplied from the action planning unit 123, and action history data stored in the memory unit 106. For example, the learning unit 122 performs the pattern recognition learning described above, or learns behavior patterns that correspond to user training.

[0067] For example, the learning unit 122 achieves the above learning using machine learning algorithms such as deep learning. Note that the learning algorithm adopted by the learning unit 122 is not limited to the above example and can be designed as appropriate.

[0068] The learning unit 122 supplies data indicating the learning results (hereinafter referred to as learning result data) to the action planning unit 123 or stores it in the storage unit 106.

[0069] The action planning unit 123 plans the actions that the autonomous mobile unit 11 will take based on the recognized or estimated situation and the learning result data. The action planning unit 123 supplies data indicating the planned actions (hereinafter referred to as action planning data) to the motion control unit 124. The action planning unit 123 also supplies data related to the actions of the autonomous mobile unit 11 to the learning unit 122 and registers it in the action history data stored in the memory unit 106.

[0070] The motion control unit 124 controls the movement of the autonomous mobile unit 11 to execute the planned action by controlling the drive unit 104 and the output unit 105 based on the action plan data. For example, the motion control unit 124 controls the rotation of the actuator 71, the display of the display 51, and the sound output of the speaker based on the action plan.

[0071] The drive unit 104 flexes and extends the multiple joints of the autonomous mobile body 11 based on the control of the motion control unit 124. More specifically, the drive unit 104 drives the actuators 71 provided in each joint based on the control of the motion control unit 124.

[0072] The output unit 105 includes, for example, a display 51, a speaker, a haptic device, etc., and outputs visual information, auditory information, tactile information, etc., based on control by the motion control unit 124.

[0073] The memory unit 106 includes, for example, non-volatile memory and volatile memory, and stores various programs and data.

[0074] In the following, when each part of the autonomous mobile unit 11 communicates with the information processing server 13, etc., via the communication unit 102 and the network 21, the phrase "via the communication unit 102 and the network 21" will be omitted as appropriate. For example, when the recognition unit 121 communicates with the information processing server 13 via the communication unit 102 and the network 21, it will simply be stated that the recognition unit 121 communicates with the information processing server 13.

[0075] <Example of the functional configuration of information processing terminal 12> Next, with reference to Figure 7, an example of the functional configuration of the information processing terminal 12 will be described. The information processing terminal 12 comprises an input unit 201, a communication unit 202, an information processing unit 203, an output unit 204, and a storage unit 205.

[0076] The input unit 201 includes various sensors such as a camera (not shown), a microphone (not shown), and an inertial sensor (not shown). The input unit 201 also includes input devices such as a switch (not shown) and a button (not shown). The input unit 201 supplies input data received via the input devices and sensor data output from the various sensors to the information processing unit 203.

[0077] The communication unit 202 communicates with the autonomous mobile unit 11, other information processing terminals 12, and information processing server 13 via or without the network 21, and sends and receives various types of data. The communication unit 202 supplies the received data to the information processing unit 203 and obtains data to be transmitted from the information processing unit 203.

[0078] Furthermore, the communication method of the communication unit 202 is not particularly limited and can be flexibly changed according to the specifications and operation.

[0079] The information processing unit 203, for example, is equipped with a processor such as a CPU, and performs various information processing and controls the various parts of the information processing terminal 12.

[0080] The output unit 204 includes, for example, a display (not shown), a speaker (not shown), a haptic device (not shown), etc., and outputs visual information, auditory information, tactile information, etc., based on control by the information processing unit 203.

[0081] The memory unit 205 includes, for example, non-volatile memory and volatile memory, and stores various programs and data.

[0082] Furthermore, the functional configuration of the information processing terminal 12 can be flexibly changed according to specifications and operation.

[0083] Furthermore, the phrase "via the communication unit 202 and network 21" will be omitted as appropriate when each part of the information processing terminal 12 communicates with the information processing server 13, etc., via the communication unit 202 and network 21. For example, when the information processing unit 203 communicates with the information processing server 13 via the communication unit 202 and network 21, it will simply be stated that the information processing unit 203 communicates with the information processing server 13.

[0084] <Example of the functional configuration of information processing server 13> Next, with reference to Figure 8, an example of the functional configuration of the information processing server 13 will be described. The information processing server 13 comprises a communication unit 301, an information processing unit 302, and a storage unit 303.

[0085] The communication unit 301 communicates with each autonomous mobile unit 11 and each information processing terminal 12 via the network 21, and sends and receives various types of data. The communication unit 301 supplies the received data to the information processing unit 302 and obtains the data to be transmitted from the information processing unit 302.

[0086] Furthermore, the communication method of the communication unit 301 is not particularly limited and can be flexibly changed according to the specifications and operation.

[0087] The information processing unit 302 includes, for example, a processor such as a CPU, and performs various information processing and controls the various parts of the information processing terminal 12. The information processing unit 302 also includes an autonomous mobile unit control unit 321 and an application control unit 322.

[0088] The autonomous mobile unit control unit 321 has the same configuration as the information processing unit 103 of the autonomous mobile unit 11. Specifically, the autonomous mobile unit control unit 321 includes a recognition unit 331, a learning unit 332, an action planning unit 333, and an operation control unit 334.

[0089] Furthermore, the autonomous mobile unit control unit 321 has the same functions as the information processing unit 103 of the autonomous mobile unit 11. For example, the autonomous mobile unit control unit 321 receives sensor data, input data, behavior history data, etc. from the autonomous mobile unit 11 and recognizes the autonomous mobile unit 11 and its surroundings. For example, the autonomous mobile unit control unit 321 controls the operation of the autonomous mobile unit 11 by generating control data that controls the operation of the autonomous mobile unit 11 based on the autonomous mobile unit 11 and transmitting it to the autonomous mobile unit 11. For example, the autonomous mobile unit control unit 321 performs pattern recognition learning and learning behavior patterns corresponding to user training, similar to the autonomous mobile unit 11.

[0090] Furthermore, the learning unit 332 of the autonomous mobile unit control unit 321 can also learn collective intelligence common to multiple autonomous mobile units 11 by performing pattern recognition learning and learning behavioral patterns corresponding to user training based on data collected from multiple autonomous mobile units 11.

[0091] The application control unit 322 communicates with the autonomous mobile unit 11 and the information processing terminal 12 via the communication unit 301 and controls the application executed by the information processing terminal 12.

[0092] For example, the application control unit 322 collects various data related to the autonomous mobile device 11 from the autonomous mobile device 11 via the communication unit 301. The application control unit 322 then transmits the collected data to the information processing terminal 12 via the communication unit 301, causing the data related to the autonomous mobile device 11 to be displayed in the application executed by the information processing terminal 12.

[0093] For example, the application control unit 322 receives data from the information processing terminal 12 via the communication unit 301, which indicates instructions for the autonomous mobile device 11 that are input via the application. The application control unit 322 then transmits the received data to the autonomous mobile device 11 via the communication unit 301, thereby giving the autonomous mobile device 11 instructions from the user.

[0094] The memory unit 303 includes, for example, non-volatile memory and volatile memory, and stores various programs and data.

[0095] Furthermore, the functional configuration of the information processing server 13 can be flexibly changed according to specifications and operation.

[0096] Furthermore, the phrase "via the communication unit 301 and network 21" will be omitted as appropriate when each part of the information processing server 13 communicates with the information processing terminal 12, etc., via the communication unit 301 and network 21. For example, when the application control unit 322 communicates with the information processing terminal 12 via the communication unit 301 and network 21, it will simply be stated that the application control unit 322 communicates with the information processing terminal 12.

[0097] <Marker handling processing> Next, the marker response process performed by the autonomous mobile unit 11 will be explained with reference to the flowchart in Figure 9.

[0098] The following explains the use of three types of markers: markers for prohibiting approach, markers for restrooms, and markers for favorite locations.

[0099] The no-approach marker is a marker used to prohibit the autonomous mobile unit 11 from approaching. For example, the autonomous mobile unit 11 recognizes a predetermined area based on the no-approach marker as a no-entry zone and acts to avoid entering the no-entry zone. The no-entry zone is set, for example, within a predetermined radius centered on the no-approach marker.

[0100] A toilet marker is a marker used to specify the location of a toilet. For example, the autonomous mobile device 11 recognizes a predetermined area based on the toilet marker as the toilet area and acts to simulate the act of defecation within the toilet area. Alternatively, for example, a user can use the toilet marker to train the autonomous mobile device 11 to perform actions simulating the act of defecation within the toilet area. The toilet area is set, for example, within a predetermined radius centered on the toilet marker.

[0101] The favorite location marker is a marker used to designate a favorite location for the autonomous mobile unit 11. For example, the autonomous mobile unit 11 recognizes a predetermined area based on the favorite location marker as its favorite area and performs predetermined actions within that area. For example, within its favorite area, the autonomous mobile unit 11 may perform actions that express positive emotions such as joy, happiness, and peace, such as dancing, singing, collecting its favorite toys, or sleeping. The favorite area is set, for example, within a predetermined radius centered on the favorite location marker.

[0102] This process starts, for example, when the power to the autonomous mobile device 11 is turned on, and ends when the power is turned off.

[0103] In step S1, the autonomous mobile unit 11 performs individual value setting processing.

[0104] Here, we will explain the details of the individual value setting process by referring to the flowchart in Figure 10.

[0105] In step S51, the recognition unit 121 recognizes the usage status of the autonomous mobile unit 11 based on the behavioral history data stored in the memory unit 106.

[0106] For example, as shown in Figure 11, the recognition unit 121 recognizes the autonomous mobile device 11's birthday, operating days, the person who often plays with it, and the toys it often plays with as part of its usage status. The autonomous mobile device 11's birthday is set, for example, to the day it was first powered on after purchase. The autonomous mobile device 11's operating days are set to the number of days it has been powered on and operating within the period from its birthday to the present.

[0107] The recognition unit 121 supplies data indicating the usage status of the autonomous mobile unit 11 to the learning unit 122 and the action planning unit 123.

[0108] In step S52, the recognition unit 121 recognizes the current situation based on the sensor data and input data supplied from the input unit 101, and the received data supplied from the communication unit 102.

[0109] For example, as shown in Figure 11, the recognition unit 121 recognizes the current date and time, the presence or absence of toys around the autonomous mobile device 11, the presence or absence of people around the autonomous mobile device 11, and the content of the user's speech as the current situation.

[0110] The recognition unit 121 supplies data indicating the current situation to the learning unit 122 and the action planning unit 123.

[0111] In step S53, the recognition unit 121 recognizes the usage status of other units. Here, "other units" refers to other autonomous mobile units 11.

[0112] Specifically, the recognition unit 121 receives data from the information processing server 13 indicating the usage status of other autonomous mobile units 11. Based on the received data, the recognition unit 121 recognizes the usage status of the other autonomous mobile units 11. For example, the recognition unit 121 recognizes the number of people that each of the other autonomous mobile units 11 has come into contact with so far.

[0113] The recognition unit 121 supplies data indicating the usage status of other autonomous mobile units 11 to the learning unit 122 and the action planning unit 123.

[0114] In step S54, the learning unit 122 and the action planning unit 123 set individual values ​​based on the usage status of the autonomous mobile unit 11, its current status, and the usage status of other units. Here, individual values ​​are values ​​that indicate the current status of the autonomous mobile unit 11 based on various perspectives.

[0115] For example, as shown in Figure 11, the learning unit 122 sets the personality, growth stage, favorite person, favorite toy, and marker preference of the autonomous mobile unit 11 based on the usage status of the autonomous mobile unit 11 and other individuals.

[0116] The personality of the autonomous mobile unit 11 is set, for example, based on the relative relationship between the usage status of the autonomous mobile unit 11 and the usage status of other units. For example, if the number of people that the autonomous mobile unit 11 has interacted with so far is greater than the average number of people that other units have interacted with, the autonomous mobile unit 11 will be set to have a shy personality.

[0117] The growth level of the autonomous mobile unit 11 is set based on, for example, the unit's birthday and operating days. For example, the growth level is set to a higher value the older the autonomous mobile unit 11 is or the more operating days it has.

[0118] Marker preference indicates the autonomous mobile unit 11's preference for its favorite location markers. Marker preference is set based on, for example, the autonomous mobile unit 11's personality and growth level. For example, the higher the autonomous mobile unit 11's growth level, the higher the marker preference value will be set. The rate at which marker preference increases also changes depending on the autonomous mobile unit 11's personality. For example, if the autonomous mobile unit 11's personality is shy, the rate at which marker preference increases will be slower. On the other hand, if the autonomous mobile unit 11's personality is wild, for example, the rate at which marker preference increases will be faster.

[0119] For example, the autonomous mobile unit 11's favorite person can be set to someone who often plays with it.

[0120] The favorite toys of the autonomous mobile robot 11 are set based on, for example, the usage status of other units and the toys that the autonomous mobile robot 11 frequently plays with. For example, for toys that the autonomous mobile robot 11 frequently plays with, the degree of preference of the autonomous mobile robot 11 for that toy is set based on the average number of times the autonomous mobile robot 11 has played with it and the average number of times other units have played with it. For example, the more times the autonomous mobile robot 11 has played with a toy compared to the average number of times other units have played with it, the higher the degree of preference for that toy will be set. For example, the less times the autonomous mobile robot 11 has played with a toy compared to the average number of times other units have played with it, the lower the degree of preference for that toy will be set.

[0121] The learning unit 122 supplies the behavior planning unit 123 with data indicating the personality, growth stage, favorite people, favorite toys, and marker preference of the autonomous mobile unit 11.

[0122] Furthermore, for example, the action planning unit 123 sets the emotions and desires of the autonomous mobile unit 11 based on the current situation, as shown in Figure 11.

[0123] Specifically, the action planning unit 123 sets the emotions of the autonomous mobile unit 11 based on, for example, the presence or absence of people in the surroundings and the content of the user's speech. For example, emotions such as joy, interest, anger, fear, surprise, and sadness are set.

[0124] For example, the action planning unit 123 sets the needs of the autonomous mobile robot 11 based on the current date and time, the presence or absence of toys in the surroundings, the presence or absence of people in the surroundings, and the emotions of the autonomous mobile robot 11. The needs of the autonomous mobile robot 11 include, for example, the need for companionship, the need for play, the need for exercise, the need for emotional expression, the need for excretion, and the need for sleep.

[0125] The desire for closeness represents the autonomous mobile device 11's desire to be close to those around it. For example, the action planning unit 123 sets a degree of the desire for closeness, indicating the level of the desire for closeness, based on the time of day, the presence or absence of people in the surroundings, and the emotions of the autonomous mobile device 11. For example, when the degree of the desire for closeness reaches a predetermined threshold, the autonomous mobile device 11 performs actions to be close to those around it.

[0126] The desire to play represents the autonomous mobile unit 11's desire to play with toys or other objects. For example, the action planning unit 123 sets a degree of desire to play based on the time of day, the presence or absence of toys in the surroundings, and the emotions of the autonomous mobile unit 11. For example, when the desire to play reaches a predetermined threshold, the autonomous mobile unit 11 performs an action to play with toys or other objects in its surroundings.

[0127] The desire for movement represents the autonomous mobile device 11's desire to move its body. For example, the action planning unit 123 sets a degree of the desire for movement, indicating the level of the desire for movement, based on the time of day, the presence or absence of toys in the surroundings, the presence or absence of people in the surroundings, and the emotions of the autonomous mobile device 11. For example, when the desire for movement reaches a predetermined threshold, the autonomous mobile device 11 performs various body movements.

[0128] The desire for emotional expression represents the autonomous mobile device 11's desire to express emotions. For example, the action planning unit 123 sets a degree of the desire for emotional expression, indicating the level of this desire, based on the date, time of day, presence or absence of people in the surroundings, and the emotions of the autonomous mobile device 11. For example, when the desire for emotional expression reaches a predetermined threshold, the autonomous mobile device 11 performs an action to express its current emotions.

[0129] The urge to defecate represents the autonomous mobile device 11's desire to perform the act of defecation. For example, the action planning unit 123 sets a degree of urge to defecate, indicating the intensity of the urge, based on the time of day and the emotions of the autonomous mobile device 11. For example, when the degree of urge to defecate exceeds a predetermined threshold, the autonomous mobile device 11 performs an action that simulates the act of defecation.

[0130] The desire for sleep represents the autonomous mobile device 11's desire to sleep. For example, the autonomous mobile device 11 sets a "desire for sleep" level, which indicates the degree of its desire for sleep, based on the time of day and the autonomous mobile device 11's emotions. For example, when the desire for sleep level exceeds a predetermined threshold, the autonomous mobile device 11 performs actions that simulate sleep.

[0131] After that, the individual value setting process is completed.

[0132] Returning to Figure 9, in step S2, the recognition unit 121 determines whether or not it has recognized the no-approach marker based on the sensor data (e.g., image data) supplied from the input unit 101. If it is determined that the approach marker has been recognized, the process proceeds to step S3.

[0133] In step S3, the autonomous mobile unit 11 avoids approaching the no-approach marker. Specifically, the recognition unit 121 supplies data indicating the location of the recognized no-approach marker to the action planning unit 123.

[0134] The action planning unit 123 plans the actions of the autonomous mobile unit 11, for example, to avoid entering a restricted area based on a no-approach marker. The action planning unit 123 supplies action planning data indicating the planned actions to the operation control unit 124.

[0135] The motion control unit 124 controls the drive unit 104 based on the action plan data to prevent the autonomous mobile unit 11 from entering the restricted area.

[0136] The process then proceeds to step S4.

[0137] On the other hand, if it is determined in step S2 that the no-approach marker has not been recognized, the process in step S3 is skipped, and the process proceeds to step S4.

[0138] In step S4, the recognition unit 121 determines whether or not it has recognized a toilet marker based on the sensor data (e.g., image data) supplied from the input unit 101. If it is determined that a toilet marker has been recognized, the process proceeds to step S5.

[0139] In step S5, the action planning unit 123 determines whether or not there is a need to defecate. Specifically, the recognition unit 121 supplies the action planning unit 123 with data indicating the location of the recognized toilet marker. If the level of need to defecate set in the processing of step S1, i.e., the level of need to defecate when the toilet marker is recognized, is above a predetermined threshold, the action planning unit 123 determines that there is a need to defecate, and the process proceeds to step S6.

[0140] In step S6, the action planning unit 123 determines whether or not to perform defecation near the toilet marker based on the growth level set in the processing of step S1. For example, if the growth level is above a predetermined threshold, the action planning unit 123 determines that the animal will perform defecation near the toilet marker (i.e., within the toilet area described above).

[0141] On the other hand, for example, if the growth level is below a predetermined threshold, the behavior planning unit 123 determines, with a probability corresponding to the growth level, whether to defecate near the toilet marker or elsewhere. For example, the higher the growth level, the higher the probability of determining that the animal will defecate near the toilet marker, and the lower the growth level, the higher the probability of determining that the animal will defecate elsewhere.

[0142] If it is determined that the animal is defecating near the toilet marker, the process proceeds to step S7.

[0143] In step S7, the autonomous mobile unit 11 performs the act of defecation near the toilet marker. For example, the action planning unit 123 plans the actions of the autonomous mobile unit 11 to urinate within the toilet area based on the toilet marker. The action planning unit 123 supplies action planning data indicating the planned actions to the action control unit 124.

[0144] The motion control unit 124 controls the drive unit 104 and the output unit 105 to perform the action of urinating within the toilet area based on the action plan data.

[0145] The process then proceeds to step S9.

[0146] On the other hand, if it is determined in step S6 that the act of defecation is performed outside the vicinity of the toilet marker, the process proceeds to step S8.

[0147] In step S8, the autonomous mobile unit 11 performs the act of defecation outside the vicinity of the toilet marker. Specifically, the action planning unit 123 plans the action of the autonomous mobile unit 11 to urinate outside the toilet area, for example, at its current location. The action planning unit 123 supplies action planning data indicating the planned action to the action control unit 124.

[0148] The motion control unit 124 controls the drive unit 104 and the output unit 105 to perform the action of urinating outside the toilet area, based on the action plan data.

[0149] The process then proceeds to step S9.

[0150] On the other hand, in step S5, if the level of urge to defecate set in step S1 is below a predetermined threshold, the action planning unit 123 determines that there is no urge to defecate, and steps S6 to S8 are skipped, and the process proceeds to step S9.

[0151] Furthermore, if it is determined in step S4 that the tray marker is not recognized, the processing in steps S5 through S8 is skipped, and the process proceeds to step S9.

[0152] In step S9, the recognition unit 121 determines whether or not it has recognized the favorite location marker based on the sensor data (e.g., image data) supplied from the input unit 101. If it is determined that the favorite location marker has been recognized, the process proceeds to step S10.

[0153] In step S10, the action planning unit 123 determines whether the marker preference is above a predetermined threshold. Specifically, the recognition unit 121 supplies the action planning unit 123 with data indicating the location of the recognized favorite location marker. The action planning unit 123 determines whether the marker preference set in the process of step S1, that is, the marker preference when the favorite location marker is recognized, is above a predetermined threshold. If it is determined that the marker preference is below the predetermined threshold, the process proceeds to step S11.

[0154] In step S11, the autonomous mobile unit 11 avoids approaching the favorite location marker. Specifically, the action planning unit 123 plans the actions of the autonomous mobile unit 11 so that it acts cautiously and avoids approaching the favorite location marker. The action planning unit 123 supplies action plan data indicating the planned actions to the action control unit 124.

[0155] The motion control unit 124 controls the drive unit 104 and the output unit 105 based on the action plan data to perform actions that prevent the robot from approaching the favorite location markers.

[0156] After that, the process returns to step S1, and the processes from step S1 onward are executed.

[0157] On the other hand, if it is determined in step S10 that the marker preference is above a predetermined threshold, the process proceeds to step S12.

[0158] In step S12, the action planning unit 123 determines whether or not there is a desire to play. If the level of desire to play set in the process of step S1, that is, the level of desire to play when the favorite place marker is recognized, is above a predetermined threshold, the action planning unit 123 determines that there is a desire to play, and the process proceeds to step S13.

[0159] In step S13, the autonomous mobile unit 11 places its favorite toy near the favorite location marker. For example, the action planning unit 123 plans the autonomous mobile unit 11's actions to place a toy with a preference level above a predetermined threshold within the favorite area based on the favorite location marker. The action planning unit 123 supplies action planning data indicating the planned actions to the action control unit 124.

[0160] The motion control unit 124 controls the drive unit 104 and the output unit 105 to perform the action of placing the favorite toy in the favorite area based on the action plan data.

[0161] After that, the process returns to step S1, and the processes from step S1 onward are executed.

[0162] On the other hand, in step S12, if the level of desire to play set in the processing of step S1 is below a predetermined threshold, the action planning unit 123 determines that there is no desire to play, and the process proceeds to step S14.

[0163] In step S14, the behavior planning unit 123 determines whether or not there is a desire for exercise. If the degree of desire for exercise set in the process of step S1, that is, the degree of desire for exercise when the favorite place marker is recognized, is above a predetermined threshold, the behavior planning unit 123 determines that there is a desire for exercise, and the process proceeds to step S15.

[0164] In step S15, the autonomous mobile robot 11 moves its body near the favorite location marker. For example, the action planning unit 123 plans the actions of the autonomous mobile robot 11 so that it moves its body within the favorite area. The actions of the autonomous mobile robot 11 set at this time are not always constant and change depending on the situation, time, the emotions of the autonomous mobile robot 11, etc. For example, usually actions such as singing or dancing are set as the actions of the autonomous mobile robot 11. And rarely, actions such as digging in the ground and finding coins are set as the actions of the autonomous mobile robot 11. The action planning unit 123 supplies action planning data indicating the planned actions to the action control unit 124.

[0165] The operation control unit 124 controls the drive unit 104 and the output unit 105 to perform the actions set within the favorite area based on the action plan data.

[0166] After that, the process returns to step S1, and the processes from step S1 onward are executed.

[0167] On the other hand, in step S14, if the exercise desire level set in the processing of step S1 is below a predetermined threshold, the action planning unit 123 determines that there is no exercise desire, and the process proceeds to step S16.

[0168] In step S16, the action planning unit 123 determines whether or not there is a need for sleep. If the level of need for sleep set in the process of step S1, that is, the level of need for sleep when the favorite place marker is recognized, is above a predetermined threshold, the action planning unit 123 determines that there is a need for sleep, and the process proceeds to step S17.

[0169] In step S17, the autonomous mobile unit 11 takes a nap near the favorite location marker. For example, the action planning unit 123 plans the autonomous mobile unit 11's actions to take a nap within the favorite area. The action planning unit 123 supplies action planning data indicating the planned actions to the motion control unit 124.

[0170] The motion control unit 124 controls the drive unit 104 and the output unit 105 to perform actions such as napping within the favorite area, based on the action plan data.

[0171] After that, the process returns to step S1, and the processes from step S1 onward are executed.

[0172] On the other hand, in step S16, if the sleep desire level set in step S1 is below a predetermined threshold, the action planning unit 123 determines that there is no sleep desire, and the process returns to step S1. Subsequently, the processes from step S1 onward are executed.

[0173] Furthermore, if it is determined in step S9 that the favorite location marker is not recognized, the process returns to step S1, and the processes from step S1 onward are executed.

[0174] <Example of installing a "No Approach" marker> Next, with reference to Figures 12 to 14, examples of how to install access restriction markers will be described.

[0175] The following describes an example where the no-approach marker is composed of a sticker on which a predetermined pattern is printed and which can be attached to and removed from desired locations.

[0176] Examples of places within a home where it is preferable that the autonomous mobile device 11 not approach or enter include the following:

[0177] It is desirable to prevent the autonomous mobile unit 11 from approaching or entering areas with water, such as kitchens, washrooms, and bathrooms, as these areas may get wet and malfunction.

[0178] It is desirable to prevent the autonomous mobile unit 11 from approaching furniture, doors, walls, etc., as these could be damaged if the unit 11 collides with them, or become immobilized if they obstruct its path.

[0179] It is desirable to prevent the autonomous mobile unit 11 from approaching places with steps or uneven surfaces, such as stairs or entrances, as there is a risk of the autonomous mobile unit 11 falling and being damaged, or tipping over and becoming immobile.

[0180] It is desirable to keep the autonomous mobile unit 11 away from heating appliances such as stoves, as the autonomous mobile unit 11 may be damaged by the heat.

[0181] In response to this, for example, a no-approach marker is installed as follows:

[0182] Figure 12 shows an example of preventing the autonomous mobile unit 11 from colliding with the TV stand 401 on which the TV 402 is installed. For example, a marker is attached to position P1 on the front of the TV stand 401. This prevents the autonomous mobile unit 11 from entering the no-entry zone A1 based on position P1, thereby preventing a collision with the TV stand 401.

[0183] In this example, because the TV stand 401 is wide, attaching multiple markers to the front of the TV stand 401 at predetermined intervals prevents the autonomous mobile unit 11 from colliding with the entire TV stand 401.

[0184] Figure 13 shows an example of preventing the autonomous mobile unit 11 from entering the bathroom 411. For example, markers are placed at position P11 near the right and bottom edge of the left wall of the bathroom 411, and at position P12 near the left and bottom edge of the bathroom door 413. This prevents the autonomous mobile unit 11 from entering the restricted area A11 based on position P11 and the restricted area A12 based on position P12.

[0185] In this case, with door 413 open, the right edge of restricted area A11 and the left edge of restricted area A12 overlap. Therefore, the entire entrance to the washroom 411 is blocked by restricted area A11 and restricted area A12, preventing the autonomous mobile unit 11 from entering the washroom 411.

[0186] Figure 14 shows an example of preventing the autonomous mobile unit 11 from entering the entrance 412. For example, stands 423-1 and 423-2 are installed between the left wall 422L and the right wall 422R of the entrance 421, with a predetermined distance between them. Markers are then placed at positions P21 on stand 423-1 and P22 on stand 423-2. This prevents the autonomous mobile unit 11 from entering the restricted area A21 based on position P21 and the restricted area A22 based on position P22.

[0187] In this case, the left edge of the restricted area A21 reaches wall 422L, and the right edge of the restricted area A22 reaches wall 422R. Also, the right edge of the restricted area A21 and the left edge of the restricted area A12 overlap. Therefore, the space between wall 422L and wall 422R is blocked by the restricted areas A11 and A12, preventing the autonomous mobile unit 11 from entering the entrance 421.

[0188] As described above, users can use markers to quickly or reliably instruct the autonomous mobile device 11 to perform desired actions. This improves the user's satisfaction with the autonomous mobile device 11.

[0189] For example, by using a no-approach marker, it is possible to reliably prevent the autonomous mobile device 11 from entering areas where it may be damaged or malfunction. This allows the user to leave the autonomous mobile device 11 powered on with peace of mind. As a result, the operational rate of the autonomous mobile device 11 increases, and the autonomous mobile device 11 feels more like a real dog.

[0190] Furthermore, users can set the toilet area to their desired location by using toilet markers. In addition, users can train the autonomous mobile device 11 to quickly and reliably perform actions simulating defecation within the toilet area, and experience the growth of the autonomous mobile device 11.

[0191] Furthermore, users can set their favorite areas to their desired locations by using markers for favorite places. In addition, users can train the autonomous mobile device 11 to perform predetermined actions quickly and reliably within the favorite area, and experience the growth of the autonomous mobile device 11.

[0192] <<2. Variant>> The following describes some modifications of the embodiments of the present technology described above.

[0193] <Variations regarding markers> First, let's explain some variations related to markers.

[0194] <Examples of marker usage> Markers are not limited to the uses described above and can be used for other purposes as well.

[0195] For example, if the autonomous mobile device 11 has a function to greet the user, the marker can be used to specify the location where the autonomous mobile device 11 will greet the user. For instance, the marker can be placed near the entrance, and the autonomous mobile device 11 can wait for the user within a predetermined area based on the marker before the time the user returns home.

[0196] For example, instead of deciding on the purpose of the markers in advance, the user can train the autonomous mobile device 11 so that it learns the purpose of the markers.

[0197] Specifically, for example, after placing a marker, the user gives commands to the autonomous mobile device 11 to perform desired actions near the marker using speech, gestures, etc. For example, the user might point to the marker and say to the autonomous mobile device 11, "Come here every morning at 7 o'clock," or "Don't go near this marker."

[0198] In response, the recognition unit 121 of the autonomous mobile unit 11 recognizes the user's command. The action planning unit 123 plans the commanded action near the marker according to the recognized command. The motion control unit 124 controls the drive unit 104 and the output unit 105 to perform the planned action.

[0199] Furthermore, the learning unit 122 learns the correspondence between markers and user commands. Then, as the user repeatedly issues similar commands near the markers, the learning unit 122 gradually learns the purpose of the markers. The action planning unit 123 plans an action for the markers based on the learned purpose of the markers. The operation control unit 124 controls the drive unit 104 and the output unit 105 to perform the planned action.

[0200] As a result, the autonomous mobile unit 11 will perform predetermined actions near the marker even without user commands. For example, the autonomous mobile unit 11 will arrive near the marker at a predetermined time. Alternatively, the autonomous mobile unit 11 will refrain from performing predetermined actions near the marker even without user commands. For example, the autonomous mobile unit 11 will not approach the marker.

[0201] In this way, the user can set the marker's purpose to their desired use.

[0202] For example, the user may be able to set the purpose of the marker in an application running on the information processing terminal 12. The information processing terminal 12 may then transmit data indicating the set purpose to the autonomous mobile unit 11, and the autonomous mobile unit 11 may recognize the purpose of the marker based on the received data.

[0203] For example, the use of the marker may be changed by updating the software of the autonomous mobile unit 11.

[0204] Specifically, for example, by installing the first version of the software on the autonomous mobile unit 11, the amount of time the autonomous mobile unit 11 spends near the marker increases. Next, by installing the second version of the software on the autonomous mobile unit 11, the autonomous mobile unit 11 will perform actions such as gathering toys near the marker. Then, by installing the third version of the software on the autonomous mobile unit 11, the autonomous mobile unit 11 will perform actions such as excavating near the marker and discovering virtual coins. In this way, by updating the software of the autonomous mobile unit 11, the uses of the marker can be increased, and the actions of the autonomous mobile unit 11 near the marker can be increased.

[0205] <When a person is wearing a marker> For example, the marker may be made from a wearable component such as clothing, wristbands, hats, accessories, badges, name tags, or armbands, so that a person can wear the marker.

[0206] In contrast, for example, the recognition unit 121 of the autonomous mobile unit 11 identifies a person based on the presence or absence and type of marker attached. The action planning unit 123 plans an action based on the result of identifying the person. The motion control unit 124 controls the drive unit 104 and the output unit 105 to perform the planned action.

[0207] For example, if the autonomous mobile device 11 is providing customer service at a theme park or commercial facility, it may treat the recognized person with extra care when it recognizes a person wearing a marker indicating that they are a valued customer. For example, the autonomous mobile device 11 may sing a song to the recognized person.

[0208] For example, if the autonomous mobile device 11 is acting as a guard dog, and it recognizes a person who is not wearing a marker that serves as a pass, the autonomous mobile device 11 may bark at that person, sound a warning, or make an alert.

[0209] For example, when the autonomous mobile device 11 goes for a walk outdoors, it may be configured to follow a person wearing a marker (for example, the owner of the autonomous mobile device 11).

[0210] <When the autonomous mobile unit 11 is equipped with a marker> For example, the marker may be made from a component that the autonomous mobile body 11 can wear, such as clothing, a collar, or an accessory, so that the autonomous mobile body 11 can attach the marker.

[0211] In response to this, for example, the recognition unit 121 of the autonomous mobile unit 11 identifies other autonomous mobile units 11 based on the presence or absence or type of marker attached. The action planning unit 123 plans an action based on the result of identifying other autonomous mobile units 11. The operation control unit 124 controls the drive unit 104 and the output unit 105 to perform the planned action.

[0212] For example, autonomous mobile unit 11 may consider other autonomous mobile units 11 wearing collars of the same type as markers as friends and act together with them. For example, autonomous mobile unit 11 may play, go for walks, or eat food with other autonomous mobile units 11 that it considers friends.

[0213] For example, if multiple autonomous mobile units 11 are divided into multiple teams and act accordingly, each autonomous mobile unit 11 may be configured to distinguish between autonomous mobile units 11 on the same team and autonomous mobile units 11 on other teams based on the type of markers worn by the other autonomous mobile units 11. For example, if multiple autonomous mobile units 11 are divided into multiple teams and play a game such as soccer, each autonomous mobile unit 11 may distinguish between teammates and opponents based on the type of markers worn by the other autonomous mobile units 11 and play the game accordingly.

[0214] <When recognizing an existing object as a marker> For example, the autonomous mobile unit 11 may recognize an existing object as a marker instead of a dedicated marker.

[0215] For example, the autonomous mobile device 11 may be configured to recognize traffic lights as markers. Alternatively, the autonomous mobile device 11 may be configured to identify traffic lights in the green light state, the yellow light state, and the red light state as different markers. This would enable the autonomous mobile device 11 to recognize traffic lights while walking and proceed across or stop at crosswalks. It would also enable the autonomous mobile device 11 to act as a guide dog to assist visually impaired individuals.

[0216] <Virtual Marker> For example, a user may place a virtual marker (hereinafter referred to as a virtual marker) on a map, and the autonomous mobile unit 11 may recognize the virtual marker.

[0217] For example, a user uses the information processing terminal 12 to place a virtual marker at any location on a map showing the layout of their home. The information processing terminal 12 uploads map data, including the map on which the virtual marker is placed, to the information processing server 13.

[0218] The recognition unit 121 of the autonomous mobile device 11 downloads map data from the information processing server 13. The recognition unit 121 recognizes the current position of the autonomous mobile device 11 and, based on the map data and the current position of the autonomous mobile device 11, recognizes the position of the virtual marker in real space. Then, the autonomous mobile device 11 performs the actions described above based on the position of the virtual marker in real space.

[0219] <Other variations> For example, the user may use the information processing terminal 12 to confirm the location of the marker recognized by the autonomous mobile unit 11.

[0220] For example, the recognition unit 121 of the autonomous mobile device 11 transmits data indicating the location and type of the recognized marker to the information processing server 13. The information processing server 13 generates map data in which the information indicating the location and type of the marker recognized by the autonomous mobile device 11 is superimposed on a map showing, for example, the floor plan of the user's home. The information processing terminal 12 downloads the map data with the information indicating the location and type of the marker superimposed from the information processing server 13 and displays it.

[0221] This allows the user to check the recognition status of the markers on the autonomous mobile unit 11.

[0222] Furthermore, for example, the information processing terminal 12 or information processing server 13 may perform some of the processing of the autonomous mobile unit 11 as described above. For example, the information processing server 13 may perform some or all of the processing of the recognition unit 121, learning unit 122, and action planning unit 123 of the autonomous mobile unit 11.

[0223] In this case, for example, the autonomous mobile device 11 transmits sensor data to the information processing server 13. The information processing server 13 performs marker recognition processing based on the sensor data and plans the actions of the autonomous mobile device 11 based on the marker recognition results. The information processing server 13 transmits action plan data indicating the planned actions to the autonomous mobile device 11. Based on the received action plan data, the autonomous mobile device 11 controls the drive unit 104 and the output unit 105 to perform the planned actions.

[0224] <<3.B>> <Example of computer configuration> The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up that software are installed on a computer. Here, a computer includes computers built into dedicated hardware, as well as general-purpose personal computers that can perform various functions by installing various programs.

[0225] Figure 15 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes described above by a program.

[0226] In computer 1000, the CPU (Central Processing Unit) 1001, ROM (Read Only Memory) 1002, and RAM (Random Access Memory) 1003 are interconnected by a bus 1004.

[0227] An input / output interface 1005 is further connected to the bus 1004. An input / output interface 1005 is connected to an input unit 1006, an output unit 1007, a recording unit 1008, a communication unit 1009, and a drive 1010.

[0228] The input section 1006 consists of input switches, buttons, a microphone, an image sensor, etc. The output section 1007 consists of a display, a speaker, etc. The recording section 1008 consists of a hard disk or non-volatile memory, etc. The communication section 1009 consists of a network interface, etc. The drive 1010 drives removable media 1011 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0229] In the computer 1000 configured as described above, the CPU 1001 loads, for example, a program stored in the recording unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, and executes it, thereby performing the series of processes described above.

[0230] The program executed by computer 1000 (CPU 1001) can be provided by recording it on removable media 1011, such as a packaged media. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0231] In computer 1000, programs can be installed in the recording unit 1008 via the input / output interface 1005 by inserting the removable media 1011 into the drive 1010. Alternatively, programs can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the recording unit 1008. Furthermore, programs can be pre-installed in the ROM 1002 or the recording unit 1008.

[0232] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0233] Furthermore, in this specification, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure or not. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device in which multiple modules are housed in one enclosure, are both considered systems.

[0234] Furthermore, the embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0235] For example, this technology can be configured as cloud computing, where a single function is shared and processed collaboratively by multiple devices via a network.

[0236] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0237] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0238] <Examples of configuration combinations> This technology can also be configured as follows:

[0239] (1) In an autonomous mobile device that operates autonomously, A recognition unit that recognizes the marker, An action planning unit that plans the actions of the autonomous mobile unit in relation to the recognized marker, A motion control unit that controls the operation of the autonomous mobile body to perform planned actions, An autonomous mobile vehicle equipped with the following features. (2) The action planning unit plans the autonomous mobile's actions toward the marker based on at least one of the following: the usage status of the autonomous mobile, the circumstances when the marker was recognized, and the usage status of other autonomous mobiles. The autonomous mobile body described in (1) above. (3) A learning unit that sets the growth level of the autonomous mobile unit based on the usage status of the autonomous mobile unit. Furthermore, The action planning unit plans the actions of the autonomous mobile unit toward the marker based on the growth rate. The autonomous mobile body described in (2) above. (4) The action planning unit controls the success rate of the action for the marker based on the growth rate. The autonomous mobile body described in (3) above. (5) The action planning unit sets the desires of the autonomous mobile body based on the circumstances when the marker is recognized, and plans the actions of the autonomous mobile body toward the marker based on those desires. An autonomous mobile body as described in any of (2) to (4) above. (6) The action planning unit plans the actions of the autonomous mobile body to perform actions based on the desire within a predetermined area based on the marker. The autonomous mobile body described in (5) above. (7) The aforementioned desires include at least one of the following: the desire to be close to others, the desire to play with objects, the desire to move one's body, the desire to express emotions, the desire to excrete, and the desire to sleep. The autonomous mobile body described in (5) or (6) above. (8) The action planning unit plans the actions of the autonomous mobile body to perform an action that simulates the act of defecation within a predetermined area based on the marker, if the degree of the urge to defecate is above a predetermined threshold. The autonomous mobile body described in (7) above. (9) The action planning unit sets a preference level for the marker based on at least one of the usage status of the autonomous mobile unit and the usage status of the other autonomous mobile units, and plans the autonomous mobile unit's actions toward the marker based on the preference level. An autonomous mobile body as described in any of (2) to (8) above. (10) The action planning unit plans the autonomous mobile body's actions to avoid approaching the marker if the preference level is below a predetermined threshold. The autonomous mobile body described in (9) above. (11) A learning unit that learns the intended use of the aforementioned markers Furthermore, The action planning unit plans the actions of the autonomous mobile unit based on the learned uses of the markers. An autonomous mobile body as described in any of (1) to (10) above. (12) The action planning unit plans the actions of the autonomous mobile unit so as not to enter a predetermined area based on the marker. An autonomous mobile body as described in any of (1) to (11) above. (13) The action planning unit plans the actions of the autonomous mobile device based on the use of the marker, which changes depending on the version of the software installed on the autonomous mobile device. An autonomous mobile body as described in any of (1) to (12) above. (14) The recognition unit identifies a person based on whether or not the marker is attached or the type of marker. The action planning unit plans the actions of the autonomous mobile unit based on the results of the person identification. An autonomous mobile body as described in any of (1) to (13) above. (15) The recognition unit identifies other autonomous mobile bodies based on whether or not the marker is attached or the type of marker. The action planning unit plans the actions of the autonomous mobile body based on the identification results of the other autonomous mobile body. An autonomous mobile body as described in any of (1) to (14) above. (16) The marker is a component that represents a predetermined two-dimensional or three-dimensional pattern. An autonomous mobile body as described in any of (1) to (15) above. (17) The recognition unit recognizes the virtual marker placed on the map data based on the current position of the autonomous mobile body. The action planning unit plans the actions of the autonomous mobile unit in relation to the virtual marker. An autonomous mobile body as described in any of (1) to (16) above. (18) A recognition unit that recognizes the marker, An action planning unit that plans the actions of the autonomous mobile unit in response to the recognized marker, An information processing device equipped with the following features. (19) Recognize the marker, Plan the actions of the autonomous mobile unit in relation to the recognized marker. Information processing methods. (20) Recognize the marker, Plan the actions of the autonomous mobile unit in relation to the recognized marker. A program that causes a computer to perform a process.

[0240] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur. [Explanation of Symbols]

[0241] 1 Information processing system, 11-1 to 11-n Autonomous mobile unit, 12-1 to 12-n Information processing terminal, 13 Information processing server, 101 Input unit, 102 Communication unit, 103 Information processing unit, 104 Drive unit, 105 Output unit, 121 Recognition unit, 122 Learning unit, 123 Action planning unit, 124 Motion control unit, 302 Information processing unit, 321 Autonomous mobile unit control unit, 322 Application control unit, 331 Recognition unit, 332 Learning unit, 333 Action planning unit, 334 Motion control unit

Claims

1. In an autonomous mobile device that operates autonomously, A recognition unit that recognizes the marker, The usage status of the aforementioned autonomous mobile device and at least one of the usage statuses of other autonomous mobile devices. An action planning unit sets a preference level for the marker based on the preference level and plans the actions of the autonomous mobile unit towards the marker based on the preference level, A motion control unit that controls the operation of the autonomous mobile body to perform planned actions, An autonomous mobile vehicle equipped with the following features.

2. A learning unit that sets the growth level of the autonomous mobile unit based on the usage status of the autonomous mobile unit. Furthermore, The action planning unit plans the actions of the autonomous mobile unit toward the marker based on the growth rate. The autonomous mobile body according to claim 1.

3. The action planning unit controls the success rate of the action for the marker based on the growth rate. The autonomous mobile body according to claim 2.

4. The action planning unit sets the desires of the autonomous mobile body based on the circumstances when the marker is recognized, and plans the actions of the autonomous mobile body toward the marker based on those desires. The autonomous mobile body according to claim 1.

5. The action planning unit plans the actions of the autonomous mobile body to perform actions based on the desire within a predetermined area based on the marker. The autonomous mobile body according to claim 4.

6. The aforementioned desires include at least one of the following: the desire to be close to others, the desire to play with objects, the desire to move one's body, the desire to express emotions, the desire to excrete, and the desire to sleep. The autonomous mobile body according to claim 4.

7. The action planning unit plans the actions of the autonomous mobile body to perform an action that simulates the act of defecation within a predetermined area based on the marker, if the degree of the urge to defecate is above a predetermined threshold. The autonomous mobile body according to claim 6.

8. The action planning unit plans the autonomous mobile body's actions to avoid approaching the marker if the preference level is below a predetermined threshold. The autonomous mobile body according to claim 1.

9. A learning unit that learns the intended use of the aforementioned markers Furthermore, The action planning unit plans the actions of the autonomous mobile unit based on the learned uses of the markers. The autonomous mobile body according to claim 1.

10. The action planning unit plans the actions of the autonomous mobile unit so as not to enter a predetermined area based on the marker. The autonomous mobile body according to claim 1.

11. The action planning unit plans the actions of the autonomous mobile device based on the use of the marker, which changes depending on the version of the software installed on the autonomous mobile device. The autonomous mobile body according to claim 1.

12. The recognition unit identifies a person based on whether or not the marker is attached or the type of marker. The action planning unit plans the actions of the autonomous mobile unit based on the results of the person identification. The autonomous mobile body according to claim 1.

13. The recognition unit identifies other autonomous mobile bodies based on whether or not the marker is attached or the type of marker. The action planning unit plans the actions of the autonomous mobile body based on the identification results of the other autonomous mobile body. The autonomous mobile body according to claim 1.

14. The marker is a member that represents a predetermined two-dimensional or three-dimensional pattern. The autonomous mobile body according to claim 1.

15. The recognition unit recognizes the virtual marker placed on the map data based on the current position of the autonomous mobile body. The action planning unit plans the actions of the autonomous mobile unit in relation to the virtual marker. The autonomous mobile body according to claim 1.

16. A recognition unit that recognizes the marker, An action planning unit sets a preference level for the marker based on at least one of the usage status of the autonomous mobile unit and the usage status of other autonomous mobile units, and plans the autonomous mobile unit's actions toward the marker based on the preference level. An information processing device equipped with the following features.

17. To recognize the marker, Setting the preference level for the marker based on at least one of the usage status of the autonomous mobile unit and the usage status of other autonomous mobile units, To plan the actions of the autonomous mobile unit toward the marker based on the aforementioned preference level. Information processing methods including

18. To recognize the marker, Setting the preference level for the marker based on at least one of the usage status of the autonomous mobile unit and the usage status of other autonomous mobile units, To plan the actions of the autonomous mobile unit toward the marker based on the aforementioned preference level. A program that causes a computer to perform a process that includes [a specific action].

Images