Method of automatically acquiring instruction to actuator for causing robot to take basic action

Abstract

To allow for automatically acquiring an instruction to cause a robot to perform basic actions even in situations where various specifications of the robot exist.SOLUTION: A method of automatically acquiring an instruction to an actuator for causing a mobile robot to perform basic actions includes: securing access to sensors by which the robot recognizes an environment, and a plurality elements of an actuator and enumerating combinations between presence / absence of respective control inputs to the plurality of elements and signal types of the control inputs (S41); causing the robot to perform a trial action by performing a control input to the actuator in each of the combinations (S45) and using outputs of the sensors before and after the trial action to identify movements of the robot before and after the trial action (S43, S46, S47); selecting movements corresponding to the basic actions from movements of the robot in the individual combinations (S51); and taking a combination of the selected movements corresponding to the basic actions as an instruction to the actuator for causing the robot to perform the basic actions.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a method for automatically acquiring instructions for actuators for performing basic operations of a robot. [Background technology]

[0002] In the field of IoT (Internet of Things), robots are being utilized. In Patent Document 1 (Name of Invention: Method and System for Detection in Data Collection Environment of Industrial Internet of Things Using Large Data Sets), robots and drones are remotely controlled and autonomously operated for the purpose of data collection (IoT) in industrial environments. In Patent Document 2 (Name of Invention: Autonomous Learning Robot Apparatus and Motion Generation Method of Autonomous Learning Robot Apparatus), robot hand motions according to the environment are learned by communicating with the robot. In Non-Patent Document 1, the systematic errors of odometry according to individual road surface environments, such as carpet indoors and pavement outdoors, are recorded in advance as a road surface environment map, and this map is used when estimating the actual position of the robot when it is actually running, thereby realizing odometry that is robust to the road surface environment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-530159 [Patent Document 2] JP 2020-028950 A [Non-patent literature]

[0004] [Non-Patent Document 1] Daichi Yamada, Akihisa Oya: "Odometry calibrating systematic errors based on road environment map - Verification of effectiveness on various road surfaces -", ​​The 30th Anniversary Academic Conference of the Robotics Society of Japan, 4J1-4 (September 2012) Summary of the Invention [Problem to be solved by the invention]

[0005] In the conventional technology described above, it is assumed that the specifications of the robots are known on the platform side, which is a system for managing the robots. Under this assumption, it is not possible to properly manage the robots when the robots to be managed from the platform side are separated into various sites, such as various stores, and each robot may have various specifications (specs determined by the robot's manufacturer and its model number, etc.). In other words, when it is assumed that the robot specifications are known, if there is a wide variety of robot specifications, it is difficult to properly manage the robots because it takes a lot of time to manually prepare control commands for each robot's specifications.

[0006] Specifically, for example, the only known specifications of a robot are that it is a vehicle-type robot that moves on tires. While there may be a large number and a wide variety of such vehicle-type robots, if individual conditions such as the size of the tires are unknown as specifications, even if one tries to control the movement of the robot, which is an elementary operation in achieving a task to be requested to be executed by the robot, it is unclear how to control the robot and how it will move, making appropriate control and management impossible.

[0007] In view of the problems with the prior art described above, the present invention aims to provide a method for appropriately managing robots by enabling automatic acquisition of instructions for causing a robot to perform basic operations even in a situation where there are a variety of robot specifications. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention is a method for automatically acquiring instructions for an actuator to perform a basic operation of a robot that moves by a moving mechanism driven by an actuator, the method comprising the steps of: ensuring access to a sensor for the robot to recognize the outside world (S1, S2); ensuring access to a control input to each of a plurality of actuator elements constituting the actuator (S1, S2); enumerating combinations of the presence or absence of a control input to each of the plurality of actuator elements and the signal type of the control input (S41); and performing a control input to the actuator for each of the enumerated combinations (S45), thereby performing a trial operation of the robot, identifying movements of the robot before and after the trial operation using outputs of the sensor before and after the trial operation (S43, S46, S47); selecting a combination corresponding to a basic operation from the movements of the robot in each of the enumerated combinations (S51); and automatically acquiring a combination corresponding to the selected basic operation as an instruction to the actuator to perform a basic operation of the robot.

[0009] In order to achieve the above object, the present invention provides a method for automatically acquiring instructions for a plurality of actuators to cause a robot that moves by a movement mechanism driven by the plurality of actuators to perform a basic movement, wherein for another robot having specifications common to the robot, a basic movement and instructions for the plurality of actuators to cause the other robot to perform the basic movement are known as templates, and a movement log of the robot is acquired (S27) in a form that links information on the movement of the robot before and after an individual movement performed by the robot by designating the presence or absence of a control input to each of the plurality of actuator elements and the signal type of the control input, and control input is performed to the actuators based on the designation information, and the movement log and the template are used to learn (S28), thereby automatically acquiring designation information on the presence or absence of a control input to each of the plurality of actuator elements and the signal type of the control input as an optimum one for realizing the basic movement. Effect of the Invention

[0010] According to the first feature, by enumerating combinations of the presence or absence of control input to each of the multiple actuator elements and the signal type of the control input, the robot is subjected to trial operations, and the movements of the robot before and after the trial operations that correspond to basic operations are selected. Even if the configuration of the movement mechanism after the actuator elements is unknown as a robot specification, it is possible to automatically acquire the configuration of control input to the multiple actuator elements for causing the robot to perform basic operations. Therefore, efficient and appropriate management can be performed even in situations where there are a variety of robot specifications.

[0011] According to the second feature, instead of the information obtained by performing trial operations of the robot in the first feature, an operation log regarding the individual operations of the robot is obtained, and by learning using this operation log and a template for another robot having common specifications, it is possible to automatically acquire the form of control input to a plurality of actuator elements for causing the robot to perform basic operations as the optimum one for the robot under the premise of common specifications.Therefore, under the premise that template information for another robot having common specifications is known, efficient and appropriate management can be performed even in a situation where there are various robot specifications. [Brief description of the drawings]

[0012] [Figure 1] 1 is an overall configuration diagram of an automatic action acquisition system according to an embodiment. [Diagram 2] FIG. 2 is a configuration diagram of a robot, a dongle, and a server of an automatic action acquisition system according to an embodiment. [Diagram 3] 1 is a flowchart of an operation of an automatic action capture system according to an embodiment. [Figure 4] 4 is a flowchart showing details of the process of a basic action acquisition unit according to one embodiment (that is, details of step S3 in FIG. 3). [Diagram 5]FIG. 2 is a diagram showing an example of a configuration of an actuator and a moving mechanism in a robot. [Figure 6] FIG. 5 is a diagram illustrating an example of explaining each step in FIG. 4 as examples EX1 to EX4. [Figure 7] 5 is a diagram illustrating an explanatory example of each step in FIG. 4 as examples EX5 to EX8. [Figure 8] FIG. 13 is a diagram illustrating an example of an operation recorded in JSON format. [Figure 9] FIG. 11 is a functional block diagram of a robot and a server in an automatic action acquisition system according to another embodiment. [Figure 10] 11 is a flowchart of the operation of an automatic action capture system according to another embodiment. [Figure 11] 11 is a flowchart showing details of learning by a learning unit according to one embodiment (details of step S28 in FIG. 10). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] FIG. 1 is a diagram showing the overall configuration of an automatic action acquisition system according to an embodiment. The automatic action acquisition system 100 includes a robot 10 and a server 50. The robot 10 and the server 50 are capable of communicating with each other via a network NW such as the Internet. The robot 10 is placed in a field F (e.g., a store, whether indoors or outdoors) where the robot 10 operates, and includes a sensor 15 including a camera or the like for recognizing the field F, and a moving mechanism 17 including wheels or the like. A detachable dongle 30 is also attached to the robot 10.

[0014] 2 is a configuration diagram of the robot 10, the dongle 30, and the server 50 of the automatic action acquisition system 100 according to an embodiment. The robot 10 includes, as its hardware configuration, a first processor 11 including a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a first RAM 12 as a primary storage device including a volatile memory providing a working area for the first processor 11, a first ROM 13 as a secondary storage device including a non-volatile memory for storing programs executed by the first processor 11 and various data, a first communication interface (IF) 14 including a network interface card for wired communication and / or wireless communication, a sensor 15 including a camera and / or a LiDAR sensor for recognizing the field F, an actuator 16 such as a motor, a moving mechanism 17 including wheels that receive a driving force from the actuator 16 to enable the robot 10 to move within the field F, a first input / output interface (IF) 18 including a USB port that enables various peripheral devices (guest devices) to be connected to the robot 10, and a first bus BS1 for transmitting and receiving data between the hardware.

[0015] The dongle 30 has, as its hardware configuration, a second input / output interface 35 formed of a USB port or the like and enabling the dongle 30 to be connected as a guest device to a host device such as the robot 10, a second processor 31 formed of a CPU (central processing unit) or a GPU (graphics processing unit) or the like, a second RAM 32 serving as a primary storage device formed of a volatile memory or the like that provides a working area for the second processor 31, a second ROM 33 serving as a secondary storage device formed of a non-volatile memory or the like that stores programs and various data, a second communication interface (IF) 34 formed of a network interface card or the like that performs wired communication and / or wireless communication functions, and a second bus BS2 that performs data transmission and reception between each of these pieces of hardware.

[0016] The server 50 has a hardware configuration similar to the robot 10 and the dongle 30 described above, which is a general computer configuration (note that in the robot 10, the sensor 15, the actuator 16, and the moving mechanism 17 are added to the configuration as a configuration for realizing autonomous movement, etc.). That is, the server 50 has a third processor 51 composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., a third RAM 52 as a primary storage device composed of a volatile memory or the like that provides a working area for the third processor 51, a third ROM 53 as a secondary storage device composed of a non-volatile memory or the like that stores programs and various data, a third communication interface (IF) 54 composed of a network interface card or the like that performs the function of wired communication and / or wireless communication, a third input / output interface 55 composed of a USB port or the like that enables connection of various input / output devices (keyboard, mouse, display, etc.), and a third bus BS3 that performs data transmission and reception between each of these hardware.

[0017] As described later (step S1 in FIG. 3), in this embodiment, the dongle 30 is connected to the robot 10 by connecting the first input / output interface 18 and the second input / output interface 35. As shown typically by the line L2, after the connection, a preparation program (PG) P21 and a basic action acquisition program (PG) P22 stored in the second ROM 33 of the dongle 30 are loaded onto the second RAM 32, and executed by the second processor 31 to realize the functional configuration, which is the control unit 20. At this time, the dongle 30 may not have its own battery, and may receive power from a battery possessed by the robot 10 to operate the second processor 31, etc., to realize the functional configuration, or the dongle 30 may independently supply power to the second processor 31, etc. of the dongle 30 to operate them.

[0018] As a modified example, the control unit 20 may be realized by the first processor 11 of the robot 10, as shown by the line L1, instead of by the second processor 31 of the dongle 30. That is, the control unit 20 may be a functional configuration realized by reading the preparation program (PG) P21 and the basic action acquisition program (PG) P22 stored in the second ROM 33 of the dongle 30 into the first RAM 12 of the robot 10 and executing them by the first processor 11 of the robot 10 after the connection. In this modified example, the second processor 31 and the second RAM 32 may be omitted as a configuration of the dongle 30. Similarly, as a further modified example, the control unit 20 may be realized by the third processor 51 of the server 50 transmitting a control command via the network NW, and the second processor 31 of the dongle 30 or the first processor 11 of the robot 10 receiving the control command to realize the control process. (That is, instead of storing the preparation program P21 and the basic action acquisition program P22 in the second ROM 33 of the dongle 30, these programs (control commands) may be transmitted from the server 50 to the robot 10 and the dongle 30, thereby realizing the control unit 20 under the control of the server 50.)

[0019] The control unit 20 includes, as functional blocks, a preparation unit 21 and a basic action acquisition unit 22. The preparation unit 21 is realized when the second processor 31 (the first processor 11 in the modified example) executes a preparation program P21, and following this process, the basic action acquisition unit 22 is realized when the second processor 31 (the first processor 11 in the modified example) executes a basic action acquisition program P22. The basic action acquisition unit 22 further includes a reference point determination unit 23, an action trial unit 24, and a basic action selection unit 25.

[0020] Since the robot 10 does not have the first communication interface 14 (a configuration in which a part is omitted from FIG. 2), the robot 10 alone does not have a communication function with the outside. However, an embodiment is also possible in which the robot 10 acquires a communication function with the outside by using the second communication interface 34 when the dongle 30 is connected.

[0021] 3 is a flowchart of the operation of the automatic action acquisition system 100 according to an embodiment. In step S1, an administrator of the robot 10 in the field F connects the dongle 30 to the robot 10, and then the process proceeds to step S2.

[0022] In order to connect the dongle 30 in step S1, the administrator or the like powers on the robot 10 in advance, causes the first processor 11 to read and execute a robot control OS (operating system) stored in the first ROM 13, and causes the sensors 15 and actuators 16 of the robot 10 to enter a standby state waiting for a control command (triggered by the connection of the dongle 30). In step S1, the administrator or the like connects the dongle 30 to the robot 10 which is in the standby state.

[0023] In step S2, the connection of the dongle 30 in step S1 is used as a trigger to cause the second processor 31 (the first processor 11 in the modified example) to load and execute the preparation program P21, causing the preparation unit 21 to perform various preparation processes, and then the process proceeds to step S3. The preparation processes in step S2 are (1) network setting, (2) hardware search, and (3) control app (application) setting, which are as follows:

[0024] (1) Regarding network settings, by activating the first communication interface 14 or the second communication interface 34, under the instruction of the control unit 20, the robot 10, the dongle 30, and the server 50 are placed in a state in which they can communicate with each other via the network NW.

[0025] (2) Regarding hardware search, the sensor 15 consisting of a camera and / or LiDAR and the actuator 16 are searched for and made accessible by the control unit 20.

[0026] Regarding the sensor 15, if it is connected and implemented as a USB device in the robot 10, for example, its presence can be searched for and made accessible using a standard function for searching USB devices in the OS (such as the lsusb command). Regarding the actuator 16, if it is connected and implemented via a general-purpose input / output port (GPIO) in the robot 10, for example, electricity can be passed through multiple pins of the GPIO one by one, and each pin that is confirmed to be energized can be identified as corresponding to the actuator 16 (there may be multiple actuators 16) and recognized as accessible.

[0027] In this embodiment, it is assumed that the configurations of the actuator 16 and the moving mechanism 17 are unknown, so this step S1 only confirms the existence of each of the actuators 16 (there can be multiple actuators 16) by checking whether electricity is flowing to the pins.

[0028] (3) Regarding the control application setting, in accordance with the result of the hardware search, a predetermined control application (such as a device driver) corresponding to each piece of hardware is read from a server or the like that distributes the application on the network NW so that each piece of hardware can be controlled by the control unit 20. For a LiDAR sensor or the like that constitutes the sensor 15, after the control application is read, coordinate calibration or the like may be performed as an initialization process.

[0029] After the preparations by the preparation unit 21 have been completed in step S2 above, in step S3, the basic operation acquisition unit 22 experimentally controls the robot 10 and organizes the results to obtain information regarding the basic operations of the robot 10, and transmits this information regarding the basic operations to the server 50 via the network NW, before proceeding to step S4.

[0030] In step S4, a command to execute a predetermined task is sent from the server 50 to the robot 10 using the basic operation acquired in step S3, and the robot 10 executes the predetermined task, thereby terminating the flow of Fig. 3. The predetermined task can be configured as a sequence of one or more predetermined instructions (which may include case distinction and repetitive control) executed by the robot 10 to realize, for example, the movement of the robot 10 or the transport of an item, and the predetermined basic operation acquired in step S3 may be set in advance as a predetermined combination of these basic operations that enables the robot 10 to realize, in whole or in part, the predetermined task to be executed by the robot 10 in step S4.

[0031] For example, if the predetermined task to be executed by the robot 10 in step S4 is realized with the movement of the robot 10 (for example, a monitoring task while moving within the field F, or transporting an item by moving within the field F), then in step S3, the robot 10 may acquire four types of basic movements, such as forward movement, marching, turning right, and turning left, as basic movements for realizing movement on an arbitrary trajectory, as described below. The basic movements may include not only movements such as forward movement, backward movement, turning right, and turning left, but also movements of other movable parts such as arms.

[0032] Thus, in step S4, in order to have the robot 10 execute a task of moving, for example, from a current location to a predetermined destination within the field F, a combination command of basic actions for realizing the trajectory of the movement is transmitted from the server 50 to the robot 10, and the robot 10 can realize the movement task according to a sequence of basic actions along the trajectory. Also, by acquiring basic actions through the action trials up to step S3 just before requesting the robot 10 to execute a predetermined task in step S4, it is possible to effectively check and grasp the operating state of the robot 10 and the state of the operating environment just before the task is executed.

[0033] Fig. 4 is a flowchart showing details of the process of basic operation acquisition unit 22 according to one embodiment (i.e., details of step S3 in Fig. 3). As shown by the configuration of the corresponding functional block on the right side of each step, among the steps in Fig. 4, steps S31 and S32 are steps in which reference point determination unit 23 determines a reference point (a reference point for defining the position of robot 10 as a coordinate in field F), steps S41 to S49 are steps in which operation trial unit 24 uses the determined reference point to control robot 10 to perform a trial operation in field F and records the results, and steps S51 to S53 are steps in which basic operation selection unit 25 organizes the trial results of operation trial unit 24 and extracts information on the basic operations of robot 10.

[0034] Each step in Figure 4 will be explained below. However, as described above, the subject of operation in each step is the reference point determination unit 23, the action trial unit 24 or the basic action selection unit 25 (i.e., the first processor 11 or the second processor 31 executing each module of the basic action acquisition program P22), so in order to simplify the explanation, we will omit any mention of the subject of operation and only explain the processing content.

[0035] As an explanatory example, the following will describe a case in which the robot 10 is configured with a four-wheeled vehicle-type robot as the moving mechanism 17, as shown in the schematic example in Figure 5 (a schematic example of the state when the robot 10 is placed on the ground of field F and viewed from the sky), and the actuator 16 and the moving mechanism 17 are configured in four different ways as shown below in (1) to (4).

[0036] (1) Actuator A1 as the first actuator 16 drives the left front wheel W1 of the four wheels as the first moving mechanism 17. (2) Actuator A2 as the second actuator 16 drives the right front wheel W2 of the four wheels as the second movement mechanism 17. (3) Actuator A3 serves as the third actuator 16 and serves as the third movement mechanism 17 to drive the left rear wheel W3 of the four wheels. (4) Actuator A4 serves as the fourth actuator 16 and serves as the fourth movement mechanism 17 to drive the right rear wheel W4 of the four wheels.

[0037] It should be noted here that the control unit 20 is in a state where it is unable to know the individual conditions of the robot 10 at the start of the flow in Fig. 4. For example, the control unit 20 is in an unknown state regarding conditions such as how many wheels are present, where they are installed on the robot 10, and the tire diameter of each wheel (conditions related to the mechanical configuration of the moving mechanism 17), as well as conditions regarding the correspondence between the actuator 16 and the moving mechanism 17 it drives.

[0038] In the example of Fig. 5, in the preparation process of the preparation unit 21 (step S2 in Fig. 3), only the fact that four actuators A1 to A4 exist as controllable actuators as the actuators 16 is known to the control unit 20. Meanwhile, although each of the four actuators A1 to A4 drives one of the moving mechanism 17, the corresponding relationship of which of the wheels W1 to W4 specifically shown in Fig. 5 each will drive is unknown to the control unit 20. Furthermore, the information that the moving mechanism 17 is in fact composed of the wheels W1 to W4 specifically shown in Fig. 5 is also unknown to the control unit 20.

[0039] In the flow of FIG. 4 of this embodiment, the only thing known is that multiple control objects as actuators 16 exist as four actuators A1 to A4, for example, as shown in FIG. 5 (with regard to the moving mechanism 17 ahead of the actuator 16, the corresponding driving relationship and mechanical configuration are unknown), and it is possible to acquire information as the basic operation of the robot 10 as to how the robot 10 can be moved by driving the four actuators A1 to A4.

[0040] As an illustrative example, the following constraints are set. (a) The four actuators A1 to A4 each drive one of the wheels, and are capable of at least forward rotation control or reverse rotation control. (b) The acquired basic movements are four basic elementary movements for moving the robot 10: forward, backward, right turn, and left turn.

[0041] Assuming that there are the above-mentioned constraints, each step in Fig. 4 will be described below. Fig. 6 and Fig. 7 are diagrams showing explanatory examples of each step in Fig. 4 as examples EX1 to EX4 and examples EX5 to EX8, respectively, corresponding to examples of the actuator 16 and the moving mechanism 17 of the robot 10 in Fig. 5, and will be referred to as appropriate below.

[0042] In step S31, the sensor 15 serving as a recognition means for the field F is driven, and the sensor output (the recognition result for the field F) is received and analyzed to search for objects around the robot 10, and then the process proceeds to step S32.

[0043] For example, if the sensor 15 is a LiDAR, the LiDAR that has been enabled by the control application settings can be started, laser light can be emitted to obtain point cloud information of the surroundings (field F) as sensor output, preprocessing such as removing the ground from the point cloud information can be performed, and the point cloud information can be clustered to detect objects. Any existing method can be used for detecting objects from point cloud information, and for example, the method of the following non-patent document 2 can be used. [Non-patent Document 2] Development of a real-time surrounding environment recognition system using LiDAR for autonomous driving

[0044] In step S32, a reference point is determined from the result of step S31, and the coordinates of the robot 10 (which is in a stationary state) are calculated based on the reference point, and then the process proceeds to step S41. These coordinates may be defined as two-dimensional coordinates defined on the plane of the field F, such as Cartesian coordinates (x, y) or polar coordinates (r, θ) corresponding thereto.

[0045] The reference point is determined by selecting an object that satisfies the following two conditions from the object detection result in step S31 as the reference point. For the second condition, the object should be located within a threshold range from the position (and orientation) of the sensor 15. - It must be a cylinder or sphere that can acquire stable position information. The robot 10 is within a distance where the sensor 15 can detect it even if the robot 10 is moving.

[0046] If no object that satisfies the above criteria exists, landmarks in the environment may be determined as reference points. For landmark detection, for example, the existing method described in Non-Patent Document 3 below can be used. [Non-patent Document 3] Development of an autonomous robot equipped with a vision system

[0047] In the example EX1 of FIG. 6, an example in which a cylindrical object within the measurement range R of the sensor 15 of the robot 10 is set as a reference point rf is shown as an example of steps S31 and S32.

[0048] As formally shown by the repeat steps S41 and S49 surrounding them, steps S41 to S49 have a flow structure in which internal steps S42 to S48 are repeated until all combinations of the actuators 16 (four actuators A1, A2, A3, and A4 in the example of FIG. 5) and their respective rotation directions (forward or reverse) are processed. When there are generally n actuators 16, the number of combinations is as follows: n C i and the number of combinations of whether to select forward or reverse rotation for the selected i actuators is 2 i The sum of the products of for each i (i=1, 2, …, n-1, n) is calculated.

[0049]

number

[0050] In the example of n=4 in FIG. 5, each of the combinations is, for example, as follows: Rotate actuator A1 forward (→This example is EX3 in Figure 6.) Flip actuator A1 Rotate actuator A2 forward. Flip actuator A2 : Rotate actuator A1 forward and actuator A2 forward Reverse actuator A1 and forward actuator A2 :

[0051] The significance of providing these combinations is as follows: As described above, in a situation where there are n actuators 16, it is unknown how the movement mechanisms 17 are mechanically arranged and which of the n actuators 16 drives which of the movement mechanisms 17, so it is necessary to prepare comprehensive combinations such as those described above and perform a test run of the robot 10 for each combination.

[0052] In step S41, an incomplete combination of the selection of the actuator 16 (plurality of actuators 16) and the selection of the rotation direction is selected as the processing target, and then the process proceeds to step S42.

[0053] In step S42, the output of the sensor 15 is analyzed to measure the position (r0, θ0) of the reference point in polar coordinates, and then the process proceeds to step S43. This measurement can be performed using an existing method (Non-Patent Document 4), for example, as follows. [Non-Patent Document 4] Proposal of a LIDAR distance correction algorithm that takes into account robot movement

[0054] (1) Measure the distance r0 to the reference point from the LiDAR point cloud information. (2) Define the local coordinate system of the robot 10. Y-axis: The center of the sensor's acquisition range is the axis, and the direction of laser light irradiation is positive X-axis: The right direction is positive, perpendicular to the Y-axis (3) Measure the inclination θ0 with respect to the X-axis.

[0055] In step S43, the initial position (x0, y0) of the robot 10 in Cartesian coordinates (the initial position before the movement of the robot 10 is attempted as a trial run in step S45 described later) is calculated, and then the process proceeds to step S44. That is, in order to grasp the position of the robot 10 as viewed from the reference point (r0, θ0) obtained in step S42, the reference point (r0, θ0) can be replaced with the origin, and the initial position (x0, y0) can be calculated by the following numbers.

[0056]

number

[0057] Example EX2 in FIG. 6 is an example in which the initial coordinates (x0, y0) of the robot 10 in the Cartesian coordinate system are calculated from the results of measuring the polar coordinate position (r0, θ0) of the reference point rf in steps S42 and S43 above.

[0058] In step S44, for the combination of the actuator 16 and its rotation direction selected in step S41, a corresponding signal input port or the like is selected to actually input a drive control signal to the actuator 16 in that combination, and then the process proceeds to step S45.

[0059] In step S45, a test run of the robot 10 is attempted by driving the actuator 16 a fixed amount in accordance with the selection, and then the process proceeds to step S46. This drive may be a fixed amount, such as one rotation, in the selected rotation direction (forward or reverse). (Note that if multiple actuators 16 are selected, each of the actuators 16 is driven simultaneously in the corresponding rotation direction. Also, actuators 16 that are not selected among the multiple actuators may simply be allowed to idle without operating. In other words, even if drive control that applies a brake (braking force) is possible for the unselected actuators 16, a signal for the brake control may not be input.)

[0060] In step S46, the position (x1, y1) of the robot 10 is calculated as the position after the movement trial in step S45, and then the process proceeds to step S47. This calculation is performed by calculating the reference point position (r1, θ1) after the movement trial in the same manner as in step S42 before the movement trial, and then calculating the position (x1, y1) of the robot 10 after the movement trial using the following formula in the same manner as in step S43 before the movement trial. Example EX4 in Fig. 6 is an example of calculating the position (x1, y1) after this movement trial.

[0061]

number

[0062] In step S47, the action during the trial count is recorded, and the process proceeds to step S48. For example, the action may be recorded in the following format, with each piece of information associated with each other. Number of trials: k=1~Σ n C i ×2 i , (As mentioned above, take the sum "Σ" for i=1,2,…,n-1,n) Initial position: Reference point and initial position of the robot 10 Position after test run: Reference point and position of robot 10 after test run - Actuator drive mode: Rotation angle: a fixed value, e.g. one rotation Rotation direction: forward / reverse / freewheeling if not selected

[0063] The operation record can be recorded, for example, as data in JSON format, which is an existing method. Fig. 8 shows an example of an operation record in JSON format when n = 4 (in the case of the configuration of the example in Fig. 5).

[0064] In step S48, the actuator 16 that was rotationally controlled in step S45 is rotated in the opposite direction to that of step S45 (reverse rotation if it is forward rotation, or forward rotation if it is reverse rotation), and the robot 10 is then moved in the opposite direction to that of step S45, and the process proceeds to step S49. (For example, if the actuator A1 is rotated once in the forward direction in step S45, the actuator A1 is rotated once in the reverse direction in step S48.) The significance of step S48 is that when the robot 10 actually moves in the movement trial in step S45, the actuator A1 is returned to its original position (almost its original position) each time, thereby preventing the sensor 15 from moving far away from its original position while various movement trials are being continuously attempted, losing sight of an object or the like that serves as a reference point, or from colliding with an unmovable part such as a wall during the movement trial (a state in which an appropriate movement trial cannot be performed). In a situation in which such prevention is not necessary, step S48 may be omitted.

[0065] In step S49, the combination (Σ n C i ×2 i It is determined whether all of the combinations (methods) have been selected, and if all have been selected and the operation trials and their recording are complete, the process proceeds to step S51, and if any combinations are incomplete, the process returns to step S41 and processing continues for the incomplete combinations.

[0066] In step S51, the operation record (Σ n C i ×2 i As described above, if four types of basic movements, for example, forward, backward, right turn, and left turn of the robot 10, are selected and defined as basic movements, then all Σ n C i ×2 i From among the various control modes of the actuator 16, the optimum one can be selected as follows.

[0067] (1)…Select the basic forward movement (1A) Action M of robot 10 in trial number k k The following two conditions are used to determine the candidate forward movement. The following formula is used to determine whether the distance in the Y-axis direction has decreased. y1-y0<0 The change in distance along the X axis is within the tolerance th f The following formula is used to determine whether the value is within the range of the allowable judgment threshold. |x1-x0| <th f In addition, as disclosed in the above-mentioned Non-Patent Document 1 (Odometry that calibrates systematic errors based on a road surface environment map), even if the vehicle is successful in moving forward, the systematic error of the odometry remains, so the allowable value th f It is desirable to set a value close to 0.

[0068] (1B) Operation M k Distance traveled d k is calculated by the following formula. Note that, by using the judgment in (1A) above and this distance calculation, even if the diameter of the tires of the wheels constituting the moving mechanism 17 and the arrangement of the actuators 17 (and their corresponding relationships as control targets with respect to each of the moving mechanisms 17) are unknown, it is possible to narrow down the basic operations corresponding to forward movement (the drive methods of the actuators 17 that realize them).

[0069]

number

[0070] (1C) Action M selected from the candidates in (1A) above k In order to select the most effective basic forward movement from among the above, the following formula is used to select the movement M with the longest distance. f is selected as the advance.

[0071]

number

[0072] (2)…Select the basic movement to retreat (2A) Action M of robot 10 in trial number k k The following two conditions are met to select a candidate for the retreat action. The following formula is used to determine whether the distance in the Y-axis direction is increasing (the opposite judgment from the case of forward movement). y1-y0>0 The change in distance along the X axis is within the tolerance th f The following formula is used to determine whether the vehicle is within the range of the allowable judgment threshold. (Same judgment as for forward movement) |x1-x0| <th f

[0073] (2B) Operation M k Distance traveled d k is calculated using the same formula as (1B) for forward travel.

[0074] (2C) Action M, which is the candidate narrowed down in (2A) above k In order to select the most effective movement for the basic movement of retreat (similar to the case of forward movement), the movement M with the longest travel distance is selected using the following formula: b is chosen as the backslide.

[0075]

number

[0076] Examples EX5 and EX6 in FIG. 7 show examples of forward and backward movements, respectively, that were selected as being most effective using the above method.

[0077] (3)…Select the basic action for turning right (3A) Operation M k The amount of change in the posture of the robot 10 in k is calculated using the following formula: dθ k =θ1-θ0 (3B) In order to select the most effective motion for turning right as a basic motion, the amount of change in the posture of the robot 10, dθ, is calculated using the following formula: k(Right turn corresponds to positive) is the largest action M r is selected as the right turn.

[0078]

number

[0079] (4)…Select the basic action for turning left (4A) Operation M k The amount of change in the posture of the robot 10 in k is calculated using the same formula as (3A) for turning right. (4B) In order to select the most effective left turning motion as the basic motion, the amount of change in the posture of the robot 10, dθ, is calculated using the following formula: k (Right turn corresponds to positive) is the smallest action M l is selected as the left turn.

[0080]

number

[0081] Examples EX7 and EX8 in FIG. 7 show examples of right-turn and left-turn movements, respectively, that were selected as being most effective using the above method.

[0082] In step S52, the amount of change in the position of the robot 10 in each of the optimal basic movements selected in step S51 above (the amount of change in each component of the Cartesian coordinates (x, y) and the amount of change in the attitude θ in the polar coordinates (r, θ)) is calculated as follows, and then the process proceeds to step S53.

[0083]

number

[0084] In step S53, the calculated change amount for each basic movement is linked to a character string (a character string representing the content of the basic movement) for issuing an instruction command to the robot 10 to perform each basic movement as follows, and is transmitted to the server 50 and stored in the server 50.

[0085]

number

[0086] As described above, according to the behavior acquisition system 100 of this embodiment, even if the robot 10 has a plurality of actuators 16 as control targets and the specific individual conditions of the moving mechanism 17, which is the mechanical driving mechanism beyond the actuators 16, are unknown, it is possible to automatically acquire the driving mode of the actuators 16 corresponding to the basic behavior of the robot 10 by performing comprehensive behavior trials and selecting the optimal one from the results. Therefore, even in a situation where robots 10 with various specifications exist in various sites, it is possible to effectively control and manage the robot 10 without the effort of manually defining a control method according to the individual specifications of the robot 10.

[0087] Various supplementary, additional, and alternative aspects are described below.

[0088] (1) For example, the following is possible as an applied use of the above-mentioned embodiment of the present invention. That is, in a situation where robots 10 with various specifications exist at various sites, a common dongle 30 is distributed in advance and the person in charge at each site connects this dongle 30 to the robot 10, so that the server 50 as a platform can remotely control various robots 10 to execute tasks in a unified manner. Therefore, it is possible to eliminate the trouble of traveling, such as having a robot control engineer travel to each site to customize the control of the robot 10 each time to execute a task customized for each site, and therefore it is possible to reduce carbon dioxide emissions by saving energy resources required for human movement, and thus it is possible to contribute to Goal 13 of the Sustainable Development Goals (SDGs) led by the United Nations, "Take urgent action to combat climate change and its impacts."

[0089] (2) In the above embodiment, it is only known that the robot 10 is, for example, a vehicle-type robot, and comprehensive trial operations are attempted for the unknown actuators 16(s) and moving mechanism 17 under the premise that the actuators 16(s) include individual actuators for driving wheels. If the specifications of the robot 10 are narrowed down to a certain range in advance, the comprehensive trial operations may be limited to that range. For example, if the robot 10 is a vehicle-type robot, the above comprehensive trial operations are attempted for "k=1 to Σ n C i ×2 i In the above, the trial operations for each k times were performed for i=1, 2, ..., n. However, if it is known that the robot 10 as a vehicle-type robot has only three or four wheels, the trial operations may be limited to i=1, 2, 3, 4, etc.

[0090] (3) In the above embodiment of the configuration and flow in Fig. 2 to Fig. 4 etc., in order to acquire basic actions of the robot 10 in the automatic action acquisition system 100, a means is used in which the robot 10 is made to perform various trial actions using the connection of the dongle 30 as a trigger, and the results are analyzed. As another embodiment, instead of connecting the dongle 30 to the robot 10 and making it perform trial actions, the robot 10 may be made to perform various actual tasks (such as transporting goods, hereinafter referred to as "real tasks") in the field F (e.g., a store, a warehouse, etc.) as usual in the field F, and the action logs obtained in the real tasks are collected and analyzed in the server 50, thereby acquiring the basic actions of the robot 10.

[0091] FIG. 9 is a functional block diagram of the robot 10 (the robot 10 may be connected to a dongle 30) and the server 50 in the automatic action acquisition system 100 according to another embodiment, and FIG. 10 is a flowchart of the operation of the automatic action acquisition system 100 according to another embodiment. The robot 10 (or the dongle 30) includes a storage unit 27 that stores the action log of the robot 10, and the server 50 includes a learning unit 28 that acquires the basic action of the robot 10 by acquiring the stored action log and learning. In the embodiment of FIG. 9, the hardware configuration of the robot 10 and the server 50 is the same as that of FIG. 2, and the storage unit 27 can be realized by the first ROM 13 of the robot 10 (or the second ROM 33 of the dongle 30), and the learning unit 28 can be realized by the third processor 51 that loads a corresponding predetermined learning program into the third RAM 53 and executes it. Each step of FIG. 10 will be described below.

[0092] The significance and premise of using learning in this embodiment are as follows. <1> Since there are generally huge volumes of action logs, by learning it is possible to acquire basic actions as optimal solutions. <2> Learning is performed using the behavior log of a robot with the same specifications as a robot that has already acquired basic movements in the past. Learning can be used to absorb differences in basic movements due to manufacturing errors (slight errors in wheel position, diameter, etc.) of the target robot (which may have dongle 30 connected as a trigger for learning using the behavior log). <3> As a premise, when the robot to be learned moves (i.e., when the motion log is being collected), it uses only one of the basic motions.

[0093] In step S27, the robot 10 is caused to execute an actual task for a certain period of time in the field F, and an operation log at that time is collected and stored in the storage unit 27. The operation log is then transmitted to the learning unit 28 of the server 50 via the network NW, and the process proceeds to step S28. The transmission and reception of the operation log is performed by the first communication IF 14 of the robot 10 (or the second communication IF 34 of the dongle 30) and the third communication IF 54 of the server 50. The collection and transmission of the operation log in step S27, or the transmission of the collected operation log in step S27, may be triggered by the connection of the dongle 30 to the robot 10. In other words, the connection of the dongle 30 to the robot 10 may be triggered by the transmission of the collected operation log to the server 50, thereby starting learning in the server 50 (the next step S28).

[0094] 9, there is only one robot 10, but it is also possible to acquire operation logs from multiple robots 10 and transmit all of them to the learning unit 28 of the server 50. (In this case, the specifications of the multiple robots 10 are the same.)

[0095] The operation log is acquired as including information similar to that of the embodiment in Figures 2 to 4 (this is the first embodiment, and another embodiment in Figures 9 and 10 (and Figure 11 described later) is the second embodiment). That is, the trial operation at each time k in the first embodiment is replaced with an individual operation of the actual task (an individual operation as a result of a single control command to the actuator 16), and an operation log L={Log(k)|k=1, 2, 3, ...} is acquired as information linked as follows: ·Individual operation: k=1,2,3,… Initial position before this individual movement: the reference point and the initial position of the robot 10 Position after this individual motion: the reference point and the position of the robot 10 after the individual motion - Actuator drive mode: Rotation angle: a fixed value, e.g. one rotation Rotation direction: forward / reverse / freewheeling if not selected

[0096] In step S28, the learning unit 28 learns using the action log L collected in step S27 as described above, thereby acquiring a basic action, and the flow in FIG. 10 ends. Since the action log L generally exists in a huge amount, learning can acquire a basic action as an optimal solution. Note that the same processing as in the first embodiment (the same processing as in the basic action selection unit 25) may be performed on the action log L to acquire a basic action based on a rule base. FIG. 11 is a flowchart showing the details of the learning by the learning unit 28 according to one embodiment (i.e., the details of step S28 in FIG. 10). Each step in FIG. 11 will be described below. Note that, regarding each of steps S281 to S285 in FIG. 11, the subject of the operation is the learning unit 28 (the third processor 51 of the server 50 which realizes the learning unit 28 by executing a predetermined learning program), so a description of this subject of the operation will be omitted, and only the processing content in each step will be described.

[0097] As an explanatory example, the specifications and basic motions of the robot 10 to be learned are assumed to be the same as those in the explanatory example in the first embodiment. That is, the robot 10 to be learned is a four-wheeled vehicle-type robot as shown in Fig. 5, and has four basic motions: forward, backward, right turn, and left turn.

[0098] In step S281, the amount of change is calculated from the motion log L (the motion log L collected in step S27 in FIG. 10), and the process proceeds to step S282. Specifically, the amount of change d Log(k) in the kth motion of the robot 10 is calculated from the individual motion logs Log(k) for each time k=1, 2, 3, ... as follows: (Note that as described above, the first and second embodiments are common explanatory examples, and therefore the variables in the following explanations are the same as those explained in the first embodiment using the formulas 2 to 10 and FIGS. 6 and 7. In other words, new variables are defined from the variables already explained.)

[0099]

number

[0100] At the stage of step S281, since it is unknown to which basic action the action log belongs, the amounts of change dx, dy, and dθ for all variables are calculated as described above.

[0101] In step S282, the change amount d Log(k) of the action log L is clustered by unsupervised learning, and the cluster center of each cluster is obtained, and then the process proceeds to step S283. For example, the k-means method or the like may be used as the unsupervised learning for achieving clustering. If the number of clusters needs to be set in advance, it may be set. For example, since there are four basic actions in this example, the number of clusters may be set to four. Any existing method may be used to obtain an appropriate clustering result using the k-means method, and for example, the initial value may be set using the method of the following non-patent document 5. [Non-Patent Document 5] Takashi Onoda; Miho Sakai; Seiji Yamada. Comparison of the performance of the k-means method using different initial value setting methods. In: Proceedings of the 27th Fuzzy System Symposium, Japan Society for Fuzzy Theory and Intelligent Informatics, Japan Society for Fuzzy Theory and Intelligent Informatics, 2011. p. 55-55.

[0102] Each cluster in the clustering result is C i (i=1, 2, ...) (in this example, there are four clusters C1, C2, C3, C4), and the cluster center dL Ci can be calculated as follows: |C i | is cluster C i is the number of elements in

[0103]

number

[0104] In step S283, each cluster C i Then, the process proceeds to step S284. Specifically, the basic actions defined in advance as templates and each cluster C i Cluster center dL Ci In this example, the template of the basic motion is the basic motion M obtained in step S53 of FIG. 4 by applying the first embodiment to a robot of the same specifications. f ,M b ,M r ,M l The amount of change dM f ,dM b ,dM r ,dM l Since the template is obtained, each cluster center dL Ci (i=1,2,3,4) can be matched as follows:

[0105]

number

[0106] In this explanatory example, it is assumed that each of the four clusters is matched with each of the basic actions as follows.

[0107]

number

[0108] In step S284, the action log with the maximum amount of change is selected within each cluster matched with the basic action, and then the process proceeds to step S285. In this illustrative example, the selection can be made as follows.

[0109]

number

[0110] The selection method in the above example is the same as the method for selecting the most effective operation described in the first embodiment. k = y1-y0. For example, the forward movement is (1A) and as mentioned above, "y1-y0<0" (i.e., dy k <0), the Y-axis direction shrinks, and the negative value of dy k The smallest value of dy corresponds to the largest distance forward. On the other hand, retreat is the opposite, and as mentioned above, dy is a positive value as (2A). k The case where is maximum corresponds to the greatest distance retreat.) That is, a basic action that is optimal in accordance with a template prepared in advance (a template that is configured with the definition of each basic action and the amount of change (movement of the robot 10) in each basic action and is configured with information output in step S53 of FIG. 4) may be selected in step S284 of the second embodiment. (That is, the selection in step S284 in the second embodiment is performed by selecting the basic action that is optimal in accordance with the template prepared in advance (a template that is configured with the definition of each basic action and the amount of change (movement of the robot 10) in each basic action and is configured with information output in step S53 of FIG. 4). iThe action log may be selected in the same manner as the processing of the basic action selection unit 25 in the first embodiment. For example, in the present illustrative example, when selecting actions that do not involve movement in the x-axis direction with respect to forward and backward movements, the judgment based on the tolerance (the above-mentioned tolerance th f It is also possible to narrow down the selection targets by adding a determination that the number of pixels falls within the range of the acceptable determination threshold.

[0111] In step S285, the action log with the maximum amount of change selected in step S284 above is saved as a basic action, and the flow in Fig. 11 ends. That is, the result output and saving in step S285 are similar (in form) to those in step S53 in Fig. 4, and the amount of change in the action log is linked to a character string for issuing an instruction to the robot 10 and saved. In this explanatory example, four basic actions are saved as follows.

[0112]

number

[0113] (4) As described above, the first embodiment may be implemented for a certain robot 10A to obtain information regarding its basic operations, and the second embodiment may be implemented for another robot 10B having common specifications with the robot 10A to learn optimal control commands for realizing the basic operations from the operation log.

[0114] For example, both robots 10A and 10B are four-wheeled vehicle-type robots as shown in Fig. 5, and the tire sizes of the four wheels W1, W2, W3, and W4 of the robot 10A are all equal (for example, radius r), and the specifications of the robot 10B are common (commonly arranged actuators A1, A2, A3, and A4). However, when an administrator replaces the tires to suit the usage environment, the left and right front wheels W1 and W2 are slightly larger (for example, radius 1.2r) and the left and right rear wheels W3 and W4 are slightly smaller (for example, radius 0.8r), the optimal control mode for realizing the basic operations of forward movement, backward movement, right turn, and left turn may be different between the robots 10A and 10B. When the moving mechanism is more complicated (for example, in the case of a vehicle-type robot with six or eight wheels), it is assumed that the optimal control mode for realizing the basic operations may be different even between the robots 10A and 10B with common specifications. The first and second embodiments of the present invention can deal with such cases. [Explanation of symbols]

[0115] 100...automatic motion acquisition system, 10...robot, 30...dongle, 50...server, 20...control unit, 21...preparation unit, 22...basic motion acquisition unit, 27...storage unit, 28...learning unit

Claims

1. A method for automatically acquiring instructions for a plurality of actuators for performing basic operations of a robot that moves using a moving mechanism driven by the plurality of actuators, comprising: A basic motion of another robot having specifications common to the robot and instructions to a plurality of actuators for causing the other robot to perform the basic motion are already known as a template; acquiring an operation log of the robot by linking the presence or absence of a control input to each of the plurality of actuator elements, designation information of a signal type of the control input when a control input is present, and movement information of the robot before and after an individual operation performed by the robot by performing the control input to the actuator based on the designation information (S27); learning (S28) using the operation log and the template, automatically acquiring designation information regarding the presence or absence of a control input to each of the plurality of actuator elements and, when a control input is present, a signal type of the control input, as an optimum one for realizing the basic operation; 11. The method according to claim 10, wherein the movement mechanism is a wheel, and a signal type of a control input to each of the plurality of actuator elements includes a signal type for forward or reverse rotation of a corresponding wheel via drive control of the actuator element.

2. The method according to claim 1, characterized in that the learning (S28) includes classifying the operation logs into clusters corresponding to each of the basic operations included in the template (S281, S282), and selecting an optimal operation log for realizing the basic operation from the operation logs for each classified cluster (S284).

3. The method according to claim 1 or 2, characterized in that the classifying into the clusters (S281, S282) includes classifying each individual action constituting the action log by unsupervised learning based on movement information of the robot before and after the individual action (S282).

4. 4. The method according to claim 1, wherein the connection of a dongle to the robot (S1) is used as a trigger to execute a method for automatically acquiring an instruction for the actuator.

5. 3. The method according to claim 2, wherein the basic movements include forward, backward, right turn, or left turn among the movements of the robot on a plane.

6. When selecting the basic motion as forward movement, backward movement, right turn, or left turn (S284, S285), Selecting, from among the operation logs for each cluster, one determined to have a moving distance corresponding to forward movement, a moving distance corresponding to backward movement, a posture change amount corresponding to a right turn, or a posture change amount corresponding to a left turn; 6. The method according to claim 5, wherein when there are a plurality of selected movements among forward movement, backward movement, right turn, or left turn, the movement that has the largest corresponding movement distance or attitude change amount is selected as the most effective movement.

Images