Control device, control method, and control program

Abstract

To achieve both safety and accessibility for the controlled object, depending on the characteristics of the task in which the controlled object operates. [Solution] The control device is characterized by comprising: a storage unit that stores a preference distribution of observations for each task target on which the controlled object operates, and obstacle degree information indicating the degree to which the task target is an obstacle; an estimation unit that estimates which task target an object in the surrounding environment is a task target based on observation data from a sensor that observes the surrounding environment of the controlled object and the preference distribution of observations; a calculation unit that calculates an obstacle spectrum indicating the degree of obstacle of the estimation result by the estimation unit based on the obstacle degree information; a generation unit that generates the action of the controlled object based on the estimation result; and a planning unit that plans control data for controlling the controlled object based on the obstacle spectrum calculated by the calculation unit and the action generated by the generation unit.

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, and a control program for controlling a controlled object. [Background technology]

[0002] Collaborative robots are robots that work in the same physical space as humans, and there are high expectations for them to complement the human workforce. However, balancing safety and accessibility (how easily collaborative robots can approach humans) remains a challenge.

[0003] Conventional collaborative robots are equipped with safety features defined by the International Organization for Standardization (ISO) standards. Conventional collaborative robots have achieved both safety and accessibility by detecting obstacles during operation and stopping or slowing down, or by stopping when the collaborative robot collides with an obstacle.

[0004] Patent Document 1 below discloses a robot that avoids contact with humans when they approach. This robot is mounted on a robot equipped with a drive unit and comprises a sensor unit that detects obstacles present around the robot and a control unit that controls the robot's drive. The control unit comprises a spatial distance calculation unit that calculates the spatial distance between the robot and the obstacle through the sensor unit, a virtual external force calculation unit that calculates a virtual external force by considering the effect of an obstacle approaching the robot as a virtual external force based on the spatial distance, an operating speed suppression unit that suppresses the programmed operating speed of the robot according to the external force by considering the effect of an obstacle approaching the robot as a virtual external force based on the spatial distance, and an operation adjustment unit that adjusts the operation of the drive unit based on the virtual external force to avoid contact between the robot and the obstacle. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2023-132333 [Overview of the project] [Problems that the invention aims to solve]

[0006] Collaborative robots are expected to be used in more flexible scenarios in the future. In cases where the robot's movements and obstacles are not predefined, accessibility is lost. Specifically, if the way obstacles are handled differs depending on the object (even among obstacles, there are differences in degree, such as those that must be avoided, those that can be approached, and those that can be touched), conventional obstacle detection methods (0 or 1 for whether or not an obstacle exists) cannot cope. Also, if the robot operates adaptively according to the situation (planning and executing a trajectory in real time), it cannot detect obstacles during trajectory planning and generate a trajectory. In such more flexible cases than before, it is necessary to achieve both safety and accessibility. Furthermore, in the above-mentioned Patent Document 1, the generated robot movements are limited to spatial distance.

[0007] The present invention aims to achieve both safety and accessibility for the controlled object, depending on the characteristics of the task in which the controlled object operates. [Means for solving the problem]

[0008] A control device according to one aspect of the disclosed technology is characterized by comprising: a storage unit that stores a preference distribution of observations for each task target on which the controlled object operates, and obstacle degree information indicating the degree to which the task target is an obstacle; an estimation unit that estimates which task target an object in the surrounding environment is a task target based on observation data from a sensor that observes the surrounding environment of the controlled object and the preference distribution of observations; a calculation unit that calculates an obstacle spectrum indicating the degree of obstacle of the estimation result by the estimation unit based on the obstacle degree information; a generation unit that generates the action of the controlled object based on the estimation result; and a planning unit that plans control data for controlling the controlled object based on the obstacle spectrum calculated by the calculation unit and the action generated by the generation unit. [Effects of the Invention]

[0009] According to a representative embodiment of the present invention, it is possible to achieve both safety and accessibility of the controlled object in accordance with the characteristics of the task target on which the controlled object operates. Problems, configurations and effects other than those described above will be made clear by the following description of the examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] [Figure 1] FIG. 1 is an explanatory diagram showing an example of robot arm control according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a control device. [Figure 3] FIG. 3 is an explanatory diagram showing an example of a task target table. [Figure 4] FIG. 4 is an explanatory diagram showing an example of an obstacle degree information table. [Figure 5] FIG. 5 is an explanatory diagram showing an example of an index determination table. [Figure 6] FIG. 6 is an explanatory diagram showing an example of a selection table. [Figure 7] FIG. 7 is a block diagram showing an example of the functional configuration of a control device. [Figure 8] FIG. 8 is an explanatory diagram showing an example of a calculation result of a probability distribution. [Figure 9] FIG. 9 is an explanatory diagram showing an example of a calculation result of an obstacle spectrum. [Figure 10] FIG. 10 is an explanatory diagram showing a first example of trajectory planning. [Figure 11] FIG. 11 is an explanatory diagram showing a second example of trajectory planning. [Figure 12] FIG. 12 is an explanatory diagram showing an example of a control operation processing procedure for a robot arm performed by the control device according to the first embodiment. [Figure 13] FIG. 13 is an explanatory diagram showing an example of a control operation processing procedure for a robot arm performed by the control device according to the second embodiment. [Figure 14] FIG. 14 is an explanatory diagram showing a first example of cart robot control according to the third embodiment. [Figure 15]Figure 15 is an explanatory diagram showing an example of trolley robot control 2 according to Embodiment 3. [Figure 16] Figure 16 is an explanatory diagram showing an example of a trajectory plan according to Embodiment 3. [Modes for carrying out the invention] [Examples]

[0011] In Example 1, we will explain an example in which a robot arm, which is an example of an actuator, is controlled to detect and avoid obstacles in a task of grasping an object. The position of the robot arm is fixed.

[0012] <Figure 1: Example of robot arm control> Figure 1 is an explanatory diagram showing an example of robot arm control according to Embodiment 1. The control system 100 includes a control device 101, a robot arm 102, and a sensor 103. The control device 101, the robot arm 102, and the sensor 103 are connected to each other via a network 104 (whether wired or wireless), such as the Internet, LAN (Local Area Network), or WAN (Wide Area Network).

[0013] The control device 101 is a computer that controls the robot arm 102. The robot arm 102 is the object controlled by the control device 101. Specifically, for example, the robot arm 102 is an actuator that performs the task of grasping an object based on control data from the control device 101. The robot arm 102 has multiple joints that operate on six axes and grasps objects with its end effector. The sensor 103 detects the surrounding environment 105 of the robot arm 102.

[0014] Sensor 103 is, for example, a camera that captures images of objects in the surrounding environment 105 and determines the three-dimensional position of the objects in the camera coordinate system. Sensor 103 may also include an infrared sensor. In other words, it is not limited to a camera as long as it can detect objects in the surrounding environment 105. Sensor 103 may also be provided on the robot arm 102.

[0015] Here, item O is a non-obstacle object to be grasped by the robot arm 102. Item O is positioned within reach of the robot arm 102 (for example, at an approach distance of 0 cm from the robot arm 102: "Contactable (0 cm approach)"). Person H1 is a worker engaged in the operation of the robot arm 102 and is always positioned around the robot arm 102.

[0016] In conventional technology, in the surrounding environment 105 of the robot arm 102, under the predefined assumption that the robot arm 102 will perform the task of grasping and moving an object O, the object O is set as a non-obstacle object (=task target), and the person H1 is set as an obstacle that is not predefined as a task target.

[0017] In contrast, in Example 1, the control device 101 estimates the obstacle spectrum in the surrounding environment 105 of the robot arm 102. The obstacle spectrum is a continuous value estimated based on the task target and real-time observation data from the sensor 103.

[0018] As a result, in a task in which the robot arm 102 grasps an object O, the object O, which is the target of the task, is grasped by the robot arm 102 and is therefore determined to be "contactable (0cm approach)". In addition, a person H1, who is a worker constantly present around the robot arm 102, is determined to be "contactable (20cm approach)" by the control device 101 predicting the movements of person H1.

[0019] On the other hand, person H2 is a person who suddenly entered the surrounding environment 105 by running from outside the surrounding environment 105. The control device 101 predicts the movements of person H2 and determines that "avoidance is necessary (approaching robot arm 102 at a distance of 200 cm): 'avoidance is necessary (approaching at 200 cm)'".

[0020] <Figure 2: Example of hardware configuration of control device 101> Figure 2 is a block diagram showing an example of the hardware configuration of the control device 101. The control device 101 includes a processor 201, a storage device 202, an input device 203, an output device 204, and a communication interface (communication IF) 205. The processor 201, storage device 202, input device 203, output device 204, and communication IF 205 are connected by a bus 206. The processor 201 controls the control device 200. The storage device 202 serves as the work area for the processor 201. The storage device 202 is also a non-temporary or temporary recording medium that stores various programs and data. Examples of storage devices 202 include ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and flash memory. The input device 203 inputs data. Examples of input devices 203 include a keyboard, mouse, touch panel, numeric keypad, scanner, microphone, and sensor. The output device 204 outputs data. Output devices 204 include, for example, displays, printers, and speakers. The communication IF 205 connects to the network 104 and sends and receives data.

[0021] For example, the control device 101 receives observation data from the sensor 103 via the communication IF 205 and transmits control data to the robot arm 102.

[0022] <Figure 3 Task Objective Table> Figure 3 is an explanatory diagram showing an example of a task objective table. The task objective table 300 includes modalities 301 and observation preference distributions 302.

[0023] Modality 301 indicates the type of task object, such as item O or person H1. The observation preference distribution 302 is the set of random variables P1 to Pn that are observed for each task object defined by modality 301. Random variables P1 to Pn are, for example, the positions observed by sensor 103. If sensor 103 is a fixed-point observation camera, then each of random variables P1 to Pn will be a fixed position in the 3D coordinate space in the camera coordinate system.

[0024] The preference distribution 302 of observations for each modality 301 is represented by a one-hot vector where, among the random variables P1, P2, ..., Pn, there is one position where the value is "1" and the values ​​of the remaining positions are "0". For example, since the value of item O is "1" only at position P2, it can be said that it is desirable for item O to be at position P2. Similarly, since the value of person H1 is "1" only at position Pn, it can be said that it is desirable for item O to be at position Pn.

[0025] Furthermore, the one-hot vectors of positions P1 to Pn, which represent the preference distribution 302 of observations for each modality 301, are referred to as the task objectives.

[0026] <Figure 4 Obstacle Level Information Table> Figure 4 is an explanatory diagram showing an example of an obstacle degree information table. The obstacle degree information table 400 is a table that manages the degree of obstacles. The obstacle degree information table 400 is stored in the storage device 202 of the control device 101. The obstacle degree information table 400 has an index 401. The index 401 is a measure that shows how the obstacle spectrum b is perceived, that is, the degree of obstacles of modality 301, and has modality 301 and obstacle degree 402. The obstacle degree 402 indicates the degree to which the task target is an obstacle. Specifically, for example, the obstacle degree 402 indicates the boundary between whether the task target defined by modality 301 is an obstacle or not.

[0027] For example, index d1 indicates the degree to which the robot arm 102 should avoid its modality 301. In the case of index d1, the obstacle degree 402 is the approach distance from the robot arm 102, i.e., the distance at which the modality 301 may approach the robot arm 102. For example, the obstacle degree 402 of item O is "0.00", so item O may come into contact with the robot arm 102, and the obstacle degree 402 of person H1 is "0.20", so person H1 may approach the robot arm 102 to within 0.20 [m].

[0028] Furthermore, index d2 indicates the operating speed of the robot arm 102. For example, a trajectory with a velocity condition is generated in which the velocity changes along the shortest path, such as a straight line.

[0029] Furthermore, index d3 indicates the flexibility of modality 301. For example, if the surface of an object is covered with a soft material or if the moving parts of a machine are passive, the object or machine will not malfunction even if it is pushed slightly. In such cases, the degree of obstacle 402 can be considered as the expected value of how modality 301 will avoid obstacles (how the observation of modality 301 changes) in response to the movement of the robot arm 102.

[0030] Furthermore, index d2 indicates the operating speed of the robot arm 102 and the flexibility of the modality 301. For example, if the surface of an object is covered with a soft material or if the moving parts of a machine are passive, pushing the object or machine slightly will not cause damage. In such cases, the degree of obstacle 402 can be considered as the expected value of how much the modality 301 will avoid obstacles (how the observation of modality 301 changes) in response to the movement of the robot arm 102.

[0031] <Figure 5: Indicator Determination Table> Figure 5 is an explanatory diagram showing an example of an index determination table. The index determination table 500 is a table that is referenced when determining the index 401. The index determination table 500 is stored in the storage device 202 of the control device 101.

[0032] The indicator determination table 500 contains actions 501 and indicators 401. Action 501 is the movement that the robot arm 102 should perform and is generated by the control device 101. For example, if the task is for the robot arm 102 to move an item O from position P2 to position P3, there are several possible actions, such as "grasp the item O at position P2", "move the grasped item O to position P3", "release the item O at position P", and "if the item O is not at position P2, stop in place". By referring to the indicator determination table 500, the indicator 401 for the task corresponding to the action 501 is identified.

[0033] <Figure 6 Selection Table> Figure 6 is an explanatory diagram showing an example of a selection table. The selection table 600 is a table referenced when selecting a trajectory planning algorithm. The selection table 600 is stored in the memory device 202 of the control device 101.

[0034] The selection table 600 has an index 401 and a trajectory planning algorithm column 601. The trajectory planning algorithm column 601 is a column that shows the priority of the trajectory planning algorithms. The leftmost column has the highest selection priority, and the rightmost column has the lowest selection priority. The trajectory planning algorithm is an algorithm that plans the trajectory of the robot arm 102 in order to accomplish the task, and is selected and executed by the control device 101.

[0035] Orbit planning algorithms include, for example, sampling-based orbit planning algorithms such as Rapidly-Exploring Random Tree (RTT) and Probabilistic Roadmap Method (RPM); orbit planning algorithms based on optimal control and numerical optimization such as Model Predictive Control (MPC) and Linear Quadratic Regulator (LQR); orbit planning algorithms based on geometric methods such as Davins curves, Reed-Schepp curves, and Bayesian curves; orbit planning algorithms based on machine learning such as deep reinforcement learning and imitation learning; and orbit planning algorithms based on task-space orbit planning such as inverse kinematics and methods using Jacobian matrices.

[0036] <Figure 7: Example of the functional configuration of the control device 101> Figure 7 is a block diagram showing an example of the functional configuration of the control device 101. The control device 101 includes a storage unit 700, an estimation unit 701, a calculation unit 702, a generation unit 703, and a planning unit 704. The storage unit 700 stores a task target table 300, an obstacle degree information table 400, an indicator determination table 500, and a selection table 600. Specifically, the storage unit 700 is implemented, for example, by the storage device 202 shown in Figure 2.

[0037] Furthermore, the estimation unit 701, calculation unit 702, generation unit 703, and planning unit 704 are specifically realized, for example, by causing the processor 201 to execute a program stored in the memory device 202 shown in Figure 2.

[0038] The estimation unit 701 uses the observed data o at time t output from the sensor 103. t It estimates the task target within and outputs the estimation result to the calculation unit 702 and the generation unit 703. Specifically, for example, the estimation unit 701 uses the observed data o at time t from the sensor 103. t The recognized object is obtained from the observation data o. tIf the input is image data, the estimating unit 701 detects, through image recognition, a subject and its position information (for example, the centroid position of the subject in a camera coordinate system) from the image data as a recognized object.

[0039] The estimating unit 701 receives observation data o at time t t as input, and calculates a probability distribution p for each recognized object through Bayesian estimation t (m|o t ). Note that the estimating unit 701 is not limited to Bayesian estimation, and may calculate the probability distribution p for each recognized object using a discrimination model configured by a convolutional neural network t (m|o t ). This discrimination model is a model in which the relationship between observation data o_t and a task target m has been learned in advance.

[0040] [Figure 8 Probability distribution p t (m|o t )] Figure 8 is an explanatory diagram showing an example of a calculation result of a probability distribution p t (m|o t ). In the calculation result 800 of the probability distribution p t (m|o t ), a recognized object ID 801 is identification information (T1, T2, ..., Tk) that uniquely identifies a recognized object obtained from the observation data o t . k is an integer of 1 or greater. Note that a recognized object whose recognized object ID 801 is Tk may be referred to as a recognized object Tk.

[0041] The probability distribution p t (m|o t ) is a probability indicating, for each recognized object Tk, which task target m defined by the modality 301 the recognized object Tk corresponds to. For example, the probability that the recognized object T1 is the article O is "0.20", and the probability that it is the person H1 is "0.70". Furthermore, the probability that the recognized object T2 is the article O is "0.80", and the probability that it is the person H1 is "0.05".

[0042] For example, observation data o at time t tIn this case, the centroid of the recognized object T1 is assumed to be closest to position Pn among positions P1 to Pn. Also, in calculation result 800, the probability distribution of the recognized object T1 is p t (m|o t In this case, the task target m with the highest probability is person H1. Therefore, the estimation unit 701 uses the observed data o at time t. t In this case, the recognized object T1 located at position Pn is identified as a person H1.

[0043] Similarly, observation data o at time t t In this case, the centroid of the recognized object T2 is assumed to be closest to position P2 among positions P1 to Pn. Also, in calculation result 800, the probability distribution of the recognized object T2 is p t (m|o t In this case, the task target m with the highest probability is item O. Therefore, the estimation unit 701 determines that the observed data at time t is t In this case, the recognized object T2 located at position P2 is identified as item O.

[0044] The same process is applied to the recognized objects T3 to Tk to estimate the task target m corresponding to each recognized object T3 to Tk.

[0045] Returning to Figure 7, the calculation unit 702 calculates the obstacle spectrum b and outputs it to the planning unit 704. Specifically, for example, the calculation unit 702 refers to the obstacle degree information table 400 to identify the estimated task target m from the modality 301 and obtains the obstacle degree 402 corresponding to the identified task target m. The value of the obtained obstacle degree 402 is denoted as pd (hereinafter sometimes referred to as obstacle degree pd).

[0046] For example, if the current index 401 is b1, the calculation unit 702 reads "0.20" from the obstacle degree information table 400 as the value pd for the obstacle degree 402 where person H1 is located. The calculation unit 702 also reads "0.00" as the value pd for the obstacle degree 402 where modality 301 is item O.

[0047] Then, the calculation unit 702 multiplies the probability that the recognized object T1 is person H1 ("0.70") by the value of the obstacle degree 402 for modality 301 being person H1 (pd = "0.20") to calculate the obstacle spectrum b = 0.14 for person H1. Similarly, the calculation unit 702 multiplies the probability that the recognized object T2 is item O ("0.80") by the value of the obstacle degree 402 for modality 301 being item O (pd = "0.00") to calculate the obstacle spectrum b = 0.00 for item O.

[0048] [Figure 9 Obstacle Spectrum b] Figure 9 is an explanatory diagram showing an example of the calculation result of the obstacle spectrum b. Note that the obstacle spectrum b is calculated similarly for task target m, which corresponds to recognized objects T3 to Tk.

[0049] Returning to Figure 7, the generation unit 703 generates the observed data o at time t. t and the probability distribution p of the recognized object Tk t (m|o t Based on this, the action of the robot arm 102 at time t is a t It generates and outputs to the planning unit 704 and the calculation unit 702. Specifically, for example, the generation unit 703 selects from a plurality of pre-set actions 501 the action a of the robot arm 102 that should be performed at time t. t Select one of the following. For example, if the task for the robot arm 102 is to move an item O from position P2 to position P3, the multiple possible actions 51 include "grasp the item O at position P2", "move the grasped item O to position P3", "release the item O at position P", and "if the item O is not at position P2, stop in place".

[0050] Therefore, for example, observation data o at time t t If an object T2 is recognized and located at position P2, then the object T2 is presumed to be article O.

[0051] The planning unit 704 plans the trajectory of the robot arm 102. Specifically, for example, the planning unit 704 refers to the selection table 600 and selects a trajectory planning algorithm corresponding to the current index 401 from the trajectory planning algorithm column 601. In this case, the unselected trajectory planning algorithm with the highest priority is selected. For example, if the current index 401 is b1, trajectory planning algorithm c1 is selected. Since index d1 is the degree to which something should be avoided, trajectory planning algorithm c1 that utilizes gradients, such as stochastic gradient descent, is selected.

[0052] The planning unit 704 may fix the index 401 and select a trajectory planning algorithm corresponding to the index 401 from the trajectory planning algorithm sequence 601, or it may select a trajectory planning algorithm corresponding to the index 401 that has been changed each time from the trajectory planning algorithm sequence 601.

[0053] The planning unit 704 uses the selected trajectory planning algorithm to determine action a from the generation unit 703 based on the obstacle spectrum b. t The robot arm 102 generates trajectory data as control data to achieve this. The trajectory data is, for example, the time-series change in the six axes of the robot arm 102. The planning unit 704 outputs the generated trajectory data to the robot arm 102. As a result, the robot arm 102 performs an action a according to the received trajectory data. t Execute this.

[0054] Furthermore, the calculation unit 702 receives instructions from the generation unit 703 regarding action a t The calculation unit 702 obtains the obtained action 501(a t An index 401 corresponding to ) may be determined. The calculation unit 702 then obtains the degree of obstacle 402 corresponding to the determined index 401 for each task target m of modality 301. In this way, the latest action a t The degree of obstacle 402 can be automatically changed in response to a change in the corresponding index 401. In this case, the calculation unit 702 outputs the latest index 401 to the planning unit 704.

[0055] Furthermore, in the calculation unit 702 and the planning unit 704, it is possible to set in advance by the user whether to fix or vary the index 401.

[0056] <Figure 10 Example of track plan 1> Figure 10 is an explanatory diagram showing trajectory planning example 1. Trajectory planning example 1 is a trajectory planning example where index 401 is index d1 (degree to avoidance). The higher the value of obstacle spectrum b, the more the planning unit 704 generates trajectory data R1 for the robot arm 102 that avoids such areas within the surrounding environment 105. In this case, for example, a trajectory planning algorithm that performs a general pathfinding using gradients, such as stochastic gradient descent, is selected, and obstacle spectrum b is applied to the gradient to generate trajectory data R1. Note that the contents shown in Figure 10 may also be displayed on a display device, which is an example of an output device 204 of the control device 101.

[0057] <Figure 11 Example of track plan 2> Figure 11 is an explanatory diagram showing trajectory planning example 2. Trajectory planning example 2 is a trajectory planning example when index 401 is index d2 (operating speed of the robot arm 102). The higher the value of the obstacle spectrum b, the more the planning unit 704 generates trajectory data R2 that represents the shortest path for the robot arm 102 operating at a low speed within the area of ​​the surrounding environment 105. The arrowheads on the trajectory data R2 represent the operating speed of the robot arm 102. That is, the larger the arrowhead, the faster the operating speed. Note that the contents shown in Figure 11 may also be displayed on a display device, which is an example of an output device 204 of the control device 101.

[0058] <Figure 12 Control Operation> Figure 12 is an explanatory diagram showing an example of the control operation processing procedure for the robot arm 102 by the control device 101 according to Embodiment 1. The vertical axis shows the passage of time, the arrows indicate data transmission and reception, and the rectangles indicate processing.

[0059] (Step S1201) The control device 101 receives the observed data o at time t from the sensor 103.t Upon receiving the data, the estimation unit 701 estimates the task target m by referring to the task target table 300. The task target m and its location information are output as estimation results to the calculation unit 702 and the generation unit 703.

[0060] (Step S1202) The control device 101, using the calculation unit 702, calculates the obstacle spectrum b for the current index 401 by referring to the obstacle degree information table 400 based on the estimation results from the estimation unit 701. The obstacle spectrum b is output to the planning unit 704.

[0061] (Step S1203) The control device 101, using the calculation unit 702, refers to the index determination table 500 and generates the action a at time t from the generation unit 703. t The corresponding index 401 is determined. The determined index 401 is output to the planning unit 704. If index 401 is fixed, step S1203 is not executed.

[0062] The control device 101 receives the observed data o t Each time a signal is received, steps S1201 to S1203 will be executed repeatedly.

[0063] (Step S1204) The control device 101 uses the generation unit 703 to determine the action a at time t based on the estimation result from the estimation unit 701. t Generates the action a at time t. t This is output to the calculation unit 702 and the planning unit 704.

[0064] (Step S1205) The control device 101, via the planning unit 704, refers to the selection table 600 and selects a trajectory planning algorithm corresponding to the index 401.

[0065] (Step S1206) The control device 101 executes trajectory planning using the trajectory planning algorithm selected in step S1205, via the planning unit 704. The trajectory data obtained from the trajectory planning is transmitted to the robot arm 102.

[0066] The control device 101 will repeatedly execute steps S1204 to S1206 each time it obtains an estimation result from the estimation unit 701.

[0067] (Step S1210) The robot arm 102 performs action a at time t according to the trajectory data. t Execute the action a at time t. t It operates as intended. The control device 101 will repeatedly execute step S1210 each time it receives trajectory data from the planning unit 704.

[0068] Thus, according to Example 1, it is possible to achieve both safety and accessibility of the robot arm 102 depending on the characteristics of the task target m of the task in which the robot arm 102 operates. [Examples]

[0069] Example 2 describes an example of determining the success or failure of a trajectory plan in Example 1. In Example 2, the focus is on explaining the differences from Example 1, so the same reference numerals are used for components identical to those in Example 1, and the explanation of redundant parts is omitted.

[0070] <Figure 13 Control Operation> Figure 13 is an explanatory diagram showing an example of the control operation processing procedure for the robot arm 102 by the control device 101 according to Embodiment 2.

[0071] (Step S1307) After the trajectory plan is executed in step S1206, the control device 101, using the planning unit 704, determines whether the trajectory plan was successful. If it is determined that the trajectory plan was successful (step S1307: Yes), the trajectory data is transmitted to the robot arm 102.

[0072] If the trajectory planning algorithm is unable to generate a trajectory plan, the trajectory planning fails. For example, if the indicator is the degree to which an object should be avoided, and the robot arm 102 is surrounded in all directions by the recognized object Rk that should be avoided, some trajectory planning algorithms will fail to plan the trajectory.

[0073] If the track plan is unsuccessful (Step S1307: No), the process proceeds to Step S1203. In this case, the control device 101, using the planning unit 704, refers to the selection table 600 and selects the unselected track plan algorithm with the highest track priority (Step S1205). As a result, the control device 101 re-executes the track plan using the newly selected track plan algorithm (Step S1206).

[0074] Furthermore, if the orbital plan is unsuccessful (step S1307: No) or if there are no unselected orbital planning algorithms for the current index 401, the process proceeds to step S1203, where the control device 101 may change the current index 401 to an unselected index 401 using the calculation unit 702. In this case, the changed index 401 is output to the planning unit 704. Therefore, the control device 101 can use the planning unit 704 to select an orbital planning algorithm corresponding to the changed index 401 (step S1205).

[0075] In such cases, the control device 101 may determine a plurality of indicators 401 in the surrounding environment 105 using the calculation unit 702, and the planning unit 704 may select a trajectory planning algorithm to which trajectory planning is possible using the plurality of indicators 401 (step S1205), and execute trajectory planning for each selected trajectory planning algorithm (step S1206).

[0076] As a result, for example, a trajectory planning algorithm capable of planning trajectories using both indices d1 and d2 is selected (step S1205), so that the larger the area of ​​the obstacle spectrum b, the more the trajectory that avoids the task target m and the slower the movement of the robot arm 102 is planned (step S1206).

[0077] In such cases, the control device 101 may also determine different indicators 401 for each divided region of the surrounding environment 105 using the calculation unit 702, select a trajectory planning algorithm for each indicator 401 of the divided region using the planning unit 704 (step S1205), and execute the trajectory plan using the trajectory planning algorithm selected for each divided region (step S1206).

[0078] As a result, for example, the control device 101, via the planning unit 704, will perform trajectory planning using a trajectory planning algorithm that allows trajectory planning with index d2 in the divided area up to 0.5m around the robot arm 102, and will perform trajectory planning using a trajectory planning algorithm that allows trajectory planning with index d1 in the divided area further away. [Examples]

[0079] Example 3 describes an example in which the actuator controlled by the control device 101 in Examples 1 and 2 is an autonomously mobile trolley robot. In Example 3, the focus is on explaining the differences from Examples 1 and 2, so the same reference numerals are used for components identical to those in Examples 1 and 2, and the explanation of redundant parts is omitted.

[0080] <Figure 14 Example of cart robot control 1> Figure 14 is an explanatory diagram showing an example of cart robot control 1 according to Embodiment 3. For example, in a picking operation in a logistics warehouse, when a person and a cart robot 1400 move together inside the warehouse, the person H1 moving together is treated as a task target m that can be approached, but an unrelated person H2 is treated as a task target that should be avoided. When the control target is the cart robot 1400, the planning unit 704 searches for a movement path for the cart robot 1400 to move to the destination point as a trajectory.

[0081] <Figure 15 Example of cart robot control 2> Figure 15 is an explanatory diagram showing an example of cart robot control example 2 according to Embodiment 3. In Figure 15, an example task is given in which a robot arm 102 picks up an item O transported by a cart robot 1400. In this case, the cart robot 1400 is the task target m that the robot arm 102 can approach. On the other hand, the cart robot 1500, which is engaged in other work, is the task target m that the robot arm 102 must avoid.

[0082] Furthermore, the control device 101 independently controls the operation of the robot arm 102 and the trolley robot 1400. Specifically, as shown in Embodiments 1 and 2, the control device 101, using the planning unit 704, plans the trajectory data R1 and R2 of the robot arm 102 and searches for a movement path for the trolley robot 1400 to move to the destination point (storage location for the robot arm 102 and item O).

[0083] <Figure 16 Example of a track plan> Figure 16 is an explanatory diagram showing an example of a trajectory plan according to Embodiment 3. Figure 16 is an example of a trajectory plan when index 401 is index d1 (degree to avoidance). The higher the value of obstacle spectrum b, the more the planning unit 704 generates trajectory data R3 as control data for the trolley robot 1400 to avoid such areas within the surrounding environment 105. The trajectory data R3 is expressed as the time-series movement direction, movement speed, and turning angle of the trolley robot 1400.

[0084] For example, a trajectory planning algorithm that performs a general pathfinding using gradients, such as stochastic gradient descent, is selected, and the obstacle spectrum b is applied to the gradient to generate trajectory data R3. Note that the contents shown in Figure 16 may also be displayed on a display device, which is an example of an output device 204 of the control device 101.

[0085] Thus, even with a movable controlled object like the trolley robot 1400, it is possible to achieve both safety and accessibility of the controlled object (trolley robot 1400) depending on the characteristics of the task object m.

[0086] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail to make the present invention easier to understand, and the present invention is not necessarily limited to having all of the described configurations. Furthermore, some of the configurations of one embodiment may be replaced with those of another embodiment. Furthermore, some of the configurations of one embodiment may be added to those of another embodiment. Furthermore, some of the configurations of each embodiment may be added, deleted, or replaced with other configurations.

[0087] Furthermore, each of the aforementioned configurations, functions, processing units, and processing means may be implemented in hardware, for example, by designing them as integrated circuits, or they may be implemented in software by having a processor interpret and execute programs that realize each function.

[0088] Information such as programs, tables, and files that implement each function can be stored in memory, hard disks, SSDs (Solid State Drives), or on recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0089] Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes and do not necessarily represent all control lines and information lines required for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0090] 100 control systems 101 Control device 102 Robot Arm 103 Sensor 104 Network 105 Surrounding Environment 200 Control device 201 Processor 202 Storage Devices 300 Task Goal Table 302 Preference distribution 400 Obstacle Level Information Table 401 indicators 500 Indicator Decision Table 600 Selection Table 700 Storage section 701 Estimation section 702 Calculation Unit 703 Generation part 704 Planning Department 800 calculation results 1400 Trolley Robots b. Obstacle spectrum

Claims

1. A storage unit that stores the preference distribution of observations for each task target on which the controlled object operates, and obstacle degree information indicating the degree to which the task target corresponds to an obstacle. An estimation unit estimates which task target an object in the surrounding environment belongs to, based on observation data from a sensor that observes the surrounding environment of the controlled object and the preference distribution of the observations. A calculation unit calculates an obstacle spectrum indicating the degree of obstacles estimated by the estimation unit based on the aforementioned obstacle degree information, A generation unit that generates the behavior of the controlled object based on the estimation result, A planning unit plans control data for controlling the controlled object based on the obstacle spectrum calculated by the calculation unit and the actions generated by the generation unit. A control device characterized by having the following features.

2. A control device according to claim 1, The preference distribution is a random variable for each position of the task target within the surrounding environment. A control device characterized by the following features.

3. A control device according to claim 1, The planning unit outputs the control data to the controlled object. The estimation unit estimates which task target an object in the surrounding environment is after the controlled object has performed the action, based on the observation data observed by the sensor after the controlled object has performed the action due to the planning unit outputting the control data to the controlled object, and the preference distribution of the observations. A control device characterized by the following features.

4. A control device according to claim 1, The memory unit stores selection information that associates priority with the algorithm for planning the control data. The planning unit attempts to plan the control data using a first algorithm with first priority, determines whether the plan is successful or not, and if it determines that the plan has failed, selects a second algorithm with second priority from the selection information and attempts to plan the control data. A control device characterized by the following features.

5. A control device according to claim 1, The aforementioned obstacle degree information is set for each of the task indicators, which is how the obstacle spectrum is perceived. The memory unit stores indicator determination information that associates the action with the indicator, and selection information that defines an algorithm for planning the control data for each indicator. The calculation unit, referring to the index determination information, determines the index corresponding to the action generated by the generation unit. The planning unit selects the algorithm for the indicator determined by the calculation unit from the selection information and attempts to plan the control data. A control device characterized by the following features.

6. A control device according to claim 1, The memory unit stores selection information that defines an algorithm for planning the control data for each task indicator, which is a way of perceiving the obstacle spectrum. The planning unit attempts to plan the control data using the first algorithm of the first indicator, determines whether the plan is successful or not, and if it determines that the plan has failed, it selects a second algorithm of a second indicator different from the first indicator from the selection information and attempts to plan the control data. A control device characterized by the following features.

7. A control device according to claim 1, The task is for the controlled object to grasp an article and move the article. A control device characterized by the following features.

8. A control device according to claim 1, The task is the task of moving the controlled object to the destination point. A control device characterized by the following features.

9. A control method performed by a control device having a processor for executing a program, a storage device for storing the program, and a communication interface capable of communicating with a sensor for observing a controlled object and its surrounding environment, The memory device, The system stores the preference distribution of observations for each task target on which the controlled object operates, and obstacle degree information indicating the degree to which the task target corresponds to an obstacle. The aforementioned processor, An estimation process that estimates which task target an object in the surrounding environment is based on the observation data from the sensor and the preference distribution of the observations. A calculation process that calculates an obstacle spectrum indicating the degree of obstacles estimated by the estimation process based on the aforementioned obstacle degree information, Based on the estimation results, a generation process is performed to generate the actions of the controlled object, A planning process that plans control data for controlling the controlled object based on the obstacle spectrum calculated by the calculation process and the actions generated by the generation process, A control method characterized by performing the following.

10. A control program to be executed by the processor of a control device having a processor for executing a program, a storage device for storing the program, and a communication interface capable of communicating with a sensor for observing a controlled object and its surrounding environment, The memory device, The system stores the preference distribution of observations for each task target on which the controlled object operates, and obstacle degree information indicating the degree to which the task target corresponds to an obstacle. The aforementioned processor, An estimation process that estimates which task target an object in the surrounding environment is based on the observation data from the sensor and the preference distribution of the observations. A calculation process that calculates an obstacle spectrum indicating the degree of obstacles estimated by the estimation process based on the aforementioned obstacle degree information, Based on the estimation results, a generation process is performed to generate the actions of the controlled object, A planning process that plans control data for controlling the controlled object based on the obstacle spectrum calculated by the calculation process and the actions generated by the generation process, A control program characterized by causing the execution of a specific action.

Images