Movement control device and movement control system
The movement control system addresses obstacle-related navigation challenges by integrating visual feedback and model-based control with machine learning to dynamically select paths, ensuring accurate and robust movement to a target position.
Patent Information
- Application Number
- JP2023083843
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-12-04
AI Technical Summary
Existing movement control systems face challenges in selecting an appropriate movement path when obstacles occur, especially in situations where visual feedback control is hindered by occlusion, making it difficult to navigate to a target position effectively.
A movement control device and system that utilize image information to determine a first movement path through visual feedback control and a second path through model-based control using machine learning, incorporating a search learning device and an evaluation learning device to select the most suitable path based on the state of the environment.
Enables the selection of an optimal movement path according to the state of the space, enhancing navigation accuracy and robustness by dynamically switching between visual feedback and model-based control paths to avoid obstacles.
Smart Images

Figure 2024167633000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a mobile control device and a mobile control system that perform movement control on a controlled object. [Background technology]
[0002] Conventionally, various control methods for moving an object to a target position have been known. For example, in the field of industrial machinery, a work method for performing an operation on a workpiece using a robot has been disclosed. The movement of an object to a target position can be performed using a controlled object or by the object itself being the controlled object.
[0003] Patent Document 1 discloses a control system for reducing the cycle time when a work target is selected from multiple workpieces and the work is performed by a robot. Specifically, Patent Document 1 discloses a technology for correcting the amount of deviation of the work target by visual feedback control using a visual sensor installed near the robot's tool. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent Publication No. 2021-024027 Summary of the Invention [Problem to be solved by the invention]
[0005] As in the system disclosed in Patent Document 1, while performing movement control using an image of a space including at least a part of a movement path from the current position of an object to a target position, a situation may occur in which an obstacle appears in the space for some reason. For example, when visual feedback control is used, depending on the occurrence of occlusion by an obstacle, it may not be possible to search for a movement path suitable for the state in the space even if a movement path to the target position actually exists.
[0006] The present invention has been made in consideration of such problems, and its purpose is to provide a movement control device and a movement control system that are capable of selecting an object's movement path according to the conditions in a space when controlling movement using an image of the space that includes at least a part of the object's movement path from its current position to a target position. [Means for solving the problem]
[0007] A movement control device in a first aspect of the present invention has one or more processors, and the one or more processors execute the following: a first generation process that determines a first movement path of the object for visual feedback control and generates first path information indicating the first movement path using image information related to an image in the space obtained by photographing the space including at least a part of the movement path from the current position of the object to a target position; a second generation process that determines a second movement path of the object for model-based control based on machine learning and generates second path information indicating the second movement path using the image information or a feature set generated from the image information; and a movement control process that selects one of the first path information and the second path information and performs movement control on a control target that is the same as or different from the object so as to move the object to the target position along the movement path specified by the one of the path information.
[0008] In a movement control device according to a second aspect of the present invention, the second generation process includes a regression calculation using a search and learner that has been subjected to machine learning, and the search and learner inputs state features indicating a state within the space, encoder information related to the controlled object, or path features indicating a movement path of the evaluation object, and outputs a search evaluation value that indicates an evaluation result for the movement path of the evaluation object.
[0009] In a mobile control device in a third aspect of the present invention, the one or more processors further execute a determination process regarding the selection of the first route information or the second route information, and in the mobile control process, select either the first route information or the second route information depending on the determination result by the determination process.
[0010] In a fourth aspect of the mobile control device of the present invention, the judgment process includes a regression calculation using an evaluation learning device that has been subjected to machine learning, and the evaluation learning device inputs state features indicating a state in the space or encoder information related to the controlled object, and outputs a selection flag indicating whether the first route information or the second route information is adopted.
[0011] In a fifth aspect of the present invention, in the mobile control device, the evaluation learning device is a learning device that has undergone reinforcement learning using the exploration evaluation value as a reward.
[0012] In a sixth aspect of the present invention, the determination process includes a regression calculation using an evaluation learning device that has been subjected to machine learning, and the evaluation learning device inputs state features indicating a state within the space or encoder information related to the object to be controlled, and outputs an evaluation value that is identical to or correlated with the search evaluation value.
[0013] In a mobile control device according to a seventh aspect of the present invention, the mobile control process selects the first route information when the evaluation value exceeds a threshold, and selects the second route information when the evaluation value is equal to or less than the threshold.
[0014] A movement control system in an eighth aspect of the present invention includes a control object, a movement control device that performs movement control on the control object, and a visual sensor that captures an image of a space including at least a part of a movement path from a current position to a target position of an object that is the same as or different from the control object, and outputs image information related to the image within the space, wherein the movement control device executes the following steps: a first generation process that uses the image information output from the visual sensor to determine a first movement path of the object for visual feedback control and generates first path information indicating the first movement path; a second generation process that uses the image information or a feature set generated from the image information to determine a second movement path of the object for model-based control based on machine learning and generates second path information indicating the second movement path; and a movement control process that selects one of the first path information and the second path information and performs the movement control so as to move the object to the target position along the movement path specified by one of the path information. Effect of the Invention
[0015] According to the present invention, when controlling movement using an image of a space including at least a portion of the movement path from the object's current position to a target position, it is possible to select the object's movement path according to the conditions in the space. [Brief description of the drawings]
[0016] [Figure 1] 1 is an overall configuration diagram of a robot system as a movement control system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a detailed block diagram of the robot system shown in FIG. [Diagram 3] FIG. 3 is a diagram illustrating input / output characteristics of the exploratory learning device of FIG. 2. [Figure 4] 3 is a diagram showing a schematic diagram of input / output characteristics of the evaluation learning device of FIG. 2. [Diagram 5] 2 is a flowchart showing an example of an operation of the robot system of FIG. 1. [Figure 6]FIG. 3 is a diagram showing an example of an image obtained by the camera of FIGS. 1 and 2. [Figure 7] 2 is a diagram showing an example of a trajectory of the tip of the cable in FIG. 1. FIG. [Figure 8] FIG. 5 is a diagram showing input / output characteristics of an evaluation learning device in an example different from that in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In order to facilitate understanding of the description, the same reference numerals are used to refer to the same components in each drawing as much as possible, and duplicated description will be omitted.
[0018] [Configuration of robot system 10] <Overall composition> 1 is an overall configuration diagram of a robot system 10 as a movement control system in one embodiment of the present invention. Specifically, the robot system 10 includes a robot 12 (corresponding to a "controlled object"), a control device 14 (corresponding to a "movement control device"), and a teaching pendant 16.
[0019] The robot 12 is a vertical joint type robot having a plurality of joint axes 42 (FIG. 2). The plurality of joint axes 42 are, for example, a rotation axis (J1 axis) that rotates the body, a lower arm axis (J2 axis) that moves the body back and forth, an upper arm axis (J3 axis) that moves the arm up and down, a wrist rotation axis (J4 axis) that rotates the arm, a "wrist bending axis" (J5 axis) that swings the wrist up and down, and a wrist rotation axis (J6 axis) that rotates the wrist. The robot 12 can perform various tasks including gripping and moving a workpiece, welding, and painting by independently driving the plurality of joint axes 42 in response to commands from the control device 14.
[0020] 1, the robot 12 performs a connection operation of inserting a tip end (e.g., a plug) of the cable 18 into a receptacle 20a on the electronic circuit board 20. Hereinafter, the horizontal direction of the electronic circuit board 20 will be referred to as the "H direction," the vertical direction of the electronic circuit board 20 as the "V direction," and the insertion direction (or depth direction) of the cable 18 as the "D direction."
[0021] An end effector 24 (or a tool or manipulator) is attached to the tip of an arm unit 22 of the robot 12. This allows the robot 12 to move the cable 18 to a desired position and posture while gripping the tip of the cable 18. In addition, a visual sensor 28 including a pair of cameras 30 (so-called stereo cameras) is attached to the tip of the arm unit 22.
[0022] The control device 14 is a computer that controls the operation of the robot 12. Specifically, the control device 14 includes a connector 32, a communication I / F 34, a processor 36, and a memory 38. The number of each component is one in the example of Fig. 1, but may be two or more.
[0023] The connector 32 is a terminal for electrically connecting to the robot 12 via a power cable or a communication cable. With this, the control device 14 supplies power and control signals to the robot 12, and also acquires measurement signals from various sensors provided on the robot 12.
[0024] The communication I / F 34 is an interface for communicating with an external device. This allows the control device 14 to exchange data with, for example, the teaching pendant 16, a work terminal not shown, or a higher-level device. Here, the teaching pendant 16 is an input device used for online teaching of the robot 12. In the case of offline teaching, the configuration of the teaching pendant 16 may be omitted.
[0025] The processor 36 may be a general-purpose processor including a CPU (Central Processing Unit) or a dedicated processor including an FPGA (Field Programmable Gate Array) or a GPU (Graphics Processing Unit). The memory 38 is a non-transitory computer-readable recording medium (or storage medium) that stores programs and data required for the processor 36 to control each component.
[0026] <Function block diagram> Fig. 2 is a detailed block diagram of the robot system 10 shown in Fig. 1. In the example of Fig. 2, a visual sensor 28, a robot 12, and a control device 14 are illustrated as device configurations of the robot system 10.
[0027] The visual sensor 28 captures an image of a space including at least a part of a moving path of an object (e.g., cable 18) from a current position to a target position, and outputs image information related to the image in the space. Specifically, the visual sensor 28 includes an image processing device 40 in addition to the camera 30 (FIG. 1).
[0028] The camera 30 is an imaging device that generates a signal for each frame by capturing images in a space and outputs the signal as an imaging signal that indicates a time series of images. The camera 30 is composed of, for example, a monocular camera such as a visible light camera, an infrared camera, or a TOF (Time Of Flight) camera, a stereo camera consisting of cameras of the same type, or a combination of cameras of different types. The camera 30 is positioned so that the end effector 24 fits within the angle of view.
[0029] The image processing device 40 performs desired image processing on the imaging signal output from the camera 30, and outputs image information including the analysis results. The analysis results include the presence or absence, type, size, position / posture, and movement (e.g., speed, acceleration, jerk, etc.) of an object. The types of object include [1] the main body or tip of the cable 18, [2] the main body or receptacle 20a of the electronic circuit board 20, or [3] other objects (e.g., obstacles, indistinguishable), etc. Note that the image information may include the imaging signal itself in addition to the above-mentioned analysis results.
[0030] The robot 12 includes a joint shaft 42, a servo motor 44, and an encoder 46. The servo motor 44 is an actuator that applies a rotational force to the joint shaft 42. The encoder 46 is a detector that outputs a detection signal that indicates encoder information such as the position, speed, and torque of the servo motor 44. For convenience of illustration, only one set of the joint shaft 42, the servo motor 44, and the encoder 46 is shown, but in reality, multiple sets (for example, six sets in the case of a six-axis robot) are provided.
[0031] The processor 36 (Figure 1) of the control device 14 functions as a feature generation unit 52, a route selection unit 54, a route generation unit 56, and a route control unit 58 by reading and executing a movement control program stored in the memory 38 (Figure 1).
[0032] The feature generator 52 generates a collection of features (hereinafter referred to as a "feature set") from the image information supplied by the visual sensor 28 or the encoder information supplied by the encoder 46. This feature set includes [1] a "state feature" related to a state in space, [2] an "encoder feature" related to encoder information of a controlled object, or [3] a "route feature" related to a moving route in space. The state feature may include an image analysis result or an imaging signal, as in the case of the image information described above. The encoder feature may be the encoder information itself, or may include time-series data or a time change amount of the encoder information. The route feature may include a vector indicating a position (start point, end point, and via point) on a moving route or an amount of movement.
[0033] The path selection unit 54 selects one of the multiple movement paths using the feature set generated by the feature generation unit 52. Examples of the movement paths include [1] a movement path of an object for visual feedback control (hereinafter also referred to as "VF") (hereinafter also referred to as "first movement path"), or [2] a movement path of an object for model-based control based on machine learning (hereinafter also referred to as "ML") (hereinafter also referred to as "second movement path"). In principle, the path selection unit 54 selects the first movement path when [1] there is no obstacle in the space or the possibility of interference with an obstacle in the space is low, and selects the second movement path when [2] there is a high possibility of interference with an obstacle in the space.
[0034] The path selection unit 54 executes a judgment process including a regression calculation using, for example, an evaluation learning device LNe that has undergone machine learning. The evaluation learning device LNe inputs a feature amount related to the state in the space (i.e., a state feature amount) or a feature amount related to the encoder information of the robot 12 (i.e., an encoder feature amount), and outputs an evaluation value (hereinafter also referred to as a "judgment evaluation value") for judging the selection of the first moving path or the second moving path. The judgment evaluation value may be a discrete value or a continuous value.
[0035] An example of a discrete value is a value flag (hereinafter, a selection flag) that indicates whether or not the first selected path is adopted. Specifically, when the value of the selection flag is "1", the first moving path is selected, whereas when the value of the selection flag is "0", the second moving path is selected. Conversely, when the value of the selection flag is "1", the second moving path is selected, whereas when the value of the selection flag is "0", the first moving path may be selected.
[0036] When the judgment evaluation value is a continuous value, it is defined such that the larger the value, the higher the reliability of the first moving path, and the smaller the value, the lower the reliability of the first moving path. In addition, this judgment evaluation value may be the same value as the search evaluation value described later, or may be a value correlated with the search evaluation value (for example, a value obtained by normalizing the search evaluation value).
[0037] The route generating unit 56 determines one travel route from the selection result by the route selecting unit 54, and generates route information indicating the travel route. Specifically, the route generating unit 56 includes a first generating unit 60 and a second generating unit 62.
[0038] The first generating unit 60 executes information processing (hereinafter referred to as "first generation processing") to determine an object's movement path (i.e., a first movement path) for visual feedback control (VF) using image information supplied from the visual sensor 28, and generate path information (hereinafter referred to as "first path information") indicating the first movement path. Here, "visual feedback control" (or VF) refers to a method of calculating the position, attitude, and movement of a controlled object through image processing, and feeding back the calculated information to the actuator drive control. Hereinafter, it is assumed that the "reference position" of the movement path is set to the position of the tip of the cable 18, and the "target position" of the movement path is set to the position of the receptacle 20a.
[0039] The first generation unit 60 may determine the first movement path regardless of the result of the movement path selection by the path selection unit 54, or may determine whether or not to determine the first movement path depending on the result of the movement path selection by the path selection unit 54. Specifically, the first generation unit 60 may not determine the first movement path when the second movement path is selected by the path selection unit 54, and may determine the first movement path only when the second movement path is not selected.
[0040] The second generation unit 62 executes information processing (hereinafter referred to as "second generation processing") to determine a movement path of an object for model-based control based on machine learning (ML) (i.e., a second movement path) using the feature set generated by the feature generation unit 52, and generate path information (hereinafter referred to as "second path information") indicating the second movement path. Here, "model-based control based on machine learning" refers to a method of modeling a system to be controlled through machine learning and controlling the drive of an actuator according to the model.
[0041] The second generation unit 62 executes a second generation process including a regression calculation using a search learning device LNs that has undergone machine learning, for example. The search learning device LNs inputs a feature amount related to the state in the space (i.e., a state feature amount), a feature amount related to the encoder information of the robot 12 (i.e., an encoder feature amount), or a feature amount related to the moving path of the evaluation target (i.e., a path feature amount), and outputs an evaluation value (hereinafter also referred to as a "search evaluation value") indicating an evaluation result for the moving path of the evaluation target. The search evaluation value is defined such that the larger the value, the higher the reliability of the moving path, and the smaller the value, the lower the reliability of the moving path.
[0042] The second generation unit 62 may search for the second movement route regardless of the result of the movement route selection by the route selection unit 54, or may determine whether or not to search for the second movement route depending on the result of the movement route selection by the route selection unit 54. Specifically, the second generation unit 62 may not search for the second movement route when the first movement route is selected by the route selection unit 54, and may search for the second movement route only when the first movement route is not selected.
[0043] The movement control unit 58 is an information process (hereinafter referred to as "movement control process") that performs movement control on a controlled object (e.g., robot 12) so as to move the object (e.g., cable 18) to a target position along a movement path specified by the path information. Specifically, the movement control unit 58 includes a target value calculation unit 64 and a servo control unit 66. For convenience of illustration, only one servo control unit 66 is shown, but in reality, the same number of servo control units 66 as the number of servo motors 44 (e.g., six in the case of a six-axis robot) are provided.
[0044] The target value calculation unit 64 selects either the first route information generated by the first generation unit 60 or the second route information generated by the second generation unit 62, and calculates target values (e.g., position, posture, speed, acceleration, jerk, etc.) for moving along the movement route specified by the route information.
[0045] The servo control unit 66 performs drive control of the servo motor 44 based on the position signal from the encoder 46 or the target value output from the target value calculation unit 64. For the drive control, for example, PWM (Pulse Width Modulation) is used, in which the current flowing through the servo motor 44 is used as the control amount.
[0046] Fig. 3 is a diagram showing a schematic diagram of input / output characteristics of the exploration learner LNs shown in Fig. 2. The exploration learner LNs is, for example, a neural network calculator in which a group of learning parameters is preset. The group of learning parameters is a collection of learning parameters for specifying the calculation rule of the exploration learner LNs. The learning parameters may include, for example, a coefficient describing the activation function of the calculation unit, a weighting coefficient corresponding to the strength of synaptic connection, the number of calculation units constituting each layer, the number of layers of intermediate layers, and the like.
[0047] The type of machine learning may be any of supervised learning, unsupervised learning, and reinforcement learning. The exploratory learner LNs in FIG. 3 is, for example, subjected to reinforcement learning. A state feature S1, an encoder feature E1, and a path feature {Ai} (i=1, 2, . . . , n) are input to an input layer of the exploratory learner LNs. An evaluation value {Ri} (i=1, 2, . . . , n) is output from an output layer of the exploratory learner LNs. The state feature S1 and the encoder feature E1 correspond to the "environment" in reinforcement learning. The path feature {Ai} corresponds to the "action" in reinforcement learning. The evaluation value {Ri} corresponds to the "reward" in reinforcement learning.
[0048] In the "search phase" of the second movement route, the route feature values {Ai} corresponding to n movement route candidates are input sequentially to the learning device LN, and n evaluation values {Ri} are output. The maximum one of the evaluation values {Ri} is selected, and the movement route candidate corresponding to the maximum value is determined as the "second movement route."
[0049] This reinforcement learning is performed using various learning algorithms. If the state, state value function, and reward at time t are S(t), V(t), and R(t), respectively, the learning error δ(t) can be calculated by the following formula (1). Note that γ corresponds to the discount rate that indicates the degree to which future rewards are discounted. δ(t)=R(t+1)+γV(S(t+1))-V(S(t)) ‥‥(1)
[0050] The reward giving rule is determined so that, for example, [1] the shorter the moving path, the higher the reward; [2] the closer to the target position, the higher the reward; or [3] the lower the reward when entering an interference area with an obstacle. In the learning phase, the moving path may be selected randomly, or a moving path based on VF may be selected. By selecting a moving path based on VF and performing reinforcement learning, there is a possibility that generalization is impaired due to overlearning, but on the other hand, there is an advantage in that the calculation accuracy of the evaluation value is higher when a moving path similar to the first moving path is always selected.
[0051] Fig. 4 is a diagram showing a schematic diagram of input / output characteristics of the evaluation learning device LNe shown in Fig. 2. The evaluation learning device LNe is composed of, for example, a neural network computing device in which a learning parameter group is preset. The learning parameter group is a collection of learning parameters for specifying the computing rule of the evaluation learning device LNe. The learning parameters may include, for example, a coefficient describing the activation function of the computing unit, a weighting coefficient corresponding to the strength of the synaptic connection, the number of computing units constituting each layer, the number of layers of the intermediate layer, and the like.
[0052] The type of machine learning may be any of supervised learning, unsupervised learning, and reinforcement learning. The evaluation learning device LNe in FIG. 4 is, for example, subjected to reinforcement learning. The state feature S2 and the encoder feature E2 are input to the input layer of the evaluation learning device LNe. A selection flag F (binary value) indicating whether or not the first movement path is adopted is output from the output layer of the evaluation learning device LNe.
[0053] The learning data set is generated from an exploration learner LNs (Fig. 3) that is in the middle of learning or has already been learned. The state feature S2, which is an input value of the learning data, corresponds to the state feature S1 in Fig. 3. The encoder feature E2, which is an input value of the learning data, corresponds to the encoder feature E1 in Fig. 3.
[0054] This reinforcement learning is performed using various learning algorithms, as in the case of FIG. 3. The reward giving rule is determined, for example, so that [1] the reward is high when the possibility of interference with an obstacle in the space is correctly determined, or [2] the reward is low when the possibility of interference with an obstacle in the space is incorrectly determined. For example, the evaluation value R described in FIG. 3 may be used as the reward. In this case, the learning efficiency can be improved by alternately repeating [1] the "first reinforcement learning" for the search learning device LNs and [2] the "second reinforcement learning" for the evaluation learning device LNe, which uses the evaluation value R obtained in the first reinforcement learning as a reward. As a result of this reinforcement learning, the evaluation learning device LNe basically outputs F=1 when [1] there is no obstacle in the space or the possibility of interference with an obstacle in the space is low, while it outputs F=0 when [2] there is a high possibility of interference with an obstacle in the space.
[0055] [Operation of robot system 10] The movement control system (i.e., robot system 10) in this embodiment is configured as described above. Next, the control operation by the robot system 10 (mainly the control device 14) will be described with reference to Fig. 5 to Fig. 8. Fig. 5 is a flow chart showing an example of the operation by the robot system 10 of Fig. 1.
[0056] <Explanation of the flow chart> (SP10: Confirmation process) 5, the control device 14 checks whether the control timing has arrived. If the control timing has not arrived yet (step SP10: NO), the control device 14 remains in step SP10 until the control timing arrives. On the other hand, if the control timing has arrived (step SP10: YES), the control device 14 proceeds to the next step SP12.
[0057] (SP12: Acquisition process) In step SP12, the control device 14 acquires image information and encoder information at the control timing arrived in step SP10. For example, the visual sensor 28 acquires image information by capturing an image of a space including at least a part of the moving path from the current position of the tip of the cable 18 to the target position.
[0058] FIG. 6 is a diagram showing an example of an image obtained by the camera 30 of FIG. 1 and FIG. 2. A plug area 72, a receptacle area 74, and a background area 76 are shown in a rectangular image area 70. The background area 76 corresponds to the entire image area 70 excluding the area of interest. The area of interest is typically the plug area 72 indicating the "start point or intermediate point" of the movement path, and the receptacle area 74 indicating the "end point" of the movement path. In the example of FIG. 6, an obstacle different from the cable 18 and the electronic circuit board 20 is shown in the image area 70 as an obstacle area 78. This obstacle means an object that may come into contact with or collide with the cable 18 or the end effector 24 during the movement of the cable 18.
[0059] (SP14: Feature generation processing) In step SP14 of FIG. 5, the feature generator 52 generates a feature set (for example, state feature, encoder feature, and path feature) to be used in the arithmetic processing, using the image information and encoder information acquired in step SP12.
[0060] (SP16: Evaluation value calculation process) In step SP16, the route selection unit 54 calculates a selection flag indicating whether or not to select the first moving route, using the feature set generated in step SP14. Specifically, by inputting this feature set to the evaluation learning device LNe (FIGS. 2 and 4), a selection flag that can take a value of 0 or 1 is output.
[0061] (SP18: Judgment process) In step SP18, the route selection unit 54 uses the selection flag calculated in step SP16 to make a decision regarding the selection of a moving route. If the value of the selection flag is "1" (step SP18: YES), the route selection unit 54 supplies the selection flag indicating the first moving route to the route generation unit 56, and then proceeds to the next step SP20.
[0062] (SP20: First generation process) In step SP20, the first generation unit 60 determines a first movement path based on the VF using the feature set (here, state features and encoder features) generated when executing step SP14, and generates first path information indicating the first movement path.
[0063] (SP22: Target value calculation process) In step SP22, the movement control unit 58 (more specifically, the target value calculation unit 64) calculates a first target value for moving along the first movement path based on the first path information generated in step SP20. Thereafter, the movement control unit 58 proceeds to step SP28, which will be described later.
[0064] Returning to step SP18, if the value of the selection flag calculated in step SP16 is “0” (step SP18: NO), the path selection unit 54 supplies a selection flag indicating the second movement path to the path generation unit 56, and then proceeds to step SP24.
[0065] (SP24: Second generation process) In step SP24, the second generation unit 62 determines a second movement path based on ML using the feature set (here, state features, encoder information, and path features) generated when step SP14 is executed, and generates second path information indicating the second movement path.
[0066] (SP26: Target value calculation process) In step SP26, the movement control unit 58 (more specifically, the target value calculation unit 64) calculates a second target value for moving along the second movement path based on the second path information generated in step SP24. Thereafter, the movement control unit 58 proceeds to the next step SP28.
[0067] (SP28: Movement control processing) In step SP28, the movement control unit 58 (more specifically, the servo control unit 66) uses the target value calculated in step SP22 or step SP26 to perform movement control on the robot 12. Specifically, the servo control unit 66 generates a control signal corresponding to the target value and outputs it to the robot 12 (more specifically, the servo motor 44).
[0068] (SP30: Drive operation) In step SP30, the servo motor 44 of the robot 12 is driven in response to the control signal output in step SP28, thereby applying a rotational force to the joint shaft 42.
[0069] In this manner, the robot system 10 completes the execution of the flowchart shown in Fig. 5. Thereafter, the robot system 10 repeatedly executes the flowchart at a predetermined control period, thereby causing the robot 12 to perform a desired task (here, the task of connecting the cable 18).
[0070] Fig. 7 is a diagram showing a schematic example of a trajectory of the reference position of the cable 18 in Fig. 1. The horizontal axis of the coordinate system indicates the D direction, and the vertical axis of the coordinate system indicates the H or V direction. Points P1 and P2 indicate initial positions, and point G indicates the target position. In other words, the control device 14 repeatedly controls the drive of the robot 12 and updates the movement path, and sequentially determines an optimal movement path connecting point P1 to point G (or point P2 to point G).
[0071] When an obstacle appears between two points P1 and G, the movement route is determined so as to avoid entering the interference area 80 with the obstacle. As a result, when the current position P is within the first area 82, the movement route based on VF is selected, and when the current position P is within the second area 84, the movement route based on ML is selected. The second area 84 is a closed area provided so as to surround a part of the interference area 80. The first area 82 is the remaining area excluding the second area 84 from the entire area. As can be seen from FIG. 6, a phenomenon occurs in which the movement route dynamically switches at the boundary between the first area 82 and the second area 84. As a result, a broken line-like trajectory is formed connecting the two points P1 and G (or between the two points P2 and G).
[0072] A route search based on VF has the advantage that the positional accuracy of the search results is relatively high, but the disadvantage is that the robustness of the search is relatively low when occlusion occurs. A route search based on ML has the advantage that the robustness of the search is relatively high when occlusion occurs, but the disadvantage is that the positional accuracy of the search results is relatively low. Therefore, by dynamically switching between the two route searches, the disadvantages of both can be mutually complemented.
[0073] <Another example of the evaluation learning machine NLe> FIG. 8 is a diagram showing input / output characteristics of an evaluation learning device LNe2 in another example different from that in FIG. 4. The evaluation learning device LNe2 is composed of, for example, a neural network calculator in which a group of learning parameters is preset. This evaluation learning device LNe2 undergoes supervised learning. A state feature S2 and an encoder feature E2 are input to the input layer of the evaluation learning device LNe2. An evaluation value R equivalent to the "reward" described in FIG. 3 is output from the output layer of the evaluation learning device LNe2.
[0074] The training data set is generated from a trained search learner LNs (Figure 3). The state feature S2, which is an input value of the training data, corresponds to the state feature S1 in Figure 3. The encoder feature E2, which is an input value of the training data, corresponds to the encoder feature E1 in Figure 3. The evaluation value R, which is the output value (correct answer) of the training data, corresponds to the evaluation value R (or reward) in Figure 3.
[0075] In this case, the path selection unit 54 outputs a selection flag "1" indicating the first path information when the evaluation value R output from the evaluation learning device LNe2 exceeds a threshold, and outputs a selection flag "0" indicating the second path information when the evaluation value R is equal to or less than the threshold. This threshold is determined in advance according to the learning result of the evaluation learning device LNe2, for example. In this way, even if the input / output characteristics shown in FIG. 8 are adopted, the path selection unit 54 can perform the same path selection function as in the case of FIG. 4.
[0076] [Summary of the embodiment] As described above, the movement control system (here, robot system 10) in this embodiment comprises a controlled object (here, robot 12), a movement control device (here, control device 14) that controls the movement of the robot 12, and a visual sensor 28 that captures an image of a space including at least a part of a movement path from a current position to a target position of an object (here, cable 18) that is the same as or different from the robot 12, and outputs image information regarding the image in the space.
[0077] The control device 14 selects either one of first path information indicating a first movement path for visual feedback control and second path information indicating a second movement path for model-based control based on machine learning, and performs movement control on the robot 12 so as to move the cable 18 to a target position along the movement path specified by either one of the path information. Since it is configured in this way, when performing movement control using an image of a space including at least a part of the movement path from the current position of the cable 18 to the target position, it is possible to select a movement path of the cable 18 according to the state in the space.
[0078] Furthermore, when the second generation process includes a regression calculation using a machine-learned exploration learner LNs, the exploration learner LNs may input state features indicating a state in the space, encoder information related to the robot 12, or path features indicating a moving path of an evaluation target, and output a search evaluation value indicating an evaluation result for the moving path of an evaluation target. This makes it possible to search for a moving path through the machine-learned exploration learner LNs.
[0079] In addition, when the control device 14 further executes a determination process regarding the selection of the first route information or the second route information, the first generation process and the second generation process may be executed alternatively depending on the determination result of the determination process. This makes it possible to omit one of the two types of generation processes, thereby reducing the computational load accordingly.
[0080] Furthermore, when the determination process includes a regression calculation using an evaluation learning device LNe that has undergone machine learning, the evaluation learning device LNe may input state features indicating a state in the space or encoder information regarding the robot 12, and output a selection flag indicating whether the first path information or the second path information is adopted. In this way, the selection flag is obtained as an index indicating the evaluation result for the first movement path through the evaluation learning device LNe.
[0081] In addition, the evaluation learning device LNe may be a learning device that has undergone reinforcement learning using the exploration evaluation value as a reward. Through reinforcement learning using the exploration evaluation value as a reward, an evaluation learning device LNe that reflects the evaluation result by the exploration learning device LNs can be obtained.
[0082] Furthermore, when the judgment process includes a regression calculation using an evaluation learning device LNe2 that has undergone machine learning, the evaluation learning device LNe2 may input state features indicating a state in the space or encoder information regarding the robot 12, and output a judgment evaluation value that is the same as or correlates with the search evaluation value. In this way, a judgment evaluation value is obtained as an index indicating an evaluation result for the first movement path through the evaluation learning device LNe2.
[0083] In addition, in the movement control process, the first route information may be selected when the judgment evaluation value exceeds a threshold, and the second route information may be selected when the judgment evaluation value is equal to or less than the threshold. In this way, the first route information is selected when the evaluation of the first movement route is high, and the second route information is selected when the evaluation of the second movement route is low.
[0084] Alternatively, the object may be a cable 18, the controlled object may be a robot 12 that grips the cable 18, and the target position may be the position of a receptacle 20a that receives the tip of the cable 18. This allows the automation of the operation of connecting the cable 18 to the electronic circuit board 20.
[0085] [Variations] The present invention is not limited to the above-described embodiment, and can be freely modified without departing from the spirit of the present invention. Alternatively, the respective configurations can be arbitrarily combined without causing any technical inconsistency. Alternatively, the order of execution of each step constituting the flow chart can be arbitrarily changed without causing any technical inconsistency.
[0086] In the above embodiment, the movement control unit 58 selects a movement route according to the output value output from the evaluation learning device LNe, but the judgment conditions for selection are not limited to this. For example, the movement control unit 58 may dynamically select a movement route according to the object detection result by image processing of the visual sensor 28. Examples of judgment conditions for selecting the second movement route include [1] when an unidentifiable object is detected, [2] when a detected object becomes undetectable (so-called lost state), and [3] when a detected target position becomes undetectable.
[0087] In the above embodiment, a vertical articulated robot has been described as an example, but the type of industrial robot is not limited to this. For example, the robot 12 may be any of a horizontal articulated robot, a parallel link robot, and an orthogonal robot.
[0088] In the above embodiment, the robot 12 is described as performing an operation of connecting the cable 18, but the type and purpose of the operation or the type of workpiece that the robot 12 is caused to perform is not limited to this. For example, the robot 12 may be caused to perform a "pickup operation" in which the robot 12 grasps a workpiece at an initial position and moves it to a target position.
[0089] In the above embodiment, the object (cable 18) and the controlled object (robot 12) are different objects, but they may be the same object. Specifically, this movement control can be applied to the movement of a moving object, including an automated guided vehicle (AGV), to a target position. [Explanation of symbols]
[0090] 10...Robot system (movement control system), 12...Robot (control object), 14...Control device (movement control device), 18...Cable (object), 28...Visual sensor, 36...Processor, 38...Memory, 54...Path selection unit, 58...Movement control unit, 60...First generation unit, 62...Second generation unit, LNe, LNe2...Evaluation learning device, LNs...Search learning device, SP14...First generation step (first generation processing), SP22...Second generation step (second generation processing), SP26...Movement control step (movement control processing)
Claims
1. one or more processors; The one or more processors: a first generation process for determining a first movement path of the object for visual feedback control using image information relating to an image in the space obtained by photographing the space including at least a part of a movement path from a current position of the object to a target position, and generating first path information indicating the first movement path; a second generation process of determining a second movement path of the object for model-based control based on machine learning using the image information or a feature set generated from the image information, and generating second path information indicating the second movement path; a movement control process for selecting one of the first route information and the second route information, and performing movement control on a control target object that is the same as or different from the object so as to move the object to the target position along a movement route specified by the one of the route information; A mobile control device that executes the above.
2. The second generation process includes a regression calculation using a machine learning search learner, The movement control device according to claim 1, characterized in that the search and learner inputs a state feature indicating a state in the space, encoder information related to the controlled object, or a path feature indicating a movement path of an evaluation object, and outputs a search evaluation value indicating an evaluation result for the movement path of the evaluation object.
3. The one or more processors further perform a determination process regarding selection of the first route information or the second route information; The mobile control device according to claim 2 , wherein the first generation process and the second generation process are alternatively executed depending on a determination result of the determination process.
4. The determination process includes a regression calculation using an evaluation learning device that has undergone machine learning, The mobile control device according to claim 3, characterized in that the evaluation learning device inputs a state feature indicating a state in the space or encoder information related to the object to be controlled, and outputs a selection flag indicating whether the first path information or the second path information is adopted.
5. The mobile control device according to claim 4 , wherein the evaluation learning device is a learning device that has undergone reinforcement learning using the search evaluation value as a reward.
6. The determination process includes a regression calculation using an evaluation learning device that has undergone machine learning, The mobile control device according to claim 3, characterized in that the evaluation learning device inputs a state feature indicating a state in the space or encoder information related to the object to be controlled, and outputs an evaluation value that is identical to or correlated with the search evaluation value.
7. A mobile control device as described in claim 6, characterized in that in the mobile control process, the first route information is selected when the evaluation value corresponding to the first mobile route exceeds a threshold value, and the second route information is selected when the evaluation value is equal to or less than the threshold value.
8. A control object; A movement control device that controls the movement of the control object; a visual sensor that captures an image of a space including at least a part of a moving path from a current position to a target position of an object that is the same as or different from the control target object, and outputs image information relating to the image of the space; The movement control device includes: a first generation process of determining a first movement path of the object for visual feedback control using the image information output from the visual sensor, and generating first path information indicating the first movement path; a second generation process of determining a second movement path of the object for model-based control based on machine learning using the image information or a feature set generated from the image information, and generating second path information indicating the second movement path; a movement control process for selecting one of the first route information and the second route information, and performing the movement control so as to move the object to the target position along a movement route specified by the one of the route information; A movement control system comprising:
Citation Information
Patent Citations
Robot control system for simultaneously performing workpiece selection and robot work
JP2021024027A
