Robot control device

Abstract

A robot control device (30) controls a robot (10) that operates in a shared work area with a person. The robot control device (30) includes an image recognition processing unit (33), a robot control processing unit (32), and a monitoring processing unit (36). The image recognition processing unit (33) recognizes first information related to a person existing in the monitoring area based on measurement data in the monitoring area obtained from a visual sensor (20). The robot control processing unit (32) controls the movement of the robot (10) according to an action program for moving the robot (10). The monitoring processing unit (36) determines the possibility of a person being sandwiched between the robot (10) and the surrounding objects based on the surrounding object data and the first information obtained from the image recognition processing unit (33). The surrounding object data is data indicating the three-dimensional configuration state of the robot (10) in the monitoring area and objects other than the robot (10), namely the surrounding objects.

Description

Technical Field

[0001] The present invention relates to a robot control device, a learning device, and an inference device for controlling a robot that shares a working space with a human and operates. Background Art

[0002] In the field of industrial robots, the development of human-coordinated robot systems in which a human, i.e., an operator, and a robot share a working space without being separated by a safety fence is progressing. In a human-coordinated robot system, the operator may intrude into the movable range of the robot, causing interference between the two. Therefore, a technology for preventing interference between the operator and the robot has been proposed.

[0003] Patent document 1 discloses a robot control method, namely, measuring the distance between the robot and an operator existing near the robot identified based on image information captured by a camera, and when the measured distance is a distance that will interfere with the operator, estimating which part of the robot and the operator will interfere and how, and restricting the movement of the robot based on the result.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2008-137127 Summary of the invention

[0005] However, according to the above-mentioned prior art, the interference between the operator and the robot is estimated based on image information only, but the objects arranged around the operator and the robot, i.e., the surrounding objects, are not considered. That is, in the prior art, there is a problem that the possibility of the operator being sandwiched between the robot and the surrounding objects is not considered. For example, in a state where a part of the operator's body is between the robot and the surrounding objects, the interference mode between the operator and the robot is estimated based on the distance between the operator and the robot. As a result, when the robot's movement is set to a low speed, since the existence of the surrounding objects is not considered, there is a possibility that a part of the operator's body is sandwiched between the robot and the surrounding objects.

[0006] The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to obtain a robot control device that can suppress the occurrence of a human being being caught between a robot and surrounding objects.

[0007] In order to solve the above-mentioned problems and achieve the purpose, the present invention is a robot control device that controls a robot that operates in a shared working area with a person, and the robot control device has an image recognition processing unit, a robot control processing unit, and a monitoring processing unit. The image recognition processing unit recognizes information related to a person existing in the monitoring area, i.e., first information, based on measurement data in the monitoring area obtained from a visual sensor. The robot control processing unit controls the movement of the robot according to an action program that causes the robot to move. The monitoring processing unit determines the possibility of a person being sandwiched between the robot and the surrounding objects based on the surrounding object data and the first information obtained from the image recognition processing unit. The surrounding object data is data representing the three-dimensional configuration state of the robot in the monitoring area and objects other than the robot, i.e., surrounding objects.

[0008] Effects of the Invention

[0009] The robot control device according to the present invention has the effect of being able to suppress the occurrence of a person being caught between the robot and surrounding objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a block diagram showing an example of the configuration of a robot system including the robot control device according to the first embodiment.

[0011] Figure 2 This is a diagram showing an example of human body influence information.

[0012] Figure 3 This is a flowchart showing an example of the procedure of the robot control method involved in the first embodiment.

[0013] Figure 4 This is a flowchart showing an example of the procedure of the robot control method involved in the first embodiment.

[0014] Figure 5 It is a graph showing the relationship between humans, robots, and surrounding objects.

[0015] Figure 6 This is a block diagram showing an example of the configuration of a robot system including a robot control device according to the second embodiment.

[0016] Figure 7 This is a diagram for explaining a method of generating an access frequency correspondence map.

[0017] Figure 8 This is a diagram showing an example of an access frequency correspondence map.

[0018] Fig. 9This is a block diagram showing an example of the configuration of a learning device in a robot control device according to the second embodiment.

[0019] Fig.10 This is a flowchart showing an example of the procedure of the learning process of the learning device included in the robot control device according to the second embodiment.

[0020] Fig.11 This is a block diagram showing an example of the configuration of an estimation device in a robot control device according to the second embodiment.

[0021] Fig.12 This is a flowchart showing an example of the procedure of the estimation process performed by the estimation device included in the robot control device according to the second embodiment.

[0022] Fig.13 This is a block diagram showing an example of the hardware configuration of the robot control device according to the first and second embodiments. DETAILED DESCRIPTION

[0023] Hereinafter, a robot control device, a learning device, and an inference device according to embodiments of the present invention will be described in detail with reference to the drawings.

[0024] Implementation method 1.

[0025] Figure 1 This is a block diagram showing an example of the configuration of a robot system including a robot control device according to Embodiment 1. The robot system 1 includes a robot 10 , a vision sensor 20 , and a robot control device 30 .

[0026] In one example, the robot 10 has a plurality of arms and a driving unit provided at each joint of each arm and controlling the angle of the joint. The robot 10 can be set to various postures in accordance with the motion command from the robot control device 30. By setting to various postures, the robot 10 can move at a predetermined range of positions centered at a fixed position. An example of the driving unit is an electric motor represented by a servo motor or a stepping motor, or a pressure cylinder using air pressure or hydraulic pressure.

[0027] The visual sensor 20 captures a predetermined area including the robot 10, that is, the monitoring area of ​​the robot system 1. The visual sensor 20 is a sensor that can obtain measurement data including a distance image and a color image. The distance image includes the depth of objects in the monitoring area, and the color image distinguishes between people and surrounding objects other than people. An example of the visual sensor 20 is a 2D camera or a 3D camera. Color information can be obtained by a 2D camera, and position information can be obtained by a 3D camera. The 3D camera can use a stereo method, a time of flight (ToF) method, or a projection method.

[0028] In the monitoring area of ​​the robot system 1, there are surrounding objects other than the robot 10. Examples of the surrounding objects are a workbench, a wall, a shelf, a door, and a processing machine.

[0029] The robot control device 30 controls the motion of the robot 10 according to a predetermined program for moving the robot 10, that is, an motion program. In the control process of the robot 10, the robot control device 30 controls the motion of the robot 10 so that the robot 10 and the person do not come into contact, based on the image capturing result of the visual sensor 20, when there is a person around the robot 10. The robot control device 30 includes an motion program storage unit 31, a robot control processing unit 32, an image recognition processing unit 33, a human body influence degree information storage unit 34, a surrounding object data storage unit 35, and a monitoring processing unit 36.

[0030] The operation program storage unit 31 stores an operation program describing the operation of the robot 10 .

[0031] The robot control processing unit 32 downloads the motion program from the motion program storage unit 31 and executes it, and controls the robot 10 according to the execution result. In addition, when the robot control processing unit 32 controls the robot 10 according to the motion program, when the monitoring processing unit 36 ​​issues an instruction to limit the motion of the robot 10, the robot control processing unit 32 controls the robot 10 according to the instruction.

[0032] The image recognition processing unit 33 recognizes the first information related to the person existing in the monitoring area based on the measurement data in the monitoring area obtained from the visual sensor 20. An example of the measurement data is a distance image or a color image. In one example, the image recognition processing unit 33 uses the color image in the measurement data to determine whether the person exists in the monitoring area, and recognizes the human body parts when the person exists in the monitoring area. The human body parts are the head, torso, upper wrist, forearm, hand, thigh, lower leg, foot, etc. The image recognition processing unit 33 can recognize the human body parts by comparing the data of the human body parts stored in advance with the measurement data. When a person exists in the monitoring area, the image recognition processing unit 33 uses the distance image in the measurement data to recognize the position and posture of the recognized person, more specifically, the first position and posture information including the position and posture related to the human body parts and the first state information including the state of the person. The first position and posture information indicates the position and posture of the person and the human body parts in the coordinate system used by the measurement data measured by the visual sensor 20, that is, the camera coordinate system. The first state information is information indicating what kind of action a human body part has performed, and in one example, is information including the moving direction and speed of the human body part. When stopped, the moving direction and speed become "0". The first position and posture information and the first state information are included in the first information. In addition, the image recognition processing unit 33 can also further identify the information related to the robot 10, that is, the second information. The second information is about the part of the robot 10, and includes the second position and posture information and the second state information, the second position and posture information is information including the position and posture, and the second state information is information including the state.

[0033] The human body influence degree information storage unit 34 stores human body influence degree information indicating the degree of influence on the human body when the robot 10 contacts the human body. The human body influence degree information defines the degree of influence exerted on the human body when the robot 10 contacts each human body part. Figure 2 1 is a diagram showing an example of human body influence information. In one example, human body influence information is information that uses the contact state between a human and a robot 10 as input data and the influence degree applied to the human body as output data. Figure 2 In the example of , the input data includes items of a contact part of a person, a state of the person, a contact part of the robot 10, and a state of the robot 10. In addition, Figure 2The items of input data are examples and are not limited to these. The output data has items of the degree of influence exerted on the human body. The items constituting the input data are information obtained based on the results of the recognition processing performed by the image recognition processing unit 33 and the results of the determination processing performed by the monitoring processing unit 36. An example of the degree of influence exerted on the human body is the degree of impact given to the human body, the degree of injury or damage to the human body. For example, in the case where damage is exerted on the human body, the degree of influence exerted on the human body is set to "high", and in the case where no damage is exerted on the human body, the degree of influence exerted on the human body is set to "low". Figure 2 In the example, when the contact part of the person is "head", the state of the person is "moving", the contact part of the robot 10 is "all parts", and the state of the robot 10 is "moving speed>0.0m / s", the influence degree applied to the human body becomes "high". In one example, the human body influence degree information is the "Biomechanical limits" shown in Table A.2 of ISO / TS (International Organization for Standardization / Technical Specifications) 15066:2016.

[0034] The peripheral object data storage unit 35 stores information including the three-dimensional configuration state of the robot 10 and objects other than the robot 10, i.e., the peripheral objects, existing in the monitoring area, i.e., the peripheral object data. The peripheral object data includes three-dimensional CAD (Computer-Aided Design) data representing the shape and size of the object and position and posture data representing the position and posture of the object. In one example, the position and posture data is data representing the position and posture based on the placement position of the robot 10. The peripheral object data is data that can reproduce the configuration state of objects including the robot 10 in the monitoring area in three dimensions. The peripheral object data is expressed using the coordinate system used by the peripheral object data, i.e., the robot coordinate system.

[0035] The monitoring processing unit 36 ​​uses the recognition result of the image recognition processing unit 33, considers the distance between the person and the robot 10, the possibility of contact, and the degree of influence when contacted, and outputs a command to restrict the action of the robot 10 to the robot control processing unit 32 when the robot 10 may exert an influence on the person. The specific processing is described below.

[0036] The monitoring processing unit 36 ​​measures the distance between the person and the robot 10, i.e., the measured distance, based on the recognition result in the image recognition processing unit 33. The monitoring processing unit 36 ​​determines whether the measured distance is a distance in which the robot 10 and the person do not contact each other due to the action of the robot 10, i.e., a non-contact distance. The non-contact distance is obtained from the action stop data, which is data indicating how long the distance is required in which direction to stop the robot 10 from the moment when it is photographed by the visual sensor 20. Thus, the monitoring processing unit 36 ​​measures the measured distance between the person and the robot 10, including the distance of inertial movement until the robot 10 stops. When the measured distance is the non-contact distance, the monitoring processing unit 36 ​​does not impose restrictions on the robot 10 and continues the current operation. When the measured distance is not the non-contact distance, the monitoring processing unit 36 ​​sends a command to the robot control processing unit 32 to change so that the action speed of the robot 10 slows down.

[0037] When the measured distance is not a non-contact distance, that is, when it is a distance at which the human and the robot 10 may come into contact, the monitoring processing unit 36 ​​determines whether the robot 10 is in contact with the human. In one example, the monitoring processing unit 36 ​​determines whether the robot 10 is in contact with the human based on the recognition result in the image recognition processing unit 33, that is, the first information of the human and the second information of the robot 10. As described above, the first information includes the first position and posture information of the human body part and the first state information, and the second information includes the second position and posture information of the robot 10 part and the second state information. In addition, when the robot 10 is in contact with the human, the monitoring processing unit 36 ​​predicts the contact part and the state of the contact between the robot 10 and the human body part based on the predicted movement of the human body part and the predicted movement of the robot 10. In one example, the monitoring processing unit 36 ​​determines the contact position and posture information including the contacted human body part and the part and posture of the robot 10, and the state of the contacted human body part and the robot 10, that is, the contact state information. In one example, the contact position and posture information includes the human body part in contact with the robot 10 and the contact angle, and the robot 10 part in contact with the human and the contact angle. The contact state information includes the moving direction and speed of the human body and the robot 10 part in contact. When the moving direction and speed are both "0" in the contact state information, it indicates that the robot is stopped.

[0038] When determining the action of the robot 10 in contact with a person, the second position and posture information and the second state information of the parts of the robot 10 may not be the recognition result of the image recognition processing unit 33, but the result of simulation of the robot 10 moving according to the action program using the robot control processing unit 32. When the robot 10 is not in contact with a person, the monitoring processing unit 36 ​​allows the action to continue while limiting the action of the robot 10.

[0039] When the robot 10 is in contact with a person, the monitoring processing unit 36 ​​extracts the degree of influence applied to the human body based on the determined contact position and state of the robot 10 and the person, and with reference to the human body influence degree information of the human body influence degree information storage unit 34. That is, the monitoring processing unit 36 ​​extracts the human body influence degree information corresponding to the combination of the position and posture information at the time of contact and the state information at the time of contact. Moreover, the monitoring processing unit 36 ​​outputs the instruction of the movement of the robot 10 to the robot control processing unit 32 in accordance with the obtained degree of influence applied to the human body. In one example, when the degree of influence applied to the human body is high, the monitoring processing unit 36 ​​sends an instruction to further restrict the movement of the robot 10 to the robot control processing unit 32. An example of limiting the movement of the robot 10 is stopping the robot 10, moving the robot 10 in a direction away from the person, etc.

[0040] In addition, when the degree of influence exerted on the human body by the contact between the robot 10 and the human is not high, the monitoring processing unit 36 ​​determines the possibility of the human being being sandwiched between the robot 10 and the surrounding objects based on the surrounding object data indicating the three-dimensional arrangement state of the robot 10 and the surrounding objects in the monitoring area and the first information obtained from the image recognition processing unit 33, and determines whether it is necessary to further restrict the movement of the robot 10 or output a warning sound. Regarding the possibility of the human being being sandwiched between the robot 10 and the surrounding objects, the monitoring processing unit 36 ​​adds the position information of the human to the surrounding object data including the position, shape and size of the robot 10 and the surrounding objects in the monitoring area based on the first information, and predicts the movement of the human and the robot 10 based on the first information. In addition, the monitoring processing unit 36 ​​includes not only the first information but also the second information, and can determine the possibility of the human being being sandwiched between the robot 10 and the surrounding objects. In this case, the monitoring processing unit 36 ​​predicts the movement of the robot 10 based on the second information including the second position posture information and the second state information.

[0041] Specifically, the monitoring processing unit 36 ​​adds the position information of the person in the camera coordinate system recognized by the image recognition processing unit 33, that is, the position information of each part of the human body, to the peripheral object data of the robot coordinate system, simulates the movement of the person and the movement of the robot 10, and determines whether the person is sandwiched between the robot 10 and the peripheral objects. At this time, the camera coordinate system and the robot coordinate system are calibrated in advance. Thus, the coordinate transformation matrix of the camera coordinate system and the robot coordinate system is calculated. By using this coordinate transformation matrix, the position of the person in the camera coordinate system recognized by the visual sensor 20 can be transformed into the robot coordinate system. In addition, the position and posture of the peripheral objects are also expressed using coordinates based on the robot 10. Therefore, the monitoring processing unit 36 ​​can grasp the positional relationship between the robot 10, the peripheral objects and the person through the robot coordinate system, and determine whether the person is in the sandwiched position based on the positional relationship. The simulation of the movement of the person can be performed based on the first information. In addition, the simulation of the movement of the robot 10 can be performed using an action program or based on the second information.

[0042] When there is no possibility of human interposition, the monitoring processing unit 36 ​​continues the movement while restricting the movement of the robot 10. That is, the monitoring processing unit 36 ​​does not further restrict the movement. In addition, when there is a possibility of human interposition, the monitoring processing unit 36 ​​outputs a command to restrict the movement of the robot 10, such as stopping the robot 10 or moving the robot 10 away from the human, to the robot control processing unit 32, or emits a warning sound.

[0043] Next, the operation of the robot control device 30 in the robot system 1 having the above-described configuration will be described. Figure 3 and Figure 4 1 is a flowchart showing an example of the procedure of the robot control method according to Embodiment 1. In addition, here, the process based on the premise that a person exists in the monitoring area is described.

[0044] First, the visual sensor 20 captures the monitored area and sends the captured data as measurement data to the robot control device 30. The image recognition processing unit 33 of the robot control device 30 recognizes the first position and posture information and the first state information of the human body part of the person existing in the monitored area based on the received measurement data (step S11). The first position and posture information is information containing the position and posture of the human body part, and the first state information is information containing the state of the human body part. Next, the monitoring processing unit 36 ​​uses the first position and posture information and the first state information of the human body part recognized by the image recognition processing unit 33 to measure the measurement distance between the recognized person and the robot 10 (step S12).

[0045] Then, the monitoring processing unit 36 ​​determines whether the measured distance is a non-contact distance where the person and the robot 10 do not come into contact (step S13). In one example, the non-contact distance is the distance that the robot 10 moves from the state captured by the visual sensor 20 until the robot 10 receives a stop signal until the robot 10 stops. If the measured distance is greater than the non-contact distance, the robot 10 is not in contact with the person, and if the measured distance is less than the non-contact distance, the robot 10 may come into contact with the person.

[0046] When the measured distance is greater than the non-contact distance (if Yes in step S13 ), the robot 10 cannot come into contact with the person, so the monitoring processing unit 36 ​​does not restrict the movement of the robot 10 , and continues the movement under the current movement program, and the process ends.

[0047] In addition, when the measured distance is less than the non-contact distance (when the answer is No in step S13), the monitoring processing unit 36 ​​outputs a command to change the motion speed of the robot 10 to the robot control processing unit 32 (step S14). If the robot control processing unit 32 receives the command, it changes the motion speed of the robot 10 according to the command (step S15). Specifically, the monitoring processing unit 36 ​​outputs a deceleration command to the robot control processing unit 32 in order to decelerate the motion speed of the robot 10. The motion speed of the robot 10 included in the deceleration command is a motion speed at which the robot 10 has little effect on the human body even if it contacts the human body, and in one example, it is 0.25 m / s.

[0048] Next, the monitoring processing unit 36 ​​uses the recognition result of the image recognition processing unit 33 to determine whether the movement of the robot 10 is likely to come into contact with a person (step S16). In one example, in step S11, the image recognition processing unit 33 recognizes the second position and posture information and the second state information of the parts of the robot 10, and uses the first position and posture information and the first state information of the human body parts and the second position and posture information and the second state information of the parts of the robot 10 to predict the movement of the person and the robot 10, and determines the possibility of the robot 10 and the person coming into contact. Alternatively, in other examples, the possibility of the robot 10 and the person coming into contact is determined by combining the simulation of the movement of the robot 10 according to the action program and the prediction using the first position and posture information and the first state information of the human body parts.

[0049] When it is determined that the movement of the robot 10 is unlikely to come into contact with a person (when step S16 is No), the monitoring processing unit 36 ​​continues the movement of the robot 10 while maintaining the movement speed of the robot 10 changed in step S15, and the processing ends. In addition, when it is determined that the movement of the robot 10 is likely to come into contact with a person (when step S16 is Yes), the monitoring processing unit 36 ​​uses the recognition result in the image recognition processing unit 33 to determine the position and posture information of the robot 10 and the human body when in contact (step S17). In addition, the monitoring processing unit 36 ​​uses the recognition result in the image recognition processing unit 33 to obtain the state information of the robot 10 and the human body when in contact (step S18). An example of the state information of the human body when in contact is the direction and moving speed of the head, the direction and moving speed of the shoulder, etc. Similarly, an example of the state information of the robot 10 when in contact is the direction and moving speed of the front end of the arm, etc.

[0050] Then, the monitoring processing unit 36 ​​obtains the influence degree applied to the human body from the contact position and posture information and contact state information of the robot 10 and the human body, and refers to the human body influence degree information to obtain the influence degree applied to the human body corresponding to the input data from the human body influence degree information (step S19). Specifically, the monitoring processing unit 36 ​​uses the combination of the contact position and posture information and contact state information of the robot 10 and the human body as input data, and obtains the influence degree applied to the human body corresponding to the input data, that is, output data, from the human body influence degree information. The monitoring processing unit 36 ​​determines whether the influence degree applied to the human body is higher than a predetermined reference value (step S20). In the case where the influence degree applied to the human body is higher than the reference value, the influence on the injury to the human body is large, and in the case where the influence degree applied to the human body is less than or equal to the reference value, the influence on the injury to the human body is slight. The influence degree applied to the human body is associated with whether the human body part contacted by the robot 10 is a vital part, the movement speed of the robot 10, the human body part that may be contacted, the position, the state, and other information.

[0051] When the degree of influence applied to the human body is less than or equal to the reference value (in the case of No in step S20), the monitoring processing unit 36 ​​adds the position information of the person recognized by the image recognition processing unit 33 to the surrounding object data (step S21), and predicts the actions of the robot 10 and the person in the environment including the surrounding objects (step S22). In one example, the monitoring processing unit 36 ​​simulates the actions of the robot 10 and the person using the surrounding object data. At this time, the position and posture information of the robot 10 and the human body at the time of contact, the state information at the time of contact, the action program, etc. can be used.

[0052] The monitoring processing unit 36 ​​determines whether the human may be caught between the robot 10 and the surrounding objects based on the prediction result and the relationship between the human, the robot 10 and the surrounding objects (step S23 ). Figure 51 is a diagram showing the relationship between a person, a robot, and surrounding objects. In this diagram, the area between the robot 10 and the workbench 510 as a surrounding object becomes a possible pinching area R1, and a hand 501 of the person 500 exists in the possible pinching area R1. In this case, it is determined that the hand 501 of the person 500 is likely to be pinched between the robot 10 and the workbench 510.

[0053] In the case where a person may be caught between the robot 10 and the surrounding objects (if step S23 is Yes), or in the case where the degree of influence applied to the human body in step S20 is higher than the reference value (if step S20 is Yes), the monitoring processing unit 36 ​​outputs an instruction to limit the movement of the robot 10 so that the person and the robot 10 do not come into contact with each other to the robot control processing unit 32 (step S24). If the robot control processing unit 32 receives the instruction, it limits the movement of the robot 10 based on the instruction (step S25). The restriction of the movement of the robot 10 includes stopping the movement of the robot 10, further deceleration, and movement of the robot 10 away from the person. In addition, in steps S24 and S25, it is shown that the monitoring processing unit 36 ​​outputs an instruction to limit the movement of the robot 10 to the robot control processing unit 32, and the robot control processing unit 32 limits the movement of the robot 10, but in step S24, the monitoring processing unit 36 ​​can also output a warning sound. The above processing ends.

[0054] If it is unlikely that a person will be caught between the robot 10 and the surrounding objects (if No in step S23 ), it is recognized that the influence on the human body is not so great, and the operation of the robot 10 is maintained as it is, and the processing ends.

[0055] According to the robot control device 30 of the first embodiment, the image recognition processing unit 33 uses the measurement data from the visual sensor 20 to obtain the first position and posture information and the first state information of the human body part of the person existing in the monitoring area. The monitoring processing unit 36 ​​adds the position of the person to the peripheral object data including the three-dimensional arrangement state, shape and size related to the robot 10 and the peripheral objects, predicts the movement of the person and the robot 10, and determines whether the person is likely to be sandwiched between the robot 10 and the peripheral objects. In the case where the person is likely to be sandwiched between the robot 10 and the peripheral objects, an instruction to restrict the movement of the robot 10 so that the person is not sandwiched between the robot 10 and the peripheral objects is output to the robot control processing unit 32. As a result, even if the impact of the robot 10 on the person is small, it is possible to suppress the possibility of the person being sandwiched between the robot 10 and the peripheral objects due to the continuation of the movement of the robot 10, thereby increasing the impact on the human body. In addition, since the robot 10 does not cause serious damage to the person due to the contact with the person, it can be used at a close distance to the person, and a robot 10 with high versatility can be realized.

[0056] Implementation method 2.

[0057] Conventionally, there has been no technology proposed for shortening the motion path of the robot 10 to a target position as much as possible while reducing the possibility of contact between the robot 10 and the person and the possibility of the person being sandwiched between the robot 10 and the surrounding objects when a person is present in the area where the robot 10 and the surrounding objects are arranged. In Embodiment 2, a robot control device is described that can shorten the motion path of the robot 10 to a target position as much as possible while reducing the possibility of contact between the robot 10 and the person and the possibility of the person being sandwiched between the robot 10 and the surrounding objects.

[0058] Figure 6 1 is a block diagram showing an example of the structure of a robot system including a robot control device according to Embodiment 2. In addition, the same reference numerals are given to the same structural elements as those in Embodiment 1, and their description is omitted. The robot control device 30 further includes a human analysis processing unit 37, an access frequency correspondence map storage unit 38, a learning device 39, a trained model storage unit 40, and an inference device 41 in the structure of Embodiment 1.

[0059] The human analysis processing unit 37 generates an access frequency map which is information indicating access conditions of humans in the monitoring area during a predetermined period, based on the recognition result of the image recognition processing unit 33 . Figure 7 This is a diagram for explaining a method for generating an access frequency correspondence graph. Figure 76 shows image data 600 of the monitoring area captured by the visual sensor 20. Here, image data 600 of the monitoring area captured from above is shown. The image data 600 of the monitoring area is divided into a plurality of small areas 601 in a rectangular shape. Figure 7 In the image data 600, the lower half is the area where the robot 10 and the surrounding objects 520 are arranged, and the upper half is the area 610 where the person 500 can move. In addition, the circular area centered on the position of the robot 10 is the range in which the robot 10 can move, that is, the robot action area 620.

[0060] In the initial state, the access frequency of each small area 601 is 0. The human analysis processing unit 37 determines to which small area 601 the position of the person 500 recognized by the image recognition processing unit 33 belongs, and adds "1" to the access frequency of the small area 601 to which the person 500 belongs. The human analysis processing unit 37 performs this processing for a predetermined period, thereby generating an access frequency correspondence map.

[0061] Figure 8 is a diagram showing an example of an access frequency correspondence graph. Figure 8 In, set it as Figure 7 The access frequency correspondence map is generated by using the image data 600 of the monitoring area. Figure 8 As shown, by performing the above-mentioned processing, the frequency of the person 500 belonging to each small area 601 is displayed. By referring to the access frequency correspondence map, it is possible to know at which position in the monitoring area the person 500 is likely to exist.

[0062] The access frequency map storage unit 38 stores the access frequency map in the monitoring area generated by the human analysis processing unit 37. The access frequency map is data prepared for machine learning.

[0063] The learning device 39 generates a trained model that suppresses the deceleration or stopping of the movement of the robot 10 based on the movement path of the robot 10 and the status of the person 500, the robot 10 and the surrounding objects 520, and learns the movement path of the robot 10 that suppresses the robot 10 from contacting the person 500 and the person 500 from being sandwiched between the robot 10 and the surrounding objects 520.

[0064] The trained model storage unit 40 stores the trained model learned by the learning device 39 .

[0065] The inference device 41 inputs the target position of the robot 10 and the states of the robot 10, the person 500 and the surrounding objects 520 into the trained model stored in the trained model storage unit 40, thereby inferring the movement path of the robot 10 suitable for the states of the robot 10, the person 500 and the surrounding objects 520.

[0066] Next, the learning in the learning device 39 and the inference in the inference device 41 are described in detail.

[0067] <Learning Stage>

[0068] Fig. 9 This is a block diagram showing an example of the configuration of a learning device in the robot control device according to Embodiment 2. The learning device 39 includes a data acquisition unit 391 and a model generation unit 392 .

[0069] The data acquisition unit 391 acquires the motion path of the robot 10 and the state data representing the state of the person 500, the robot 10, and the surrounding objects as learning data. The state data includes the first information of the person 500, the target position and second information of the robot 10, and the surrounding object data. Here, the case where the first information is the first position and posture information and the second information is the second position and posture information is described. The motion path and target position of the robot 10 can be obtained by simulating the motion program. The motion path of the robot 10 is the motion path in the combination of the state of the person 500, the robot 10, and the surrounding objects 520, that is, the first position and posture information of the person 500, the target position and second position and posture information of the robot 10, and the surrounding object data.

[0070] The model generation unit 392 learns the movement path of the robot 10 that suppresses the deceleration or stop of the movement of the robot 10 from the state of the robot 10, the person 500, and the surrounding objects 520, and suppresses the contact of the robot 10 with the person 500 and the sandwiching of the person 500 between the robot 10 and the surrounding objects 520, based on the learning data including the movement path and state data of the robot 10. In other words, the model generation unit 392 generates a trained model for inferring the movement path of the robot 10 that reduces the possibility of contact with the person 500 and sandwiching of the person 500 from the state of the person 500, the robot 10, and the surrounding objects 520.

[0071] The learning algorithm used by the model generation unit 392 can use well-known learning algorithms such as teacher learning, teacher-free learning, and reinforcement learning. As an example, the case where reinforcement learning is applied is described. In reinforcement learning, an agent acting in a certain environment observes the current state, i.e., the parameters of the environment, and decides the action to be taken. The environment changes dynamically through the actions of the agent, and rewards are given to the agent in accordance with the changes in the environment. The agent repeats and learns the action policy that obtains the most rewards through a series of actions. As representative methods of reinforcement learning, Q-learning, TD-learning, etc. are known. For example, in the case of Q-learning, the general update formula of the action value function Q(s, a) is expressed by the following formula (1).

[0072] [Formula 1]

[0073]

[0074] In formula (1), s t represents the state of the environment at time t, a t represents the action at time t. By action a t , the state changes to s t+1 . r t+1 represents the reward brought by the change of its state, γ represents the discount rate, and α represents the learning coefficient. In addition, γ is in the range of 0<γ≤1, and α is in the range of 0<α≤1. The action path of robot 10 becomes action a t , the state of the person 500, the robot 10 and the surrounding object 520 becomes state s t , for the state s at time t t The best action in a t To study.

[0075] The update formula represented by formula (1) is that if the action value Q of action a with the highest Q value at time t+1 is greater than the action value Q of action a executed at time t, then the action value Q is increased, and in the opposite case, the action value Q is decreased. In other words, the action value function Q(s, a) is updated so that the action value Q of action a at time t is close to the best action value at time t+1. Thus, the best action value Q in a certain environment is continuously propagated to the action value Q in its previous environment.

[0076] As described above, when a trained model is generated by reinforcement learning, the model generation unit 392 includes a reward calculation unit 393 and a function update unit 394 .

[0077] The reward calculation unit 393 calculates the reward based on the motion path of the robot 10, the person 500, and the state of the robot 10 and the surrounding objects 520. The reward calculation unit 393 calculates the reward r based on at least one of the reward criteria, namely, the motion time of the robot 10, the degree of influence exerted on the human body when the robot 10 contacts the person 500, and the access frequency of the person 500 in the robot motion area 620. In one example, the motion time of the robot 10 is the time taken to move from a certain position A to another position B as a target position. The shorter the motion time of the robot 10, the higher the productivity becomes, so it is preferred that the motion time is short. As a method of shortening the motion time, a method of increasing the motion speed of the robot 10 and a method of shortening the motion path of the robot 10 are conceivable.

[0078] For example, when the action time of the robot 10 is reduced, when the degree of influence exerted on the human body during contact is low, or when the access frequency of the person 500 is low, the reward calculation unit 393 increases the reward r. In one example, the reward calculation unit 393 increases the reward r by assigning a value of the reward, i.e., "1". In addition, the value of the reward is not limited to "1". On the other hand, when the action time of the robot 10 is increased, when the degree of influence exerted on the human body during contact is high, or when the access frequency of the person 500 is high, the reward calculation unit 393 decreases the reward r. In one example, the reward calculation unit 393 decreases the reward r by assigning a value of the reward, i.e., "-1". In addition, the value of the reward is not limited to "-1".

[0079] Furthermore, efficient learning can be performed by combining the operation time of the robot 10 , the degree of influence exerted on the human body when the robot 10 contacts the human 500 , and the access frequency of the human 500 in the robot operation area 620 as a reward criterion.

[0080] The function updating unit 394 updates the function for determining the action path of the robot 10 according to the reward calculated by the reward calculating unit 393, and outputs it to the trained model storage unit 40. For example, in the case of Q learning, the action value function Q(s t , a t ) is used as a function for calculating the motion path of the robot 10. The above learning is repeatedly performed.

[0081] The trained model storage unit 40 stores the action-value function Q(s) updated by the function updating unit 394. t , a t ), that is, the trained model is stored.

[0082] Next, the learning process performed by the learning device 39 will be described. Fig.10 This is a flowchart showing an example of the procedure of the learning process of the learning device included in the robot control device according to the second embodiment.

[0083] First, the data acquisition unit 391 acquires the motion path of the robot 10 and state data indicating the states of the person 500, the robot 10, and the surrounding objects 520 as learning data (step S51). In one example, the state data includes the first position and posture information of the person 500, the target position and second position and posture information of the robot 10, and the surrounding object data.

[0084] Next, the model generation unit 392 calculates the reward based on the action path of the robot 10 and the state data indicating the state of the person 500, the robot 10, and the surrounding objects 520, and determines whether to increase the reward (step S52). Specifically, the reward calculation unit 393 obtains the action path of the robot 10 and the state of the person 500, the robot 10, and the surrounding objects 520, and determines whether to increase or decrease the reward based on at least one of the action time of the robot 10, the degree of influence exerted on the human body when the robot 10 contacts the person 500, and the visit frequency of the person 500 in the robot action area 620, i.e., a predetermined reward benchmark.

[0085] If it is determined in step S52 that the reward is to be increased, the reward calculation unit 393 increases the reward (step S53). On the other hand, if it is determined in step S52 that the reward is to be decreased, the reward calculation unit 393 decreases the reward (step S54).

[0086] After step S53 or S54, the function updating unit 394 updates the action value function Q(s) represented by equation (1) stored in the trained model storage unit 40 based on the reward calculated by the reward calculation unit 393. t , a t ) is updated (step S55).

[0087] The learning device 39 repeatedly executes the above steps S51 to S55, and generates the action-value function Q(s t , a t ) is stored as a trained model in the trained model storage unit 40.

[0088] The learning device 39 according to Embodiment 2 stores the trained model in the trained model storage unit 40 which is provided outside the learning device 39 , but the trained model storage unit 40 may be provided inside the learning device 39 .

[0089] <Effective use stage>

[0090] Fig.11 This is a block diagram showing an example of the configuration of an estimation device in the robot control device according to Embodiment 2. The estimation device 41 includes a data acquisition unit 411 and an estimation unit 412 .

[0091] The data acquisition unit 411 acquires state data indicating the states of the person 500, the robot 10, and the surrounding objects 520. In one example, the state data includes first position and posture information of the person 500, target position and second position and posture information of the robot 10, and surrounding object data.

[0092] The inference unit 412 uses the trained model to infer the motion path of the robot 10. That is, by inputting the state data acquired by the data acquisition unit 411, i.e., the states of the person 500, the robot 10, and the surrounding objects 520, into the trained model, the motion path of the robot 10, more specifically, the motion path of the robot 10 suitable for the states of the person 500, the robot 10, and the surrounding objects 520, can be inferred.

[0093] In addition, here, the output of the motion path of the robot 10 using the trained model learned by the model generation unit 392 of the learning device 39 of the robot control device 30 is described, but the trained model can also be obtained from other robot systems 1 and the motion path of the robot 10 can be output based on the trained model.

[0094] Next, the processing performed by the estimation device 41 to obtain the motion path of the robot 10 will be described. Fig.12 This is a flowchart showing an example of the procedure of the estimation process performed by the estimation device included in the robot control device according to the second embodiment.

[0095] First, the data acquisition unit 411 acquires state data indicating the states of the person 500 , the robot 10 , and the surrounding objects 520 as estimation data (step S71 ).

[0096] Next, the inference unit 412 inputs the state data, i.e., the states of the person 500, the robot 10, and the surrounding objects 520, as inference data to the trained model stored in the trained model storage unit 40 (step S72), and obtains the motion path of the robot 10. Then, the inference unit 412 outputs the data, i.e., the obtained motion path of the robot 10, to the robot control processing unit 32 (step S73).

[0097] Then, the robot control processing unit 32 controls the robot 10 using the outputted motion path of the robot 10 (step S74). Thus, the deceleration or stop of the motion of the robot 10 is suppressed, and the influence on the human body is reduced, so that the control of the robot 10 can be performed while taking into account the influence on the human body of the person 500 and the suppression of the pinching of the person 500 and the improvement of the operation rate of the robot 10, and a highly versatile robot system 1 can be realized.

[0098] In Embodiment 2, reinforcement learning is applied to the learning algorithm used by the inference unit 412, but the present invention is not limited thereto. Regarding the learning algorithm, in addition to reinforcement learning, supervised learning, unsupervised learning, or semi-supervised learning can also be applied.

[0099] In addition, as a learning algorithm used in the model generation unit 392, deep learning (Deep Learning) that learns the extraction of feature quantities themselves can also be used, or machine learning can be performed according to other well-known methods such as neural networks, genetic programming, functional logic programming, support vector machines, etc.

[0100] In addition, Figure 6 , the learning device 39 and the inference device 41 are shown as being built into the robot control device 30, but the learning device 39 and the inference device 41 may be connected to the robot control device 30 via a network, for example, and may be separate devices from the robot control device 30. In addition, the learning device 39 and the inference device 41 may also exist on a cloud server.

[0101] Furthermore, the model generation unit 392 can learn the motion path of the robot 10 using the learning data obtained from the plurality of robot control devices 30. In addition, the model generation unit 392 can also obtain the learning data from the plurality of robot control devices 30 used in the same area, and can also learn the motion path of the robot 10 using the learning data collected from the plurality of robot control devices 30 independently operating in different areas. In addition, the robot control device 30 that collects the learning data can be added to the object midway, or removed from the object. Furthermore, the learning device 39 that has learned the motion path of the robot 10 with respect to a certain robot control device 30 can be applied to other robot control devices 30, and the motion path of the robot 10 with respect to the other robot control devices 30 can be re-learned and updated.

[0102] In the second embodiment, the learning device 39 learns the motion path of the robot 10 that suppresses the deceleration or stop of the motion of the robot 10 from the state of the human 500, the robot 10, and the surrounding objects 520, and suppresses contact with the human 500 and the pinching of the human 500 based on the learning data including the motion path and state data of the robot 10. This has the effect that the motion path of the robot 10 that suppresses the deceleration or stop of the motion of the robot 10 from the state of the human 500, the robot 10, and the surrounding objects 520 and reduces the degree of influence on the human body can be learned.

[0103] In the second embodiment, the inference device 41 uses the trained model to suppress the deceleration or stop of the robot 10 from the state of the human 500, the robot 10, and the surrounding objects 520, and infers the motion path of the robot 10 that reduces the influence applied to the human body, and outputs it to the robot control processing unit 32. In this way, a highly versatile robot system 1 that takes into account both the reduction of the influence applied to the human body and the improvement of the operation rate of the robot 10 can be realized.

[0104] Here, yes Figure 1 and Figure 6 The hardware structure of the robot control device 30 shown in FIG. Fig.13 This is a block diagram showing an example of the hardware configuration of the robot control device 30 according to the first and second embodiments.

[0105] The robot control device 30 can be implemented by a hardware structure including a computing device 301 and a storage device 302. An example of the computing device 301 is a CPU (also called a Central Processing Unit, a central processing unit, a processing device, a microprocessor, a microcomputer, a processor, a DSP (Digital Signal Processor)) or a system LSI (Large Scale Integration). An example of the storage device 302 is a RAM (Random Access Memory) or a ROM (Read Only Memory).

[0106] The robot control device 30 is implemented by the computing device 301 reading and executing a program stored in the storage device 302 for executing the actions of the robot control device 30. In addition, the program can be said to cause the computer to execute the procedures or methods of the robot control device 30. In one example, the program causes the computer to execute Figure 3 and Figure 4 The robot control method shown.

[0107] The storage device 302 stores the action program, human body influence information, surrounding object data, access frequency correspondence map, and trained models. The storage device 302 is also used as a temporary memory when the computing device 301 executes various processes.

[0108] The program executed by the computing device 301 can be provided as a computer program product by storing it in a computer-readable storage medium in an installable or executable format. In addition, the program executed by the computing device 301 can also be provided to the robot control device 30 via a network such as the Internet.

[0109] In addition, the robot control device 30 may be implemented by dedicated hardware. In addition, regarding the functions of the robot control device 30, part of it may be implemented by dedicated hardware, and part of it may be implemented by software or firmware.

[0110] The configuration described in the above embodiment is merely an example, and may be combined with other known technologies, the embodiments may be combined with each other, and part of the configuration may be omitted or changed without departing from the gist.

[0111] Description of the label

[0112] 1 robot system, 10 robot, 20 visual sensor, 30 robot control device, 31 action program storage unit, 32 robot control processing unit, 33 image recognition processing unit, 34 human body influence information storage unit, 35 surrounding object data storage unit, 36 monitoring processing unit, 37 human analysis processing unit, 38 access frequency correspondence map storage unit, 39 learning device, 40 trained model storage unit, 41 inference device, 391, 411 data acquisition unit, 392 model generation unit, 393 feedback calculation unit, 394 function update unit, 412 inference unit, 500 person, 501 hand, 510 workbench, 520 surrounding objects, 600 image data, 601 small area, 620 robot action area.

Claims

1. A robot control device that controls a robot that operates in a shared work area with a human. The robot control device is characterized by having: an image recognition processing unit that recognizes first information related to a person existing in the monitoring area based on measurement data in the monitoring area obtained from the visual sensor; a robot control processing unit for controlling the movement of the robot according to a movement program for causing the robot to move; a monitoring processing unit that adds the position information of the person and the human body part based on the first information obtained from the image recognition processing unit to the surrounding object data including the position, shape and size indicating the three-dimensional arrangement state of the robot and objects other than the robot, i.e., surrounding objects, in the monitoring area, and predicts the movement of the person and the robot in an environment including the surrounding objects based on the added data, and determines the possibility of the person being sandwiched between the robot and the surrounding objects by determining whether the person is located at a position where the person is likely to be sandwiched, based on the positional relationship of the coordinates of the robot, the surrounding objects and the person obtained on the surrounding object data; and a human body influence degree information storage unit for storing human body influence degree information indicating the degree of influence on the human body when the robot and the human are in contact, The image recognition processing unit further recognizes second information related to the robot. The first information includes: first position and posture information including the position and posture of the person; and first state information including the moving direction and speed of the person, and the second information includes: second position and posture information including the position and posture of the robot; and second state information including the moving direction and speed of the robot, The monitoring processing unit determines the possibility of contact between the robot and the person using the first position and posture information, the first state information, the second position and posture information, and the second state information. When it is determined that there is a possibility of contact between the robot and the person, the monitoring processing unit uses the first position and posture information, the first state information, the second position and posture information, and the second state information to obtain the respective contact position and posture information and contact state information of the robot and the human body part, the contact position and posture information including the respective parts and postures of the contacted human body part and the robot, and the contact state information including the respective states of the contacted human body part and the robot. The monitoring processing unit obtains the influence exerted on the human body by referring to the human body influence degree information based on the contact position and posture information and the contact state information of each of the robot and the human. The monitoring processing unit determines whether the degree of influence is higher than a predetermined reference value. If the degree of influence is lower than the reference value, the monitoring processing unit determines the possibility of the person being sandwiched between the robot and the surrounding objects based on the positional relationship and whether the person is located in a position where the person may be sandwiched.

2. The robot control device according to claim 1, characterized in that: The monitoring processing unit adds the position information of the person based on the first information and adds information related to the robot based on the second information to the surrounding object data, and determines the possibility of the person being sandwiched between the robot and the surrounding objects based on the added data.

3. The robot control device according to claim 2, characterized in that: The monitoring processing unit predicts the movement of the robot based on the second position and posture information and the second state information.

4. The robot control device according to claim 1, characterized in that: The monitoring processing unit predicts the movement of the person based on the first information, predicts the movement of the robot based on the movement program, and determines the possibility of the person being sandwiched between the robot and the surrounding object.

5. The robot control device according to any one of claims 2 to 4, characterized in that: The monitoring processing unit predicts the movement of the person based on the first position and posture information and the first state information.

6. The robot control device according to claim 5, characterized in that: The monitoring processing unit outputs a command to the robot control processing unit to restrict the movement of the robot so that the person does not come into contact with the robot, when the influence degree is higher than a reference value.

7. The robot control device according to any one of claims 1 to 4 and 6, characterized in that: The monitoring processing unit is configured to output a command to restrict the movement of the robot to the robot control processing unit when there is a possibility that the person may be caught. The instruction for limiting the movement of the robot is to stop or slow down the robot or to move the robot away from the person.

8. The robot control device according to claim 1 or 4, characterized in that: Also provided is a learning device, the learning device having: a data acquisition unit that acquires learning data, the learning data including state data indicating states of the person, the robot, and the surrounding objects, and a motion path of the robot under the states of the person, the robot, and the surrounding objects; as well as a model generating unit, which generates a trained model using the learning data, wherein the trained model is used to infer the movement path of the robot so as to suppress the deceleration or stop of the movement of the robot from the state data and reduce the possibility of contact with the person and the person being sandwiched between the robot and the surrounding objects, The state data includes the first information, the target position of the robot, second information which is information related to the robot, and the peripheral object data.

9. The robot control device according to claim 2 or 3, characterized in that: Also provided is a learning device, the learning device having: a data acquisition unit that acquires learning data, the learning data including state data indicating states of the person, the robot, and the surrounding objects, and a motion path of the robot under the states of the person, the robot, and the surrounding objects; as well as a model generating unit, which generates a trained model using the learning data, wherein the trained model is used to infer the movement path of the robot so as to suppress the deceleration or stop of the movement of the robot from the state data and reduce the possibility of contact with the person and the person being sandwiched between the robot and the surrounding objects, The state data includes the first information, a target position of the robot, the second information, and the peripheral object data.

10. The robot control device according to claim 1 or 4, characterized in that: Also provided is an inference device, the inference device having: a data acquisition unit that acquires status data indicating the status of the person, the robot, and the surrounding objects; and an inference unit that outputs the movement path of the robot from the state data acquired by the data acquisition unit, using a trained model for suppressing the deceleration or stop of the movement of the robot based on the state of the person, the robot, and the surrounding objects, and for inferring the movement path of the robot that reduces the possibility of contact with the person and the person being sandwiched between the robot and the surrounding objects, The state data includes the first information, the target position of the robot, second information which is information related to the robot, and the peripheral object data.

11. The robot control device according to claim 2 or 3, characterized in that: Also provided is an inference device, the inference device having: a data acquisition unit that acquires status data indicating the status of the person, the robot, and the surrounding objects; and an inference unit that outputs the movement path of the robot from the state data acquired by the data acquisition unit, using a trained model for suppressing the deceleration or stop of the movement of the robot based on the state of the person, the robot, and the surrounding objects, and for inferring the movement path of the robot that reduces the possibility of contact with the person and the person being sandwiched between the robot and the surrounding objects, The state data includes the first information, a target position of the robot, the second information, and the peripheral object data.

12. A robot control device for controlling a robot that operates in a shared working area with a human being. The robot control device is characterized by having: an image recognition processing unit that recognizes first information related to a person existing in the monitoring area based on measurement data in the monitoring area obtained from the visual sensor; a robot control processing unit for controlling the movement of the robot according to a movement program for causing the robot to move; a monitoring processing unit that adds the position information of the person and the human body part based on the first information obtained from the image recognition processing unit to the surrounding object data including the position, shape and size indicating the three-dimensional arrangement state of the robot and objects other than the robot, i.e., surrounding objects, in the monitoring area, and predicts the movement of the person and the robot in an environment including the surrounding objects based on the added data, and determines the possibility of the person being sandwiched between the robot and the surrounding objects by determining whether the person is located at a position where the person is likely to be sandwiched, based on the positional relationship of the coordinates of the robot, the surrounding objects and the person obtained on the surrounding object data; and Learning device, The learning device comprises: a data acquisition unit that acquires learning data, the learning data including state data indicating states of the person, the robot, and the surrounding objects, and a motion path of the robot under the states of the person, the robot, and the surrounding objects; as well as a model generating unit, which generates a trained model using the learning data, wherein the trained model is used to infer the movement path of the robot so as to suppress the deceleration or stop of the movement of the robot from the state data and reduce the possibility of contact with the person and the person being sandwiched between the robot and the surrounding objects, The state data includes the first information, the target position of the robot, second information related to the robot, and the surrounding object data. The model generation unit has: a reward calculation unit that calculates a reward based on at least one reward criterion among an operation time of the robot, a degree of influence exerted on a human body when the robot contacts the human body, and a visit frequency of the human body within an operation area of ​​the robot; as well as A function updating unit updates a function for determining a motion path of the robot according to the reward calculated by the reward calculating unit.

Images