Work robot system, marker member for work robot system, control method of work robot system, travel program generation device, travel program generation method, and program

Abstract

A work robot system includes: a work robot including a cart capable of traveling on a ground, a robot arm mounted on the cart and capable of arbitrarily changing a position and a posture of a distal end, and a vision sensor mounted on the distal end of the robot arm; and a marker member provided at a prescribed position on the ground, the marker member including: a stand column rising upward from the ground; and a marker provided at an upper end portion of the stand column and including information related to a position on the ground, the work robot including: a control device that controls traveling of the cart based on the marker included in an image captured by the vision sensor.

Description

Technical Field

[0001] This invention relates to a work robot system, a marking component for the work robot system, a control method for the work robot system, a driving program generation device, a driving program generation method, and a program. This application claims priority to Japanese Patent Application No. 2024-148420, filed August 30, 2024, the contents of which are incorporated herein by reference. Background Technology

[0002] One current concept involves mounting a robot (robotic arm) on an Automated Guided Vehicle (AGV) and automatically transporting the robot to the work location (e.g., see Patent Document 1). Another current concept involves mounting a robot on a platform and controlling the movement of the platform via the robot's control device (e.g., see Patent Document 2).

[0003] Previous technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent No. 2680298

[0006] Patent Document 2: Japanese Patent No. 7395877 Summary of the Invention

[0007] The technical problem to be solved by the invention

[0008] AGVs equipped with robots navigate autonomously using guidance methods such as magnetic induction or optical guidance. Generally, commercially available AGVs are equipped with sensors corresponding to the guidance method or various functions for autonomous driving control, offering high functionality but at a high price. Therefore, when introducing commercially available AGVs into a factory, features not used in the factory (such as automated material handling scheduling or map creation functions) may be included, potentially making the equipment structure or cost excessive for the intended purpose.

[0009] Furthermore, if the AGV cannot move precisely to the correct working position, it will cause a misalignment between the robot and the workpiece, making it difficult for the robot to move the tool to the correct working point on the workpiece for operation. This will lead to a decrease in the accuracy of robot-based operations.

[0010] Furthermore, for example, Patent Document 2 describes a system where, if the operator determines that the deviation between the target position and the actual stopping position is large, the robot returns to its original position and attempts to move towards the target position again. Therefore, in the system described in Patent Document 2, the operator needs to constantly monitor the moving robot and perform operations to make it move again, which correspondingly increases the operator's operating costs.

[0011] The present invention was made to solve the above-mentioned problems, and its purpose is to provide a work robot system that can move a work robot with a simple and low-cost structure, a marking component for the work robot system, a control method for the work robot system, a driving program generation device, a driving program generation method, and a program.

[0012] means for solving technical problems

[0013] To address the aforementioned issues, the present invention relates to a work robot system comprising: a work robot including a trolley capable of traveling on the ground, a robotic arm mounted on the trolley and capable of arbitrarily changing the position and posture of its end effector, and a vision sensor mounted on the end effector of the robotic arm; and a marking component disposed at a predetermined position on the ground, the marking component comprising: a column erected from the ground upwards; and a marking disposed at the upper end of the column and including information related to its position on the ground, the work robot comprising a travel robot control device (control device) 10 for controlling the trolley based on the marking included in an image captured by the vision sensor.

[0014] The marking component for a work robot system according to the present invention is used in a work robot system comprising a work robot, the work robot including a trolley capable of traveling on the ground, a robotic arm mounted on the trolley and capable of arbitrarily changing the position and posture of its end effector, and a vision sensor mounted on the end effector of the robotic arm. The marking component for the work robot system comprises: a column, disposed at a predetermined position on the ground and erected upward from the ground; and a mark, disposed at the upper end of the column and including information related to the position on the ground.

[0015] The control method for the work robot system involved in this invention is a control method for the work robot system, which includes the following steps: causing the trolley to travel along a specified path; capturing the mark through the vision sensor; and controlling the travel of the trolley according to the mark included in the image captured by the vision sensor.

[0016] Invention Effects

[0017] The work robot system, the marking component for the work robot system, and the control method for the work robot system according to the present invention enable the work robot to move with a simple and low-cost structure. Attached Figure Description

[0018] Figure 1 This is a diagram showing the overall structure of the robot system according to the first embodiment.

[0019] Figure 2 From and Figure 1 The diagram shows the working robot involved in the first embodiment viewed from different angles.

[0020] Figure 3 This is a perspective view showing the marking component according to the first embodiment.

[0021] Figure 4 This is a block diagram illustrating the functional structure of the robot control device and the driving control device according to the first embodiment.

[0022] Figure 5 This is a flowchart illustrating an example of the trolley's driving process based on the robot system according to the first embodiment.

[0023] Figure 6 This is a diagram illustrating an example of the path information involved in the first embodiment.

[0024] Figure 7 This is a diagram used to illustrate the driving process of the robot system according to the first embodiment.

[0025] Figure 8 This is a diagram illustrating an example of the hardware structure of a robot control device and a motion control device.

[0026] Figure 9 This is a block diagram illustrating the functional structure of the driving procedure generation device and the robot control device according to the second embodiment.

[0027] Figure 10 This is a flowchart illustrating an example of the processing of the driving procedure generation apparatus according to the second embodiment.

[0028] Figure 11 This diagram illustrates the processing of the driving procedure generation apparatus according to the second embodiment.

[0029] Figure 12 This is a flowchart illustrating an example of the trolley's movement processing based on the robot system according to the second embodiment.

[0030] Figure 13 This diagram illustrates the processing of the driving procedure generation apparatus according to the third embodiment.

[0031] Figure 14 This is Figure 1, which is used to explain the processing of the driving procedure generation device according to the fourth embodiment.

[0032] Figure 15 This is Figure 2, which is used to explain the processing of the driving procedure generation device according to the fourth embodiment. Detailed Implementation

[0033] Hereinafter, with reference to the accompanying drawings, embodiments of the robot system for implementing the present invention, the marking component for the robot system, and the control method for the robot system will be described. However, the present invention is not limited to this embodiment.

[0034] <First Embodiment>

[0035] (Overall structure of the robot system)

[0036] Figure 1 This is a diagram showing the overall structure of the robot system according to the first embodiment. (Example) Figure 1 As shown, the work robot system 100 includes a work robot 1, a marking component 7, and an operating PC 3. The work robot 1 automatically performs tasks instructed by an operator in a workspace such as a factory or warehouse. The work robot 1 includes a trolley 2, a robotic arm 11, a vision sensor 12, and a robot control device 10.

[0037] (Structure of the trolley)

[0038] Figure 2 The first embodiment involves the relationship between and Figure 1 Images showing the robot being operated from different angles.

[0039] The trolley 2 is, for example, an automated guided vehicle (AGV). In this embodiment, the trolley 2 is rectangular when viewed from above. Figure 1 , Figure 2 As shown, the trolley 2 has a top plate 2t and side plates 2s. The top plate 2t faces upward and can carry items transported by the work robot 1. The height of the top plate 2t from the ground F is set in the range of approximately 10cm to 30cm. The side plates 2s extend downward from the four sides of the top plate 2t.

[0040] like Figure 1 As shown, the trolley 2 is equipped with a driving control device 20, a drive device 21, wheels 23 and stops 24.

[0041] The driving control device 20, which will be described in detail later, controls the drive device 21 in a manner that enables the trolley 2 to move (drive), stop, change speed, etc., according to the driving instructions of the robot control device 10.

[0042] The drive unit 21 is a motor that moves the wheels 23 of the trolley 2. In this embodiment, the wheels 23 are Mecanum wheels (registered trademark) with multiple rollers, and the drive unit 21 has one motor for each wheel 23. The drive unit 21 changes the combination of rotating wheels 23 or the rotation speed and direction of each wheel 23 according to the control commands of the travel control unit 20. Thus, the trolley 2 can move in any direction. Alternatively, in other embodiments, the wheels 23 may be ordinary wheels instead of Mecanum wheels. In this case, the drive unit 21 has a motor for rotating the wheels and a steering mechanism for changing the orientation of the wheel axles.

[0043] Additionally, in the following description, the main direction of travel of the trolley 2 based on multiple wheels 23 (Mecanum wheels) is sometimes referred to as the X-axis direction, and the width direction of the trolley 2 that intersects the X-axis direction in the horizontal plane is referred to as the Y-axis direction.

[0044] The stop block 24 is a device used to fix the trolley 2 in place after it has been moved to a designated position.

[0045] The trolley 2 may also be equipped with a driving assistance sensor (not shown). The driving assistance sensor is used to detect the position or direction of travel of the trolley 2, enabling the trolley 2 to move autonomously when there is no driving instruction from the robot control device 10 described later. In this embodiment, the driving assistance sensor includes a position detection sensor that detects the amount of movement of the trolley 2 in the horizontal direction (X-axis and Y-axis). The position detection sensor uses the same technology as a so-called optical mouse, reading patterns on the ground to detect the amount of movement of the trolley 2 in the X-axis and Y-axis directions. Furthermore, the driving assistance sensor may also include a gyrocompass that detects the direction of travel of the trolley 2.

[0046] Furthermore, an obstacle detection sensor 26 is installed on the trolley 2. The obstacle detection sensor 26 detects obstacles around the trolley 2. The trolley 2 is configured such that, during operation, it decelerates or stops when the obstacle detection sensor 26 detects an obstacle within a preset range, under the control of the travel control device 20. In this embodiment, the obstacle detection sensor 26 is, for example, installed at the two diagonal corners of the trolley 2, which is rectangular when viewed from above. The number and position of the obstacle detection sensors 26 can be appropriately changed.

[0047] (Structure of robotic arm and vision sensor)

[0048] like Figure 1 , Figure 2As shown, the robotic arm 11 is mounted on the trolley 2. In this embodiment, when viewed from above, the robotic arm 11 is positioned on the outside of the top plate 2t of the trolley 2. In this embodiment, the working robot 1 has a support base 15 for supporting the robotic arm 11. The support base 15 is configured to protrude outward when viewed from above the top plate 2t of the trolley 2. The support base 15 is configured to protrude from the upper end of the side 2s on one side of the width direction in the trolley 2 towards the width direction.

[0049] The robotic arm 11 is mounted on the support base 15. The robotic arm 11 is a multi-joint robotic arm with multiple joints. The robotic arm 11 is capable of rotating each joint and arbitrarily changing the position and posture of the end effector 11a. Furthermore, the structure of the robotic arm 11, such as the number of joints, is not limited in any way and can be any structure.

[0050] A tool 13 is detachably mounted on the end effector 11a of the robotic arm 11. The tool 13 is an instrument used to perform various tasks. The tasks performed by the robot 1 include, for example, welding, assembly tasks such as screw tightening or drilling, picking tasks such as holding and moving parts, finishing, ultrasonic inspection, and appearance inspection. The tool 13 corresponding to the task is mounted on the end effector 11a of the robotic arm 11.

[0051] A vision sensor 12 is mounted on the end effector 11a of the robotic arm 11. The vision sensor 12 is a camera that captures images of the surrounding area of ​​the robot 1. Figure 1 An example of a vision sensor 12 with a CCD camera 12A and a 3D camera 12B is shown.

[0052] The vision sensor 12 is capable of capturing images of markings L set in the area where the robot 1 moves. In this embodiment, the markings L include a guide line L1 and a marker L2.

[0053] (Structure of the guide line)

[0054] Guide line L1 is set on the ground F in the area where the work robot 1 travels. Guide line L1 is used to guide the travel direction of work robot 1, and therefore extends in a specified direction on the ground F.

[0055] (Structure of the marking component)

[0056] Figure 3 This is a perspective view showing the marking component according to the first embodiment.

[0057] like Figures 1-3As shown, the marking component 7 is disposed at a predetermined position on the ground F. Multiple marking components 7 are disposed, for example, at positions overlapping with the guide line L1, along the path of the working robot 1, at the stopping position of the working robot 1, and at positions where the traveling direction of the working robot 1 is changed. In this embodiment, multiple marking components 7 are disposed, for example, at predetermined intervals along the extension direction of the guide line L1.

[0058] Each of the multiple marking components 7 has a column 71 and a marker L2.

[0059] The column 71 rises upward from the ground F. The column 71 is, for example, formed as a column extending in the vertical direction. The column 71 can also be formed as a wall (plate) extending in the vertical direction. The column 71 has an upward-facing upper surface 71t. Furthermore, when viewed from above, the column 71 is rectangular or polygonal in shape, and has multiple side surfaces 71s facing the sides, intersecting the vertical direction.

[0060] A mark L2 is disposed at the upper end of the column 71. The mark L2 includes information related to its position on the ground F. In this embodiment, the mark L2 is, for example, a QR code (registered trademark). In this embodiment, the mark L2 has an upper mark L2a and a side mark L2b. The upper mark L2a is disposed on the upper surface 71t of the column 71. The side mark L2b is disposed on at least one of the plurality of side surfaces 71s of the column 71.

[0061] The marking component 7 is preferably configured at a height detectable by the obstacle detection sensor 16. That is, the marking component 7 is preferably configured to extend upwards beyond the obstacle detection sensor 16. This helps to prevent collisions between the moving trolley 2 and the marking component 7.

[0062] Furthermore, the marking component 7 is preferably lower than the support base 15. Thus, even if the moving trolley 2 deviates from the path, collision between the support base 15 and the marking component 7 can be prevented.

[0063] Furthermore, in order to prevent workers from tripping while walking on the ground F, the marking component 7 preferably has a height of 5 cm or more.

[0064] (Structure of the robot control device)

[0065] A robot control device 10 is mounted on the work robot 1. The robot control device 10 controls the movements of various parts of the work robot 1. In this embodiment, the robot control device 10 causes the trolley 2 to travel to a designated location. The designated location is, for example, the work position where the work robot 1 performs its work or the storage position of the work robot 1. The designated location is specified by the operator of the work robot 1 via the operation PC 3.

[0066] (Structure of operating a PC)

[0067] The operation PC3 is a computer operated by the operator of the work robot 1. The operation PC3 communicates wirelessly with the work robot 1. The operation PC3 receives commands from the operator and instructs the work robot 1 on the designated location as the destination and the path (via location, etc.) to reach the designated location.

[0068] (Functional structure of robot control device)

[0069] Figure 4 This is a block diagram illustrating the functional structure of the robot control device and the driving control device according to the first embodiment. For example... Figure 4 As shown, the robot control device 10 includes a path acquisition unit 101, a sensor information acquisition unit 102, a detection unit 103, a driving instruction unit 104, and a robotic arm control unit 105.

[0070] The path acquisition unit 101 acquires path information D1, which represents the path from the current position of the trolley 2 to the specified position, from the operation PC3.

[0071] The sensor information acquisition unit 102 acquires the image D2 captured by the vision sensor 12. For example, in this embodiment, the sensor information acquisition unit 102 acquires the image D2 captured by the CCD camera 12A of the vision sensor 12 when the trolley 2 is moving.

[0072] The detection unit 103 detects the offset of the carriage 2 from the specified path by the guide line L1 and the mark L2 (upper mark L2a, side mark L2b) included in the image captured by the vision sensor 12 (CCD camera 12A).

[0073] The driving instruction unit 104 outputs driving instructions D4 to the trolley 2 (driving control device 20), indicating the driving position, stopping, speed, etc. For example, the driving instruction unit 104 instructs the trolley 2 to control its driving direction (angle) based on the driving direction of the trolley 2 detected by the detection unit 103 according to the guide line L1. Furthermore, the driving instruction unit 104 instructs the trolley 2 to correct its driving position or driving direction (angle) based on the offset of the trolley 2 detected by the detection unit 103 according to the marker L2.

[0074] The robotic arm control unit 105 controls the robotic arm 11 to position or pose the vision sensor 12 or tool 13 in any manner. For example... Figure 1 As shown, during the movement of the trolley 2, the robotic arm control unit 105 controls the robotic arm 11 so that the vision sensor 12 adopts a first downward orientation.

[0075] (Functional structure of the driving control device)

[0076] like Figure 4 As shown, the driving control device 20 includes a motor control unit 201.

[0077] The motor control unit 201 controls the drive unit 21 to move, stop, or change the speed of the trolley 2. Regarding the motor control unit 201, when there is no driving instruction from the robot control unit 10, it can control the drive unit 21 based on the sensor values ​​(movement amount and direction of travel) of the driving assistance sensor (not shown) to allow the trolley 2 to autonomously travel to the designated position.

[0078] Furthermore, when the trolley 2 is moved, its position and direction of travel may deviate from the designated path due to factors such as uneven ground in the factory, slippage of the wheels 23 on the ground F, and inability to read the guide line L1. Therefore, in this embodiment, when the mark L2 is captured by the vision sensor 12, the robot control device 10 of the work robot 1 detects the deviation of the trolley 2 from the path based on the captured mark L2, and issues a driving instruction to the trolley 2's driving control device 20 to correct the driving position. The motor control unit 201 of the driving control device 20 controls the drive device 21 to adjust the driving position of the trolley 2 according to the driving instruction of the robot control device 10.

[0079] Furthermore, as described above, the wheels 23 of the trolley 2 in this embodiment are Mecanum wheels. The motor control unit 201 may include a calculation unit 201A for calculating control command values ​​representing the presence, direction, and speed of rotation of the travel motors of each wheel 23. For example, if the coordinates of the destination are assigned, the calculation unit 201A calculates the control command values ​​for each travel motor to orient the trolley 2 toward the coordinates of the destination. The motor control unit 201 controls the travel motors based on the control command values ​​calculated by the calculation unit 201A.

[0080] (Rotation processing of robot-based trolleys)

[0081] Figure 5 This is a flowchart illustrating an example of the trolley driving process performed by the robot according to the first embodiment. Figure 6 This is a diagram illustrating an example of the path information involved in the first embodiment. Figure 7 This is a diagram illustrating the driving process of the robot according to the first embodiment. Here, refer to... Figures 5-7 The process of handling the movement of the trolley 2 based on the robot control device 10 of the work robot 1 is explained.

[0082] The path acquisition unit 101 acquires path information D1, representing the path from the current position of the trolley 2 to the specified position, from the operation PC3. Figure 4 (Step S101). The specified location is, for example, the working position of the work robot 1. The specified location and path are input by the operator via the operation PC3. Figure 6 As shown, the path information D1 represents the coordinates (X, Y, Z) of the starting position (current position), the passing position and the specified position (destination), as well as the posture (R) of the trolley 2 at each position.

[0083] If path information D1 is obtained, the robot control device 10 causes the trolley 2 to start moving according to the path information D1 (step S102).

[0084] After the trolley 2 starts moving, the robot control device 10 acquires image D2 captured by the vision sensor 12 (step S103). Figure 1 and Figure 7 As shown, in this embodiment, firstly, the robot 1 takes an image D2 with the vision sensor 12 facing downward (to the ground) in a vertical direction (first posture).

[0085] If the guide line L1 is detected from the acquired image D2 (step S104; "Yes"), the detection unit 103 detects the traveling direction of the carriage 2 and the amount of offset of the carriage 2 in the Y direction based on the detection information D3 of the guide line L1 (step S105). For example, the detection unit 103 detects the amount of offset indicating how much the position of the carriage 2 has shifted in the Y direction based on the position of the guide line L1 in image D2. Furthermore, the detection unit 103 detects the amount of offset indicating how much the traveling direction of the carriage 2 has shifted based on the angle of the guide line L1 in image D2.

[0086] Furthermore, if the upper mark L2a is detected from image D2 (step S104; "Yes"), the detection unit 103 detects the coordinates (X, Y) and angle (angle R around the Z-axis) of the detection position of the upper mark L2a in image D2. Based on the coordinates and angle of the upper mark L2a, the detection unit 103 detects the amount of offset indicating how much the position and direction of travel of the trolley 2 have shifted (step S105).

[0087] If the detection unit 103 cannot detect the upper mark L2a in the first posture (step S104; "No"), the robotic arm control unit 105 changes the position and posture of the robotic arm 11 so that the vision sensor 12 adopts a second posture facing forward in the direction of travel. Then, if the side mark L2b is detected from the image D2 taken in the second posture (step S104; "Yes"), the detection unit 103 detects the coordinates (X, Y) and angle (angle R around the Z-axis) of the detection position of the side mark L2b in image D2. Based on the coordinates and angle of the side mark L2b, the detection unit 103 detects the amount of offset indicating how much the position and direction of travel of the trolley 2 have deviated (step S105).

[0088] Next, the driving instruction unit 104 outputs a driving instruction D4 to the trolley 2 in a manner that reduces the offset of the trolley 2 based on the detected offset and driving position (step S106).

[0089] For example, such as Figure 5 As shown in (7), when a deviation in the Y direction is detected according to the guide line L1, the travel instruction unit 104 outputs a travel instruction D4 to correct the position of the trolley 2 in the Y direction. This travel instruction is represented, for example, by the relative coordinates (X, Y) of the corrected position relative to the current position of the trolley 2. If the trolley 2 can change its direction of travel via the steering mechanism, the relative angle (R) of the corrected direction of travel relative to the current direction of travel can also be included in the travel instruction D4. In addition, the travel instruction unit 104 can also output a travel instruction D4 to correct the travel position of the trolley 2 if the deviation is above a preset allowable value. That is, if the deviation is less than the allowable value, the position correction of the trolley 2 may not be performed.

[0090] And, as Figure 5As shown in (b), when the travel position of the trolley 2 is detected based on the marker L2, the travel instruction unit 104 outputs a travel instruction D4 indicating the coordinates of the trolley 2's next destination, stopping, and speed changes, based on the trolley 2's travel position and path information D1. For example, suppose the path information D1 includes a path for moving in the Y direction after reaching a certain work area (area3) in the X direction. In this case, if the trolley 2 is detected to have reached the work area (area3) based on the reading result of the marker L2, the travel instruction unit 104 outputs a travel instruction D4 specifying the coordinates of the next destination to the trolley 2. In addition, if the reading result of the marker L2 includes information indicating "stop", "90° turn", or "travel speed", the travel instruction unit 104 can include instructions such as stopping, 90° turn, and speed changes (acceleration or deceleration) in the travel instruction D4 based on the reading result. Furthermore, when the trolley 2 is stopped, the driving instruction unit 104 detects the offset of the detection position of the marker L2, outputs a driving instruction D4 for adjusting the position and posture of the trolley 2, and outputs a driving instruction D4 indicating that the trolley 2 has been stopped after adjusting the position and posture of the trolley 2.

[0091] Then, the robot control device 10 determines whether the trolley 2 is stopped at the designated location (step S107). For example, if it was indicated in step S105 that the trolley 2 should stop at the designated location (within the work area as the destination) (step S107; "Yes"), the robot control device 10 stops the trolley 2. On the other hand, if it was not indicated in step S105 that the trolley 2 should stop at the designated location (step S107; "No"), the robot control device 10 returns to step S102 and continues the trolley 2's movement (steps S102 to S107).

[0092] In step S107, when the trolley 2 stops at the designated position (step S107; "Yes"), the work robot 1 reads the mark L2 (upper mark L2a) through the vision sensor 12 (step S108). Based on the information included in the read mark L2 (QR code), the work robot 1 performs prescribed processes such as self-positioning or interlocking before the next action.

[0093] (Functions and Effects)

[0094] As described above, the work robot system 100 according to this embodiment includes: a work robot 1, including a trolley 2 capable of traveling on the ground F, a robotic arm 11 mounted on the trolley 2 and capable of arbitrarily changing the position and posture of its end effector, and a vision sensor 12 installed on the end effector 11a of the robotic arm 11; and a marking component 7, which is set at a predetermined position on the ground F. The marking component 7 includes: a column 71 that rises upward from the ground F; and a mark L2 that is set at the upper end of the column 71 and includes information related to its position on the ground F. The work robot 1 includes a robot control device 10, which controls the movement of the trolley 2 based on the mark L2 included in the image captured by the vision sensor 12.

[0095] The robot system 100 guides the trolley 2 by using a vision sensor 12 mounted on the robotic arm 11, eliminating the need for a separate mechanism for guiding the trolley 2. This allows the trolley 2 to be constructed with minimal equipment and cost, while enabling the trolley 2 to move with high precision along the path.

[0096] Furthermore, the marker L2 is positioned at the upper end of the column 71 that rises upwards from the ground F. Therefore, compared to placing the marker L2 directly on the ground F, it is less prone to soiling or damage. Consequently, when the vision sensor 12, mounted on the end effector 11a of the robotic arm 11 of the work robot 1, is used to photograph the marker L2, it is easy to bring the vision sensor 12 close to the marker L2. As a result, the marker L2 can be photographed easily and accurately, thereby enabling effective control of the trolley 2's movement.

[0097] Furthermore, the column 71 also functions as a protective component to prevent collisions with surrounding workers when the trolley 2 leaves the path.

[0098] Furthermore, the column 71 has an upward-facing upper surface 71t, and the mark L2 has an upper mark L2a disposed on the upper surface 71t.

[0099] Therefore, while the trolley 2 is moving, by making the vision sensor 12 point downwards to take pictures, it is possible to capture the upper mark L2a set on the upper surface 71t of the column 71.

[0100] Furthermore, by setting the side mark L2b on the side 71s of the column 71, the mark L2 (side mark L2b) can be captured by the vision sensor 12 even when it is far away from the column 71 component. Thus, the mark L2 can be easily detected even when the trolley 2 is off the path.

[0101] Furthermore, the robotic arm 11 is mounted on a support 15 that protrudes outward when viewed from above the top plate 2t of the trolley 2. Therefore, compared to the case where the robotic arm 11 is mounted on the top plate 2t, the extent to which the end point 11a of the robotic arm 11 reaches the side of the support 15 relative to the top plate 2t is wider. Thus, even when the marking member 7 is far from the travel path of the trolley 2, the marking L2 of the marking member 7 can be easily detected using the vision sensor 12. As a result, the degree of freedom in the placement of the marking member 7 is increased.

[0102] Furthermore, the height of the marking component 7 from the ground F is less than the height of the support base 15 from the ground F.

[0103] Therefore, even if the trolley 2 deviates from the path, it can suppress the collision between the support seat 15 and the marking component 7.

[0104] Furthermore, since the marker component 7 is set in a manner that can be detected by the obstacle detection sensor 16, collisions between the trolley 2 and the marker component 7 can be suppressed.

[0105] Furthermore, the robot control device 10 detects the offset of the carriage 2 relative to the specified path based on the marker L2 included in the image captured by the vision sensor 12, and corrects the travel position or posture of the carriage 2 according to the offset. As a result, the carriage 2 can move along the path with high precision.

[0106] In the marking component 7 of the work robot system 100 described above, the marking L2 is located at the upper end of the column 71 that rises from the ground F. Therefore, compared to placing the marking L2 on the ground F, it is less likely for the marking L2 to become soiled or damaged. Consequently, when the vision sensor 12 is used to capture the marking L2 at the end 11a of the robotic arm 11 of the work robot 1, it is easy to bring the vision sensor 12 close to the marking L2. As a result, the marking L2 can be captured easily and accurately, thereby enabling effective control of the trolley 2's movement.

[0107] Furthermore, the column 71 also functions as a protective component to prevent collisions with surrounding workers when the trolley 2 leaves the path.

[0108] According to the control method of the work robot system 100 described above, the marker L2 can be easily and well captured, thereby enabling effective control of the trolley 2's movement.

[0109] <Hardware Structure>

[0110] Figure 8This diagram illustrates an example of the hardware structure of a robot control device and a motion control device. The computer 900 includes a CPU 901, a main storage device 902, an auxiliary storage device 903, an input / output interface 904, and a communication interface 905.

[0111] The robot control device 10 and the driving control device 20 are respectively installed in the computer 900. Furthermore, the aforementioned functions are stored as programs in the auxiliary storage device 903. The CPU 901 reads the program from the auxiliary storage device 903 and expands it to the main storage device 902, executing the aforementioned processing according to the program. The CPU 901 also secures a storage area in the main storage device 902 according to the program. The CPU 901 also secures a storage area in the auxiliary storage device 903 for the data being processed according to the program.

[0112] Alternatively, programs for implementing all or part of the functions of the robot control device 10 and the driving control device 20 can be recorded on a computer-readable recording medium. The computer system can then read and execute the program recorded on this medium to perform processing based on each functional unit. Here, "computer system" includes hardware such as the operating system and supporting devices. Furthermore, if a WWW system is used, the "computer system" also includes a webpage providing environment (or display environment). "Computer-readable recording medium" refers to portable media such as CDs, DVDs, and USB drives, and storage devices such as hard drives built into the computer system. When the program is transmitted to the computer 900 via a communication line, the receiving computer 900 can expand the program to the main storage device 902 and execute the aforementioned processing. Furthermore, the program can be used to implement a portion of the aforementioned functions, and can also achieve the aforementioned functions by combining it with programs already recorded in the computer system.

[0113] <Second Implementation>

[0114] Next, refer to Figures 9-12 The second embodiment will be described in detail. Components identical to those in the above embodiments will be labeled with the same symbols and detailed descriptions will be omitted.

[0115] (Functional structure of the driving program generation device)

[0116] Figure 9 This is a block diagram illustrating the functional structure of the robot control device and the driving program generation device according to the second embodiment. In this embodiment, an example of the operating PC3 functioning as the driving program generation device 30 will be described. In other embodiments, the driving program generation device 30 may be hardware different from the operating PC3. Figure 9As shown, the driving procedure generation device 30 includes a path generation unit 301, a driving procedure generation unit 302, and a transmission processing unit 303.

[0117] The path generation unit 301 generates path information D5, which includes the coordinates of the designated position and the passing position of the work robot 1, as well as the target posture of the work robot 1 at the designated position and the passing position. The designated position is the destination of the work robot 1. The passing position is the waypoint between the current position of the work robot 1 and the designated position. The path generation unit 301 generates the path information D5, for example, by receiving input operations of the coordinates and postures of each position from the operator of the work robot 1.

[0118] The driving program generation unit 302 calculates the amount of movement between each position and the direction change position of the working robot 1 based on the path information D5, and generates a driving program D6 that sequentially combines the following actions: the action of moving between each position of the working robot 1, the action of turning at the direction change position, and the action of correcting the position and posture of the working robot 1 based on the mark L2 included in the image D2 captured by the vision sensor 12 of the working robot 1 at the direction change position.

[0119] The sending processing unit 303 sends the driving program D6 generated by the driving program generation unit 302 to the working robot 1.

[0120] (Functional structure of robot control device)

[0121] In this embodiment, the robot control device 10 controls the operation of the work robot 1 according to the driving program D6 generated by the driving program generation device 30.

[0122] like Figure 9 As shown, the robot control device 10 according to this embodiment includes a driving program acquisition unit 106 instead of a path acquisition unit 101.

[0123] The driving program acquisition unit 106 acquires the driving program D6 from the driving program generation device 30.

[0124] The driving instruction unit 104 outputs a driving instruction D4 to the driving control device 20 based on the driving program D6 acquired by the driving program acquisition unit 106. Furthermore, if the driving program D6 includes position and posture correction actions, the driving instruction unit 104 outputs a detection instruction D7 for marker L2 to the robotic arm control unit 105.

[0125] When the detection instruction D7 is input, the robotic arm control unit 105 controls the posture of the robotic arm 11 so that the vision sensor 12 can capture the mark L2 (the upper mark L2a or the side mark L2b) and controls the vision sensor 12 to capture the image.

[0126] In the first embodiment, the robot control device 10 continuously performs corrections based on the capture of mark L2 from the vision sensor 12 and the image captured by the vision sensor 12, determining the driving position and direction of travel. In contrast, in the second embodiment, the robot control device 10 corrects the position and posture of the work robot 1 (trolley 2) only when the position and posture correction action is indicated in the driving procedure D6. This correction is based on the capture of mark L2 from the vision sensor 12 and the image D2 captured by the vision sensor 12.

[0127] (Driving program generation and processing)

[0128] Figure 10 This is a flowchart illustrating an example of the processing of the driving procedure generation apparatus according to the second embodiment. Figure 11 This is a diagram illustrating the processing of the driving procedure generation apparatus according to the second embodiment. Here, refer to... Figures 10-11 The process of generating the driving program D6 of the working robot 1 (cart 2) by the driving program generation device 30 is explained.

[0129] The path generation unit 301 generates path information D5 based on the operator's input (step S201). For example... Figure 11 As shown, the path information D5 includes the coordinates (X, Y, Z) of the starting position (current position), the passing position (via location), and the designated position (destination) of the robot 1, as well as the posture (R) of the robot 1 at each position. The coordinates of the starting position, the passing position, and the designated position are absolute coordinates within the travel area of ​​the robot 1.

[0130] The operator inputs the coordinates (X, Y, Z) of the passing positions and the specified positions according to the movement sequence of the robot 1, and also inputs the target posture (R) of the robot 1 at the passing positions and the specified positions. For example, the operator inputs the coordinates of passing positions P2 to P4 and the specified position P5 according to the movement sequence. In this way, the path generation unit 301 generates path information D5, which arranges the coordinates of each position P1 to P5 from the starting position P1 to the specified position P5 in the movement sequence. In addition, the coordinates and posture (R) of the starting position P1 can be automatically obtained from the current position and posture of the robot 1, or they can be input by the operator after confirming the status of the robot 1.

[0131] Furthermore, if the operator wants to change the direction of travel of the work robot 1 at position P2, the operator inputs the direction of travel (target posture) of the work robot 1 at position P2. In this way, the path generation unit 301 generates path information D5 that includes the step of changing the direction of travel (direction change) at position P2 without changing the position (coordinates) of the work robot 1.

[0132] Furthermore, the operator can input an instruction for the robot 1 to temporarily standby after changing its direction of travel at position P2. In this case, the path generation unit 301 generates path information D5 that includes steps indicating a state (standby) where the position (coordinates) and posture of the robot 1 are not changed at position P2.

[0133] The driving sequence generation unit 302 calculates the amount of movement (relative movement) between each position of the working robot 1 or the direction change position, standby position, etc. of the working robot 1 changing the direction of travel based on the path information D5 generated by the path generation unit 301, and generates the driving sequence of the working robot 1 (step S202).

[0134] exist Figure 11 In the example, the driving procedure generation unit 302 generates a driving sequence consisting of the following actions (1) to (6) based on the path information D5.

[0135] (1) The driving program generation unit 302 generates a motion (1) in which the work robot 1 moves 1950mm forward and 40mm to the right, based on the movement amount calculated from the coordinates of the starting position P1 and the passing position P2 of the path information D5. In addition, the wheels 23 of the trolley 2 of the work robot 1 are Mecanum wheels, which are capable of omnidirectional movement. Therefore, the driving program generation unit 302 can generate a motion of the work robot 1 moving in the forward and backward direction, or a motion of the work robot 1 moving in all directions, including left and right or tilting directions.

[0136] (2) At the passing position P2 in path information D5, the posture (R) changes from the previous step, therefore the passing position P2 is calculated as the direction change position. In this embodiment, the driving program generation unit 302 automatically adds the action of correcting the position and posture of the working robot 1 at the direction change position to the driving sequence. Therefore, in Figure 11 In the example, after the driving program generation unit 302 moves to the passing position P2 through the action (1), the working robot 1 corrects its position and posture based on the image D2 obtained by capturing the mark L2 set near the passing position P2, and then generates the action (2) of turning 90° to the left. Alternatively, the action (2) can also be set as the action of the working robot 1 turning 90° to the left and then correcting its position and posture based on the mark L2.

[0137] (3) At the passing position P2 of path information D5, both the position (coordinates) and posture have not changed since the previous step, so the passing position P2 is calculated as the standby position. Therefore, the driving program generation unit 302 generates the operation (3) for the working robot 1 to perform a standby (stop) action at the passing position P2 for a specified standby time. The standby time can be preset or the standby time input by the operator in step S201 can be used.

[0138] (4) The driving program generation unit 302 generates a movement (4) of the working robot 1 moving (backward) 700mm based on the movement amount calculated from the coordinates of the passing position P2 and the next passing position P3 of the path information D5.

[0139] (5) The driving program generation unit 302 generates a movement (5) of the working robot 1 moving 400mm to the right based on the movement amount calculated from the coordinates of the passing position P3 and the next passing position P4 of the path information D5.

[0140] (6) The driving program generation unit 302 generates a motion (6) for the working robot 1 to move straight (forward) by 10mm and laterally move 10mm to the left based on the movement amount calculated from the coordinates of the passing position P4 and the next designated position P5.

[0141] Furthermore, the driving program generation unit 302 can, after the work robot 1 reaches the designated position P5 (after performing action (6)), add actions to correct the position and posture of the work robot 1 based on the image obtained by capturing the image of the marker L2 set near the designated position P5. Thus, when the work robot 1 reaches the designated position P5, the positional relationship between the work robot 1 and the work object can be appropriately corrected.

[0142] If the driving sequence is generated, the driving program generation unit 302 generates a driving program D6 that converts it into program code for the robot control device 10 (step S203).

[0143] The sending processing unit 303 sends the driving program D6 generated by the driving program generation unit 302 to the robot control device 10 (step S204). The sending processing unit 303 sends the driving program D6 to the robot control device 10, for example, via an FTP server. Furthermore, in other embodiments, the sending processing unit 303 may send the driving program D6 to a portable recording medium (not shown), and the robot control device 10 may read the driving program D6 from the recording medium.

[0144] (Rotation processing of robot-based trolleys)

[0145] Figure 12This is a flowchart illustrating an example of the movement processing of a trolley based on the robot system according to the second embodiment. (See reference) Figure 12 The process of handling the movement of the trolley 2 based on the robot control device 10 of the work robot 1 is explained.

[0146] The driving program acquisition unit 106 acquires the driving program D6 from the driving program generation device 30 (or recording medium) (step S301).

[0147] The driving instruction unit 104 executes the instructions included in the acquired driving procedure D6 sequentially by repeating steps S302 to S308, thereby controlling the driving control device 20, robotic arm 11 and vision sensor 12 of the control vehicle 2.

[0148] For example, if the control issued by the driving procedure D6 is for position and posture correction (step S302; "Yes"), the driving instruction unit 104 outputs a detection instruction D7 to the robotic arm control unit 105 (step S303). In this way, the robotic arm control unit 105 outputs a control instruction to make the robotic arm 11 adopt a posture (e.g., the first posture) that can read the mark L2, and outputs a control instruction to the vision sensor 12 to capture the mark L2.

[0149] The detection unit 103 acquires image D2 captured by the vision sensor 12 (step S304). If the detection unit 103 cannot detect marker L2 in image D2 captured in the current posture (step S305; "No"), the robotic arm control unit 105 changes the posture of the robotic arm 11. For example, the robotic arm control unit 105 outputs a control command to the robotic arm 11 to change from the current posture to another posture (e.g., from posture 1 to posture 2) or to offset the position and orientation (angle) of the vision sensor 12 from the current posture (posture 1 or posture 2) by a predetermined amount. Then, the detection unit 103 acquires image D2 captured in the new posture (step S304).

[0150] If marker L2 is detected in image D2 (step S305; "Yes"), the detection unit 103 detects the offset of the position and posture (direction of travel) of the trolley 2 based on the coordinates and angle of the detection position of marker L2 in image D2 (step S306), and outputs detection information D3 including the offset to the travel instruction unit 104. The travel instruction unit 104 outputs a travel instruction D4 that reduces the offset of the trolley 2 based on the offset and travel position included in the detection information D3 (step S307). The wheels 23 of the trolley 2 are Mecanum wheels, which can move in all directions. Therefore, the travel instruction unit 104 moves the trolley 2 in any direction, including forward, backward, left, right, and diagonal, to correct the position and posture of the trolley 2. In addition, the processing of steps S304 to S307 is the same as in the first embodiment ( Figure 5 The processing of steps S103 to S106 is the same.

[0151] Furthermore, if the control commanded by the driving procedure D6 is for something other than position and posture correction (step S302; "No"), the driving instruction unit 104 outputs a driving instruction D4 (straight, lateral, turn, standby, etc.) corresponding to the control commanded by the driving procedure D6 to the trolley 2 (driving control device 20) (step S310).

[0152] Then, the driving instruction unit 104 determines whether all driving procedures D6 have been executed (step S308). If the driving procedure D6 has not been executed to the end (step S308; "No"), the driving instruction unit 104 returns to step S302 and continues the driving process of the trolley 2 (steps S302 to S307, S310).

[0153] If the driving procedure D6 has been executed to its final stage (step S308; "Yes"), the work robot 1 reads marker L2 via vision sensor 12 (step S309). Based on the information included in the read marker L2, the work robot 1 performs prescribed processes such as self-positioning and interlocking before the next action. This process is similar to that in the first embodiment (…). Figure 5 The process of step S108 is the same.

[0154] (Functions and Effects)

[0155] As described above, the driving program generation apparatus 30 according to this embodiment includes: a path generation unit 301, which accepts input from an operator and generates path information D5, the path information D5 including the coordinates of a designated location as the destination of the work robot 1 and the coordinates of each of the passing locations until reaching the designated location, and a target posture indicating the direction of travel of the work robot 1 at each of the designated location and the passing locations; and a driving program generation unit 302, which calculates the amount of movement between each location and the direction change position of the work robot 1 to change its direction of travel based on the path information D5, and generates a driving program D6 consisting of actions that sequentially combine the following actions: the action of moving between each location of the work robot 1, the action of turning around at the direction change position, and the action of correcting the position and posture of the work robot 1 based on the marker L2 included in the image D2 captured by the vision sensor 12 of the work robot 1 at the direction change position.

[0156] Thus, by generating a driving program D6 that includes position and posture corrections at the direction change positions, the driving program generation device 30 can automatically correct the position and posture of the robot 1 each time it reaches a direction change position. Therefore, the driving program generation device 30 can improve the reliability of the robot 1 autonomously moving to the designated position (destination), thereby reducing the need for operator monitoring or operation of the moving robot 1. Furthermore, the operator, in preparing for the robot 1's movement, only needs to specify the designated position, the coordinates of the passing position, and the posture of the robot 1 at each position, eliminating the need for tedious tasks such as calculating the movement between positions and creating driving programs based on the robot's path. In other words, it can reduce the operator's workload or time required for driving program generation.

[0157] <Third Implementation>

[0158] Next, refer to Figure 13 The third embodiment will be described in detail. Components identical to those in the embodiments described above will be labeled with the same symbols and detailed descriptions will be omitted.

[0159] Figure 13 This diagram illustrates the processing of the driving procedure generation apparatus according to the third embodiment. In this embodiment, the path generation unit 301 of the driving procedure generation apparatus 30... Figure 10 In step S201, Figure 13 The operating area screen is displayed to the operator, and path information D5 is generated and acquired based on the position and posture of the robot 1 set by the operator on the operating area screen. The operating area screen includes the location of the operating area where the robot 1 performs its work, map information of the operating area including the setting position of marker L2, and start position information indicating the current position and posture of the robot 1. Furthermore, the operating area screen may also include channel information indicating the paths that the robot 1 can traverse.

[0160] The operator sets the designated position and passing position of the work robot 1 by selecting any location on the driving area screen (tap, click, etc.). The path generation unit 301 displays an icon of the work robot 1 at the location selected by the operator. Furthermore, the operator manipulates the icon to change the designated position and passing position, as well as the posture (direction of travel) of the work robot 1 at the designated position and passing position. If the operator completes the setting operation, the path generation unit 301 generates and acquires path information D5, including the coordinates of the starting position, designated position, and passing position, and the posture of the work robot 1 at each position, based on the position and posture of each icon on the driving area screen.

[0161] Therefore, the driving program generation device 30 can automatically generate path information D5 based on the position set by the operator on the driving area screen and the posture (orientation of the icon) of the working robot 1. As a result, the operator can easily specify the movement path of the working robot 1 without manually inputting the coordinates of the specified position and the position passed through, or the posture values ​​of the working robot 1.

[0162] Furthermore, the path generation unit 301 can display a travel area screen that includes channel information indicating the channels that the work robot 1 can travel on. In this case, if the operator selects a location other than a traversable channel, the path generation unit 301 displays a warning message on the travel area screen stating "Unable to move to the selected location." Alternatively, the path generation unit 301 can restrict the selection of locations other than traversable channels on the travel area screen.

[0163] Therefore, the driving program generation device 30 can easily generate a path plan for the work robot 1, even for inexperienced operators unfamiliar with the driving area. This reduces the likelihood that the work robot 1 will take an inappropriate path, such as one that is too close to surrounding objects.

[0164] Furthermore, the path generation unit 301 can display a travel area screen showing the setting position of marker L2. In this case, if the operator designates a position a certain distance from the setting position of marker L2 as the direction-changing position for the work robot 1, the path generation unit 301 displays a warning on the travel area screen stating "Unable to change direction at the selected position." Alternatively, the path generation unit 301 can restrict the movement of the work robot 1 to a position a certain distance from the setting position of marker L2, as not shown on the travel area screen.

[0165] Therefore, the driving program generation device 30 can generate path information D5 and driving program D6 that can more reliably perform position and posture correction based on marker L2 when the work robot 1 changes direction. This suppresses the work robot 1 from deviating significantly from the movement path after changing direction due to the accumulation of position and posture errors. Consequently, the reliability of autonomous movement to the designated location (destination) can be further improved.

[0166] <Fourth Implementation>

[0167] Next, refer to Figure 14 The fourth embodiment will be described in detail. Components identical to those in the embodiments described above will be labeled with the same symbols and detailed descriptions will be omitted.

[0168] Figure 14 and Figure 15This diagram illustrates the processing of the driving procedure generation apparatus according to the fourth embodiment. In this embodiment, the driving procedure generation unit 302 of the driving procedure generation apparatus 30... Figure 10 In step S202, if the amount of movement of the work robot 1 along a straight path between the operator-set pass position and other positions is above a threshold, or if a marker L2 is set on the straight path, a driving program D6 is generated to add the setting position of the marker L2 on the straight path to correct the position and posture of the work robot 1.

[0169] like Figure 14 and Figure 15 As shown, assuming that through operator input, the path generation unit 301 generates path information D5 for the work robot 1 to move straight from the starting position P21 to the passing position P22, and then from the passing position P22 to the passing position P23. Here, if the movement amount of path 1 from the starting position P21 to the passing position P22 is below a threshold, or if no marker L2 is set on path 1, the driving program generation unit 302 does not add position and posture correction actions on path 1.

[0170] And, for example, such as Figure 14 As shown, markers L2 (L2-1, L2-2, L3, ...) are set at equal intervals. Here, when the amount of movement along path 2 from passing position P22 to passing position P23 is greater than or equal to a threshold, the driving program generation unit 302 sets intermediate passing positions on path 2 at unit distances. The unit distance is the setting interval of markers L2. Therefore, the intermediate passing positions are set at the positions where markers L2 are set. Then, the driving program generation unit 302 generates a driving sequence for each intermediate passing position, correcting the position and posture according to markers L2. Figure 14 In the example, path 2 is divided into a segmented path 2A from the passing position P22 to the intermediate passing position P22A, and a segmented path 2B from the intermediate passing position P22A to the passing position P23. Then, the driving program generation unit 302 generates a driving sequence including the action of moving from the passing position P22 to the intermediate passing position P22A, the action of correcting the position and posture at the intermediate passing position P22A according to the mark L2-2, and the action of moving from the intermediate passing position P22A to the passing position P23 after correction.

[0171] And, as Figure 15As shown in the example, multiple markers L2 (L2-1, L2-3, L2-4, L3) are set at unequal intervals between passing position P22 and passing position P23. Here, if the amount of movement along path 2 from passing position P22 to passing position P23 is greater than or equal to a threshold, the driving program generation unit 302 sets intermediate passing positions on path 2 corresponding to the positions where markers L2 are set. Then, the driving program generation unit 302 generates a driving sequence for each intermediate passing position, correcting the position and posture according to markers L2. Figure 15 In the example, path 2 is divided into a segmented path 2C from the passing position P22 to the intermediate passing position P22B, a segmented path 2D from the intermediate passing position P22B to the intermediate passing position P22C, and a segmented path 2E from the intermediate passing position P22C to the passing position P23. Then, the driving program generation unit 302 generates a driving sequence including the action of moving from the passing position P22 to the intermediate passing position P22B, the action of correcting the position and posture at the intermediate passing position P22B according to the marker L2-3, the action of moving from the intermediate passing position P22B to the intermediate passing position P22C after correction, the action of correcting the position and posture at the intermediate passing position P22C according to the marker L2-4, and the action of moving from the intermediate passing position P22C to the passing position P23 after correction.

[0172] In addition, Figure 15 In the example, the driving sequence generation unit 302 can generate a driving sequence in which the position and posture are corrected at intermediate passing positions P22B and P22C, regardless of whether the movement amount of path 2 is above a threshold.

[0173] Driving program generation unit 302 Figure 10 In step S203, driving procedure D6 is generated based on the driving sequence of the segmented path as described above.

[0174] Therefore, the driving program generation device 30 can automatically generate a driving program that includes actions to correct the position and posture of the robot 1 at the setting position of marker L2, for example, when the distance of linear movement between the passing positions set by the operator is long. This reduces the risk that the robot 1 may lose its position due to error accumulation during long-distance movement or may deviate from the intended path and come into contact with objects in the surrounding environment.

[0175] <Another implementation>

[0176] The embodiments have been described in detail above with reference to the accompanying drawings, but the specific structure is not limited to the above structure, and various design changes can be made. That is, in another embodiment, the order of the above processes can be appropriately changed. Furthermore, some processes can be performed in parallel.

[0177] <Postscript>

[0178] The operation robot system 100 and the control method of the operation robot system 100 described in the embodiments can be understood as follows.

[0179] (1) The operation robot system 100 involved in the first method includes: an operation robot 1, including a trolley 2 capable of traveling on the ground F, a robotic arm 11 mounted on the trolley 2 and capable of arbitrarily changing the position and posture of the end 11a, and a vision sensor 12 installed on the end 11a of the robotic arm 11; and a marking component 7, which is set at a predetermined position on the ground F. The marking component 7 includes: a column 71 that rises upward from the ground F; and a mark L2 that is set at the upper end of the column 71 and includes information related to the position on the ground F. The operation robot 1 includes: a control device 10 that controls the travel of the trolley 2 according to the mark L2 included in the image captured by the vision sensor 12.

[0180] The robot system 100 guides the trolley 2 by using a vision sensor 12 mounted on the robotic arm 11, eliminating the need for a separate mechanism for guiding the trolley 2. This allows the trolley 2 to be constructed with minimal equipment and cost, while enabling the trolley 2 to move with high precision along the path.

[0181] Furthermore, the marker L2 is positioned at the upper end of the column 71 that rises upwards from the ground F. Therefore, compared to placing the marker L2 directly on the ground F, it is less prone to soiling or damage. Consequently, when the vision sensor 12, mounted on the end effector 11a of the robotic arm 11 of the work robot 1, is used to photograph the marker L2, it is easy to bring the vision sensor 12 close to the marker L2. As a result, the marker L2 can be photographed easily and accurately, thereby enabling effective control of the trolley 2's movement.

[0182] Furthermore, the column 71 also functions as a protective component to prevent collisions with surrounding workers when the trolley 2 leaves the path.

[0183] (2) The operation robot system 100 involved in the second method is the operation robot system 100 of (1), wherein the column 71 has an upper surface 71t facing upward, and the mark L2 has an upper mark L2a disposed on the upper surface 71t.

[0184] Therefore, while the trolley 2 is moving, by pointing the vision sensor 12 downwards to take pictures, it is easy to take pictures of the upper mark L2a set on the upper surface 71t of the column 71.

[0185] (3) The operation robot system 100 involved in the third method is the operation robot system 100 of (1) or (2), wherein the column 71 has a side 71s facing to the side, and the mark L2 has a side mark L2b disposed on the side 71s.

[0186] Therefore, by setting the side mark L2b on the side 71s of the column 71, the mark L2 (side mark L2b) can be captured by the vision sensor 12 even when it is far away from the column 71 component. Thus, the mark L2 can be easily detected even when the trolley 2 is off the path.

[0187] (4) The operation robot system 100 involved in the fourth method is any one of (1) to (3) operation robot system 100, wherein the operation robot 1 further comprises: a support base 15, which is configured to protrude outward when the top plate 2t of the trolley 2 is viewed from above, and the mechanical arm 11 is disposed on the support base 15.

[0188] Therefore, the robotic arm 11 is mounted on a support 15, which is configured to protrude outwards when viewed from above the top plate 2t of the trolley 2. Thus, compared to the case where the robotic arm 11 is mounted on the top plate 2t, the extent to which the end 11a of the robotic arm 11 reaches the side of the top plate 2t where the support 15 is mounted is wider. Therefore, even when the marking member 7 is far from the travel path of the trolley 2, the marking L2 of the marking member 7 can be easily detected using the vision sensor 12. As a result, the degree of freedom in positioning the marking member 7 is increased.

[0189] (5) The operation robot system 100 involved in the fifth method is the operation robot system 100 of (4), wherein the height of the marking component 7 from the ground F is less than the height of the support 15 from the ground F.

[0190] Therefore, even if the trolley 2 deviates from the path, it can suppress the collision between the support seat 15 and the marking component 7.

[0191] (6) The operation robot system 100 involved in the sixth method is any one of (1) to (5) operation robot system 100, wherein the operation robot 1 further comprises: an obstacle detection sensor 16 for detecting obstacles around the trolley 2, and the marking component 7 is configured to be detectable by the obstacle detection sensor 16.

[0192] Therefore, the marker component 7 is configured to be detectable by the obstacle detection sensor 16, thus preventing collisions between the trolley 2 and the marker component 7.

[0193] (7) The work robot system 100 involved in the seventh method is any one of (1) to (6) of the work robot system 100, wherein the control device 10 includes: a detection unit 103, which detects the offset of the trolley 2 relative to the specified path based on the mark L2 included in the image captured by the vision sensor 12; and a driving instruction unit 104, which issues an instruction to the trolley 2 in a manner that corrects the driving position or driving posture of the trolley 2 based on the offset.

[0194] Therefore, based on the marker L2 included in the image captured by the vision sensor 12, the offset of the trolley 2 relative to the specified path is detected, and the driving position or driving posture of the trolley 2 is corrected according to the offset, thereby enabling the trolley 2 to move along the path with high precision.

[0195] (8) The marking component 7 is used in the robot system 100 involved in the eighth method. The robot system 100 includes a robot 1, which includes: a trolley 2 capable of traveling on the ground F; a robotic arm 11 mounted on the trolley 2 and capable of arbitrarily changing the position and posture of the end effector; and a vision sensor 12 installed at the end effector 11a of the robotic arm 11. The marking component 7 includes: a column 71 set at a predetermined position on the ground F and standing upright from the ground F; and a mark L2 set at the upper end of the column 71 and including information related to the position on the ground F.

[0196] In the marking component 7 of the work robot system 100, the marker L2 is located at the upper end of the column 71 that rises from the ground F. Therefore, compared to placing the marker L2 on the ground F, it is less likely for the marker L2 to become soiled or damaged. Thus, when the vision sensor 12, mounted on the end effector 11a of the robotic arm 11 of the work robot 1, is used to photograph the marker L2, it is easy to bring the vision sensor 12 close to the marker L2. As a result, the marker L2 can be photographed easily and accurately, thereby enabling effective control of the trolley 2's movement.

[0197] Furthermore, the column 71 also functions as a protective component to prevent collisions with surrounding workers when the trolley 2 leaves the path.

[0198] (9) The control method of the operation robot system 100 involved in the ninth method is any one of the control methods of the operation robot system 100 in (1) to (7), which includes: step S101, causing the trolley 2 to travel along the specified path; step S102, taking a picture of the mark L2 by the vision sensor 12; and step S106, controlling the travel of the trolley 2 according to the mark L2 included in the image taken by the vision sensor 12.

[0199] (10) The driving program generation device 30 according to the 10th method includes: a path generation unit 301, which accepts input operation from the operator and generates path information D5, the path information D5 including the coordinates of a designated position as the destination of the working robot 1 and the coordinates of the passing positions passed up to the designated position, and a target posture indicating the direction of travel of the working robot 1 in the designated position and the passing positions; and a driving program generation unit 302, which calculates the amount of movement between each position and the direction change position of the working robot 1 to change the direction of travel based on the path information D5, and generates a driving program D6 consisting of actions that sequentially combine the following actions: the action of moving between each position of the working robot 1, the action of turning around at the direction change position, and the action of correcting the position and posture of the working robot 1 based on the mark L2 included in the image D2 captured by the vision sensor 12 of the working robot 1 at the direction change position.

[0200] Thus, by generating a driving program D6 that includes position and posture corrections at the direction change positions, the driving program generation device 30 can automatically correct the position and posture of the robot 1 each time it reaches a direction change position. Therefore, the driving program generation device 30 can improve the reliability of the robot 1 autonomously moving to the designated position (destination), thereby reducing the need for operator monitoring or operation of the moving robot 1. Furthermore, the operator, in preparing for the robot 1's movement, only needs to specify the designated position, the coordinates of the passing position, and the posture of the robot 1 at each position, eliminating the need for tedious tasks such as calculating the movement between positions and creating driving programs based on the robot's path. In other words, it can reduce the operator's workload or time required for driving program generation.

[0201] (11) The driving procedure generation device 30 involved in the 11th method is the driving procedure generation device 30 of (10), wherein the path generation unit 301 performs the following: displays a driving area screen including map information of the driving area of ​​the work robot 1 and starting position information indicating the current position and posture of the work robot 1 in the driving area; and generates path information D5 including the coordinates of the passing position and the designated position of the work robot 1, and the target posture in the passing position and the designated position, based on the position and posture of the work robot 1 set by the operator on the driving area screen.

[0202] Thus, the driving program generation device 30 can automatically generate path information D5 based on the position set by the operator on the driving area screen or the posture of the working robot 1. As a result, the operator can easily specify the movement path of the working robot 1 without manually inputting the coordinates of the specified position and the position passed through, or the numerical values ​​of the posture of the working robot 1.

[0203] (12) The driving procedure generation device 30 involved in the 12th method is the driving procedure generation device 30 of (11), wherein the driving area screen also includes channel information indicating the channel that the working robot 1 can drive in the driving area.

[0204] Therefore, the driving program generation device 30 can easily generate a path plan for the work robot 1, even for inexperienced operators unfamiliar with the driving area. This reduces the likelihood that the work robot 1 will take an inappropriate path, such as one that is too close to surrounding objects.

[0205] (13) The driving procedure generation device 30 involved in the 13th method is the driving procedure generation device 30 of (11), wherein the driving area screen also includes information indicating the setting position of the marker L2 in the driving area.

[0206] Therefore, the driving program generation device 30 can generate path information D5 and driving program D6 that can more reliably perform position and posture correction based on marker L2 when the work robot 1 changes direction. This suppresses the work robot 1 from deviating significantly from the movement path after changing direction due to the accumulation of position and posture errors. Consequently, the reliability of autonomous movement to the designated location (destination) can be further improved.

[0207] (14) The driving program generation device 30 involved in the 14th method is any one of (10) to (13) driving program generation devices 30, wherein the driving program generation unit 302 sets an intermediate passing position at the setting position of the mark L2 on the straight path when the amount of movement of the working robot 1 in a straight path between the passing position and other positions is more than a threshold, and generates a driving program D6 that adds an action to correct the position and posture of the working robot 1 at the intermediate passing position.

[0208] Therefore, the driving program generation device 30 can automatically generate a driving program that includes actions to correct the position and posture of the robot 1 at the setting position of marker L2, for example, when the distance of linear movement between the passing positions set by the operator is long. This reduces the risk that the robot 1 may lose its position due to error accumulation during long-distance movement or may deviate from the intended path and come into contact with objects in the surrounding environment.

[0209] (15) The driving program generation device 30 involved in the 15th method is any one of (10) to (13) driving program generation devices 30, wherein, when a marker L2 is set on a straight path that causes the working robot 1 to move in one direction between the passing position and other positions, the driving program generation unit 302 sets an intermediate passing position at the setting position of the marker L2 on the straight path and generates a driving program D6 that adds an action to correct the position and posture of the working robot 1 at the intermediate passing position.

[0210] Thus, the driving program generation device 30 can automatically generate a driving program that includes actions to correct the position and posture of the work robot 1 at the setting position of marker L2. This reduces the risk that the work robot 1 may lose its position due to accumulated errors during movement or deviate from the intended path and come into contact with objects in the surrounding environment.

[0211] (16) The driving program generation method involved in the 16th method includes the following steps: accepting the input operation of the operator and generating path information D5, the path information D5 including the coordinates of the designated position as the destination of the working robot 1 and the passing positions passed up to the designated position, and the target posture indicating the direction of travel of the working robot 1 in the designated position and the passing positions; and calculating the amount of movement between each position and the direction change position of the working robot 1 to change the direction of travel according to the path information D5, and generating a driving program D6 consisting of the following actions in sequence: the action of moving between each position of the working robot 1, the action of turning at the direction change position, and the action of correcting the position and posture of the working robot 1 according to the mark L2 included in the image D2 captured by the vision sensor 12 of the working robot 1 at the direction change position.

[0212] (17) The program involved in the 17th method causes the driving program generation device 30 to perform the following steps: accept the input operation of the operator and generate path information D5, the path information D5 including the coordinates of the designated position as the destination of the working robot 1 and the passing positions passed up to the designated position, and the target posture indicating the direction of travel of the working robot 1 in the designated position and the passing positions; and calculate the amount of movement between each position and the direction change position of the working robot 1 to change the direction of travel according to the path information D5, and generate a driving program D6 consisting of the following actions in sequence: the action of moving between each position of the working robot 1, the action of turning around at the direction change position, and the action of correcting the position and posture of the working robot 1 according to the mark L2 included in the image D2 captured by the vision sensor 12 of the working robot 1 at the direction change position.

[0213] Industrial availability

[0214] The work robot system, the marking component for the work robot system, and the control method for the work robot system according to the present invention enable the work robot to move with a simple and low-cost structure.

[0215] Symbol Explanation

[0216] 1-Working robot, 2-Trolley, 2s-Side, 2t-Top plate, 7-Marking component, 10-Robot control device (control unit), 11-Mechanical arm, 11a-End effector, 12-Vision sensor, 15-Support base, 16-Obstacle detection sensor, 26-Obstacle detection sensor, 71-Column, 71s-Side, 71t-Top surface, L2-Marking, L2a-Top marking, L2b-Side marking, 100-Working robot system, 103-Detection unit, 104-Travel instruction unit, 30-Travel program generation device, 301-Path generation unit, 302-Travel program generation unit, 303-Transmission and processing unit, F-Ground

Claims

1. A work robot system, comprising: The work robot includes a trolley capable of traveling on the ground, a robotic arm mounted on the trolley and capable of arbitrarily changing the position and posture of its end effector, and a vision sensor mounted on the end effector of the robotic arm; and The marking component is placed at a designated location on the ground. The marking component includes: A column, erected from the ground upwards; and A marker is placed at the upper end of the column and includes information related to its position on the ground. The robot has the following features: The control device controls the movement of the trolley based on the markings included in the image captured by the vision sensor.

2. The work robot system according to claim 1, wherein, The column has an upward-facing upper surface. The mark has an upper mark disposed on the upper surface.

3. The work robot system according to claim 1 or 2, wherein, The column has a side facing to the side. The mark has a side mark disposed on the side.

4. The work robot system according to claim 1 or 2, wherein, The robot also has the following features: The support base is configured to protrude outwards when the top plate of the trolley is viewed from above. The robotic arm is mounted on the support base.

5. The work robot system according to claim 4, wherein, The height of the marking component from the ground is less than the height of the support base from the ground.

6. The work robot system according to claim 1 or 2, wherein, The robot also has the following features: Obstacle detection sensors detect obstacles around the trolley. The marking component is configured to be detectable by the obstacle detection sensor.

7. The work robot system according to claim 1 or 2, wherein, The control device includes: The detection unit detects the offset of the trolley relative to a specified path based on the markings included in the image captured by the vision sensor; and The driving instruction unit issues instructions to the trolley in a manner that corrects the trolley's driving position or driving posture based on the offset amount.

8. A marking component for a work robot system, comprising a work robot including a trolley capable of traveling on the ground, a robotic arm mounted on the trolley and capable of arbitrarily changing the position and posture of its end effector, and a vision sensor mounted on the end effector of the robotic arm. The marking component of the robot system includes: A column, positioned at a predetermined location on the ground and erected upwards from the ground; and A marker is placed on the upper end of the column and includes information related to its location on the ground.

9. A control method for a work robot system, which is the control method for the work robot system according to claim 1 or 2, comprising the following steps: To make the trolley travel along the specified path; The mark is captured by the visual sensor; and The movement of the trolley is controlled based on the markings included in the images captured by the vision sensor.

10. A driving procedure generation apparatus, comprising: The path generation unit accepts operator input and generates path information, which includes a designated location as the destination of the robot, the coordinates of each of the passing locations until reaching the designated location, and a target posture indicating the robot's direction of travel at each of the designated location and each passing location; and The driving program generation unit calculates the amount of movement between each position and the direction change position of the working robot based on the path information, and generates a driving program that sequentially combines the following actions: the action of moving between each position of the working robot, the action of turning around at the direction change position, and the action of correcting the position and posture of the working robot based on the marks included in the image captured by the visual sensor of the working robot at the direction change position.

11. The driving procedure generation apparatus according to claim 10, wherein, The path generation unit performs the following: The display shows the driving area, which includes map information of the robot's driving area and starting position information indicating the robot's current position and posture within the driving area. Based on the position and posture of the working robot set by the operator on the driving area screen, path information is generated. The path information includes the coordinates of the passing position and the designated position of the working robot, as well as the target posture in each of the passing position and the designated position.

12. The driving procedure generation apparatus according to claim 11, wherein, The driving area screen also includes channel information indicating the paths that the robot can travel within the driving area.

13. The driving procedure generation apparatus according to claim 11, wherein, The driving area screen also includes information indicating the location of the marker within the driving area.

14. The driving procedure generation apparatus according to any one of claims 10 to 13, wherein, When the amount of movement of the work robot along a straight path between the passing position and other positions is greater than or equal to a threshold, the driving program generation unit sets an intermediate passing position at the location of the mark on the straight path and generates a driving program that adds an action to correct the position and posture of the work robot at the intermediate passing position.

15. The driving procedure generation apparatus according to any one of claims 10 to 13, wherein, When the marker is set on a straight path that causes the work robot to move in one direction between the passing position and other positions, the driving program generation unit sets an intermediate passing position at the location of the marker on the straight path and generates a driving program that adds an action to correct the position and posture of the work robot at the intermediate passing position.

16. A method for generating a driving procedure, comprising the following steps: The system accepts operator input and generates path information, which includes the coordinates of a designated location as the destination of the robot, the coordinates of each of the locations traversed up to the designated location, and a target pose indicating the robot's direction of travel at each of the designated location and each of the traversed locations; and Based on the path information, the amount of movement between each position and the direction change position of the robot are calculated, and a driving program is generated by sequentially combining the following actions: the action of moving between each position of the robot, the action of turning around at the direction change position, and the action of correcting the position and posture of the robot based on the marks included in the image captured by the robot's vision sensor at the direction change position.

17. A program that causes a driving procedure generation device to perform the following steps: The system accepts operator input and generates path information, which includes the coordinates of a designated location as the destination of the robot, the coordinates of each of the locations traversed up to the designated location, and a target pose indicating the robot's direction of travel at each of the designated location and each of the traversed locations; and Based on the path information, the amount of movement between each position and the direction change position of the robot are calculated, and a driving program is generated by sequentially combining the following actions: the action of moving between each position of the robot, the action of turning around at the direction change position, and the action of correcting the position and posture of the robot based on the marks included in the image captured by the robot's vision sensor at the direction change position.

Images