Control device and program

Abstract

A control device according to the present disclosure includes: a storage unit that stores data relating to a plurality of holding modes for holding a workpiece by an end effector of a robot; a selection unit that selects one of a plurality of holding modes for the workpiece; a control unit that controls the robot on the basis of the selected one of the holding modes; a creation unit that creates a plurality of holding pattern candidates on the basis of the holding pattern; a receiving unit that receives a user instruction for adding at least one of the created candidates for the holding mode as the holding mode; and a storage control unit that stores the received holding pattern candidates in the storage unit as holding patterns.

Description

Technical Field

[0001] The present disclosure relates to a control device and a program. Background Art

[0002] A technology for bulk unloading a plurality of workpieces, irregularly spaced and arranged in a container such as a box, is known (e.g., Patent Document 1). For example, in bulk unloading, the robot is pre-taught with candidate positions for gripping the workpieces using its onboard hands.

[0003] Prior art literature Patent Literature Patent Document 1: Japanese Patent Application Laid-Open No. 2019-028775. Summary of the Invention

[0004] Problems to be solved by the invention Because robots can only remove workpieces at set gripping positions, they are sometimes unable to remove workpieces from a box due to interference with the box or the workpieces by the robot's hands. To address this, users observe the position and orientation of workpieces remaining in the box that cannot be gripped by hand and teach the gripping positions for workpieces that can be removed by hand. Since teaching gripping positions requires thinking about which gripping positions should be taught to remove the workpiece by hand while using an operating device such as a teach pendant, it is a burdensome task for users who are not accustomed to bulk removal settings. Therefore, it is desirable to provide a technology that reduces the effort required to teach gripping positions for bulk removal.

[0005] Means used to solve problems The control device disclosed herein is intended to control a robot that performs bulk workpiece removal operations, and comprises: a storage unit that stores data related to a plurality of holding modes, wherein the holding modes are used to hold the workpiece using the end effector of the robot; a selection unit that selects one of the plurality of holding modes for the workpiece; a control unit that controls the robot based on the selected holding mode; a production unit that produces a plurality of holding mode candidates based on the holding mode; a receiving unit that receives a user instruction to add at least one of the produced holding mode candidates as a holding mode; and a storage control unit that stores the accepted holding mode candidate as a holding mode in the storage unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 1 is a diagram showing a robot system including the control device according to the first embodiment.

[0007] Figure 2 yes Figure 1 Hardware structure diagram of the control device.

[0008] Figure 3 yes Figure 1 Functional block diagram of the control device.

[0009] Figure 4 It shows Figure 3 FIG. 1 is a diagram showing an example of a holding mode management table stored in a storage unit of FIG.

[0010] Figure 5 Is used to illustrate Figure 4 Supplementary figure of the retention pattern.

[0011] Figure 6 Is used to illustrate Figure 4 Supplementary diagram of the preparation process of the candidate for the holding mode.

[0012] Figure 7 Is shown by Figure 3 An example of a receiving screen created by the screen creation unit.

[0013] Figure 8 This is a flowchart showing an example of a procedure for the control device according to the first embodiment to perform a workpiece removal operation.

[0014] Figure 9 It shows Figure 8 A flowchart of an example of the procedure of step S22.

[0015] Figure 10 Is shown click Figure 7 This is an example of the update acceptance screen after clicking the Create button.

[0016] Figure 11 Is shown click Figure 10 An example of an update acceptance screen after clicking the Add button.

[0017] Figure 12 Is shown click Figure 7 This is another example of the update acceptance screen after clicking the Create button.

[0018] Figure 13 Is shown click Figure 11 This figure shows an example of the update acceptance screen after clicking the Execute button.

[0019] Figure 14 This is a functional block diagram of a control device according to the second embodiment.

[0020] Figure 15 An example of an acceptance screen presented to the user by the control device according to the second embodiment is shown.

[0021] Figure 16 It shows Figure 15The image of the receiving screen when the cursor is aligned with the interference information.

[0022] Figure 17 It shows Figure 15 Figure 2 shows the acceptance screen when the adjustment button is clicked. DETAILED DESCRIPTION

[0023] Below, with reference to the attached Figure 1 In the following description, components having substantially the same function and structure are denoted by the same reference numerals, and repeated description is given only when necessary.

[0024] In this embodiment, the terms are defined as follows.

[0025] Holding mode: This is a parameter set for the robot to perform workpiece removal operations. A holding mode typically includes the position (holding position) at which the robot's hand (end effector) holds the workpiece, and the posture (holding posture) at which the workpiece is held. Of course, a holding mode may simply represent either the holding position or the holding posture. Typically, multiple holding modes are prepared for bulk removal. The control device selects one of the multiple holding modes for each workpiece to be removed and controls the robot based on the selected holding mode. This allows the workpiece to be removed from a box, etc.

[0026] Candidates for holding patterns are parameters registered as candidates for holding patterns for the robot or the control device that controls the robot. Candidates for holding patterns are created based on existing holding patterns.

[0027] In this embodiment, the concept of "holding" encompasses suction, gripping, and other aspects. Specifically, when using a hand capable of gripping a workpiece, the holding mode (holding position, holding posture) can be renamed the gripping mode (gripping position, gripping posture). Similarly, when using a hand capable of gripping a workpiece, the holding mode (holding position, holding posture) can be renamed the suction mode (suction position, suction posture).

[0028] The control device disclosed herein is a computer device having: a function of creating candidates for a holding mode based on a plurality of pre-registered holding modes; a function of accepting user instructions for additionally registering at least one candidate for a holding mode from the created candidates for the holding mode; and a function of storing at least one accepted candidate for the holding mode as a holding mode.

[0029] (First embodiment) like Figure 1As shown, a robot system 1 including a control device 2 according to the first embodiment includes: a robot 10 equipped with a robot hand 11 as an end effector for removing bulk workpieces W from a box 15; a 3D vision camera 13 positioned to overlook the opening of the box 15 housing the workpieces W and to image the box 15; and a control device 2 for controlling the robot 10 and the 3D vision camera 13. The 3D vision camera 13 generates image data regarding the workpieces W housed in the box 15. This image data is also referred to as point cloud data.

[0030] like Figure 2 As shown, the control device 2 is configured by hardware connections including a processor 5 such as a CPU, an operating device 6 , a display device 7 , a communication device 8 , and a storage device 9 .

[0031] The operating device 6 is implemented by a keyboard, a mouse, a jog device, etc. The operating device 6 can be implemented by a touch panel that also serves as a display device 7, or by a dedicated teaching pendant of the robot 10. The user can input various information to the control device 2 via the operating device 6. The display device 7 is implemented by an LCD, etc., and displays various screens according to the control of the processor 5. The communication device 8 is implemented by a communication module according to an arbitrary communication specification, and sends and receives various data between the robot 10 and the three-dimensional vision camera 13 according to the control of the processor 5. The storage device 9 is implemented by an HDD, an SSD, etc. The storage device 9 stores a control program and a support program. The support program can be constituted as a part of the control program, or it can be executed in conjunction with the control program.

[0032] like Figure 3 As shown, the control program stored in the storage device 9 is executed together with the support program by the processor 5, so that the control device 2 functions as a receiving unit 21, a display unit 22, an acquisition unit 23, a storage unit 24, a robot control unit 25, a workpiece detection unit 26, an object workpiece determination unit 27, a judgment unit 28, a holding mode selection unit 29, a holding mode candidate creation unit 30, a storage control unit 31, and a screen creation unit 32.

[0033] The receiving unit 21 receives the user operation via the operating device 6. Figure 2 Specifically, the receiving unit 21 receives user operations on the receiving screen displayed by the display device 7. User operations on the receiving screen include clicking the create button, clicking the execute button, clicking the add button, selecting at least one holding mode candidate stored in the storage unit 24 as a holding mode from a plurality of holding mode candidates, and inputting the priority of at least one holding mode candidate stored in the storage unit 24. The display unit 22 displays the receiving screen produced by the screen producing unit 32. The display unit 22 is operated by Figure 2The acquisition unit 23 acquires the image data of the box 15 from the 3D vision camera 13. The acquisition unit 23 acquires the image data of the box 15 from the 3D vision camera 13. Figure 2 The functions of the communication device 8 shown are realized.

[0034] The storage unit 24 stores various data required for the robot 10 to perform bulk removal. Figure 2 The various data required for the motion control of the robot 10 include an action sequence, setting data, and a holding pattern management table. The action sequence describes the order for the robot 10 to perform bulk removal. The setting data includes the robot 10's motion speed, motion form, interpolation form, etc. The holding pattern management table stores the holding pattern of the robot hand 11 for the workpiece W in a table format. The details of the holding pattern management table will be described later. In addition, the holding pattern of the robot hand 11 for the workpiece W may be stored in the storage unit 24 in another format rather than a table format.

[0035] The storage unit 24 stores various data required for support processing to assist the user in adding operations in the hold mode. This data includes data required for screen creation by the screen creation unit 32 and data required for determination processing by the determination unit 28. The data required for screen creation includes screen formats, text data, image data, and basic models for schematic diagrams. The data required for determination processing includes data on a three-dimensional model of the robot 10 including the robot hand 11, data on a three-dimensional model of the workpiece W, and data on a three-dimensional model of the box 15, all necessary for simulating the robot hand 11 holding the workpiece W contained in the box 15.

[0036] The robot control unit 25 controls the robot 10 and the 3D vision camera 13 according to the action sequence so that the robot 10 can remove the bulk workpieces W from the box 15. While holding the target workpieces W, the robot control unit 25 controls the robot 10 based on the holding mode selected by the holding mode selection unit 29, which will be described later.

[0037] The workpiece detection unit 26 detects the workpiece W inside the box 15 based on the image data acquired from the three-dimensional vision camera 13. For example, the detection of the workpiece W can employ known image processing methods such as template matching using pre-registered template data, three-dimensional matching using shape data of a three-dimensional model of the workpiece W stored in the storage unit 24, or extraction of the inner area of the box 15 and pixel thresholding of the extracted inner area.

[0038] The target workpiece determination unit 27 determines the workpiece W to be held (referred to as the target workpiece W) based on the image data acquired from the 3D vision camera 13. For example, the target workpiece determination unit 27 randomly determines the target workpiece W from the workpieces W detected in their entirety in the image data, in other words, from the workpieces W exposed in their entirety at the opening of the box 15.

[0039] The determination unit 28 determines the possibility of holding the target workpiece W in the holding mode (a candidate holding mode) based on the image data acquired from the 3D vision camera 13. Specifically, the determination unit 28 calculates the position and posture of the target workpiece W and the positions and postures of other workpieces W in the box 15 based on the image data. The determination unit 28 then calculates the holding position and holding posture of the target workpiece W based on the holding mode. It also calculates the possibility of interference between the robot hand 11 and the box 15 or other workpieces W when the robot hand 11 is positioned at the holding position for the target workpiece W in the holding mode. The determination unit 28 then calculates the holding possibility based on the interference possibility.

[0040] For example, the possibility of interference between the robot hand 11 and the box 15 or other workpiece W is calculated as follows. Specifically, if the area corresponding to the robot hand 11 overlaps with the area corresponding to the box 15 or other workpiece W, the possibility of interference is calculated as 100%. If the area corresponding to the robot hand 11 does not overlap with the area corresponding to the box 15 or other workpiece W, and the areas are separated by a predetermined distance, the possibility of interference is calculated as 0%. If the area corresponding to the robot hand 11 does not overlap with the area corresponding to the box 15 or other workpiece W, but the areas are close to each other by less than a predetermined distance, the possibility of interference is calculated based on the distance.

[0041] The determination unit 28 calculates the holding probability by subtracting the interference probability from 100%. That is, an interference probability of 100% indicates a holding probability of 0%, and an interference probability of 0% indicates a holding probability of 100%. The determination unit 28 compares the holding probability with a preset threshold to determine whether the target workpiece W can be held using each holding mode.

[0042] The holding mode selection unit 29 selects a holding mode for holding the target workpiece W from among the plurality of holding modes. Specifically, the holding mode selection unit 29 selects a holding mode determined by the determination unit 28 to be highly likely to hold the target workpiece W. If a priority order is set for the holding modes, the holding mode selection process sequentially verifies the holding modes starting with the highest priority.

[0043] The holding pattern candidate creation unit 30 creates holding pattern candidates based on existing holding patterns. Details of the holding pattern candidate creation process will be described later.

[0044] The storage control unit 31 stores the candidate for the holding pattern selected by the user operation as a holding pattern in the storage unit 24. Thus, the accepted candidate for the holding pattern is added to the existing holding pattern.

[0045] The screen creating unit 32 creates an acceptance screen in accordance with a predetermined format. The details of the acceptance screen will be described later.

[0046] Below, refer to Figure 4 The hold mode management table will be described. Figure 4 An example of the hold mode management table is shown in FIG. Figure 4 As shown, in the holding mode management table, multiple holding modes are managed, along with registration date and time information and priority. A holding mode includes parameters related to the holding position and parameters related to the holding posture. Typically, these parameters are expressed in a workpiece coordinate system. The workpiece coordinate system is a coordinate system defined by the target workpiece W.

[0047] Below, refer to Figure 5 Here, we will explain the hold mode. Figure 4 The maintenance mode H1 managed in the maintenance mode management table is used as an example for explanation. Figure 5 As shown, the workpiece W is a cylindrical object. The workpiece coordinate system defined by the workpiece W and the hand coordinate system defined by the robot hand 11 are defined as follows. Specifically, the center position of the cylinder is defined as the origin Ow of the workpiece coordinate system, the axis parallel to the centerline of the cylinder is defined as the Xw axis, and two axes orthogonal to the Xw axis and to each other are defined as the Yw axis and the Zw axis, respectively. Furthermore, the grip center position of a pair of fingers is defined as the origin Oh of the hand coordinate system, the axis along the opening and closing direction of the pair of fingers is defined as the Yh axis, the axis along the centerline of the robot hand 11 is defined as the Zh axis, and the axis orthogonal to the Yh and Zh axes is defined as the Xh axis.

[0048] When the workpiece coordinate system and the hand coordinate system are defined as above, Figure 5 As shown, the holding pattern H1 ((Xw, Yw, Zw, Ww, Pw, Rw) = (10 mm, 0, 0, 0, 0, 0)) means that the robot hand 11 holds the workpiece W at a position offset by 10 mm from the center position (Ow) of the workpiece W along the center axis (Xw axis) in a posture in which the robot hand 11 does not tilt relative to the workpiece W.

[0049] In addition, when the axes of the workpiece coordinate system cannot be specified only by the characteristics of the workpiece W, such as a workpiece W having a line-symmetrical or point-symmetrical shape such as a cylinder, a sphere, or a cube, at least one of the three axes of the workpiece coordinate system can be defined based on other fixed coordinate systems such as the box coordinate system (Xc, Yc, Zc).

[0050] In addition, the holding pattern of the workpiece W may be expressed not by the workpiece coordinate system but by another coordinate system such as the box coordinate system.

[0051] Below, refer to Figure 6 , the production process of holding mode candidates performed by the holding mode candidate production unit 30 is described. Typically, the holding mode candidate production unit 30 produces holding mode candidates based on the holding mode, the three-dimensional model of the box 15, the three-dimensional model of the target workpiece W, and the three-dimensional model of the robot hand 11. Specifically, the holding mode candidate production unit 30 extracts multiple holding mode candidates by searching within a specified range of the distance holding mode, and produces at least one holding mode candidate from the extracted multiple holding mode candidates. The specified range can be set according to the user specification received via the receiving unit 21, and can also be automatically set according to the target workpiece W. As Figure 6 As shown, for example, when the length of the target workpiece W is Lw, the maximum value of the predetermined range of search targets as holding pattern candidates is the range where the holding position Xw is less than ±Lw / 2. The narrower the search range, the faster the calculation processing speed can be.

[0052] The following is based on Figure 4 The example of making a candidate for the holding mode is explained by using the holding mode H1 managed by the holding mode management table. Figure 6 As shown, the holding pattern candidate creation unit 30 searches within the length Lw of the target workpiece W, extracting multiple holding pattern candidates (H11, H12, ..., H20) by shifting the holding pattern H1 by a unit length ΔL along the Xw-axis. From these multiple extracted holding pattern candidates (H11, H12, ..., H20), a holding pattern candidate (H18) that is capable of holding the target workpiece W (i.e., without interfering with surrounding objects) and closest to the existing holding pattern H1 is extracted and created. Furthermore, whether the candidate holding pattern can hold the target workpiece W—in other words, whether the robot hand 11 operating with the candidate holding pattern will interfere with surrounding objects—can be determined through simulation using the three-dimensional model of the box 15, the three-dimensional model of the target workpiece W, and the three-dimensional model of the robot hand 11, similar to the processing of the determination unit 28.

[0053] Below, refer to Figure 7 , explaining the acceptance screen. Figure 7 FIG. 8 shows an example of an acceptance screen displayed by the display device 7. Figure 7As shown, the receiving screen 100 has a first area 110, a second area 113, and a third area 115. The first area 110 displays the shooting data related to the box 15' and an execution button 111. The second area 113 displays the holding mode stored in the storage unit 24 together with the priority 121. For example, the holding mode is represented as a schematic diagram that imitates the workpiece W and the robot hand 11 so that the holding position and holding posture relative to the workpiece W can be known. The third area 115 displays a production button 117. Although Figure 7 , but multiple schematic diagrams representing multiple holding mode candidates are displayed in the third area (see Figure 10 、 Figure 11 ).

[0054] Below, refer to Figure 8 , the bulk taking-out action performed by the robot 10 is described. Figure 8 This is a flowchart showing an example of a control procedure of the robot 10 by the control device 2 when causing the robot 10 to execute bulk removal.

[0055] like Figure 8 As shown, when the control program is executed, the control device 2 controls the 3D vision camera 13 to acquire image data regarding the workpiece W contained in the box 15 (S11). Based on the acquired image data, the control device 2 determines whether a workpiece W is present inside the box 15 (S12). If a workpiece W is present inside the box 15 (S13: Yes), the control device 2 determines the target workpiece W to be retained based on the image data (S14) and determines the likelihood of retaining the determined target workpiece W based on multiple retention patterns (S15). The determination of the likelihood of retaining the target workpiece W based on the multiple retention patterns is performed sequentially according to the priority of the retention patterns. Furthermore, by assigning priorities, step S15 is terminated when a retention pattern with a high likelihood of retaining the target workpiece W is discovered. This prioritizes frequently used retention patterns, thereby minimizing the increase in the processing speed of step S15.

[0056] If the probability of holding the target workpiece W is greater than or equal to a predetermined value, in other words, if the target workpiece W can be held in any of the multiple holding modes (S16: No), the control device 2 selects a holding mode that can hold the target workpiece W from the multiple holding modes (S17) and controls the robot 10 according to the selected holding mode (S18). As a result of step S18, the target workpiece W is removed from the box 15. On the other hand, if the probability of holding the target workpiece W is less than a predetermined value, in other words, if none of the multiple holding modes can hold the target workpiece W (S16: Yes), the control device 2 determines whether there are any other unverified workpieces W based on the image data (S19). These other unverified workpieces W are workpieces W that have not been subjected to the determination process in step S15.

[0057] If there are other unverified workpieces W, in other words, if there is still a possibility that the workpieces W remaining in the box 15 can be held using the existing holding pattern (S20: Yes), the control device 2 changes the target workpiece W to an unverified workpiece W (S21) and returns the process to step S15. On the other hand, if there are no other unverified workpieces W, in other words, if the workpieces W remaining in the box 15 cannot be held using the existing holding pattern (S20: No), the control device 2 performs additional registration processing for a new holding pattern (S22) and returns the process to step S15.

[0058] The series of processes from steps S11 to S22 is repeatedly executed until all the workpieces W contained in box 15 are removed and no workpieces W are left inside box 15. When no workpieces W are left inside box 15, control of robot 10 by controller 2 ends, and bulk removal by robot 10 is completed (S13: NO).

[0059] Below, refer to Figure 7 、 Figure 9 、 Figure 10 、 Figure 11 , the user support process performed by the control device 2 according to the first embodiment will be described. Figure 9 It shows Figure 8 A flowchart of an example of the procedure of step S22. Figure 10 The acceptance screen updated when the Create button is clicked is shown. Figure 11 The updated acceptance screen is shown when the add button is clicked.

[0060] like Figure 9 As shown, when the workpiece W remaining in the box 15 cannot be held in the existing holding mode, the control device 2 displays the acceptance screen (S221). Through the processing of step S221, the display device 7 displays the following Figure 7The control device 2 then waits for the production process of the next step until the production button 117 displayed on the acceptance screen 100 is clicked (S222: No).

[0061] When the creation button 117 is clicked (S222: Yes), the control device 2 creates a candidate for the holding mode based on the holding mode (S223). Then, the control device 2 creates the acceptance screen 100 in which the schematic diagram corresponding to the candidate for the holding mode is inserted according to a predetermined format, and updates the acceptance screen 100 displayed on the display device 7 (S224). Through the processing of step S224, Figure 7 The acceptance screen 100 shown is updated to Figure 10 The acceptance screen 100 is shown. Figure 10 As shown, the updated third area 115 of the acceptance screen 100 displays multiple schematic diagrams 130, each representing a plurality of candidate holding modes created in step S223. In the schematic diagrams 130, a check box 131 and a priority input field 132 are displayed in association with each other. The control device 2 then places the additional registration process for the next step on standby until the add button 119 displayed on the acceptance screen 100 is clicked (S225: No). By clicking the check box 131 displayed on the acceptance screen 100, the user can select a candidate holding mode to be additionally registered. Furthermore, by inputting into the input field 132, the priority can be set.

[0062] When the add button 119 is clicked (S225: Yes), at least one holding mode candidate selected by the user from the plurality of holding mode candidates displayed on the acceptance screen 100 is added and registered in the holding mode management table (S226) according to the priority order set by the user. Figure 10 The acceptance screen 100 shown is updated to Figure 11 The acceptance screen 100 is shown. Figure 11 As shown, the schematic diagram 120 corresponding to the candidate for the hold mode selected by the user and displayed in the third area 115 is added to the second area 113 of the updated acceptance screen 100 .

[0063] The priority order set by the user can be defined as the priority order among the candidates for the additional hold mode to be registered, or as the priority order within the entire group of existing hold modes. The definition of the priority order can be switched according to user instructions. In addition, the priority order of the existing hold mode can be changed at any time according to user instructions. For example, in the case of a priority order defined as a candidate for the additional hold mode to be registered, a lower priority order than the priority order assigned to the existing hold mode is assigned to the candidate for the additional hold mode to be registered according to the priority order set by the user.

[0064] The control device 2 of the first embodiment provides the following advantages. Specifically, the control device 2 of the first embodiment can present the user with a reception screen 100 including candidate holding modes, where the candidate holding modes are additionally registered. The user does not need to teach the holding mode by directly inputting coordinates or operating on a simulation screen. Instead, the user simply confirms the presented candidate holding modes, selects the candidate holding mode to be additionally registered, and clicks the add button 119. This allows the selected candidate holding mode to be additionally registered (taught) as the holding mode. Therefore, even users unfamiliar with bulk removal settings can easily perform additional (teaching) holding modes for workpieces W.

[0065] The aforementioned additional registration of holding patterns can be performed when the workpieces W remaining in the box 15 cannot be held using the existing holding patterns. Therefore, the new holding patterns added through the additional registration of holding patterns are only those that can hold target workpieces W that cannot be held using the existing holding patterns. This allows cumulative addition of only those holding patterns that can hold workpieces W that cannot be held using the existing holding patterns. This prevents an unnecessary increase in holding patterns and gradually reduces the time the robot 10 stops due to an inability to hold a workpiece W.

[0066] Furthermore, the candidate holding modes presented to the user are likely to retain the target workpiece W without interfering with surrounding objects through simulation. However, even if a candidate holding mode is determined by simulation to be capable of retaining the target workpiece W, it may not actually be able to retain the target workpiece W due to the state of the robot 10, the workpiece W, or the surrounding environment. In the first embodiment, instead of automatically adding the generated candidate holding modes as holding modes, only the candidate holding modes ultimately confirmed by the user are added. This allows only candidate holding modes with a high probability of retaining a workpiece W that cannot be retained by existing holding modes to be added. This helps to appropriately prevent the unnecessary increase in holding modes.

[0067] In the first embodiment, the holding pattern candidate creation unit 30 creates holding pattern candidates based on the holding pattern, the three-dimensional model of the box 15, the three-dimensional model of the target workpiece W, and the three-dimensional model of the robot hand 11. This allows the user to narrow down the holding pattern candidates to those capable of holding the target workpiece W and present them to the user. Therefore, regardless of which one the user selects, they can additionally register a holding pattern with a high probability of holding the target workpiece W. However, because the user ultimately determines whether holding is possible, the holding pattern candidates may be presented to the user regardless of whether the target workpiece W can be held.

[0068] Therefore, the holding pattern candidate generation unit 30 can generate holding pattern candidates based solely on the holding pattern or based on the holding pattern and the three-dimensional model of the target workpiece W. Specifically, the holding pattern candidate generation unit 30 can generate holding pattern candidates by adding or subtracting a predetermined offset from the holding pattern. Furthermore, the holding pattern candidate generation unit 30 can extract multiple holding pattern candidates by searching within a predetermined range from the holding pattern H1 and generate a single holding pattern candidate that is closest to the center of gravity of the target workpiece W based on the extracted multiple holding pattern candidates.

[0069] In the first embodiment, the acceptance screen is displayed when the workpieces W remaining in the box 15 cannot be held in the existing holding mode. However, the timing of displaying the acceptance screen can be arbitrary. For example, the acceptance screen may be displayed continuously after bulk removal starts.

[0070] In the first embodiment, the preparation process of the candidate holding mode is executed based on the click of the preparation button displayed on the acceptance screen, that is, the user operation. However, it can also be automatically executed when it is determined that the workpieces W remaining in the box 15 cannot be held in the existing holding mode.

[0071] exist Figure 10 In the reception screen 100 shown, the user must determine which workpiece W' is the candidate for holding each of the plurality of holding mode candidates. Figure 12 As shown, in order to clarify which workpiece W' is being suggested as a candidate for holding each of the multiple holding mode candidates, a balloon is displayed, for example, to indicate the correspondence between the workpiece W' and the holding mode candidate. This further reduces the user's effort in additionally selecting the holding mode candidates. The method for displaying the correspondence between the workpiece W' and the holding mode candidate that can hold the workpiece W' is not limited to balloons; for example, existing identification methods such as color and hatching can be used.

[0072] The reception screen 100 is provided with an execution button 111, which allows a user to check through simulation whether the workpiece W' remaining in the box 15 can be held in the newly added or existing holding mode. When the execution button 111 is clicked, a simulation is executed based on the image data, and information is displayed on whether the workpiece W' remaining in the box 15 can be held in a plurality of holding modes including a candidate for the newly added holding mode. For example, Figure 13 As shown, if the execution button 111 is clicked, the correspondence between the workpiece W' and the holding mode is displayed through a dialog box so that the user can know which holding mode can be used to hold the workpiece W' remaining in the box 15. In this way, the user can reconfirm whether the workpiece W' remaining in the box 15' can be held in the newly added holding mode. If the added holding mode cannot hold any of the target workpieces W', the newly added holding mode can be deleted because the holding mode is not needed. In this way, the situation where the holding modes are unnecessarily increased can be avoided. The method of displaying the correspondence between the workpiece W' and the holding mode that can hold the workpiece W' is not limited to a dialog box. For example, existing identification methods such as color and hatching can be used.

[0073] (Second embodiment) The control device 2' of the second embodiment is a device in which a function of adjusting the candidate of the holding mode is added to the control device 2 of the first embodiment. Since the hardware structure of the control device 2' of the second embodiment is the same as that of the control device 2 of the first embodiment, its description is omitted (see Figure 2 ).

[0074] Below, refer to Figure 14 、 Figure 15 、 Figure 16 、 Figure 17 , the control device 2 of the second embodiment will be described. Figure 14 : is a functional block diagram of the control device 2 of the second embodiment. Figure 14 As shown, the control device 2' of the second embodiment is a control device 2 of the first embodiment (see Figure 3 ) is composed of an additional adjustment part.

[0075] The accepting unit 21 accepts a user instruction regarding adjustment of a plurality of hold mode candidates as a user operation on the acceptance screen. The adjusting unit 33 adjusts the hold mode candidates according to the user adjustment instruction accepted by the accepting unit 21.

[0076] Figure 15 An example of an acceptance screen presented to the user by the control device 2 according to the second embodiment is shown. Figure 16 It shows Figure 15 The image of the receiving screen when the cursor is aligned with the interference information. Figure 17 It shows Figure 15 This is the image of the acceptance screen when the adjustment button is clicked. Figure 15 Compared with the first embodiment Figure 10 correspond.

[0077] like Figure 15 As shown, in the acceptance screen 100' presented to the user by the control device 2 of the second embodiment, a check box 131, a priority input field 132, an adjustment button 133, and interference information 134 are displayed in a corresponding manner in the diagram 130. The interference information 134 is information related to the interference between the surrounding objects of the target workpiece W and the robot hand 11. For example, Figure 15 As shown, if the robot hand 11 interferes with a surrounding object, "interference present" is displayed as interference information 134. If there is no interference, "no interference" is displayed as interference information 134. In the absence of interference, interference information 134 may not be displayed. Surrounding objects include the box 15 and other workpieces W other than the target workpiece W.

[0078] exist Figure 15 In the example, if the cursor is aligned to the interference information 134, Figure 16 As shown, a detail screen 135 describing the details of the interference information is popped up and displayed. Alternatively, the details of the interference information may be displayed on the reception screen 100' instead of the interference information 134 "there is interference".

[0079] exist Figure 15 If the adjustment button 133 is clicked, Figure 17 As shown, an adjustment screen 136 for accepting adjustment of the candidate holding mode is popped up. The adjustment screen 136 includes a virtual space in which the three-dimensional model of the workpiece W, the three-dimensional model of the box 15, and the three-dimensional model of the robot hand 11 are arranged based on the candidate holding mode. The user can adjust the candidate holding mode by inputting movement operations and rotation operations on the three-dimensional model of the robot hand 11, and replace the candidate holding mode with the adjusted candidate holding mode by clicking the save button 137. The user can register the adjusted candidate holding mode as an additional holding mode. As long as the candidate holding mode can be adjusted, the adjustment method is not limited to this. For example, these coordinates can be displayed so that the coordinates of the holding position and the coordinates of the holding posture corresponding to the candidate holding mode can be adjusted.

[0080] According to the control device 2 of the second embodiment, the following effects are achieved. That is, the control device 2 of the first embodiment can only prompt the user with candidates for holding modes that do not interfere with surrounding objects. Thus, the user can simply add and register holding modes that can hold the target workpiece W by simply selecting the prompted candidates for holding modes without causing the robot hand 11 to interfere with surrounding objects. On the other hand, in a case where the surrounding objects relative to the target workpiece W are complexly configured, etc., it may not be possible to automatically generate candidates for holding modes that can hold the target workpiece W. The control device 2 of the second embodiment not only displays candidates for holding modes that do not interfere with surrounding objects, but also displays candidates for holding modes that interfere with surrounding objects as selection objects for the user, and can adjust the candidates for holding modes according to user instructions. Thus, the above-mentioned situation where candidates for holding modes that can hold the target workpiece W cannot be automatically generated can be appropriately avoided.

[0081] Various data stored in the storage device 9 may be recorded in a removable medium and distributed to users, or may be downloaded to the control device 2 via the network W and distributed thereto.

[0082] The following supplementary notes are further disclosed regarding this embodiment and its modifications.

[0083] (Note 1) A control device 2 controls a robot for performing a removal operation of a bulk workpiece W, comprising: a storage unit 24 for storing data related to a plurality of holding modes for holding the workpiece W; a selection unit 29 for selecting one of the plurality of holding modes for the workpiece; a control unit 25 for controlling the robot based on the selected holding mode; a production unit 30 for producing a plurality of holding mode candidates based on the holding mode; an acceptance unit 21 for accepting a user instruction for adding at least one of the produced holding mode candidates as a holding mode; and a storage control unit 31 for storing the accepted holding mode candidate as a holding mode in the storage unit 24.

[0084] (Note 2) The creation unit 30 of Supplementary Note 1 creates a plurality of hold pattern candidates in response to a user's instruction.

[0085] (Note 3) The control device 2 described in Supplementary Note 1 further includes an acquisition unit 23 that acquires image data of the workpiece W, and a determination unit 28 that determines the holding possibility of the workpiece W by the holding pattern added to the storage unit 24 based on the image data.

[0086] (Note 4) The creation unit 30 of Supplementary Note 3 creates a plurality of candidates for the holding pattern when the holding possibility of any of the plurality of holding patterns is lower than a predetermined value.

[0087] (Note 5) In the control device 2 of Supplementary Note 1, the storage unit 24 further stores data of the three-dimensional model of the workpiece W, and the creation unit 30 creates a plurality of holding pattern candidates based on the holding pattern and the data of the three-dimensional model of the workpiece W.

[0088] (Note 6) The storage unit 24 of Supplementary Note 5 further stores data of a three-dimensional model of a hand for holding a workpiece W. The creation unit 30 creates a plurality of holding pattern candidates based on the holding pattern, the data of the three-dimensional model of the workpiece W, and the data of the three-dimensional model of the hand.

[0089] (Note 7) In the control device 2 of Supplementary Note 1, the holding pattern includes at least a holding position for holding the workpiece W, and the creating unit 30 creates holding pattern candidates within a predetermined range from the holding position of the holding pattern.

[0090] (Note 8) The accepting unit 21 in Supplementary Note 7 accepts a user's designation of a predetermined range.

[0091] (Note 9) The control device 2 of Supplementary Note 1 further includes an acquisition unit 23 that acquires image data of the workpiece W, and a determination unit 28 that determines the possibility of interference with surrounding objects of the workpiece W based on the image data for each of the plurality of holding pattern candidates.

[0092] (Note 10) The control device 2 of Note 9 also has a display unit 22, which displays an acceptance screen for accepting a user instruction to add at least one of the candidates for the prepared holding mode as a holding mode, and only displays the candidates for the holding mode that are determined to have a lower possibility of interference with surrounding objects than a specified value.

[0093] (Note 11) The control device 2 of Note 9 also has a display unit 22, which displays an acceptance screen for accepting a user instruction to add at least one of the prepared holding mode candidates as a holding mode, and among the holding mode candidates displayed on the acceptance screen, holding mode candidates that are judged to have a possibility of interference with surrounding objects that is higher than a specified value and information related to surrounding objects with a high possibility of interference are displayed together.

[0094] (Note 12) In any one of the control devices 2 according to Supplementary Notes 9 to 11, the surrounding objects include any one of the box 15 storing the workpiece W and another workpiece W different from the workpiece W.

[0095] (Note 13) In any one of the control devices 2 in Notes 1 to 12, priorities are set for multiple holding modes respectively, the receiving unit 21 receives at least one of the candidates for the prepared holding modes together with the priority, and the storage control unit 31 adds the accepted candidates for the holding modes to the multiple holding modes according to the priority.

[0096] (Note 14) The receiving unit 21 of the control device 2 in any one of Supplementary Notes 1 to 13 receives a user instruction regarding adjustment of a plurality of holding pattern candidates.

[0097] (Note 15) A program that can control a robot in order to perform a removal operation of a bulk workpiece W, the program causing a computer storing data related to multiple holding modes for holding the workpiece W to implement: a unit for creating multiple holding mode candidates based on the holding mode; a unit for accepting a user instruction for adding at least one of the created holding mode candidates as a holding mode; and a unit for storing the accepted holding mode candidates as a holding mode.

[0098] Although the embodiments of the present disclosure have been described in detail, the present disclosure is not limited to the above-mentioned embodiments. These embodiments may be subjected to various additions, replacements, changes, partial deletions, etc., without departing from the scope of the main idea of the invention or the scope of the idea and main idea of the invention derived from the contents recorded in the claims and their equivalents. For example, in the above-mentioned embodiments, the order of each action or the order of each process is shown as an example, but is not limited to this. In addition, the same applies to the case where numerical values or mathematical formulas are used in the description of the above-mentioned embodiments.

[0099] Description of Reference Numerals 1: Robot system, 2: Control device, 5: Processor, 6: Operating device, 7: Display device, 8: Communication device, 9: Storage device, 10: Robot, 11: Robot hand, 13: 3D vision camera, 15: Box, 21: Receiving unit, 22: Display unit, 23: Acquisition unit, 24: Storage unit, 25: Robot control unit, 26: Workpiece detection unit, 27: Target workpiece determination unit, 28: Judgment unit, 29: Holding mode selection unit, 30: Holding mode candidate creation unit, 31: Storage control unit, 32: Screen creation unit, 33: Adjustment unit.

Claims

1. A control device for controlling a robot to perform a bulk workpiece removal operation, characterized in that: have: a storage unit storing data related to a plurality of holding modes for holding the workpiece using an end effector of the robot; a selection unit that selects one of the plurality of holding modes for the workpiece; a control unit that controls the robot based on the one holding pattern; a preparation unit that prepares a plurality of holding pattern candidates based on the holding pattern; an accepting unit that accepts a user instruction to add at least one of the created hold pattern candidates as the hold pattern; and The storage control unit stores the accepted holding pattern candidate as the holding pattern in the storage unit.

2. The control device according to claim 1, characterized in that The creating unit creates the plurality of holding pattern candidates in response to a user's instruction.

3. The control device according to claim 1, characterized in that Also features: an acquiring unit, which acquires the imaging data of the workpiece; and The determination unit determines, based on the image data, whether the holding pattern added to the storage unit can hold the workpiece.

4. The control device according to claim 3, characterized in that The creating unit creates candidates for the plurality of holding patterns when the holding possibility is lower than a predetermined value for any one of the plurality of holding patterns.

5. The control device according to claim 1, characterized in that The storage unit also stores data of the three-dimensional model of the workpiece. The creating unit creates the plurality of holding pattern candidates based on the holding pattern and data of the three-dimensional model of the workpiece.

6. The control device according to claim 5, characterized in that The storage unit further stores data of a three-dimensional model of a hand holding the workpiece. The creating unit creates the plurality of holding pattern candidates based on the holding pattern, the data of the three-dimensional model of the workpiece, and the data of the three-dimensional model of the hand.

7. The control device according to claim 1, characterized in that The holding mode at least includes maintaining the holding position of the workpiece, The creating unit creates candidates for the holding pattern within a predetermined range from a holding position of the holding pattern.

8. The control device according to claim 7, characterized in that: The accepting unit accepts a user's designation of the predetermined range.

9. The control device according to claim 1, characterized in that Also features: an acquiring unit, which acquires the imaging data of the workpiece; and The determination unit determines, for each of the plurality of holding pattern candidates, a possibility of interference with an object surrounding the workpiece based on the image data.

10. The control device according to claim 9, characterized in that: The apparatus further comprises a display unit configured to display an acceptance screen for accepting a user instruction to add at least one of the prepared holding mode candidates as the holding mode. Only the holding pattern candidates determined to have a possibility of interference with the surrounding object lower than a predetermined value are displayed on the reception screen.

11. The control device according to claim 9, characterized in that: The apparatus further comprises a display unit configured to display an acceptance screen for accepting a user instruction to add at least one of the prepared holding mode candidates as the holding mode. Among the holding pattern candidates displayed on the reception screen, holding pattern candidates determined to have a high possibility of interference with the surrounding object equal to or higher than a predetermined value are displayed together with information on the surrounding object with a high possibility of interference.

12. The control device according to any one of claims 9 to 11, characterized in that: The surrounding objects include any one of a box storing the workpiece and other workpieces different from the workpiece.

13. The control device according to any one of claims 1 to 12, characterized in that: Prioritizing the plurality of holding modes, The receiving unit receives at least one of the prepared holding mode candidates together with the priority order. The storage control unit adds the accepted holding pattern candidates to the plurality of holding patterns in order of priority.

14. The control device according to any one of claims 1 to 13, characterized in that: The receiving unit receives a user instruction regarding adjustment of the plurality of hold pattern candidates.

15. A program capable of controlling a robot to perform a task of removing bulk workpieces, characterized in that: causing a computer having a storage device storing data related to a plurality of holding modes for holding the workpiece to implement: a unit for preparing a plurality of holding pattern candidates based on the holding pattern; means for accepting a user instruction to add at least one of the prepared holding pattern candidates as the holding pattern; as well as means for storing the accepted holding pattern candidate as the holding pattern in a storage device.

Images