Workpiece removal number calculation device, hand system, and display device

Abstract

This workpiece retrieval count calculation device comprises: a three-dimensional information acquisition unit for acquiring three-dimensional information pertaining to a subject space in which a plurality of workpieces are loaded; a workpiece shape information acquisition unit for acquiring shape information pertaining to the workpieces; and a workpiece retrieval count calculation unit for calculating a range of workpiece retrieval counts, in consideration of cavities within a workpiece loading region, on the basis of the three-dimensional information pertaining to the subject space and the shape information pertaining to the workpieces from before and after retrieval of the workpieces.

Description

[Technical field]

[0001] The present invention relates to a hand technology, and more particularly to a workpiece pick-up number calculation device, a hand system, and a display device. [Background technology]

[0002] Various types of pick-and-place hands are known, such as multi-finger gripping, magnetic suction, vacuum suction, and Bernoulli types. In applications where one or a desired number of workpieces are picked from multiple workpieces that are piled up or stacked, a visual sensor is used to detect the position (and posture, if necessary) of the workpiece. As a visual sensor, three-dimensional visual sensors using various principles such as triangulation, TOF (time of flight), and focusing are used. However, in the case of small, thin, sheet-like, or tightly adhered workpieces, the hand may mistakenly pick up multiple workpieces at once. There is a method for calculating the number of workpieces picked up by using additional sensors such as force sensors, electromagnetic induction sensors, pressure sensors, and photoelectric sensors after picking up the workpieces, but performing additional measurements with sensors separate from the visual sensor extends the cycle time. In addition, since an additional sensor is required in addition to the visual sensor, the system becomes more complex and expensive.

[0003] On the other hand, when calculating the number of workpieces taken out using a visual sensor, a method is known in which the difference (i.e., volume difference) between three-dimensional information such as distance images and three-dimensional point cloud data before and after the workpieces are taken out is divided by the volume of each workpiece to calculate the number of workpieces taken out (see Patent Document 1 described below). However, when multiple workpieces are piled up or stacked, it is not possible to know whether a workpiece or a cavity exists in an area not visible to the visual sensor, and therefore it is not possible to determine whether the difference in the three-dimensional information before and after the workpieces are taken out is a difference caused by mistakenly taking out multiple workpieces at once, or a difference caused by the exposure of a cavity part. Therefore, the number of workpieces taken out obtained by dividing the difference in the three-dimensional information before and after the workpieces are taken out by the volume of each workpiece is not necessarily an accurate value. The following documents are disclosed as prior art related to the present application.

[0004] Patent document 1 describes a method and device for detecting the collapse of a load of items that occurs when items are transferred by a picking robot, which estimates the volume difference of the load of items before and after the transfer operation from a distance image before the operation and a distance image after the operation generated based on a three-dimensional measurement means, divides the estimated volume difference of the load of items by the volume per item calculated from item shape data, compares the number of items actually transferred with the desired number to be transferred, and determines whether or not the load has collapsed after all the steps of the transfer operation have occurred based on the degree of agreement.

[0005] Patent Document 2 describes a robot manipulator equipped with a suction gripper that lifts and moves a box using a 3D sensor that detects information about the environment.

[0006] Patent Document 3 describes a method in which a predetermined number of workpieces are held at the tip of a robot arm that is randomly stacked on a pallet and removed from the pallet. The number of remaining workpieces when an area where no workpieces are present occurs at the bottom of the pallet is almost uniformly determined depending on the size of the pallet, and the number of workpieces to be removed when that remaining number is reached is calculated in advance by reverse calculation. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2007-179301 A [Patent Document 2] Special Publication No. 2017-520417 [Patent Document 3] JP 2010-300878 A Summary of the Invention [Problem to be solved by the invention]

[0008] In view of the problems in the related art, an object of the present invention is to provide a technique for accurately estimating the number of workpieces to be taken out. [Means for solving the problem]

[0009] One aspect of the present disclosure provides a work removal number calculation device comprising: a three-dimensional information acquisition unit that acquires three-dimensional information of a target space in which multiple workpieces are loaded; a work shape information acquisition unit that acquires shape information of the workpieces; and a work removal number calculation unit that calculates a range of the number of workpieces to be removed based on the three-dimensional information of the target space before and after workpiece removal and the work shape information, taking into account cavities in the work loading area. Another aspect of the present disclosure provides a hand system comprising a hand for picking up and dispensing workpieces, a transport device for transporting the workpieces using the hand, a control device for controlling the operation of at least one of the hand and the transport device, and a visual sensor for outputting three-dimensional information of a target space in which multiple workpieces are loaded, wherein the control device comprises a three-dimensional information acquisition unit for acquiring three-dimensional information of the target space from the visual sensor, a work shape information acquisition unit for acquiring shape information of the workpieces, a workpiece removal number calculation unit for calculating a range of the number of workpieces to be removed based on the three-dimensional information of the target space before and after workpiece removal and the work shape information, taking into account cavities in the work loading area, and an operation control unit for controlling the operation of at least one of the transport device and the hand based on the range of the number of workpieces to be removed. Another aspect of the present disclosure provides a display device that visually displays the process of calculating the number of workpieces to be removed from a plurality of loaded workpieces, and includes a display unit that displays a superimposed image in which a changed area in the workpiece loading area after the workpieces are removed is superimposed on at least one of the three-dimensional information of the target space before and after the workpieces are removed. Effect of the Invention

[0010] According to one aspect of the present disclosure, the range of the number of workpieces to be removed is calculated taking into account cavities in the workpiece loading area, so that the number of workpieces to be removed can be accurately estimated. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a configuration diagram of a hand system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram of a hand system according to the first embodiment. [Figure 3A] 1 is a side view of a workpiece loading area showing an example of the workpiece loading area before a workpiece is removed. FIG. [Figure 3B] 11 is a side view of the workpiece loading area showing an example of the workpiece loading area after the workpiece has been removed. FIG. [Figure 4] 4 is a flowchart showing the operation of the hand system of the first embodiment. [Diagram 5] 4 is a flowchart showing the operation of the hand system of the first embodiment. [Figure 6] 10 is a flowchart showing the operation of the hand system of the second embodiment. [Figure 7A] 11 is a side view showing an example of a workpiece loading area before a workpiece is removed. FIG. [Figure 7B] 11 is a side view showing an example of a workpiece loading area after the workpiece is removed. FIG. [Figure 8A] 13 is a side view showing another example of the workpiece loading area before the workpiece is removed. FIG. [Figure 8B] 13 is a side view showing another example of the workpiece loading area after the workpiece is removed. FIG. [Figure 9A]13 is a side view showing another example of the workpiece loading area before the workpiece is removed. FIG. [Figure 9B] 13 is a side view showing another example of the workpiece loading area after the workpiece is removed. FIG. [Figure 10A] 13 is a side view showing yet another example of the workpiece loading area before the workpiece is removed. FIG. [Figure 10B] 13 is a side view showing yet another example of the workpiece loading area after the workpiece is removed. FIG. [Figure 11] FIG. 13 is a configuration diagram of a hand system according to a third embodiment. [Figure 12] FIG. 11 is a distance image diagram showing an example of three-dimensional information after the workpiece has been removed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In each drawing, the same or similar components are given the same or similar reference numerals. In addition, the embodiments described below do not limit the technical scope and meaning of the terms of the invention described in the claims.

[0013] The hand system 1 of the first embodiment will be described below. FIG. 1 is a configuration diagram of the hand system 1 of the first embodiment. The hand system 1 includes a hand 2, a conveying device 3, a control device 4, and a visual sensor 5. The hand system 1 may further include a presence sensor 6 and a display device 7. In this embodiment, a plurality of cylindrical workpieces W are loaded (loosely piled or stacked) in the container, but in other embodiments, the workpieces W may be in other shapes such as rectangular, thin plate, or sheet, and the workpieces W may be loaded on a jig. The hand system 1 detects the position (and posture, if necessary) of the workpiece W based on the three-dimensional information of the visual sensor 5. The hand system 1 moves the conveying device 3 to a workpiece removal position based on the position (and posture, if necessary) of the workpiece W, and controls the hand 2 to operate and remove the workpiece W. When removing the workpiece W, the hand 2 may mistakenly remove a number of workpieces W different from the desired number (one in this embodiment), so the hand system 1 calculates the range of the number of workpieces to be removed based on the three-dimensional information of the visual sensor 5. The hand system 1 controls the transfer device 3 to a workpiece ejection position and the hand 2 to eject the workpiece W based on the calculated range of the number of workpieces to be taken out.

[0014] The hand 2 is attached to the transport device 3. The hand 2 takes out and delivers the workpiece W. The hand 2 includes a multi-fingered gripping hand, a magnetic suction hand, a vacuum suction hand, a Bernoulli hand, and the like. The hand 2 may be appropriately selected according to the material, shape, size, and the like of the workpiece W. For example, if the friction coefficient of the workpiece W is high, a multi-fingered gripping hand may be used, if the workpiece is a magnetic material, a magnetic suction hand may be used, if the workpiece is large or thin plate-like, a vacuum suction hand may be used, and if the workpiece is light or sheet-like, a Bernoulli hand may be used. The hand 2 of this embodiment is a multi-fingered gripping hand, and takes out and delivers the workpiece W by operating multiple fingers that can move relative to each other. The hand 2 is equipped with a motor and a motor drive device that operate the multiple fingers, and the motor drive device is connected to the control device 4 via a wire or wirelessly.

[0015] The transport device 3 transports the workpiece W using the hand 2. The transport device 3 includes a robot, a conveyor, an automated guided vehicle (AGV), and the like. The transport device 3 in this embodiment is a vertical articulated robot, but in other embodiments, it may be a horizontal articulated robot, an orthogonal robot, a parallel link robot, a humanoid, and the like. The transport device 3 in this embodiment includes a plurality of links connected to enable relative movement, and transports the workpiece W by operating the plurality of links. The transport device 3 includes a motor and a motor drive device that operate the links, and the motor drive device is connected to the control device 4 via a wire or wirelessly.

[0016] The control device 4 controls the operation of at least one of the hand 2, the transport device 3, and the visual sensor 5. In this embodiment, the control device 4 controls all of the operations of the hand 2, the transport device 3, and the visual sensor 5, but in other embodiments, the control device 4 controls only the operations of the hand 2 and the transport device 3, and an external device (not shown) connected to the control device 4 so as to be able to communicate via wire or wirelessly may control the operation of the visual sensor 5. The control device 4 is a computer device equipped with a processor, memory, and the like. The processor in this embodiment includes semiconductor integrated circuits such as a central processing unit (CPU) and a micro processing unit (MPU) that execute computer programs, but in other embodiments, it may be other semiconductor integrated circuits such as a field programmable gate array (FPGA) and an application specific integrated circuit (ASIC) that do not execute computer programs. The memory in this embodiment is a semiconductor integrated circuit such as a random access memory (RAM) and a read only memory (ROM) that store various data.

[0017] The visual sensor 5 measures and outputs three-dimensional information of a target space including a workpiece loading area where multiple workpieces W are loaded (loosely or stacked). The visual sensor 5 includes three-dimensional visual sensors using various principles such as triangulation, TOF, and focusing. Alternatively, the visual sensor 5 may be a three-dimensional visual sensor combining a distance sensor and a two-dimensional camera. The visual sensor 5 of this embodiment is a TOF sensor that outputs a distance image. The visual sensor 5 is attached to the hand 2 or the transport device 3. Although the visual sensor 5 of this embodiment is attached to the hand 2, in other embodiments, the visual sensor 5 may be installed at a fixed point in a location other than the hand 2 or the transport device 3.

[0018] It should be noted that the presence sensor 6 is not an essential component. The presence sensor 6 outputs presence information indicating whether or not one or more workpieces W have been picked up by the hand 2. The presence sensor 6 includes a force sensor, an electromagnetic induction sensor, a pressure sensor, a photoelectric sensor, a camera, a three-dimensional sensor, and the like. As will be described in the embodiment described later, the visual sensor 5 may also be used as the presence sensor 6. The presence sensor 6 in this embodiment is a force sensor that detects force. The presence sensor 6 outputs force sense information as the presence information. The presence sensor 6 is attached to the hand 2 or the transport device 3, but in other embodiments, the presence sensor 6 may be installed at a fixed point other than the hand 2 or the transport device 3.

[0019] It should be noted that the display device 7 is not an essential component. The display device 7 visually displays the calculation process of the number of workpieces to be removed. The display device 7 is provided with a display unit (not shown) configured with a liquid crystal display, an organic EL display, or a touch panel display using these. The display device 7 in this embodiment is a teaching device communicatively connected to the control device 4, but is not limited thereto. For example, in another embodiment, the display device 7 may be a display device associated with an external device communicatively connected to the control device 4 via wire or wirelessly.

[0020] 2 is a block diagram of the hand system 1 of the first embodiment. The control device 4 of this embodiment includes a workpiece take-up number calculation device 40, but in other embodiments, an external device (not shown) communicatively connected to the control device 4 via wire or wirelessly may include the workpiece take-up number calculation device 40.

[0021] The workpiece removal number calculation device 40 is a computer device equipped with a processor, memory, etc. The processor in this embodiment includes a semiconductor integrated circuit such as a central processing unit (CPU) or a micro processing unit (MPU) that executes a computer program, but in other embodiments, may be another semiconductor integrated circuit such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) that does not execute a computer program. The memory in this embodiment includes a semiconductor storage device such as a random access memory (RAM) or a read only memory (ROM) that stores various data, but in other embodiments, the memory may be a magnetic storage device such as a hard disk drive (HDD).

[0022] The workpiece removal number calculation device 40 includes a three-dimensional information acquisition unit 41, a workpiece shape information acquisition unit 42, and a workpiece removal number calculation unit 44. The workpiece removal number calculation device 40 may further include a work presence information acquisition unit 43. The workpiece removal number calculation device 40 may further include an operation control unit 45. In this specification, the term "unit" refers to a part or all of a computer program, or a part or all of a semiconductor integrated circuit that does not execute a computer program. The workpiece removal number calculation device 40 calculates the range of the number of workpieces W to be removed, taking into account cavities in the loading area of ​​the workpieces W, based on the three-dimensional information and the workpiece shape information of the target space before and after the workpieces are removed.

[0023] The three-dimensional information acquisition unit 41 acquires three-dimensional information of the target space in which multiple workpieces W are loaded (loosely piled or stacked) from the visual sensor 5. The three-dimensional information includes a distance image including a distance value from a reference position for each pixel, and three-dimensional point cloud data including coordinate values ​​in a three-dimensional coordinate system. The three-dimensional information in this embodiment is a distance image output from the TOF sensor. The three-dimensional information acquisition unit 41 may have a function for converting data between the distance image and other data formats such as three-dimensional point cloud data. The three-dimensional information acquisition unit 41 sends the three-dimensional information of the target space to the workpiece removal number calculation unit 44.

[0024] The workpiece shape information acquisition unit 42 acquires workpiece shape information from an external memory or an internal memory. The workpiece shape information includes a three-dimensional shape model, a distance image, three-dimensional point cloud data, etc. The workpiece shape information in this embodiment is a three-dimensional shape model such as three-dimensional CAD (computer-aided design) data. The workpiece shape information acquisition unit 42 may have a function for converting data between the three-dimensional shape model and other data formats such as a distance image, three-dimensional point cloud data, etc. The workpiece shape information acquisition unit 42 sends the workpiece shape information to the workpiece removal number calculation unit 44.

[0025] In addition, it is preferable that at least one of the three-dimensional information acquisition unit 41 and the work shape information acquisition unit 42 has a function of aligning parameters such as size, scale, coordinate system, etc. of one of the three-dimensional information of the target space and the work shape information with parameters such as size, scale, coordinate system, etc. of the other for subsequent processing.

[0026] It should be noted that the attendance information acquisition unit 43 is not an essential component. The attendance information acquisition unit 43 acquires attendance information indicating whether or not one or more workpieces W have been taken out by the hand 2 from the attendance sensor 6. The attendance information in this embodiment is force sense information from a force sensor, and the attendance information acquisition unit 43 sends attendance information indicating that one or more workpieces W have been taken out to the workpiece take-out number calculation unit 44 when the force sense information exceeds the gravity per workpiece W. In other embodiments, the number of workpieces taken out estimated by dividing the force sense information by the gravity per workpiece W may be sent to the workpiece take-out number calculation unit 44 as attendance information.

[0027] Although not essential, the workpiece removal number calculation unit 44 executes a calculation process of the range of the number of workpieces to be removed when the attendance information indicates that one or more workpieces W have been removed, and does not execute a calculation process of the range of the number of workpieces to be removed when the attendance information indicates that no workpieces W have been removed, and sends the number of workpieces to be removed as 0 to the operation control unit 45. This makes it possible to prevent unnecessary calculation processes by the workpiece removal number calculation unit 44. Furthermore, when the attendance information includes the number of workpieces to be removed, the workpiece removal number calculation unit 44 may check the calculated number of workpieces to be removed based on the number of workpieces removed in the attendance information when the range of the calculated number of workpieces to be removed becomes one value (the minimum value and the maximum value are the same value) and the desired number of workpieces to be removed (which may be multiple).

[0028] The workpiece removal number calculation unit 44 calculates the range of the number of workpieces to be removed, taking into consideration the cavity in the workpiece loading area, based on the three-dimensional information of the target space before and after the workpiece removal and the workpiece shape information. Here, the calculation process of the number of workpieces to be removed in the first embodiment will be described with reference to Figs. 3A and 3B. Figs. 3A and 3B are side views showing an example of the workpiece loading area before and after the workpiece removal. Fig. 3A shows the workpiece loading area before the workpiece removal, in which all rectangular workpieces W having short and long sides are turned sideways and stacked vertically. Fig. 3B shows the workpiece loading area after the workpiece removal, in which none of the workpieces W are removed and two workpieces W are collapsed. The visual sensor 5 captures the workpiece loading area from above and outputs three-dimensional information 51 (shown by a dashed line) before the workpiece removal and three-dimensional information 52 (shown by a dashed line) after the workpiece removal.

[0029] In the invisible area in the work loading area that cannot be seen by the visual sensor 5, it is not possible to determine whether a work W exists or whether a cavity exists. Therefore, as a basic concept, the number of workpieces to be taken out can be accurately estimated by calculating the range of the number of workpieces to be taken out based on the existence of a cavity in the disappearance area A (shown by the thick dashed line) that disappeared from the work loading area after the workpieces were taken out. In other words, the workpiece take-out number calculation unit 44 calculates the range of the number of disappeared workpieces that can be accommodated in the disappearance area A (1 to 2 in the example of FIG. 3A) as the range of the number of workpieces to be taken out (1 to 2).

[0030] The disappearance area A is obtained as a difference area (positive area or negative area) obtained by subtracting the three-dimensional information 51, 52 of the target space before and after the workpieces are removed. The maximum number of disappeared workpieces that can be accommodated in the disappearance area A (2 in the example of FIG. 3A) is obtained by repeatedly subtracting the workpieces W obtained from the workpiece shape information from the disappearance area A, or by dividing the volume of the disappearance area A by the volume per workpiece W obtained from the workpiece shape information. The minimum number of disappeared workpieces that can be accommodated in the disappearance area A (1 in the example of FIG. 3A) is obtained as 1 when the maximum number of disappeared workpieces is 1 or more, and as 0 when the maximum number of disappeared workpieces is 0.

[0031] However, as shown in Fig. 3B, the workpieces W may collapse after removal due to the loading method of the workpieces W. In addition, since it is not possible to determine whether a workpiece exists or a cavity exists in the invisible area in the workpiece loading area that is not visible to the visual sensor 5, the number of workpieces to be removed can be more accurately estimated by calculating the range of the number of workpieces to be removed while also taking into account the cavity in the collapsed load area B (shown by the thick dashed line) that newly appears in the workpiece loading area after the workpieces are removed. In other words, the workpiece removal number calculation unit 44 may calculate the range of the number of workpieces to be removed while taking into account the range of the number of collapsed workpieces that can be accommodated in the collapsed load area B that newly appears in the workpiece loading area after the workpieces are removed (1 to 2 in the example of Fig. 3A).

[0032] The load collapse area B is obtained as a difference area (negative area or positive area) obtained by subtracting the three-dimensional information 51, 52 of the target space before and after the workpieces are removed. The maximum number of collapsed workpieces that can be accommodated in the load collapse area B (2 in the example of FIG. 3B) is obtained by repeatedly subtracting the workpieces W obtained from the workpiece shape information from the load collapse area B, or by dividing the volume of the load collapse area B by the volume of one workpiece W obtained from the workpiece shape information. The minimum number of collapsed workpieces that can be accommodated in the load collapse area B (1 in the example of FIG. 3B) is obtained as 1 when the maximum number of collapsed workpieces is 1 or more, or as 0 when the maximum number of collapsed workpieces is 0.

[0033] Next, the workpiece removal number calculation unit 44 subtracts the minimum number of workpiece collapse numbers (1 in the example of FIG. 3B) from the maximum number of workpiece disappearance numbers (2 in the example of FIG. 3A) to obtain the maximum number of workpiece removal numbers (1), and subtracts the maximum number of workpiece collapse numbers (2 in the example of FIG. 3B) from the minimum number of workpiece disappearance numbers (1 in the example of FIG. 3A) to obtain the minimum number of workpiece removal numbers (0 in the case of a negative number, since it is impossible to remove -1 workpiece W), thereby calculating the range of the number of workpiece removal numbers (0 to 1). Therefore, the calculation formula for the range R of the number of workpiece removal numbers is as follows.

[0034]

number

[0035] In formula 1, J+H is the maximum number of workpieces that disappeared, J is the minimum number of workpieces that disappeared, K+I is the maximum number of workpieces that collapse, and K is the minimum number of workpieces that collapse. max(J-(K+I),0) is the minimum number of workpieces that can be removed, and max((J+H)-K,0) is the maximum number of workpieces that can be removed. Note that the max(argument 1, argument 2) function returns the larger of argument 1 and argument 2. In other words, when the max function calculates the maximum or minimum number of workpieces that can be removed, it returns 0 if the number is negative.

[0036] In addition, when a loading method that does not have the possibility of the workpieces W collapsing is adopted, there is no need to specify the collapse area B. In this case, the workpiece removal number calculation unit 44 sets the minimum number K of the collapsed workpieces to 0, sets the maximum number K+I of the collapsed workpieces to 0, and calculates the range of the number of workpieces to be removed (1 to 2) based only on the range J to J+H of the number of vanishing workpieces that can be accommodated in the vanishing area A (1 to 2 in the example of FIG. 3A).

[0037] In addition, when a loading method is adopted in which there are no cavities in the work loading area, the minimum number J of workpieces that disappear is set to the maximum number J of workpieces that disappear + H (2 in the example of Figure 3A), the minimum number K of workpieces that collapse is set to the maximum number K of workpieces that collapse + I (2 in the example of Figure 3B), and the range of the number of workpieces to be removed (0 to 0) can be calculated.

[0038] Furthermore, when the workpiece take-out number calculation unit 44 determines whether or not to execute the calculation process for the range of the number of workpieces to be taken out based on the attendance information acquired from the attendance information acquisition unit 43, since it is known that the number of workpieces to be taken out is always 1 or more, it is preferable to calculate the maximum or minimum value of the number of workpieces to be taken out and modify the max function to return 1 when the number is 0 or less. In this case, the calculation formula for the range R of the number of workpieces to be taken out is as follows.

[0039]

number

[0040] Referring again to FIG. 2, the workpiece removal number calculation unit 44, although not essential, sends to the display device 7 a superimposed image in which a change area Q (including disappearance area A and possibly load collapse area B) in the loading area of ​​the workpieces W after the workpieces are removed is superimposed on at least one of the three-dimensional information of the target space before and after the workpieces are removed, and the display unit of the display device 7 may display the superimposed image sent from the workpiece removal number calculation unit 44. For example, the superimposed image may be a side view of the loading area of ​​the workpieces as shown in FIG. 3A and FIG. 3B. The display unit may distinguish and display the change area Q, disappearance area A, and possibly load collapse area B, for example, by color coding, different types of encircling lines, or display switching. This visualizes the calculation process of the number of workpieces to be removed, and it is possible to check whether there is a calculation error in the number of workpieces to be removed.

[0041] Further, although not essential, the workpiece removal number calculation unit 44 sends at least one of the range J to J+H of the number of workpieces lost, the range K to K+I of the number of workpieces collapsed in some cases, the range R of the number of workpieces taken out, and the number of workpieces taken out when the maximum value and the minimum value of the range R of the number of workpieces taken out match to the display device 7, and the display unit of the display device 7 may display at least one of the range J to J+H of the number of workpieces lost, the range K to K+I of the number of workpieces collapsed in some cases, the range R of the number of workpieces taken out, and the number of workpieces taken out when the maximum value and the minimum value of the range R of the number of workpieces taken out match, sent from the workpiece removal number calculation unit 44. Further, although not essential, the workpiece removal number calculation unit 44 sends to the display device 7 a superimposed image in which the shape model of the workpiece W is superimposed on the places (places indicated by J, H, K, and I) that are the basis for calculating these number ranges, and the display unit of the display device 7 may display the superimposed image sent from the workpiece removal number calculation unit 44. This makes it possible to visualize the process of calculating the number of workpieces to be taken out, making it possible to check whether there have been any calculation errors in the number of workpieces to be taken out.

[0042] The workpiece take-out number calculation unit 44 sends the calculated range R of the number of workpieces to be taken out to the operation control unit 45. The operation control unit 45 controls the operation of at least one of the hand 2 and the transport device 3 in accordance with the operation program of at least one of the hand 2 and the transport device 3. The operation control unit 45 controls (corrects) the operation of at least one of the hand 2 and the transport device 3 based on the range R of the number of workpieces to be taken out. In other words, the operation control unit 45 commands the hand 2 and the transport device 3 to repeat the operation of taking out and dispensing the workpieces W until the range R of the number of workpieces to be taken out becomes a single value (the minimum value and the maximum value are the same value) and the desired number of workpieces is taken out. If the range R of the number of workpieces to be taken out is not a single value or the desired number of workpieces to be taken out is not reached, the operation control unit 45 instructs the hand 2 and the conveying device 3 to eject the workpiece W to its original location or to a temporary placement table and to remove the workpiece W again; if the range R of the number of workpieces to be taken out is a single value and the desired number of workpieces to be taken out is reached, the operation control unit 45 instructs the hand 2 and the conveying device 3 to transport the workpiece W to the ejection position and eject it.

[0043] 4 and 5 are flowcharts showing the operation of the hand system 1 of the first embodiment. In step S1, it is determined whether or not three-dimensional information after the previous workpiece removal is in memory. If three-dimensional information after the previous workpiece removal is not in memory (NO in step S1), three-dimensional information 51 of the target space before the workpiece removal is acquired from the visual sensor 5 in step S2. If three-dimensional information after the previous workpiece removal is in memory (YES in step S1), in step S3, the three-dimensional information after the previous workpiece removal is used as three-dimensional information 51 before the workpiece removal.

[0044] In step S4, it is determined whether the position (and posture, if necessary) of the workpiece W before the workpiece is removed is in the memory. If the position (and posture, if necessary) of the workpiece W before the workpiece is removed is not in the memory (NO in step S4), in step S5 the position (and posture, if necessary) of the workpiece W is detected from the three-dimensional information 51 before the workpiece is removed. If the position (and posture, if necessary) of the workpiece W in the three-dimensional information 51 before the workpiece is removed is in the memory (YES in step S4), proceed to step S6.

[0045] In step S6, the transport device 3 is moved to the workpiece removal position based on the position (and posture, if necessary) of the workpiece W, and the hand 2 is operated to remove the workpiece W. In step S7, it is determined whether one or more workpieces W have been removed based on the presence information of the presence sensor 6. However, it should be noted that step S7, which determines whether or not to perform a calculation process for the range of the number of workpieces to be removed based on the presence information, is not a required step. If no workpieces W have been removed (NO in step S7), the process returns to step S1 and a workpiece W is removed again. If one or more workpieces W have been removed (YES in step S7), the process proceeds to step S8. In step S8, three-dimensional information 52 after the workpieces have been removed is acquired from the visual sensor 5.

[0046] In step S9, a change area Q (including disappearance area A and possibly load collapse area B) that has changed in the work loading area after the workpiece is removed is identified by subtracting the three-dimensional information 51, 52 before and after the workpiece is removed. The change area Q is an area that is not 0 after the subtraction. Unnecessary calculation processing can be prevented by limiting the subsequent processing of step S9 to the change area Q. The change area Q may be replaced with a circumscribed rectangle, circumscribed circle, etc. that circumscribes the change area Q, or may be replaced with a circumscribed rectangle, circumscribed circle, etc. that corresponds to the shape of the workpiece W. The change area Q may also be limited by using an area of ​​a predetermined radius from the workpiece removal position.

[0047] If the positive area after subtracting the three-dimensional information 51, 52 before and after the workpiece is taken out is the disappearance area A, the negative area becomes the load collapse area B. On the other hand, if the negative area after subtracting the three-dimensional information 51, 52 before and after the workpiece is taken out is the disappearance area A, the positive area becomes the load collapse area B. However, if a loading method is adopted in which there is no possibility of the workpiece W collapsing, it is not necessary to identify the load collapse area B. Note that the change area Q, disappearance area A, and possibly the load collapse area B identified in step S9 may be displayed by, for example, color coding, different encircling lines, or display switching. This visualizes the calculation process of the number of workpieces to be taken out, and it is possible to check whether there is a calculation error in the number of workpieces to be taken out. In addition, as a subsequent process of step S9, it is preferable to detect the position (and posture, if necessary) of the workpiece W after the workpiece is taken out in the change area Q and store it in memory for the next workpiece takeout.

[0048] In step S10, the range J to J+H of the number of workpieces that can be removed and accommodated in the removal area A and the range K to K+I of the number of workpieces that can be removed and accommodated in the load collapse area B are calculated based on the workpiece shape information. The maximum number J+H of workpieces that can be removed is calculated by repeatedly subtracting the workpieces W obtained from the workpiece shape information from the removal area A, or by dividing the volume of the removal area A by the volume per workpiece W obtained from the workpiece shape information. The minimum number J of workpieces that can be removed is calculated as 1 when the maximum number J+H of workpieces that can be removed is 1 or more, and as 0 when the maximum number J+H of workpieces that can be removed is 0. The maximum number K+I of workpieces that can be removed is calculated by repeatedly subtracting the workpieces W obtained from the workpiece shape information from the load collapse area B, or by dividing the volume of the load collapse area B by the volume per workpiece W obtained from the workpiece shape information. The minimum number K of the number of collapsed workpieces is calculated as 1 when the maximum number K+I of the number of collapsed workpieces is 1 or more, and as 0 when the maximum number K+I of the number of collapsed workpieces is 0. In step S10, the calculated range J to J+H of the number of vanished workpieces and, in some cases, the calculated range K to K+I of the number of collapsed workpieces may be displayed on the display device 7. As shown in Figs. 3A and 3B, a superimposed image in which the shape model of the workpiece W is superimposed on the locations (locations indicated by J, H, K, and I) that are the basis for calculating these number ranges may be displayed on the display device 7. This visualizes the calculation process of the number of workpieces to be removed, making it possible to check for any calculation errors.

[0049] In step S11, the range R(max(J-(K+I),1) to max((J+H)-K,1)) of the number of workpieces to be removed is calculated based on the range of the number of workpieces lost and, in some cases, the range of the number of workpieces that have collapsed. That is, the maximum number of workpieces to be removed is obtained by subtracting the minimum number K of workpieces that have collapsed from the maximum number J+H of workpieces lost, and the minimum number of workpieces to be removed is obtained by subtracting the maximum number K+I of workpieces that have collapsed from the minimum number J of workpieces lost, thereby calculating the range R of the number of workpieces to be removed. However, if a loading method is adopted in which there is no possibility of workpieces W collapsing, the minimum number K of workpieces that have collapsed is set to 0, the maximum number K+I of workpieces that have collapsed is set to 0, and the range of the number of workpieces to be removed is calculated based only on the range of the number of workpieces lost. Furthermore, when it is not determined whether one or more workpieces W have been taken out based on the attendance information (when the process of step S7 is not performed), a range R of the number of taken workpieces of 0 or more is calculated based on max(J-(K+I),0) to max((J+H)-K,0)) in formula 1. Note that in step S11, the calculated range R of the number of taken workpieces may be displayed on the display device 7.

[0050] In step S12, the operation of at least one of the hand 2 and the transport device 3 is controlled based on the range R of the number of workpieces to be taken out. The hand 2 and the transport device 3 are commanded to repeat the operation of taking out and discharging the workpieces W until the range R of the number of workpieces to be taken out becomes a single value (the minimum value and the maximum value are the same value) and the desired number of workpieces is taken out. In other words, if the range R of the number of workpieces to be taken out does not become a single value or the desired number of workpieces is not taken out, the workpieces W are discharged to their original location or to the temporary placement table, and the process returns to step S1 and the workpieces W are taken out again (step S6). If the range R of the number of workpieces to be taken out becomes a single value and the desired number of workpieces is taken out, the workpieces W are transported to the discharge position and discharged.

[0051] According to the hand system 1 of the first embodiment, the range of the number of workpieces to be taken out is calculated taking into account cavities in the workpiece loading area (disappearance area A and possibly load collapse area B), so the number of workpieces to be taken out R can be accurately estimated.

[0052] A hand system 1 according to a second embodiment will be described. The hand system 1 according to the second embodiment differs from the hand system 1 according to the first embodiment in that the number of workpieces to be picked up is calculated more precisely. Please note that the following description will omit descriptions of the same configurations and operations as those of the hand system 1 according to the first embodiment.

[0053] Here, the calculation process for the number of workpieces to be taken out in the second embodiment will be described with reference to Fig. 6, Fig. 7A, and Fig. 7B. Fig. 6 is a flowchart showing the operation of the hand system 1 in the second embodiment, and Fig. 7A and Fig. 7B are side views showing an example of the workpiece loading area before and after the workpieces are taken out. The flowchart in Fig. 6 includes steps S20 to S24 for more precisely calculating the process of step S10 in Fig. 4. Figs. 7A and 7B show the same workpiece loading area as Figs. 3A and 3B.

[0054] In step S20, in the change area Q (including the disappearance area A and possibly the load collapse area B), invisible three-dimensional information 53, 54 (shown by dashed lines) is generated by removing one work W from the visible area seen by the visual sensor 5 of the three-dimensional information 51, 52 before and after removal.

[0055] In step S21, in the change area Q, invisible areas (positive areas) in the disappearance area A are identified by subtracting the invisible three-dimensional information 52 after removal from the invisible three-dimensional information 53 before removal, and the number of invisible disappearance workpieces H that can be accommodated in the invisible areas in the disappearance area A is calculated. The number of invisible disappearance workpieces H is found by repeatedly subtracting the workpieces W obtained from the work shape information from the invisible areas in the disappearance area A, or by dividing the volume of the invisible areas in the disappearance area A by the volume per workpiece W obtained from the work shape information.

[0056] In step S22, in the change area Q, the three-dimensional information 51 before removal is subtracted from the invisible three-dimensional information 54 after removal to identify invisible areas (positive areas) in the load collapse area B, and calculate the number I of invisible workpieces due to load collapse that can be accommodated in the invisible areas in the load collapse area B. The number I of invisible workpieces due to load collapse is found by repeatedly subtracting the workpieces W obtained from the work shape information from the invisible areas in the load collapse area B, or by dividing the volume of the invisible areas in the load collapse area B by the volume of one workpiece W obtained from the work shape information. However, if a loading method is adopted in which there is no possibility of the workpieces W collapsing, step S22 is not necessary.

[0057] In step S23, in the change area Q, the visible area (positive area) in the disappearance area A is identified by subtracting the invisible three-dimensional information 53 before removal from the three-dimensional information 51 before removal, and the number of visible disappearance workpieces J that can be accommodated in the visible area in the disappearance area A is calculated. The number of visible disappearance workpieces J is found by repeatedly subtracting the workpieces W obtained from the work shape information from the visible area in the disappearance area A, or by dividing the volume of the visible area in the disappearance area A by the volume of one workpiece W obtained from the work shape information.

[0058] In step S24, in the change area Q, the invisible three-dimensional information 53 before removal and the invisible three-dimensional information 54 after removal are subtracted from the three-dimensional information 52 after removal to identify the visible area (positive area) in the load collapse area B, and the number K of visible workpieces of load collapse that can be accommodated in the visible area in the load collapse area B is calculated. However, if a loading method is adopted in which there is no possibility of the workpieces W collapsing, step S24 for calculating the number K of visible workpieces of load collapse is unnecessary. The number K of visible workpieces of load collapse can be found by repeatedly subtracting the workpieces W obtained from the work shape information from the visible area in the load collapse area B, or by dividing the volume of the visible area in the load collapse area B by the volume of each workpiece W obtained from the work shape information.

[0059] In step S11, the range R (max(J-(K+I),1) to max((J+H)-K,1)) of the number of workpieces to be taken out is calculated based on the range of the number of workpieces disappeared (J to J+H) and, in some cases, the range of the number of workpieces that have collapsed (K to K+I). That is, the maximum number of workpieces to be taken out is obtained by subtracting the minimum number K of the number of workpieces that have collapsed from the maximum number J+H of the number of workpieces disappeared, and the minimum number of workpieces to be taken out is obtained by subtracting the maximum number K+I of the number of workpieces that have collapsed from the minimum number J of the number of workpieces disappeared, thereby calculating the range R of the number of workpieces to be taken out. However, if a loading method is adopted in which there is no possibility of the workpieces W collapsing, the minimum number K of the number of workpieces that have collapsed is set to 0, the maximum number K+I of the number of workpieces that have collapsed is set to 0, and the range of the number of workpieces to be taken out is calculated based only on the range of the number of workpieces disappeared. In addition, if it is not determined whether one or more workpieces W have been taken out based on the attendance information (if step S7 is not performed), the range R of the number of workpieces taken out that is 0 or more is calculated based on max(J-(K+I),0) to max((J+H)-K,0)) in equation 1.

[0060] According to the second embodiment of the hand system 1, by calculating the number H of invisible disappearance workpieces that can be accommodated in the invisible area within the disappearance area A and, in some cases, the number I of invisible collapsed workpieces that can be accommodated in the invisible area within the load collapse area B, the range R of the number of workpieces to be removed can be calculated more precisely taking into account cavities in the work loading area (disappearance area A and, in some cases, load collapse area B), thereby making it possible to more accurately estimate the number R of workpieces to be removed.

[0061] Other examples of the workpiece loading area will be described below. Figures 8A and 8B are side views showing other examples of the workpiece loading area before and after the workpieces are removed. Figure 8A shows the workpiece loading area before the workpieces are removed, in which three rectangular workpieces W having short and long sides are stacked vertically, with one workpiece W in the middle being vertical and the remaining two workpieces W being laid horizontally, including a cavity for two workpieces W. Figure 8B shows the workpiece loading area after the workpieces are removed, in which one workpiece W has been removed and none of the workpieces W has collapsed. The visual sensor 5 captures an image of the workpiece loading area from above, and outputs three-dimensional information 51 (shown by dashed lines) before the workpieces are removed and three-dimensional information 52 (shown by dashed lines) after the workpieces are removed.

[0062] According to the calculation process for the number of workpieces to be removed in the first embodiment (see step S10), the range J to J+H of the number of workpieces that can be removed and stored in the removal area A is 1 to 3. Since the load collapse area B does not exist, the range K to K+I of the number of workpieces that will collapse is 0. Therefore, the range R of the number of workpieces to be removed (max(J-(K+I),1) to max((J+H)-K,1)) is 1 to 3. Since the range R of the number of workpieces to be removed is not a single value (the minimum and maximum values ​​are the same value) or is not the desired number (1 in this embodiment, but it may be multiple), the workpiece W is ejected to its original location or to the temporary placement table, and the workpiece W is removed again.

[0063] According to the calculation process of the number of workpieces to be taken out in the second embodiment (see steps S21 to S24), the invisible disappearance number H of workpieces that can be accommodated in the invisible area in the disappearance area A is 2. Since the load collapse area B does not exist, the invisible collapse number I of workpieces that can be accommodated in the invisible area in the load collapse area B is 0. The visible disappearance number J of workpieces that can be accommodated in the visible area in the disappearance area A is 1. Since the load collapse area B does not exist, the visible collapse number K of workpieces that can be accommodated in the visible area in the load collapse area B is 0. Therefore, the range R (max(J-(K+I),1) to max((J+H)-K,1)) of the number of workpieces to be taken out is 1 to 3. Since the range R of the number of workpieces to be taken out is not a single value (the minimum value and the maximum value are the same value) or is not the desired number to be taken out (1 in this embodiment), the workpiece W is discharged to the original location or the temporary placement table, and the workpiece W is taken out again.

[0064] Another example of the work loading area will be described. Figures 9A and 9B are side views showing another example of the work loading area before and after the work is taken out. Figure 9A shows the work loading area before the work is taken out, in which three rectangular workpieces W having short and long sides are piled up horizontally, two workpieces W are tilted sideways, and the remaining workpiece W on the left side is leaning against the middle workpiece W. Figure 9B shows the work loading area after the workpiece is taken out, in which the middle workpiece W is taken out and the leftmost workpiece W collapses. The visual sensor 5 captures an image of the work loading area from above, and outputs three-dimensional information 51 (shown by a dashed line) before the workpiece is taken out and three-dimensional information 52 (shown by a dashed line) after the workpiece is taken out.

[0065] According to the calculation process for the number of workpieces to be taken out in the first embodiment (see step S10), the range J to J+H of the number of workpieces that can be lost and accommodated in the loss area A is 2 to 2. The range K to K+I of the number of workpieces that can be collapsed and accommodated in the collapse area B is 1 to 1. Therefore, the range R of the number of workpieces to be taken out (max(J-(K+I),1) to max((J+H)-K,1)) is 1 to 1. Since the range R of the number of workpieces to be taken out is a single value (the minimum and maximum values ​​are the same value) and is the desired number to be taken out (1 in this embodiment), the workpiece W will be transported to the removal position.

[0066] According to the calculation process of the number of workpieces to be taken out in the second embodiment (see steps S21 to S24), since there is no invisible area in the disappearance area A, the invisible number H of workpieces that can be accommodated in the invisible area in the disappearance area A is 0. Since there is no invisible area in the load collapse area B, the invisible number I of workpieces that can be accommodated in the invisible area in the load collapse area B is 0. The visible number J of workpieces that can be accommodated in the visible area in the disappearance area A is 2. The visible number K of workpieces that can be accommodated in the visible area in the load collapse area B is 1. Therefore, the range R of the number of workpieces to be taken out (max(J-(K+I),1) to max((J+H)-K,1)) is 1 to 1. Since the range R of the number of workpieces to be taken out is one value (the minimum value and the maximum value are the same value) and is the desired number to be taken out (1 in this embodiment), the workpiece W is transported to the discharge position.

[0067] Another example of the workpiece loading area will be described. Figures 10A and 10B are side views showing another example of the workpiece loading area before and after the workpieces are removed. Figure 10A shows the workpiece loading area before the workpieces are removed, in which two rectangular workpieces W having short and long sides are laid sideways, and one workpiece W on the left side is caught on one workpiece W on the right side and floats up by one workpiece W. Figure 10B shows the workpiece loading area after the workpieces are removed, in which one workpiece W on the right side is removed and the workpiece W on the left side has fallen as if the ground has subsided. The visual sensor 5 captures an image of the workpiece loading area from above, and outputs three-dimensional information 51 (shown by dashed lines) before the workpieces are removed and three-dimensional information 52 (shown by dashed lines) after the workpieces are removed.

[0068] In the calculation process of the number of workpieces to be taken out in the first embodiment (see step S10), the range J to J+H of the number of workpieces that can be accommodated in the disappearance area A is 2 to 2. On the other hand, even if the three-dimensional information 51 before the workpieces are taken out and the three-dimensional information 52 after the workpieces are taken out are subtracted, the load collapse area B (negative area or positive area) cannot be specified, so even if the load collapse area B actually exists, the range K to K+I of the number of workpieces that can be accommodated in the load collapse area B is erroneously 0 to 0 (originally 1 to 1). Therefore, the range R (max (J-(K+I), 1) to max ((J+H)-K, 1)) of the number of workpieces to be taken out is erroneously 2 to 2, and becomes one value (the minimum value and the maximum value are the same value). However, since the number of workpieces to be taken out (2 pieces) is not the desired number to be taken out (1 piece in this embodiment), the workpieces W are delivered to the original location or the temporary placement table, and the workpieces W are taken out again, so that the malfunction of the hand system 1 can be prevented. However, when the desired number of workpieces to be taken out is two, there is a possibility that only one workpiece W will be mistakenly transported to the delivery position. However, when the attendance information includes the number of workpieces to be taken out (one), the calculated number of workpieces to be taken out (two pieces) can be checked based on the number of workpieces to be taken out (one piece) in the attendance information, thereby preventing malfunction of the hand system 1.

[0069] In the calculation process for the number of workpieces to be removed in the second embodiment (see steps S21 to S24), since there is no invisible area in the disappearance area A, the invisible number H of workpieces that can be accommodated in the invisible area in the disappearance area A is 0. Since there is no invisible area in the load collapse area B, the invisible number I of workpieces that can be accommodated in the invisible area in the load collapse area B is 0. The visible number J of workpieces that can be accommodated in the visible area in the disappearance area A is 2. However, since the visible area (positive area) in the load collapse area B cannot be identified even if the load collapse area B actually exists, the visible number K of workpieces that can be accommodated in the visible area in the load collapse area B is erroneously 0, even if the load collapse area B actually exists. Therefore, the range R(max(J-(K+I),1) to max((J+H)-K,1)) of the number of workpieces to be taken out will erroneously be 2 to 2, resulting in one value (the minimum and maximum values ​​being the same). However, since the number of workpieces to be taken out (2) is not the desired number to be taken out (1 in this embodiment), the workpiece W will be taken out to its original location or to the temporary placement table, and the workpiece W will be taken out again, thereby preventing malfunction of the hand system 1. However, when the desired number of workpieces to be taken out is 2, there is a possibility that only one workpiece W will be mistakenly transported to the take-out position. However, when the attendance information includes the number of workpieces to be taken out (1), the calculated number of workpieces to be taken out (2) can be checked based on the number of workpieces to be taken out (1) in the attendance information, thereby preventing malfunction of the hand system 1.

[0070] As shown in Fig. 10A and Fig. 10B, when the workpiece W falls straight down so that only one workpiece sinks to the ground, it is not possible to determine whether the workpiece W has been removed and a disappearance area A has been generated, or whether the workpiece W has simply sunk to the ground. Therefore, there is a possibility that the range R of the number of workpieces to be removed will be calculated incorrectly. However, by combining this with other methods, such as determining whether the range R of the number of workpieces to be removed is the desired number to be removed (one in this embodiment), or checking the range R of the number of workpieces to be removed based on the attendance information, it is considered that the possibility of causing a malfunction of the hand system 1 is reduced. Note that, if two or more workpieces W fall, it is preferable that the workpiece removal number calculation unit 44 outputs an error.

[0071] A hand system 1 of the third embodiment will be described. The hand system 1 of the third embodiment differs from the hand system 1 of the first embodiment in that the visual sensor 5 is used as a presence sensor and does not include the presence sensor 6. The visual sensor 5 measures and outputs three-dimensional information of a target space including a workpiece loading area where a plurality of workpieces W are loaded (loosely or stacked) and the hand 2 that picks up the workpieces W. In this embodiment, the visual sensor 5 is installed at a fixed point other than the hand 2 and the transport device 3, so that the visual sensor 5 outputs three-dimensional information of the target space including the hand 2 that picks up the workpieces W. In other embodiments, when the visual sensor 5 outputs three-dimensional information of the target space including the hand 2, the visual sensor 5 may be attached to the hand 2 or the transport device 3 as in the first embodiment. Please note that the same configuration and operation as those of the hand system 1 of the first embodiment will not be described below.

[0072] 2 again, when the operation control unit 45 moves the transport device 3 to the standby position after the workpieces are removed, the presence information acquisition unit 43 acquires three-dimensional information 52 after the workpieces are removed from the visual sensor 5. The presence information acquisition unit 43 generates presence information indicating whether or not one or more workpieces W have been removed by the hand 2 based on the three-dimensional information 52 after the workpieces are removed, and sends the generated presence information to the workpiece removal number calculation unit 44.

[0073] FIG. 12 is a distance image diagram showing an example of the three-dimensional information 52 after the workpiece is taken out. In the present embodiment, the attendance information acquisition unit 43 preliminarily sets the attendance confirmation area 52a including the hand 2 in the three-dimensional information 52 after the workpiece is taken out. The attendance information acquisition unit 43 detects the workpiece W from the attendance confirmation area 52a using image processing such as matching processing, and generates attendance information indicating whether one or more workpieces W have been taken out by the hand 2. If the attendance information indicates that one or more workpieces W have been taken out, the workpiece take-out number calculation unit 44 executes a calculation process for the range of the number of workpieces to be taken out. If the attendance information indicates that no workpieces W have been taken out, the workpiece take-out number calculation unit 44 does not execute a calculation process for the range of the number of workpieces to be taken out, and sends the number of workpieces taken out as 0 to the operation control unit 45. This prevents unnecessary calculation processes by the workpiece take-out number calculation unit 44.

[0074] Furthermore, the attendance information acquisition unit 43 may use image processing to calculate the number of workpieces to be taken out from the attendance confirmation area 52a. When the attendance information includes the number of workpieces to be taken out, the workpiece take-out number calculation unit 44 may check the calculated number of workpieces to be taken out based on the number of workpieces to be taken out in the attendance information when the range of the calculated number of workpieces to be taken out becomes one value (the minimum value and the maximum value are the same value) and becomes the desired number of workpieces to be taken out (1 in this embodiment).

[0075] According to the hand system 1 of the third embodiment, since the visual sensor 5 is used as the presence sensor, it is not necessary to prepare a presence sensor separate from the visual sensor 5, and it is possible to provide a low-cost hand system 1. In addition, it is possible to prevent unnecessary calculation processing by the workpiece removal number calculation unit 44.

[0076] The above-mentioned computer program may be provided by being recorded on a computer-readable non-transitory recording medium, such as a CD-ROM, or may be provided by being distributed from a server device on a WAN (wide area network) or LAN (local area network) via wired or wireless connections.

[0077] Although various embodiments have been described herein, it should be appreciated that the present invention is not limited to the above-described embodiments, but may be modified in various ways within the scope of the appended claims. [Explanation of symbols]

[0078] 1 Hand System 2 Hands 3. Conveyor 4. Control device 5. Visual Sensor 6 Presence sensor 7 Display device 40 Workpiece removal number calculation device 41 3D information acquisition section 42 Work shape information acquisition unit 43 Enrollment Information Acquisition Department 44 Workpiece removal number calculation section 45 Motion control section 51 3D information before removing the workpiece 52 3D information after workpiece removal 52a Enrollment Verification Area 53 Invisible 3D information before workpiece removal 54 Invisible 3D information after workpiece removal A Disappearance area B. Cargo collapse area H Number of invisible workpieces I Number of invisible workpieces J Number of visible workpieces that have disappeared K Number of visible collapsed workpieces Q Change Area W work

Claims

1. a three-dimensional information acquisition unit that acquires three-dimensional information of a target space in which a plurality of workpieces are loaded; A workpiece shape information acquisition unit that acquires shape information of the workpiece; a workpiece take-out number calculation unit that calculates a range of the number of workpieces to be taken out, taking into account cavities in the workpiece loading area, based on three-dimensional information of the target space before and after the workpieces are taken out and shape information of the workpieces; The workpiece removal number calculation device includes:

2. The workpiece take-out number calculation device according to claim 1 , wherein the workpiece take-out number calculation unit calculates the range of the number of workpieces to be taken out based on a range of a number of lost workpieces that can be accommodated in a disappearance area that has disappeared from the loading area after the workpieces have been taken out.

3. The workpiece removal number calculation device according to claim 2, wherein the workpiece removal number calculation unit calculates the range of the number of workpieces to be removed by taking into account a range of the number of workpieces that can be accommodated in a load collapse area that has newly appeared in the loading area of ​​the workpiece after the workpiece is removed.

4. 4. The workpiece take-out number calculation device according to claim 3, wherein the workpiece take-out number calculation unit calculates a range of the number of workpieces to be taken out by subtracting the minimum number of the number of workpiece collapses from the maximum number of the number of workpieces disappeared to obtain the maximum number of the number of workpiece collapses, and subtracting the maximum number of the number of workpiece collapses from the minimum number of the number of workpieces disappeared to obtain the minimum number of the number of workpieces to be taken out.

5. A range of the number of workpieces that can be accommodated in the disappearance area that disappears from the workpiece loading area after the workpiece is removed; In some cases, the range of the number of workpieces that can be accommodated in a workpiece collapse area that has newly appeared in the workpiece loading area after the workpiece is removed; A range of the number of workpieces to be taken out calculated based on the range of the number of workpieces lost and, in some cases, the range of the number of workpieces collapsed; When the maximum value and the minimum value of the range of the number of workpieces to be taken out match, The workpiece take-out number calculation device according to claim 1 , further comprising a display unit that displays at least one of the above.

6. The work removal number calculation device according to claim 5, wherein the display unit displays a superimposed image in which a shape model of the work is superimposed on a location that is the basis for calculating at least one of the range of the number of workpieces lost, the range of the number of workpieces collapsed, and the range of the number of workpieces removed.

7. The workpiece removal number calculation unit identifies a change area in the loading area after workpiece removal by subtracting the three-dimensional information of the target space before and after workpiece removal. The workpiece removal number calculation device according to any one of claims 1 to 6.

8. The workpiece removal number calculation device according to claim 7, wherein the change area includes at least one of a disappearance area that disappears from the three-dimensional information of the target space after workpiece removal, and a load collapse area that newly appears in the three-dimensional information of the target space after workpiece removal.

9. 9. The work take-out number calculation device according to claim 1, further comprising a work information acquisition unit that acquires work information indicating whether or not one or more of the workpieces have been taken out, and the work take-out number calculation unit determines whether or not to execute a calculation process for the range of the number of workpieces to be taken out based on the work information.

10. The workpiece removal number calculation device according to any one of claims 1 to 9, further comprising an operation control unit that controls the operation of at least one of a hand that removes and removes the workpieces and a transport device that transports the workpieces using the hand, based on the range of the number of workpieces to be removed.

11. The workpiece take-out number calculation device according to claim 10 , wherein the operation control unit instructs the hand to perform an operation of repeating the take-out and removal of the workpieces until the range of the take-out number of the workpieces becomes one value and a desired take-out number is reached.

12. A hand that takes out and puts out the workpiece; A conveying device that conveys a workpiece using the hand; A control device for controlling an operation of at least one of the hand and the transport device; A visual sensor that outputs three-dimensional information of a target space in which a plurality of the workpieces are loaded; Equipped with The control device includes: a three-dimensional information acquisition unit that acquires three-dimensional information of the target space from the visual sensor; A workpiece shape information acquisition unit that acquires shape information of the workpiece; a workpiece take-out number calculation unit that calculates a range of the number of workpieces to be taken out, taking into account cavities in the workpiece loading area, based on three-dimensional information of the target space before and after the workpieces are taken out and shape information of the workpieces; an operation control unit that controls an operation of at least one of the hand and the transport device based on the range of the number of workpieces to be taken out; A hand system comprising:

13. The hand system according to claim 12, further comprising a presence sensor that outputs presence information indicating whether or not one or more workpieces have been picked up, and the workpiece pick-up number calculation unit determines whether or not to perform a calculation process for the range of the number of workpieces to be picked up based on the presence information.

14. A display device that visually displays the calculation process for the number of workpieces to be removed from a plurality of loaded workpieces, the display device having a display unit that displays a superimposed image that identifiably displays a changed area in the workpiece loading area after the workpieces are removed, superimposed on at least one of the three-dimensional information of the target space before and after the workpieces are removed.

15. The display device according to claim 14, wherein the change area includes at least one of a disappearance area that disappears from the loading area after the workpiece is removed and, optionally, a load collapse area that newly appears in the loading area after the workpiece is removed, and the display unit identifies and displays at least one of the change area, the disappearance area, and, optionally, the load collapse area.

16. A method for controlling a computer-implemented program comprising: The processor is a control device that sends to a display device a superimposed image that identifiably displays the changed area in the work loading area after the work is removed on at least one of the three-dimensional information of the target space before and after the work is removed, in order to visually display the calculation process of the number of workpieces to be removed from a plurality of loaded workpieces.

17. A computer program which causes a processor to execute the operation of sending to a display device a superimposed image which identifiably displays the area of ​​change in the work loading area after the work is removed, over at least one of the three-dimensional information of the target space before and after the work is removed, in order to visually display the process of calculating the number of workpieces to be removed from among multiple loaded workpieces.

Images