Device discrimination system, device discrimination method, and device discrimination program

Abstract

To allow for configuring auxiliary devices, such as workpiece turnover devices, for assisting work on workpieces loaded in a loading area in a freely placeable manner, and accurately identifying placement positions and orientations thereof.SOLUTION: A device identification system provided herein comprises: freely placeable, portable auxiliary devices for assisting work on workpieces loaded in a loading area; an image capturing device capable of capturing an image of the loading area; and a control device configured to identify placement positions and orientations of the auxiliary devices in the loading area using the captured image captured by the image capturing device. Each auxiliary device comprises a device main body, a movable unit capable of moving relative to the device main body and contacting a workpiece in the loading area, and indicators provided at least on the movable unit so as to be present in the captured image and designed to enable identification of a position and orientation of the auxiliary device, where the control device is configured to detect the indicators in the captured image and identify the placement position and orientation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a device discrimination system, a device discrimination method, and a device discrimination program.

Background Art

[0002] Conventionally, a work such as a plurality of plate-like members loaded in a loading area has been supplied by a loading device such as a supply robot to a processing machine such as a press brake that performs bending processing or the like. And at the time of loading, for example, a magnet float that floats a work by magnetic force and separates it from other works, or an air separator that separates a work from other works by the injection pressure of air (for example, see Patent Document 1) and other work turning devices (work single-piece taking devices) are used.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, a conventional work turning device including an air separator described in Patent Document 1 is provided in a state of being fixed to an installation table installed at a position adjacent to the loading area. For this reason, a plurality of works loaded on the loading area are placed at one location, and one work turning device is arranged on the installation table corresponding to these works.

[0005] However, since the workpiece turning device is fixedly arranged on the installation base, it cannot actually be installed at an arbitrary position on the workpiece, and the workpiece is loaded so as to correspond to the installation position of the workpiece turning device. Therefore, when loading a workpiece that is sufficiently large with respect to the loading area, there may be inconvenience due to spatial problems in the loading area when loading the workpiece corresponding to the installation position of the workpiece turning device. Thus, there has been a desire to install the workpiece turning device not fixedly but at an arbitrary position.

[0006] However, in the case of loading a workpiece that is sufficiently large with respect to the loading area as described above, even if the workpiece turning device is configured to be installable at an arbitrary position, it is necessary to install it at an arbitrary position around the loading area. For this reason, for example, even if an attempt is made to image the loading area with an imaging device to detect the workpiece turning device, the workpiece turning device cannot be completely imaged within the imaging range, so it is difficult to appropriately detect the device itself, and it has been very difficult to determine its arrangement position and arrangement orientation.

[0007] The present invention has been made in view of the above circumstances, and an apparatus discrimination system, an apparatus discrimination method, and an apparatus discrimination program are provided that are configured to be able to freely arrange an auxiliary device that assists operations on a workpiece loaded in a loading area such as a workpiece turning device, and to accurately determine its arrangement position and arrangement orientation.

Means for Solving the Problems

[0008] The device discrimination system according to the present invention includes an auxiliary device having a portable and deployable nature for assisting in operations on a workpiece loaded within a loading area, an imaging device capable of imaging the loading area, and a control device for discriminating the arrangement position and orientation of the auxiliary device with respect to the loading area based on an imaging image captured by the imaging device. The auxiliary device has a device body and a movable part provided to be movable relative to the device body and capable of contacting the workpiece within the loading area, and has an index provided at least on the movable part shown in the imaging image and capable of identifying the position and direction of the auxiliary device. The control device detects the index included in the imaging image and discriminates the arrangement position and the arrangement orientation.

[0009] In one embodiment of the present invention, the index is provided on the upper surface side near the tip of the movable part.

[0010] In another embodiment of the present invention, the control device compares the detected index with the index shown in a pre-stored index model image to discriminate the arrangement position and the arrangement orientation.

[0011] In still another embodiment of the present invention, the loading area is formed in a rectangular shape in plan view, the auxiliary device is arranged such that the device body is located outside the loading area, and the control device detects the index included in an inner rectangular area of the outer peripheral edges of a set of intersecting long side and short side within the loading area from the imaging image and discriminates the arrangement position and the arrangement orientation.

[0012] In still another embodiment of the present invention, a plurality of the auxiliary devices are provided, and the control device determines an arrangement situation including the arrangement position and the arrangement orientation for each auxiliary device with respect to the loading area discriminated based on the detected index.

[0013] In still another embodiment of the present invention, the auxiliary device has a top surface on the device body, and the index is provided on the top surface of the device body.

[0014] In still another embodiment of the present invention, the auxiliary device has an inclined surface portion that inclines from the top surface portion of the device body, and the indicator is provided on the inclined surface portion of the device body.

[0015] In still another embodiment of the present invention, the indicator provided on the inclined surface portion is provided at a location farthest from the top surface portion.

[0016] In still another embodiment of the present invention, the indicator provided on the top surface portion is larger than the indicators provided on the movable portion and the inclined surface portion, and the indicator provided on the inclined surface portion is smaller than the indicators provided on the movable portion and the top surface portion.

[0017] In still another embodiment of the present invention, the auxiliary device has a pair of side surface portions that are connected to the top surface portion of the device body, and the indicator is provided on the pair of side surface portions of the device body.

[0018] In still another embodiment of the present invention, the indicator has light transmissibility, and the auxiliary device includes light sources inside the movable portion and the device body, respectively.

[0019] In still another embodiment of the present invention, the control device detects the auxiliary device disposed in the loading area based on the captured image, and determines the arrangement position and the orientation of the arrangement based on the detected indicator.

[0020] In still another embodiment of the present invention, the control device detects the workpiece in the loading area based on the captured image, and identifies the auxiliary device to be operated on the workpiece based on the determination result of the arrangement status.

[0021] In still another embodiment of the present invention, a detection means capable of detecting the auxiliary device stored in a storage area provided at a location separated from the loading area is provided, and the control device determines the storage status of the auxiliary device in the storage area based on the detection result from the detection means.

[0022] In still another embodiment of the present invention, there is provided notification means capable of notifying a predetermined alarm, and the control device controls the notification means to notify the predetermined alarm based on the determination result of the arrangement status and / or the storage status.

[0023] The apparatus discrimination method according to the present invention includes a step of imaging, by an imaging device, a loading area in which at least auxiliary devices having a portable and deployable property for assisting work on a workpiece loaded in the loading area are arranged around; and a step of discriminating, by a control device, the arrangement position and the arrangement orientation of the auxiliary devices with respect to the loading area based on an imaging image captured by the imaging device. In the discriminating step, an index capable of identifying the position and the direction of the auxiliary device, which is provided at least on a movable part that is provided to be movable with respect to the device main body and capable of contacting the workpiece in the loading area, and appears in the imaging image, is detected from the imaging image to discriminate the arrangement position and the arrangement orientation.

[0024] In one embodiment of the present invention, in the discriminating step, the index included in an inner rectangular region of an outer periphery of a set of intersecting long-side and short-side in the loading area formed in a rectangular shape in plan view is detected from the imaging image to discriminate the arrangement position and the arrangement orientation.

[0025] In another embodiment of the present invention, in the imaging step, the loading area in which a plurality of the auxiliary devices are arranged such that the device main body is located outside the loading area is imaged, and in the discriminating step, an arrangement status including the arrangement position and the arrangement orientation of each auxiliary device with respect to the loading area discriminated based on the detected index is determined.

[0026] In still another embodiment of the present invention, in the step of imaging, the loading area in which a plurality of the auxiliary devices are further arranged in the loading area is imaged, and in the step of determining, the arrangement status including the arrangement position and the arrangement orientation of each auxiliary device in the loading area determined based on the detected index is determined.

[0027] In still another embodiment of the present invention, the step of detecting the workpiece in the loading area based on the captured image by the control device, and specifying an auxiliary device to operate on the workpiece based on the determination result of the arrangement status is included.

[0028] In still another embodiment of the present invention, the step of detecting the auxiliary device stored in a storage area provided at a location separated from the loading area by a detection means, and the step of determining the storage status of the auxiliary device in the storage area by the control device based on the detection result from the detection means are included.

[0029] In still another embodiment of the present invention, the step of controlling a notification means capable of notifying a predetermined alarm by the control device based on the determination result of the arrangement status and / or the storage status, and causing the predetermined alarm to be notified is included.

[0030] An apparatus determination program according to the present invention causes a computer to acquire a captured image obtained by imaging, by an imaging device, a loading area in which at least auxiliary devices having a portable and movable property that can assist work on a workpiece loaded in the loading area are arranged around. And a step of determining an arrangement position and an arrangement orientation of the auxiliary device with respect to the loading area based on the acquired captured image, and in the step of determining, at least provided on a movable part of the auxiliary device that has a device main body and a movable part that is movable with respect to the device main body and can come into contact with the workpiece in the loading area. An index capable of identifying the position and direction of the auxiliary device is detected from the captured image, and the arrangement position and the arrangement orientation are determined.

Advantages of the Invention

[0031] According to the present invention, an auxiliary device for assisting work on a work piece loaded in a loading area such as a work turning device is configured to be freely arranged, and its arrangement position and arrangement direction can be accurately determined.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0033] Hereinafter, with reference to the accompanying drawings, an apparatus discrimination system, an apparatus discrimination method, and an apparatus discrimination program according to an embodiment of the present invention will be described in detail. However, the following embodiments do not limit the invention according to each claim, and not all combinations of the features described in the embodiments are essential for the solution means of the invention. In this embodiment, the scales and dimensions of each component may be exaggerated, or some components may be omitted.

[0034] FIG. 1 is a perspective view schematically showing a configuration example of an apparatus discrimination system 100 according to an embodiment of the present invention. As shown in FIG. 1, the apparatus discrimination system 100 according to this embodiment includes a work turning device 10 as an auxiliary device for assisting work (for example, loading onto a bending machine or the like) on a work W loaded in a loading area TA on a pallet P, and a camera 20 as an imaging device capable of imaging the loading area TA.

[0035] Further, the apparatus discrimination system 100 has functions of an NC device or the like for controlling the entire apparatus discrimination system 100, and also has functions of an image processing device for performing general image processing such as image analysis and image correction based on the captured image captured by the camera 20, and includes a control device 30 for discriminating the arrangement position and the arrangement direction of the work turning device 10 with respect to the loading area TA.

[0036] In this embodiment, the control device 30 includes a display 69 as a display device and an input unit 35a including an input device such as a keyboard and a mouse. However, as long as it can have equivalent functions instead of these display 69 and input unit 35a (for example, display means and input means that can be used remotely), it is not limited to this.

[0037] In addition, in the present embodiment, for example, although illustration of processing machines such as bending machines and loading devices such as workpiece holding robots is omitted, these processing machines, loading devices, etc. may be included in the device discrimination system 100 in a state controllable by the control device 30.

[0038] Also, in the present embodiment, a storage area 50 for storing the necessary number of workpiece turning devices 10 corresponding to the workpiece W loaded in the loading area TA is provided at a location separated from the loading area TA of the pallet P. And a storage sensor 51, which is a detection means for individually detecting the stored workpiece turning devices 10, is provided in this storage area 50. The control device 30 can determine the storage status of the workpiece turning devices 10 in the storage area 50 based on the detection signal from the storage sensor 51.

[0039] In addition, in the present embodiment, the workpiece turning device 10 is described as an example of the auxiliary device, but the auxiliary device is not limited to this as long as it can assist various operations on the workpiece W. Further, although illustration is omitted, each of the workpiece turning devices 10, the camera 20, and the control device 30 are connected so as to be able to transmit and receive data and signals to and from each other by wire or wirelessly.

[0040] The workpiece turning device 10 has portability that allows it to be freely arranged, and can be freely arranged at any location inside or outside the loading area TA so as to correspond to the workpiece W arranged in the loading area TA on the pallet P. In the present embodiment, the workpiece turning device 10 can be arranged to correspond one-to-one to the workpiece W from outside the loading area TA.

[0041] More specifically, the work turning device 10 of the present embodiment is arranged as follows, for example. Generally, for a loading area TA formed in a rectangular shape in plan view, another auxiliary device (not shown) such as a magnetic floater is installed on one long side of the loading area TA, and an end face bar (not shown) for abutting the work W and the like is installed on one short side. And the work turning device 10 is arranged on the long side or the short side of the loading area TA together with this other auxiliary device and the end face bar so that at least the turning unit 10B can exist within the loading area TA.

[0042] That is, in FIG. 1, the work turning device 10 is arranged on one long side of the loading area TA on which a large-area work W is loaded so that the device main body 10A is located outside the loading area TA. However, for example, as shown by the two-dot chain line, it may be arranged in the same way on one short side.

[0043] FIG. 2 is a perspective view schematically showing the work turning device 10 in the device discrimination system 100. As shown in FIG. 2, the work turning device 10 has a device main body 10A having a rectangular outer shape in top view, and a turning unit 10B which is a movable part provided in a so-called cantilever shape with respect to the device main body 10A so as to be movable with respect to the device main body 10A and capable of contacting and holding the work W in the loading area TA.

[0044] Note that the turning unit 10B of the work turning device 10 has a function of turning and separating the uppermost work W among a plurality of plate-shaped works W loaded in the loading area TA on the pallet P from the other works W by, for example, the injection pressure and adsorption force of air. In this way, the work turning device 10 can be used as a single-piece picking device for picking up one work W by the turning unit 10B. However, since the detailed configuration of the single-piece picking device is well-known, the description thereof is omitted here. In the work turning device 10 of the present embodiment, the arrangement side of the turning unit 10B is defined as the front side of the device.

[0045] The device body 10A of the workpiece turning device 10 has a top surface portion 11 provided on the upper side of the device, inclined surface portions 12 that incline at two locations on the front side of the device from this top surface portion 11, and a pair of side surface portions 13 that connect to the left and right sides of the top surface portion 11 (and the inclined surface portions 12). Markers 14, 15, 16, and 17 are provided at predetermined locations on the top surface portion 11, the inclined surface portions 12, and the pair of side surface portions 13 of the device body 10A and at a predetermined location of the turning unit 10B as indicators capable of identifying the position and direction of the workpiece turning device 10, respectively.

[0046] Specifically, a plurality of markers 14 and 16 are provided at predetermined locations on the top surface portion 11 and the pair of side surface portions 13 of the device body 10A, and a plurality of markers 15 are provided at predetermined locations on the inclined surface portions 12 of the device body 10A. Further, the marker 17 is provided at a location near the tip of the upper surface portion 10a of the turning unit 10B.

[0047] More specifically, the markers 14 are provided at two locations on the top surface portion 11 of the device body 10A near the rear side of the device, and the markers 16 are provided at locations near the upper side of the device on the pair of side surface portions 13 of the device body 10A, respectively. That is, the markers 16 are provided laterally and directly below the markers 14. Further, the markers 15 are provided at locations near both ends in the inclination direction on the two inclined surface portions 12 of the device body 10A, respectively.

[0048] FIG. 3 is a diagram showing an example of the markers 14 to 17 provided on the workpiece turning device 10, and FIG. 4 is a diagram showing another example of the markers 14 to 17. The markers 14, 15, and 16 provided on the device body 10A of the workpiece turning device 10 and the marker 17 provided on the turning unit 10B can be constituted by, for example, AR (augmented reality) markers including known ArUco markers, ChArUco markers, and the like.

[0049] As shown in FIG. 3, the markers 17 (14 to 16) are constituted by a film-like member or a plate-like member including at least an overall region 18 representing the entire marker and an identification region 19 for identifying the position and orientation of the marker. In a predetermined drawing including FIGS. 1 and 2, illustration of the identification regions 19 of the markers 14 to 17 is omitted.

[0050] The shape of the identification regions 19 of the markers 14 to 17 is, for example, a shape combining simple rectangles in the example shown in FIG. 3. However, as shown in FIGS. 4(a) to 4(g), it is not limited to the illustrated shape as long as it can identify at least the position and orientation of the work turning device 10 alone by combining a plurality of rectangles or squares or changing the position of the identification region 19 within the overall region 18, and various shapes such as more complex shapes or simpler shapes can be adopted.

[0051] Also, the dimensions (such as the size and shape of the overall region 18 and the identification region 19) of the markers 14 to 17 can be set according to the imaging performance of the camera 20 such as the number of pixels and the imaging resolution. However, it is sufficient as long as it can be detected so that the arrangement position and the arrangement orientation of the work turning device 10 can be sufficiently determined from the captured image of the loading area TA taken from above.

[0052] In the present embodiment, it is sufficient that at least one marker 17 is provided in the turning unit 10B. That is, when the work turning device 10 is in a normal arrangement state around the loading area TA, only the turning unit 10B is reflected in the captured image. When the work turning device 10 is arranged within the loading area TA, it is sufficient that at least one marker 14 is provided on the top surface portion 11 of the device main body 10A among the markers 14 to 16. This is because the top surface portion 11 of the device main body 10A is most likely to be reflected in the captured image.

[0053] When markers 14 to 16 are provided on the apparatus main body 10A, the detection accuracy and recognition accuracy from the captured images of the markers 14 and 15 can be further improved by providing, for example, marker sets 14 and 15 at a plurality of locations on the top surface portion 11 and the inclined surface portion 12, but it is not limited thereto.

[0054] Also, from the viewpoint of further improving the detection accuracy and the like, the marker 17 is provided at a location farthest from the apparatus main body 10A, that is, in the vicinity of the tip portion of the turning unit 10B as described above. For example, the marker 15 provided on the inclined surface portion 12 of the apparatus main body 10A may be provided at a location farthest from the top surface portion 11, that is, in the vicinity of the inclined direction end portion on the front side of the apparatus. Thereby, the marker 17 of the turning unit 10B, the marker 14 of the top surface portion 11, and the marker 15 of the inclined surface portion 12 can be arranged as far apart from each other as possible, so that false detection can be reduced (or detection error can be reduced).

[0055] Note that the marker 14 provided on the top surface portion 11 preferably has larger dimensions in both the entire region 18 and the identification region 19 than the markers 17 and 15 provided on the turning unit 10B and the inclined surface portion 12 so as to be more easily detectable. In this way, if the marker 14 on the top surface portion 11 is large, it is possible to easily detect it even if the imaging performance of the camera 20 is somewhat lower than that generally used in this field. And the marker 15 provided on the inclined surface portion 12 is configured to be smaller in both the entire region 18 and the identification region 19 than the markers 17 and 14 provided on the turning unit 10B and the top surface portion 11, as shown in FIG. 2 and the like.

[0056] Note that the markers 16 provided on the pair of side surface portions 13 of the apparatus main body 10A are provided to be able to identify the position and direction of the workpiece turning device 10 (in this case, the arrangement position and the arrangement direction when it falls so that the marker 16 appears in the loading area TA) even when the workpiece turning device 10 falls due to some accident (for example, collides with a robot arm of a loading device not shown).

[0057] On the other hand, the reason why the markers 16 are not provided on the front and rear surfaces connected to the pair of side surfaces 13 of the apparatus main body 10A is that, even when the workpiece turning device 10 falls, the outer shape of the apparatus main body 10A is such that it is almost impossible for these front and rear surfaces to come to rest facing upward. Needless to say, depending on the outer shape of the apparatus main body 10A, the markers 16 may be provided on the front and rear surfaces of the apparatus main body 10A.

[0058] Furthermore, the markers 14 to 17 may be configured to have light transmissibility so that their entire regions 18 and identification regions 19 can be recognized well even in a captured image taken in a dark environment, for example. When the markers 14 to 17 have light transmissibility, the workpiece turning device 10 may be further configured to include light sources (such as LEDs), not shown, inside the turning unit 10B and the apparatus main body 10A (at least inside the turning unit 10B in this embodiment).

[0059] In this way, even when the lighting equipment described later is OFF in a dark environment such as at night, if the light sources inside the turning unit 10B and the apparatus main body 10A are turned on, the markers 14 to 17 can be clearly recognized and made to stand out in the captured image. Thus, it becomes possible to perform so-called night vision on the marker 17 (and markers 14 to 16) of the turning unit 10B of the workpiece turning device 10 at least within the loading area TA.

[0060] On the other hand, the camera 20 is constituted by, for example, an inexpensive and highly versatile monocular camera (that is, one camera), and is arranged directly above the center of the loading area TA of the pallet P via a support member such as a camera stand 21 so as to be able to image the entire loading area TA. Although not shown in the figure, lighting equipment having a plurality of light-emitting diodes (LEDs) or the like for irradiating irradiation light onto the workpiece W loaded in the loading area TA is provided at predetermined locations around the pallet P.

[0061] When imaging the loading area TA with the camera 20, if the lighting equipment irradiates the workpiece W with irradiation light, the brightness of the captured image can be appropriately adjusted to proper exposure. If the markers 14 to 17 having the above-described light transmissibility and light sources are used, for example, even when the lighting equipment is provided at a position where the irradiation light from the lighting equipment does not reach the top surface portion 11 of the apparatus main body 10A, the markers 14 to 17 in the captured image can be made to stand out sufficiently recognizable.

[0062] FIG. 5 is a block diagram schematically showing a functional configuration of the control device 30 in the apparatus discrimination system 100, and FIG. 6 is a diagram showing an image of the index model image stored in the storage unit 34 of the control device 30. Further, FIG. 7 is a configuration diagram schematically showing a hardware configuration of the control device 30, and FIG. 8 is a plan view showing an image of the marker 17 detected from the captured image captured by the camera 20 in the control device 30.

[0063] As shown in FIG. 5, the image data of the captured image captured by the camera 20 is input to the control device 30. Functionally, the control device 30 includes an image acquisition unit 31, an image processing unit 32, an arithmetic unit 33, a storage unit 34, and an operation unit 35. Note that the discrimination of the arrangement position and the arrangement direction of the workpiece turning device 10 with respect to the loading area TA (hereinafter, such processing may be referred to as "discrimination processing") is mainly performed by the image processing unit 32 of the control device 30.

[0064] Here, the image acquisition unit 31 acquires the image data of the captured image output from the camera 20. The image processing unit 32 has, for example, a marker detection unit 32a and a matching processing unit 32b, and performs various image conversion and image analysis processes (hereinafter, such processes are collectively referred to as "image processing") including binarization processing, contour detection processing (edge detection processing) using the Sobel method, morphological processing, approximation processing using Newton's method, template matching processing, etc. on the image data acquired by the image acquisition unit 31 to execute the above-described discrimination processing.

[0065] Based on the discrimination result from the image processing unit 32, the arithmetic unit 33 performs various arithmetic processes for controlling the entire device discrimination system 100 including the work turning device 10, and outputs control signals to each unit. The storage unit 34 can read and write various data used by the control device 30, such as CAD data representing the work W and the work turning device 10, image data of model (marker model) images serving as references for the markers 14 to 17, and various program data.

[0066] As shown in FIG. 6, the marker model images 17c1 to 17c n are a plurality of types of corresponding images (for example, images that appear tilted, distorted, or have different sizes based on the height position) considering the height positions and rotation states of the markers 14 to 17 in the work turning device 10, and are stored in a database (DB) of the storage unit 34 as template images.

[0067] Therefore, by comparing the marker 17 (14 to 16) detected from the captured image with the marker model images 17c1 to 17c n and performing matching, it becomes possible to determine the posture such as the position, height, and orientation of the actual work turning device 10 with respect to the loading area TA. The operation unit 35 receives operation inputs from the user of the control device 30 via the input unit 35a.

[0068] As shown in FIG. 7, as a hardware configuration, the control device 30 includes, for example, a CPU 61, a RAM 62, a ROM 63, an HDD (hard disk drive) 64, and an SSD (solid state drive) 65. The control device 30 also includes an input I / F (interface) 66, an output I / F (interface) 67, and a communication I / F (interface) 68. Each component 61 to 68 is interconnected by a bus 60.

[0069] The CPU 61 controls the entire device discrimination system 100 including the control device 30 by executing various programs stored in the RAM 62, ROM 63, HDD 64, SSD 65, etc., and realizes the functions of the image processing unit 32 and the arithmetic unit 33 by executing the device discrimination program.

[0070] The RAM 62 can be used as a work area for the CPU 61. The ROM 63 stores at least the various programs in a readable manner. The HDD 64 and SSD 65 read and write the various data described above and realize the function of the storage unit 34 together with the RAM 62 and ROM 63.

[0071] The camera 20 is connected to the input I / F 66 to acquire the captured image. Therefore, the input I / F 66 realizes the function of the image acquisition unit 31. At the same time, a touch panel 69a that functions as the input unit 35a of the operation unit 35 is connected to the input I / F 66 to receive information accompanying the operation input from the user. Note that input means such as a keyboard and a mouse (not shown) can also be connected to the input I / F 66.

[0072] For example, the display 69 (see FIG. 1) of the control device 30 with the touch panel 69a built in is connected to the output I / F 67, and various information to be monitor-displayed is output. Note that the control device 30 can be connected to a network such as the Internet (not shown) or an external device via the communication I / F 68.

[0073] In the discrimination process by the control device 30 configured as described above, specifically, first, the marker detection unit 32a of the image processing unit 32 detects the marker 17 provided on the upper surface portion 10a of the turning unit 10B of the work turning device 10 from the captured image 22 that has been imaged and image-processed for the loading area TA as shown in FIG. 8 by edge detection processing or the like.

[0074] When detecting the marker 17, in order to reduce the processing burden, instead of detecting the marker 17 from the entire captured image 22, first, attention is paid to one long side and one short side of the loading area TA where the workpiece turning device 10 is likely to be arranged. That is, the marker 17 is detected from the detection ranges D1 and D2 which are a set of inner rectangular areas intersecting within the loading area TA in the captured image.

[0075] Then, the image processing unit 32 records the position information (pixel positions) of, for example, the pixels at the four corners of the marker 17 detected by the marker detection unit 32a in the storage unit 34. Here, a pixel refers to the smallest unit or smallest element having color information (such as color tone and gradation) when the control device 30 processes the captured image.

[0076] Next, the image processing unit 32, for the marker 17 in the captured image 22, based on the dimensional information of each part of the device, the loading area TA, etc. in the CAD data stored in the storage unit 34, and the image data of the marker model images 17c1 to 17c n obtains a position model representing the actual position of the marker 17 in the workpiece turning device 10.

[0077] Then, the image processing unit 32, by the matching processing unit 32b, (1) applies the position model and the pixel positions to an approximation process such as the Newton method to calculate the physical position (x, y, z position) and the rotation state (rz) of the marker 17 within the loading area TA, (2) generates a marker image reflecting the physical position (x, y, z position) and the rotation state (rz) based on the calculation results, and (3) compares the marker 17 shown by the generated marker image with various marker model images 17c1 to 17c pre-stored in the storage unit 34. nand, for example, search by performing template matching processing, and update the pixel position of the marker 17 stored in the storage unit 34 based on the matching information of the marker indicated by the searched marker model image. That is, by repeating the above processes (1) to (4) a plurality of times to improve the discrimination accuracy, the actual position and orientation (i.e., the placement position and the orientation of placement) of the work turning device 10 with respect to the loading area TA are discriminated.

[0078] Note that, as described above, when the marker 17 exists within the detection ranges D1 and D2, during the above processes (1) to (4) in the matching processing unit 32b, the detected marker 17 is compared with the marker model images 17c1 to 17c in the DB of the storage unit 34. n Among them, after preferentially matching with two types of marker model images with an image angle of 0° or 90° and searching, the detailed orientation of the work turning device 10 (for example, the rotation angle has a few degrees, etc.) is specified, and the placement position and the orientation of placement are discriminated. In this way, it is possible to further reduce the processing load of the matching processing unit 32b.

[0079] As a result, for example, it becomes possible to accurately grasp the placement position and the orientation of placement of the work turning device 10 that is arranged so as to be freely placed with respect to the loading area TA and determine the placement situation. And it becomes possible to surely perform various operation controls and the like according to the work turning device 10 whose placement situation has been discriminated.

[0080] Note that, as shown in FIGS. 9 and 10, even when a plurality of workpieces W are arranged in the loading area TA and a plurality of work turning devices 10 are arranged in a one-to-one relationship with respect to the workpiece W, without detecting the marker 17 from the entire captured image 22, by paying attention to the detection ranges D1 and D2 and detecting the marker 17 included in each detection range D1 and D2 and performing discrimination processing. In this way, it is possible to accurately grasp the placement position and the orientation of placement for each work turning device 10 while reducing the processing load, determine the placement situation, and perform various operation controls and the like according to this for each device.

[0081] In this embodiment, the discrimination process described above is merely an example and is not limited thereto. Any method that can detect markers 17 etc. by image recognition to identify the position and orientation of the work turning device 10 and discriminate the arrangement position and orientation with respect to the loading area TA is applicable. Further, when the work turning device 10 is also arranged within the loading area TA depending on the loading mode of the work W, the markers 14 to 16 may be detected based on the captured image of the loading area TA and the discrimination process may be similarly performed.

[0082] Next, an operation example when actually arranging and operating, for example, a plurality of work turning devices 10 inside and outside the loading area TA will be described. In this operation, the control device 30 is configured to include an alarm device (not shown) as a notification means capable of notifying a predetermined alarm. The alarm device is configured to be able to notify a predetermined alarm by lighting a warning lamp or outputting an alarm by voice.

[0083] FIG. 11 is a diagram showing an example of a part of a setting screen displayed on the display 69 of the control device 30, and FIG. 12 is a flowchart showing an example of an operation associated with the discrimination process of the control device 30. Here, in the device discrimination system 100, a case where up to two work turning devices 10 can be arranged and stored in either the inside or outside of the loading area TA or in the storage area 50 will be described.

[0084] As shown in FIG. 11, the setting screen 70 displayed on the display 69 of the control device 30 includes, for example, an operation input area 71 having numeric keys and cursor keys, and a parameter setting area 72 for setting various parameters related to the operation of the work turning device 10. Here, when the user installs the work turning device 10 inside and outside the loading area TA, for example, the user operates the operation input area 71 to set the setting value of "turning device" in the parameter setting area 72 to "ON". When the work turning device 10 is not installed with respect to the loading area TA, the above setting value is set to "OFF".

[0085] That is, in this operation, as shown in FIG. 12, the control device 30 first determines whether the arrangement of the device inside and outside the loading area TA of the workpiece turning device 10 is ON or not based on, for example, the setting status of the set value of the above-mentioned "turning device" (step S100). If it is determined that the arrangement of the device is not ON, that is, it is OFF (the workpiece turning device 10 is not arranged) (No in step S100), the processing of this flowchart is terminated. However, if it is determined that the arrangement of the device is ON (that is, the workpiece turning device 10 is arranged) (Yes in step S100), the following processing is performed.

[0086] That is, first, an imaging image obtained by imaging the loading area TA with the camera 20 is acquired, and the image data is analyzed as described above (step S101), and it is determined whether there is a marker 17 within the detection ranges D1 and D2 (step S102). If it is determined that the marker 17 is not within the detection ranges D1 and D2 (No in step S102), it is assumed that there is no arrangement of the workpiece turning device 10 outside the loading area TA. Next, markers 14 to 16 are detected from the imaging image, and for example, a device detection process capable of discriminating the arrangement position and arrangement direction of each of the plurality of workpiece turning devices 10 within the loading area TA is performed (step S103). Also, if it is determined that the marker 17 is within the detection ranges D1 and D2 (Yes in step S102), the arrangement position and arrangement direction of the workpiece turning device 10 outside the loading area TA are discriminated as described above, and a device detection process is performed (step S106).

[0087] Then, in the device detection process of step S103, if the device is not detected (No in step S104), although it is determined that the arrangement of the device inside the loading area TA is ON, there is a problem that the device is actually not detected (for example, the device is forgotten to be installed), the alarm device is controlled to issue an alarm (step S105), and the processing according to this flowchart is terminated.

[0088] On the other hand, when a device is detected (Yes in step S104), and after the arrangement status of the work turning device 10 outside the loading area TA is determined in step S106, it is determined whether the number of detected work turning devices 10 is one or two within the loading area TA or outside the loading area TA (step S107). When it is determined that the number of detected work turning devices 10 is one (1 in step S107), for example, based on the captured image, image processing such as extracting the contour of the work W in the loading area TA with a line image is performed to detect the work W in the loading area TA (step S108).

[0089] Then, for the detected work W, interference calculation is performed to determine the presence or absence of interference by these, such as the work turning device 10 and the robot arm whose arrangement position and arrangement orientation were determined in step S103 or step S106 (step S109), and the processing according to this flowchart is terminated.

[0090] Also, when it is determined that the number of detected work turning devices 10 is two (2 in step S107), in the same manner as step S108, the work W in the loading area TA is detected (step S110), and based on the arrangement status including the arrangement position and arrangement orientation of each work turning device 10 in the loading area TA or outside the loading area TA determined in step S103 or step S1062 respectively, for example, it is determined whether two work turning devices 10 are installed for the same work W (step S111).

[0091] When it is determined that two work turning devices 10 are not installed for the same work W (No in step S111), in the same manner as step S109, interference calculation is performed to determine the presence or absence of interference (step S112), and the processing according to this flowchart is terminated.

[0092] On the other hand, when it is determined that two workpiece turning devices 10 are installed for the same workpiece W ( "Yes" in step S111), although one workpiece turning device 10 should be arranged one-to-one for one workpiece W inside or outside the loading area TA, if there is a problem with the arrangement mode (such as overlapping installation of devices), similar to step S105 above, the alarm device is controlled to give an alarm (step S113), and the processing according to this flowchart is terminated. Thus, according to this operation, when the workpiece turning device 10 is arranged inside and outside the loading area TA, it is possible to prevent the loading of the workpiece W from being performed in a state where the above-mentioned problem remains.

[0093] Figure 13 is a flowchart showing another example of the operation associated with the discrimination process of the control device 30. Hereinafter, descriptions of parts that overlap with the content already described will be omitted. Here, the operation based on the detected number and stored number of the workpiece turning devices 10 inside and outside the loading area TA and in the storage area 50 will be described.

[0094] As shown in Figure 13, in this operation, the control device 30 images the loading area TA with the camera 20 and performs image analysis (step S120) on the captured image in the same manner as in step S101 above, and determines whether there are markers 17 within the detection ranges D1 and D2 (step S121). When it is determined that the marker 17 is not within the detection ranges D1 and D2 ( "No" in step S121), it is assumed that there is no arrangement of the workpiece turning device 10 outside the loading area TA. Next, markers 14 to 16 are detected from the captured image, and a device detection process is performed to determine the arrangement position and arrangement direction of each device within the loading area TA in the same manner as in step S103 above (step S122).

[0095] And, in the device detection process of step S122, if a device is detected (Yes in step S123), similar to step S110 etc., the workpiece W in the loading area TA is detected (step S125), and similar to step S111, it is determined whether two workpiece turning devices 10 are installed for the same workpiece W (step S126).

[0096] If it is determined that two workpiece turning devices 10 are installed for the same workpiece W (Yes in step S126), assuming there is a problem (such as duplicate installation of devices) in the arrangement mode of the workpiece turning device 10 as described above, similar to step S113 etc., the alarm device is controlled to give an alarm (step S127), and the processing according to this flowchart is terminated.

[0097] On the other hand, if it is determined that two workpiece turning devices 10 are not installed for the same workpiece W (No in step S126), similar to step S112 etc., interference calculation is performed (step S128). If it is determined that there is interference (Yes in step S129), the process proceeds to step S127 and the subsequent processing is repeated.

[0098] Also, if it is determined that there is no interference (No in step S129), similar to step S107, it is determined whether the number of detected workpiece turning devices 10 is 1 or 2 within or outside the loading area TA (step S130). And, for example, if it is determined that the number of detected workpiece turning devices 10 is 1 (1 in step S130), based on the detection signal from the storage sensor 51, it is determined whether the storage number of the workpiece turning devices 10 stored in the storage area 50 is 0 or 2 (step S131).

[0099] Here, when it is determined that the storage number is one (i.e., "1" in step S131), it can be determined that there is no abnormality in the number of workpiece turning devices 10 detected inside or outside the loading area TA and the number of workpiece turning devices 10 stored in the storage area 50. Therefore, assuming that the workpiece turning device 10 is properly installed and managed, the process proceeds to the loading process of the workpiece W (step S132), and the processing according to this flowchart is terminated.

[0100] On the other hand, when it is determined that the storage number is zero or two (i.e., "0" or "2" in step S131), it can be determined that there is an abnormality (such as installation error or improper storage, etc.) in both the number of workpiece turning devices 10 inside or outside the loading area TA and the number of workpiece turning devices 10 inside the storage area 50. For this reason, assuming that the workpiece turning device 10 is not properly installed and managed, similar to steps S127 etc., the alarm device is controlled to give an alarm (step S135), and the processing according to this flowchart is terminated.

[0101] Also, when it is determined that the number of workpiece turning devices 10 detected inside or outside the loading area TA is two (i.e., "2" in step S130), similar to step S131, it is determined whether the storage number of the workpiece turning devices 10 stored in the storage area 50 is any one of zero to two (step S134).

[0102] Here, when it is determined that the storage number is zero (i.e., "0" in step S134), it can be determined that there is no abnormality in the number of workpiece turning devices 10 inside or outside the loading area TA and inside the storage area 50. Therefore, assuming that the workpiece turning device 10 is properly installed and managed, the process proceeds to step S132 above to perform loading, and the processing according to this flowchart is terminated.

[0103] On the other hand, when it is determined that the number of stored units is 1 or 2 ( "1" or "2" in step S134), it can be determined that there is an abnormality in the number of workpiece turning devices 10 both inside or outside the loading area TA and inside the storage area 50. Therefore, assuming that the workpiece turning device 10 is not properly installed and managed, the process proceeds to step S135 above, where the alarm device is controlled to issue an alarm, and the processing according to this flowchart is terminated.

[0104] In addition, in the device detection process of step S122 above, when the device is not detected ( "No" in step S123), based on the detection signal from the storage sensor 51, it is determined whether the number of workpiece turning devices 10 stored in the storage area 50 is either 0 or 2 (step S133).

[0105] Then, when it is determined that the number of stored units is 2 ( "2" in step S133), it can be determined that there is no abnormality in the number of workpiece turning devices 10 both inside or outside the loading area TA and inside the storage area 50. Therefore, assuming that the workpiece turning device 10 is properly installed and managed, the process proceeds to step S132 above to perform loading, and the processing according to this flowchart is terminated.

[0106] On the other hand, when it is determined that the number of stored units is 0 or 1 ( "0" or "1" in step S133), it can be determined that there is an abnormality in the number of workpiece turning devices 10 inside the storage area 50 despite the fact that the workpiece turning device 10 is not arranged inside or outside the loading area TA. Therefore, assuming that the workpiece turning device 10 is not properly installed and managed as described above, the process proceeds to step S135 above, where the alarm device is controlled to issue an alarm, and the processing according to this flowchart is terminated.

[0107] Thus, according to this operation, it is possible to accurately determine the arrangement and storage status not only inside and outside the loading area TA but also for the plurality of work turning devices 10 that are arranged and stored in a freely positionable manner in the storage area 50, and it is possible to surely prevent the loading of the work W and the like from being performed under the occurrence of problems.

[0108] As described above, the embodiments of the present invention have been described. However, this embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention and are included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0109] 10 Work turning device (auxiliary device) 10A Device main body 10B Turning unit (movable part) 10a Upper surface part 11 Top surface part 12 Inclined surface part 13 Side surface part 14, 15, 16, 17 Marker (indicator) 18 Entire area 19 Identification area 20 Camera (imaging device) 21 Camera stand 22 Captured image 30 Control device 50 Storage area 51 Storage sensor 100 Device discrimination system D1, D2 Detection range P Pallet TA Loading area W Work

Claims

1. A loading device capable of performing a work supply operation for supplying a work loaded in a loading area to a processing machine, An auxiliary device having a portable property that can be arranged freely to assist the work supply operation by the loading device, An imaging device capable of imaging the loading area, A control device that determines the arrangement position and the orientation of the auxiliary device with respect to the loading area based on an imaging image captured by the imaging device, comprising: The auxiliary device has a device main body and a movable part provided so as to be movable with respect to the device main body and capable of contacting the work in the loading area, and has at least an index provided on the movable part shown in the imaging image, capable of identifying the position and direction of the auxiliary device. The control device detects the index and the work included in the imaging image, and determines the arrangement position and the orientation with respect to the work. Device discrimination system.

2. The index is provided on the upper surface side near the tip of the movable part. The device discrimination system according to claim 1.

3. The loading area is formed in a rectangular shape in plan view. The auxiliary device is arranged such that the device main body is located outside the loading area. The control device detects the index included in an inner rectangular area of the outer peripheral edges of a set of intersecting long side and short side in the loading area from the imaging image, and determines the arrangement position and the orientation. The device discrimination system according to claim 1 or 2.

4. The auxiliary device has a top surface portion on the device main body. The index is provided on the top surface portion of the device main body. The control device detects the auxiliary device arranged in the loading area based on the imaging image, and determines the arrangement position and the orientation based on the detected index. The device discrimination system according to any one of claims 1 to 3.

5. A plurality of the auxiliary devices are provided. It includes a detection means capable of detecting the auxiliary device stored in a storage area provided at a location separated from the loading area. The control device determines an arrangement situation including the arrangement position and the orientation of each auxiliary device with respect to the loading area determined based on the detected index. The control device determines the storage situation of the auxiliary device in the storage area based on the detection result from the detection means. The device discrimination system according to any one of claims 1 to 4.

6. Comprising notification means capable of notifying a predetermined alarm, Based on the determination result of the arrangement status and / or the storage status, the control device controls the notification means to notify the predetermined alarm. The device discrimination system according to claim 5.

7. A step of imaging, by an imaging device, the loading area in which at least auxiliary devices with portable flexibility capable of assisting the work supply operation by a loading device capable of performing a work supply operation of supplying a work loaded in the loading area to a processing machine are arranged around; A step of discriminating, by a control device, the arrangement position and the arrangement orientation of the auxiliary device with respect to the loading area based on the captured image captured by the imaging device, the method including: In the discriminating step, at least provided on the movable part of the auxiliary device that appears in the captured image, the auxiliary device having a device main body and a movable part that is movable with respect to the device main body and is provided so as to be able to contact the work in the loading area, an index capable of identifying the position and direction of the auxiliary device and the work are detected from the captured image, and the arrangement position and the arrangement orientation with respect to the work are discriminated. Device discrimination method.

8. A step of detecting, by a detection means, the auxiliary device stored in a storage area provided at a location separated from the loading area; A step of judging, by the control device, the storage status of the auxiliary device in the storage area based on the detection result from the detection means, the method including: In the imaging step, the loading area in which a plurality of the auxiliary devices are arranged so that the device main body is located outside the loading area is imaged. In the discriminating step, the arrangement status including the arrangement position and the arrangement orientation of each auxiliary device with respect to the loading area discriminated based on the detected index is judged. The device discrimination method according to claim 7.

9. Based on the determination result of the arrangement status and / or the storage status, the method includes a step of controlling, by the control device, notification means capable of notifying a predetermined alarm to notify the predetermined alarm. The device discrimination method according to claim 8.

10. On a computer, A step of causing the computer to acquire a captured image obtained by imaging, by an imaging device, the loading area in which at least auxiliary devices with portable flexibility capable of assisting the work supply operation by a loading device capable of performing a work supply operation of supplying a work loaded in the loading area to a processing machine are arranged around; A device discrimination program that causes a process to be executed for discriminating the arrangement position and orientation of the auxiliary device with respect to the loading area based on the acquired captured image. In the discriminating process, at least provided on the movable part of the auxiliary device that has a device body and a movable part that is movable with respect to the device body and is provided so as to be able to contact the workpiece in the loading area and is shown in the captured image, a reference that can identify the position and orientation of the auxiliary device and the workpiece are detected from the captured image, and the arrangement position and orientation with respect to the workpiece are discriminated. Device discrimination program.

Images