Image processing device, image processing method, and robot control system

Abstract

This image processing device is applied to a recognition device that performs image processing including edge detection on an image obtained using a designated exposure adjustment value and recognizes an object identified by a detected edge. The image processing device comprises: a region setting unit that receives setting of a region including the object within the image; a detection unit that performs edge detection on the basis of the luminance or the saturation within the region set by the region setting unit; and an output unit that displays an image of the object including the detected edge, and an evaluation value based on the length and the intensity of the detected edge.

Description

Image processing device, image processing method, and robot control system

[0001] This specification discloses an image processing device, an image processing method, and a robot control system.

[0002] Conventionally, a robot system for detecting a workpiece using an image has been proposed, which determines the aperture value of a camera when capturing an image of the workpiece based on the measurement value of an illuminance meter. For example, Patent Document 1 discloses a system having a vision device that, when capturing an image of the workpiece, determines the aperture value corresponding to the illuminance measured by the illuminance meter from optimal aperture values ​​previously learned in a memory for each illuminance as the aperture value when capturing the image of the workpiece (the aperture value at which the workpiece can be most easily detected). In this robot system, the vision device changes the aperture value of the camera for each illuminance and learns the aperture value that maximizes the contrast between the workpiece area and the background in the image as the appropriate aperture value for that illuminance.

[0003] Japanese Patent Application Publication No. 11-272845

[0004] When capturing an image of a workpiece, the worker may set the imaging conditions. In such cases, it is desirable for the worker to be able to judge the appropriateness of the set imaging conditions. Furthermore, when extracting a workpiece from an image, the position, angle, and surface pattern of the workpiece may be recognized based on the contour of the subject recognized in the image, or scratches and stains on the workpiece surface may be inspected. In such cases, even if the contrast between the workpiece area and the background is high, the pattern, scratches, stains, etc. on the workpiece surface may not be detected, and the workpiece may not be properly detected from the image. For example, if the workpiece area in the image is overexposed, it may be difficult to detect the pattern, scratches, stains, etc. on the workpiece surface.

[0005] A primary object of the present disclosure is to enable an operator to determine whether or not the exposure adjustment value that has been set is appropriate.

[0006] The image processing system of the present disclosure employs the following means to achieve the above-mentioned main object.

[0007] The image processing device of the present disclosure is an image processing device that is applied to a recognition device that performs image processing including edge detection on an image obtained using a specified exposure adjustment value and recognizes an object identified by the detected edge, and is equipped with: an area setting unit that accepts the setting of an area including the object within a color image; a detection unit that performs edge detection based on brightness or saturation within the area set by the area setting unit; and an output unit that displays an image of the object including the detected edge and an evaluation value based on the length and strength of the detected edge.

[0008] The image processing device of the present disclosure displays an image of the object including the detected edge and an evaluation value based on the length and strength of the detected edge, allowing the operator to check the edge of the object and the evaluation value based on the edge length and strength and determine whether the exposure adjustment value is appropriate.

[0009] The image processing method of the present disclosure is an image processing method used to capture an image of an object with a camera using a specified exposure adjustment value, perform image processing including edge detection on the obtained image, and recognize an object identified by a detected edge, and includes: a setting process for setting an exposure adjustment value when capturing an image of the object with a camera; an image acquisition process for capturing an image of the object with the camera using the set exposure adjustment value; an edge detection process for performing edge detection on each pixel within a predetermined range of the image; an evaluation value derivation process for deriving an evaluation value based on the length and strength of the detected edge; and an exposure adjustment value determination process for determining an exposure adjustment value when capturing an image to be used to recognize the object based on the derived evaluation value.

[0010] This image processing method has the same effects as the image processing device of the present disclosure.

[0011] The robot control system of the present disclosure is a robot control system that recognizes a workpiece and controls a robot that performs work on the workpiece, and is provided with: a camera that captures an image of the workpiece using a specified exposure adjustment value; an exposure adjustment value setting unit that sets the exposure adjustment value when capturing an image of the workpiece with the camera; an image capture unit that controls the camera to capture an image of the workpiece using the set exposure adjustment value and captures the image; a detection unit that performs edge detection on each pixel within a predetermined range of the image; an output unit that outputs an evaluation value based on the length and strength of the detected edge; a reception unit that receives specification of the exposure adjustment value to be used for capturing the image of the workpiece; and a recognition unit that performs edge detection from the image captured with the specified exposure adjustment value and recognizes objects in the image based on the detected edges.

[0012] This robot control system has the same effects as the image processing device of the present disclosure.

[0013] 1 is a schematic configuration diagram of a robot system 1. FIG. 2 is a block diagram showing the electrical connection relationships of the robot system 1. FIG. 3 is a flowchart showing an example of the first half of the evaluation process. FIG. 4 is a flowchart showing an example of the second half of the evaluation process. FIG. 5 is an explanatory diagram showing a state in which a work center is specified on a work area setting screen Sc1. FIG. 6 is an explanatory diagram showing a state in which a work area is specified on a work area setting screen Sc1. FIG. 7 is an explanatory diagram showing an example of an evaluation condition setting screen Sc2. FIG. 8 is an explanatory diagram showing an example of an evaluation result screen Sc3 when an ultra-high brightness alarm is issued. FIG. 9 is an explanatory diagram showing an example of an evaluation result screen Sc3 when the proportion of ultra-high brightness areas K or high brightness areas L is high. FIG. 10 is an explanatory diagram showing an example of an evaluation result screen Sc3 when the proportion of low brightness areas M or ultra-low brightness areas N is high.

[0014] Next, an embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic configuration diagram of a robot system 1. Fig. 2 is a block diagram showing electrical connections in the robot system 1.

[0015] The robot system 1 is a working robot system that, for example, picks up workpieces W (components) supplied by a workpiece supply unit P and mounts them on an object (board C). As shown in FIGS. 1 and 2 , the robot system 1 includes a workbench 2, a robot device 10 having a robot arm 20, a hand camera 60 attached to the robot arm 20, a workpiece camera 70 as an external camera, a control device 90 that controls the robot device 10, an input device 95, and a display device 96.

[0016] In this embodiment, the robot device 10 is a SCARA robot, and includes a base 11 and a robot arm 20 .

[0017] The base 11 is fixed to the workbench 2 and supports the base end side of the robot arm 20. The robot arm 20 includes a first arm 21, a first arm driver 30, a second arm 22, a second arm driver 40, a shaft 23, and a shaft driver 50. The first arm 21 has a base end connected to the base 11 via a first joint axis J1 and is configured to be rotatable (horizontally pivotable) within a horizontal plane relative to the base 11 by rotation of the first joint axis J1. The second arm 22 has a base end connected to a tip end of the first arm 21 via a second joint axis J2 and is configured to be rotatable (horizontally pivotable) within a horizontal plane relative to the first arm 21 by rotation of the second joint axis J2. The shaft 23 is connected to a tip end of the second arm 22 via a third joint axis J3 and is configured to be rotatable around the third joint axis J3 relative to the second arm 22 and to be movable up and down along the axial direction of the third joint axis J3. In the robot system 1 of this embodiment, a workpiece holding unit 24 that serves as an end effector and picks up and holds the workpiece W is provided at the tip of the shaft 23. Examples of the workpiece holding unit 24 include a suction nozzle that attracts the workpiece W by negative pressure, a mechanical chuck that grips the workpiece W with a pair of claws, and an electromagnetic chuck that attracts the workpiece W by an electromagnet.

[0018] The first arm driver 30 includes a motor 32 and an encoder 34. The rotation shaft of the motor 32 is connected to the first joint shaft J1 via a reducer (not shown). By driving the motor 32, the first arm driver 30 transmits torque to the first joint shaft J1 via the reducer, causing the first arm 21 to rotate around the first joint shaft J1 as a fulcrum. The encoder 34 is attached to the rotation shaft of the motor 32 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 32.

[0019] Similar to the first arm driver 30, the second arm driver 40 includes a motor 42 and an encoder 44. The rotation shaft of the motor 42 is connected to the second joint shaft J2 via a reducer (not shown). By driving the motor 42, the second arm driver 40 transmits torque to the second joint shaft J2 via the reducer, causing the second arm 22 to rotate around the second joint shaft J2 as a fulcrum. The encoder 44 is attached to the rotation shaft of the motor 42 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 42.

[0020] The shaft driving unit 50 includes a shaft rotating unit including a motor 52a and an encoder 54a, and a shaft lifting unit including a motor 52b and an encoder 54b. The rotational axis of the motor 52a of the shaft rotating unit is connected to the shaft 23 via a belt (not shown). The shaft rotating unit rotates the shaft 23 around its axis by driving the motor 52a. The rotational axis of the motor 52b of the shaft lifting unit is connected to the shaft 23 via a ball screw mechanism (not shown). The shaft lifting unit drives the motor 52b, and the ball screw mechanism converts the rotational motion of the motor 52b into linear motion, thereby lifting the shaft 23 up and down. The encoder 54a is configured as a rotary encoder that detects the rotational displacement of the shaft 23. The encoder 54b is configured as a linear encoder that detects the lift position of the shaft 23.

[0021] 2, the control device 90 includes a CPU 91, a ROM 92 that stores a processing program, a RAM 93 as a work memory, a storage device 94 such as an HDD or SSD, and an input / output interface (not shown). Position signals from the encoders 34, 44, 54a, and 54b, image signals from the hand camera 60, image signals from the work camera 70, operation signals from the input device 95, and the like are input to the control device 90 via the input / output interface. Drive signals to the motors 32, 42, 52a, and 52b, drive signals to the hand camera 60, drive signals to the work camera 70, image signals to the display device 96, and the like are output from the control device 90 via the input / output interface.

[0022] The input device 95 is, for example, an input device such as a mouse or a keyboard. The display device 96 is, for example, a display device such as a liquid crystal display or an organic EL display.

[0023] The hand camera 60 is attached to the tip of the second arm 22. The hand camera 60 is configured as a color camera capable of capturing color images in which each pixel has a gradation value of the three primary colors RGB (R, G, B). The gradation values ​​(R, G, B) each have 256 levels ranging from 0 to 255. The hand camera 60 has a fixed aperture value, and exposure can be adjusted by adjusting the exposure time. The hand camera 60 captures an image of the workpiece W in the workpiece supply unit P from above and outputs the captured image to the control device 90. The control device 90 recognizes the position of the workpiece W by processing the captured image.

[0024] The work camera 70 is installed between the work supply unit P and the board C on the work table 2. The work camera 70 is a camera similar to the hand camera 60. The work camera 70 captures an image of the work W held by the work holding unit 24 of the robot device 10 from below, and outputs the captured image to the control device 90. The control device 90 processes the captured image to determine whether the work W is being held properly by the work holding unit 24.

[0025] Next, the operation of the robot system 1 configured as described above will be described. The operation of the robot system 1 includes a workpiece position recognition operation, a collection operation (picking operation), a collection confirmation operation, and a mounting operation (placement operation). In the workpiece position recognition operation, the hand camera 60 captures an image of the workpiece W using an exposure time specified by the operator, detects the edges of the workpiece W from the captured image, recognizes the workpiece W based on the detected edges, and identifies the position of the workpiece W. In the collection operation, the hand (workpiece holding unit 24) of the robot system 1 moves to the position of the recognized workpiece W to pick up the workpiece W. In the collection confirmation operation, the workpiece W picked up by the hand is moved above the workpiece camera 70, and the workpiece W is imaged by the work camera 70. The edges are detected from the captured image, the workpiece W is recognized based on the detected edges, and the quality of the workpiece W is confirmed. In the mounting operation, the picked workpiece W is mounted on the board C.

[0026] Incidentally, in the workpiece position recognition operation, the workpiece W is recognized based on the edge of the workpiece W in the image, so an appropriate exposure time must be set in order to obtain an image suitable for edge detection. Furthermore, because the degree of light reflection varies depending on the type of workpiece W, the worker must also set an appropriate exposure time depending on the type of workpiece W. For this reason, it is preferable for the worker to be able to determine whether the exposure time set by the worker is appropriate in an image captured with the exposure time set by the worker. Therefore, the robot system 1 executes an evaluation process to evaluate the appropriateness of the exposure time setting value in an image captured with the exposure time set by the worker.

[0027] 3 and 4 are flowcharts showing an example of the evaluation result display process, which is executed by the CPU 91 of the control device 90 after the operator operates the input device 95 to issue an instruction to start the process.

[0028] To start this process, the CPU 91 first controls the motors 32 and 42 to move the hand camera 60 to a predetermined imaging position (S100). Next, the CPU 91 controls the hand camera 60 to capture a work area setting image Im1 (S102). The CPU 91 then displays the work area setting screen Sc1 on the display device 96 (S104) and accepts specification of the center position of the work W and the work area, which is the area of ​​the work W (S106). The center position of the work W is given as X-axis and Y-axis coordinate values ​​on an XY plane, with the reference point of the work area setting image Im1 as the origin, the left-right direction of the work area setting image Im1 as the X-axis, and the up-down direction of the work area setting image Im1 as the Y-axis. The work area is, for example, a rectangular area, and the area includes dimensions and inclination. The dimensions are given as the length of the short side (vertical width) and the length of the long side (horizontal width) of the work W in the work area setting image Im1, and the inclination is given as the angle between the X axis and the long side of the work W.

[0029] The work area setting screen Sc1 includes a display field for displaying the work center, dimensions, and tilt. When the worker operates the input device 95 (clicking the mouse at the position where the worker wants to specify the center of the workpiece W) to specify a position on the work area setting image Im1, the CPU 91 displays the specified position as the center position of the workpiece W on the work area setting image Im1 in the display field of the work area setting screen Sc1, as shown in FIG. 5. When the worker operates the input device 95 (clicking one point with the mouse and dragging to another point) to specify two points on the work area setting image Im1, the CPU 91 displays a rectangular area with the two points as diagonal corners. Then, the CPU 91 displays the length of the specified rectangular area in the Y-axis direction and the length of the specified rectangular area in the display field of the work area setting screen Sc1, as shown in FIG. 6. When the worker operates the input device 95 (clicking the displayed rectangular area with the mouse and dragging in the rotation direction) to rotate the rectangular area, the CPU 91 displays the angle at which the rectangular area is rotated in the display field. Furthermore, the work area setting screen includes an OK button that is selected when the worker has completed specifying the work center and work area, and a cancel button that is selected when the worker wants to redo the specification of the work center and work area.

[0030] Next, the CPU 91 determines whether or not the OK button has been selected (S108). If it is determined that the OK button has not been selected, the CPU 91 determines whether or not the Cancel button has been selected (S110). If it is determined that the Cancel button has been selected, the CPU 91 clears the display field (S112) and returns to S106. If it is determined that the Cancel button has not been selected, the CPU 91 returns to S106.

[0031] If it is determined that the OK button has been selected, the CPU 91 calculates the number of pixels on the outer edge of the specified work area as the perimeter A of the work area (S114). The CPU 91 then displays the evaluation condition setting screen Sc2 on the display device 96 (S116) and accepts various settings (S118). These settings include the exposure time, the ultra-high brightness threshold B1, the high brightness threshold B2, the low brightness threshold B3, the ultra-low brightness threshold B4, whether or not to enable an ultra-high brightness alarm, and the ultra-high brightness alarm threshold B5. The exposure time is the exposure time used to capture the evaluation image Im2 in S124 of this process. The thresholds B1 to B4 are thresholds used to display the evaluation image Im2 in different colors in S142 of this process, which will be described later. The whether or not to enable an ultra-high brightness alarm indicates whether or not to notify the user of the fact that the proportion of pixels whose brightness is equal to or greater than the ultra-high brightness threshold B1 is equal to or greater than the ultra-high brightness alarm threshold B5 in S138 and S140 of this process, which will be described later.

[0032] The evaluation condition setting screen Sc2 includes input fields for inputting the exposure time and each threshold value B1 to B5, a check box for selecting whether or not to enable an ultra-high brightness alarm, an OK button that is selected when the operator has finished inputting various information, and an end button that is selected when the operator wishes to end this process.

[0033] Next, the CPU 91 determines whether or not the OK button has been selected (S120). If it is determined that the OK button has not been selected, the CPU 91 determines whether or not the End button has been selected (S122). If it is determined that the End button has been selected, the CPU 91 ends this processing. If it is determined that the End button has not been selected, the CPU 91 returns to S118 again.

[0034] Next, the CPU 91 controls the hand camera 60 to capture an evaluation image Im2 using the exposure time input in S118 (S124). The CPU 91 then applies an edge enhancement filter to the evaluation image Im2 (S126). Examples of edge enhancement filters include a Prewitt filter and a Sobel filter. The CPU 91 calculates the luminance Y based on the R, G, and B gradation values ​​of each pixel and applies the edge enhancement filter to the luminance Y. As a result, the edge portions in the evaluation image Im2 have a higher luminance Y value than the other portions, resulting in a line image of a predetermined thickness. The CPU 91 then extracts the edge O of the workpiece W from the evaluation image Im2 after applying the edge enhancement filter (S128). Specifically, the CPU 91 extracts pixels whose luminance Y is equal to or greater than a predetermined threshold as the edge O of the workpiece W in the evaluation image Im2.

[0035] Next, the CPU 91 detects the length D of the edge O based on the number of pixels within the extracted edge O, as shown in FIG. 4 , and calculates the strength F of the edge O based on the luminance within the edge O (S130). Specifically, the CPU 91 calculates the number of pixels within the edge as the edge length D. The CPU 91 then calculates the average luminance Y of the pixels within the edge O as the strength F of the edge O. The CPU 91 then normalizes the length D and the strength F of the edge O (S132). Specifically, the CPU 91 calculates the normalized length E of the edge O by dividing the length D by the product of the perimeter A' and an integer value N, as shown in Equation (1) below. Here, the perimeter A' is the value obtained by converting the value of the perimeter A into a value obtained when the outer edge of the work area is made the same thickness as the edge O after edge enhancement. The integer value N is an integer value determined experimentally, and is 4 in this embodiment. This is because it was confirmed through experiments that the length D of the edge O that can be detected from the evaluation image Im2 is at most about four times the perimeter A calculated in S114.

[0036] E=D / (N×A')...(1)

[0037] Further, the intensity G of the edge O after normalization is calculated by dividing the intensity F by 255 as shown in the following equation (2). Here, the intensity F of the edge O is divided by 255 because the average maximum value of the luminance Y is 255.

[0038] G=F / 255...(2)

[0039] Next, the CPU 91 determines whether or not an ultra-high brightness alarm has been set (S134). If it is determined that an ultra-high brightness alarm has been set, the CPU 91 calculates the percentage H of pixels in the work area whose brightness Y is equal to or greater than the ultra-high brightness threshold B1 (S136). Next, the CPU 91 determines whether or not the percentage H is equal to or greater than the ultra-high brightness alarm threshold B5 (S138). If it is determined that the percentage H is equal to or greater than the ultra-high brightness alarm threshold B5, the CPU 91 determines that the workpiece W is overexposed in the evaluation image Im2 and that the exposure time when the evaluation image Im2 was captured is inappropriate for performing the workpiece position recognition operation. The CPU 91 then applies hatching to the ultra-high brightness region K, which is a collection of pixels with Y≧B1, and applies hatching different from that of the ultra-high brightness region K, to the high brightness region L, which is a collection of pixels with B2≦Y<B1, and causes the display device 96 to display an evaluation result screen Sc3 (S140), which includes an evaluation image Im2 in which the detected edge O is colored, meters showing the length E and intensity G in red, and a predetermined warning display (exclamation mark, etc.). An example of the evaluation result screen Sc3 at this time is shown in FIG. 8.

[0040] In this way, in the evaluation process, if the proportion H of ultra-high brightness pixels in the evaluation image Im2 is equal to or greater than the ultra-high brightness alarm threshold B5, the CPU 91 determines that the evaluation image Im2 is overexposed and issues an ultra-high brightness alarm, which allows the operator to recognize that the exposure time used to capture the evaluation image Im2 is inappropriate.

[0041] On the other hand, after determining in S134 that the ultra-high brightness alarm has not been set or after determining in S138 that the ratio H is less than the ultra-high brightness alarm threshold B5, the CPU 91 causes the display device 96 to display an evaluation result screen Sc3 (S142) that includes an evaluation image Im2 in which the detected edges are colored, coloring the ultra-high brightness areas red, the high brightness areas yellow, the low brightness areas M where pixels satisfying B4≦Y<B3 cyan, and the ultra-low brightness areas N where pixels satisfying Y≦B4 blue, and meters indicating the length E and intensity G in green. An example of the evaluation result screen Sc3 at this time is shown in FIGS. 9 and 10.

[0042] In this way, in the evaluation process, the edge O detected from the evaluation image Im2 is displayed in color, and the length E of the edge O and the strength G of the edge O are displayed by meters, and are displayed in different colors according to the range of luminance Y. This allows the operator to intuitively grasp whether the exposure time used when capturing the evaluation image Im2 is appropriate.

[0043] After S134 or S136, the CPU 91 waits until the evaluation result screen Sc3 is closed (S144). Next, the CPU 91 displays the evaluation condition setting screen Sc2 on the display device 96 (S146). The CPU 91 then determines whether or not the End button has been selected (S148). If End has not been selected, the CPU 91 returns to S118 shown in FIG. 3 again and accepts input of various conditions. If it is determined that the End button has been selected, the CPU 91 displays an exposure time specification screen (not shown) and accepts from the operator a specification of the exposure time for capturing an image of the workpiece in the workpiece recognition operation (S150), and ends this processing.

[0044] Here, the correspondence between the main elements of the embodiment and the main elements of the present disclosure described in the claims will be described. That is, the control device 90 of the present embodiment corresponds to the recognition device of the present disclosure, the CPU 91 that performs the evaluation process corresponds to the image processing device, the CPU 91 that performs the process of S128 of the evaluation process corresponds to the detection unit in the image processing device of the present disclosure, and the CPU 91 that performs the processes of S140 and S142 of the evaluation process corresponds to the output unit in the image processing device of the present disclosure. The hand camera 60 corresponds to the camera in the image processing device of the present disclosure. Furthermore, the CPU 91 that performs the process of S124 of the evaluation process corresponds to the image acquisition unit. Furthermore, the robot device 10 corresponds to the robot in the robot control system of the present disclosure, the control device 90 and the hand camera 60 correspond to the robot control system of the present disclosure, the hand camera 60 corresponds to the camera in the robot control system of the present disclosure, the CPU 91 that performs the processing of S124 of the evaluation process corresponds to the image acquisition unit in the robot control system, the CPU 91 that performs the processing of S150 of the evaluation process corresponds to the reception unit in the robot control system, and the CPU 91 that performs the work position recognition operation corresponds to the recognition unit in the robot control system.

[0045] It goes without saying that the present disclosure is not limited to the above-described embodiments, and can be implemented in various forms as long as they fall within the technical scope of the present disclosure.

[0046] For example, in the above-described embodiment, the robot device 10 is configured as a horizontal articulated robot (SCARA robot), but is not limited to this and may be any other type of robot device, such as a vertical articulated robot, as long as a camera is attached to a movable part such as an arm. Also, the camera may be fixed so as to be able to capture an image of an area including the work supply unit.

[0047] In the above-described embodiment, the image processing device (control device 90) is applied to the robot system 1. However, the image processing device may be applied to anything that performs edge detection on an image of an object and recognizes the object, and may be applied to, for example, an inspection system that uses an image to inspect whether an object is good or bad.

[0048] In the above-described embodiment, the control device 90 detected the edge O based on the luminance of each pixel in the evaluation image Im2 in S126 of the evaluation process. However, if the workpiece W or the background is captured in color, the control device 90 may detect the edge O based on the saturation of each pixel in the evaluation image Im2. In this case, the CPU 91 may calculate the saturation S based on the R, G, and B gradation values ​​of each pixel in the evaluation image Im2 in S126 of the evaluation process and apply an edge enhancement filter to the saturation S. In S128 of the evaluation process, the CPU 91 can extract pixels whose saturation S is equal to or greater than a predetermined threshold as the edge O of the workpiece W in the evaluation image Im2.

[0049] In the above-described embodiment, the evaluation process is executed prior to the work position recognition operation to evaluate the exposure time when the work W is imaged by the hand camera 60. However, the evaluation process may be executed prior to the collection confirmation operation to evaluate the exposure time when the work W is imaged by the work camera 70.

[0050] In the above-described embodiment, the presence or absence of an ultra-high brightness alarm is input by the operator. However, the ultra-high brightness alarm may be set regardless of whether or not the operator has input the information. Alternatively, the control device 90 may not issue an ultra-high brightness alarm.

[0051] In the above-described embodiment, when reporting an ultra-high brightness alarm, the CPU 91 displays the evaluation image Im2 in which the ultra-high brightness region K and the high brightness region L are hatched differently. However, the CPU 91 may display the evaluation image Im2 in which the ultra-high brightness region K and the high brightness region L are colored differently. Alternatively, the CPU 91 may display the evaluation image Im2 in which the ultra-high brightness region K and the high brightness region L are distinguished by shades of the same color.

[0052] In the above-described embodiment, the CPU 91 displays the evaluation image Im2 in different colors according to the value of the luminance Y in S140 of the evaluation process. However, the CPU 91 may also display the evaluation image Im2 in S140 of the evaluation process with the same color shaded according to the value of the luminance Y.

[0053] In the above-described embodiment, the ultra-high brightness threshold B1, the high brightness threshold B2, the low brightness threshold B3, the ultra-low brightness threshold B4, and the ultra-high brightness alarm threshold B5 are input by the operator, but each threshold may be determined in advance.

[0054] In the above-described embodiment, the pixels in the evaluation image Im2 are displayed in different colors according to the luminance Y of each pixel. However, the pixels do not have to be displayed in different colors.

[0055] In the above-described embodiment, the exposure adjustment value of the present disclosure is the exposure time when capturing an image with the hand camera 60. However, the exposure adjustment value of the present disclosure may be at least one of the exposure time and the aperture value.

[0056] The image processing device of the present disclosure described above displays the detected edges and the evaluation values ​​of the lengths and strengths of the detected edges together with the color image, allowing the operator to determine the appropriateness of the exposure adjustment value based on the length and strength of the edges.

[0057] The image processing device of the present disclosure may also include an exposure adjustment value setting unit that sets an exposure adjustment value when capturing an image of an object with a camera. In this case, the image capturing unit may also include an image capturing unit that controls the camera to capture an image of the object using the exposure adjustment value set by the exposure adjustment value setting unit and captures the image. In this way, an evaluation value is output based on the image captured using the set exposure adjustment value.

[0058] In the image processing device of the present disclosure, the detection unit may perform edge enhancement prior to the edge detection, and the output unit may include a calculation unit that calculates the length of the edge based on the number of pixels of the edge extracted by the edge enhancement and calculates the strength based on the luminance of the extracted edge. In this way, evaluation values ​​of the edge length and edge strength can be calculated relatively easily.

[0059] In the image processing device of the present disclosure, the output unit may display the image used for the evaluation by dividing low-brightness areas and high-brightness areas into different colors, thereby allowing the operator to periodically determine whether the exposure adjustment value used to capture the image is appropriate.

[0060] In the image processing device of the present disclosure, the output unit may determine that the image has overexposed highlights when the ratio of pixels whose luminance exceeds a threshold value exceeds a reference value, and output a warning. This allows an operator to relatively easily recognize overexposed highlights in the image.

[0061] The present disclosure is not limited to the form of an image processing device, but may also be in the form of an image processing method or a robot control system.

[0062] This specification also discloses the technical idea of ​​changing "the image processing device according to any one of claims 1 to 3" in claim 5 as originally filed to "the image processing device according to any one of claims 1 to 4." Also, it discloses the technical idea of ​​changing "the image processing device according to any one of claims 1 to 3" in claim 6 as originally filed to "the image processing device according to any one of claims 1 to 5."

[0063] The present disclosure is applicable to the manufacturing industry of image processing devices and the like.

[0064] 1 Robot system, 2 Work table, 10 Robot device, 11 Base, 20 Robot arm, 21 First arm, 22 Second arm, 23 Shaft, 24 Work holder, 30 First arm drive unit, 32 Motor, 34 Encoder, 40 Second arm drive unit, 42 Motor, 44 Encoder, 50 Shaft drive unit, 52a Motor, 52b Motor, 54a Encoder, 54b Encoder, 60 Hand camera, 70 Work camera, 90 Control device, 91 CPU, 92 ROM, 93 RAM, 94 Storage, 95 Input device, 96 Display device, A Circumference, B1 Ultra-high brightness threshold, B2 High brightness threshold, B3 Low brightness threshold, B4 Ultra-low brightness threshold, B5 Ultra-high brightness alarm threshold, C Substrate, F Intensity, G Intensity, H Ratio, Im1 Work area setting image, Im2 Evaluation image, J1 First joint axis, J2 second joint axis, J3 third joint axis, K ultra-high brightness area, L high brightness area, M low brightness area, N ultra-low brightness area, O edge, P work supply unit, Sc1 work area setting screen, Sc2 evaluation condition setting screen, Sc3 evaluation result screen, W work, Y brightness.

Claims

1. An image processing device applicable to a recognition device that performs image processing including edge detection on an image obtained using a specified exposure adjustment value and recognizes an object identified by a detected edge, comprising: an area setting unit that accepts the setting of an area including the object within the image; a detection unit that performs edge detection based on brightness or saturation within the area set by the area setting unit; and an output unit that displays an image of the object including the detected edge and an evaluation value based on the length and strength of the detected edge.

2. An image processing device according to claim 1, comprising an exposure adjustment value setting unit that sets an exposure adjustment value when capturing an image of an object with a camera.

3. An image processing device according to claim 2, further comprising an image acquisition unit that acquires an image by controlling a camera to capture an image of an object using the exposure adjustment value set by the exposure adjustment value setting unit.

4. An image processing device according to any one of claims 1 to 3, wherein the detection unit performs edge enhancement prior to the edge detection, and the output unit includes a calculation unit that calculates the length of the edge based on the number of pixels of the edge extracted by the edge enhancement, and calculates the intensity based on the luminance or saturation of the extracted edge.

5. An image processing device according to any one of claims 1 to 3, wherein the output unit displays the image used for the evaluation by dividing the image into low brightness areas and high brightness areas by color.

6. An image processing device according to any one of claims 1 to 3, wherein the output unit determines that the image has whiteout when the ratio of pixels whose luminance exceeds a threshold value exceeds a reference value, and outputs a warning.

7. An image processing method used to capture an image of an object with a camera using a specified exposure adjustment value, perform image processing including edge detection on the obtained image, and recognize an object identified by a detected edge, comprising: a setting step of setting an exposure adjustment value when capturing an image of the object with a camera; an image acquisition step of capturing an image of the object with the camera using the set exposure adjustment value, an edge detection step of performing edge detection on each pixel within a predetermined range of the image; an evaluation value derivation step of deriving an evaluation value based on the length and strength of the detected edge; and an exposure adjustment value determination step of determining an exposure adjustment value when capturing an image to be used in recognizing the object, based on the derived evaluation value.

8. A robot control system that recognizes a workpiece and controls a robot that performs work on the workpiece, comprising: a camera that images the workpiece using a specified exposure adjustment value; an exposure adjustment value setting unit that sets the exposure adjustment value when capturing an image of the workpiece with the camera; an image capture unit that controls the camera to capture an image of the workpiece using the set exposure adjustment value and captures the image; a detection unit that performs edge detection on each pixel within a predetermined range of the image; an output unit that outputs an evaluation value based on the length and strength of the detected edge; a reception unit that receives specification of the exposure adjustment value to be used for capturing the image of the workpiece; and a recognition unit that performs edge detection from the image captured with the specified exposure adjustment value, and recognizes objects in the image based on the recognized edges.

Images