Image processing apparatus

Abstract

An image processing device detects a feature portion in a component using an image obtained by capturing an image of the component having a main body and a plurality of feature portions provided on a lower surface of the main body, the image processing device comprising: an image acquisition unit that acquires a plurality of images obtained by capturing the component a plurality of times by irradiating the component with light according to a plurality of illumination conditions; the plurality of illumination conditions include a first illumination condition using a first light source capable of irradiating light to an element at a predetermined position from a side and a second illumination condition using a second light source capable of irradiating light to an element at a predetermined position from a lower side; a first detection unit that detects a first feature portion, which is a part of a plurality of feature portions of the component, using a first image obtained by capturing the component under the first illumination condition; and a second detection unit that detects, using a second image obtained by capturing an image of the component under a second illumination condition, a second feature portion that is a part of a plurality of feature portions of the component and that has a distance from the first light source that is closer than the distance from the first feature portion to the first light source at the time of capturing the image.

Description

TECHNICAL FIELD

[0001] The present specification discloses an image processing apparatus. BACKGROUND

[0002] Conventionally, an image processing apparatus is known, which is applied to a component mounting machine that mounts components on a substrate, and which performs inspection of components by using images obtained by photographing components from a lower surface side under a plurality of illumination conditions in which light is irradiated from different illuminations. For example, Patent Literature 1 discloses an image processing apparatus that detects a shift of a component with respect to a suction nozzle by using an image obtained by photographing a component sucked by the suction nozzle in a state in which light is irradiated from above, and that detects a bump on the component by using an image obtained by photographing the component in a state in which light is irradiated from the side. PRIOR ART DOCUMENTS PATENT LITERATURE

[0003] Patent Literature 1: Japanese Patent Application Publication No. 2005-107716 SUMMARY PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] However, in the image obtained by photographing a component from below in a state in which light is irradiated from the side, a bump that is close to a light source that irradiates light sometimes strongly reflects light, and the outline is imaged to be expanded. In this case, if the image is used to detect the bump, sufficient detection accuracy cannot be obtained.

[0005] The main object of the present disclosure is to improve detection accuracy of a feature in a component. MEANS FOR SOLVING PROBLEMS

[0006] An image processing apparatus of the present disclosure detects a feature in a component using an image obtained by photographing the component, the component having a main body and a plurality of features provided on a lower surface of the main body, and the gist of the image processing apparatus is to include: an image acquisition section that acquires a plurality of images obtained by photographing the component a plurality of times according to a plurality of illumination conditions in which light is irradiated to the component, the plurality of illumination conditions including a first illumination condition and a second illumination condition, the first illumination condition using a first light source that can irradiate light to a component at a predetermined position from the side, and the second illumination condition using a second light source that can irradiate light to the component at the predetermined position from the lower side; a first detection section that detects a first feature, which is a part of the plurality of features of the component, using a first image obtained by photographing the component under the first illumination condition; and The second detecting section detects a second feature portion using a second image obtained by capturing the component under the second lighting condition, the second feature portion being a part of the plurality of feature portions of the component and being closer to the first light source than the first feature portion is to the first light source at the time of capturing.

[0007] In the image processing apparatus, in a first image obtained by capturing the component under a first lighting condition, an outline of the second feature portion is imaged to be easily expanded in a predetermined direction compared to an outline of the first feature portion. This is because the second feature portion is closer to the first light source than the first feature portion is to the first light source, and the second feature portion strongly reflects light from the first light source in the predetermined direction compared to the first feature portion. Therefore, when the first image is used to detect the first feature portion and the second feature portion, the second feature portion is sometimes not detected with sufficient accuracy because the second feature portion is imaged to be expanded in the predetermined direction. In a second image obtained by capturing the component under a second lighting condition, the outline of the second feature portion is not imaged to be expanded in the predetermined direction compared to the outline of the first feature portion as in the first image. This is because the distance from the second light source to each feature portion is substantially equal at the time of capturing, and the second feature portion does not strongly reflect light from the second light source in the predetermined direction compared to the first feature portion. In the image processing apparatus, since the second feature portion is detected using the second image in which the second feature portion is not expanded in the predetermined direction compared to the first image, the detection accuracy of the feature portion can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a perspective view showing the outline structure of a component mounting machine 10. Figure 2A is a longitudinal sectional view before the electronic component C is mounted to the substrate S. Figure 2B is a longitudinal sectional view after the electronic component C is mounted to the substrate S. Figure 3 is a perspective view of the side light source 26. Figure 4 is a longitudinal sectional view of the lower light source 28. Figure 5 is a block diagram showing the electrical connection relationship of the component mounting system 1. Figure 6 is a flowchart showing an example of a component mounting processing routine. Figure 7A is an explanatory diagram showing an example of a side irradiation image Im1. Figure 7B is an explanatory diagram showing an example of a lower irradiation image Im2. Figure 8 is a flowchart showing an example of a temporary center calculation processing subroutine. Figure 9 is a flowchart showing an example of the center calculation processing subroutine. Figure 10A is an explanatory diagram showing an example of the binary image Im4. Figure 10B is an explanatory diagram showing the position of the center Q of the reference point with respect to the binary image Im4. DETAILED DESCRIPTION

[0009] Next, a mode for carrying out the present disclosure will be described with reference to the drawings. Figure 1 is a perspective view showing the outline structure of the component mounting machine 10. Figure 2A 、 2B is a longitudinal sectional view of the electronic component C and the substrate S. Figure 3 is a perspective view of the side light source 26. Figure 4 is a longitudinal sectional view of the lower light source 28. Figure 5 is a block diagram showing the electrical connection relationship of the component mounting system 1. In addition, in Figure 1 、 3 , the left-right direction is taken as the X-axis direction, the front-rear direction is taken as the Y-axis direction, and the up-down direction is taken as the Z-axis direction.

[0010] As shown in Figure 1 , the component mounting system 1 is provided with the component mounting machine 10 and the management device 70. The component mounting machine 10 takes out the electronic component C supplied from the component supply device 21 and mounts it to the substrate S. As shown in Figure 1 , the component mounting machine 10 is provided with the component supply device 21, the conveyance device 22, the head moving device 40, the head 50, the part camera 24, the mark camera 25, the side light source 26 (see Figure 3 ), the lower light source 28 (see Figure 4 ), the scrap box 38, and the control device 60 (see Figure 5 ). For example, as shown in Figure 2A 、 2B , the electronic component C has a main body B that is rectangular in plan view and a plurality of pins P arranged at a predetermined interval on the back surface of the main body B. In the present embodiment, the reflectance of the pin P when irradiated with light is higher than the reflectance of the back surface of the main body B. Therefore, in an image obtained by photographing the electronic component C from the back surface side of the main body B, the luminance value of the region corresponding to the pin P is higher than the luminance value of the surrounding region. Therefore, the pin P is a characteristic portion that represents a characteristic of the electronic component C in the image of the electronic component C.

[0011] Examples of component supply devices 21 include a tray supply device 21a that supplies trays containing multiple storage bags for electronic components C, and a belt feeder 21b that feeds a belt containing multiple chambers for electronic components C. In the tray supply device 21a of this embodiment, one tray contains electronic components C of the same type. Therefore, each electronic component C contained in the same tray has the same shape data. Similarly, in the belt feeder 21b of this embodiment, multiple chambers provided on one belt contain electronic components C of the same type. Therefore, each electronic component C contained in the same belt has the same shape data. Furthermore, the shape data will be explained later.

[0012] The transport device 22 transports the substrate S, which is the object to be inserted, from left to right by driving a pair of conveyor belts. The transport device 22 has, for example, a pair of conveyor belts that are arranged at predetermined intervals along the front-to-back (Y-axis direction) and erected along the left-to-right (X-axis direction).

[0013] The head moving device 40 causes the head 50 to move back and forth and left and right (in the XY axis direction), such as Figure 1 As shown, it includes an X-axis slider 42 and a Y-axis slider 44. The X-axis slider 42 is supported on a pair of upper and lower X-axis guide rails 43 that are arranged on the front surface of the Y-axis slider 44 in a manner extending in the left-right direction (X-axis direction). The X-axis slider 42 is connected to an X-axis actuator 46 (see reference). Figure 5 Driven by the X-axis actuator 48, the Y-axis slider 44 moves along the X-axis guide rail 43 in the X-axis direction. The Y-axis slider 44 is supported on a pair of left and right Y-axis guide rails 45 extending in the front-rear direction (Y-axis direction) of the upper section of the housing 12. The Y-axis slider 44 is driven by the Y-axis actuator 48 (see reference 48). Figure 5 Driven by the X-axis position sensor 47, the X-axis slider 42 moves along the Y-axis guide rail 45 in the Y-axis direction. Additionally, the X-axis slider 42 is controlled by the X-axis position sensor 47 (see reference 48). Figure 4 The position in the X-axis direction is detected. Additionally, the Y-axis slider 44 is connected to the Y-axis position sensor 49 (see reference). Figure 5 The head 50 is mounted on the X-axis slider 42. Therefore, by driving the head moving device 40 (X-axis actuator 46 and Y-axis actuator 48), the head 50 moves along the XY plane (horizontal plane).

[0014] The head 50 has a suction nozzle 51 for picking up (adsorbing) and holding the electronic component C. Although not shown, the suction nozzle 51 is connected to a negative pressure source via a solenoid valve (switching valve), and receives a negative pressure supply from the negative pressure source to adsorb the electronic component C. Additionally, the suction nozzle 51 is connected to a Z-axis actuator 52 (see reference 52). Figure 5 Driven by the Z-axis position sensor 53, the nozzle 51 moves along the vertical direction (Z-axis direction). Figure 5Detect the position in the Z-axis direction.

[0015] When the component camera 24 picks up the electronic component C supplied by the component supply device 21 at the head 50 and mounts (inserts) it onto the substrate S transported by the transport device 22, it takes a picture of the electronic component C from below as the electronic component C, which is attracted by the nozzle 51, passes above the component camera 24. Figure 1 As shown, the component camera 24 is positioned between the component supply device 21 and the conveying device 22. The images captured by the component camera 24 are output to the control device 60.

[0016] The marking camera 25 takes pictures from above of the substrate S, which is moved in by the conveying device 22, and the electronic components C, which are supplied by the component supply device 21. For example... Figure 1 As shown, the marking camera 25 is mounted on the X-axis slider 42 and moves along the XY axis with the head 50 via the head moving device 40. The images captured by the marking camera 25 are output to the control device 60.

[0017] like Figure 3 As shown, the side light source 26 illuminates the electronic component C (the front end of pin P) from the side when the component camera 24 captures an image of the electronic component C. The side light source 26 is directed towards the optical axis 36a of the component camera 24 (see reference). Figure 4 The laser is irradiated in an orthogonal direction.

[0018] like Figure 4As shown, the lower light source 28 irradiates light from below (directly below or diagonally below) the electronic component C when the image of the electronic component C is captured by the component camera 24. The lower light source 28 has a housing 29, a connecting portion 30, a downlight 31, and a side light 34. The housing 29 is a bowl-shaped member with an open upper surface and a lower surface. The connecting portion 30 is a cylindrical member that connects the housing 29 and a camera body 36. Light irradiated from the downlight 31 and light received by the camera body 36 pass through the internal space of the connecting portion 30. The downlight 31 is used to irradiate light from directly below the electronic component C held by the suction nozzle 51. The downlight 31 has a half mirror 33 and a plurality of LEDs 32 that irradiate light toward the half mirror 33 in a direction (horizontal direction) perpendicular to the optical axis 36a. The plurality of LEDs 32 are installed to the inner peripheral surface of the connecting portion 30. The half mirror 33 is disposed inside the connecting portion 30 so as to be inclined by 45° with respect to the optical axis 36a. The half mirror 33 reflects light from the LEDs 32 in the horizontal direction upward. Thus, the downlight 31 irradiates light from directly below the electronic component C. In addition, the half mirror 33 transmits light from above toward the camera body 36. The side light 34 is used to irradiate light from diagonally below the electronic component C held by the suction nozzle 51. The side light 34 has a plurality of LEDs 35a installed to the upper section of the inner peripheral surface of the housing 29, a plurality of LEDs 35b installed to the middle section, and a plurality of LEDs 35c installed to the lower section. In the following description, the LEDs 35a to 35c are collectively referred to as LEDs 35 when no particular distinction is required.

[0019] The disposal box 38 is a box for disposing of the electronic component C that has occurred an abnormality. The disposal box 38 is disposed between the component supply device 21 and the conveyance device 22 adjacent to the component camera 24.

[0020] As Figure 5As shown, the control device 60 is configured as a microprocessor centered on a CPU 61. In addition to the CPU 61, it also includes a ROM 62, a storage device (e.g., HDD, SSD) 63, RAM 64, and an input / output interface 65. These are electrically connected via a bus 66. Position signals from the X-axis position sensor 47, Y-axis position sensor 49, and Z-axis position sensor 53 are input to the control device 60. Image signals from the part camera 24 and the marking camera 25 are also input to the control device 60. Furthermore, the control device 60 uses the image input from the part camera 24 to calculate the adsorption offset of the electronic component C adsorbed by the suction nozzle 51 and to calculate the positional offset (bending amount) of each pin P relative to the body B of the electronic component C. On the other hand, the control device 60 outputs drive signals to the component supply device 21, the conveying device 22, the X-axis actuator 46, the Y-axis actuator 48, and the Z-axis actuator 52. Additionally, the control device 60 outputs control signals to the part camera 24, the marking camera 25, the side light source 26, and the bottom light source 28. The storage device 63 stores shape data.

[0021] like Figure 5 As shown, the management device 70 is configured as a microprocessor centered on a CPU 71, and in addition to the CPU 71, it also includes a ROM 72, a storage device 73, and a RAM 74. The management device 70 is communicatively connected to the control device 60. The storage device 73 stores shape data, production plans, etc., for each electronic component C mounted on the substrate S. The shape data stores information required for mounting the electronic component C on the substrate S, such as the shape of the electronic component C, the number of pins P, the type of electronic component C mounted on the substrate S, the design relative position of each pin P based on the representative position of the main body B in the electronic component C (e.g., the center Q of the main body B) (e.g., the center position of the outline of each pin P when viewed from below), i.e., the pin relative design position R (in the following description, the pin relative design position R of the h-th pin P is referred to as the h-th pin relative design position Rh (h is an integer greater than or equal to 1 and less than or equal to the number of pins P)), and the information required for calculating the positional offset of each pin P relative to the main body B. The production plan stores data on which electronic components C are mounted on the substrate S in the component mounting machine 10 in what order, and how many substrates S (products) are to be manufactured in this manner.

[0022] Next, the operation of the component mounting system 1 configured as described will be explained. First, using... Figure 6 to Figure 1 0. The component mounting process performed by the CPU 61 of the component mounting machine 10 will be described. Figure 6This is a flowchart illustrating an example of a component mounting process. This process is executed by the CPU 61 of the control device 60 after the management device 70 inputs the production start instruction, shape data, and production plan, and stores the shape data and production plan in the storage device 63. Furthermore, in this embodiment, an electronic component C having pins P101 to P104 disposed approximately at the center of the back surface of the main body B, and pins P201 to P212 disposed on the back surface of the main body B at a location peripheral to the outer edge of pins P101 to P104, will be described as an example.

[0023] When this routine begins, CPU 61 causes nozzle 51 to pick up electronic component C supplied from component supply device 21 (S100). Specifically, CPU 61 controls head moving device 40 (X-axis actuator 46 and Y-axis actuator 48) to move nozzle 51 above the supply position of electronic component C supplied by component supply device 21, then controls Z-axis actuator 52 to lower nozzle 51, and controls solenoid valve to supply negative pressure to nozzle 51.

[0024] Next, CPU 61 illuminates the side light source 26 (S102) and uses the part camera 24 to photograph the lower surface of the electronic component C adsorbed by the nozzle 51 (S104). Specifically, CPU 61 first controls the head moving device 40 to move the nozzle 51, which adsorbs the electronic component C, upwards towards the part camera 24. Next, CPU 61 controls the Z-axis actuator 52 to lower the front end of the pin P of the electronic component C to a position illuminated by the light from the side light source 26. Then, CPU 61 controls the part camera 24 to photograph the electronic component C from the lower surface side. In this embodiment, the image of the electronic component C captured is called the side-illuminated image Im1. Figure 7A An example of a side-lit image Im1 is shown. In the side-lit image Im1, as... Figure 7A As shown, pins P201 to P212 will be photographed as being more tilted and expanded than they actually are. Additionally, in Figure 7A In the image, pins P201 to P212 are shown in dashed lines when captured in their original position and size. This is because when capturing the side-lit image Im1, pins P201 to P212 are closer to the side light source 26 than pins P101 to P104, thus reflecting the light from the side light source 26 more strongly. Furthermore, the CPU 61 turns off the side light source 26 (S106). In the following description, pins P101 to P104 are referred to as first pins P1, and pins P201 to P212 are referred to as second pins P2. Moreover, the CPU 71 of the management device 70 stores the information about whether a particular pin P is first pin P1 or second pin P2 in shape data based on operator input.

[0025] Next, the CPU 61 turns on the LEDs 32 of the downlight 31 and the LEDs 35a to 35c of the sidelight 34 in the lower light source 28 (S108), and captures the lower surface of the electronic component C held by the suction nozzle 51 with the part camera 24 (S110). In the present embodiment, the captured image of the electronic component C is referred to as a down irradiation image Im2. In this case, the light from the LEDs 32, 35b, 35c is roughly equally irradiated to the pins P201 to P212 and the pins P101 to P104 compared to when the sidelight 26 is turned on and the image is captured. However, the pins P201 to P212 strongly reflect the light from the LED 35a compared to the pins P101 to P104. This is because the pins P201 to P212 are closer to the LED 35a compared to the pins P101 to P104. Therefore, in the down irradiation image Im2, as shown in FIG. 12, the region of the second pin P2 is not captured in a state of being obliquely expanded as in the image sidelight image Im1, but the luminance value is higher compared to the region of the first pin P1. In addition, in the down irradiation image Im2, as shown in FIG. 12, the main body B of the electronic component C is captured together with the pins P. This is because the light emitted from the lower light source 28 is reflected on the lower surface of the main body B. Then, the CPU 61 turns off the lower light source 28 (S112). Figure 7B Figure 7B

[0026] Next, the CPU 61 executes the temporary center position calculation processing subroutine shown in FIG. 13, and calculates a temporary center position of the electronic component C with respect to a reference point of the image (for example, the left front corner of the image), that is, a temporary center M (S114), and executes the center calculation processing subroutine shown in FIG. 14, and calculates a center Q of the electronic component C with respect to the reference point of the image (S116). The temporary center position calculation processing subroutine and the center calculation processing subroutine will be described later. In addition, the temporary center M and the center Q are calculated as X-axis coordinate values and Y-axis coordinate values on an XY plane with the reference point of the image as an origin O, with the left-right direction of the image as an X-axis, and with the front-back direction of the image as a Y-axis. Then, the CPU 61 sets the value of the pin number h to 1 (S118). Figure 8 Figure 9

[0027] ​​​​Next, the CPU 61 obtains the hth pin relative design position Rh (ΔXrh, ΔYrh) with the center Q of the electronic component C as a reference from the shape data (S120). Next, the CPU 61 converts the hth pin relative design position Rh with respect to the center Q of the electronic component C to the hth pin design position Ah (Xah, Yah) with respect to the reference point (origin O) of the image based on the center Q with respect to the reference point (origin O) of the image and the obtained hth pin relative design position Rh (S122). When the hth pin Ph is arranged at the hth pin relative design position Rh as specified in the shape data, the hth pin design position Ah is the design center position of the hth pin Ph with respect to the reference point of the image. Specifically, the CPU 61 adds the X-axis relative value ΔXrh of the hth pin relative design position Rh to the X-axis coordinate value Xq of the center Q of the electronic component C with respect to the reference point of the image to calculate the X-axis coordinate value Xah of the hth pin design position Ah (Xq + ΔXrh). Also, the CPU 61 adds the Y-axis relative value ΔYrh of the hth pin relative design position Rh to the Y-axis coordinate value Yq of the center Q to calculate the Y-axis coordinate value Yah of the hth pin design position Ah (Yq + ΔYrh). Next, the CPU 61 obtains the hth pin actual position Dh (Xdh, Ydh) which is the center position of the hth pin Ph with respect to the reference point of the image (S124). The hth pin actual position Dh is the position of the pin P corresponding to the hth pin Ph with respect to the reference point of the image among the first pin actual position D and the second pin actual position U calculated in the temporary center calculation processing subroutine or the center calculation processing subroutine described later.

[0028] Also, the CPU 61 calculates the position offset δ of the hth pin Ph (hereinafter, hth pin position offset δh) (S126). Specifically, the CPU 61 subtracts the X-axis coordinate value Xdh of the hth pin actual position Dh from the X-axis coordinate value Xah of the hth pin design position Ah to calculate the X-axis direction position offset ΔXh (Xah - Xdh). Also, the CPU 61 subtracts the Y-axis coordinate value Ych of the hth pin actual position Dh from the Y-axis coordinate value Yah of the hth pin design position Ah to calculate the Y-axis direction position offset ΔYh (Yah - Ydh).

[0029] Next, the CPU 61 determines whether the hth pin positional displacement δh is within the allowable range (S128). If both the X-axis directional positional displacement ΔXh and the Y-axis directional positional displacement ΔYh are within the predetermined range, the CPU 61 determines that the hth pin positional displacement δh is within the allowable range, and increments the value of the pin number h by 1 (S130). Next, the CPU 61 determines whether the value of the pin number h is greater than the number of pins P (16 in the present embodiment) (S132). If it is determined that the value of the pin number h is 16 or less, the CPU 61 determines that there are still pins P for which the positional displacement δ should be calculated, and returns to S120 again to calculate the positional displacement δ for the other pins P. If it is determined that the value of the pin number h is greater than the number of pins P, the CPU 61 determines that all of the pins P do not have a positional displacement that exceeds the allowable range, and mounts the electronic component C on the substrate S (S134). Specifically, the CPU 61 controls the head moving device 40 so that the center Q of the electronic component C that is being sucked by the suction nozzle 51 is moved above the mounting position of the substrate S. Also, the CPU 61 controls the Z-axis actuator 52 so that the electronic component C is pressed against the substrate S, and controls the electromagnetic valve so that the suction of the electronic component C is released, and ends the present processing. Furthermore, the CPU 61 can also correct the mounting position of the electronic component C in consideration of the positional displacement δ of each pin P in the electronic component C.

[0030] If at least one of the X-axis directional positional displacement ΔXh and the Y-axis directional positional displacement ΔYh is outside the predetermined range, the CPU 61 determines that the hth pin positional displacement δh is outside the allowable range, and discards the electronic component C (S136). Specifically, the CPU 61 controls the head moving device 40 so that the suction nozzle 51 is moved above the discard box 38. Also, the CPU 61 controls the electromagnetic valve so that the suction of the electronic component C is released, and ends the present routine.

[0031] Next, the temporary center calculation processing subroutine will be described. Figure 8 is a flowchart showing an example of the temporary center calculation processing subroutine. This processing is executed by the CPU 61 after S112 of the component mounting processing routine described above.

[0032] When the processing starts, the CPU 61 first performs a binarization process on the side irradiation image Im1 (S200). Specifically, the CPU 61 sets pixels having a pixel luminance value less than a first threshold value to black and sets pixels having a pixel luminance value of a predetermined value or more to white. Here, the first threshold value is a threshold value for distinguishing the pin P from other portions, and a value inputted in advance by an operator is adopted. The first threshold value is included in the shape data. Thus, the leading end portion of the pin P is white, and other portions are black. The image obtained in this way is referred to as a binarization image Im3 (not shown). Next, the CPU 61 detects the first pin P1 from the binarization image Im3 (S202). Specifically, the CPU 61 detects, as the first pin P1, a region included in a first pin detection range that has been determined in advance in the binarization image Im3, in a region in which white pixels are concentrated in the binarization image Im3. The first pin detection range is a range determined experimentally for each kind of electronic component C. Also, the CPU 61 sets the value of the first pin number i to 1 (S204).

[0033] Next, the CPU 61 calculates the actual center position of the i-th first pin P1 (hereinafter, i-th first pin P1i) with respect to a reference point of the binarization image Im3 (for example, the left front corner of the binarization image Im3), that is, the first pin actual position D (hereinafter, i-th first pin actual position Di) (S206). The i-th first pin actual position Di is calculated as an X-axis coordinate value (Xdi) and a Y-axis coordinate value (Ydi) of the center position of the i-th first pin P1i, on an XY plane with the reference point of the binarization image Im3 as an origin O, the left-right direction of the binarization image Im3 as an X-axis, and the front-back direction of the binarization image Im3 as a Y-axis. Next, the CPU 61 increments the value of the first pin number i by 1 (S208). Also, the CPU 61 determines whether the value of the first pin number i is greater than the number of first pins (4 in this embodiment) (S210). If it is determined that the value of the first pin number i is equal to or less than the number of first pins P1, the CPU 61 determines that there is a first pin P1 for which the first pin actual position D must be calculated, and returns to S206 again to calculate the first pin actual position D of the other first pins P1.

[0034] If it is determined that the value of the first pin number i is greater than the number of the first pins P1, the CPU 61 determines that the first pin actual positions D of all the first pins P1 have been calculated, and calculates a provisional center position of the electronic component C, i.e., a provisional center M (Xm, Ym) with respect to the reference point (origin O) of the binarized image Im3, based on the first pin actual positions D (S212). The calculation of the provisional center M can be performed by the least square method. That is, the CPU 61 sets a provisional provisional center of the electronic component C with respect to the reference point of the binarized image Im3, i.e., a provisional provisional center. Next, the CPU 61 determines an i-th first pin provisional position Ei (Xei, Yei) which is a provisional center of the i-th first pin Pi with respect to the reference point when the i-th first pin Pi is present at a position which is set to be apart from the i-th first pin relative design position Ri (i is an integer of 1 or more and 4 (the number of first pins) or less) specified in the shape data by a distance determined with the provisional provisional center as a reference. Further, the CPU 61 calculates the provisional provisional center which minimizes the sum of squares of distances from the first pin actual positions D to the first pin provisional positions E as the provisional center M of the electronic component C with respect to the reference point of the binarized image Im3, and ends the process. Thus, the provisional center M of the electronic component C with respect to the reference point of the binarized image Im3 can be calculated using the first pin actual positions D.

[0035] Next, the center calculation processing subroutine will be described. Figure 9 is a flowchart showing an example of the center calculation processing subroutine. This process is executed by the CPU 61 after the provisional center calculation processing subroutine is executed and the provisional center M of the electronic component C with respect to the image is calculated.

[0036] When the process is started, the CPU 61 first performs a binarization process on the lower irradiation image Im2 (S300). Specifically, the CPU 61 sets a pixel having a pixel luminance value less than a second threshold value to black, and sets a pixel having a pixel luminance value of a predetermined value or more to white. The second threshold value is a threshold value for distinguishing the second pins P2 from other portions, and is a value input by the operator in advance. The second threshold value is, for example, a value which is greater than the luminance value of the region of the first pins P1 and less than the luminance value of the region of the second pins P2 in the lower irradiation image Im2, and is a value determined through experiments. The second threshold value is, for example, set to a value greater than the first threshold value. The second threshold value is included in the shape data. Thus, the leading end portions of the second pins easily become white, and other portions easily become black. The image obtained in this way is referred to as a binarized image Im4. Figure 10A An example of the binarized image Im4 is shown in Figure 10AIn this case, the image of the first pin P1 and the portion of the body B is black, and the image of the second pin P2 is white. The CPU 61 detects a region in which white pixels are concentrated in the binary image Im4 as a candidate for the second pin P2 (S302). Next, the CPU 61 determines whether the number of detected second pin candidates is equal to or greater than the number of second pins P2 in the shape data (S304). If it is determined that the number of second pin candidates is less than the number of second pins P2 in the shape data, the CPU 61 determines that there is an error, outputs the error by displaying an error message or the like on a display device not shown, and proceeds to S136 of the component mounting processing routine, discarding the electronic component C held by the nozzle 51. This is because if the number of detected second pin candidates is less than the number of second pins P2 specified in the shape data, a certain second pin P2 is omitted in the detection of the second pin candidates.

[0037] If it is determined that the number of second pin candidates is equal to or greater than the number of second pins P2 in the shape data, the CPU 61 determines that all of the second pins are included in the detected second pin candidates, and sets the value of the second pin candidate number j to 1 (S306). In addition, due to individual differences in the electronic component C, diffuse reflection of light, or the like, other than the second pins P2, other portions are sometimes detected as second pin candidates. Next, the CPU 61 calculates the center position of the jth second pin candidate (hereinafter, the jth second pin candidate) with respect to the reference point (for example, the front left corner) of the binary image Im4, that is, the second pin candidate position F (hereinafter, the jth second pin candidate position Fj) (S310). The jth second pin candidate position Fj is calculated as an X-axis coordinate value (Xfj) and a Y-axis coordinate value (Yfj) of the jth second pin candidate on an XY plane with the reference point of the binary image Im4 as an origin O, with the left-right direction of the binary image Im4 as an X-axis, and with the front-back direction of the binary image Im4 as a Y-axis. Next, the CPU 61 increments the value of the second pin candidate number j by 1 (S312). The CPU 61 then determines whether the value of the second pin candidate number j is greater than the number of second pin candidates detected in S302 (S314). If it is determined that the value of the second pin candidate number j is equal to or less than the number of second pin candidates detected in S302, the CPU 61 determines that there is still a second pin candidate for which the second pin candidate position F must be calculated, and returns to S310 again to calculate the second pin candidate position F of the other second pin candidate.

[0038] If it is determined that the value of the second pin candidate number j is greater than the number of second pin candidates detected in S302, the CPU 61 determines that the second pin candidate position F has been calculated for all second pin candidates, and sets the temporary center M calculated in the temporary center calculation process with respect to the reference point of the binary image Im3 as the temporary center N (Xn, Yn) with respect to the reference point (origin O) of the binary image Im4 (S316). Next, the CPU 61 sets the value of the second pin number k to 1 (S318). Then, the CPU 61 acquires the kth second pin relative design position Rk (ΔXrk, ΔYrk) of the kth second pin P2 (hereinafter, kth second pin P2k) with respect to the center Q of the electronic component C from the shape data (S320). Also, the CPU 61 calculates the second pin temporary design position G (hereinafter, kth second pin temporary design position Gk) of the kth second pin P2k with respect to the reference point of the binary image Im4, based on the temporary center N and the kth second pin relative design position Rk (S322). The kth second pin temporary design position Gk is a design center position of the kth second pin P2k with respect to the reference point of the binary image Im4 when the kth second pin P2k is disposed according to the kth second pin relative design position Rk. Specifically, the CPU 61 adds the X-axis relative value ΔXrk of the acquired kth second pin relative design position Rk to the X-axis coordinate value Xn of the temporary center N, and calculates the X-axis coordinate value Xgk of the kth second pin temporary design position Gk (Xn + ΔXrk). Also, the CPU 61 adds the Y-axis relative value ΔYrk of the acquired kth second pin relative design position Rk to the Y-axis coordinate value Yn of the temporary center N, and calculates the Y-axis coordinate value Ygk of the kth second pin temporary design position Gk (Yn + ΔYrk).

[0039] Next, the CPU 61 sets the value of the reference distance L to the distance Lmax (S324). Also, the reference distance L will be described later. Furthermore, the distance Lmax is set to be longer than the diagonal length of the main body B in the electronic component C. Then, the CPU 61 sets the value of the second pin candidate number j to 1 (S326). Also, the CPU 61 calculates the distance Ljk from the jth second pin candidate position Fj to the kth second pin temporary design position Gk (S328).

[0040] Next, the CPU 61 determines whether the distance Ljk is shorter than the reference distance L (S330). If it is determined that the distance Ljk is shorter than the reference distance L, the CPU 61 updates the value of the reference distance L to the distance Ljk (S332). Next, the CPU 61 sets the jth second pin candidate position Gj (Xgj, Ygj) to the kth second pin actual position Uk (Xuk, Yuk) (S334). When the value of the second pin candidate number j is 1, since the reference distance L is set to the distance Lmax, the distance Llk is necessarily shorter than the reference distance L. Therefore, the CPU 61 updates the reference distance L to the distance Llk, and sets the first second pin candidate position Gl to the kth second pin actual position Uk. In addition, when the value of the second pin candidate number j is 2 or more, if the distance Ljk is smaller than the reference distance L, the CPU 61 updates the value of the reference distance L to the distance Ljk, and updates the jth second pin candidate position Gj to the kth second pin actual position Uk. By repeatedly performing such processing, the jth second pin candidate position Fj closest to the kth second pin temporary design position Gk is set to the kth second pin actual position Uk.

[0041] After S334, or after it is determined that the distance Ljk is not shorter than the reference distance L, the CPU 61 adds 1 to the value of the second pin candidate number j (S336). Also, the CPU 61 determines whether the value of the second pin candidate number j is greater than the number of second pin candidates (S338). If it is determined that the value of the second pin candidate number j is the number of second pin candidates detected in S302 or less, the CPU 61 determines that there still exists a second pin candidate for which the reference distance L to the kth second pin temporary design position Gk must be calculated, and returns to S328 again to calculate the distance Ljk to the kth second pin temporary design position Gk.

[0042] If it is determined that the value of the second pin candidate number j is greater than the number of second pin candidates detected in S302, the CPU 61 determines that the second pin candidate position Fj closest to the kth second pin temporary design position Gk has been set to the kth second pin actual position Uk, and adds 1 to the value of the second pin number k (S340). Next, the CPU 61 determines whether the value of the second pin number k is greater than the number of second pins P2 (S342). If it is determined that the value of the second pin number k is the number of second pins P2 or less, the CPU 61 determines that there still exists a second pin P2 for which the second pin actual position U must be calculated, and returns to S320 again to acquire the kth second pin relative design position Rk of the other kth second pin P2k.

[0043] Thus, the CPU 61 calculates the distance Ljk of the kth second pin temporary design position Gk from all the second pin candidate positions F by successively adding 1 to the value of the second pin candidate number j and repeatedly executing the processes of S328 to S338. Also, the CPU 61 sets the second pin candidate closest to the kth second pin temporary design position Gk among the plurality of second pin candidates detected in S302 as the kth second pin actual position Uk. Thus, even when a number of second pin candidates greater than the number of second pins P2 are detected in S302, the second pins can be correctly detected therefrom. Further, the CPU 61 can correctly set the second pin actual positions U of all the second pins P2 by successively adding 1 to the value of the second pin number k and repeatedly executing these processes.

[0044] If it is determined that the value of the second pin number k is greater than the number of second pins P2, the CPU 61 calculates the center Q (Xq, Yq) of the electronic component C with respect to the reference point (origin O) of the binarized image from the first pin actual positions D and the second pin actual positions U (S344). This process can be performed by the least squares method. That is, the CPU 61 sets a provisional center of the electronic component C in the binarized image Im4, i.e., a provisional center. Next, the CPU 61 sets the ith first pin provisional position Ei (Xei, Yei) which is a provisional center position of the ith first pin Pi (i is an integer of 1 or more and 4 (the number of first pins Pi) or less) with respect to the reference point of the binarized image Im4 when it is assumed that the ith first pin Pi exists at a position deviating from the ith first pin relative design position Ri with the provisional center as a reference. Next, the CPU 61 determines the jth second pin provisional position Vj (Xvj, Yvj) which is a provisional center position of the jth second pin P2j with respect to the reference point of the binarized image Im4 when it is assumed that the jth second pin P2j exists at a position deviating from the jth pin relative design position Rj (j is an integer of 1 or more and 12 (the number of second pins P2) or less) with the provisional center position as a reference. Also, the CPU 61 calculates the center Q (Xq, Yq) of the electronic component C with respect to the reference point of the binarized image Im4 as shown in FIG. 34 from the provisional center which minimizes the sum of the distance squares from the first pin actual positions D to the first pin provisional positions E and the distance squares from the second pin actual positions U to the second pin provisional positions V, and ends this process. Thus, the center Q of the electronic component C with respect to the reference point of the binarized image Im4 can be calculated using the first pin actual positions D and the second pin actual positions U. In addition, in Figure 10B Figure 10B ​In the above-described embodiment, the CPU 61 calculates the center Q of the electronic component C. However, the CPU 61 can not calculate the center Q with respect to the reference point of the image.

[0045] Here, the correspondence between the constituent elements of the present embodiment and the constituent elements of the present disclosure is made clear. That is, the control device 60 of the present embodiment corresponds to the image processing device of the present disclosure; the CPU 61 that executes the processes of S102 to S106 and S108 to S112 of the component mounting processing routine corresponds to the image acquisition section; the CPU 61 that executes the processes of S200 to S210 of the temporary center calculation processing corresponds to the first detection section; the CPU 61 that executes the process of S212 of the temporary center calculation processing and the processes of S300 to S342 of the center calculation processing corresponds to the second detection section; and the CPU 61 that executes the process of S344 of the center calculation processing corresponds to the component position calculation section.

[0046] Note that the present disclosure is not limited at all to the above-described embodiment, and can obviously be implemented in various forms within the technical scope of the present disclosure.

[0047] In the above-described embodiment, the CPU 61 generates the binarized image Im3 from the side-view image Im1 in the temporary center calculation processing subroutine and detects the first pin P1 from the binarized image Im3. However, the CPU 61 can also directly detect the first pin P1 from the side-view image Im1 by previously setting a range of luminance values corresponding to the color of the first pin P1. In addition, the CPU 61 generates the binarized image Im4 from the bottom-view image Im2 in the center calculation processing subroutine and detects the second pin P2 from the binarized image Im4. However, the CPU 61 can also directly detect the second pin P2 from the bottom-view image Im2 by previously setting a range of luminance values corresponding to the color of the second pin P2.

[0048] In the above-described embodiment, the CPU 61 executes the processes of S302 to S342 of the center calculation processing to calculate the second pin actual position U. However, the position of each second pin candidate with respect to the reference point of the binarized image Im4 can also be set as the second pin actual position U by the same processes as S200 to S210 of the temporary center calculation processing subroutine.

[0049] In the above-described embodiment, the CPU 61 is configured to calculate the center Q. However, the CPU 61 can not calculate the center Q with respect to the reference point of the image.

[0050] In the above-described embodiment, it is configured to set which pin P is the first pin P1 or the second pin P2 based on the operation of the operator. However, it can also be configured to be automatically set by the CPU 71 based on the pin relative design position R. In this case, the CPU 71 of the management device 70 can set the pin P disposed at the outer circumferential side of the main body B among the plurality of pins P to the second pin P2 and the remaining pins P to the first pin P1. Alternatively, the CPU 71 can set the pin P within a predetermined range from the center Q to the first pin P1 and the pin P outside the predetermined range from the center Q to the second pin P2.

[0051] In the above-described embodiment, the CPU 61 captures the electronic component C in a state where the LEDs 32 of the downlight 31 and the LEDs 35a to 35c of the side light 34 are lit to obtain the lower irradiation image Im2. However, it can also be configured to capture the electronic component C in a state where only the LEDs 32 of the downlight 31 are lit to obtain the lower irradiation image Im2 and capture the electronic component C in a state where only the LEDs 35 of the side light 34 are lit to obtain the lower irradiation image Im2.

[0052] In the above-described embodiment, it is configured that the CPU 61 proceeds to S136 of the component mounting processing routine after S308 of the center calculation processing subroutine. However, it can also be configured that the CPU 61 returns to S102 or S108 of the component mounting processing routine after S308 of the center calculation processing subroutine and recaptures the side irradiation image Im1 or the lower irradiation image Im2.

[0053] As described above, in the image processing device of the present disclosure, in a first image obtained by capturing a component in a first illumination condition, a profile of a second feature portion is imaged to be expanded in a predetermined direction more than a profile of a first feature portion. This is because the second feature portion is closer to a first light source than the first feature portion, and the second feature portion reflects light from the first light source in the predetermined direction more strongly than the first feature portion. Therefore, if it is intended to detect the first feature portion and the second feature portion using the first image, the second feature portion is imaged to be expanded in the predetermined direction, and thus the second feature portion cannot be detected with sufficient accuracy. In a second image obtained by capturing the component in a second illumination condition, the profile of the second feature portion is not imaged to be expanded in the predetermined direction more than the profile of the first feature portion as in the first image. This is because the distance from a second light source to each feature portion is substantially equal at the time of capturing, and the second feature portion does not reflect light from the second light source in the predetermined direction more strongly than the first feature portion. In the image processing device, since the second feature portion is detected using the second image in which the second feature portion is not expanded in the predetermined direction compared to the first image, it is possible to improve the detection accuracy of the feature portion.

[0054] In the image processing apparatus of the present disclosure, the second detection section can binarize the second image in a manner that can distinguish the feature section including at least the second feature section from a background including the main body, and detect the second feature section using the binarized image. When the second image of the component is captured in a state where light is irradiated in the second illumination condition, a portion other than the second feature section can be captured, and thus it can be difficult to detect the second feature section from the second image. Therefore, it is significant to binarize the second image in a manner that can distinguish the second feature section from the background.

[0055] In the image processing apparatus of the present disclosure, the first detection section can calculate the position of the first feature section with respect to the first image, the second detection section can calculate a temporary representative position of the component, which is a temporary representative position, based on the position of the first feature section, calculate a candidate position of the position of the second feature section with respect to the second image, which is a candidate position, acquire a design position of the second feature section with respect to the representative position of the component, which is a design position, calculate a design position of the second feature section with respect to the second image based on the temporary representative position and the design position, and calculate the position of the second feature section in the second image based on the candidate position and the design position. In this way, the second feature section can be detected with sufficient accuracy.

[0056] In the image processing apparatus of the present disclosure, the component position calculation section can calculate the representative position of the component based on the position of the first feature section and the position of the second feature section. In this way, since the positions of the respective feature sections with respect to the image can be detected with accuracy, the representative position of the component with respect to the image can be detected with accuracy. Industrial applicability

[0057] The present disclosure can be applied to manufacturing industries and the like of component mounting machines, component mounting systems, and the like. Explanation of reference numerals

[0058] 1 component mounting system, 10 component mounting machine, 12 housing, 21 component supply device, 21a tray supply device, 21b tape feeder, 22 conveyance device, 24 part camera, 25 mark camera, 26 side light source, 28 lower light source, 29 housing, 30 connecting portion, 31 drop illumination, 32 LED, 33 half mirror, 34 side illumination, 35, 35a-35c LED, 36 camera main body, 36a optical axis, 38 scrap box, 40 head moving device, 42 X-axis slide, 43 X-axis guide rail, 44 Y-axis slide, 45 Y-axis guide rail, 46 X-axis actuator, 47 X-axis position sensor, 48 Y-axis actuator, 49 Y-axis position sensor, 50 head, 51 suction nozzle, 52 Z-axis actuator, 53 Z-axis position sensor, 60 control device, 61 CPU, 62 ROM, 63 storage device, 64 RAM, 65 input / output interface, 66 bus, 70 management device, 71 CPU, 72 ROM, 73 storage device, 74 RAM, 101 pin, B main body, C electronic component, Im1 side irradiation image, Im2 lower irradiation image, Im3 binarized image, Im4 binarized image, M temporary center, N temporary center, O second pin temporary design position, P pin, P101-P104, P201-P212, P1 first pin, P2 second pin, Q center, R pin relative design position.

Claims

1. An image processing apparatus for detecting feature portions in a component using an image obtained by capturing an image of the component, the component having a main body and a plurality of feature portions disposed on a lower surface of the main body, the image processing apparatus comprising: The image acquisition unit acquires multiple images obtained by taking multiple pictures of the element under multiple lighting conditions, including a first lighting condition and a second lighting condition. The first lighting condition uses a first light source that can illuminate the element at a predetermined position from the side, and the second lighting condition uses a second light source that can illuminate the element at the predetermined position from below. The first detection unit uses a first image obtained by taking a picture of the element under the first illumination condition to detect a first feature portion, which is a part of a plurality of feature portions of the element; as well as The second detection unit uses a second image obtained by taking a picture of the element under the second illumination condition to detect a second feature portion, which is a part of a plurality of feature portions of the element, and the distance from the second feature portion to the first light source is closer than the distance from the first feature portion to the first light source during the picture taking.

2. The image processing apparatus according to claim 1, wherein, The second detection unit binarizes the second image in a manner that distinguishes the feature containing at least the second feature from the background containing the subject, and uses the binarized image to detect the second feature.

3. The image processing apparatus according to claim 1 or 2, wherein, The first detection unit calculates the position of the first feature relative to the first image; The second detection unit calculates a temporary representative position of the element based on the position of the first feature, calculates a candidate position of the second feature relative to the second image, obtains the design relative position of the second feature relative to the representative position of the element, calculates the design position of the second feature relative to the second image based on the temporary representative position and the relative position, and calculates the position of the second feature in the second image based on the candidate position and the design position.

4. The image processing apparatus according to claim 3, wherein, The image processing apparatus includes a component position calculation unit that calculates a representative position of the component based on the position of the first feature portion and the position of the second feature portion.

Images