Image processing device

Abstract

An image processing device detects a characteristic section in a component comprising a main body and several characteristic sections arranged on a bottom surface of the main body, using an image obtained by imaging the component, and includes an image acquisition section configured to capture multiple images obtained by repeatedly imaging the component by emitting light to the component under multiple illumination conditions, including a first illumination condition using a first light source configured to emit light to a component at a predetermined position from a lateral side, and a second illumination condition using a second light source configured to emit light to the component at the predetermined position from a bottom surface, and a first acquisition section configured toto capture a first characteristic section, which is part of the several characteristic sections of the component, using a first image obtained by imaging the component under the first illumination condition, and a second capture section, configured to capture a second characteristic section, which is part of the several characteristic sections of the component and is closer to the first light source than the first characteristic section during imaging, using a second image obtained by imaging the component under the second illumination condition.

Description

Technical field

[0001] The present description discloses an image processing device. State of the art

[0002] A known image processing device is used in a component assembler for placing a component onto a plate. This device captures an image of the component, which is illuminated from below under various lighting conditions, and inspects the component based on the captured image. For example, patent literature 1 discloses an image processing device that detects a component's deviation from being picked up by a nozzle using an image captured when the component is illuminated from above, and detects unevenness of the component using an image captured when the component is illuminated from the side, in relation to the component picked up by the nozzle. List of patent literature citations

[0003] Patent Literature 1: JP2005-107716 Summary of the invention; Technical task

[0004] In some cases, however, when an image is taken from below while a component is illuminated from the side, a protrusion closer to the light source reflects the light strongly, making its contour appear enlarged. In such a case, sufficient detection accuracy cannot be achieved if the protrusion is identified from an image.

[0005] A primary objective of the present disclosure is to improve the detection accuracy of a characteristic section in a component. Solution to the problem

[0006] An image processing device of the present disclosure is an image processing device for capturing a characteristic section in a component comprising a main body and several characteristic sections arranged on a bottom side of the main body, using an image obtained by imaging the component, and comprises an image acquisition section configured to capture multiple images obtained by repeatedly imaging the component by emitting light onto the component under multiple illumination conditions, including a first illumination condition in which a first light source is used that is configured to emit light onto a component at a predetermined position from a lateral side, and a second illumination condition in which a second light source is used that is configured tothat it emits light onto the component at the predetermined position from a bottom side, a first detection section configured to detect a first characteristic section, which is part of the multiple characteristic sections of the component, using a first image obtained by imaging the component under the first illumination condition, and a second detection section configured to detect a second characteristic section, which is part of the multiple characteristic sections of the component and is closer to the first light source than the first characteristic section during imaging, using a second image obtained by imaging the component under the second illumination condition.

[0007] In the image processing device, in the first image obtained by imaging the component under the first illumination condition, the contour of the second characteristic section appears slightly extended in a predetermined direction compared to the contour of the first characteristic section. This is because the second characteristic section is closer to the first light source than the first, and the light from the first light source is reflected more strongly in the predetermined direction than by the first characteristic section. Therefore, if both the first and second characteristic sections are to be captured using the first image, the second characteristic section appears extended in the predetermined direction, which in some cases prevents it from being captured with sufficient accuracy.In the second image, obtained by imaging the component under the second illumination condition, the contour of the second characteristic section does not appear extended in the predetermined direction compared to the contour of the first characteristic section, as it did in the first image. This is because, during imaging, the distances from the second light source to the respective characteristic sections are essentially the same, and the second characteristic section does not reflect the light from the second light source more strongly in the predetermined direction than the first characteristic section. In this image processing device, the second characteristic section is detected using the second image, in which the second characteristic section is not extended in the predetermined direction compared to the first image, thus improving the detection accuracy of the characteristic section. Brief description of the characters Fig. Figure 1 is a perspective view showing a schematic configuration of the component assembler 10. Fig. 2A is a vertical cross-sectional view before the electronic component C is mounted on the plate S. Fig. Figure 2B is a vertical cross-sectional view after the electronic component C has been mounted on the plate S. Fig. Figure 3 is a perspective view of the side light source 26. Fig. Figure 4 is a vertical cross-sectional view of the lower light source 28. Fig. Figure 5 is a block diagram illustrating an electrical connection relationship of the component assembly system 1. Fig. Figure 6 is a flowchart illustrating an example of an assembly order for the component. Fig. Figure 7A is a diagram illustrating an example of a side emission image Im1. Fig. 7B is a diagram illustrating an example of a lower emission image Im2. Fig. Figure 8 is a flowchart illustrating an example of a temporary subroutine for calculating the center. Fig. Figure 9 is a flowchart illustrating an example of a subroutine for calculating the center. Fig. 10A is a diagram illustrating an example of a binaryized image Im4. Fig. 10B is a diagram illustrating the position of the center Q for a reference point of the binarized image Im4. Description of the embodiments

[0008] Next, embodiments for implementing the invention of the present disclosure will be described with reference to the figures. Fig. Figure 1 is a perspective view illustrating a schematic configuration of the component assembler 10. Fig. 2A and Fig. Figure 2B shows vertical cross-sectional views of the electronic component C and the plate S. Fig. Figure 3 is a perspective view of the side light source 26. Fig. Figure 4 is a vertical cross-sectional view of the lower light source 28. Fig. Figure 5 is a block diagram illustrating an electrical connection relationship of the component assembly system 1. In the Fig. 1 and Fig. 3. The left-right direction is referred to as the X-axis direction, the front-back direction as the Y-axis direction, and the top-bottom direction as the Z-axis direction.

[0009] As in Fig. As shown in Figure 1, the component assembly system 1 comprises a component assembler 10 and a management device 70. The component assembler 10 picks up the electronic component C supplied by the component feeder 21 and mounts the electronic component C onto the plate S. ... Fig. As shown in Figure 1, the component assembler 10 comprises a component feeding device 21, a conveying device 22, a head movement device 40, a head 50, a component camera 24, a marking camera 25, and a side light source 26 (see Figure 1). Fig. 3), a lower light source 28 (see Fig. 4), a disposal container 38 and a control device 60 (see Fig. 5) As in the Fig. 2A and Fig. As shown in Figure 2B, the electronic component C comprises a body B, which has a rectangular shape in plan view, and several pins P arranged at predetermined intervals on the back side of the body B. In the present embodiment, the reflectance of the light-illuminated pin P is higher than the reflectance of the back side of the body B. Accordingly, in an image of the electronic component C taken from the back side of the body B, an area corresponding to the pin P exhibits a higher luminance value than the surrounding area. Therefore, the pin P is a characteristic section that indicates the properties of the electronic component C in the image of the electronic component C.

[0010] The component feeding device 21 can, for example, comprise a tray feeding device 21a, which feeds a tray with many receiving pockets for holding electronic components C, a belt feeder 21b, which feeds a belt with many cavities for holding electronic components C, and the like. In the tray feeding device 21a according to the present embodiment, electronic components C of the same type are received in a tray. Accordingly, electronic components C received in the common tray have common shape data. Furthermore, in the belt feeder 21b according to the present embodiment, electronic components C of the same type are received in the multiple recesses arranged in a belt. Accordingly, electronic components C received in the common belt have common shape data. The shape data are described below.

[0011] The conveyor device 22 conveys the plate S, which serves as the insertion target, from left to right by driving a pair of conveyor belts. For example, the conveyor device 22 comprises a pair of conveyor belts that are installed at a predetermined distance in a front-back direction (Y-axis direction) and laid out in a left-right direction (X-axis direction).

[0012] The head movement device 40 moves the head 50 in the forward-backward and left-right direction (XY axis direction) and comprises an X-axis slider 42 and a Y-axis slider 44, as shown in Fig. Figure 1 shows the X-axis slider 42 being supported by a pair of upper and lower X-axis guide rails 43, which are attached to a front face of the Y-axis slider 44 to extend in the left-right direction (X-axis direction). The X-axis slider 42 moves in the X-axis direction along the X-axis guide rail 43 by driving the X-axis actuator 46 (see Figure 1). Fig. 5) The Y-axis slider 44 is supported by a pair of left and right Y-axis guide rails 45, which are attached to an upper stepped section of the housing 12 to extend in the front-to-back (Y-axis) direction. The Y-axis slider 44 moves in the Y-axis direction along the Y-axis guide rail 45 by driving the Y-axis actuator 48 (see Fig. 5) The position of the X-axis slider 42 in the X-axis direction is detected by the X-axis position sensor 47 (see Fig. 4) In addition, the position of the Y-axis slider 44 in the Y-axis direction is detected by the Y-axis position sensor 49 (see Fig. 5) The head 50 is attached to the X-axis slider 42. Accordingly, the head 50 moves along an XY plane (horizontal plane) by driving and controlling the head movement device 40 (X-axis actuator 46 and Y-axis actuator 48).

[0013] The head 50 includes a suction nozzle 51 that picks up (draws in) and holds the electronic component C. Although not shown, a vacuum source is connected to the suction nozzle 51 via a solenoid valve (on / off valve), and the suction nozzle 51 picks up the electronic component C by drawing in a vacuum supplied by the vacuum source. Furthermore, the suction nozzle 51 moves in an up-down direction (Z-axis direction) by the drive of the Z-axis actuator 52 (see Fig. 5) The position of the suction nozzle 51 in the Z-axis direction is detected by the Z-axis position sensor 53 (see Fig. 5).

[0014] When the head 50 picks up the electronic component C supplied by the component feeder 21 and mounts (inserts) the electronic component C onto the plate S conveyed by the conveyor 22, the component camera 24 picks up the electronic component C from below as the electronic component C, picked up by the suction nozzle 51, passes over the component camera 24. As in Fig. As shown in Figure 1, the component camera 24 is installed between the component feeder 21 and the conveyor 22. The image captured by the component camera 24 is output to the control device 60.

[0015] The marking camera 25 images the plate S transported by the conveyor device 22 and the electronic component C fed by the component feeder 21 from above. As in Fig. As shown in Figure 1, the marking camera 25 is attached to the X-axis slider 42 and moves together with the head 50 in the XY-axis direction via the head movement device 40. An image captured by the marking camera 25 is output to the control unit 60.

[0016] As in Fig. As shown in Figure 3, the lateral light source 26 shines light onto the electronic component C (front end section of the pin P) from the side when the component camera 24 takes an image of the electronic component C. The lateral light source 26 emits laser light in a direction orthogonal to the optical axis 36a (see Figure 3). Fig. 4) the component camera 24.

[0017] As in Fig. As shown in Figure 4, the lower light source 28 shines light from below (directly below or obliquely below the electronic component C) onto the electronic component C when the component camera 24 captures an image of the electronic component C. The lower light source 28 comprises a housing 29, a connecting element 30, an incident light illuminator 31, and a side light illuminator 34. The housing 29 is a shell-shaped element with an open top and bottom. The connecting element 30 is a tubular element that connects the housing 29 to the camera main body 36. The light emitted by the incident light illuminator 31 and the light received by the camera main body 36 pass through an interior space of the connecting element 30. The incident light illuminator 31 shines light directly from below onto the electronic component C, which is held by the suction nozzle 51.The incident light illumination 31 comprises a half-mirror 33 and several LEDs 32, which emit light in one direction (horizontal direction) perpendicular to the optical axis 36a onto the half-mirror 33. The several LEDs 32 are mounted on an inner circumferential surface of the connecting device 30. The half-mirror 33 is positioned within the connecting device 30 such that it is inclined at 45° with respect to the optical axis 36a. The half-mirror 33 reflects the light upwards in a horizontal direction from the LED 32. As a result, the incident light illumination 31 shines light directly from below onto the electronic component C. Additionally, the half-mirror 33 allows light to pass through it from above towards the camera main body 36. The side illumination 34 shines light obliquely from below onto the electronic component C, which is held by the suction nozzle 51.The side lighting 34 comprises several LEDs 35a attached to an upper part, several LEDs 35b attached to a middle part, and several LEDs 35c attached to a lower part, on an inner circumferential surface of the housing 29. Hereinafter, the LEDs 35a to 35c are simply referred to as LEDs 35 unless otherwise specified.

[0018] The disposal container 38 is a container for disposing of electronic components C that exhibit an anomaly. The disposal container 38 is installed next to the component camera 24, between the component feeder 21 and the conveyor 22.

[0019] As in Fig. As shown in Figure 5, the control device 60 is configured as a microprocessor comprising a CPU 61 as its main element. In addition to the CPU 61, it includes a ROM 62, a memory 63 (e.g., an HDD or SSD), a RAM 64, and an input / output interface 65. The elements are electrically interconnected via a bus 66. Position signals from the X-axis position sensor 47, the Y-axis position sensor 49, and the Z-axis position sensor 53 are input to the control device 60. Furthermore, image signals or similar signals from the component camera 24 and the marking camera 25 are also input to the control device 60.Furthermore, the control device 60 calculates a pickup deviation of the electronic component C picked up by the suction nozzle 51, or calculates a position deviation (bending deviation) of each pin P for the body B of the electronic component C using an image input from the component camera 24. Meanwhile, the control device 60 outputs control signals to the component feeder 21, the conveyor 22, the X-axis actuator 46, the Y-axis actuator 48, and the Z-axis actuator 52. In addition, the control device 60 also outputs control signals to the component camera 24, the marking camera 25, the side light source 26, and the bottom light source 28. The memory 63 stores shape data.

[0020] As in Fig. As shown in Figure 5, the management device 70 is configured as a microprocessor comprising a CPU 71 as its main element and, in addition to the CPU 71, also contains a ROM 72, a memory 73, and a RAM 74. The management device 70 is interconnected with the control device 60. The memory 73 stores shape data, a production plan, and the like for each electronic component C to be mounted on the plate S. The shape data stores information required for mounting the electronic component C on the plate S, information required for calculating the positional deviation of each pin P relative to the body B, and the like, such as the outer shape of the electronic component C, the number of pins P, and the type of electronic component C to be mounted on the plate S.The relative design position R of the pins (hereinafter, the relative design position R of the h-th pin P is referred to as the h-th relative design position Rh (h is an integer greater than or equal to 1 and less than or equal to the number of pins P)), which is a relative position in design (for example, a midpoint of a contour of each pin P in a case where the electronic component C is viewed from below) of each pin P with respect to a representative position (for example, center Q of body B) of body B in the electronic component C, and the like. The production plan stores which electronic components C are to be mounted on the board S in the component assembler 10 in which sequence, how many boards S (products) assembled in this way are to be produced, and the like.

[0021] Next, the operation of component assembly system 1, configured as described above, will be described. First, with reference to the Fig. Sections 6 to 10 describe a component assembly process that is carried out by the CPU 61 of the component assembler 10. Fig. Figure 6 is a flowchart illustrating an example of a component assembly process. The present routine is executed by the CPU 61 of the control device 60 after a production start command, mold data, and a production plan have been entered by the management device 70 and the mold data and production plan have been stored in memory 63. In the present embodiment, an example is given of a case in which an electronic component C, comprising pins P101 to P104 arranged substantially in the center of a rear surface of the body B and pins P201 to P212 arranged at an outer edge region of the rear surface of the body B, is mounted on the plate S.

[0022] When the present routine starts, the CPU 61 causes the suction nozzle 51 to pick up the electronic component C supplied by the component feeder 21 (S100). In particular, the CPU 61 controls the head movement device 40 (X-axis actuator 46 and Y-axis actuator 48) so that the suction nozzle 51 moves to a position above a feed position of the electronic component C by the component feeder 21, controls the Z-axis actuator 52 so that the suction nozzle 51 is lowered, and controls a solenoid valve so that a vacuum is applied to the suction nozzle 51.

[0023] Next, the CPU 61 switches on the side light source 26 (S102) and images the underside of the electronic component C, which has been picked up by the suction nozzle 51, with the component camera 24 (S104). Specifically, the CPU 61 first controls the head movement device 40 so that the suction nozzle 51, which has picked up the electronic component C, moves over the component camera 24. Next, the CPU 61 controls the Z-axis actuator 52 so that a front end section of the pin P of the electronic component C is lowered to a position onto which the light emitted by the side light source 26 strikes. Then, the CPU 61 controls the component camera 24 so that the electronic component is imaged from the underside. In the present embodiment, a captured image of the electronic component C is referred to as a side emission image Im1. Fig. Figure 7A shows an example of a side emission image Im1. In the side emission image Im1, as in Fig. Figure 7A shows pins P201 to P212 in a state where they are extended more obliquely than in their actual state. Fig. In Figure 7A, pins P201 to P212, depicted in their original positions and sizes, are represented by dashed lines. This is because pins P201 to P212 are closer to the side light source 26 than pins P101 to P104 and reflect more light from the side light source 26 when the side emission image Im1 is acquired. The CPU 61 then switches off the side light source 26 (S106). Hereafter, pins P101 to P104 are referred to as first pins P1 and pins P201 to P212 as second pins P2. Furthermore, in response to an operator action, the CPU 71 of the management device 70 stores in the shape data which pin P is the first pin P1 or the second pin P2.

[0024] The CPU 61 then switches on the LED 32 of the incident light illumination 31 and the LEDs 35a to 35c of the side illumination 34 in the lower light source 28 (S108) and images the underside of the electronic component C, which was picked up by the suction nozzle 51, with the component camera 24 (S110). In the present embodiment, the captured image of the electronic component C is referred to as the lower emission image Im2. In this case, the light from the LEDs 32, 35b, and 35c is emitted substantially uniformly onto the pins P201 to P212 and the pins P101 to P104, compared to a case in which the side light source 26 is switched on to perform the imaging. However, the pins P201 to P212 reflect the light from LED 35a more strongly than the pins P101 to P104. This is because pins P201 to P212 are closer to LED 35a than pins P101 to P104. Accordingly, in the image Im2, the emission is lower, as shown in Fig. As shown in Figure 7B, an area of ​​the second pin P2 is not depicted in a state where it is obliquely extended with lateral emission as in Figure Im1, but rather exhibits a higher luminance value than an area of ​​the first pin P1. Furthermore, in Figure Im2, the lower emission is shown, as in Fig. Figure 7B shows the body B of the electronic component C together with the pin P. This is because the light emitted by the lower light source 28 is reflected from the underside of body B. Then the CPU 61 switches off the lower light source 28 (S112).

[0025] Next, the CPU 61 introduces a feature in Fig. 8. The temporary center position calculation subroutine shown is executed to calculate the temporary center M, which represents a temporary center of the electrical component C for a reference point (for example, a left front corner of an image) of the image (S114), and performs a Fig. The center calculation subroutine shown in Figure 9 is executed to calculate the center Q of the electrical component C for the image's reference point (S116). The temporary center position calculation subroutine and the center calculation subroutine are described below. The temporary center M and the center Q are calculated as X-axis and Y-axis coordinate values ​​in an XY plane, where an image's reference point is designated as the origin point O, the left-right direction of an image is designated as the X-axis, and the front-back direction of an image is designated as the Y-axis. The CPU 61 then sets the value of pin number h to 1 (S118).

[0026] Next, the CPU 61 determines the relative design position Rh (ΔXrh and ΔYrh) of the h-th pin with respect to the center Q of the electronic component C from the shape data (S120). The CPU 61 then converts the relative design position Rh of the h-th pin for the center Q of the electronic component C into the design position Ah (Xah and Yah) of the h-th pin for a reference point (origin point O) of an image, based on the center Q for the reference point (origin point O) of the image and the acquired relative design position Rh of the h-th pin (S122). In a case where the h-th pin Ph is located at the relative design position Rh of the h-th pin defined in the shape data, the design position Ah of the h-th pin is a center in design of the h-th pin Ph for the reference point of the image described above.Specifically, the CPU 61 adds the X-axis relative value ΔXrh of the h-th relative design position Rh of the pin in the shape data to the X-axis coordinate value Xq of the center Q of the electronic component C for the image reference point to calculate the X-axis coordinate value Xah (Xq+ΔXrh) of the h-th design position Ah of the pin. Furthermore, the CPU 61 adds the Y-axis relative value ΔYrh of the h-th pin relative design position Rh in the shape data to the Y-axis coordinate value Yq of the center Q to calculate the Y-axis coordinate value Yah (Yq+ΔYrh) of the h-th pin design position Ah. Subsequently, the CPU 61 acquires the actual position Dh (Xdh and Ydh) of the h-th pin, which represents a center of the h-th pin Ph for the image reference point (S124).The actual position Dh of the h-th pen is a position for the reference point of the image in pen P, corresponding to the h-th pen Ph, between the first actual position D of the pen and the second actual position U of the pen, which are calculated in a temporary center calculation subroutine or a center calculation subroutine described below.

[0027] The CPU 61 then calculates the position deviation δ (hereinafter referred to as the h-th pin position deviation δh) of the h-th pin Ph (S126). Specifically, the CPU 61 calculates the position deviation magnitude ΔXh (Xah - Xdh) in the X-axis direction by subtracting the X-axis coordinate value Xch of the actual position Dh of the h-th pin from the X-axis coordinate value Xah of the design position Ah of the h-th pin. Furthermore, the CPU 61 calculates the position deviation ΔYh (Yah - Ych) in the Y-direction by subtracting the Y-axis value Ych of the actual position Dh of the h-th pin from the Y-axis value Yah of the design position Ah of the h-th pin.

[0028] Next, the CPU 61 determines whether the position deviation δh of the h-th pin is within a permissible range (S128). If both the position deviation ΔXh in the X-axis direction and the position deviation ΔYh in the Y-axis direction are within the specified range, the CPU 61 determines that the h-th pin position deviation δh is within the permissible range and increments the value of the pin number h by 1 (S130). The CPU 61 then determines whether the value of the pin number h is greater than the number of pins P (16 in the present embodiment) (S132). If it is determined that the value of the pin number h is equal to or less than the number of pins P, the CPU 61 determines that there is one more pin P for which the position deviation amount δ is to be calculated, returns to S120 again and calculates the position deviation amount δ of another pin P.If the value of the pin number h is determined to be greater than the number of pins P, the CPU 61 determines that none of the pins P have a positional deviation exceeding the permissible range and mounts the electronic component C onto the plate S (S134). Specifically, the CPU 61 controls the head movement device 40 so that the center Q of the electronic component C, picked up by the suction nozzle 51, moves over a mounting position on the plate S. The CPU 61 then controls the Z-axis actuator 52 so that the electronic component C is pressed against the plate S and controls a solenoid valve so that the electronic component C is released, thus terminating the current operation. The CPU 61 can correct the mounting position of the electronic component C by taking into account the positional deviation δ of each pin P in the electronic component C.

[0029] If at least one of the position deviation values ​​ΔXh in the X-axis direction and ΔYh in the Y-axis direction lies outside the specified range, the CPU 61 determines that the h-th pin position deviation value δh is outside the permissible range and disposes of component C (S136). Specifically, the CPU 61 controls the head movement device 40 so that the suction nozzle 51 moves over the disposal container 38. Then, the CPU 61 causes the solenoid valve to release the electronic component C and terminates the routine.

[0030] Next, the subroutine for calculating the temporary center will be described. Fig. Figure 8 is a flowchart illustrating an example of the subroutine for calculating the temporary center. This processing is performed by CPU 61 after S112 of the component assembly routine described above.

[0031] When this processing begins, the CPU 61 first binarizes the side emission image Im1 (S200). Specifically, the CPU 61 sets a pixel with a brightness value below a first threshold to black and a pixel with a brightness value greater than or equal to a predetermined value to white. Here, the first threshold is a threshold to distinguish pen P from the others, and a value previously entered by the operator is used. The first threshold is contained in the shape data. As a result, one leading end section of pen P is white, and the other sections are black. An image obtained in this way is called the binarized image Im3 (not shown). Next, the CPU 61 detects the first pen P1 from the binarized image Im3 (S202).Specifically, the CPU 61 recognizes as the first pin P1 a region contained within a predetermined first pin detection region in the binarized image Im3, among the regions where white pixels are collected in the binarized image Im3. The first pin detection region is a region experimentally determined for each type of electronic component C. The CPU 61 then sets the value of the first pin number i to 1 (S204).

[0032] Next, the CPU 61 calculates the actual position D of the first pin (hereafter referred to as the actual position Di of the i-th first pin), which is an actual center of the i-th first pin P1 (hereafter referred to as the i-th first pin P1i) for a reference point (for example, a left front corner of the binarized image Im3) of the binarized image Im3 (S206). The i-th actual position of the first pin Di is calculated as the X-axis coordinate value Xdi and Y-axis coordinate value Ydi of a center of the i-th first pin P1i in an XY plane, where a reference point of the binarized image Im3 is referred to as the origin point O, with the left-right direction of the binarized image Im3 being referred to as the X-axis and the front-back direction of the binarized image Im3 being referred to as the Y-axis. The CPU 61 then increments the value of the first pin number i by 1 (S208).Then, CPU 61 determines whether the value of the first pin number i is greater than the number of first pins (4 in the present embodiment) (S210). If it is determined that the value of the first pin number i is equal to or less than the number of first pins P1, CPU 61 determines that there is another first pin P1 for which the actual position D of the first pin must be calculated, returns to S206, and calculates the actual position D of another first pin P1.

[0033] If it is determined that the value of the first pin number i is greater than the number of first pins P1, the CPU 61 determines that the calculation of the actual positions D of all first pins P1 is complete and calculates the temporary center M (Xm and Ym), which represents a temporary center of the electronic component C for a reference point (origin point O) of the binarized image Im3, from the actual positions D of the first pins (S212). The temporary center M can be calculated using the least squares method. That is, the CPU 61 establishes a temporary center, which is a temporary center of the electronic component C for a reference point of the binarized image Im3.Next, CPU 61 determines the i-th provisional first pin position Ei (Xei and Yei), which is a provisional center of the i-th first pin P1i for the reference point, if the i-th first pin P1i is located at a position separated by a distance defined in the relative design position Ri of the i-th first pin (i is an integer of 1 or more and 4 (the number of first pins) or less), which is defined in the shape data with reference to the provisional center. Then, CPU 61 calculates the provisional center where the sum of the squares of the distances from the actual position D of the first pin to the provisional position E of the first pin is minimal, as the temporary center M of the electronic component C for a reference point of the binary image Im3, and terminates the current processing.This allows the temporary center M of the electronic component C to be calculated for the reference point of the binarized image Im3 using the actual position D of the first pin.

[0034] Next, the subroutine for calculating the center will be described. Fig. Figure 9 is a flowchart illustrating an example of the subroutine for calculating the center. This processing is performed by CPU 61 after execution of the subroutine for calculating the temporary center, in order to calculate the temporary center M of the electronic component C for an image.

[0035] When this processing begins, the CPU 61 first binarizes the lower-emission image Im2 (S300). Specifically, the CPU 61 sets a pixel with a luminance value below a second threshold to black and a pixel with a luminance value equal to or greater than a predetermined value to white. The second threshold is used to distinguish the second pen P2 from the others, and it uses a value previously entered by the operator. For example, the second threshold is a value greater than the luminance value of an area of ​​the first pen P1 and less than the luminance value of an area of ​​the second pen P2 in the lower-emission image Im2, and it is a value determined through experimentation. The second threshold is, for example, set to a value greater than the first threshold. The second threshold is contained in the shape data.This causes the leading end section of a second pen to tend to be white, and the other sections to tend to be black. An image obtained in this way is called a binarized image Im4. Fig. Figure 10A shows an example of a binaryized image Im4. Fig. In 10A, the images of the first pin P1 and the body B are black, and one pixel of the second pin P2 is white. The CPU 61 then identifies an area where white pixels are collected in the binarized image Im4 as a candidate for the second pin P2 (S302). The CPU 61 then determines whether the number of identified candidates for the second pin is equal to or greater than the number of second pins P2 in the shape data (S304). If the number of candidates for the second pin is found to be less than the number of second pins P2 in the shape data, the CPU 61 determines that an error has occurred, indicates the error by displaying an error message on a display device (not shown) or the like (S308), proceeds to S136 of the component assembly processing routine, and discards the electronic component C picked up by the suction nozzle 51.This is because if the number of detected second pen candidates is less than the number of second pens defined in the shape data, one of the second pen candidates P2 will be missing during the detection of the second pen candidates.

[0036] If the number of second-pin candidates is found to be equal to or greater than the number of second-pin P2s in the shape data, the CPU 61 determines that all second-pins are contained within the detected second-pin candidates and sets a value of the second-pin candidate number j to 1 (S306). In some cases, due to individual variations in the electronic component C, irregular light reflection, or the like, other parts are detected as second-pin candidates in addition to the second-pin P2. Next, the CPU 61 calculates the second-pin candidate position F (hereafter referred to as the j-th second-pin candidate position Fj), which is the center of a j-th second-pin candidate (hereafter referred to as the j-th second-pin candidate) for a reference point (for example, a front-left corner) of the binarized image Im4 (S310).The j-th second pen candidate position Fj is calculated as the X-axis coordinate value Xfj and the Y-axis coordinate value Yfj of the j-th second pen candidate on an XY plane, where a reference point of the binarized image Im4 is designated as the origin point O, with the left-right direction of the binarized image Im4 being designated as the X-axis and the front-back direction of the binarized image Im4 being designated as the Y-axis. The CPU 61 then increments the value of the second pen candidate number j by 1 (S312). The CPU 61 then determines whether the value of the second pen candidate number j is greater than the number of second pen candidates detected in S302 (S314).If it is determined that the value of the second pen candidate number j is equal to or less than the number of second pen candidates detected in S302, CPU 61 determines that there is one more second pen candidate for which the second pen candidate position F needs to be calculated, returns to S310 again and calculates the second pen candidate position F of another second pen candidate.

[0037] If it is determined that the value of the second pin candidate number j is greater than the number of second pin candidates detected in S302, CPU 61 determines that the calculation of the position F of the second pin candidate is complete for all second pin candidates and sets the temporary center M for a reference point of the binarized image Im3, calculated in the temporary center calculation, to the temporary center N (Xn and Yn) for a reference point (original point O) of the binarized image Im4 (S316). Next, CPU 61 sets a value of the second pin number k to 1 (S318). Subsequently, CPU 61 determines the k-th relative position Rk (ΔXrk and ΔYrk) of the k-th second pin P2 (hereinafter k-th second pin P2k) with respect to the center Q of the electrical component C from the shape data (S320).Then, the CPU 61 calculates the temporary layout position G of the k-th second pin P2k (hereinafter referred to as the temporary layout position Gk of the k-th second pin) for the reference point of the binaryized image Im4 from the temporary center N and the relative layout position Rk of the k-th second pin (S322). In a case where the k-th second pin P2k is located at the k-th relative layout position Rk of the second pin, the temporary layout position Gk of the k-th second pin is a center in the layout of the k-th second pin P2k for the reference point of the binaryized image Im4 of the second pins. In particular, the CPU 61 adds the detected X-axis relative value ΔXrk of the relative design position Rk of the k-th second pin to the X-axis coordinate value Xn of the temporary center N in order to calculate the X-axis coordinate value Xgk (Xn+ΔXrk) of the k-th temporary design position Gk of the second pin.In addition, the CPU 61 adds the detected Y-axis relative value ΔYrk of the relative design position Rk of the k-th second pin to the Y-axis coordinate value Yn of the temporary center N in order to calculate the Y-axis coordinate value Ygk (Yn+ΔYrk) of the temporary design position Gk of the k-th second pin.

[0038] Next, CPU 61 sets a value for the reference distance L to the distance Lmax (S324). The reference distance L is described below. Furthermore, the distance Lmax is set to be greater than the length of a diagonal of body B in the electronic component C. Then, CPU 61 sets a value for the second pin candidate number j to 1 (S326). Finally, CPU 61 calculates the distance Ljk from the j-th second pin candidate position Fj to the temporary design position Gk of the k-th second pin (S328).

[0039] Next, CPU 61 determines whether the distance Ljk is shorter than the reference distance L (S330). If it is determined that the distance Ljk is shorter than the reference distance L, CPU 61 updates a value of the reference distance L to the distance Ljk (S332). Then, CPU 61 sets the j-th second candidate pin position Gj (Xgj and Ygj) to the k-th second actual pin position Uk (Xuk and Yuk) (S334). In a case where a value of the second candidate pin number j is 1, the distance Lmax is set as the reference distance L, and accordingly, the distance L1k is always shorter than the reference distance L. Thus, CPU 61 updates the reference distance L to the distance L1k and sets the first second candidate pin position G1 to the k-th second actual pin position Uk.If the value of the second pin candidate j is 2 or greater, and the reference distance Ljk is less than the reference distance L, CPU 61 updates a value of the reference distance L to the distance Ljk and updates the j-th second pin candidate position Gj to the k-th second pin actual position Uk. By repeatedly performing this processing, the j-th second pin candidate position Fj, which is closest to the temporary design position Gk of the k-th second pin, is set as the k-th second pin actual position Uk.

[0040] After S334, or after determining that the distance Ljk is not shorter than the reference distance L, CPU 61 increments the value of the second pin candidate number j by 1 (S336). CPU 61 then determines whether the value of the second pin candidate number j is greater than the number of second pin candidates (S338). If it is determined that the value of the second pin candidate number j is equal to or less than the number of second pin candidates identified in S302, CPU 61 determines that there is one more second pin candidate for which the reference distance L to the temporary design position Gk of the k-th second pin must be calculated, returns to S328, and calculates the distance Ljk to the temporary design position Gk of the k-th second pin.

[0041] If it is determined that the value of the second pin candidate with the number j is greater than the number of second pin candidates detected in S302, CPU 61 determines that the position Fj of the second pin candidate closest to the temporary design position Gk of the k-th second pin is set as the k-th second pin actual position Uk, and increments the value of the second pin number k by 1 (S340). Next, CPU 61 determines whether the value of the second pin number k is greater than the number of second pins P2 (S342). If it is determined that the value of the second pin number k is equal to or less than the number of second pins P2, CPU 61 determines that there is one more second pin P2 for which the second pin actual position U needs to be calculated, returns to S320, and determines the k-th relative design position Rk of another k-th second pin P2k.

[0042] In this way, CPU 61 increments the value of the second pin candidate number j by 1 and repeatedly processes S328 to S338 to calculate the distance Ljk between a specific temporary design position Gk of the k-th second pin and all second pin candidate positions F. Furthermore, CPU 61 designates a second pin candidate with the shortest distance Ljk to the temporary design position Gk of the k-th second pin from among the multiple second pin candidates detected in S302 as the k-th second pin position Uk. Accordingly, even in a case where the number of second pin candidates detected in S302 is greater than the number of second pins P2, the second pins can be accurately identified among the second pin candidates.Furthermore, the CPU 61 increments the value of the second pin number k by 1 and performs the processing repeatedly, thereby accurately determining the actual positions U of all second pins P2.

[0043] If it is determined that the value of the second pin number k is greater than the number of second pins P2, the CPU 61 calculates the center Q of the electronic component C for a reference point (origin point O) of a binary image from the actual position D of the first pin and the actual position U of the second pin (S344). This processing can be carried out by a least squares method. That is, the CPU 61 establishes a preliminary center, which is a preliminary center of the electronic component C in the binary image Im4.Next, in a case where the i-th first pin P1i (i is an integer greater than or equal to 1 and less than or equal to 4 (the number of first pins P1)) is at a position separated by the i-th relative position Ri of the first pin with respect to a provisional center, the CPU 61 sets the i-th provisional position Ei (Xei and Yei) of the first pin, which represents a provisional center of the i-th first pin P1i for a reference point of the binaryized image Im4.Subsequently, in a case where the j-th second pin P2j is located at a position separated by a j-th relative pin position Rj (j is an integer greater than or equal to 1 and less than or equal to 12 (the number of second pins P2)) with respect to a provisional center, the CPU 61 determines the provisional position Vj (Xvj and Yvj) of the j-th second pin, which represents a provisional center of the j-th second pin P2j for the reference point of the binary image Im4. Then, the CPU 61 computes a provisional center at which the sum of the squares of the distances from the actual position D of the first pin to the provisional position E of the first pin and the sum of the squares of the distances from the actual position U of the second pin to the provisional position V of the second pin are minimized, as the center Q (Xq and Yq) of the electronic component C for the reference point of the binary image Im4, as shown in [reference]. Fig. 10B is displayed, and the current processing is terminated. This allows the center Q of the electronic component C to be calculated for the reference point of the binaryized image Im4 using the actual position D of the first pin and the actual position U of the second pin. Fig. In Figure 10B, the first pins P1 (P101 to P104) are shown with dashed lines for ease of description. The center Q of the electronic component C, determined in this way, is used to calculate the actual position D of the pin for a reference point (original point O) of an image.

[0044] Here, a correspondence relationship between elements of the present embodiment and elements of the present disclosure is clarified. That is, the control device 60 of the present embodiment corresponds to the image processing device of the present disclosure; the CPU 61, which performs the processing of S102 to S106 and S108 to S112 of the component assembly processing routine, corresponds to an image acquisition section; the CPU 61, which performs the processing of S200 to S210 of the temporary center calculation processing, corresponds to a first detection section; the CPU 61, which performs the processing of S212 of the temporary center calculation and the processing of S300 to S342 of the center calculation, corresponds to a second detection section; and the CPU 61, which performs the processing of S344 of the center calculation, corresponds to a component position calculation section.

[0045] The present disclosure is in no way limited to the embodiments described above, and it is understood that the present disclosure can be realized in various aspects, as long as the various aspects fall within the technical scope of the present disclosure.

[0046] In the embodiment described above, the CPU 61 generates a binary image Im3 from the side emission image Im1 and detects the first pin P1 from the binary image Im3 in the temporary subroutine for center calculation. However, the CPU 61 can detect the first pin P1 directly from the side emission image Im1 by predefining a luminance value range that corresponds to a color of the first pin P1. Furthermore, the CPU 61 generates a binary image Im4 from the bottom emission image Im2 and detects the second pin P2 from the binary image Im4 in the subroutine for center calculation. However, the CPU 61 can detect the second pin P2 directly from the bottom emission image Im2 by predefining a luminance value range that corresponds to a color of the second pin P2.

[0047] In the embodiment described above, the CPU 61 performs the processing of S302 to S342 of the center calculation processing routine to calculate the actual position U of the second pin. However, the position of each second pin candidate for a reference point of the binarized image Im4 can be set to the actual position U of the second pin by the same processing as S200 to S210 of the temporary center calculation processing routine.

[0048] In the embodiment described above, the CPU 61 calculates the center Q. However, the CPU 61 does not need to calculate the center Q for a reference point of an image.

[0049] In the embodiment described above, an operator determines which pin P is the first pin P1 or the second pin P2. However, the CPU 71 can automatically determine this based on the relative pin position R. In this case, the CPU 71 of the management device 70 can designate the pin P located on an outer circumferential side of the body B as the second pin P2, and designate the remaining pins P as the first pins P1. Alternatively, the CPU 71 can designate the pin P within a predetermined range from the center Q as the first pin P1 and the pin P outside the predetermined range from the center Q as the second pin P2.

[0050] In the embodiment described above, the CPU 61 images the electronic component C in a state in which the LED 32 of the incident light illumination 31 and the LEDs 35a to 35c of the side illumination 34 are switched on, in order to obtain an image Im2 with lower emission. However, the image Im2 with lower emission can also be obtained by imaging the electronic component C in a state in which only the LED 32 of the incident light illumination 31 is switched on, or the image Im2 with lower emission can be obtained by imaging the electronic component C in a state in which only the LED 35 of the side illumination 34 is switched on.

[0051] In the embodiment described above, after S308 of the subroutine for center calculation, the CPU 61 proceeds to S136 of the component assembly processing routine. However, after S308 of the subroutine for center calculation, the CPU 61 can return to S102 or S108 of the component assembly processing routine and reacquire the lateral emission image Im1 or the image with lower emission Im2.

[0052] As described above, in the image processing device of the present disclosure, in the first image obtained by imaging a component under the first illumination condition, a contour of the second characteristic section appears slightly extended in a predetermined direction compared to a contour of the first characteristic section. This is because the second characteristic section is closer to the first light source than the first characteristic section, and the light from the first light source is reflected more strongly in the predetermined direction than by the first characteristic section.Therefore, if the first characteristic section and the second characteristic section are to be captured using the first image, the second characteristic section appears extended in the predetermined orientation, which in some cases prevents the second characteristic section from being captured with sufficient accuracy. In the second image, obtained by imaging the component under the second illumination condition, the contour of the second characteristic section does not appear extended in the predetermined direction compared to the contour of the first characteristic section, as is the case in the first image.This is because, during imaging, the distances from the second light source to the respective characteristic sections are essentially the same, and the second characteristic section does not reflect the light from the second light source more strongly in the predetermined direction than the first characteristic section. In this image processing device, the second characteristic section is captured using the second image, in which the second characteristic section is not extended in the predetermined direction compared to the first image, thus improving the capture accuracy of the characteristic section.

[0053] In the image processing device of the present disclosure, the second recognition section can binarize the second image such that the characteristic section, which includes at least the second characteristic section, and a background, which includes the main body, can be distinguished from one another, and can recognize the second characteristic section using the binarized image. If the second image of a component is taken in a state in which light is emitted under the second illumination condition, a section other than the second characteristic section is also depicted, and it can be difficult to recognize the second characteristic section from the second image. Accordingly, it is important in the second image to binarize the second characteristic section and the background in a distinguishable manner.

[0054] In the image processing device of the present disclosure, the first recognition section can calculate a position of the first characteristic section for the first image, and the second recognition section can calculate a temporary representative position, which is a temporary representative position of the component based on a position of the first characteristic section; calculate a candidate position, which is a candidate position of a position of the second characteristic section for the second image; detect a relative position in interpretation of the second characteristic section for the representative position of the component; and calculate an interpretation position, which is a position in interpretation of the second characteristic section for the second image based on the temporary representative position and the relative position.and calculate the position of the second characteristic section in the second image based on the candidate position and the design position. With this configuration, the second characteristic section can be identified sufficiently and accurately.

[0055] The image processing device of the present disclosure can further comprise a component position calculation section that calculates the representative position of the component based on the position of the first characteristic section and the position of the second characteristic section. With this configuration, the position of each characteristic section for an image can be accurately detected, and thus a representative position of the component for the image can be accurately detected. Industrial applicability

[0056] The present disclosure can be applied in the manufacturing industry and the like for component assemblers and component assembly systems. List of reference symbols

[0057] 1 Component assembly system, 10 Component assembler, 12 Housing, 21 Component feeder, 21a Tray feeder, 21b Belt feeder, 22 Conveyor, 24 Component camera, 25 Marking camera, 26 Side light source, 28 Bottom light source, 29 Housing, 30 Connecting device, 31 Incident light illumination, 32 LED, 33 Semi-mirror, 34 Side light, 35, 35a to 35c LED, 36 Camera main body, 36a Optical axis, 38 Disposal container, 40 Head movement device, 42 X-axis slider, 43 X-axis guide rail, 44 Y-axis slider, 45 Y-axis guide rail, 46 X-axis actuator, 47 X-axis position sensor, 48 Y-axis actuator, 49 Y-axis position sensor, 50 head, 51 suction nozzle, 52 Z-axis actuator, 53 Z-axis position sensor, 60 control device, 61 CPU, 62 ROM, 63 memory, 64 RAM, 65 input / output interface, 66 bus, 70 management device, 71 CPU, 72 ROM, 73 memory, 74 RAM, 101 pin, B housing, C electronic component, Im1 side emission imageIm2 lower emission image, Im3 binary image, Im4 binary image, M temporary center, N temporary center, O temporary design position of the second pin, P pin, P101 to P104, P201 to P212, P1 first pin, P2 second pin, Q center, R relative design position of the pin. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2005-107716

[0003]

Claims

[1] Image processing device for capturing a characteristic section in a component comprising a main body and several characteristic sections arranged on a bottom surface of the main body, using an image obtained by imaging the component, wherein the image processing device comprises: an image acquisition section configured to capture multiple images obtained by repeatedly imaging the component by emitting light onto the component under multiple illumination conditions, including a first illumination condition using a first light source configured to emit light onto a component at a predetermined position from a lateral side, and a second illumination condition using a second light source configured to emit light onto the component at the predetermined position from a bottom side; a first acquisition section that is configured to acquire a first characteristic section, which is part of the multiple characteristic sections of the component, using a first image obtained by imaging the component under the first illumination condition; and a second capture section configured to capture a second characteristic section, which is part of the multiple characteristic sections of the component and is closer to the first light source than the first characteristic section during imaging, by using a second image obtained by imaging the component under the second illumination condition. [2] The image processing device according to claim 1, wherein the second detection section binarizes the second image in such a way that the characteristic section comprising at least the second characteristic section and a background comprising the main body are distinguishable from each other, and detects the second characteristic section using the binarized image. [3] The image processing device according to claim 1 or 2, wherein the first acquisition section calculates a position of the first characteristic section for the first image and The second recognition section calculates a temporary representative position, which is a temporary representative position of the component based on the position of the first characteristic section; calculates a candidate position, which is a candidate position of the second characteristic section for the second image; determines a relative position in the design of the second characteristic section for the representative position of the component; calculates a design position, which is a position in the design of the second characteristic section for the second image based on the temporary representative position and the relative position; and calculates the position of the second characteristic section in the second image based on the candidate position and the design position. [4] The image processing device according to claim 3, which further comprises: a component position calculation section that is configured to calculate the representative position of the component based on the position of the first characteristic section and the position of the second characteristic section.

Images