Image processing device, installation device, installation system, image processing method and installation method

Abstract

The image processing device disclosed in the present invention is used for a mounting device having a mounting portion, wherein the mounting portion inserts a protrusion of a component having a main body and one or more protrusions formed on the main body into an insertion portion formed on a processing object and arranges the component on the processing object. The image processing device detects the shape of the main body, the position of the protrusion, the number of protrusions, and the front end of the protrusion based on a photographed image of the protrusion irradiating the component from the side and the main body irradiating the component from below, acquires information of the insertion portion, and generates a component model including a contour shape model including the shape and direction of the main body and a protrusion model including the position, number, and front end features of the protrusion.

Description

Technical Field

[0001] This specification discloses an image processing device, an installation device, an installation system, an image processing method, and an installation method. Background Art

[0002] In the past, as an installation device, a device has been proposed that uses a reflector to guide the image of an electronic component adsorbed by a nozzle into a camera and shoot the image with the camera (for example, see Patent Document 1, etc.). In this device, a structure that can achieve miniaturization of an assembly head and device, reduce costs, and shorten the production interval of production can be provided.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2005-50887 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, the above-mentioned Patent Document 1 does not consider the process of picking up a component whose pin protrudes from a main body and inserting the pin into a hole formed in a substrate for mounting. Thus, it is desired to more appropriately identify a component having a main body and a protrusion protruding from the main body and more appropriately arrange it on a processing object such as a substrate.

[0008] The present disclosure is completed in view of such problems, and its main purpose is to provide an image processing device, an installation device, an installation system, an image processing method and an installation method that can more reliably perform installation processing with a simpler structure when installing a component having a protrusion protruding from a main body.

[0009] Technical solutions to solve problems

[0010] In the present disclosure, the following means are adopted to achieve the above-mentioned main object.

[0011] The image processing device disclosed in the present invention is used for an installation device having an installation portion, wherein the installation portion allows an element having a main body and one or more protrusions formed on the main body to be inserted into an insertion portion formed on a processing object and arrange the element on the processing object. The image processing device includes an image control portion, which acquires a captured image, detects the shape of the main body, the position of the protrusion, the number of the protrusions and the front end of the protrusion based on the acquired captured image, acquires information of the insertion portion, and generates an element model including a contour shape model and a protrusion model. The captured image is obtained by photographing under side illumination and / or epi-illumination of a photographing unit, the photographing unit having an illumination portion capable of lighting the side illumination for illuminating the element from the side and the epi-illumination for illuminating the element from below, and a photographing portion having a depth of field of at least the distance between the main body and the protrusion and photographing the element from below, the contour shape model includes the shape and direction of the main body, and the protrusion model includes the characteristics of the position, number and front end of the protrusion.

[0012] In the image processing device, the shape of the main body, the position of the protrusion, the number of protrusions, and the front end of the protrusion are detected based on the captured image captured by at least one of the side illumination for irradiating light to the component from the side and the epi-illumination for irradiating light to the component from the bottom, and the information of the insertion part is obtained, and the component model including the outline shape model and the protrusion model is generated. The outline shape model includes the shape and direction of the main body, and the protrusion model includes the position, number, and characteristics of the front end of the protrusion. In the image processing device, for example, there is no need for a special lighting unit and a shooting unit such as irradiating a laser to the front end of the protrusion to focus on the front end and shoot the image. By the side illumination from the side and the epi-illumination from the bottom, a captured image that can more reliably detect the protrusion and the main body can be obtained. In the image processing device, the captured image obtained in this way can be used to construct a component model that becomes the reference data of the component. Moreover, the mounting device can perform recognition of the component having the protrusion using the component model and perform mounting processing. Therefore, in the image processing device, when mounting a component having a protrusion protruding from the main body, the mounting processing can be reliably performed with a simpler structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic explanatory diagram showing an example of the mounting system 10 .

[0014] Figure 2 It is a schematic explanatory diagram showing an example of the imaging unit 14 .

[0015] Figure 3 It is an explanatory diagram showing an example of information stored in the storage unit 42 .

[0016] Figure 4 This is a flowchart showing an example of a component model generation processing procedure.

[0017] Figure 5 This is an explanatory diagram showing the difference between captured images obtained by side illumination and epi-illumination.

[0018] Figure 6 This is a flowchart showing an example of the installation processing procedure.

[0019] Figure 7 This is an explanatory diagram for explaining the outline of image recognition processing.

[0020] Figure 8 This is an explanatory diagram for correcting the setting position of the protrusion area A by taking parallax into account.

[0021] Fig. 9 It is an explanatory diagram of the length of the protruding portion and the distance between the element 31 and the imaging unit 15 .

[0022] Fig.10 It is an explanatory diagram showing an example of the insertion permission determination using the insertion portion model 46.

[0023] Fig.11 It is an explanatory diagram showing an example of another mounting system 10B. DETAILED DESCRIPTION

[0024] Hereinafter, the present embodiment will be described with reference to the drawings. Figure 1 It is a schematic explanatory diagram showing an example of the mounting system 10 of the present disclosure. Figure 2 It is a schematic explanatory diagram showing an example of the imaging unit 14 . Figure 3 4 is an explanatory diagram showing an example of information stored in the storage unit 42. In addition, in the present embodiment, the left-right direction (X axis), the front-back direction (Y axis), and the up-down direction (Z axis) are as follows: Figure 1 , Figure 2 As shown.

[0025] The mounting system 10 is a device for mounting a component 31 on a processing object 30, for example, and includes a mounting device 11 and a supply device 25. The mounting device 11 includes a conveying device 12, an imaging unit 14, a mounting portion 20, and a control device 40. The processing object 30 is not particularly limited as long as it includes an insertion portion into which the protrusion 33 of the component 31 is inserted, and examples thereof include a substrate, a three-dimensional base material (three-dimensional object), and the like.

[0026] The component 31 is an electronic component having a main body 32 and a protrusion 33. The protrusion 33 is a terminal pin, and a plurality of protrusions 33 are formed on the main body 32. The protrusion 33 of the component 31 is inserted into the insertion portion of the processing object 30 and fixed. For the protrusion 33, a tapered portion 34 having a tapered surface formed to become thinner toward the front end and a front end portion 35 as a front end surface are formed on the front end side thereof. The tapered portion 34 is formed, for example, to facilitate insertion of the protrusion 33 into the insertion portion of the processing object 30. The tapered portion 34 is appropriately formed in a shape suitable for the shape of the protrusion 33, such as a flat surface or a curved surface, depending on the type of component. In addition, when the component 31 has a polarity as a terminal of the protrusion 33, a mark for specifying the direction of the main body 32 is formed on the lower surface of the main body 32 (see Figure 3 In addition, for the convenience of explanation, the protrusion 33 will also be referred to as a "terminal" or "pin" below.

[0027] The transport device 12 is a unit that carries out carrying in, transporting, fixing at a mounting position, and carrying out of the processing object 30. The transport device 12 transports the processing object 30 by a pair of conveyor belts provided at a distance.

[0028] The imaging unit 14 is a device for imaging the image of the lower surface side of the component 31 in the state of being picked up and held by the mounting unit 20. The imaging unit 14 is disposed between the conveying device 12 and the supply device 25. The imaging unit 14 includes an imaging unit 15 and an illumination unit 16. Figure 2 As shown, the photographing range of the photographing section 15 is the upper part of the photographing section 15. The photographing section 15 has a photographing element such as a CMOS image sensor or a CCD image sensor that generates electric charge by receiving light and outputs the generated electric charge. The photographing section 15 has a depth of field of at least the distance between the main body 32 and the protruding portion 33. The lighting section 16 is a light source that irradiates light to the element 31 held by the mounting portion 20 and can adjust the brightness (light amount) of the light irradiated to the element 31, the wavelength of the light, and the irradiation position of the light. The lighting section 16 has at least a side light 17 that irradiates light to the element 31 from the side and a downlight 18 that irradiates light to the element 31 from below. The lighting section 16 is configured to be able to light the side light 17 and the downlight 18 simultaneously or separately. In addition, the lighting section 16 is configured to be able to simultaneously illuminate the respective objects of the front end of the protruding portion 33 and the main body 32 that are at different distances from the photographing section 15. The imaging unit 14 is configured to capture images of the main body 32 and the protrusion 33 by side illumination from the sidelight 17 and images of the main body 32 and the protrusion 33 by side illumination from the sidelight 18 without the element 31 moving (see FIG. Figure 2). In addition, the imaging unit 14 can also light up the side light 17 and the downlight 18 at the same time to capture the main body 32 and the protrusion 33 as the subject. The imaging unit 14 captures the image of the component 31 when the multi-joint arm robot 21 holding the component 31 stops above the imaging unit 15 or moves, and outputs the captured image to the control device 40. The control device 40 can perform inspections on whether the shape and position of the component 31 are normal, and detect the position, rotation, etc. of the component 31 when it is picked up, etc. based on the captured image.

[0029] The mounting part 20 is a device for placing the component 31 on the processing object 30. The mounting part 20 includes a multi-joint arm robot 21, a picking component 22, and a camera 23. The multi-joint arm robot 21 is a working robot that is connected to a first arm and a second arm that rotate around a rotation axis. The picking component 22 is a component that performs the operation of picking up the component 31 and placing it on the processing object 30, and is rotatably connected to the front end of the multi-joint arm robot 21. The picking component 22 may be, for example, a suction nozzle that absorbs and picks up the component 31 by negative pressure, or a mechanical chuck that physically holds and picks up the component 31. The camera 23 takes a picture of the bottom, for example, the component 31 placed on the supply device 25, and is used when identifying the posture of the component 31. The camera 23 is fixed to the front end of the multi-joint arm robot 21.

[0030] The supply device 25 is a device for supplying components 31. The supply device 25 includes a supply placement section 26, a vibrating section 27, and a component supply section 28. The supply placement section 26 has a placement surface, places components 31 of uncertain posture, and is configured as a conveyor that moves in the front-rear direction. The vibrating section 27 applies vibration to the supply placement section 26 to change the posture of the components 31 on the supply placement section 26. The component supply section 28 is a device that is disposed at the upper rear portion (not shown) of the supply placement section 26, receives components 31, and supplies them to the supply placement section 26. The component supply section 28 releases components 31 to the supply placement section 26 when the number of components on the placement surface is less than a predetermined number, or releases components 31 to the supply placement section 26 regularly. The mounting section 20 uses the multi-jointed arm robot 21, the picking component 22, and the camera 23 to recognize components 31 in a posture that can be picked up and arranged on the supply placement section 26 and pick up the components 31.

[0031] The control device 40 has a microprocessor centered around the CPU 41 and is responsible for controlling the entire device including the transport device 12, the imaging unit 14, the mounting unit 20, and the supply device 25. The CPU 41 has the functions of an image control unit in the image processing device that performs image processing of the imaging unit 14 and an installation control unit in the installation device that controls the multi-jointed arm robot 21. The control device 40 outputs control signals to the transport device 12, the imaging unit 14, the mounting unit 20, and the supply device 25, and on the other hand, inputs signals from the transport device 12, the imaging unit 14, the mounting unit 20, and the supply device 25. In addition to the CPU 41, the control device 40 also includes a storage unit 42, a display device 47, and an input device 48. The storage unit 42 is a large-capacity storage medium such as an HDD or a flash memory that stores various application programs and various data files. The display device 47 is a display that displays information related to the installation system 10, various input screens, etc. The input device 48 includes a keyboard, a mouse, etc. for input by the operator.

[0032] The storage unit 42 stores, for example, installation condition information for installing the component 31 on the processing object 30, a component model 43 for identifying the component 31, and the like. The installation condition information includes information such as the configuration position and the configuration number of the component 31 installed on the processing object 30. The component model 43 is reference data of the component 31, such as Figure 3As shown, it includes component ID, lighting method, outline shape model 44, protrusion model 45 and insertion part model 46. Component ID is identification information for specifying component 31, and is assigned in advance for each type of component 31. Lighting method is information for specifying the lighting method when capturing the image for generating component model 43, and is at least one of side lighting and epi-illumination. Outline shape model 44 is reference data for specifying main body 32. Outline shape model 44 specifies the size of main body 32 in addition to the shape and direction (polarity of terminal) of main body 32. Outline shape model 44 also includes the shape and configuration position of the mark for specifying the direction of main body 32. Protrusion model 45 is reference data for specifying protrusion 33 such as terminal. Protrusion model 45 includes the position, number and characteristics of the front end of protrusion 33 including the pitch of protrusion 33. Characteristics of the front end include, for example, the shape of the front end 35, its size and signal level. The shape of the front end includes, for example, circle, square, rectangle, trapezoid and other arbitrary shapes. The size includes information such as radius (circle), width / height (rectangle), width 1 / width 2 / height (trapezoid), width / height, etc. The signal level includes the signal level of the front end when photographed by predetermined lighting, the signal level of its peripheral part, etc. In addition, the protrusion model 45 also includes the size of the protrusion 33, the information of the protrusion area A including one or more protrusions 33 arranged on the main body 32, etc. The insertion part model 46 is the reference data that specifies the insertion part formed 30 on the processing object for the protrusion 33 to be inserted. The insertion part model 46 includes information such as the position of the insertion part including the spacing of the insertion part, the number of the insertion part, the characteristics of the shape of the insertion part, and its size.

[0033] Next, the operation of the mounting system 10 of the present embodiment configured as described above will be described. First, the process of generating the component model 43 by the control device 40 will be described. Figure 44 is a flowchart showing an example of a component model generation processing program executed by the CPU 41. The program is stored in the storage unit 42 and is executed before the component 31 is used. When the program is started, first, the CPU 41 controls the multi-joint arm robot 21 in a manner to hold the component 31 at the imaging position, and sets the illumination method of the illumination unit 16 (S110). The CPU 41 sets the initial value of the illumination method to side illumination, and then to epi-illumination. Next, the CPU 41 adjusts and sets the illumination intensity and exposure time of the imaging unit 14 in a manner to produce contrast between the tapered portion 34 and the front end portion 35 (S120). Next, the CPU 41 images the component 31 under the set conditions (S130), determines whether the component 31 is imaged by all the illumination methods (S140), and executes the processing below S110 when the image is not captured by all the illumination methods. That is, the illumination method is changed to epi-illumination, and the illumination intensity and exposure time are determined in a manner to produce contrast between the tapered portion 34 and the front end portion 35 to image the component 31. In S110, the image of the device 31 may be captured by an illumination method in which the side illumination and the epi-illumination are illuminated simultaneously. Simultaneous illumination may improve the detection efficiency.

[0034] Figure 5 is an explanatory diagram showing the difference between images captured by side illumination and by epi-illumination. Figure 5 As shown, if the protrusion 33 is photographed by side illumination, the image obtained by reflection of the illumination is that the tapered portion 34 is white (bright portion) and the front end portion 35 is black (dark portion). On the other hand, if the protrusion 33 is photographed by incident illumination, the image obtained by reflection of the illumination is that the tapered portion 34 is black (dark portion) and the front end portion 35 is white (bright portion). In an electronic component with pins such as the component 31, side illumination is more effective in obtaining an image of the front end portion of the protrusion 33 in most cases, but incident illumination is sometimes better for some components and calibration, so the imaging unit 14 is configured to be able to obtain a photographic image by at least one of the two illuminations.

[0035] Next, the CPU 41 selects an illumination method and a captured image suitable for image recognition of the component 31 (S150). The CPU 41 may select, for example, a captured image in which the front end portion 35 of the protrusion 33 is easily detected from the captured images of the side illumination and the epi-illumination. The CPU 41 may select, for example, a captured image in which the edge portion (region of the tapered portion 34) of the front end portion 35 of the protrusion 33 is easily distinguished from the main body 32. That is, the CPU 41 may select an illumination method and a captured image by adding the contrast with the main body 32 in addition to the contrast between the tapered portion 34 and the front end portion 35. Alternatively, in S150, the operator may select an appropriate illumination method and a captured image.

[0036] Next, CPU41 obtains component information and insertion part information (S160). Component information is information provided by the manufacturing company of component 31, for example, including information such as the size, shape, direction, position, quantity and spacing (distance) between multiple projections of main body 32 of protrusion 33. The position of projection 33 also includes information such as row / column number, row / column spacing. This component information does not include information such as the front end shape of projection 33. In addition, insertion part information is information provided by the manufacturing company of processing object 30, for example, including information such as the position, quantity, shape, size of insertion part, more specifically, information such as aperture, row / column number, number of pins, number of exceptional pins, column spacing, and row spacing. Exception number of pins refers to the number of pins that do not exist in the equally spaced pin columns. CPU41 can also obtain component information and insertion part information in the data provided, can also extract and obtain component information and insertion part information from CAD data, and can also obtain data input by the operator from the specification booklet.

[0037] Next, the CPU 41 uses the captured image to generate a contour shape model 44 including information on the shape and direction of the main body 32 and also information on the size (S170). When generating the contour shape model 44, the CPU 41 may also use the component information. In addition, the CPU 41 also extracts features such as marks indicating the polarity and direction of the main body 32 as a contour. Next, the CPU 41 sets the protrusion area A in a manner that surrounds the portion where the front end portion 35 of the protrusion 33 exists (S180, see below). Figure 7 (C)). The CPU 41 uses the position information of the protrusion 33 to set a predetermined margin relative to the area where the protrusion 33 exists and sets the protrusion area A. For the predetermined margin, for example, a value that can better suppress the erroneous detection of the protrusion 33 can be empirically obtained and determined as the value. Next, the CPU 41 generates an insertion part model 46 including the position, number and size of the hole of the insertion part and the shape and other features based on the insertion part information (S190).

[0038] Next, the CPU 41 detects the position, number and front end of the protrusions 33 based on the captured image, and analyzes the characteristics thereof (S200), and based on the analyzed information, generates a protrusion model 45 (S210) including information on the position, number, characteristics of the front end 35 of the protrusions 33, and size information. When generating the protrusion model 45, the CPU 41 may also use the component information. In addition, the CPU 41 may also assign numbers to the detected protrusions 33 in S200. In addition, the position of the protrusion 33 also includes information such as the number of rows / columns and the row / column spacing. In addition, the characteristics of the front end 35 may also include the shape of the front end 35 (circle, square, rectangle, trapezoid, etc.) and its size, the signal level of the front end and its surrounding area, etc. Furthermore, the CPU 41 generates the component model 43 (refer to Figure 3 ), and store it in the storage unit 42, and end the program. In this way, the control device 40 can use the mounting system 10 to generate a component model 43 as the basic data and shape template of the component 31.

[0039] Next, a mounting process of inserting the protruding portion 33 into the insertion portion of the processing target 30 using the component model 43 to arrange the component 31 will be described. Figure 6 This is a flowchart showing an example of the installation processing program executed by the CPU 41. Figure 7 is an explanatory diagram for explaining the outline of the image recognition processing of the element 31. Figure 7 (A) is an explanatory diagram of a component image 51. Figure 7 (B) is the contour shape recognition processing of the main body 52, Figure 7 (C) is the setting of the protrusion area A, Figure 7 (D) is the identification of the front end portion 35, Figure 7 (E) is an explanatory diagram to which the insertion unit is applied. The mounting process program is stored in the storage unit 42 and is executed by a start instruction from an operator.

[0040] If the program is started, first, the CPU 41 reads out and obtains the installation condition information and the component model 43 (S300). Next, the CPU 41 causes the conveying device 12 to convey the processing object 30 to the installation position and perform a fixing process (S310), and controls the component supply unit 28 in a manner that supplies the component 31 to the supply placement unit 26 (S320). Next, the CPU 41 performs the following processing: the multi-joint arm robot 21 is controlled to move the camera 23 to the top of the supply placement unit 26, and the camera 23 photographs the component 31 on the supply placement unit 26 to identify the pickable component 31 (S330). The CPU 41 identifies the component 31 on the supply placement unit 26 in which the protrusion 33 is on the lower side and the main body 32 is on the upper side as a pickable component. In addition, when there is no pickable component 31, the vibration unit 27 is driven to change the posture of the component 31. When the component 31 is not present on the supply placement portion 26 , the component supply portion 28 is driven to supply the component 31 onto the supply placement portion 26 .

[0041] Next, the component 31 drives the multi-joint arm robot 21 and the picking component 22, so that the picking component 22 picks up the component 31 and moves to the upper part of the imaging unit 14 (S340). In the mounting portion 20, the imaging position and the control of the multi-joint arm robot 21 are pre-adjusted so that the center position of the picking component 22 moves to the center of the image. Next, the CPU 41 causes the imaging unit 14 to image the component 31 from the protrusion 33 side under the imaging condition including the lighting method corresponding to the component type (S350). The CPU 41 obtains the lighting method and the imaging condition from the component model 43. Next, the CPU 41 extracts the area of ​​the component 31 from the captured image ( Figure 7 (A)), identify the center position of the body 32 and the direction (polarity) of the body 32 (S360). Figure 7 As shown in (A), the component image 51 includes a main body 52, a protrusion 53, a front end 55, etc. as an image. The region of the component 31 can be extracted by pattern matching of the contour shape model 44. By detecting a mark or a shape feature that can identify a direction and performing pattern matching, the direction of the main body 32 can be identified ( Figure 7 Next, the CPU 41 uses the detected center position of the element 31 to perform parallax correction and set the position of the protruding region A (S370).

[0042] Figure 8 2 is an explanatory diagram for correcting the setting position of the protrusion area A by taking into account parallax. Figure 8 (A) is an illustration before parallax correction. Figure 8 (B) is an illustration after parallax correction. Fig. 9 1 is an explanatory diagram of the length of the protruding portion and the distance between the element 31 and the imaging unit 15. In the element 31, as Fig. 9 As shown, compared with the distance between the imaging unit 15 and the front end 35, the distance between the imaging unit 15 and the main body 32 is farther, so as Figure 8 As shown, if the element 31 deviates from the center of the image, parallax occurs in such a way that the sides of the main body 32 and the protrusion 33 are visible. There is no problem when the element 31 is in the center of the image, but when it is in the peripheral part of the image, the position of the front end 35 deviates from the main body 32. If the protrusion area A is set in this state, it will be impossible to properly detect the front end 35. The CPU 41 performs parallax correction on the captured image based on the center of the image, sets the protrusion area A on the element image 51, and then detects the position of the protrusion 33. In the parallax correction, when the resolution of each pixel of the front end 35 of the protrusion 33 is set to Rp (μm / pixel), the resolution change per predetermined length, that is, the unit resolution is set to Ru (μm / pixel), and the length of the protrusion 33 from the main body 32 is set to PL (mm), the CPU 41 performs a parallax correction of Rp-Ru×PL ( Figure 8 By performing such parallax correction, the CPU 41 can recognize the front end portion 35 more reliably.

[0043] Next, the CPU 41 performs processing to estimate and recognize the position of the tip 35 of each protrusion 33 and the center position of the tip 35 in the area inside the protrusion area A (S380). The CPU 41 can recognize the position of the tip 35 by performing pattern matching in the protrusion area A using the protrusion model 45. The protrusion model 45 includes information on the position and number of the protrusions 33, and the CPU 41 can estimate the approximate position of the tip 35 (see Figure 3 ). In addition, the element model 43 includes the shape feature of the front end portion 35, so the CPU 41 uses the shape feature to automatically generate a shape model and perform pattern matching. At this time, the front end portion 35 has individual differences in shape, size, and signal level for each front end portion, so the CPU 41 performs pattern matching while changing these parameters. In particular, regarding the shape, there is a situation where even if the matching score of the circle is high in the individual when the model is generated, the matching score of the rectangle is higher in other individuals. Therefore, when the CPU 41 cannot detect the number of protrusions 33 recorded in the protrusion model 45, it can generate shape models of other shapes and perform pattern matching again to improve the recognition rate. If the CPU 41 detects the positions of all the front ends 35, it calculates the center position of all the front ends 35.

[0044] If the center position of the front end 35 is detected, the CPU 41 calculates the optimal correction position (S390) according to the positional relationship between the center position of the front end 35 and the insertion portion. The CPU 41 obtains the correction value according to the positional relationship between the spacing of the position of the front end 35 and the deviation of the position and the hole of the insertion portion. More specifically, the CPU 41 measures the distance between the central position of each front end 35 and the hole of the insertion portion, and takes the position where the maximum value of the distance becomes the minimum as the optimal correction value. Next, the CPU 41 determines whether the element 31 can be inserted into the insertion portion (S400). The protrusion 33 is not always that its front end 35 is always equidistant due to bending, etc. The CPU 41 calculates the distance between the hole of the insertion portion and the front end shape of the protrusion 33, and determines that the element 31 can be inserted when the distance between all protrusions 33 and the hole is above a predetermined allowable threshold. The allowable threshold is, for example, a value empirically obtained that the protrusion 33 does not interfere with the inner wall of the insertion portion.

[0045] Fig.10 4 is an explanatory diagram showing an example of whether or not insertion is possible using the insertion portion model 46. The CPU 41 can, in a state where the above correction value is applied, Fig.10 As shown, while reducing the aperture of the insertion portion model 46 from the standard aperture, the interference between the hole and the front end portion 35 of the protrusion 33 is detected, and the judgment is made based on whether the difference between the aperture when the hole interferes with the front end 35 and the standard aperture exceeds the allowable threshold. In this way, the CPU 41 assumes at least an insertion portion of a first size and a second size smaller than the first size to judge whether interference occurs when the protrusion 33 is inserted into the insertion portion.

[0046] In S400, when the component 31 can be inserted into the insertion portion, the CPU 41 performs position correction to arrange the component 31 at a predetermined arrangement position of the processing object 30 (S410). At this time, if the CPU 41 arranges the component 31 to the processing object 30, the judgment result is the same, and the protrusion 33 is inserted into the insertion portion of the processing object 30. On the other hand, in S400, when the component 31 cannot be inserted into the insertion portion, the CPU 41 stops the use of the component 31 and discards it, and reports a message to this effect to the operator (S420). The report to the operator is, for example, a message or an icon showing that the component 31 cannot be inserted is displayed on the display device 47.

[0047] After S420 or after S410, the CPU 41 determines whether there is a component 31 to be arranged next (S430), and when there is a next component 31, the processing after S320 is executed. That is, the CPU 41 supplies the component 31 to the supply placement section 26 as needed, identifies the component 31 that can be picked up, picks up the component 31 and shoots it with the imaging unit 14, and repeatedly executes the processing of identifying the front end 35 and determining whether it can be inserted and arranged. On the other hand, in S430, when there is no next component to be arranged on the processing object 30, the CPU 41 determines whether there is a processing object 30 to which the component 31 should be arranged next (S440), and when there is a next processing object 30, the processing after S310 is executed. That is, the CPU 41 discharges the processing object 30 for which the component 31 has been arranged, and moves in the next processing object 30, and executes the processing after S320. On the other hand, when there is no next processing object 30 in S440, the CPU 41 ends the program.

[0048] Here, the correspondence between the structural elements of this embodiment and the structural elements of the present disclosure is clarified. The CPU 41 of this embodiment is equivalent to the image control unit and the installation control unit of the present disclosure, the installation unit 20 is equivalent to the installation unit, the shooting unit 15 is equivalent to the shooting unit, the lighting unit 16 is equivalent to the lighting unit, and the supply device 25 is equivalent to the supply device. In addition, the control device 40 is equivalent to the image processing device, and the installation device 11 is equivalent to the installation device. In addition, in this embodiment, an example of the image processing method and the installation method of the present disclosure is also clarified by explaining the operation of the installation system 10.

[0049] In the control device 40 of the present embodiment described above, the shape of the main body 32, the position of the protrusion 33, the number of the protrusions 33, and the front end 35 of the protrusion 33 are detected based on the captured images of the protrusion 33 irradiated with light from the side of the component 31 and the main body 32 irradiated with light from the bottom of the component 31, and the information of the insertion portion is acquired, and the component model 43 including the outline shape model 44 including the shape and direction of the main body 32 and the protrusion model 45 including the position, number, and characteristics of the front end 35 of the protrusion 33 is generated. In the control device 40, for example, a special lighting unit and a capturing unit such as irradiating a laser beam to the front end 35 of the protrusion 33 to focus on the front end 35 and capturing the image can be obtained, and the protrusion 33 and the main body 32 can be more reliably detected by side illumination from the side and epi-illumination from the bottom. In the control device 40, the component model 43 serving as the reference data of the component 31 can be constructed using the captured images obtained in this way. Furthermore, the mounting device 11 can use the component model 43 to perform recognition of the component 31 having the protrusion 33 and perform mounting processing. Therefore, in the control device 40, when mounting the component 31 having the protrusion 33 protruding from the main body 32, the mounting processing can be performed more reliably with a simpler structure. In addition, the control device 40 can generate the component model 43 including the information of the insertion part model 46 based on the information of the insertion part, and the insertion part model 46 includes the position, number and size of the insertion part.

[0050] Furthermore, in the mounting device 11 of the present embodiment, the shape of the main body 32, the position of the protrusion 33, the number of the protrusions 33, and the tip 35 of the protrusion 33 are detected based on the captured image obtained by at least one of the side illumination and the epi-illumination, and the contour shape model 44 including the shape and direction of the main body 32 and the protrusion model 45 including the position, number, and features of the tip 35 of the protrusion 33 are generated, and the insertion part model 46 including the position, number, and size of the insertion part is generated based on the information of the insertion part, and the component model 43 including the contour shape model 44, the protrusion model 45, and the insertion part model 46 is generated. Furthermore, in the mounting device 11, the shape of the main body 32, the position of the protrusion 33, the number of the protrusion 33, and the tip 35 of the protrusion are detected based on the captured image of the component 31, and the mounting part 20 is caused to arrange the component 31 on the processing object 30 using the component model 43. In the mounting device 11, a captured image of the component 31 picked up by the mounting section 20 is acquired by using the lighting used in the generated component model 43, and the shape of the main body 32, the position of the protrusion 33, the number of the protrusions 33, and the front end 35 of the protrusion 33 are detected based on the acquired captured image. In addition, the mounting device 11 causes the mounting section 20 to place the component 31 on the processing object 30 using the component model 43. Therefore, in the mounting device 11, when mounting the component 31 having the protrusion 33 protruding from the main body 32, the mounting process can be performed more reliably with a simpler structure.

[0051] In addition, the CPU 41 determines whether the component 31 can be arranged in the processing object 30 using the component model 43. In the component 31 having the protrusion 33, the protrusion 33 may not be inserted into the insertion part due to its shape due to deformation or the like. In the mounting device 11, the information that the component 31 can be used can be obtained in advance using the component model 43, and the mounting process can be performed more reliably. Moreover, the CPU 41 uses the insertion part model 46 to determine whether the protrusion 33 interferes when being inserted into the insertion part by assuming at least a first size and a second size smaller than the first size. In the mounting device 11, by changing the size of the insertion part to be smaller, it is possible to obtain information whether the component 31 can be used without interfering with the insertion part. In addition, the CPU 41 performs parallax correction on the captured image based on the center of the image to detect the position of the protrusion 33. In the imaging unit 15 having a depth of field of at least the distance between the main body 32 and the protrusion 33, since the main body 32 is farther than the front end 35 of the protrusion 33, parallax is more generated on the peripheral side of the image further deviated from the center of the image. In the mounting device 11, by correcting such parallax, the position of the protrusion 33 can be detected more appropriately, and the component model 43 can be used more appropriately. At this time, when the resolution per pixel of the tip 35 of the protrusion 33 is set to Rp, the resolution change per predetermined length, that is, the unit resolution is set to Ru, and the length of the protrusion from the main body is set to PL, the CPU 41 performs parallax correction of Rp-Ru×PL. In the mounting device 11, more appropriate parallax correction can be performed in consideration of the resolution and the length of the protrusion.

[0052] The mounting system 10 of this embodiment includes the above-mentioned mounting device 11 and a supply device 25 for supplying the component 31 to the placement area in an indefinite direction, and the mounting section 20 includes a multi-joint arm robot 21. In the mounting system 10, the component having the protrusion 31 can be easily supplied, and the component 31 can be arranged by inserting the protrusion 33 into the processing object 30 using the general-purpose multi-joint arm robot 21.

[0053] In the mounting device 11, the imaging unit 15 has a depth of field of at least the distance between the main body 32 and the protrusion 33, so that the main body 32 and the protrusion 33 can be photographed on the same image. In addition, the CPU 41 selects a photographed image obtained by at least one of side illumination and epi-illumination as a photographed image for setting the component model 43, so that a more appropriate component model 43 can be generated. At this time, the CPU 41 selects a photographed image in which the front end 35 of the protrusion 33 is easy to detect from among the photographed images of the side illumination and the photographed images of the epi-illumination, so that the front end 35 of the protrusion 33 of the component 31 can be detected by appropriately distinguishing between the use of side illumination and epi-illumination. In addition, the CPU 41 selects a photographed image in which the edge of the front end 35 of the protrusion 33 is easy to distinguish from the main body 32 from among the photographed images of the side illumination and the photographed images of the epi-illumination. In the control device 40, the photographed color of the edge of the tip 35 can be changed by either side lighting or epi-illumination. By making the photographed color of the edge different from the color of the main body 32, the tip 35 of the protrusion 33 can be more reliably identified.

[0054] In addition, the CPU 41 generates the component model 43 in which the protrusion region A including one or more protrusions 33 is included in the protrusion model 45. In the control device 40, the position of the protrusion 33 can be processed in units of regions, so the protrusion 33 can be recognized more efficiently. In addition, the CPU 41 generates the contour shape model 44 including the direction of the component 31, so the component model 43 including the direction of the component body can be constructed.

[0055] In addition, the mounting system 10 can use the mounting device 11 having the multi-joint arm robot 21 and the supply device 25 that carries the component 31 in an indefinite direction to perform the mounting process of inserting the terminal into the insertion portion, so that a general device can be used to perform a complex mounting process at a relatively low cost. Moreover, in the mounting system 10, if a trace parameter is input, the component model 43 can be generated, so the trouble of the operator's image processing setting can be suppressed to a minimum. In addition, for a special device using a special laser illumination and a special shooting part with a small depth of field, it is impossible to shoot the front end 35 and the main body 32 of the protrusion 33 in the same image, but for the mounting system 10, the front end 35 and the main body 32 can be shot by one shot, and the direction of the front end 35 and the main body 32 can be executed by one image, so the processing time can be shortened. Moreover, in the mounting system 10, for the component 31 with a deviation in the terminal length, the front end 35 can also be identified by one shot. Furthermore, the mounting system 10 also recognizes the terminals one by one and measures the distance between the terminals, etc., so that it is possible to detect and discard defective terminals such as broken or bent terminals, thereby further improving the workability at the production site.

[0056] In addition, the present disclosure is not limited to the above-mentioned embodiments, and it goes without saying that the present disclosure can be implemented in various forms as long as it belongs to the technical scope of the present disclosure.

[0057] For example, in the above-mentioned embodiment, the control device 40 of the mounting device 11 has the function of an image processing device, but the present invention is not particularly limited thereto, and the image processing device may be provided separately from the mounting device 11. Similarly, in the above-mentioned embodiment, the CPU 41 has the functions of an image control unit and a mounting control unit, but the present invention is not particularly limited thereto. The image control unit may also be provided separately from the CPU 41. In the mounting device 11, it is also possible to more reliably perform mounting processing with a simpler structure when mounting a component having a protruding portion protruding from the main body.

[0058] In the above-mentioned embodiment, the outline shape model 44 specifies the size of the main body 32 in addition to the shape and direction of the main body 32 (polarity of the terminal), but is not limited to this, and may include other factors, or may at least include the shape and direction of the main body 32. In addition, in the above-mentioned embodiment, the protrusion model 45 includes the position, number, characteristics of the front end, size of the protrusion 33, and information of the protrusion area A, but is not limited to this, and may include other factors, or may at least include the position, number, and characteristics of the front end 35 of the protrusion 33. Moreover, in the above-mentioned embodiment, the insertion part model 46 includes information such as the position, number, characteristics including the shape of the insertion part, and size of the insertion part including the spacing of the insertion part, but is not limited to this, and may include other factors, or may at least include the position, number, and size of the insertion part. In addition, in the above-mentioned embodiment, the component model 43 includes the outline shape model 44, the protrusion model 45, and the insertion part model 46, but is not particularly limited to this. For example, the insertion part model 46 may be omitted. The component model 43 may appropriately include information that enables the protrusion 33 to be inserted into the insertion part.

[0059] In the above-mentioned embodiment, when performing parallax correction of the captured image, the resolution Rp, the unit resolution Ru, and the protrusion length PL are used to perform parallax correction of Rp-Ru×PL, but the present invention is not limited thereto, and other factors may be used in addition to or instead of these factors to perform parallax correction. Alternatively, the CPU 41 may omit the parallax correction. In this case, the CPU 41 is expected to perform processing for recognizing the front end portion 35 in a wider range without using the protrusion area A.

[0060] In the above-mentioned embodiment, the CPU 41 generates the component model using any one of the captured images captured by the side illumination and the captured images captured by the epi-illumination, but it is not particularly limited thereto, and the component model can also be generated based on the captured images of the side illumination and the epi-illumination simultaneously lit. In addition, the CPU 41 can also perform image processing on the captured images of the side illumination, perform image processing on the captured images of the epi-illumination, synthesize the two image processing results and finally obtain one image processing result, thereby generating the component model. The CPU 41 can select an appropriate lighting method and image processing method according to the component 31. In addition, when the CPU 41 uses the component model to perform image processing on the component 31 picked up by the mounting portion 20, it can also adopt the same lighting method and image processing method as described above.

[0061] In the above-described embodiment, the case where the protrusion 33 of the element 31 is a terminal pin has been described, but the protrusion 33 is not particularly limited thereto as long as it is a portion to be inserted into the insertion portion (hole portion) of the treatment target 30 .

[0062] In the above-mentioned embodiment, the mounting portion 20 is described as having a multi-joint arm robot 21, but the present invention is not particularly limited thereto as long as the protrusion 33 can be inserted into the insertion portion and the component 31 can be arranged on the processing object 30. For example, the structure of an XY robot may also be provided. Fig.11 1 is an explanatory diagram showing an example of another mounting system 10B. The mounting system 10B includes a mounting device 11B and a management device 60. The mounting device 11B includes: a conveying device 12B for conveying a substrate 30B, a supply device 25B including a feeder 29, a photographing unit 14 capable of performing side illumination and epi-illumination, a mounting section 20B, and a control device 40. The mounting section 20B includes: a mounting head 21B, a pickup component 22B mounted on the mounting head 21B, and a head moving section 24 for moving the mounting head 21B in the XY direction. The photographing unit 14 and the control device 40 are the same as those in the above-mentioned embodiment. Even when such a mounting device 11B is used, it is possible to more reliably perform the mounting process by a simpler structure when mounting a component having a protrusion protruding from a main body.

[0063] In the above-mentioned embodiment, the image processing device of the present invention is described as the control device 40, but it is not particularly limited to this, and it may be an image processing method, or the image processing method may be a program executed by a computer. In addition, in the above-mentioned embodiment, the installation device of the present invention is described as the installation device 11, but it is not particularly limited to this, and it may be an installation method, or the installation method may be a program executed by a computer.

[0064] Here, the image processing device and the mounting device disclosed in the present invention may also be configured as follows. For example, in the image processing device disclosed in the present invention, the image control unit may generate a component model including an insertion unit model based on the information of the insertion unit, and the insertion unit model includes the position, quantity and size of the insertion unit. In the image processing device, a component model including the information of the insertion unit can be generated.

[0065] The installation device disclosed in the present invention comprises: a mounting portion, which inserts the protrusion of an element having a main body and one or more protrusions formed on the above-mentioned main body into an insertion portion formed on a processing object and arranges the above-mentioned element on the above-mentioned processing object; a lighting portion, which can light up side lighting for irradiating light on the above-mentioned element from the side and falling lighting for irradiating light on the above-mentioned element from below; a photographing portion, which has a depth of field of at least the distance between the above-mentioned main body and the above-mentioned protrusion and photographs the above-mentioned element from below; and a mounting control portion, which detects the shape of the above-mentioned main body, the position of the above-mentioned protrusion, the number of the above-mentioned protrusions and the front end portion of the above-mentioned protrusion based on a photographed image obtained by photographing the above-mentioned element picked up by the above-mentioned mounting portion using the lighting used in the above-mentioned element model generated by the above-mentioned image processing device, and uses the above-mentioned element model generated by the above-mentioned image processing device to make the above-mentioned mounting portion arrange the element on the above-mentioned processing object.

[0066] In the installation device, a captured image is obtained by photographing a component picked up by the installation unit using side lighting and / or falling lighting used in the component model generated by the above-mentioned image processing device, and the shape of the main body, the position of the protrusion, the number of protrusions, and the front end of the protrusion are detected based on the acquired captured image. In addition, the installation device uses the component model generated by the image processing device to make the installation unit configure the component to the processing object. Therefore, in the installation device, when installing a component having a protrusion protruding from the main body, the installation process can be performed more reliably with a simpler structure. In addition, in the installation device, an effect corresponding to the method using the above-mentioned image processing device can be obtained.

[0067] Alternatively, in the installation device disclosed herein, the installation control unit may use the component model to determine whether the component can be placed on the processing object. In a component having a protruding portion, the protruding portion may not be inserted into the insertion portion due to deformation of the protruding portion or the like depending on its shape. In this installation device, information on the usability of the component can be acquired in advance using the component model, and the installation process can be performed more reliably.

[0068] Alternatively, in the installation device disclosed in the present invention, the installation control unit uses the insertion portion model to assume at least a first size and a second size smaller than the first size of the insertion portion to determine whether interference will occur when the protrusion is inserted into the insertion portion. In this installation device, by using multiple sizes of the insertion portion, information can be obtained on whether the component can be used without interfering with the insertion portion, and the installation process can be performed more reliably.

[0069] Alternatively, in the installation device disclosed in the present invention, the installation control unit performs parallax correction on the captured image based on the center of the image, and detects the position of the protrusion. In the shooting unit having a depth of field of at least the distance between the main body and the protrusion, since the main body is farther than the front end of the protrusion, more parallax is generated on the peripheral side of the image that is further away from the center of the image. In the installation device, by correcting such parallax, the position of the protrusion can be more appropriately detected and the component model can be used more appropriately. At this time, it is also possible that when the resolution of each pixel at the front end of the protrusion is set to Rp, the resolution change per predetermined length, that is, the unit resolution, is set to Ru, and the length of the protrusion from the main body is set to PL, the installation control unit performs the parallax correction of Rp-Ru×PL. In the installation device, more appropriate parallax correction can be performed in consideration of the resolution and the length of the protrusion.

[0070] The mounting system disclosed in the present invention comprises: the mounting device described in any one of the above items; and a supply device for supplying the components to the loading area in an indefinite direction, and the mounting part has a multi-joint arm robot. In the mounting system, the components with protruding parts can be easily supplied, and a relatively common multi-joint arm robot can be used to arrange them on the processing object.

[0071] The image processing method disclosed in the present invention is used for an installation device having an installation portion, wherein the installation portion allows an element having a main body and one or more protrusions formed on the main body to be inserted into an insertion portion formed on an object to be processed and the element is arranged on the object to be processed, and the image processing method comprises the following steps: (a) acquiring a captured image, wherein the captured image is captured under side illumination and / or epi-illumination of a capturing unit, wherein the capturing unit comprises: an illumination unit capable of lighting the side illumination for illuminating the element from the side and the epi-illumination for illuminating the element from below; and a capturing unit having at least the main body and the protrusions. (b) detecting the shape of the main body, the position of the protrusion, the number of the protrusions and the front end of the protrusions according to the acquired captured image; (c) acquiring information of the insertion part; (d) generating a component model including a contour shape model, a protrusion model and an insertion part model based on the detection result of the step (b) and the information acquired in the step (c), wherein the contour shape model includes the shape and direction of the main body, the protrusion model includes the position, number and front end features of the protrusions, and the insertion part model includes the position, number and size of the insertion part.

[0072] In this image processing method, similar to the above-mentioned image processing device, no special lighting unit or shooting unit is required, and a shooting image that can more reliably detect the protrusion can be obtained by using multiple lighting modes, and the main body and the protrusion can be shot at one time. Therefore, in this image processing method, when installing a component having a protrusion protruding from the main body, the installation process can be more reliably performed with a simpler structure. In addition, in this image processing method, various forms of the above-mentioned image processing device can also be adopted, and steps that realize the various functions of the above-mentioned image processing device can also be added.

[0073] The installation method disclosed in the present invention is performed by an installation device, which comprises: an installation part, which enables an element having a main body and one or more protrusions formed on the main body to insert the protrusions into an insertion part formed on a processing object and arrange the element on the processing object; an illumination part, which can light up side illumination for irradiating light on the element from the side and downward illumination for irradiating light on the element from below; and a photographing part, which has a depth of field of at least the distance between the main body and the protrusions and photographs the element from below, and the installation method comprises the following steps: (e) obtaining a photographed image of the element picked up by the installation part using the illumination used in the element model generated by the image processing method; and (f) detecting the shape of the main body, the position of the protrusions, the number of the protrusions and the front end of the protrusions based on the acquired photographed image, and using the element model to enable the installation part to arrange the element on the processing object.

[0074] In this installation method, the shape of the main body, the position of the protrusion, the number of protrusions, and the front end of the protrusion are detected based on the captured image obtained by using the lighting used in the component model generated by any of the above-mentioned image processing devices. Moreover, in this installation method, the component model is used to make the installation part configure the component to the processing object. Therefore, in this installation method, when installing a component with a protrusion protruding from the main body, the installation process can be performed more reliably through a simpler structure. In addition, an effect corresponding to the method of the image processing method adopted can be obtained. In addition, in this installation method, various methods of the above-mentioned installation device can also be adopted, and steps such as realizing the various functions of the above-mentioned installation device can also be added.

[0075] Industrial Applicability

[0076] The image processing device, mounting device, mounting system, and image processing method disclosed herein can be utilized in, for example, the field of mounting electronic components.

[0077] Description of Reference Numerals

[0078] 10, 10B...Installation system 11, 11B...Installation device 12, 12B...Transportation device 13...Transportation unit 14...Image capture unit 15...Image capture unit 16...Illumination unit 17...Sidelight 18...Downlight 20, 20B...Installation unit 21...Multi-joint arm robot 21B...Installation head 22, 22B...Pickup component 23...Camera 24...Head moving unit 25, 25B...Supply device 26...Supply placement unit 27...Vibration unit 28...Component supply unit 29...Material feeding Device 30...Processing object 30B...Substrate 31...Component 32...Main body 33...Protrusion 34...Taper 35...Front end 40...Control device 41...CPU 42...Storage unit 43...Component model 44...Contour shape model 45...Protrusion model 46...Insertion model 47...Display device 48...Input device 51...Component image 52...Main body 53...Protrusion 55...Front end 56...Insertion 60...Management device A...Protrusion area

Claims

1. An image processing device for use in an installation device, the installation device comprising an installation portion, the installation portion inserting the protrusion of an element having a main body and one or more protrusions formed on the main body into an insertion portion formed on a processing object and arranging the element on the processing object, wherein: The image processing device includes an image control unit, which acquires a captured image, detects the shape of the main body, the position of the protrusion, the number of the protrusions and the front end of the protrusion based on the acquired captured image, acquires information of the insertion part, and generates a component model including a contour shape model and a protrusion model, wherein the captured image is obtained by capturing under side lighting and / or epi-illumination of a capturing unit, the capturing unit has an illumination unit capable of illuminating the side lighting for irradiating light to the component from the side and the epi-illumination for irradiating light to the component from below, and a capturing unit having a depth of field of at least the distance between the main body and the protrusion and capturing the component from below, the contour shape model includes the shape and direction of the main body, and the protrusion model includes the characteristics of the position, number and front end of the protrusion.

2. The image processing device according to claim 1, wherein: The image control unit generates a component model further including an insertion unit model based on the information of the insertion unit, the insertion unit model including the position, number, and size of the insertion unit.

3. A mounting device comprising: A mounting portion that inserts the protrusions of a component having a main body and one or more protrusions formed on the main body into an insertion portion formed on the object to be processed and arranges the component on the object to be processed; an illumination unit capable of lighting a side illumination for illuminating the element from the side and a falling illumination for illuminating the element from below; a photographing unit having a depth of field of at least the distance between the main body and the protruding portion and photographing the element from below; and The installation control unit detects the shape of the main body, the position of the protrusions, the number of the protrusions and the front end portion of the protrusions based on a captured image obtained by photographing the component picked up by the installation unit using the lighting used in the component model generated by the image processing device of claim 2, and uses the component model generated by the image processing device to cause the installation unit to configure the component to the processing object.

4. The mounting device according to claim 3, wherein: The mounting control unit determines whether the component can be arranged on the processing object using the component model.

5. The mounting device according to claim 3, wherein: The mounting control unit uses the insertion unit model to assume at least a first size and a second size smaller than the first size of the insertion unit, and determines whether interference will occur when the protrusion is inserted into the insertion unit.

6. The mounting device according to claim 4, wherein: The mounting control unit uses the insertion unit model to assume at least a first size and a second size smaller than the first size of the insertion unit, and determines whether interference will occur when the protrusion is inserted into the insertion unit.

7. The mounting device according to any one of claims 3 to 6, wherein: The mounting control unit performs parallax correction on the captured image with the center of the image as a reference, and detects the position of the protruding portion.

8. The mounting device according to claim 7, wherein: When the resolution of each pixel at the tip of the protrusion is set to Rp, the resolution change per predetermined length, i.e., the unit resolution, is set to Ru, and the length of the protrusion from the main body is set to PL, the mounting control unit performs the parallax correction of Rp-Ru×PL.

9. A mounting system comprising: The mounting device according to any one of claims 3 to 8; and A supply device supplies the component to the loading area in an indefinite direction, The installation part has a multi-joint arm robot.

10. An image processing method for an installation device, the installation device comprising an installation portion, the installation portion inserting the protrusion of an element having a main body and one or more protrusions formed on the main body into an insertion portion formed on a processing object and arranging the element on the processing object, wherein: The image processing method comprises the following steps: (a) acquiring a captured image, wherein the captured image is captured under side illumination and / or epi-illumination of a capturing unit, wherein the capturing unit comprises: an illumination unit capable of lighting the side illumination for illuminating the element from the side and the epi-illumination for illuminating the element from below; and a capturing unit having a depth of field of at least the distance between the main body and the protruding portion and capturing the element from below; (b) detecting the shape of the main body, the position of the protrusion, the number of the protrusions, and the front end of the protrusion based on the acquired captured image; (c) obtaining information of the insertion portion; and (d) Based on the detection result of step (b) and the information obtained in step (c), generate a component model including a contour shape model, a protrusion model and an insertion part model, wherein the contour shape model includes the shape and direction of the main body, the protrusion model includes the position, number and front end characteristics of the protrusion, and the insertion part model includes the position, number and size of the insertion part.

11. A mounting method, performed by a mounting device, the mounting device comprising: a mounting portion, which inserts the protruding portion of a component having a main body and one or more protruding portions formed on the main body into an insertion portion formed on a processing object and arranges the component on the processing object; an illumination unit capable of lighting side lighting for illuminating the element from the side and epi-illumination for illuminating the element from below; and a photographing unit having a depth of field of at least the distance between the main body and the protruding portion and photographing the element from below, wherein The installation method comprises the following steps: (e) acquiring a photographed image obtained by photographing the component picked up by the mounting section using the lighting used in the component model generated by the image processing method according to claim 10; and (f) detecting the shape of the main body, the positions of the protrusions, the number of the protrusions, and the tip ends of the protrusions based on the acquired captured image, and causing the mounting section to place the component on the processing object using the component model.

Images