Image processing system, mounting device, and abnormality determination method
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- FUJI CORP
- Filing Date
- 2025-01-24
- Publication Date
- 2026-07-30
Smart Images
Figure JP2025002326_30072026_PF_FP_ABST
Abstract
Description
Image Processing System, Mounting Device, and Abnormality Determination Method
[0001] This specification discloses an image processing system, a mounting device, and an abnormality determination method.
[0002] Conventionally, when mounting a component on a substrate, there is known a method of processing an image obtained by imaging the component adsorbed by an adsorption member or the side surface of the adsorption member. For example, in Patent Document 1, in a configuration in which the adsorption member is disposed slidably in the vertical direction and biased downward by a biasing member, the side surface image of the adsorption member is processed to determine whether the tip of the adsorption member is within a preset normal range, and if it is not within the normal range, it is determined as abnormal.
[0003] Japanese Patent Application Laid-Open No. 2006-114534
[0004] Here, the determination of abnormalities in the adsorption member, such as a stacked state where the adsorption member does not return from an upwardly slid state or dirt on the side surface of the adsorption member, is an important issue for correctly adsorbing and mounting components. Therefore, it is required to perform such abnormality determination with higher accuracy.
[0005] The main object of the present disclosure is to more accurately determine abnormalities in the stacked state and side surface dirt of the adsorption member using the side surface image of the adsorption member.
[0006] The present disclosure has adopted the following means to achieve the above main object.
[0007] The present disclosure provides an image processing system for processing an image of a suction member in a mounting device that detachably holds a suction member for picking up components from a supply unit and mounting them on a substrate, wherein the suction member is slidable in the vertical direction and biased downward by a biasing member, and slides upward against the biasing force when pressed against the supply unit to pick up components and when pressed against the substrate to mount components, and comprises: an acquisition unit that acquires a side image of the suction member held by the mounting device; a learning unit that learns a plurality of side images of the suction members acquired by the acquisition unit and creates a determination model for determining abnormalities of the suction member; and a determination unit that processes an image of the side image of the suction member to be determined acquired by the acquisition unit and applies it to the determination model to determine a stuck state in which the suction member to be determined does not return from a state in which it has slid upward and / or an abnormality of dirt on the side of the suction member.
[0008] This image processing system processes a side view image of the adsorption material to be evaluated and applies it to a judgment model to determine the stacking state and / or any abnormalities in the side surface contamination of the adsorption material. Because the system creates a judgment model capable of detecting abnormalities in the adsorption material with high accuracy through learning, it can more accurately determine the stacking state and side surface contamination abnormalities of the adsorption material. Furthermore, this image processing system contributes to the realization of smart factories.
[0009] A perspective view showing an outline of the configuration of the implementation system 1. A configuration diagram showing an outline of the side configuration of the head unit 50. A configuration diagram showing an outline of the bottom configuration of the head unit 50. A block diagram showing the electrical connection relationships of the implementation system 1. A flowchart showing an example of the judgment model creation process. An explanatory diagram showing an example of how the target image It is extracted from the side image Is. An explanatory diagram showing an example of the target image It with dirt D on the nozzle 51. A flowchart showing an example of the nozzle abnormality judgment process. An explanatory diagram showing an example of how the target image It is extracted from the side image Is. An explanatory diagram showing an example of the side image Is of a comparative example.
[0010] Next, embodiments of the present disclosure will be described with reference to the drawings. Figure 1 is a perspective view showing an outline of the configuration of the mounting system 1. Figure 2 is a configuration diagram showing an outline of the side configuration of the head unit 50. Figure 3 is a configuration diagram showing an outline of the bottom configuration of the head unit 50. Figure 4 is a block diagram showing the electrical connection relationships of the mounting system 1. The mounting system 1 includes a mounting device 10 that performs mounting processing to mount components P such as electronic components onto a substrate S, and a management device 100 that manages the entire system. In this embodiment, the left-right direction in Figure 1 is the X-axis direction, the front-back direction is the Y-axis direction, and the up-down direction is the Z-axis direction.
[0011] As shown in Figure 1, the mounting device 10 includes a component supply device 20, a substrate transport device 30, a moving device 40, a head unit 50, and a control device 90 (see Figure 4).
[0012] The component supply device 20 is, for example, a tape feeder equipped with a reel 22 containing components P on tape at predetermined intervals, and supplies the components P to the supply position by pulling the tape from the reel 22 by the drive of a motor (not shown). The substrate transport device 30 is, for example, equipped with a pair of conveyor belts 32 that are spaced apart in the front-to-back direction (Y-axis direction) and spanned in the left-to-right direction, and transports the substrate S from left to right in Figure 1 by driving the conveyor belts 32 by the drive of a motor (not shown). The moving device 40 is equipped with a guide rail 46 provided along the Y-axis direction, a Y-axis slider 48 that moves along the guide rail 46, a guide rail 42 provided on the Y-axis slider 48 along the X-axis direction, and an X-axis slider 44 that moves along the guide rail 42. A head unit 50 is attached to the X-axis slider 44. The moving device 40 moves the head unit 50 in the XY direction by moving the X-axis slider 44 and the Y-axis slider 48.
[0013] As shown in Figure 2, the head unit 50 includes a cylindrical rotary head 54 on which a plurality of nozzles 51 for adsorbing parts P and a plurality of nozzle holders 52 for holding the nozzles 51 are arranged. The head unit 50 also includes an R-axis actuator 56 for rotating the rotary head 54, a Q-axis actuator 59 (see Figure 4) for individually rotating each nozzle holder 52, a Z-axis actuator 60 for moving the nozzle holders 52 in the Z-axis direction, and a side camera 70 for imaging the sides of the nozzles 51 and parts P.
[0014] The nozzle holder 52 is configured as a hollow cylindrical member extended in the Z-axis direction. The upper end portion 52a of the nozzle holder 52 is formed in a circular shape that is slightly larger than the shaft portion of the nozzle holder 52. The nozzle holder 52 also has a flange portion 52b formed at a predetermined position below the upper end portion 52a, which has a diameter slightly larger than the shaft portion. A spring (coil spring) 55 is positioned between the annular surface below this flange portion 52b and a recess (not shown) formed on the upper surface of the rotary head 54. Therefore, the spring 55 biases the nozzle holder 52 (flange portion 52b) upward, using the recess on the upper surface of the rotary head 54 as a spring receiver.
[0015] The nozzle 51 comprises a tip portion 51a (nozzle tip) that attracts a part P at its tip (lower end), a substantially cylindrical nozzle pipe 51b to which the tip portion 51a is attached, and a substantially cylindrical sleeve 51c that holds the nozzle pipe 51b so as to be slidable in the vertical direction. The nozzle 51 is detachably attached to the lower end of the nozzle holder 52. The nozzle 51 (sleeve 51c) is also slidable in the vertical direction relative to the inner circumferential surface of the nozzle holder 52 while attached to the nozzle holder 52. Inside the nozzle holder 52 are an inner spring 53a with the lower surface of a stepped portion (not shown) positioned as a spring receiver, and an inner pipe 53b that is slidable in the vertical direction inside the nozzle holder 52. The inner pipe 53b is positioned between the inner spring 53a and the nozzle 51 so as to abut the lower end of the inner spring 53a and abut the upper end of the nozzle 51 (nozzle pipe 51b). Therefore, the inner spring 53a biases the nozzle 51 downward via the inner pipe 53b. In this way, the nozzle 51 of this embodiment is mounted on the nozzle holder 52 so that it can slide vertically and is biased downward by the inner spring 53a. Furthermore, the nozzle 51 attracts the part P by supplying negative pressure through the nozzle holder 52 (inner pipe 53b) to an air passage that penetrates vertically along the central axis between the nozzle pipe 51b and the tip portion 51a.
[0016] As shown in Figure 3, the rotary head 54 has multiple (for example, 12) nozzles 51 arranged at predetermined angular intervals (for example, 30-degree intervals) on a circumference coaxial with the rotation axis. A cylindrical reflector 54a capable of reflecting light is attached to the center of the lower surface of the rotary head 54. The Q-axis actuator 59, although not shown in the figure, comprises a drive gear meshed with a gear provided on the cylindrical outer circumference of the nozzle holder 52, and a drive motor connected to the rotation axis of the drive gear. Therefore, in this embodiment, multiple nozzle holders 52 can each rotate individually around the axis (Q direction), and each nozzle 51 can also rotate individually.
[0017] As shown in Figure 2, the R-axis actuator 56 comprises a rotating shaft 57 connected to the rotary head 54 and a drive motor 58 connected to the rotating shaft 57. The R-axis actuator 56 intermittently rotates the rotary head 54 by a predetermined angle (for example, 30 degrees) by intermittently driving the drive motor 58. As a result, each nozzle 51 positioned on the rotary head 54 moves by a predetermined angle in the circumferential direction. Of the multiple positions to which the nozzle 51 can move, the position at 12 o'clock in Figure 3 is called the working position A0.
[0018] As shown in Figure 2, the Z-axis actuator 60 is configured as a feed screw mechanism comprising a screw shaft 64 extending in the Z-axis direction and moving a ball screw nut 62, a Z-axis slider 66 attached to the ball screw nut 62, and a drive motor 68 whose rotation axis is connected to the screw shaft 64. The Z-axis actuator 60 moves the Z-axis slider 66 in the Z-axis direction by rotationally driving the drive motor 68. The Z-axis slider 66 has a substantially L-shaped lever portion 67 that protrudes toward the rotary head 54. The lever portion 67 can contact the upper end portion 52a of the nozzle holder 52 located within a predetermined range including the working position A0. As the Z-axis slider 66 moves in the Z-axis direction, the lever portion 67 moves in the Z-axis direction, causing the nozzle holder 52 and the nozzle 51 attached to the nozzle holder 52 to move in the Z-axis direction. Therefore, at the working position A0, the nozzle 51 can pick up parts P supplied from the parts supply device 20 and mount parts P onto the substrate S. In Figure 3, the 11 o'clock position is the position immediately before (just before) working position A0 when the nozzle 51 moves in the circumferential direction (direction of the arrow in the figure), and is called the position A1 immediately before suction or attachment. In Figure 3, the 1 o'clock position is the position immediately after (just after) working position A0 when the nozzle 51 moves in the circumferential direction, and is called the position A2 immediately after suction or attachment.
[0019] As shown in Figure 3, the side camera 70 consists of a camera body 72 located at the bottom of the head unit 50 and an optical system unit 74 that forms an optical path to the camera body 72. The optical system unit 74 has three entrance ports on the rotary head 54 side: a central entrance port 76a, a left entrance port 76b, and a right entrance port 76c, and an exit port 76d on the camera body 72 side. The central entrance port 76a is located opposite the working position A0. The left entrance port 76b is located opposite the immediate forward position A1. The right entrance port 76c is located opposite the immediate backward position A2. The optical system unit 74 also includes multiple light-emitting elements 77, such as LEDs, that emit light toward the reflector 54a of the rotary head 54, and mirrors that reflect light and prisms that refract light to form the optical paths from each entrance port 76a, 76b, 76c to the exit port 76d. As a result, the camera body 72 captures images of the sides of the nozzle 51 and the side of the part P at the work position A0, the immediate front position A1, and the immediate back position A2, and generates captured images (side images).
[0020] The mounting device 10 also includes a parts camera 80, a mark camera 82, and a nozzle stocker 84. The parts camera 80 is located between the parts supply device 20 and the substrate transport device 30. The parts camera 80 has an imaging range above it and captures the lower surface of an object such as a part P that is attracted to the nozzle 51 to generate an image (lower surface image).
[0021] The mark camera 82 is located on the underside of the X-axis slider 44. The mark camera 82 captures the top surface of an object and generates an image (top image). Examples of objects that the mark camera 82 can capture include parts P supplied from the tape feeder of the parts supply device 20, marks on a circuit board S, and marks (ID marks) on nozzles 51 in the nozzle stocker 84.
[0022] The nozzle stocker 84 is configured to accommodate multiple nozzles 51 in each of its storage compartments. The nozzles 51 stored in the nozzle stocker 84 can be automatically replaced by the rotary head 54. Furthermore, while the mounting device 10 is stopped, the operator can remove nozzles 51 that are not needed for the mounting process from the nozzle stocker 84 and store the nozzles 51 that are needed for the mounting process. The nozzles 51 that are needed for the mounting process are, for example, nozzles 51 of a size and shape corresponding to the type of part to be mounted.
[0023] As shown in Figure 4, the control device 90 is configured as a microprocessor centered around a CPU 91, and in addition to the CPU 91, it includes a ROM 92, HDD 93, RAM 94, input / output interface (I / F) 95, etc. These are connected via a bus 96. The control device 90 receives image signals from the side camera 70, part camera 80, and mark camera 82 via the input / output interface 95. Position sensors (not shown) are provided on the X-axis slider 44, Y-axis slider 48, R-axis actuator 56, Q-axis actuator 59, and Z-axis actuator 60, and the control device 90 also receives position information from these position sensors. Furthermore, the control device 90 outputs drive signals to the component supply device 20, the substrate transport device 30, the X-axis actuator 45 that moves the X-axis slider 44, the Y-axis actuator 49 that moves the Y-axis slider 48, the R-axis actuator 56 (drive motor 58), the Q-axis actuator 59 (drive motor), the Z-axis actuator 60 (drive motor 68), the nozzle stocker 84, etc., via the input / output interface 95. The control device 90 also stores the judgment model 93a, etc., in the HDD 93. The judgment model 93a was created through learning to determine abnormalities in the nozzle 51, and its details will be described later.
[0024] The management device 100 is, for example, a general-purpose computer. As shown in Figure 4, the management device 100 includes a CPU 101, ROM 102, an HDD 103 for storing production jobs for the substrate S, RAM 104, an input / output interface 105, and the like. These are connected via a bus 106. The management device 100 receives input signals from input devices 107 such as a mouse or keyboard via the input / output interface 105. The management device 100 also outputs image signals to the display 108 via the input / output interface 105. Here, the production job for the substrate S is information that defines which components P to mount on the substrate S in what order by the mounting device 10, and how many substrates S with components P mounted in that manner to be manufactured. The production job also includes information about the components P to be mounted, such as the mounting order and mounting position of the components P, the type, size, and shape of the components P, and the type, size, and shape of the nozzle 51 suitable for picking up the components P. This production job is transmitted from the control device 100 to the mounting device 10.
[0025] Next, the operation of the mounting device 10 of the mounting system 1 configured in this way will be described. In the process of mounting components P onto a substrate S, the control device 90 first transports the substrate S to a predetermined position using the substrate transport device 30 and holds it there. Next, the control device 90 has the component supply device 20 supply components P to the supply position, and moves the head unit 50 above the supply position using the moving device 40 to pick up the components P with the nozzle 51. Subsequently, the control device 90 moves the head unit 50 above the parts camera 80 using the moving device 40, and has the parts camera 80 capture an image of the components P that have been picked up by the nozzle 51. The control device 90 processes the captured image to correct the target mounting position of the components P so that any misalignment of the components P is eliminated, and moves the head unit 50 using the moving device 40 so that the nozzle 51 is positioned above the target mounting position. Furthermore, the control device 90 has the side camera 70 capture images of the nozzle 51 immediately after picking up the component P and immediately before mounting, and determines from these side images whether there are any abnormalities in the nozzle 51 or any abnormalities in the picking of the component P. After determining that there are no such abnormalities, the control device 90 mounts the component P to the target mounting position. Once the mounting of the necessary components P is complete, the control device 90 releases the substrate S from the substrate transport device 30 and transports it out of the device. The control device 90 performs this mounting process based on production jobs transmitted from the management device 100.
[0026] As described above, the nozzle 51 attached to the nozzle holder 52 is biased downward by the inner spring 53a. Also, when the nozzle 51 moves downward to pick up a component P supplied to the supply position by the component supply device 20, it is pressed against the component supply device 20 (component P) and receives an upward reaction force, causing it to slide upward against the biasing force of the inner spring 53a. Similarly, when the nozzle 51 moves downward to mount a component to the mounting position (target mounting position) on the substrate S, it is pressed against the substrate S (component P) and receives an upward reaction force, causing it to slide upward against the biasing force of the inner spring 53a. When the nozzle 51 no longer receives these reaction forces, it is biased downward by the inner spring 53a and returns to its original position. However, due to aging deterioration of the inner spring 53a or contamination of foreign matter, an abnormality may occur in which the nozzle 51 is not properly biased and does not return to its original position, a so-called stuck state (hereinafter referred to as "stuck"). In addition to stacking, dirt may also accumulate on the side surface of the tip 51a of the nozzle 51 due to the attachment of foreign matter. In this embodiment, such abnormalities of the nozzle 51, such as stacking and dirt, are determined using a side view image. The details of abnormality determination using the side view image will be described below.
[0027] First, the creation of the judgment model 93a used for abnormality detection will be explained. Figure 5 is a flowchart showing an example of the judgment model creation process. This flowchart is executed, for example, by the CPU 101 of the management device 100, but may also be executed by the CPU 91 of the control device 90. In the judgment model creation process, the CPU 101 first collects side images Is of nozzles 51 for each type of nozzle 51, for which the presence or absence of abnormalities such as stacking or dirt is known (S100). For example, as side images Is, multiple images are collected taken by the side camera 70 when the nozzle 51, which is in a state where the presence or absence of abnormalities such as stacking or dirt has been determined, is attached to the rotary head 54 (nozzle holder 52). Next, the CPU 101 obtains a target image It from the collected side images Is, which is a predetermined range with the ideal position of the tip of the nozzle 51 as the lower end (S110). The ideal position of the tip of the nozzle 51 is a position determined according to the size (type) of the nozzle 51 when there are no abnormalities such as stacking in the nozzle 51.
[0028] Figure 6 is an explanatory diagram showing an example of how a target image It is extracted from a side image Is. Figure 6A is an example of a side image Is acquired in S100. Figure 6B is an example of a target image It acquired in S110. As the side image Is in Figure 6A, for example, an image of a nozzle 51 that is known to be free from stacking or dirt abnormalities is shown. The ideal position of the tip of the nozzle 51 is defined as the lower end line BL. The range below the lower end line BL is deleted from the side image Is in Figure 6A, and the range above the lower end line BL (a predetermined range) is extracted to obtain the target image It. As a result, the target image It is an image that includes only the nozzle 51 and no parts P. Figure 7 is an explanatory diagram showing an example of a target image It with dirt D on the nozzle 51. The target image It in Figure 7 is extracted from the side image Is of a nozzle 51 that is known to have dirt D. The side image Is of a nozzle 51 with stacking will be described later.
[0029] Next, the CPU 101 learns the target image It for each type of nozzle 51, creates a judgment model for determining whether or not there is an abnormality (S120), registers it as judgment model 93a in the HDD 93 of the mounting device 10 (S130), and terminates this process. The judgment model is created by learning using multiple target images It and whether or not there is an abnormality in the nozzle 51 of each target image It as training data. There are no particular limitations on the learning method, but well-known methods such as autoencoders and anomaly detection can be used.
[0030] Next, the process for determining abnormalities in the nozzle 51 during the mounting process will be described. Figure 8 is a flowchart showing an example of the nozzle abnormality determination process. This flowchart is executed, for example, by the CPU 91 of the control device 90, but may also be executed by the CPU 101 of the management device 100. In the nozzle abnormality determination process, the CPU 91 determines whether there is a nozzle 51 that is about to pick up a component (S200), whether there is a nozzle 51 that is about to mount a component (S210), and whether there is a nozzle 51 that has just mounted a component (S220). For example, when the CPU 91 moves the head unit 50 above the supply position of the component P by the component supply device 20 and sequentially picks up the component P with each nozzle 51, if a nozzle 51 that has not picked up a component P moves to the previous position A1, the CPU 91 determines in S200 that there is a nozzle 51 that is about to pick up a component. Furthermore, when the CPU 91 moves the head unit 50 above the substrate S and sequentially mounts the components P held by each nozzle 51 onto the substrate S, if the nozzle 51 holding a component P moves to the previous position A1, the CPU 91 determines in S210 that there is a nozzle 51 that is about to mount a component. In addition, if the nozzle 51 that has mounted a component P moves to the next position A2, the CPU 91 determines in S220 that there is a nozzle 51 that has just mounted a component.
[0031] If the CPU 91 determines in S200 to S220 that there are no nozzles 51, it terminates the nozzle abnormality detection process. On the other hand, if the CPU 91 determines in S200 to S220 that there is any nozzle 51, it targets that nozzle 51 for detection and captures a side image Is with the side camera 70 (S230). Next, the CPU 91 obtains a target image It extracted from the side image Is (S240). In S240, the CPU 91 obtains the target image It by deleting the area below the lower end line BL, which corresponds to the ideal position of the tip of the nozzle 51, and extracting the area above it (a predetermined area), similar to S110 described above.
[0032] Next, the CPU 91 processes the acquired target image It and applies it to a determination model 93a corresponding to the type of nozzle 51 to be judged to determine whether there are any abnormalities such as stacking or dirt (S250), and determines whether there are any abnormalities (S260). If the CPU 91 determines that there are no abnormalities, it terminates the nozzle abnormality determination process. On the other hand, if the CPU 91 determines that there are abnormalities, it executes abnormality response processing (S270) and terminates the nozzle abnormality determination process. In the abnormality response processing of S270, the CPU 91 executes processes such as storing the abnormal nozzle 51 in the nozzle stocker 84 and attaching another nozzle 51 of the same type to the nozzle holder 52. Alternatively, the CPU 91 may execute processes such as notifying the operator of the abnormality of the nozzle 51.
[0033] Figure 9 is an explanatory diagram showing an example of how a target image It is extracted from a side image Is. Figure 9A is an example of a side image Is acquired in S230, for example, a side image Is of the nozzle 51 immediately before adsorption or immediately before mounting. Figure 9B is an example of a target image It acquired in S240. The side image Is in Figure 9A is an image of the nozzle 51 where stacking has occurred. Because of the stacking, the tip of the nozzle 51 is located above the ideal position (lower end line BL). Therefore, the upper part of the component P is also located above the lower end line BL. Consequently, the target image It, which is extracted from the range above the lower end line BL (a predetermined range), includes not only the nozzle 51 but also the upper part of the component P. By applying this target image It to the judgment model 93a, the presence or absence of stacking can be determined with high accuracy. Although not shown in the illustration, contamination of the nozzle 51 can also be determined with high accuracy.
[0034] Figure 10 is an explanatory diagram showing an example of a side view image Is of a comparative example. In the comparative example, unlike in this embodiment, the stacking of the nozzle 51 is determined without extracting the target image It from the side view image Is. For example, the ideal position of the lower end of the component P adsorbed by the nozzle 51 is determined based on the type (size) of the nozzle 51 and the type (size) of the component P, and the stacking of the nozzle 51 is determined using the lower end line BL1 (reference) as the ideal position. In the side view image Is of Figure 10A, the component P is located above the lower end line BL1, so it is possible to determine the stacking of the nozzle 51. On the other hand, in the side view image Is of Figure 10B, the same stacking as in Figure 10A is also occurring on the nozzle 51. However, in the side view image Is of Figure 10B, the component P is adsorbed at an angle, so the position of the lower end of the component P is on the lower end line BL1. For this reason, it is not possible to determine the stacking of the nozzle 51 in the side view image Is of Figure 10B, and it is incorrectly determined that no stacking has occurred. The CPU 91 in this embodiment determines stacking based on the ideal position of the tip of the nozzle 51, thus preventing such misjudgments. Furthermore, since the CPU 91 in this embodiment extracts the target image It from the side image Is and makes a judgment, it can eliminate the influence of component P and accurately determine abnormalities in the nozzle 51.
[0035] Here, the correspondence between the components of this embodiment and the components of the present disclosure will be clarified. In this embodiment, the nozzle 51 corresponds to an example of an adsorption member, the management device 100 (CPU 101) that executes the judgment model creation process S100 and S110 and the control device 90 (CPU 91) that executes the nozzle abnormality determination process S230 and S240 correspond to an example of an acquisition unit, the management device 100 (CPU 101) that executes the judgment model creation process S120 and S130 corresponds to an example of a learning unit, and the control device 90 (CPU 91) that executes the nozzle abnormality determination process S250 and S260 corresponds to an example of a determination unit. That is, the control device 90 and the management device 100 correspond to an example of an image processing system of the present disclosure. The side camera 70 corresponds to an example of an imaging unit, and the HDD 93 corresponds to an example of a storage unit. In this embodiment, an example of an image processing method is also clarified by describing the operation of the implementation system 1 including the image processing system.
[0036] The image processing system of the embodiment described above processes a side image Is of the nozzle 51 to be judged and applies it to a judgment model 93a to determine abnormalities such as nozzle stacking and side contamination. Furthermore, by learning the side image Is, a judgment model 93a capable of determining abnormalities of the nozzle 51 with high accuracy is created, thus enabling more accurate determination of nozzle stacking and side contamination abnormalities. In particular, the nozzle 51 is slidably mounted on the nozzle holder 52 and biased downward by an inner spring 53a. Due to aging deterioration of the inner spring 53a or contamination of foreign matter, the nozzle is prone to being biased incorrectly and stacking is likely to occur. Therefore, applying this disclosure to accurately determine abnormalities is highly significant. In addition, this image processing system contributes to the realization of smart factories.
[0037] Furthermore, since the CPU 101 learns side image Is of the nozzle 51, which is known to have stack or side contamination abnormalities, the learning accuracy can be increased, and a highly accurate judgment model 93a can be created. As a result, abnormality can be judged with even greater accuracy.
[0038] Furthermore, the CPU 101 learns a predetermined range in the side view image Is, with the ideal position of the tip of the nozzle 51 as the lower end. The CPU 91 processes the predetermined range in the side view image Is. This eliminates the influence of the component P adsorbed to the nozzle 51, allowing for more accurate detection of abnormalities in the nozzle 51.
[0039] Furthermore, the CPU 101 removes the area below the ideal position from the side image Is of the nozzle 51 and obtains a target image It from which a predetermined area has been extracted. This allows the processing to be performed quickly by narrowing the range of the processing target.
[0040] Furthermore, the CPU 91 acquires a side view image Is of the nozzle 51 to be judged at the timing immediately before the nozzle 51 picks up the component P, the timing immediately before the nozzle 51 mounts the component P, and the timing immediately after the nozzle 51 mounts the component P, and determines whether there is an abnormality in the nozzle 51. By determining the abnormality of the nozzle 51 at the timing immediately before the component P is picked up, poor picking of the component P can be suppressed. Also, by determining the abnormality of the nozzle 51 at the timing immediately before the component P is mounted, poor mounting of the component P can be suppressed. Moreover, by determining the abnormality of the nozzle 51 immediately after the component P is mounted, the abnormality can be detected and addressed early before the nozzle 51 moves on to the next picking operation of the component P.
[0041] Furthermore, the mounting device 10 executes an abnormality response process for any nozzle 51 that is determined to be abnormal, and prevents that nozzle 51 from being used. This suppresses problems such as poor suction and mounting defects of the component P, and allows the component P to be mounted properly.
[0042] It goes without saying that this disclosure is not limited in any way to the embodiments described above, and can be implemented in various forms as long as they fall within the technical scope of this disclosure.
[0043] In the embodiment, the side image Is of the nozzle 51 to be determined was acquired at the timing immediately before the adsorption of the component P, the timing immediately before the mounting of the component P, and the timing immediately after the mounting of the component P, and the abnormality of the nozzle 51 was determined. However, the present invention is not limited to this. That is, the determination of the abnormality of the nozzle 51 is not limited to being performed at these three timings, and the abnormality of the nozzle 51 may be determined at least at the timing immediately before the mounting of the component P. Alternatively, among these three timings, the abnormality of the nozzle 51 may be determined at any one or two of the timings. Further, the timing immediately before the mounting of the component P may be any timing from immediately after the adsorption of the component P by the nozzle 51 to immediately before the mounting. Therefore, for example, at the timing when the nozzle 51 that has adsorbed the component P has moved to the immediately subsequent position A2, the side image Is of the nozzle 51 may be acquired to determine the abnormality.
[0044] In the embodiment, the target image It was acquired by cutting out a predetermined range from the side image Is of the nozzle 51. However, the present invention is not limited to this. Image processing may be performed on a predetermined range within the side image Is without cutting out the predetermined range from the side image Is. In addition, the side image Is captured by the side camera 70 may be transmitted to the outside such as the management device 100 or the cloud server from the mounting device 10 to perform the nozzle abnormality determination process. In such a case, cutting out the target image It from the side image Is has an advantage because the image size can be reduced for transmission. When the nozzle abnormality determination process is performed outside the mounting device 10, it is not necessary to store the determination model 93a in the HDD 93 (storage unit) of the mounting device 10.
[0045] In the embodiment, a predetermined range having the ideal position of the tip of the nozzle 51 in the side image Is as the lower end was learned, and the predetermined range was subjected to image processing. However, the present invention is not limited to this. For example, a range below the ideal position of the tip may be included in the predetermined range. Alternatively, the entire range of the side image Is may be the target of image processing. However, in order to perform the processing promptly, it is preferable to adopt the method of the embodiment.
[0046] In the embodiment, the side image Is of the nozzle 51, for which it is known whether there is an abnormality in the stain on the stack or the side surface, is learned, but the present invention is not limited to this. For example, learning may be performed including a side image Is for which it is unknown whether there is an abnormality.
[0047] In the embodiment, the abnormality of the stain on the stack and the side surface of the nozzle 51 to be determined is determined, but the present invention is not limited to this, and it may be determined whether there is an abnormality in either the stain on the stack or the side surface of the nozzle 51 to be determined.
[0048] In the embodiment, the rotary head 54 is exemplified as the head unit 50, but the present invention is not limited to this. For example, a plurality (for example, two) of nozzles 51 may be arranged in parallel, or only one nozzle 51 may be arranged.
[0049] In this embodiment, a model-based determination of nozzle 51 abnormalities was performed using a determination model 93a created through learning. However, rule-based determination can also be performed as follows. For example, CPU 91 and CPU 101 acquire multiple side images Is of nozzles 51 that do not have stacking or side contamination abnormalities, and generate an average image for each type of nozzle 51 by deriving the average brightness value of each pixel in the side image Is, and store it in HDD 93 or HDD 103. Next, CPU 91 and CPU 101 acquire a side image Is of the nozzle 51 to be determined during the implementation process and calculate the similarity to the average image corresponding to the type of nozzle 51. The similarity can be calculated using well-known methods such as normalized cross-correlation or SSD (Sum of Squared Difference). Then, CPU 91 and CPU 101 can detect stacking or contamination abnormalities if the similarity is lower than a predetermined threshold. The predetermined threshold can be arbitrarily set by a user such as an operator or designer, or a fixed value determined in advance through experiments may be set. The mounting device 10 may selectively switch between such rule-based determination and the model-based determination of the embodiment to perform nozzle abnormality determination processing. For example, the mounting device 10 may switch the determination processing to be performed depending on whether it is mounting a component P with normal mounting accuracy or a component P that requires relatively high mounting accuracy. Alternatively, it may perform rule-based nozzle abnormality determination processing in place of, or in addition to, the model-based nozzle abnormality determination processing of the embodiment.
[0050] Herein, the mounting apparatus of the present disclosure is a mounting apparatus that detachably holds a suction member for adsorbing components from a supply unit and mounting them onto a substrate, wherein the suction member is slidable in the vertical direction and biased downward by a biasing member, and slides upward against the biasing force when pressed against the supply unit to adsorb components and when pressed against the substrate to mount components, and comprises an imaging unit that captures a side image of the suction member held by the mounting apparatus, a storage unit that stores a determination model created by learning side images of a plurality of the suction members for determining abnormalities in the suction member, and a determination unit that determines a stuck state in which the suction member to be determined does not return from a state in which it has slid upward and / or an abnormality of contamination on the side of the suction member by image processing the side image of the suction member to be determined captured by the imaging unit and applying it to the determination model.
[0051] This mounting device, similar to the image processing system described above, can more accurately determine the stacking state of the adsorption material and any abnormalities in the contamination of its sides using side images of the adsorption material. Furthermore, this mounting device contributes to the realization of smart factories.
[0052] Furthermore, the abnormality determination method of this disclosure is an abnormality determination method for determining an abnormality of a suction member in a mounting device that detachably holds a suction member for adsorbing components from a supply unit and mounting them on a substrate, wherein the suction member is slidable in the vertical direction and biased downward by a biasing member, and slides upward against the biasing force when pressed against the supply unit to adsorb components and when pressed against the substrate to mount components, and the method includes: (a) acquiring a side image of the suction member held by the mounting device; (b) learning the side images of a plurality of suction members acquired in step (a) to create a determination model for determining an abnormality of the suction member; and (c) image processing the side image of the suction member to be determined acquired in step (a) and applying it to the determination model to determine a stuck state in which the suction member to be determined does not return from a state in which it has slid upward and / or an abnormality of dirt on the side of the suction member.
[0053] The anomaly detection method of this disclosure, similar to the image processing system described above, can more accurately determine the stacking state of the adsorption member and any abnormalities in the contamination of its sides using a side image of the adsorption member. Furthermore, this anomaly detection method contributes to the realization of smart factories. In this anomaly detection method, a step to implement any of the functions of the image processing system described above may be added.
[0054] This specification also discloses a technical concept in which the "image processing system described in claim 1 or 2" in the original claim 5 has been changed to "image processing system described in any one of claims 1 to 4".
[0055] This disclosure is applicable to the technical field for determining abnormalities in suction members that hold components.
[0056] 1 Mounting system, 10 Mounting device, 20 Parts supply device, 22 Reel, 30 Board transport device, 32 Conveyor belt, 40 Moving device, 42, 46 Guide rail, 44 X-axis slider, 45 X-axis actuator, 48 Y-axis slider, 49 Y-axis actuator, 50 Head unit, 51 Nozzle, 51a Tip (nozzle tip), 51b Nozzle pipe, 51c Sleeve, 52 Nozzle holder, 52a Upper end, 52b Flange, 53a Inner spring, 53b Inner pipe, 54 Rotary head, 54a Reflector, 55 Spring, 56 R-axis actuator, 57 Rotating shaft, 58 Drive motor, 59 Q-axis actuator, 60 Z-axis actuator, 62 Ball screw nut, 64 Screw shaft, 66 Z-axis slider, 67 Lever part, 68 Drive motor, 70 Side camera, 72 Camera body, 74 Optical unit, 76a Middle entrance port, 76b Left entrance port, 76c Right entrance port, 76d Exit port, 77 Light emitter, 80 Parts camera, 82 Mark camera, 84 Nozzle stocker, 90 Control device, 91, 101 CPU, 92, 102 ROM, 93, 103 HDD, 93a Detection line information, 94, 104 RAM, 95, 105 Input / output interface, 96, 106 Bus, 100 Management device, 107 Input device, 108 Display, Is Side image, It Target image, P Part, S Circuit board.
Claims
1. An image processing system for processing an image of a suction member in a mounting device that detachably holds a suction member for picking up components from a supply unit and mounting them onto a substrate, wherein the suction member is slidable in the vertical direction and biased downward by a biasing member, and slides upward against the biasing force when pressed against the supply unit to pick up components and when pressed against the substrate to mount components, the image processing system comprises: an acquisition unit that acquires a side image of the suction member held by the mounting device; a learning unit that learns the side images of a plurality of suction members acquired by the acquisition unit and creates a determination model for determining abnormalities of the suction member; and a determination unit that processes the side image of the suction member to be determined acquired by the acquisition unit and applies it to the determination model to determine a stuck state in which the suction member to be determined does not return from a state in which it has slid upward and / or an abnormality of dirt on the side of the suction member.
2. The image processing system according to claim 1, wherein the learning unit learns side images of the adsorption member, in which it is known whether or not it is in the abnormal state.
3. The image processing system according to claim 1 or 2, wherein the learning unit learns a predetermined range in the side image with the ideal position of the tip of the adsorption member as the lower end, and the determination unit processes the predetermined range in the side image.
4. The image processing system according to claim 3, wherein the acquisition unit acquires an image of a predetermined range by removing the range below the ideal position from an image of the side view of the adsorption member.
5. The image processing system according to claim 1 or 2, wherein the acquisition unit acquires a side view image of the suction member to be determined at at least the time between immediately after the suction member has picked up the part and immediately before mounting, among the timings of immediately after the suction member has picked up the part and immediately before mounting.
6. A mounting apparatus for detachably holding a suction member for picking up components from a supply unit and mounting them onto a substrate, wherein the suction member is slidable in the vertical direction and biased downward by a biasing member, and slides upward against the biasing force when pressed against the supply unit to pick up components and when pressed against the substrate to mount components, the mounting apparatus comprising: an imaging unit for capturing a side image of the suction member held by the mounting apparatus; a storage unit for storing a determination model created by learning side images of a plurality of the suction members for determining abnormalities in the suction member; and a determination unit for determining an abnormality in the suction member by image processing the side image of the suction member to be determined captured by the imaging unit and applying it to the determination model, thereby determining a stuck state in which the suction member to be determined does not return from a state in which it has slid upward and / or an abnormality in the side surface of the suction member.
7. An abnormality determination method for determining an abnormality in a mounting device that detachably holds a suction member for picking up components from a supply unit and mounting them on a substrate, wherein the suction member is slidable in the vertical direction and biased downward by a biasing member, and slides upward against the biasing force when pressed against the supply unit to pick up components and when pressed against the substrate to mount components, the abnormality determination method comprising: (a) acquiring a side image of the suction member held by the mounting device; (b) learning the side images of a plurality of suction members acquired in step (a) to create a determination model for determining an abnormality in the suction member; and (c) image processing the side image of the suction member to be determined acquired in step (a) and applying it to the determination model to determine an abnormality of a stuck state in which the suction member to be determined does not return from a state in which it has slid upward and / or an abnormality of dirt on the side of the suction member.