Component mounting system and component mounting apparatus

Abstract

To recognize the surface state of a substrate in a short time.SOLUTION: The component mounting system includes head 32 which mounts component E on board B, transport unit 13 which transports board B toward a component mounting position which is a position where head 32 mounts component E on board B, TOF camera 40 which acquires a three dimensional image of the surface of board B indicating a height of the surface of board B by irradiating board B transported to the component mounting position with light L and receiving reflected light L, and an information processing unit which recognizes a surface state of board B transported to the component mounting position based on the three dimensional image acquired by TOF camera 40 and determines a mounting operation of component E on board B by head 32 according to the recognition result.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a component mounting system and a component mounting apparatus. [Background technology]

[0002] A component mounting device described in JP 2021-180287 A (Patent Document 1 below) is known in the prior art. This component mounting device includes a head unit that is movable horizontally. The head unit is provided with a moving head for suction-holding components, and a height sensor. A laser displacement sensor is exemplified as the height sensor. The height sensor measures the board height at multiple measurement points while the board is positioned and held at the work position. Board warpage data is generated based on the board height measurement results obtained by the height sensor. Components are then mounted on the board based on this board warpage data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-180287 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above configuration, in order to measure the board height at multiple measurement points, it is necessary to move the height sensor to each measurement point, which may result in a long time being required to generate board warpage data. [Means for solving the problem]

[0005] The component mounting system of the present disclosure includes a head that mounts components on a board, a transport unit that transports the board toward a component mounting position where the head mounts the components on the board, a TOF camera that irradiates light onto the board transported to the component mounting position and receives the reflected light to obtain a three-dimensional image of the surface of the board that indicates the height of the surface of the board, and an information processing unit that recognizes the surface condition of the board transported to the component mounting position based on the three-dimensional image obtained by the TOF camera, and determines the mounting operation of the components on the board by the head based on the recognition result. [Effects of the Invention]

[0006] According to the present disclosure, the surface condition of a substrate can be recognized in a short time. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic plan view of a component mounting apparatus according to a first embodiment. [Figure 2] FIG. 2 is a schematic side view of the component mounting apparatus. [Figure 3] FIG. 3 is a block diagram showing the electrical configuration of the component mounting apparatus. [Figure 4] FIG. 4 shows a three-dimensional image of the substrate and a cross-sectional view of the substrate. [Figure 5] FIG. 5 is a diagram showing the height of the surface of the substrate in the three-dimensional image of FIG. 4 in numerical terms. [Figure 6] FIG. 6 is a plan view of the substrate on which the solder paste has been applied. [Figure 7] FIG. 7 is a three-dimensional image of the substrate shown in FIG. [Figure 8] FIG. 8 is a plan view of a substrate including a foreign particle. [Figure 9] FIG. 9 is a three-dimensional image of the substrate shown in FIG. [Figure 10] FIG. 10 is a difference image obtained by subtracting FIG. 7 from FIG. [Figure 11]FIG. 11 is a plan view of a board on which components are mounted. [Figure 12] FIG. 12 is a three-dimensional image of the substrate shown in FIG. [Figure 13] FIG. 13 is a plan view of the board including the unmounted portion. [Figure 14] FIG. 14 is a three-dimensional image of the substrate shown in FIG. [Figure 15] FIG. 15 is a difference image obtained by subtracting FIG. 14 from FIG. [Figure 16] FIG. 16 is a flowchart showing the operation of recognizing the surface condition of the board and mounting components according to the first embodiment. [Figure 17] FIG. 17 is a flowchart showing the recognition of the surface state of the substrate after component mounting according to the first embodiment. [Figure 18] FIG. 18 is a block diagram showing the electrical configuration of the component mounting system according to the second embodiment. [Figure 19] FIG. 19 is a block diagram of a production management device that uses machine learning. [Figure 20] FIG. 20 is a diagram showing a learning model. DETAILED DESCRIPTION OF THE INVENTION

[0008] [Description of the embodiments of the present disclosure] First, embodiments of the present disclosure will be listed and described. [1] The component mounting system of the present disclosure includes a head that mounts components on a board, a transport unit that transports the board toward a component mounting position where the head mounts the components on the board, a TOF camera that irradiates light onto the board transported to the component mounting position and receives the reflected light to obtain a three-dimensional image of the surface of the board that indicates the height of the surface of the board, and an information processing unit that recognizes the surface condition of the board transported to the component mounting position based on the three-dimensional image obtained by the TOF camera and determines the mounting operation of the components on the board by the head based on the recognition result.

[0009] With this configuration, the surface condition of the substrate can be recognized from a three-dimensional image of the substrate surface acquired by the TOF camera. Because image data can be acquired within the field of view of the TOF camera, the surface condition of the substrate can be recognized in a shorter time than when, for example, a laser displacement meter is used to measure the height of the substrate surface at multiple points and the surface condition of the substrate is estimated based on the measurement results.

[0010] [2] In the above [1], it is preferable that the information processing unit recognizes the surface condition of the substrate, including the degree of warping of the surface, based on the three-dimensional image acquired by the TOF camera, and determines the mounting operation of the component onto the substrate by the head according to the degree of warping of the surface of the substrate.

[0011] [3] It is preferable that the component mounting system of [1] or [2] above further comprises a housing that covers the component mounting position, the head, and the TOF camera, and that the TOF camera is attached to the housing so as to be fixedly positioned above the component mounting position.

[0012] With this configuration, there is no need to move the TOF camera when capturing a three-dimensional image of the board transported to the component mounting position.

[0013] [4] It is preferable that the component mounting system of any one of [1] to [3] above further comprises a moving body to which the head is attached, and a moving body drive unit that moves the head by operating the moving body, and that the information processing unit controls the moving body drive unit so that, when the TOF camera captures an image of the board transported to the component mounting position, the moving body is positioned so that it does not overlap the component mounting position when viewed from above.

[0014] With this configuration, the imaging field of view of the TOF camera can be prevented from being blocked by a moving object.

[0015] [5] In any one of [1] to [4] above, it is preferable that the information processing unit measures the surface condition of the substrate, including the presence or absence of foreign matter, based on the three-dimensional image acquired by the TOF camera, and determines the mounting operation of the component on the substrate by the head according to the measurement results.

[0016] [6] In any one of [1] to [5] above, it is preferable that the information processing unit determines whether or not the component is mounted on the board based on the three-dimensional image acquired by the TOF camera, and determines the mounting operation of the component on the board by the head according to the determination result.

[0017] [7] It is preferable that any one of the component mounting systems [1] to [6] above further has a learning model generated by machine learning using the three-dimensional image acquired by the TOF camera and the surface condition of the substrate captured in the three-dimensional image as a data set, and that the information processing unit estimates and recognizes the surface condition of the substrate by inputting image information of the three-dimensional image acquired by the TOF camera into the learning model.

[0018] With this configuration, it is possible to estimate the surface condition of the substrate through machine learning and detect abnormalities in the surface condition of the substrate without having to set a threshold value for the height of the substrate or the like in advance.

[0019] [8] It is preferable that the component mounting system according to [7] above further comprises a learning unit that updates the parameters of the learning model.

[0020] [9] The component mounting device of the present disclosure includes a head that mounts components on a board, a transport unit that transports the board toward a component mounting position where the head mounts the components on the board, a TOF camera that irradiates light onto the board transported to the component mounting position and receives the reflected light to obtain a three-dimensional image of the surface of the board indicating the height of the surface of the board, and an information processing unit that recognizes the surface condition of the board transported to the component mounting position based on the three-dimensional image obtained by the TOF camera and determines the mounting operation of the components on the board by the head based on the recognition result.

[0021] [Details of the embodiments of the present disclosure] Embodiments of the present disclosure will be described below. The present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims. In each drawing, for the convenience of explanation, some components may be exaggerated or simplified. Furthermore, the dimensional proportions of each part may differ from one drawing to another.

[0022] <Embodiment 1> A first embodiment of the present disclosure will be described with reference to Fig. 1 to Fig. 17. As shown in Fig. 1, the first embodiment illustrates a component mounting apparatus 10 that mounts a component E on a board B such as a printed circuit board. The component mounting apparatus 10 is also an example of a component mounting system.

[0023] (Overall configuration of component mounting device) The component mounting apparatus 10 includes a base 11, component supply units 12A to 12D, a transport unit 13, a backup device 15 (see FIG. 2), a head unit 30, a drive unit 20, a TOF camera 40 (see FIG. 2), etc. In the following description, the longitudinal direction of the base 11 (the left-right direction in FIG. 1) will be referred to as the X-axis direction, the depth direction of the base 11 (the up-down direction in FIG. 1) will be referred to as the Y-axis direction, and the up-down direction in FIG. 2 (the direction in which the head 32 moves up and down) will be referred to as the Z-axis direction.

[0024] The transport unit 13 is disposed in the center of the base 11. The transport unit 13 is equipped with a pair of transport belts 14 that are driven to rotate in the left-right direction, and transports the board B on the transport belts 14 to the right by friction with the transport belts 14. In this embodiment, the board B is carried into the component mounting apparatus 10 from the left side through the transport unit 13. The carried-in board B is transported by the transport unit 13 to the component mounting position (position indicated by the two-dot chain line) in the center of the base 11, where it is stopped. Here, the component mounting position is the position of the board B when the head 32 mounts components E on the board B. The board B on which components E have been mounted is transported to the right by the transport unit 13 from the component mounting position, and is then transported out of the component mounting apparatus 10.

[0025] 2, backup device 15 fixes board B, which has been transported to the component mounting position by conveyor belt 14, at the component mounting position. Backup device 15 includes a clamp unit 15A that positions board B in the Y-axis direction, a pressing unit 15B that presses down the end of board B from above, multiple backup pins 15C that support board B from below, and an elevating unit 15D that raises and lowers backup pins 15C.

[0026] 1, four component supply units 12A to 12D are provided on base 11 so as to surround the periphery of the component mounting position. Component supply units 12A to 12D supply components E to be mounted on board B by head 32. Component supply units 12A to 12D are provided with a number of tape feeders 16 adjacent to each other in the left-right direction. Component supply units 12A to 12D may also be provided with tray feeders.

[0027] The driving device 20 transports the head unit 30 in the X-axis and Y-axis directions within a predetermined movable range. The driving device 20 includes an X-axis beam 21 (an example of a moving body), a Y-axis frame 22 (an example of a moving body driving section), an X-axis servo motor 23 (an example of a moving body driving section), and a Y-axis servo motor 24 (an example of a moving body driving section). Here, the head unit 30 is supported by the X-axis beam 21 and can be moved back and forth in the X-axis direction by the X-axis servo motor 23. The X-axis beam 21 is supported by the Y-axis frame 22 and can be moved back and forth in the Y-axis direction by the Y-axis servo motor 24.

[0028] The head unit 30 has a box-shaped head unit main body 31, a plurality of heads 32 (five in this embodiment), and a board imaging camera 17.

[0029] The board imaging camera 17 is disposed on the head unit 30 with its imaging surface facing downward. The board imaging camera 17 is configured to capture an image of a fiducial mark (not shown) on the board B in order to recognize the position and orientation of the board B.

[0030] As shown in Fig. 2, the multiple heads 32 protrude downward from the head unit main body 31. A suction nozzle 33 is provided at the bottom end of each head 32. An air path is provided in each head 32, and negative or positive pressure is supplied to each air path by an air supply device. Supplying negative pressure to the air path of the head 32 enables the suction nozzle 33 to pick up the component E. Supplying positive pressure to the air path of the head 32 allows the component E picked up and held by the suction nozzle 33 to be released.

[0031] Each head 32 is attached to a Z-axis servo motor 34 (see FIG. 3) provided inside the head unit main body 31. The head 32 can be raised and lowered in the vertical direction by the Z-axis servo motor 34. Also provided inside the head unit main body 31 is an R-axis servo motor 35 (see FIG. 3) that rotates each head 32 around an axis extending in the vertical direction.

[0032] 1, a component imaging camera 18 is disposed with its imaging surface facing upward on the base 11. The component imaging camera 18 captures an image (bottom surface image) of the component E held by the head 32.

[0033] (TOF camera) 2, the TOF camera 40 is disposed (immovably fixed) above the component mounting position with its imaging surface facing downward. The TOF camera 40 includes a light projecting unit 41 and a light receiving unit 42. The light projecting unit 41 irradiates light L toward the board B disposed at the component mounting position, and the light receiving unit 42 receives the reflected light L. The light L is, for example, infrared light.

[0034] The TOF camera 40 is a camera that measures the distance to an object based on the time of flight of light L, i.e., the time from when the light L is emitted until when the reflected light L is received. The TOF camera 40 acquires a three-dimensional image of the surface of the board B placed at the component mounting position. The field of view of the TOF camera 40 is wider than the component mounting position. Therefore, a single image taken by the TOF camera 40 can acquire a three-dimensional image of the entire board B at the component mounting position.

[0035] The three-dimensional image of the substrate B acquired by the TOF camera 40 is an image (depth image) that visualizes the distribution of the surface height of the substrate B, and includes information about the surface height of the substrate B.

[0036] The component mounting apparatus 10 includes a housing 50 that covers the component mounting position, the head unit 30, and the TOF camera 40. In this embodiment, the TOF camera 40 is fixed to the housing 50 so as to be disposed above the component mounting position. The TOF camera 40 may be fixed directly to the ceiling portion of the housing 50. Here, the ceiling portion refers to the portion of the housing 50 that is disposed above the base 11. The TOF camera 40 may also be indirectly attached to a side wall or the like of the housing 50 via a frame or the like.

[0037] (Electrical configuration of component mounting device) 3, the component mounting apparatus 10 has a controller 60 (an example of an information processing unit). The controller 60 includes a CPU 61, which is a calculation unit, and a memory 62.

[0038] The controller 60 is connected via a bus BS to the transport unit 13, backup device 15, motor control unit 63, tape feeder 16, board imaging camera 17, component imaging camera 18, TOF camera 40, display unit 64, operation unit 65, etc. The motor control unit 63 is connected to the X-axis servo motor 23, Y-axis servo motor 24, Z-axis servo motor 34, and R-axis servo motor 35.

[0039] The controller 60 is a main control unit of the component mounting apparatus 10, and typically controls the transport and fixation of the board B and the mounting of the components E by the head 32. The controller 60 also recognizes the surface condition of the board B based on a three-dimensional image of the surface of the board B acquired by the TOF camera 40. Then, the controller 60 determines the mounting operation of the components E on the board B by the head 32 according to the recognition result.

[0040] (3D image) Hereinafter, the correspondence between various surface states of the substrate B and the three-dimensional images of the substrate B captured by the TOF camera 40 will be described with reference to the drawings. The lower part of Figure 4 is a cross-sectional view of substrate B in a warped state. The upper part of Figure 4 is a three-dimensional image obtained by capturing substrate B shown in the lower part of Figure 4 using TOF camera 40. Substrate B has a rectangular shape with its long side in the X-axis direction and its short side in the Y-axis direction. As shown in the cross-sectional view, substrate B is warped in an upwardly convex shape. Specifically, the central portion of the surface of substrate B is positioned upward, and both left and right ends of the surface of substrate B are positioned downward. Here, the difference between the height of the highest point on the surface of substrate B and the reference height is defined as the warpage amount ΔZ of substrate B. The reference height is a predetermined height positioned below substrate B, and may be, for example, the height of the upper surface of conveyor belt 14.

[0041] The three-dimensional image G1 is an image that represents the distribution of the height of the surface of substrate B, measured with respect to a reference height, by changing the display color (gradation). That is, the display color of each pixel in the three-dimensional image G1 indicates the height of the portion of the surface of substrate B that corresponds to that pixel. In this figure, the display color becomes darker as the height of the surface of substrate B increases. As shown in the cross-sectional view, the surface of substrate B is highest near the center and decreases with increasing distance from the center of the surface of substrate B. Therefore, in the three-dimensional image G1, the display color becomes lighter with increasing distance from the center of the surface of substrate B.

[0042] As shown in FIG. 5, the three-dimensional image G2 may be an image that numerically represents the height distribution of the surface of the substrate B.

[0043] The controller 60 acquires the amount of warpage ΔZ of the surface of the substrate B from the three-dimensional image, and if the amount of warpage ΔZ is equal to or greater than a predetermined threshold, stops automatic operation and prevents the mounting of the component E. If the amount of warpage ΔZ is smaller than the predetermined threshold, the component E is mounted.

[0044] Typically, before the substrate B is loaded into the component mounting apparatus 10, solder paste SP is applied to predetermined locations on the substrate B using a screen printing device or the like. In the component mounting apparatus 10, the component E is attached to the substrate B so that the electrodes and fixing portions of the component E come into contact with the areas where the solder paste SP has been applied. FIG. 6 is a plan view of the substrate B on which the solder paste SP has been applied. FIG. 7 is a three-dimensional image G3 obtained by capturing the substrate B shown in FIG. 6 using the TOF camera 40. In FIG. 7, the areas where the solder paste SP has been applied are higher than the surface of the substrate B, and are therefore displayed darker than the surface of the substrate B. For simplicity, in FIGS. 6 and 7, and in FIGS. 8 to 15 described below, it is assumed that the substrate B is not warped.

[0045] FIG. 8 is a plan view of a substrate B coated with solder paste SP, and further including a foreign substance C1. The foreign substance C1 is located in the upper left portion of the surface of the substrate B. Examples of the foreign substance C1 include dirt and dust. The foreign substance C1 may adhere to the substrate B before it is transported into the component mounting apparatus 10. FIG. 9 is a three-dimensional image G4 obtained by capturing an image of the substrate B shown in FIG. 8 using the TOF camera 40. The foreign substance C1 is located higher than the surface of the substrate B and the solder paste SP, and is displayed dark in FIG. 9. Therefore, the foreign substance C1 can be found from the three-dimensional image G4.

[0046] Furthermore, if the foreign substance C1 is not particularly bulky, it may be displayed in the 3D image G4 in a color tone similar to that of the solder paste SP. Even in such cases, the foreign substance C1 may be more easily recognized by a differential image (an example of a 3D image). For example, FIG. 10 shows a differential image G5 obtained by subtracting the 3D image G3 of FIG. 7, which does not include the foreign substance C1, from the 3D image G4 of FIG. 9, which includes the foreign substance C1. Specifically, this differential image G5 is created by subtracting the height (value) of each pixel in FIG. 7 from the height (value) of each pixel in FIG. 9. In FIG. 10, the solder paste SP is offset and not displayed, so only the foreign substance C1 is displayed darkly. This makes it easier to recognize the foreign substance C1.

[0047] FIG. 11 is a plan view of the board B on which the component E has been mounted by the head 32. The component E is placed on the board B so that the electrodes and the like come into contact with the positions of the solder paste SP (see FIG. 6). In FIG. 11, the solder paste SP is placed in a position that overlaps the component E when viewed from above. Therefore, in the plan view, the solder paste SP is hidden by the component E and cannot be seen. FIG. 12 is a 3D image G6 obtained by capturing the board B shown in FIG. 11 with the TOF camera 40. The component E is placed in a position higher than the surface of the board B, and is displayed darker than the board B. Furthermore, the component E is placed in a position higher than the solder paste SP, and is displayed darker than the solder paste SP (see FIG. 7).

[0048] FIG. 13 is a plan view of board B including an unmounted portion where component E has not been mounted. Specifically, in FIG. 13, component E, which should be arranged in the second column from the left and the second row from the top, has not been mounted on board B. In the unmounted portion, solder paste SP is exposed upward. FIG. 14 is a 3D image G7 obtained by capturing an image of board B shown in FIG. 13 using TOF camera 40. The solder paste SP in the unmounted portion is lower than component E and higher than the surface of board B. Therefore, the solder paste SP in the unmounted portion is displayed lighter than component E and darker than the surface of board B. Therefore, the difference in display color makes it possible to recognize that there is an unmounted portion on board B.

[0049] Fig. 15 is a difference image G8 obtained by subtracting the three-dimensional image G7 including the unmounted portion in Fig. 14 from the three-dimensional image G6 not including the unmounted portion in Fig. 12. More specifically, this difference image G8 is created by subtracting the height (numerical value) of each pixel in Fig. 14 from the height (numerical value) of each pixel in Fig. 12. In Fig. 15, component E is offset and not displayed, so only the unmounted portion is displayed dark. This makes it easy to recognize the unmounted portion.

[0050] (Controller control flow) Next, with reference to FIG. 16, a flow of mounting a component E onto a board B, including recognition of the surface state of the board B using a three-dimensional image, will be described. First, the controller 60 controls the transport unit 13 to transport the board B into the component mounting apparatus 10 (S1). The board B is transported to the component mounting position. The controller 60 controls the backup device 15 to fix the board B at the component mounting position (S2).

[0051] Next, the controller 60 determines whether the movable body (X-axis beam 21) and head unit 30 overlap the component mounting position when viewed from above (S3). If the movable body and head unit 30 overlap the component mounting position (S3: YES), the controller 60 controls the movable body drive units (X-axis servo motor 23, Y-axis servo motor 24, etc.) to move the movable body and head unit 30 to positions where they do not overlap the component mounting position when viewed from above (S4).

[0052] When S4 is completed, or when the moving body is not overlapping with the component mounting position (S3: NO), the moving body and head unit 30 are out of the field of view of the TOF camera 40. Then, the controller 60 captures a 3D image of the surface of the substrate B using the TOF camera 40 (S5). The 3D image of the surface of the substrate B includes not only height information of the surface of the substrate B, but also height information of the solder paste SP applied to the surface of the substrate B and foreign matter C1 adhering to the surface of the substrate B.

[0053] The controller 60 recognizes the surface condition of the substrate B transported to the component mounting position based on the three-dimensional image acquired by the TOF camera 40 (S6). Specifically, the controller 60 estimates the surface condition of the substrate B from height information of the surface of the substrate B for each pixel of the three-dimensional image, etc.

[0054] Then, the controller 60 determines whether or not there is an abnormality in the surface state of the substrate B (S7). Here, the abnormality in the surface state of the substrate B specifically includes the following items. The warpage ΔZ of substrate B is equal to or greater than a predetermined threshold. · Foreign matter C1 is attached to the surface of substrate B (for example, Figure 9 or Figure 10). · Solder paste SP is not applied to the correct location on the surface of board B.

[0055] In recognizing the surface condition of substrate B (S6) and determining whether or not there is an abnormality (S7), for example, height information of the surface of normal substrate B stored in advance in memory 62 is compared with the three-dimensional image. At this time, a difference image as shown in Fig. 10 may be created.

[0056] The controller 60 determines the mounting operation of the component E on the board B by the head 32 according to the result of S7. Here, the determination of the mounting operation includes a determination of whether or not to perform mounting, and a change of the mounting conditions. If it is determined that there is no abnormality in the surface condition of the board B (S7: NO), the controller 60 determines that mounting of the component E is possible, and starts mounting of the component E (S8). At this time, the mounting conditions may be changed based on the recognition result of the surface condition of the board B. For example, if the warpage ΔZ of the board B is finite and less than a threshold value, the amount of descent of the head 32 may be reduced according to the warpage ΔZ of the board B.

[0057] If it is determined that there is an abnormality in the surface condition of substrate B (S7: YES), controller 60 determines to stop mounting of component E and does not start mounting (S9). Then, controller 60 may cause display unit 64 to display an error message indicating that the mounting operation has been stopped due to an abnormality in the surface condition of substrate B (S10). The error message may include the type of abnormality detected in the surface condition of substrate B (substrate warpage, foreign matter, etc.) and an image in which the location where the abnormality was detected is identified by a marker in the 3D image.

[0058] Next, with reference to FIG. 17, recognition of the surface condition of the board B after component mounting will be described. When the mounting of the component E on the board B by the head 32 is completed (S11), the controller 60 causes the TOF camera 40 to capture a three-dimensional image of the surface of the board B again (S12).

[0059] The controller 60 recognizes the surface condition of the board B after the components are mounted based on the three-dimensional image acquired by the TOF camera 40 (S13). Then, the controller 60 determines whether or not there is an unmounted portion on the board B (S14).

[0060] For example, when it is determined from the three-dimensional images shown in FIG. 14 or 15 that there is an unmounted portion on the board B (S14: YES), the controller 60 performs control to mount the component E on the unmounted portion (S15).

[0061] For example, if it is determined from a three-dimensional image such as that shown in FIG. 12 that there are no unmounted portions on the board B (S14: NO), or if a component E is mounted on the unmounted portion in S15, the controller 60 controls the transport unit 13 to transport the board B outside the component mounting device 10 (S16).

[0062] (Effects of the first embodiment) (1-1) The component mounting system (component mounting device 10) according to the first embodiment includes a head 32 that mounts a component E on a substrate B, a transport unit 13 that transports the substrate B toward a component mounting position where the head 32 mounts the component E on the substrate B, a TOF camera 40 that irradiates light L onto the substrate B transported to the component mounting position and receives the reflected light L to obtain a three-dimensional image of the surface of the substrate B that indicates the height of the surface of the substrate B, and an information processing unit (controller 60) that recognizes the surface condition of the substrate B transported to the component mounting position based on the three-dimensional image obtained by the TOF camera 40 and determines the mounting operation of the component E on the substrate B by the head 32 according to the recognition result.

[0063] With this configuration, the surface condition of substrate B can be recognized from a three-dimensional image of the surface of substrate B acquired by TOF camera 40. Because image data can be acquired within the field of view of TOF camera 40, the surface condition of substrate B can be recognized in a shorter time than when, for example, a laser displacement meter is used to measure the height of the surface of substrate B at multiple points and the surface condition of substrate B is estimated based on the measurement results.

[0064] (1-2) In embodiment 1, the information processing unit recognizes the surface condition of the substrate B, including the degree of warping, based on the three-dimensional image acquired by the TOF camera 40, and determines the mounting operation of the component E on the substrate B by the head 32 according to the degree of warping of the surface of the substrate B.

[0065] (1-3) The component mounting system of embodiment 1 further includes a housing 50 that covers the component mounting position, the head 32, and the TOF camera 40, and the TOF camera 40 is attached to the housing 50 so as to be fixedly positioned above the component mounting position.

[0066] With this configuration, there is no need to move the TOF camera 40 when capturing a three-dimensional image of the board B that has been transported to the component mounting position.

[0067] (1-4) The component mounting system of embodiment 1 further includes a moving body (X-axis beam 21) to which a head 32 is attached, and a moving body drive unit (Y-axis frame 22, X-axis servo motor 23, Y-axis servo motor 24) that moves the head 32 by operating the moving body, and when the TOF camera 40 captures an image of the board B transported to the component mounting position, the information processing unit controls the moving body drive unit so that the moving body is positioned so as not to overlap the component mounting position when viewed from above.

[0068] With this configuration, the imaging field of view of the TOF camera 40 can be prevented from being blocked by a moving object.

[0069] (1-5) In embodiment 1, the information processing unit measures the surface condition, including the presence or absence of foreign matter C1 on the substrate B, based on the three-dimensional image acquired by the TOF camera 40, and determines the mounting operation of component E on the substrate B by the head 32 based on the measurement results.

[0070] (1-6) In embodiment 1, the information processing unit determines whether or not component E is mounted on board B based on the three-dimensional image acquired by the TOF camera 40, and determines the mounting operation of component E on board B by head 32 based on the determination result.

[0071] <Embodiment 2> A second embodiment of the present disclosure will be described with reference to Fig. 18 to Fig. 20. As shown in Fig. 18, the second embodiment illustrates a component mounting system 100 including a component mounting apparatus 10 configured similarly to the first embodiment, and an AI station 70 capable of communicating with the component mounting apparatus 10. Unlike the first embodiment, the second embodiment is different in that the AI ​​station 70 inputs image information of a three-dimensional image into a learning model M1, thereby estimating and recognizing the surface condition of the board B. That is, in the second embodiment, the controller 60 of the component mounting apparatus 10 does not recognize the surface condition of the board B based on the three-dimensional image.

[0072] 19 is a block diagram showing in detail the configuration of the AI ​​station 70. The AI ​​station 70 includes a control unit 71 and a storage unit 72. The control unit 71 includes a display control unit 77, a learning model generation unit 78, and an estimation unit 79. The storage unit 72 includes an image storage unit 73, a dataset storage unit 74, a learning model storage unit 75, and a recognition result storage unit 76.

[0073] The image storage unit 73 stores image information of the three-dimensional image of the substrate B captured by the TOF camera 40. The dataset storage unit 74 stores a dataset of the three-dimensional image of the substrate B and the surface state of the substrate B captured in the three-dimensional image. The learning model generation unit 78 performs machine learning using the dataset stored in the dataset storage unit 74, and creates and updates the learning model M1. The learning model generation unit 78 is an example of a learning unit.

[0074] The learning model storage unit 75 stores information constituting the learning model M1 created by the learning model generation unit 78. The learning model generation unit 78 updates the parameters of the learning model M1 through machine learning of a new data set, and updates the information of the existing learning model M1 stored in the learning model storage unit 75. The recognition result storage unit 76 stores the recognition result of the substrate B by the estimation unit 79.

[0075] The display control unit 77 displays predetermined information on the display unit 64. The predetermined information is, for example, various types of information stored in the storage unit 72, the recognition result of the estimation unit 79, and the like.

[0076] When a three-dimensional image of the substrate B is captured by the TOF camera 40, the estimation unit 79 uses the captured three-dimensional image as input data and estimates and recognizes the surface state of the substrate B based on the learning model M1 stored in the learning model storage unit 75. The estimation unit 79 is an example of an information processing unit.

[0077] The learning model M1 is constructed by a neural network model including an input layer, a hidden layer (intermediate layer), and an output layer, as shown in FIG. 20, for example.

[0078] The input layer is made up of, for example, a number of neurons corresponding to the number of pixels in the three-dimensional image of substrate B. The altitude value (surface height data) x of each pixel in the three-dimensional image of substrate B is input to each neuron in the input layer.

[0079] The hidden layer consists of multiple neurons connected to each neuron in the input layer. The hidden layer is a layer that extracts features from the 3D image of substrate B, and in this example, there are two hidden layers.

[0080] The output layer consists of two neurons connected to each neuron in the hidden layer. The output layer outputs the probability of the surface condition of substrate B being good or bad.

[0081] The connection weights w, v, and q of each layer are parameters of the learning model M1. The learning model generation unit 78 optimizes these parameters based on a new data set, thereby improving the estimation accuracy of the learning model M1.

[0082] By determining the surface condition of substrate B using machine learning learning model M1, it is expected that abnormalities in the surface condition of substrate B can be accurately determined even in cases where it is difficult to determine using threshold values ​​alone, such as when there is a gradual change in the surface condition.

[0083] In this embodiment, the memory unit 72 of the AI ​​station 70 stores the learning model M1, and the control unit 71 of the AI ​​station 70 uses the learning model M1 to recognize the surface condition of the board B. This reduces the consumption of memory capacity and the burden of calculation processing in the component mounting apparatus 10.

[0084] When the estimation unit 79 recognizes the surface condition of the substrate B and the recognition result is stored in the recognition result memory unit 76, the controller 60 of the component mounting device 10 determines the mounting operation of the component E on the substrate B by the head 32 in accordance with the recognition result, in the same manner as in embodiment 1.

[0085] (Effects of the second embodiment) (2-1) The component mounting system 100 according to the second embodiment further includes a learning model M1 generated by machine learning using a data set of the three-dimensional image acquired by the TOF camera 40 and the surface condition of the substrate B shown in the three-dimensional image, and the information processing unit (estimation unit 79) estimates and recognizes the surface condition of the substrate B by inputting the image information of the three-dimensional image acquired by the TOF camera 40 into the learning model M1.

[0086] According to this configuration, even if a threshold value for the height of the substrate B or the like is not set in advance, the surface state of the substrate B can be estimated by machine learning, and an abnormality in the surface state of the substrate B can be detected.

[0087] (2-2) The component mounting system 100 of the second embodiment further includes a learning unit (learning model generation unit 78) that updates the parameters of the learning model M1.

[0088] (Other embodiments) The above embodiment can be modified as follows: The above embodiment and the following modifications can be combined with each other within the scope of technical compatibility.

[0089] In the above-described embodiment 1, the TOF camera 40 images the surface of the substrate B after the substrate B is fixed to the component mounting position. However, the TOF camera may also image the surface of the substrate after the substrate is transported to the component mounting position and without being fixed.

[0090] In the above-described second embodiment, the AI ​​station 70 has a learning model M1 and recognizes the surface condition of the substrate B using the learning model M1, but the component mounting device may have a learning model and recognize the surface condition of the substrate using the learning model.

[0091] In the second embodiment, a neural network having an input layer, two hidden layers, and an output layer is used as the learning model M1 based on machine learning, but the learning model may be a neural network with a single hidden layer, etc. Also, a learning model other than a neural network may be used. [Explanation of symbols]

[0092] 10: Component mounting equipment 11: base, 12A to 12D: component supply unit, 13: conveying unit, 14: conveying belt, 15: backup device, 15A: clamping unit, 15B: holding unit, 15C: backup pin, 15D: lifting unit, 16: tape feeder, 17: board imaging camera, 18: component imaging camera 20: Drive unit, 21: X-axis beam, 22: Y-axis frame, 23: X-axis servo motor, 24: Y-axis servo motor 30: Head unit, 31: Head unit body, 32: Head, 33: Suction nozzle, 34: Z-axis servo motor, 35: R-axis servo motor 40: TOF camera, 41: light projecting unit, 42: light receiving unit 50: Cabinet 60: Controller, 61: CPU, 62: Memory, 63: Motor control unit, 64: Display unit, 65: Operation unit 70: AI station, 71: control unit, 72: memory unit, 73: image memory unit, 74: data set memory unit, 75: learning model memory unit, 76: recognition result memory unit, 77: display control unit, 78: learning model generation unit, 79: estimation unit 100: Component mounting system B: Board, BS: Bus, C1: Foreign object, E: Part, G1-G8: 3D image, L: Light, M1: Learning model, SP: Solder paste

Claims

1. a head that mounts components on the board; a transport unit that transports the board toward a component mounting position where the head mounts the component on the board; a time-of-flight (TOF) camera that irradiates light onto the board transported to the component mounting position and receives the reflected light to acquire a three-dimensional image of the surface of the board, which indicates the height of the surface of the board; an information processing unit that recognizes a surface condition of the substrate transported to the component mounting position based on the three-dimensional image acquired by the TOF camera, and determines an operation of mounting the components on the substrate by the head in accordance with the recognition result; A component mounting system comprising:

2. 2. The component mounting system of claim 1, wherein the information processing unit recognizes the surface condition of the substrate, including the degree of warping of the surface, based on the three-dimensional image acquired by the TOF camera, and determines the mounting operation of the component onto the substrate by the head according to the degree of warping of the surface of the substrate.

3. a housing that covers the component mounting position, the head, and the TOF camera; 3. The component mounting system according to claim 1, wherein the TOF camera is attached to the housing so as to be fixedly disposed above the component mounting position.

4. a moving body to which the head is attached; a moving body driving unit that moves the head by operating the moving body, 3. The component mounting system according to claim 1, wherein the information processing unit controls the moving body driving unit so that, when the TOF camera captures an image of the substrate transported to the component mounting position, the moving body is positioned so as not to overlap the component mounting position when viewed from above.

5. 3. The component mounting system according to claim 1, wherein the information processing unit measures the surface condition of the substrate, including the presence or absence of foreign matter, based on the three-dimensional image acquired by the TOF camera, and determines the mounting operation of the component on the substrate by the head according to the measurement results.

6. 3. The component mounting system according to claim 1, wherein the information processing unit determines whether or not the component is mounted on the board based on the three-dimensional image acquired by the TOF camera, and determines the mounting operation of the component on the board by the head based on the determination result.

7. The apparatus further includes a learning model generated by machine learning using the three-dimensional image acquired by the TOF camera and the surface state of the substrate captured in the three-dimensional image as a data set, 3. The component mounting system according to claim 1, wherein the information processing unit estimates and recognizes the surface condition of the substrate by inputting image information of the three-dimensional image acquired by the TOF camera into the learning model.

8. The component mounting system according to claim 7 , further comprising a learning unit that updates parameters of the learning model.

9. a head that mounts components on the board; a transport unit that transports the board toward a component mounting position where the head mounts the component on the board; a time-of-flight (TOF) camera that irradiates light onto the board transported to the component mounting position and receives the reflected light to acquire a three-dimensional image of the surface of the board, which indicates the height of the surface of the board; an information processing unit that recognizes a surface condition of the substrate transported to the component mounting position based on the three-dimensional image acquired by the TOF camera, and determines an operation of mounting the components on the substrate by the head in accordance with the recognition result; A component mounting device comprising:

Images