Mounting device, image processing device, and image processing method

Abstract

To more effectively execute processing regarding a super-resolution image.SOLUTION: A mounting device for acquiring an imaging image obtained by imaging a plurality of components collected for a plurality of collecting members, and mounting a component on the basis of a processing result of a super-resolution image based on the imaging image, comprises: an allocation processing part executed so as to allocate allocation processing for allocating a generation of the super-resolution image in each component to a plurality of tasks in order from a component earlier in a mounting order among the plurality of components, and so as to be allocated in order that an integration time integrating a generation schedule time of the super-resolution image of the component to be allocated in each task is small; and a super-resolution processing part that generates a super-resolution image in each component in each execution of the plurality of tasks in parallel processing, and processes the generated super-resolution image.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] This specification discloses a mounting apparatus, an image processing apparatus, and an image processing method. [Background technology]

[0002] Conventionally, a mounting device has been proposed that processes images of components captured from below and mounts the components based on the results (see, for example, Patent Document 1). In this mounting device, when processing a super-resolution image generated from multiple captured images, the image processing is performed efficiently by performing the image processing in parallel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 123534 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, when a captured image includes multiple components, in order to minimize the image processing area and shorten the processing time, it is conceivable to generate super-resolution images of areas determined for each component in the captured image and process the multiple super-resolution images. In addition, in order to improve the mounting efficiency of components, it is required to efficiently execute not only image processing after the generation of the super-resolution images, but also the generation of the super-resolution images.

[0005] A primary object of the present disclosure is to more efficiently execute processing related to super-resolution images. [Means for solving the problem]

[0006] In order to achieve the above-mentioned main object, the present disclosure has adopted the following means.

[0007] The mounting device of the present disclosure is A mounting apparatus that acquires captured images of a plurality of components picked up by a plurality of picking members, and mounts components based on a processing result of a super-resolution image based on the captured images, comprising: an allocation processing unit that executes an allocation process for allocating the generation of the super-resolution image for each component to a plurality of tasks in order of the components that are mounted first among the plurality of components, and in order of the tasks that have the shortest cumulative time, which is calculated by accumulating the planned times for generating the super-resolution image for each task, for the allocated components; a super-resolution processing unit that generates the super-resolution image for each component by executing the plurality of tasks in parallel processing and processes the generated super-resolution image; The gist of the project is to provide the following:

[0008] In the mounting device of the present disclosure, the allocation process of allocating the generation of super-resolution images for each component to multiple tasks is executed so that the tasks are allocated in order from the component that is mounted first, and the tasks are allocated in order from the task that has the shortest cumulative time for generating the super-resolution images of the assigned components. In addition, the multiple tasks are executed in parallel to generate super-resolution images for each component, and the generated super-resolution images are processed. This makes it possible to efficiently generate super-resolution images by parallel processing while suppressing bias in the cumulative time for each task. In addition, since the generation of super-resolution images is started in order from the component that is mounted first, it is easy to process the super-resolution images in the order of mounting, and it is possible to suppress delays in processing subsequent components due to waiting for processing of earlier components. Therefore, the processing related to the super-resolution images can be executed more efficiently. [Brief description of the drawings]

[0009] [Figure 1] 1 is a configuration diagram showing an outline of the configuration of a mounting system 1 including a mounting device 10. [Diagram 2] FIG. 2 is a diagram showing the outline of the configuration of a mounting head 40. [Diagram 3] FIG. 2 is an explanatory diagram showing the electrical connection relationship of the mounting system 1. [Figure 4] FIG. 4 is an explanatory diagram showing an example of rotational positions A to X of a nozzle 44 and the orientation of a part Pa. [Diagram 5] 11 is a flowchart showing an example of an implementation process. [Figure 6] FIG. 1 is an explanatory diagram showing an example of how a super-resolution image is generated. [Figure 7] 11 is a flowchart showing an example of a super-resolution related process. [Figure 8] FIG. 11 is an explanatory diagram showing a generation region Gr of a super-resolution image for each rotation position and a scheduled generation time. [Figure 9] 11 is an explanatory diagram showing the mounting order and the estimated time for generating a super-resolution image at each rotational position. [Figure 10] 11 is a flowchart showing an example of an allocation process. [Figure 11] An explanatory diagram showing how the generation of super-resolution images is assigned to tasks 1 to 4. [Figure 12] 11 is a flowchart showing an example of super-resolution image processing. [Figure 13] FIG. 13 is an explanatory diagram in which generation of a super-resolution image is assigned to tasks 1 to 4 in a comparative example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic diagram showing the configuration of a mounting system 1 including a mounting device 10. Fig. 2 is a schematic diagram showing the configuration of a mounting head 40. Fig. 3 is an explanatory diagram showing the electrical connection relationship of the mounting system 1. Note that the left-right direction in Fig. 1 is the X-axis direction, the front-rear direction is the Y-axis direction, and the up-down direction is the Z-axis direction.

[0011] As shown in Fig. 1, the mounting system 1 includes a mounting apparatus 10 and a management apparatus 100 that manages the mounting apparatus 10. The mounting system 1 includes a mounting line in which a plurality of mounting apparatuses 10 that mount components Pa (see Fig. 4) on a board Su are arranged along the transport direction (X-axis direction) of the board Su. For ease of explanation, Fig. 1 shows only one mounting apparatus 10. Note that the mounting system 1 may include a solder printer, an inspection machine, a reflow oven, and the like on the mounting line.

[0012] The mounting apparatus 10 includes a component supplying device 20, a board transporting device 25, an XY robot 30, a mounting head 40, a part camera 27, a mark camera 28, a side camera 29, and a control device 80 (see FIG. 3).

[0013] A plurality of component supply devices 20 are provided in line in the left-right direction on the front side of the mounting device 10. Each component supply device 20 is configured as a tape feeder that pulls out a tape 22, which contains components Pa in cavities formed at a predetermined pitch, from a reel 21 and sends it to a component supply position at a predetermined pitch. The components Pa in the cavities are protected by a film covering the surface of the tape 22, and the film is peeled off just before the component supply position to expose the components Pa.

[0014] The substrate transport device 25 includes a pair of conveyor belts 26 (only one of which is shown in FIG. 1) that are spaced apart from each other and span the left-right direction. When the substrate Su is transported by the conveyor belts 26 and reaches a predetermined position, it is supported by a number of support pins 24 that are erected on the back surface side.

[0015] The XY robot 30 includes a Y-axis guide rail 33 provided along the front-rear direction, and a Y-axis slider 34 that moves along the Y-axis guide rail 33 by driving a Y-axis motor 38. The XY robot 30 also includes an X-axis guide rail 31 provided along the left-right direction on the front side of the Y-axis slider 34, and an X-axis slider 32 that moves along the X-axis guide rail 31 by driving an X-axis motor 36. A mounting head 40, a mark camera 28, and a side camera 29 are attached to the X-axis slider 32. The mounting head 40, the mark camera 28, and the side camera 29 are moved to any position on the XY plane by driving the XY robot 30.

[0016] As shown in FIG. 2, the mounting head 40 includes a head body 41, a nozzle holder 42, and a nozzle (collection member) 44. The head body 41 is a disk-shaped rotating body, and a cylindrical reflector 41a (see FIG. 4) capable of reflecting light is attached to the center of the bottom surface. The mounting head 40 is configured as a rotary head, and a plurality of nozzle holders 42 are provided at predetermined intervals on the same circumference of the head body 41. The nozzles 44 are replaceably attached to the tip of each nozzle holder 42, and collect the components Pa from the component supply device 20. The nozzles 44 suck (collect) and hold the components Pa at their tips (lower ends) by utilizing the supplied negative pressure, and release the sucking of the components Pa by utilizing the supplied positive pressure or atmospheric pressure to mount the components Pa on the board Su. Note that, in FIG. 2, the reflector 41a is omitted, and eight nozzle holders 42 and eight nozzles 44 are shown, but in this embodiment, the number of nozzles 44 is 24 as shown in FIG. 4. Therefore, the number of nozzle holders 42 is also 24. Note that the mounting head 40 is not limited to 24, and may have 12 or 16 nozzle holders 42, as long as a plurality of nozzles 44 can be attached thereto. The mounting head 40 also includes an R-axis driving device 50, a Q-axis driving device 60, and a Z-axis driving device 70. In Fig. 2, a position for engaging with the Z-axis driving device 70 and two nozzle holders 42 and nozzles 44 at positions symmetrical to that position are indicated by solid lines, and other nozzle holders 42 and nozzles 44 are indicated by dashed and dotted lines.

[0017] The R-axis driving device 50 is a mechanism for revolving the nozzles 44 by rotating the head body 41, and includes an R-axis 51, an R-axis motor 54, and an R-axis position sensor 55 (see FIG. 3) for detecting the rotational position of the R-axis motor 54. The R-axis 51 extends in the vertical direction, and its lower end is attached to the central axis of the head body 41. The R-axis motor 54 rotates a gear 53 that meshes with an R-axis gear 52 provided at the upper end of the R-axis 51. The R-axis driving device 50 rotates the R-axis 51 by a predetermined angle (for example, 15 degrees) via the gear 53 and the R-axis gear 52 by the R-axis motor 54, thereby rotating the head body 41 by a predetermined angle in the direction of the arrow in FIG. 4 (clockwise direction). As a result, the nozzle holders 42 and the nozzles 44 revolve (revolve) by a predetermined angle in the circumferential direction.

[0018] The Q-axis driving device 60 is a mechanism for synchronously rotating (spinning) the multiple nozzles 44, and includes upper and lower two-stage Q-axis gears 61, 62, gears 63, 64, a Q-axis motor 65, and a Q-axis position sensor 66 (see FIG. 3) that detects the rotational position of the Q-axis motor 65. The Q-axis gears 61, 62 are inserted coaxially with and relatively rotatable relative to the R-axis 51. The gear 63 is provided on the upper portion of each nozzle holder 42, and meshes with the lower-stage Q-axis gear 61 so as to be slidable in the vertical direction. The Q-axis motor 65 drives and rotates the gear 64 that meshes with the upper-stage Q-axis gear 62. The Q-axis drive unit 60 rotates the Q-axis gears 61, 62 using the Q-axis motor 65, thereby rotating a gear 63 meshing with the Q-axis gear 61, and rotates (spins) each nozzle holder 42 (nozzle 44) about its central axis in the same rotational direction by the same rotational amount (rotational angle). Because the gear 63 meshes with the Q-axis gear 61, when each nozzle holder 42 (nozzle 44) is rotated by the drive of the R-axis drive unit 50, each nozzle holder 42 spins while rotating on its own axis.

[0019] For this reason, as shown in FIG. 4, the orientation (rotation angle θ) of the component Pa may differ depending on the rotation positions A to X of the nozzle 44. The 6 o'clock position in FIG. 4 is set as the rotation position A, and the rotation positions B, C, W, and X are set in the order of the position that reaches the rotation position A earliest by the circumferential rotation. The rotation position A is the position where the nozzle 44 is raised and lowered by the Z-axis driving device 70, and is the position where the nozzle 44 picks up and mounts the component Pa. For this reason, the rotation positions A to X are the order of picking and mounting the component Pa. In addition, the rotation positions A, G, M, and S where the rotation angle θ is the smallest (0 degrees) are also referred to as first rotation positions. The rotation positions B, F, H, L, N, R, T, and X where the rotation angle θ is larger than the first rotation position are also referred to as second rotation positions. The rotation positions C, E, I, K, O, Q, U, and W where the rotation angle θ is larger than the second rotation position are also referred to as third rotation positions. The rotational positions D, J, P, and V having a rotational angle θ larger than that of the third rotational position are also referred to as fourth rotational positions.

[0020] The Z-axis drive device 70 is provided at one location on the orbit of the revolving (revolving) of the nozzle holder 42, and is configured to be able to individually raise and lower the nozzle holder 42, which is at, for example, a rotational position A (see FIG. 4). The Z-axis drive device 70 includes a Z-axis slider 71, a Z-axis motor 73, and a Z-axis position sensor 74 (see FIG. 3) that detects the vertical position of the Z-axis slider 71. The Z-axis slider 71 is attached to a ball screw 72 extending in the vertical direction so as to be able to rise and fall, and includes a clamping portion 71a that clamps an engagement piece 42a extending laterally from the nozzle holder 42. The Z-axis motor 73 raises and lowers the Z-axis slider 71 by rotating the ball screw 72. When the nozzle holder 42 revolves and stops at the rotational position A, the engagement piece 42a is clamped by the clamping portion 71a of the Z-axis slider 71. Therefore, the Z-axis drive device 70 drives the Z-axis motor 73 to move the Z-axis slider 71 up and down along the ball screw 72, thereby enabling the nozzle holder 42 and the nozzle 44 to be raised and lowered.

[0021] The part camera 27 is disposed between the component supply device 20 and the board transport device 25, and images the components Pa held by each nozzle 44 from below. The mark camera 28 is disposed on the lower surface of the X-axis slider 32, and images the components Pa held on the tape 22 of the component supply device 20 and the marks attached to the board Su from above. The side camera 29 is disposed below the mounting head 40. The side camera 29 receives light that is emitted toward the reflector 41a of the head main body 41 and reflected, thereby imaging the components Pa adsorbed to the nozzles 44 at a predetermined rotational position from the side.

[0022] The control device 80 includes a control unit 81 having a CPU and an internal memory such as an SRAM and controlling the entire mounting device 10, an image processing unit 83 processing images, and a memory 89 such as a DRAM or a ROM. The control device 80 is connected to the management device 100 so as to be capable of bidirectional communication, and exchanges data and control signals with each other. The control unit 81 outputs control signals to the component supply device 20, the board transport device 25, the XY robot 30, the mounting head 40 (R-axis motor 54, Q-axis motor 65, Z-axis motor 73), the part camera 27, the mark camera 28, the side camera 29, and the like. The control unit 81 receives data from the component supply device 20, the board transport device 25, the XY robot 30, and the mounting head 40 (R-axis position sensor 55, Q-axis position sensor 66, Z-axis position sensor 74), and captured images captured by the part camera 27, the mark camera 28, and the side camera 29. The captured images are stored in the memory 89.

[0023] The image processing unit 83 includes a multi-core processor 85 having multiple cores and an internal memory 87 such as an SRAM, and performs image processing on images captured by the part camera 27, the mark camera 28, and the side camera 29. The multi-core processor 85 has, for example, four cores, 85a, 85b, 85c, and 85d, and executes parallel processing in which each core simultaneously processes an assigned task. The number of cores of the multi-core processor 85 is not limited to four, and may be any number. In addition to parallel processing, the multi-core processor 85 can also execute parallel processing in which one core processes multiple tasks while switching between them.

[0024] The management device 100 is a general-purpose computer, and includes a control unit 102, an input device 104 such as a keyboard or a mouse, a display 106, and a storage device 108 such as a HDD or an SSD. The control unit 102 is composed of a CPU, a ROM, a RAM, etc., and receives an input signal from the input device 104 and outputs an image signal to the display 106. The storage device 108 stores a production program for the board Su. The production program for the board Su includes information on which components Pa are to be mounted at which positions on the board Su and in what order, and information on how many boards Su are to be produced. The production program also includes information on the size of the components Pa and information on whether super-resolution processing is required. The super-resolution processing is, for example, a process of generating a higher quality super-resolution image from a plurality of captured images (multi-frame super-resolution processing). The management device 100 outputs a command signal to the control device 80 to mount the components Pa according to the production program.

[0025] The following is an explanation of the mounting process of the mounting device 10, particularly the mounting process of the component Pa that requires super-resolution processing. FIG. 5 is a flow chart showing an example of the mounting process executed by the control device 80. In the mounting process, the control unit 81 (CPU) controls the XY robot 30 to move the mounting head 40 above the component supply device 20, and causes each nozzle 44 to pick up the component Pa supplied to the component supply position by the component supply device 20 (S100). The control unit 81 repeats the operation of controlling the R-axis drive device 50 to sequentially move each nozzle 44 to the rotation position A and controlling the Z-axis drive device 70 to raise and lower the nozzle 44 at the rotation position A to pick up the component Pa, until the planned number of components Pa are picked up. As a result, the component Pa is picked up by each nozzle 44 (see FIG. 4).

[0026] Next, the control unit 81 captures a bottom surface image for super-resolution with the part camera 27 (S110). The control unit 81 controls the XY robot 30 to move the mounting head 40 above the part camera 27, and captures a bottom surface image including each component Pa with the part camera 27. When the bottom surface image is stored in the memory 89, the image processing unit 83 generates a super-resolution image of each component Pa and processes each super-resolution image. Therefore, after executing the process of S110, the control unit 81 waits to obtain the result of the super-resolution processing by the image processing unit 83 (S120).

[0027] FIG. 6 is an explanatory diagram showing an example of how a super-resolution image is generated. As shown in the figure, a super-resolution image Gs (high-resolution image) is generated from, for example, four images (low-resolution images) G1 to G4. The images G1 to G4 are bottom surface images captured by the part camera 27, shifted, for example, within a range of pixel pitch other than an integer (for example, 0.5 pixels, 1.5 pixels, etc.). By superimposing these images G1 to G4, a super-resolution image Gs with increased information between pixels is generated. In addition, since the images G1 to G4 that are actually captured are used, a super-resolution image Gs with higher reliability can be obtained compared to one in which information between pixels is estimated and interpolated.

[0028] FIG. 7 is a flowchart showing an example of a super-resolution related process. This process is executed by the image processing unit 83 (multi-core processor 85). The image processing unit 83 first judges whether or not it is time for super-resolution processing (S200). The image processing unit 83 judges that it is time for super-resolution processing when the bottom surface image captured in S110 is stored in the memory 89, and if not, judges that it is not time for super-resolution processing and ends this process. When it is judged that it is time for super-resolution processing, the image processing unit 83 sets the generation area Gr of the super-resolution image of each part Pa in the bottom surface image as a rectangular area based on the rotational position (nozzle position) of the nozzle 44 suctioning each part Pa and the size of the bottom surface of the part Pa (part size) (S210). The vertical and horizontal directions of the generation area Gr, which is a rectangular area, are extracted parallel to the vertical and horizontal arrangement of the pixels of the part camera 27.

[0029] FIG. 8 is an explanatory diagram showing the generation area Gr and the planned generation time of the super-resolution image for each rotation position, and FIG. 9 is an explanatory diagram showing the mounting order and the planned generation time of the super-resolution image for each rotation position. As described above, the rotation angle θ of the component Pa differs depending on the rotation position of the nozzle 44. In this embodiment, a rectangular area circumscribing the component Pa is set for each rotation position as the generation area Gr of the super-resolution image in the bottom surface image, with a slight margin added. As shown in FIG. 8, for example, at the first rotation position, a first generation area Gr1 that is slightly larger than the bottom surface size of the component Pa is set, and at the second rotation position, a second generation area Gr2 that is larger than the first generation area Gr1 is set. At the third rotation position, a third generation area Gr3 that is larger than the second generation area Gr2 is set, and at the fourth rotation position, a fourth generation area Gr4 that is larger than the third generation area Gr3 is set. Each generation area Gr of such a size is set based on each nozzle position in the bottom surface image. Also, the planned generation time required for generating the super-resolution image becomes longer as the area of ​​the generation area Gr becomes larger. In this embodiment, as shown in FIGS. 8 and 9, the expected generation times required for generating super-resolution images in the generation regions Gr1, Gr2, Gr3, and Gr4 are set to t1, t2, t3, and t4 (t1 <t2<t3<t4)とする。

[0030] After setting the generation area Gr in this way, the image processing unit 83 executes an allocation process for allocating the generation of super-resolution images in each generation area Gr to a plurality of tasks (S220). In this embodiment, the four cores 85a, 85b, 85c, and 85d execute tasks 1, 2, 3, and 4, respectively, and the generation of super-resolution images of each part Pa is allocated to the four tasks 1 to 4. However, the number of cores and the number of tasks are merely an example and are not limited to four each. Next, the image processing unit 83 causes each task to generate the allocated super-resolution image and executes super-resolution image processing for processing the generated super-resolution image (S230), and ends this process. The allocation process of S220 and the super-resolution image processing of S230 will be described in detail below.

[0031] 10 is a flowchart showing an example of the allocation process. In the allocation process, the image processing unit 83 acquires an accumulated time by accumulating the planned generation time of the super-resolution image allocated to tasks 1 to 4 for each task (S300). The accumulated time is accumulated in S350, which will be described later. Next, the image processing unit 83 determines whether there is one task with the shortest accumulated time (S310). At the beginning of the allocation process, all the accumulated times of the tasks are 0, so the image processing unit 83 determines that the tasks with the shortest accumulated time are all tasks 1 to 4, and not just one.

[0032] If the image processing unit 83 determines that there is not one task with the shortest integrated time but multiple tasks, it selects one task with the smallest serial number among the tasks with the shortest integrated time (S320) and proceeds to S340. For example, when S320 is executed initially when the allocation process starts, task 1 is selected from tasks 1 to 4. Also, if the image processing unit 83 determines in S310 that there is one task with the shortest integrated time, it selects that task (S330) and proceeds to S340.

[0033] Next, the image processing unit 83 assigns to the selected task the generation of a super-resolution image in the generation area Gr of the part Pa that is mounted earliest (the part Pa that is mounted at the next rotational position) (S340). When S340 is executed at the beginning of the allocation process, the generation of a super-resolution image in the generation area Gr1 of the part Pa that is at the rotational position A is assigned to task 1. Then, the image processing unit 83 accumulates the planned generation time (any of t1 to t4) of the super-resolution image of the part Pa (generation area Gr) allocated in S340 to update the accumulated time of the task (S350), and determines whether the allocation of all the parts Pa has been completed (S360). If the image processing unit 83 determines that the allocation of all the parts Pa has not been completed, it returns to S300 and repeats the process, and if it determines that the allocation of all the parts Pa has been completed, it ends this process.

[0034] FIG. 11 is an explanatory diagram in which the generation of a super-resolution image is assigned to each task 1 to 4. In the explanation of FIG. 11, for example, "assigning the generation of a super-resolution image in the generation area Gr1 of the part Pa at the rotation position A" is simply explained as "assigning the rotation position A". As described above, when S340 is executed at the beginning of the allocation process, the rotation position A is assigned to the task 1. In that state, the next rotation position B is assigned to the task 2 with the smallest serial number among the tasks 2, 3, and 4 with the shortest accumulated time, and the next rotation position C is assigned to the task 3 with the smallest serial number among the tasks 3 and 4 with the shortest accumulated time when assigned up to the rotation position B. In addition, the next rotation position D is assigned to the task 4 with the shortest accumulated time when assigned up to the rotation position C. The next rotation position E is assigned to the task 1 with the shortest accumulated time when assigned up to the rotation position D, the next rotation position F is assigned to the task 2 with the shortest accumulated time when assigned up to the rotation position E, and the next rotation position G is assigned to the task 3 with the shortest accumulated time when assigned up to the rotation position F. In the state where the allocation has been completed up to rotational position G, the accumulated times of tasks 1 to 4 are approximately the same, so the next rotational position I is allocated to task 1, which has the smallest serial number. The allocation process is similarly executed thereafter, and allocation to tasks 1 to 4 is completed as shown in Fig. 11. In this embodiment, a rotational position that is earlier in the mounting order starts generation of a super-resolution image before a rotational position that is later in the mounting order, or at the same time as a rotational position that is later in the mounting order. In other words, a rotational position that is later in the mounting order does not start generation of a super-resolution image before a rotational position that is earlier in the mounting order.

[0035] FIG. 12 is a flowchart showing an example of super-resolution image processing. In the super-resolution image processing, each of the cores 85a to 85d of the multi-core processor 85 of the image processing unit 83 processes tasks 1 to 4 in parallel, respectively, to sequentially generate super-resolution images of the assigned part Pa (S400). Next, the image processing unit 83 determines whether a super-resolution image has been generated in any of the tasks (S410), and when a super-resolution image has been generated in any of the tasks, determines whether super-resolution images have been generated simultaneously in two or more tasks, that is, whether multiple super-resolution images have been generated simultaneously (S420). When the image processing unit 83 determines that multiple super-resolution images have been generated simultaneously, it sets the super-resolution images of the part Pa (nozzle 44) that is mounted first as the target of image processing in order (S430). For example, in FIG. 11, the super-resolution images of the rotational positions D to G are generated substantially simultaneously. Therefore, the rotational positions D, E, F, and G are set as the target of image processing in the order of mounting. On the other hand, if the image processing unit 83 determines that a plurality of super-resolution images have not been generated simultaneously, it skips S430 since it is sufficient to perform image processing on the one generated super-resolution image.

[0036] Next, the image processing unit 83 performs image processing on the super-resolution image (S440) and stores the processing result in the memory 89 (S450). The image processing unit 83 performs image processing on the generated super-resolution image while generating another super-resolution image by parallel processing in any one of the cores of the multi-core processor 85. The processing result includes the amount of positional deviation of the part Pa with respect to the nozzle 44. Each time image processing of a super-resolution image of one part Pa is performed, the processing result is stored in the memory 89. The image processing unit 83 may transmit the processing result of the super-resolution image to the control unit 81. Then, the image processing unit 83 determines whether or not there is an unprocessed super-resolution image among the generated super-resolution images (S460), and if it determines that there is an unprocessed super-resolution image, the process returns to S440. For example, when super-resolution images for rotational positions D to G are generated simultaneously and the super-resolution image for rotational position D has been image-processed, it is determined that there are unprocessed super-resolution images for rotational positions E to G, and the process returns to S440 to image-process the super-resolution image for rotational position E. On the other hand, when the image processing unit 83 determines that there are no unprocessed super-resolution images, it determines whether or not the processing of all super-resolution images has been completed (S470), and if it determines that the processing of all super-resolution images has not been completed, it returns to S410, and if it determines that the processing of all super-resolution images has been completed, it ends this process.

[0037] In S120 of the mounting process in FIG. 5, when the control unit 81 determines that the processing result of the super-resolution image in S450 in FIG. 11 has been acquired, the control unit 81 mounts the component Pa at the corrected mounting position based on the processing result (S130). The control unit 81 controls the XY robot 30 to move the mounting head 40 (rotation position A) to the corrected mounting position, and controls the Z-axis driving device 70 to raise and lower the nozzle 44 at the rotation position A to mount the component Pa on the board Su. After mounting the component Pa, the control unit 81 controls the R-axis driving device 50 to rotate the next component Pa in the mounting order to the rotation position A. Next, the control unit 81 determines whether mounting of each component Pa has been completed (S140), and if it is determined that it has not been completed, the process returns to S120, and if it is determined that it has been completed, the process ends. In this way, each time the processing result of the super-resolution image is obtained, the component Pa is mounted at the mounting position based on the processing result. Even if the processing of the super-resolution image of a later component Pa is completed when a later component Pa has not been mounted, the component Pa cannot be mounted and remains awaits mounting.

[0038] Here, FIG. 13 is an explanatory diagram in which the generation of super-resolution images is assigned to each task 1 to 4 in the comparative example. The assignment process in the comparative example will be described using rotational positions as in FIG. 11, but each rotational position A to X is assigned to tasks 1 to 4 in order of four positions from the earliest mounting order (see FIG. 9). That is, the rotational positions A, B, C, and D are assigned to each of tasks 1 to 4 in the first time, the rotational positions E, F, G, and H are assigned to the second time, the rotational positions I, J, K, and L are assigned to the third time, the rotational positions M, N, O, and P are assigned to the fourth time, the rotational positions Q, R, S, and T are assigned to the fifth time, and the rotational positions U, V, W, and X are assigned to the sixth time. For this reason, for example, when the generation of rotational position K is completed in task 3, the generation of rotational position O, which is later in the mounting order than rotational position N, is started first, and the start of the generation of rotational position N is delayed. Furthermore, when the generation of rotational position Q is completed in task 1, the generation of rotational position U, which is later in the mounting order than rotational position T, is started first, and the start of the generation of rotational position T is put on hold. Similarly, when the generation of rotational position S is completed in task 3, the generation of rotational position W, which is later in the mounting order than rotational positions T and V, is started first, and the start of the generation of rotational positions T and V is put on hold.

[0039] In this way, in the comparative example, the generation of a super-resolution image may be started for a rotational position with a later mounting order before the generation of a super-resolution image for a rotational position with an earlier mounting order. Therefore, even if the generation and processing of a super-resolution image for a component Pa with a later mounting order is completed, the generation and processing of a super-resolution image for the earlier component Pa is not completed, so that a wait for mounting of the later component Pa is likely to occur. On the other hand, in this embodiment, the tasks are assigned in order from the rotational position (component Pa) with the earliest mounting order, and are assigned in order to the task with the shortest cumulative time. Also, when there are multiple tasks with the shortest cumulative time, the tasks are assigned to the task with the smallest serial number. Therefore, as described in FIG. 11, the generation of a super-resolution image is started in order from the rotational position with the earliest mounting order, and the generation of a super-resolution image for a rotational position with a later mounting order is started first, so that the start of the generation of a super-resolution image for a rotational position with an earlier mounting order is not made to wait. Therefore, compared to the comparative example, it is possible to suppress the occurrence of a wait for mounting of the component Pa.

[0040] Here, the correspondence between the components of this embodiment and the components of this disclosure will be clarified. The image processing unit 83 that executes the allocation process of S220 of the super-resolution related process of this embodiment corresponds to the allocation processing unit, and the image processing unit 83 that executes the super-resolution image process of S230 corresponds to the super-resolution processing unit. The mounting head 40 corresponds to the rotary head. Also, by explaining the operation of the mounting device 10, an example of the image processing device and image processing method of the present disclosure will be clarified.

[0041] As described above, in the mounting device 10 of the present embodiment, the allocation process of allocating the generation of a super-resolution image for each component Pa to a plurality of tasks is executed so that the tasks are allocated in order from the component Pa (rotation position) that is mounted first, and the tasks are allocated in order from the task with the shortest cumulative time of the planned generation time of the super-resolution image of the allocated component Pa. In addition, the super-resolution image is generated for each component Pa by executing a plurality of tasks in parallel processing, and the generated super-resolution image is processed. This suppresses bias in the cumulative time for each task, and therefore the generation of the super-resolution image by parallel processing can be efficiently performed. In addition, since the generation of the super-resolution image is started in order from the component Pa that is mounted first, the processing of the super-resolution image is also easily performed in the mounting order, and it is possible to suppress delays in the processing of the subsequent components Pa due to waiting for the processing of the previous components Pa. Therefore, the processing related to the super-resolution image can be executed more efficiently.

[0042] Furthermore, if there are a plurality of tasks with the shortest integrated time, the task with the smallest serial number is assigned, so that the bias in the integrated time of each task can be suppressed by a simple process.

[0043] Furthermore, since the allocation process is completed before parallel processing of multiple tasks begins, there is no need to interrupt the parallel processing and perform the allocation process. Each task only needs to sequentially generate super-resolution images of the assigned parts Pa, allowing parallel processing to be performed efficiently.

[0044] Furthermore, a super-resolution image is generated for each part Pa using a generation area Gr for each part Pa that is determined based on the position (rotational position) of each part Pa in the captured image (bottom image). The generation area Gr has a different size depending on the position of each part Pa in the captured image, and times t1 to t4 that are determined in advance according to the size of the generation area Gr are used as the estimated generation time for the super-resolution image. Therefore, since there is no need to calculate the estimated generation time during the allocation process, the integrated time can be calculated quickly and the allocation process can be executed efficiently.

[0045] Moreover, the generation region Gr is a rectangular region whose size is determined based on the circumferential positions (rotational positions) of the multiple nozzles 44 and the size of the bottom surface of the component Pa. In the mounting head 40, which is a rotary head, the rotation angle of the component Pa varies depending on the rotational position of the nozzles 44, and the size of the generation region Gr differs even for the same component Pa, resulting in a different expected generation time. For this reason, it is highly meaningful to calculate the integrated time using a predetermined expected generation time.

[0046] Furthermore, each time a super-resolution image for each component Pa is generated, that super-resolution image is processed, and each time a super-resolution image is processed, the mounting process for the component Pa is executed based on the processing result. As a result, while a delay in the processing of the subsequent components Pa can easily occur due to waiting for the processing of the super-resolution image of the component Pa that is earlier in the mounting order, the generation of the super-resolution image is started in order of the component Pa that is earlier in the mounting order, making it easier to process the super-resolution image in the mounting order as well. This prevents a delay in mounting the component Pa from occurring due to a processing delay, and allows the mounting process for multiple components Pa to be executed more efficiently.

[0047] It goes without saying that the present disclosure is in no way limited to the above-described embodiment, and can be embodied in various forms as long as it falls within the technical scope of the present disclosure.

[0048] In the above-described embodiment, each time a super-resolution image is generated for each part Pa, the super-resolution image is processed, and each time a super-resolution image is processed, the part Pa is mounted based on the processing result, but this is not limited to the above. For example, all super-resolution images may be generated and then each super-resolution image may be processed in sequence, or all super-resolution images may be processed and then each part Pa may be mounted in sequence. However, in order to efficiently process the super-resolution images and mount the part Pa based on the processing result, it is preferable to use the embodiment.

[0049] In the embodiment, the mounting head 40 is a rotary head, but it may be any head that picks up a plurality of components Pa using a plurality of nozzles 44. For example, the mounting head 40 may be one that includes a plurality of nozzle holders 42 (nozzles 44) arranged in parallel.

[0050] In the embodiment, the times t1 to t4 predetermined according to the size of the generation area Gr are used as the expected generation time of the super-resolution image, but the expected generation time may be calculated based on the size of the generation area Gr during the allocation process. Also, the size of the generation area Gr is illustrated as being different depending on the position (rotational position) of each component Pa in the captured image, but it may be different depending on the size of each component Pa without changing depending on the position of each component Pa. Also, when generating the super-resolution image, an image (bottom image) containing all the components Pa held by the mounting head 40 (each nozzle 44) is captured, but this is not limited thereto, and a required number of images containing only a part (one or more) of the components Pa may be captured.

[0051] In the embodiment, the allocation process is completed before the generation of the super-resolution image is started, but the generation of the super-resolution image may be started before the allocation process is completed. That is, the generation of the super-resolution image may be started in the middle of the allocation process. Furthermore, when there are multiple tasks with the shortest integrated time among multiple tasks, the task with the smallest serial number is selected, but a task may be selected in another manner, such as selecting the task with the smallest number of allocations for the generation of the super-resolution image.

[0052] In the embodiment, the present disclosure has been described in the form of the mounting device 10, but it may also be in the form of an image processing device or an image processing method.

[0053] This specification also discloses the technical idea of ​​changing "the mounting device according to claim 1 or 2" in claim 4 at the time of the application to "the mounting device according to any one of claims 1 to 3" and the technical idea of ​​changing "the mounting device according to claim 1 or 2" in claim 6 at the time of the application to "the mounting device according to any one of claims 1 to 5". [Industrial Applicability]

[0054] The present disclosure can be used in the technical field of mounting components. [Explanation of symbols]

[0055] 1 Mounting system, 10 Mounting device, 20 Component supply device, 21 Reel, 22 Tape, 24 Support pin, 25 Board transport device, 26 Conveyor belt, 27 Parts camera, 28 Mark camera, 29 Side camera, 30 XY robot, 31 X-axis guide rail, 32 X-axis slider, 33 Y-axis guide rail, 34 Y-axis slider, 36 X-axis motor, 38 Y-axis motor, 40 Mounting head, 41 Head body, 41a Reflector, 42 Nozzle holder, 44 Nozzle, 50 R-axis drive device, 51 R-axis, 52 R-axis gear, 53 Gear, 54 R-axis motor, 55 R-axis position sensor, 60 Q-axis drive device, 61 Q-axis gear, 62 Q-axis gear, 63, 64 Gear, 65 Q-axis motor, 66 Q-axis position sensor, 70 Z-axis drive device, 71 Z-axis slider, 71a Clamping unit, 72 ball screw, 73 Z-axis motor, 74 Z-axis position sensor, 80 control device, 81, 102 control unit, 83 image processing unit, 85 multi-core processor, 85a to 85d cores, 87 internal memory, 89 memory, 100 management device, 104 input device, 106 display, 108 storage device, Gr generation area, Pa component, Su board.

Claims

1. A mounting apparatus that acquires captured images of a plurality of components picked up by a plurality of picking members, and mounts components based on a processing result of a super-resolution image based on the captured images, comprising: an allocation processing unit that executes an allocation process for allocating the generation of the super-resolution image for each component to a plurality of tasks in order of the components that are mounted first among the plurality of components, and in order of the tasks that have the shortest cumulative time, which is calculated by accumulating the planned times for generating the super-resolution image for each task, for the allocated components; a super-resolution processing unit that generates the super-resolution image for each component by executing the plurality of tasks in parallel processing and processes the generated super-resolution image; A mounting device comprising:

2. The plurality of tasks are assigned sequential numbers, the allocation processing unit, when there are a plurality of tasks having the shortest integrated time among the plurality of tasks, allocates the task to the task having the smallest serial number; The mounting device according to claim 1 .

3. the allocation processing unit completes the allocation processing before the super-resolution processing unit starts executing the parallel processing of the plurality of tasks. The mounting device according to claim 1 .

4. the super-resolution processing unit generates the super-resolution image for each component by using a generation area for each component that is determined based on a position of each component in the captured image; The generation area may have a different size depending on the position of each part in the captured image, the allocation processing unit uses a time that is predetermined depending on a size of the generation area as the planned generation time of the super-resolution image. The mounting device according to claim 1 .

5. A rotary head is provided in which the plurality of sampling members are arranged on the same circumference so as to be rotatable in a circumferential direction and the sampling members rotate about their own axes when rotating, The generation area is a rectangular area whose size is determined based on the circumferential positions of the plurality of collection members and the size of the part. The mounting device according to claim 4 .

6. the super-resolution processing unit sequentially processes the super-resolution images each time the super-resolution processing unit generates the super-resolution images for each component; each time the super-resolution image is processed by the super-resolution processing unit, a mounting process for the components is sequentially executed based on a processing result of the super-resolution image. The mounting device according to claim 1 .

7. An image processing device that acquires captured images of a plurality of parts picked up by a plurality of picking members and processes a super-resolution image based on the captured images, an allocation processing unit that executes an allocation process for allocating the generation of the super-resolution image for each component to a plurality of tasks in order of the components that are mounted first among the plurality of components, and in order of the tasks that have the shortest cumulative time, which is calculated by accumulating the planned times for generating the super-resolution image for each task, for the allocated components; a super-resolution processing unit that generates the super-resolution image for each component by executing the plurality of tasks in parallel processing and processes the generated super-resolution image; An image processing device comprising:

8. 1. An image processing method for acquiring captured images of a plurality of parts picked up by a plurality of picking members, and processing a super-resolution image based on the captured images, comprising: (a) executing an allocation process for allocating the generation of the super-resolution image for each component to a plurality of tasks in a manner that the generation is performed in order from the components that are mounted earliest among the plurality of components, and in order to allocate the generation of the super-resolution image for the assigned component to a task having a smallest cumulative time, which is calculated by accumulating a planned time for generating the super-resolution image for each task; (b) generating the super-resolution image for each part by executing the plurality of tasks in parallel, and processing the generated super-resolution image; An image processing method comprising:

Images