Illumination condition creation support device, captured image estimation method, and illumination condition creation method

Abstract

To provide a lighting condition creation support device, a captured image estimation method, and a lighting condition creation method that can acquire in a short time captured images of a component under a plurality of lighting conditions.SOLUTION: A captured image estimation method includes: a first imaging step of capturing images of a component lit by lighting means under a plurality of lighting conditions with imaging means; a learning step (ST1) of generating a learning model 41 on the basis of the plurality of captured images obtained in the first imaging step and the lighting conditions (captured image data 40) at the time of imaging; a second imaging step of imaging one component D99 lit by the lighting means under one lighting condition 1 with the imaging means; and an image estimation step (ST2) of estimating other captured images (estimated image data 42) obtained when the imaging means images the one component D99 lit by the lighting means under other lighting conditions 6 and 7 from the captured image (captured image data 40) of the component D99 obtained in the second imaging step using the generated learning model 41.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an illumination condition creation support device that supports the creation of illumination conditions for an illumination means that illuminates a component, a captured image estimation method, and an illumination condition creation method. [Background technology]

[0002] The component mounting device uses a nozzle to pick up components supplied by a component supply device (such as a tape feeder), illuminates the components held by the nozzle with lighting means while capturing an image with a camera (imaging means), and recognizes the holding state of the components, such as the amount of positional deviation of the components relative to the nozzle, before mounting the components held by the nozzle on a board. The optimal lighting conditions under which the lighting means illuminates the components differ for each component depending on the material and shape of the component, so work is done to set the optimal lighting conditions before production begins.

[0003] Patent Document 1 discloses a component mounting device (electronic component mounting device) that, when an image of a component captured by a camera during the production of mounted boards is determined to be defective, switches to an arrangement mode in which an operator changes the lighting conditions and has the camera re-image the component, and then resets the lighting conditions of the lighting means consisting of four types of lighting units while viewing the captured image displayed on an operation screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2004-186175 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional technologies including Patent Document 1, it takes several seconds for a camera to capture an image of a component, and furthermore, in order to determine the optimal lighting conditions for an illumination means composed of multiple illumination units, it is necessary to capture an image under a combination of multiple conditions. This causes problems in that it takes a long time to capture images of a large number of components mounted on a single mounting board under multiple lighting conditions and then set the optimal lighting conditions based on the captured images.

[0006] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide an illumination condition creation support device, an image capture estimation method, and an illumination condition creation method that are capable of acquiring images of a part under a plurality of illumination conditions in a short period of time. [Means for solving the problem]

[0007] The illumination condition creation support device of the present invention is an illumination condition creation support device that supports creation of illumination conditions of an illumination means that illuminates a component imaged by an imaging means, and includes a learning unit that generates a learning model based on a plurality of captured images captured by the imaging means of a component illuminated by the illumination means under a plurality of illumination conditions and the illumination conditions at the time of capturing the images, and an image estimation unit that uses the generated learning model to estimate, from one captured image captured by the imaging means of a component illuminated by the illumination means under one illumination condition, another captured image that would be obtained when the imaging means captures the one component illuminated by the illumination means under another illumination condition. an input receiving unit that receives an input of the other illumination condition; and a display unit that displays the other captured image estimated by the image estimation unit based on the input other illumination condition; Equipped with:

[0008] The captured image estimation method of the present invention includes a first imaging step of imaging a component illuminated by a lighting means under a plurality of lighting conditions with an imaging means, a learning step of generating a learning model based on the plurality of captured images obtained by the first imaging step and the lighting conditions at the time of imaging, a second imaging step of imaging a component illuminated by the lighting means under a single lighting condition with the imaging means, and an image estimation step of estimating, using the generated learning model, from the captured image of the component obtained by the second imaging step, another captured image that would be obtained if the imaging means captured the component illuminated by the lighting means under a different lighting condition. an input receiving step of receiving an input of the other lighting condition; and a display step of displaying the other captured image estimated in the image estimation step based on the input other lighting condition; Includes.

[0009] The illumination condition creation method of the present invention is a method for creating illumination conditions used when the illumination means illuminates a part imaged by the imaging means, the method including: a first imaging step of imaging a part illuminated by the illumination means under a plurality of illumination conditions with the imaging means; a learning step of generating a learning model based on a plurality of captured images obtained by the first imaging step and the illumination conditions at the time of imaging; a second imaging step of imaging a part illuminated by the illumination means under a single illumination condition with the imaging means; an image estimation step of estimating, from the one captured image obtained by the second imaging step, another captured image that will be obtained when the imaging means images the part illuminated by the illumination means under a different illumination condition, using the learning model generated by the learning step; and an illumination condition creation step of creating illumination conditions suitable for the illumination means to illuminate the part, based on the other captured image of the part estimated by the image estimation step. an input receiving step of receiving an input of the other lighting condition; and a display step of displaying the other captured image estimated in the image estimation step based on the input other lighting condition; Includes. Effect of the Invention

[0010] According to the present invention, images of a component can be captured under a plurality of illumination conditions in a short period of time. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a configuration explanatory diagram of a component mounting system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a configuration explanatory diagram of a component mounting device according to an embodiment of the present invention; [Diagram 3] FIG. 1 is a configuration explanatory diagram of a component recognition camera provided in a component mounting device according to an embodiment of the present invention; [Figure 4] FIG. 1 is a block diagram showing a configuration of a control system of a component mounting apparatus according to an embodiment of the present invention. [Diagram 5] FIG. 1 is an explanatory diagram of a captured image estimation according to an embodiment of the present invention; [Figure 6] FIG. 13 is a diagram showing an example of an illumination condition input screen displayed on a touch panel provided in the component mounting apparatus according to the embodiment of the present invention; [Figure 7] FIG. 13 is a diagram showing an example of an illumination condition input screen displayed on a touch panel provided in the component mounting apparatus according to the embodiment of the present invention; [Figure 8] FIG. 13 is a diagram showing an example of an illumination condition input screen displayed on a touch panel provided in the component mounting apparatus according to the embodiment of the present invention; [Figure 9] FIG. 2 is a flow diagram of a captured image estimation method according to an embodiment of the present invention. [Figure 10] FIG. 2 is a flow diagram of a method for creating illumination conditions according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An embodiment of the present invention will be described in detail below with reference to the drawings. The configurations, shapes, etc. described below are examples for explanation, and can be appropriately changed according to the specifications of the component mounting device, the component recognition camera, the management computer, and the offline camera device. In the following, the same reference numerals are used for corresponding elements in all the drawings, and duplicated explanations are omitted. In FIG. 2 and some parts described later, the X-axis (perpendicular to the paper surface in FIG. 2) in the board transport direction and the Y-axis (left-right direction in FIG. 2) perpendicular to the board transport direction are shown as two axes perpendicular to each other in the horizontal plane. In FIG. 2 and some parts described later, the Z-axis (up-down direction in FIG. 2) is shown as the height direction perpendicular to the horizontal plane.

[0013] First, the configuration of component mounting system 1 will be described with reference to Fig. 1. Component mounting system 1 is configured by connecting multiple component mounting apparatuses M1 to M3. Component mounting apparatuses M1 to M3 are connected to management computer 3 via communication network 2. Management computer 3 manages the production of mounted boards by component mounting system 1, and also assists in creating lighting conditions used by component mounting apparatuses M1 to M3, which will be described later. Note that the number of component mounting apparatuses M1 to M3 provided in component mounting system 1 is not limited to three, and may be one, two, four or more.

[0014] The component mounting system 1 includes an offline camera device 4 that is not connected to the component mounting devices M1 to M3. The offline camera device 4 includes an offline camera having a configuration similar to that of a component recognition camera 20 (see FIG. 3) included in the component mounting devices M1 to M3 described later. In the offline camera device 4, prior to the production of a mounted board, an image of a new component to be mounted on the board is captured by the offline camera. Images captured by the offline camera are transmitted to the management computer 3 and used to create lighting conditions and the like to be used by the component mounting devices M1 to M3. Alternatively, images captured by the offline camera are processed by an information processing device included in the offline camera device 4 to create lighting conditions to be used by the component mounting devices M1 to M3.

[0015] Next, the configuration of component mounting devices M1 to M3 will be described with reference to FIG. 2. The component mounting devices M1 to M3 have the same configuration, and only the component mounting device M1 will be described below. The component mounting device M1 has a function of mounting a component D on a board B. A board transport mechanism 6 provided on the upper surface of a base 5 transports, positions, and holds the board B along the X-axis. A head moving mechanism 7 provided above the base 5 moves a mounting head 8, which is detachably attached via a plate 7a, in horizontal directions (X-axis direction and Y-axis direction). A suction nozzle 9 is detachably attached to the lower end of the mounting head 8.

[0016] A plurality of tape feeders 10 are mounted in line along the X-axis on feeder base 11a on the upper part of dolly 11, which is coupled to base 5 on the side of substrate transport mechanism 6. Carrier tape 12, which stores components D to be supplied to component mounting device M1, is wound and stored on reel 13 and held on dolly 11. Carrier tape 12 inserted into tape feeder 10 is pitch-fed, and components D stored on carrier tape 12 are sequentially supplied to component supply port 10a provided on the upper part of tape feeder 10.

[0017] 2, in the component mounting operation, mounting head 8 is moved above tape feeder 10 by head moving mechanism 7, and picks up component D supplied to component supply port 10a of tape feeder 10 by vacuum suction with suction nozzle 9 (arrow a). Mounting head 8 holding component D is moved above board B held by board transport mechanism 6 by head moving mechanism 7, and mounts component D at a predetermined component mounting position Ba on board B (arrow b).

[0018] 2, a board recognition camera 14 with its optical axis facing in the negative direction of the Z axis is attached to the plate 7a. The board recognition camera 14 moves integrally with the mounting head 8 in the positive and negative directions of the X axis and the positive and negative directions of the Y axis by the head moving mechanism 7. As the mounting head 8 moves, the board recognition camera 14 moves above the board B positioned by the board transport mechanism 6, and captures an image of a board mark (not shown) provided on the board B.

[0019] A component recognition camera 20 with its optical axis facing upward is attached to the upper surface of the base 5 between the board transport mechanism 6 and the tape feeder 10. The component recognition camera 20 captures an image of the holding posture of the component D held by the suction nozzle 9 when the suction nozzle 9 that picked up the component D passes above. When the mounting head 8 mounts the component on the board B, the mounting position is corrected taking into account the image of the component D captured by the component recognition camera 20 and the image of the board position captured by the board recognition camera 14.

[0020] 2, a touch panel 15 operated by an operator is provided in front of component mounting apparatus M1 at a position where the operator works. Touch panel 15 displays various information on its display section, and the operator inputs data and operates component mounting apparatus M1 using operation buttons or the like, which are an input section displayed on the display section.

[0021] Next, the configuration of the component recognition camera 20 will be described with reference to Fig. 3. The component recognition camera 20 includes an imaging unit 22 having an imaging element such as a one-dimensional CCD or a two-dimensional CMOS sensor inside a housing 21. A ceiling board 21a made of plate glass or the like is installed on the upper surface of the housing 21. A lens 23 is disposed above the imaging unit 22. The imaging unit 22 is installed with its optical axis 22a facing upward, and captures an image of the component D held by the suction nozzle 9 through the lens 23 and the ceiling board 21a. In other words, the imaging unit 22 and the lens 23 form an imaging means C that captures an image of the component D from below.

[0022] A reflective illumination unit 24, a transmitted illumination unit 25, and a side illumination unit 26 each configured with a plurality of LED chips and the like are installed inside the housing 21. The light emitted from the reflective illumination unit 24 mainly illuminates the bottom surface Da of the component D. The light emitted from the transmitted illumination unit 25 is mainly reflected by the flange 9a of the suction nozzle 9 and illuminates the component D from above. The light emitted from the side illumination unit 26 mainly illuminates bumps Db and the like formed on the bottom surface Da of the component D.

[0023] 3, the component recognition camera 20 includes an illumination driver 27. The illumination driver 27 supplies currents for the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 to emit light. The illumination driver 27 changes the current values ​​of the currents supplied to the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 based on illumination values ​​included in the illumination conditions transmitted from a control device 30 (see FIG. 4) included in the component mounting device M1. In other words, the illumination conditions are a combination of the illumination values ​​of the multiple illumination units.

[0024] Illumination conditions are registered for each type of part D, and the intensity (brightness) of light irradiated from the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 is changed according to the part D being imaged. In this way, the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26 are illumination means L that illuminates the part D imaged by the imaging means C (imaging unit 22, lens 23). In other words, the illumination means L includes multiple illumination units (reflective illumination unit 24, transmitted illumination unit 25, side illumination unit 26) that illuminate the part D, and the intensity of light irradiated by the multiple illumination units is changed according to the illumination value.

[0025] 3, in a state where the lighting means L is irradiating light according to lighting conditions, the component mounting apparatus M1 images (scans) the component D with the imaging means C while moving the suction nozzle 9 holding the component D in the horizontal direction (arrow c). The direction in which the component D is moved is changed as appropriate according to the specifications of the mounting head 8 and the component recognition camera 20. In this way, scanning and imaging to image the component D while moving it takes about several seconds.

[0026] Next, the configuration of the control system of component mounting devices M1 to M3 will be described with reference to Fig. 4, focusing on the function of creating the illumination conditions for component recognition camera 20. Component mounting devices M1 to M3 have the same configuration, and below, component mounting device M1 will be described. Component mounting device M1 is equipped with a control device 30, a board transport mechanism 6, a head moving mechanism 7, a mounting head 8, a tape feeder 10, a board recognition camera 14, a touch panel 15, and a component recognition camera 20.

[0027] The control device 30 includes a storage unit 31, a mounting control unit 32, an imaging processing unit 33, a learning unit 34, an image estimation unit 35, an input receiving unit 36, an illumination condition creation unit 37, and an illumination condition feasibility determination unit 38. The storage unit 31 is a storage device, and stores mounting data 39, captured image data 40, a learning model 41, estimated image data 42, illumination condition data 43, inter-device correction data 44, etc. In the mounting data 39, various information such as the component type and size of component D to be mounted on board B, and the coordinates of component mounting position Ba on board B is stored for each board type of mounting boards to be produced.

[0028] 4, the mounting control unit 32, based on mounting data 39, executes a component mounting operation in which the mounting head 8 picks up a component D supplied by the tape feeder 10 using the suction nozzle 9 and mounts the component at a component mounting position Ba on the board B. The imaging processing unit 33 scans and images the component D held by the suction nozzle 9 using the imaging means C (imaging unit 22, lens 23) while moving the suction nozzle 9 holding the component D above the component recognition camera 20 during the component mounting operation, and recognizes the holding orientation of the component D.

[0029] When scanning and imaging the component D, the imaging processing unit 33 controls the illumination driving unit 27 according to the illumination conditions included in the illumination condition data 43 to irradiate a predetermined amount of light from the illumination means L (the reflective illumination unit 24, the transmitted illumination unit 25, and the side illumination unit 26). The imaging processing unit 33 also executes a learning data creation process in which the illumination means L illuminates the component D held by the suction nozzle 9 under a plurality of illumination conditions while the imaging means C scans and images the component D to obtain a plurality of captured images. The imaging processing unit 33 associates information identifying the captured component D and the illumination conditions at the time of capturing the image with the captured image and stores the information as captured image data 40 in the storage unit 31. The learning data creation process is executed during spare time during component mounting work, during a changeover work to change the board type of the mounted board to be produced, and the like.

[0030] 4, the learning unit 34 uses a learning algorithm using machine learning or the like to generate a learning model 41 that estimates an image of the part D (to be described later) by using, as teacher data, a plurality of captured images captured by the imaging means C of the part D illuminated under a plurality of illumination conditions by the illumination means L included in the captured image data 40 and the illumination conditions at the time of capturing the images. As the learning algorithm, a neural network (including deep learning using a multi-layered neural network), genetic programming, a decision tree, a Bayesian network, a support vector machine (SVM), or the like can be used. The generated learning model 41 is stored in the storage unit 31.

[0031] In this way, the learning unit 34 provided in the component mounting device M1 generates a learning model 41 based on multiple captured images of the component D illuminated under multiple lighting conditions by the lighting means L provided in the component mounting device M1, captured by the imaging means C provided in the component mounting device M1, and the lighting conditions at the time of capturing the images.

[0032] 4, the image estimation unit 35 estimates (generates) from one captured image of one part D illuminated by the lighting means L under one lighting condition by the imaging means C another captured image (hereinafter referred to as an "estimated image") obtained when the imaging means C captures one part D illuminated by the lighting means L under another lighting condition, using the generated learning model 41. The image estimation unit 35 associates the generated estimated image with information identifying the part D and the lighting condition, and stores the result in the storage unit 31 as estimated image data 42.

[0033] Here, the captured image estimation process for generating an estimated image will be described with reference to Fig. 5. First, the learning unit 34 generates a learning model 41 based on the captured images of the part D captured under a plurality of illumination conditions included in the captured image data 40 and the illumination conditions at the time of capturing the images (ST1: learning process). The created learning model 41 is stored in the storage unit 31. Next, the image estimation unit 35 uses the generated learning model 41 to estimate (generate) a captured image (estimated image) obtained when the part D is captured under one illumination condition from the captured image of the part D captured under another illumination condition (ST2: image estimation process). The generated estimated image is stored in the storage unit 31 as estimated image data 42.

[0034] In the example of Figure 5, the image estimation unit 35 uses the learning model 41 to generate an estimated image that would be obtained if part D99 were captured under lighting condition 6 and lighting condition 7 from the captured image of part D99 captured under lighting condition 1 included in the captured image data 40 (ST2), and stores the estimated image in the estimated image data 42.

[0035] In FIG. 4, the input receiving unit 36 ​​displays an illumination condition input screen for inputting illumination conditions on the touch panel 15, and receives an input of illumination conditions when the image estimation unit 35 generates an estimated image.

[0036] 6, an example of the lighting condition input screen 50 displayed on the touch panel 15 will be described. An image display frame 51, an image information display frame 52, a lighting condition input frame 53, an automatic adjustment button 54, an image capture button 55, a verification button 56, and a decision button 57 are displayed on the lighting condition input screen 50. The image display frame 51 displays a captured image of the part D captured by the imaging means C, or an estimated image of the part D generated by the image estimation unit 35. The image information display frame 52 displays "captured image" when the image displayed in the image display frame 51 is an image captured by the imaging means C, and displays "estimated image" when the image is generated by the image estimation unit 35.

[0037] The illumination condition input frame 53 displays an illumination value display section 53a that displays the illumination values ​​set for the reflective illumination section 24, the transmitted illumination section 25, and the side illumination section 26, an increase button 53b that increases the illumination value, and a decrease button 53c that decreases the illumination value. When the operator operates the increase button 53b, the illumination value displayed in the illumination value display section 53a increases by a predetermined amount, and when the operator operates the decrease button 53c, the illumination value decreases by a predetermined amount. The operator operates the increase button 53b or the decrease button 53c to set the illumination value between "0" and "100."

[0038] 6 displays an image captured by the imaging means C while the illumination means L is illuminating the component D99 held by the suction nozzle 9 under illumination condition 1 (reflection: 20, transmission: 0, side: 10). The illumination value display section 53a of the illumination condition input frame 53 displays the illumination value included in the illumination condition 1 at the time of imaging. The image display frame 51 shows the main body Dc of the component D99 and five electrodes Dd formed on the bottom surface of the main body. Under illumination condition 1, the amount of light irradiated from the reflection illumination section 24 and the side illumination section 26 is insufficient, so the main body Dc and electrodes Dd of the component D99 are not clearly visible.

[0039] 7 shows an illumination condition input screen 58 in which the illumination value is changed by operating the increase button 53b or the decrease button 53c on the illumination condition input screen 50 shown in FIG. 6. On the illumination condition input screen 58, the illumination condition is changed to illumination condition 6 (reflection: 70, transmission: 0, side 40). When the increase button 53b or the decrease button 53c is operated on the illumination condition input screen 58 to change the illumination value, the image estimation unit 35 generates an estimated image in which the imaging means C captures the part D99 illuminated by the illumination means L under the set illumination condition. The generated estimated image is displayed in the image display frame 51, and "estimated image" is displayed in the image information display frame 52.

[0040] Since the generation process of an estimated image by the image estimation unit 35 is completed in an extremely short time, when the worker operates the increase button 53b or the decrease button 53c to change the illumination value, an estimated image under the changed illumination conditions is immediately displayed in the image display frame 51. That is, every time the worker operates the increase button 53b or the decrease button 53c, the estimated image displayed in the image display frame 51 is updated to one under the changed illumination conditions. In this way, the touch panel 15 is a display unit that displays another captured image (estimated image) estimated by the image estimation unit 35 based on the input illumination condition 6 (another illumination condition).

[0041] 7, the operator operates the increase button 53b or the decrease button 53c while viewing the estimated image displayed in the image display frame 51 to search for the lighting conditions under which the main body Dc and the electrodes Dd of the component D99 are clearly visible. When the automatic adjustment button 54 is operated, the lighting condition creation unit 37 automatically searches for the lighting conditions (combination of lighting values) that maximize the contrast difference between the main body Dc of the component D99 and the background, and between the main body Dc and the electrodes Dd. The determined lighting values ​​are then displayed in the lighting value display unit 53a, and the estimated image under the determined lighting conditions is displayed in the image display frame 51. Note that the method by which the lighting condition creation unit 37 searches for the lighting conditions that are considered optimal is not limited to the method of maximizing the contrast difference, and various methods can be used.

[0042] 7, when the imaging button 55 is operated, the imaging processing unit 33 scans and images the component D99 held by the suction nozzle 9 with the imaging means C under illumination by the illumination means L under the illumination conditions 6 set on the illumination condition input screen 58. When imaging is completed, the captured image of the component D99 is displayed in the image display frame 51. On the illumination condition input screen 58, the user operates the increase button 53b or the decrease button 53c while viewing the estimated image displayed in the image display frame 51, or operates the automatic adjustment button 54 to input illumination conditions thought to be optimal, and then operates the imaging button 55 to check the captured image of the component D99 actually captured under those illumination conditions, thereby making it possible to efficiently determine whether the illumination conditions are acceptable in a short time.

[0043] Fig. 8 shows an illumination condition input screen 59 in which the verification button 56 is operated on the illumination condition input screen 58 shown in Fig. 7 and the illumination condition suitability determination unit 38 executes the illumination condition suitability determination process. In the illumination condition suitability determination process, it is determined whether or not the position and orientation of the component D99 and the positions of the electrodes Dd can be correctly recognized from an image captured by the imaging means C under illumination condition 6 set in the illumination condition input frame 53 by the illumination means L.

[0044] When the verification button 56 is operated, first, the illumination condition determination unit 38 causes the illumination means L to illuminate the component D99 under the illumination condition 6 set in the illumination condition input frame 53, rotates the component D99 held by the suction nozzle 9 around the Z axis by a predetermined angle (here, 30 degrees), and causes the imaging means C to scan and image the component D99. The captured image is displayed in the image display frame 51, and "Captured Image" is displayed in the image information display frame 52.

[0045] In Fig. 8, the illumination condition suitability determination unit 38 then performs image processing on the captured image to extract the center position of the main body Dc of the component D99 and the center position of the electrode Dd, and determines whether the extracted positions are as expected. If the extracted center positions of the main body Dc and the electrode Dd are within a predetermined range from the expected positions, the illumination condition suitability determination unit 38 determines that the illumination conditions are acceptable. In the captured image of the component D99 displayed in the image display frame 51, cross marks are superimposed on the extracted center positions of the main body Dc and the electrode Dd. In the example of Fig. 8, the illumination conditions are judged to be acceptable, and the judgment result "OK" is displayed in the image display frame 51.

[0046] In this way, the illumination condition suitability determination unit 38 determines whether illumination condition 6 is acceptable based on the captured image captured by the imaging means C while changing the posture (rotating 30 degrees around the Z axis) of the component D99 (one component) illuminated by the illumination means L under illumination condition 6 (another illumination condition). This makes it possible to confirm in advance whether or not the component D99 can be correctly recognized even if the holding posture deviates within a predetermined range from the normal state when the component recognition camera 20 images the component D99 under illumination condition 6 and recognizes the holding posture during component mounting work.

[0047] 8, when the decision button 57 is operated on the lighting condition input screen 59, the lighting condition creation unit 37 creates lighting conditions suitable for the lighting means L to illuminate the component D99 (one component) based on the lighting conditions 6 set in the lighting condition input frame 53. The lighting condition creation unit 37 associates information identifying the component D99 with the created lighting conditions and stores them in the storage unit 31 as lighting condition data 43. That is, the lighting condition creation unit 37 creates lighting conditions suitable for the lighting means L to illuminate the component D99 based on the captured image (estimated image) of D99 (one component) estimated by the image estimation unit 35.

[0048] Specifically, when creating lighting conditions to be used by the lighting means L that illuminates the component D99 held by the suction nozzle 9 of the mounting head 8 imaged by the imaging means C included in the component mounting device M1, the lighting condition creation unit 37 stores the lighting condition 6 set in the lighting condition input frame 53 as it is as the lighting condition data 43. When creating lighting conditions to be used by the component mounting device M2, the lighting condition creation unit 37 creates the lighting conditions based on information regarding the difference in lighting conditions between the component recognition cameras 20 of the component mounting devices M1 and M2, which is included in the inter-device correction data 44, and the lighting condition 6 set in the lighting condition input frame 53.

[0049] For example, if the illumination value of the reflective illumination unit 24 is stored as "-10" in the inter-apparatus correction data 44 as the correction value of the component recognition cameras 20 of the component mounting apparatuses M1 and M2, the illumination condition creation unit 37 creates illumination condition 7 (reflection: 60, transmission: 0, side: 40) in which the illumination value of the reflective illumination unit 24 is corrected to -10 as the illumination condition, and stores it in the storage unit 31 of the component mounting apparatus M2 as the illumination condition data 43. That is, the inter-apparatus correction data 44 includes information for correcting the characteristic difference between the illumination means L and the imaging means C of the component recognition cameras 20 provided in the component mounting apparatuses M1 to M3, which has been acquired in advance. This allows the component mounting apparatus M1 to create the illumination conditions of the component recognition cameras 20 of the other component mounting apparatuses M2 or M3.

[0050] Next, a captured image estimation method for estimating a captured image of the component D will be described with reference to FIG. 7 along the flow of FIG. 9. Hereinafter, the same steps as those in the captured image estimation process described with reference to FIG. 5 are given the same reference numerals, and detailed description will be omitted. First, the imaging processing unit 33 causes the imaging unit C to scan and image the component D illuminated by the illumination unit L under a plurality of illumination conditions, and creates learning data in which information identifying the captured component D and the illumination conditions at the time of imaging are associated with the captured image (ST11: first imaging step). The learning data is stored as captured image data 40. The imaging processing unit 33 also stores the captured image of the component D captured during the component mounting work as learning data.

[0051] Next, a learning step (ST1) is executed, and a learning model 41 is generated based on the learning data included in the captured image data 40. That is, the learning model 41 is generated based on the multiple captured images obtained in the first imaging step (ST11) and the illumination conditions at the time of imaging. Next, the imaging processing unit 33 causes the imaging means C to scan and image the part D99 (one part) illuminated by the illumination means L under one illumination condition 1 (ST12: second imaging step). Next, the input receiving unit 36 ​​inputs an illumination condition 6 (illumination value) by operating the increase button 53b or decrease button 53c on the illumination condition input screen 58 (see FIG. 7) displayed on the touch panel 15 (ST13: illumination condition input step).

[0052] In FIG. 9, when the illumination condition 6 is input, an image estimation step (ST2) is executed. That is, using the generated learning model 41, another captured image (estimated image) obtained when the component D99 illuminated by the illumination means L under another illumination condition 6 is captured by the imaging means C is estimated from the captured image of D99 (one component) obtained by the second imaging step (ST12) (see FIG. 7). Next, until any one of the imaging button 55, the verification button 56, and the decision button 57 is operated on the illumination condition input screen 58 (No in ST14), the process returns to the illumination condition input step (ST13) and the image estimation step (ST2) is repeatedly executed. In this way, the captured image estimation method of the present embodiment can estimate captured images of the component D99 under a plurality of illumination conditions in a short time based on one captured image of the component D99 captured under the illumination condition 1.

[0053] Next, along with the flow of Fig. 10, a method for creating lighting conditions for creating lighting conditions used when the lighting means L illuminates the part D imaged by the imaging means C will be described with reference to Fig. 7 and Fig. 8. Hereinafter, the same steps as those in the captured image estimation process described with reference to Fig. 5 and the captured image estimation method described with reference to Fig. 9 are given the same reference numerals, and detailed descriptions thereof will be omitted. The first imaging step (ST11) to the image estimation step (ST2) are the same as those in the captured image estimation method, and descriptions thereof will be omitted.

[0054] When the image capture button 55 is operated (the "photograph button" is operated in ST21) while the estimated captured image (another captured image) of part D99 in the image estimation step (ST2) is displayed in the image display frame 51 of the lighting condition input screen 58 shown in FIG. 7, the imaging processing unit 33 causes the imaging means C to scan and capture the part D99 held by the suction nozzle 9 under conditions where the lighting means L illuminates under the lighting condition 6 set on the lighting condition input screen 58 (ST22: part imaging step).

[0055] 10, when the verification button 56 is operated on the illumination condition input screen 58 (operating the "Verification button" in ST21), the illumination condition suitability determination unit 38 causes the illumination means L to illuminate the component D99 under illumination condition 6, and rotates the component D99 held by the suction nozzle 9 at a predetermined angle around the Z axis (ST23: component rotation step). Next, the illumination condition suitability determination unit 38 causes the imaging means C to scan and image the component D99 (ST24: third imaging step). That is, in the third imaging step (ST24), the illumination means L illuminates the component D99 (one component) under illumination condition 6 (another illumination condition), and the imaging means C images the component D99 with its posture changed.

[0056] Next, the illumination condition suitability determination unit 38 performs image processing on the captured image to extract the center position of the main body Dc of the component D99 and the center position of the electrode Dd, and determines whether the extracted positions are as expected (ST25: illumination condition suitability determination step) (see FIG. 8). That is, based on the captured image obtained in the third imaging step (ST24), the suitability of illumination condition 6 (another illumination condition) is determined.

[0057] 10, after the component imaging step (ST22) or the illumination condition suitability determination step (ST25), when the decision button 57 is operated on the illumination condition input screen 59 (see FIG. 8) (Yes in ST26), the illumination condition creation unit 37 creates illumination conditions suitable for the illumination means L to illuminate the component D99 based on another captured image of the component D99 (one component) estimated in the image estimation step (ST2) (ST27: illumination condition creation step). Until any one of the imaging button 55, the verification button 56, or the decision button 57 is operated on the illumination condition input screen 59 (No in ST26), the process returns to the illumination condition input step (ST13) and the image estimation step (ST2) is repeatedly executed.

[0058] The illumination condition creation step (ST27) is also executed when the decision button 57 is operated on the illumination condition input screen 58 (operating the "decision button" in ST21). In the illumination condition creation step (ST27), the illumination condition creation unit 37 creates illumination conditions to be used by the component mounting apparatus M1, and can also use the inter-apparatus correction data 44 to create illumination conditions to be used by the other component mounting apparatuses M2 and M3.

[0059] That is, in the illumination condition creation step (ST27), illumination conditions are also created for the illumination means L of the other component mounting apparatuses M2 and M3, which are different from the illumination means L of the component mounting apparatus M1 that illuminated D99 (one component) in the second imaging step (ST12). In this way, the illumination condition creation method of this embodiment can create appropriate illumination conditions in a short time by estimating captured images of the component D99 under multiple illumination conditions based on one captured image of the component D99 captured under illumination condition 1.

[0060] As described above, the component mounting apparatuses M1 to M3 of the present embodiment are an illumination condition creation support device that supports the creation of illumination conditions for the illumination means L that illuminates the component D that is imaged by the imaging means C, and includes a learning unit 34 that generates a learning model 41 based on multiple captured images captured by the imaging means C of the component D illuminated by the illumination means L under multiple illumination conditions and the illumination conditions at the time of capturing images (captured image data 40), and an image estimation unit 35 that estimates another captured image (estimated image) that is obtained when the imaging means C captures a component illuminated by the illumination means L under another illumination condition from one captured image captured by the imaging means C of a component (component D99) illuminated by the illumination means L under one illumination condition, using the generated learning model 41. This allows the captured images of the component D99 under multiple illumination conditions to be acquired (estimated) in a short time, and illumination conditions to be created appropriately.

[0061] Incidentally, the illumination condition creation support device, which includes the learning unit 34 and the image estimation unit 35 and supports the creation of illumination conditions for the illumination means L that illuminates the component D imaged by the imaging means C, is not limited to the component mounting devices M1 to M3, and may be configured by the management computer 3 or the offline camera device 4. For example, in the case of the offline camera device 4, a plurality of captured images are acquired by the imaging means C of the component D illuminated by the illumination means L under a plurality of illumination conditions, using the offline camera included in the offline camera device 4. Also, the inter-apparatus correction data 44 includes information regarding the difference in illumination conditions between the offline camera device and the illumination means L included in the component mounting devices M1 to M3.

[0062] In the case of the management computer 3, the learning unit 34 generates the learning model 41 using the captured images captured by the offline camera device 4 as learning data (teacher data). That is, the learning unit 34 included in the management computer 3 generates the learning model 41 based on the lighting means L included in the offline camera device 4, which is different from the lighting means L included in the component mounting devices M1 to M3, illuminating the component D under multiple lighting conditions, and multiple captured images captured by the imaging means C (offline camera) included in the offline camera device 4, which is different from the imaging means C included in the component mounting devices M1 to M3, and the lighting conditions at the time of capturing the captured images. The learning unit 34 included in the management computer 3 may generate the learning model 41 using the captured images (captured image data 40) captured by the component mounting devices M1 to M3 as learning data (teacher data). [Industrial Applicability]

[0063] The illumination condition creation support device, captured image estimation method, and illumination condition creation method of the present invention have the effect of making it possible to obtain captured images of components under a plurality of illumination conditions in a short period of time, and are useful in the field of mounting components on a board. [Explanation of symbols]

[0064] 3. Management computer (lighting condition creation support device) 4. Offline camera device (lighting condition creation support device) 8 Mounting Head 15 Touch panel (display) 24 Reflective lighting section (lighting section) 25 Transillumination unit (illumination unit) 26 Side lighting section (lighting section) C. Imaging means D, D99 parts L lighting means M1~M3 Component mounting device (lighting condition creation support device)

Claims

1. 1. A lighting condition creation support device that supports creation of lighting conditions for a lighting means that illuminates a part imaged by an imaging means, comprising: a learning unit that generates a learning model based on a plurality of captured images of a component illuminated by a lighting unit under a plurality of lighting conditions and the lighting conditions at the time of capturing the captured images; an image estimation unit that estimates, from one captured image of a part illuminated by the lighting means under one lighting condition and captured by the imaging means, another captured image that is obtained when the imaging means captures the one part illuminated by the lighting means under another lighting condition, using the generated learning model; an input receiving unit that receives an input of the other lighting conditions; a display unit that displays the other captured image estimated by the image estimation unit based on the other lighting condition that is input.

2. 2. The lighting condition creation support device according to claim 1, further comprising: a lighting condition creation unit that creates lighting conditions suitable for a lighting means to illuminate the one part based on the other captured image of the one part estimated by the image estimation unit.

3. 3. The lighting condition creation support device according to claim 1, further comprising an illumination condition suitability determination unit that determines suitability of the other lighting condition based on an image captured by an imaging means of the one component illuminated by the illumination means under the other lighting condition.

4. 4. The lighting condition creation support device according to claim 3, wherein the lighting condition suitability determination unit determines suitability of the other lighting condition based on an image captured by the imaging means while changing an attitude of the one part illuminated by the illumination means under the other lighting condition.

5. 3. The illumination condition creation support device according to claim 2, wherein the illumination condition creation unit creates illumination conditions used for illumination means that illuminates a component held by a mounting head that is imaged by imaging means included in the component mounting device.

6. 6. The lighting condition creation support device according to claim 5, wherein the learning unit generates a learning model based on a plurality of captured images of components illuminated under a plurality of lighting conditions by a lighting means provided in the component mounting device, the captured images being captured by an imaging means provided in the component mounting device, and the lighting conditions at the time of capturing the captured images.

7. The lighting condition creation support device according to claim 5, wherein the learning unit generates a learning model based on a plurality of captured images of components illuminated under a plurality of lighting conditions by a lighting means other than the lighting means provided in the component mounting device, the captured images being captured by an imaging means other than the imaging means provided in the component mounting device, and the lighting conditions at the time of capturing the captured images.

8. The illumination means includes a plurality of illumination units for illuminating the component, The illumination condition creation support device according to claim 1 , wherein the illumination condition is a combination of illumination values ​​of the plurality of illumination units.

9. a first imaging step of imaging the component illuminated by the illumination means under a plurality of illumination conditions with the imaging means; A learning process of generating a learning model based on the plurality of captured images obtained by the first imaging process and the lighting conditions at the time of capturing the images; a second imaging step of imaging the component illuminated by the illumination means under one illumination condition with the imaging means; an image estimation step of estimating, from the captured image of the one component obtained in the second imaging step, another captured image that would be obtained when the imaging means captures an image of the one component illuminated by the illumination means under another lighting condition, using the generated learning model; an input receiving step of receiving an input of the other lighting conditions; a display step of displaying the other captured image estimated in the image estimation step based on the other lighting condition that has been input.

10. The illumination means includes a plurality of illumination units for illuminating the component, The captured image estimating method according to claim 9 , wherein the illumination condition is a combination of illumination values ​​of the plurality of illumination units.

11. 1. A method for creating lighting conditions for use when a lighting unit illuminates a part imaged by an imaging unit, comprising: a first imaging step of imaging the component illuminated by the illumination means under a plurality of illumination conditions with the imaging means; A learning process of generating a learning model based on the plurality of captured images obtained by the first imaging process and the lighting conditions at the time of capturing the images; a second imaging step of imaging, by the imaging means, one component illuminated by the illumination means under one illumination condition; an image estimation step of estimating, from the one captured image obtained in the second imaging step, another captured image that is obtained when the imaging means captures the one component illuminated by the illumination means under another illumination condition, using the learning model generated in the learning step; an illumination condition creating step of creating an illumination condition suitable for illumination of the one component by an illumination means based on the other captured image of the one component estimated by the image estimation step; an input receiving step of receiving an input of the other lighting conditions; a display step of displaying the other captured image estimated in the image estimation step based on the other lighting condition that has been input.

12. a third imaging step of imaging the one component illuminated by the illumination means under the different illumination condition with the imaging means; 12. The illumination condition creating method according to claim 11, further comprising an illumination condition suitability determining step of determining suitability of the other illumination condition based on the captured image obtained in the third imaging step.

13. The illumination condition creating method according to claim 12 , wherein in the third imaging step, an image of the one component is taken with a changed attitude.

14. The illumination means includes a plurality of illumination units for illuminating the component, The illumination condition creating method according to claim 11 , wherein the illumination condition is a combination of illumination values ​​of the plurality of illumination units.

15. 15. The illumination condition creating method according to claim 11, wherein in the illumination condition creating step, an illumination condition for an illumination means different from the illumination means that illuminated the one component in the second imaging step is created.

Images