Focusing detection device, automatic focusing system, and substrate processing system

Abstract

Automatically and accurately determines whether or not the image is in focus. [Solution] The focus determination device 12 includes a sharpness calculation means 510 that calculates the sharpness of a hole image showing a hole processed on a substrate, and a focus determination means 520 that determines whether or not the hole image is in focus based on the sharpness calculated by the sharpness calculation means 510.

Description

Technical Field

[0001] The present invention relates to a focus determination device, an autofocus system, and a substrate processing system.

Background Art

[0002] Conventionally, a printed circuit board hole position and hole diameter inspection machine has been proposed that images a drilled printed circuit board, binarizes the image of the imaged printed circuit board, and calculates the center position and hole diameter of the hole from the position and shape of the hole region (see Patent Document 1). Further, Patent Document 1 describes that the optical focus of the imaging device is adjusted manually or automatically.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, the appearance of the image of the printed circuit board imaged by the imaging device varies widely depending on the type, surface treatment, hole processing method, lighting conditions, etc. of the printed circuit board. For this reason, for example, even when trying to focus the imaging device based on the captured image, it has been difficult to appropriately determine whether the focus is achieved.

[0005] Therefore, an object of the present invention is to automatically and accurately determine whether the focus is achieved.

Means for Solving the Problems

[0006] One aspect of the present invention is a focus determination device including sharpness calculation means for calculating the sharpness of a hole image in which a hole processed in a substrate is imaged, and focus determination means for determining whether the focus of the hole image is achieved based on the sharpness calculated by the sharpness calculation means. [Effects of the Invention]

[0007] According to the present invention, it is possible to automatically and accurately determine whether or not the image is in focus. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of a laser processing system according to the first embodiment. [Figure 2] This is a diagram showing a hole image. [Figure 3] This is a flowchart showing the process for calculating sharpness. [Figure 4] This is a flowchart showing the details of the focusing process. [Figure 5] This graph shows the relationship between the Z-axis position during the focusing process and the measured and calculated sharpness. [Figure 6] This is a diagram illustrating the configuration of the hole recognition system. [Figure 7] This is a conceptual diagram showing an example of ground truth data created for the hole image in Figure 2. [Figure 8] This is a conceptual diagram showing an example of ground truth data created for the hole image in Figure 2. [Figure 9] This is a conceptual diagram of a hole image illustrating how to set up the second sharpness calculation unit. [Figure 10] This is a diagram showing the configuration of a laser processing system according to a second embodiment. [Figure 11] This is a diagram showing the configuration of the hole recognition system according to the second embodiment. [Figure 12] This is a conceptual diagram of a ground truth image showing an example of ground truth data for the vicinity of a hole according to the second embodiment. [Figure 13] This is a conceptual diagram of a hole image illustrating the method for calculating sharpness according to the second embodiment. [Figure 14] This diagram shows the setting method for the sharpness calculation unit, which differs from Figure 13. [Modes for carrying out the invention]

[0009] <First Embodiment> The laser processing system 1000, which is a hole-punching system (substrate processing system) according to the first embodiment of the present invention, will be described below with reference to the drawings. As shown in Figure 1, the laser processing system 1000 according to this embodiment comprises a laser processing device 100 and a hole recognition system 10. The laser processing device 100 is a device that uses a laser to drill holes in a printed circuit board 14 mounted on a positioning table 15 in the XY direction. The laser beam emitted from the laser source 121 is deflected in the XY direction by a galvanoscanner 131 and a galvanoscanner 132, and the laser beam is projected onto the surface of the printed circuit board 14 by an Fθ lens 133 to perform hole processing.

[0010] The galvanoscanners 131 and 132 are devices that deflect the direction of the laser beam incident on the mirror to a desired angle by attaching a mirror to one end of an axis that oscillates using an electromagnetic actuator and providing feedback control of the oscillation angle. The pair of galvanoscanners 131 and 132 are equipped with an oscillation angle detector, and the detected oscillation angle is fed back to the galvanoscanner control device 130, which controls the oscillation angle to match the angle command value. The Fθ lens 133 deflects the incident light ray passing through the front focal point to a ray parallel to the optical axis and at a distance proportional to the incident angle.

[0011] Furthermore, a camera 134 is provided at a position opposite the printed circuit board 14, enabling the capture of images of the holes processed on the printed circuit board. This camera 134 is a fixed-focus camera and, together with the galvanoscanners 131 and 132 and the Fθ lens 133 described above, constitutes the processing head 135. The processing head 135 is movable in the Z direction (hereinafter also referred to as the vertical direction) perpendicular to the upper surface, which is the loading surface of the positioning table 15, by a focusing mechanism (drive mechanism, see Figure 6) 151. As the processing head 135 is driven and positioned, the distance between the focal point of the laser processing optical system and the camera optical system, which are used as the drilling means for drilling holes, and the printed circuit board 14 is adjusted.

[0012] In addition, the camera 134 is configured to be communicable with the hole recognition system 10. The hole recognition system 10 includes a hole recognition device 12 into which an image of the substrate 14 subjected to the drilling process captured by the above-described camera 134 is input as digital image data, and a learning device 11 that learns the learning model mounted on the hole recognition device 12. In the present embodiment, the hole recognition system 10 is configured as an independent device that can transmit and receive data to and from the laser processing device 100, but it may be configured as a part of the laser processing device 100. Also, only the hole recognition device 12 within the hole recognition system 10 may be configured as a part of the laser processing device 100.

[0013] The hole recognition device 12 is configured to be able to execute a focusing determination process that reverses whether or not the image captured by the above-described camera 134 is in focus, a specifying process that specifies the holes drilled from the image data of the substrate 14, and a feature amount extraction process that extracts the feature amounts of the holes. Specifically, the hole recognition device 12 includes a sharpness calculation means 510 and a sharpness evaluation means 520 as sharpness detection means 5 for detecting sharpness, an inference means 3, and a feature amount extraction means 4.

[0014] The sharpness calculation means 510 and the sharpness evaluation means 520, together with the laser processing control device 140 which is the control device of the laser processing device 100 and the laser source control device 120 which is a drive control device that drives the focusing mechanism 151 based on a command from the laser processing control device 140, are configured to be able to execute an autofocusing process for the camera 134 and the laser processing optical system. Note that the above-described hole recognition device 12 and the control devices of the laser processing device 100 (for example, the laser processing control device 140 and the laser processing control device 140) constitute the information processing device of the laser processing system 1000.

[0015] The inference means 3 includes a multi-layer neural network model 222 which is a learned learning model (hereinafter also referred to as a learned model), and is configured to execute an inference process for inferring and specifying the drilled holes from the image of the substrate imaged using this multi-layer neural network model 222. That is, the inference means 3 takes the image data of the substrate after the drilling process as an input, and is configured to output hole region information (in this embodiment, image information specifying the hole region) that specifies the hole region in the input image of the substrate using the multi-layer neural network model 222.

[0016] The hole region information obtained by the inference means 3 is passed to the feature amount extraction means 4. The feature amount extraction means 4 is configured to calculate feature amounts such as the position, diameter, area, etc. of the holes from the hole region information output from the inference means 3. The feature amounts of the holes calculated by the feature amount extraction means 4 are input to a laser processing control device 140 connected to the hole recognition device 12. The laser processing control device 140 corrects various processing conditions such as the processing position error of the holes and the error of the focal distance based on the feedback (feature amounts) regarding the quality of the processed holes from the hole recognition device 12. Then, the laser processing control device 140 controls the positioning table control device 110, the laser source control device 120, and the galvanometer scanner control device 130 according to a predetermined processing condition and a processing program, and executes hole processing on the printed substrate. Note that the feature amounts of the holes may not be directly input to the laser processing control device 140, and may be configured to be notified to the operator by displaying them on a display (not shown). In this case, depending on the judgment of the operator, the hole processing may be executed by reflecting it in the setting of the processing conditions in the laser processing control device 140. <Sharpness calculation process>

[0017] Next, prior to explaining the autofocus process, the sharpness calculation process performed during the autofocus process will be explained based on Figures 2 and 3. Figure 2 is a grayscale hole image 51 of holes processed in the substrate 14, captured by the camera 134. This hole image 51 is either the image data itself captured by the camera 134 converted to grayscale, or a grayscale partial image of an image of an area of ​​the substrate 14 with multiple holes processed.

[0018] The hole image 51 shows the opening 52 of the processed hole near its center, and spatter and molten material deposits 53 resulting from the processing are present around this opening 52. In addition, a sharpness calculation unit 54, composed of four line segments that form the sides of a rectangle, is set up in an area other than the hole (a position outside the hole area, in this embodiment, near the outer edge where there is no hole).

[0019] When the sharpness calculation process is started, the sharpness calculation means 510 first sets a sharpness calculation unit 54 as a calculation area for calculating the sharpness of the hole image 51 (step S70 in Figure 3).

[0020] In addition, when setting the sharpness calculation unit 54, the area around the hole 53 is recognized using a predetermined recognition algorithm, and the sharpness calculation means 510 sets the sharpness calculation unit 54 in an area other than this area around the hole. That is, the area around the hole 53 is an area on the substrate surface where the effects of hole processing, such as sputtering and molten material, may occur, and the sharpness calculation means 510 sets the sharpness calculation unit 54 at a position outside the area around the hole 53 in the hole image.

[0021] This is because, after the autofocusing process, the machining hole is identified by the inference means 3, so the focus position is on the surface of the substrate 14. Furthermore, in the vicinity of the hole where sputter or molten material is attached, such as in the surrounding area, the surface condition of the substrate is not uniform, and there is a risk that an appropriate sharpness cannot be calculated. In this embodiment, the laser machining control device 140 is configured to set the sharpness calculation unit 54 by having it perform a process to set an area a predetermined amount away from the expected machining hole and its surrounding area, based on hole machining instruction data that indicates the position and size of the machining hole supplied to the laser machining control device 140.

[0022] When the sharpness calculation unit 54 is set, the sharpness calculation means 510 selects one of the four sides constituting the sharpness calculation unit 54 as the processing target (step S71, line segment selection process), and sets the starting point of the selected line segment as the starting pixel coordinate (step S72). Then, it performs a pixel value filter calculation from this starting pixel coordinate (step S73). In the pixel value filter calculation, pixels on the line segment are sequentially referenced in x or y coordinates, and the amount of change with the pixel value of neighboring pixels is calculated using a Sobel filter. Due to the differential action of the Sobel filter, the calculated value corresponds to the amount of change in pixel value with respect to adjacent pixels.

[0023] When a filter calculation is performed on the pixel in question, the sharpness calculation means 510 accumulates the absolute values ​​of the pixel value change amount (which has a positive or negative sign) (step S74). Then, it determines whether the pixel for which the filter calculation was performed is the endpoint of the line segment (step S75). If it is not the endpoint (No. in step S75), it updates the coordinates along the line segment and sets the coordinates to be one pixel further toward the endpoint (step S76). Then, it returns to step S73 and repeats the above accumulation process sequentially until it reaches the pixel that is the endpoint of the line segment.

[0024] Then, when the endpoint of the line segment is reached (Yes in step S75), the sharpness calculation means 510 checks whether the above process has been completed for all line segments constituting the four sides of the sharpness calculation unit 54 (step S77). If there are any unfinished line segments (No in step S77), the process returns to step S71 and a new line segment is selected. The above process is then repeated until there are no more unfinished line segments (S71-S76).

[0025] When the calculation of the pixel value change for all the pixels of the line segments constituting the sharpness calculation unit 54 is completed (Yes in step S77), and the absolute values ​​of the change are accumulated, the sharpness calculation means 510 uses this calculated accumulated value as the sharpness value of the target hole image (step S78). In this embodiment, the accumulated absolute value of the Sobel filter output was used as the sharpness value, but other numerical indicators that can evaluate the sharpness of an image may be used.

[0026] <Automatic focusing process> Next, the automatic focusing process will be explained using Figures 4 and 5. Figure 4 is a flowchart of the automatic focusing process using sharpness detection and Z-axis movement. Figure 5 is a graph showing the relationship between the Z-axis position and the measured and calculated sharpness during the focusing process. When executing the automatic focusing process, the laser processing control device 140 drives the focusing mechanism 151 via the laser source control device 120 to position the camera 134 (processing head 135) at its initial position and to image the substrate 14 with the camera 134. Note that this initial position is set so that the focal point position of the camera 134 is different from (higher in this embodiment than) the object to be focused on (for example, the substrate surface).

[0027] When the perforated substrate 14 is imaged by the camera 134 and a hole image (hereinafter also referred to as the initial image) is acquired, the sharpness calculation means 510 performs a sharpness calculation process on this initial image and stores the sharpness of the initial image calculated together with the initial image (step S81 in Figure 4). At this time, since the initial position is considerably far from the focal point position, the sharpness value of the initial image is low.

[0028] Next, the laser processing control device 140 drives the focusing mechanism 151 to lower the camera 134 by a predetermined amount (step S82). This predetermined amount of descent is set to be small compared to the thickness of the insulating layer between the surface copper foil and the inner copper foil that forms the bottom of the hole of the substrate 14, but large enough to produce a significant difference in sharpness.

[0029] When the camera 134 is lowered by a predetermined amount, the laser processing control device 140 causes the camera 134 to image the substrate 14 at this lowered position and acquires a hole image. Once this hole image is acquired, the sharpness calculation means 510 performs a sharpness calculation process on the acquired hole image and calculates the sharpness of the hole image (step S83).

[0030] Once the most recent sharpness (hereinafter referred to as "current sharpness") is calculated, the sharpness evaluation means 520 of the hole recognition device 12 performs an evaluation process to evaluate the calculated sharpness. Specifically, the sharpness evaluation means 520 compares the current sharpness value with the stored sharpness (the sharpness obtained in the previous sharpness calculation process during the same focusing process; for example, the sharpness of the initial image described above; hereinafter referred to as "previous sharpness") and determines whether the sharpness value has decreased (step S84).

[0031] Then, if the sharpness value has not decreased (No. in step S84), the current sharpness value and the corresponding hole image (image of the captured substrate) are stored, and the process returns to step S82 to lower the position of the camera 134 by a predetermined amount again.

[0032] Subsequently, steps S82 to S85 are repeated until the current sharpness value is smaller than the previous sharpness value. When the sharpness value decreases (Yes in step S84), the sharpness evaluation means 520 outputs the stored hole image and terminates the process.

[0033] For example, in the example shown in Figure 5, the sharpness increases from the initial position (1) up to position (4), and then decreases at position (5). Therefore, the hole image (substrate image) captured at position (4) is output to the inference means 3, for example.

[0034] Thus, in this embodiment, the sharpness evaluation means 520 is a focus determination means that determines whether or not the hole image is in focus based on the sharpness calculated by the sharpness calculation means 510. The hole recognition device 12, which includes the sharpness calculation means 510 and the sharpness evaluation means 520, can be said to constitute a focus determination device.

[0035] Furthermore, the hole image captured at position (4) in the example described above can be said to be a first hole image captured when the distance between the camera 134's focal point position and the substrate 14 is a first distance, and the hole image captured at position (5) can be said to be a second hole image captured when the distance between the camera 134's focal point position and the substrate 14 is a second distance that is a predetermined distance different from the first distance.The sharpness evaluation means 520, which is the focus determination means, determines that the first hole image is in focus when the second sharpness of the second hole image is smaller than the first sharpness of the first hole image.As a result, it is possible to output a hole image captured at the position where the sharpness is at its peak (i.e., the position where it is in focus).

[0036] Furthermore, if the laser processing control device 140 subsequently takes images of the substrate or holes processed in the substrate, or performs hole processing on the substrate, it returns the Z-axis position of the processing head 135 to the position where the output image was captured (for example, position (4)) and focuses on the substrate surface. The laser processing control device 140, together with the camera 134, the focusing mechanism 151 configured to change the distance between the camera 134's focal point position and the substrate 14, the hole recognition device 12, etc., constitute an automatic focusing system.

[0037] Furthermore, in the above-described embodiment, a fixed-focus camera is used as the camera 134, and the camera 134 is configured to be movable in the Z-axis direction (focal length direction) by the focusing mechanism 151. However, instead of moving the camera, the focusing mechanism may be a mechanism that raises and lowers the XY table (mounting table) 15 on which the substrate 14 is placed. Alternatively, the camera that images the substrate 14 may be a camera equipped with a lens with a variable focal position. In this case, the focusing process may be performed while changing the focal position (focal length) of the lens, and the mechanism that changes the focal position of the lens becomes the focusing mechanism.

[0038] <Details of the hole detection system> Next, the configuration of the hole recognition system 10 for hole recognition will be explained based on Figure 6. As shown in Figure 6, the learning device 11 of the hole recognition system 10 is a learning means for training a learning model for inferring hole regions in an image from an image of a substrate that has been drilled, and comprises an image accumulation unit 200, a training data creation unit 210, and a learning processing unit 220.

[0039] The image accumulating unit 200 stores numerous digital hole images taken under similar or identical conditions to the images used by the hole recognition device 12 for hole recognition (identification). The data includes images focused on the vicinity of the substrate surface and images focused on the vicinity of the hole bottom. The training data creation unit 210 creates training data for the learning processing unit 220 to train the learning model based on each image accumulated in the image accumulating unit 20.

[0040] Specifically, the training data creation unit 210 includes a ground truth data creation unit 211, a training data conversion processing unit 212, and a data splitting processing unit 213. The ground truth data creation unit 211 is configured to perform a ground truth data creation process for images sequentially acquired from the image accumulation unit 200, creating ground truth data that defines the regions (e.g., location and shape) of holes present in the images. In this embodiment, this ground truth data creation process is performed automatically using a program called an annotation tool or based on human operation.

[0041] Figure 7(a) shows the result of defining the hole 52 using an annotation tool. In the figure, the black circles (hereinafter referred to as markers) are connected in a chain, representing the boundary of the hole defined by polygonal approximation, and the area inside this boundary is the hole. In other words, in Figure 7(a), the boundary of the hole 52 is defined by a point cloud. The markers represent the vertices of the polygon, and the line segments connecting a marker to an adjacent marker represent the sides of the polygon. Here, the polygon 55 is displayed as a graphic, but in actual processing, it is defined as a list of vertex coordinate values. Figure 7(b) shows the opening 52 of the hole defined by a rectangle that circumscribes it. Here, the rectangle 56 is displayed as a graphic, but in actual processing, it is defined by the x and y coordinates of two diagonally opposite vertices, or by the x and y coordinates of one vertex along with its width and height. Depending on the shape of the opening 52, the opening 52 may be defined by a square instead of a rectangle. This is also true when defining the hole bottom 57 (rectangle 59), which will be described later.

[0042] Figure 8(a) is a cross-sectional view of the substrate surface including the processed hole (corresponding to the cross-sectional view of the hole from the left side in Figure 8(b)). As shown in Figure 8(a), this substrate 14 has an insulating layer 62 below the surface copper foil 61 (left side in the figure), and an inner layer copper foil 63 is provided below that. The hole in the surface copper foil 61 is the opening 52, and the part of the inner layer copper foil that is visible from the surface after the insulating layer has been removed is the hole bottom 57. Figure 8(b) shows the result of defining the hole bottom 57 in the hole image using an annotation tool. In Figure 8(b), the hole bottom in the hole image is defined by a polygon 58, and the part enclosed by the polygon 58 in the figure defines the hole bottom. Also, in Figure 8(c), the hole bottom 57 is defined by a rectangle 59 that circumscribes it.

[0043] When performing hole recognition using semantic segmentation, polygons 55 and 58 mentioned above are used as ground truth data, while when performing hole recognition using object detection, rectangles 56 and 59 are used as ground truth data. The ground truth data in Figures 7(a) and 8(b) are used for training hole recognition using semantic segmentation, and the ground truth data in Figures 7(b) and 8(c) are used for training hole recognition using object detection.

[0044] The training data processing unit 212 creates training data by associating the correct answer data created by the correct answer data creation unit 211 with the corresponding image data (digital hole image). The training data is then compiled from many such pairs of original image data and correct answer data. In other words, the training data consists of multiple sets of datasets, each containing image data of a substrate with processed holes and correct answer data that defines the hole area in the image data of the substrate with processed holes.

[0045] The data splitting processing unit 213 splits the training data generated by the above-mentioned training data processing unit 212 into training data used to train the learning model in "supervised learning" (hereinafter also referred to as training training data) and validation data used to verify the learning effect (hereinafter also referred to as validation training data).

[0046] The training data creation unit 210 and the learning processing unit 220 are connected in a communication manner, and the training data (training training data and verification training data) created by the training data creation unit 210 is communicated to the learning processing unit 220. The learning processing unit 220 includes a multilayer neural network model 221, which is an example of a learning model in this embodiment, and the training training data acquired by the learning processing unit 220 is input to the multilayer neural network model 221. When the training training data is input, the multilayer neural network model 221 performs forward processing to calculate the output.

[0047] The calculated output is compared with the correct data of the training data, and the parameters of the multilayer neural network model 221 are adjusted by backpropagation processing 223 using intermediate variables in the calculation process to reduce the error in the above comparison. The learning processing unit 220 then updates the input training data and repeats the above process to optimize the parameters of the multilayer neural network model 221, and completes the learning when the evaluation function value of the error has decreased sufficiently. This makes it possible to create a multilayer neural network model 221 that identifies the region where the hole opening exists using polygons or rectangles for hole images that are similar to or identical to the given integrated hole image 200. For example, a learning model such as a single-shot multiple-box detector or a pyramidal image analysis network can be used as the multilayer neural network model 221. Furthermore, since the integrated hole image 200 includes images of the hole bottom as described above, this trained multilayer neural network model 221 can distinguish and recognize whether it is an image of the top of the hole near the substrate surface or an image of the hole bottom.

[0048] The hole recognition device 12 is configured to communicate with the learning device 11, and the trained multilayer neural network model 222, learned by the learning device 11, is introduced into the inference means 3. The inference means 3 performs an inference process to recognize holes by inputting the trained multilayer neural network model 222, and obtains hole position data and hole shape data as inference results. The feature extraction means 4 calculates feature quantities such as hole position, hole diameter, and hole area based on the data obtained by the inference means 3. These calculated feature quantities are displayed on the display device 235 connected to the hole recognition device 12 and are also input to the laser processing control device 140.

[0049] Thus, in this embodiment, since the focus of the image is determined based on the sharpness of the hole image, it is possible to accurately and automatically determine whether the hole image is in focus at the desired position, even if the hole image appears in a wide variety of ways depending on the condition of the substrate surface, the processing condition of the hole, and the imaging conditions. For this reason, the camera 134 can be focused with high accuracy.

[0050] In particular, in this embodiment, since the sharpness calculation unit 54 is set to a location other than the hole region 52 on the image, it is possible to focus accurately on the surface of the substrate 14.

[0051] Furthermore, since the input data is an image captured in such a focused state, the multilayer neural network model 222 of the inference means 3 can accurately recognize and identify holes in the image. As a result, the accuracy of the extracted hole features is also improved. These features can then be used to improve the accuracy of determining the quality of the processed holes and setting the processing conditions of the laser processing device 100.

[0052] In other words, the hole recognition device 12 and the laser processing device 100 incorporating it automatically perform precise focusing, enabling stable hole recognition that is resistant to changes in the surface treatment and structure of the substrate, as well as changes in laser processing conditions and illumination conditions, and is less affected by dirt, scratches, sputter, and their shadows on the substrate surface, thereby achieving highly reliable and high-precision drilling.

[0053] <Recognition of the bottom of the hole> In the above description, we explained the process of focusing on the substrate surface and recognizing the holes (openings 52) in the substrate 14. However, the laser processing system 1000 according to this embodiment is also configured to focus on the bottom of the hole 57 and recognize this bottom of the hole 57. The process for recognizing the bottom of the hole 57 will be described below.

[0054] Figure 9 shows an image when the second sharpness calculation unit 66 is set for the hole image 51 when the focus is on the bottom of the hole 57. Within the hole image 51, the opening 52 of the machined hole can be identified, for example, as a set of pixels or a bounding box. When the opening 52 is identified as a set of pixels or a bounding box, the sharpness calculation means 510 defines a rectangle 65 that circumscribes this region, using the maximum and minimum coordinates if it is a set of pixels, or the bounding box itself if it is a bounding box.

[0055] The sharpness calculation means 510 then sets a second rectangle 66 near the center of the position of the rectangle 65, using the dimensions of the rectangle 65 as a reference, with dimensions that fit within the hole bottom 57, and each side of this rectangle 66 becomes a second sharpness calculation unit 66. Once the second sharpness calculation unit 66 is set, an image focused on the hole bottom 57 can be obtained by performing a focusing process in the same manner as described in Figures 3 and 4.

[0056] In other words, the focusing process starts with the Z-axis positioned at an initial position higher than the focal point relative to the hole bottom (hole bottom surface), and images of the substrate 14 are captured while changing the distance between the focal point and the hole bottom 57 by a predetermined distance. When the sharpness decreases, the state before that point is set to the state where focusing on the hole bottom surface 57 is completed, and an image with the focal point aligned to the hole bottom 57, which has been stored, can be obtained.

[0057] As described above, the multilayer neural network model 222 of the inference means 3 is also capable of recognizing the hole bottom 57. Therefore, by inputting a hole image with the focus point aligned to the hole bottom 57 into the multilayer neural network model 222, the hole bottom 57 can be inferred and identified. The feature extraction means 4 can extract feature quantities of the hole bottom 57 based on the hole bottom information (in this embodiment, image information in which the hole bottom 57 is identified) output from the inference means 3. Based on these feature quantities, for example, the hole recognition device 12 can function as a quality determination device to determine the quality of the processed hole and provide feedback to the laser processing control device 140.

[0058] <Second Embodiment> Next, a second embodiment will be described. This embodiment differs from the first embodiment in that, when setting the sharpness calculation unit for the hole image, it uses a trained multilayer neural network model to recognize holes in the hole image. In the following description, only the differences from the first embodiment will be described, and similar components will be denoted by the same reference numerals and their descriptions will be omitted.

[0059] Figure 10 is an overall configuration diagram of the laser processing system 1000 according to the second embodiment. Unlike the first embodiment, the laser processing system 1000 in this embodiment includes two models 222a and 222b as multilayer neural network models for the hole recognition device 12. In addition, the results of the inference processing are also input to the sharpness calculation means 510.

[0060] The multilayer neural network model 222a described above is an object detection model trained using training data created in the manner described later, targeting integrated hole images that also include images that are out of focus. It is used when setting the sharpness calculation unit 54 (see Figure 13), which is performed prior to the sharpness calculation process. The training method and application to the hole recognition device of the other multilayer neural network model 222b are the same as those of the multilayer neural network model 222 in the first embodiment described above, and it is used for recognizing hole openings and hole bottoms.

[0061] Figure 11 is an overall configuration diagram of the hole recognition system 10 according to this embodiment. Reference numerals with "a" appended to the end indicate parts added in this embodiment. Reference numerals with "b" appended to the end indicate the same configuration as those described in the first embodiment.

[0062] In this embodiment, the hole recognition device 12 includes inference means 3a, 3b, sharpness detection means 5a, and feature extraction means 4b. The inference means 3a is used when setting the sharpness calculation unit 54, which is performed prior to the sharpness calculation process. Otherwise, the inference means 3b and feature extraction means 4b are used. That is, in this embodiment, the inference means 3a includes a trained model 221a that has been trained to infer the area around the hole, and functions as a means for identifying the hole area using this trained model 221a. These two systems, the inference means 3a and sharpness detection means 5a, and the inference means 3b and feature extraction means 4b, are switched by processing selection functions 236 and 237.

[0063] Furthermore, the image integrator 200a integrates hole images captured with a focal plane set at the substrate surface and at a different location from the substrate surface (for example, above the substrate surface). The hole images in the image integrator 200a are sequentially acquired during the training data creation process, and first, the ground truth data creation processing unit 211a creates ground truth data that defines the location and shape of the area near the holes present in the image.

[0064] Figure 12 is a conceptual diagram of a ground truth image showing an example of ground truth data created for a hole image 51, where the vicinity of the hole opening 52 is defined by a rectangle 65 that circumscribes it. This differs from Figure 7(b) of the first embodiment in that the rectangle 65 defines not the hole opening 52, but the vicinity of the hole including the spatter / molten material deposit area 53 resulting from processing around the opening. In this embodiment, object detection is used as the object recognition algorithm.

[0065] The learning process flow by the learning processing unit 220a, which creates the multilayer neural network model 222a through learning based on the above training data, is the same as the flow described in Figure 6. Note that the learning model used before the multilayer neural network model 222a is trained can be the same as or different from the multilayer neural network model 222b.

[0066] The inference means 3a of the hole recognition device 12 is equipped with the trained multilayer neural network model 222a. When setting the sharpness detection unit, which is performed prior to the sharpness calculation process, the hole recognition device 12 connects the processing selection functions 236 and 237 to the inference means 3a.

[0067] As described above, the inference means 3a inputs digital image data including the hole into a trained multilayer neural network model 222a to perform recognition of the area near the hole, and obtains a rectangular region 65 defining the area near the hole as an inference result. This result is passed to the sharpness calculation means 510 and used when setting the sharpness calculation unit 54.

[0068] Figure 13 is a conceptual diagram of a hole image taken by a camera 134 of a machined hole on a substrate 14, and a diagram illustrating the method of calculating the sharpness of the image. In Figure 13, the hole image 51 shows two machined hole openings 52, 52, and spatter / molten material deposits 53, 53 resulting from the machining process are present around these openings 52, 52. In addition, a sharpness calculation unit 54 is provided near the outer edge where there are no holes in the hole image 51.

[0069] The sharpness calculation unit 54 is composed of four line segments, which are the sides of a rectangle, but the inference means 3a detects two hole vicinity areas 64. Here, the rectangles of the hole vicinity areas 64 overlap the sharpness calculation unit 54 in the left and lower right sections. These areas are excluded from the sharpness calculation unit 54, and the line segments that make up the remaining sides are then used to create the sharpness calculation unit 54 again. The sharpness on each line segment is evaluated using the sharpness calculation process described with reference to Figure 3, targeting the grayscaled hole image 51. Furthermore, the autofocus process using this sharpness evaluation can be performed by the process described with reference to Figures 4 and 5.

[0070] In addition, as shown in Figure 14, for example, the sharpness calculation unit 54 may be set up in a rectangular shape, avoiding the vicinity 64 of the two holes. In this case, the cumulative value described above can be calculated over the entire circumference of the rectangle constituting the sharpness calculation unit 54. In the embodiment described above, the shape of the sharpness calculation unit 54 was set to a rectangle, but the shape of the sharpness calculation unit 54 is not limited to this, and may be a circle, a cross shape, a grid shape, or other shapes. In these cases, the cumulative value described above is calculated over the circumference or line segments constituting these shapes.

[0071] In this way, by using the multilayer neural network model 222a, it is possible to accurately recognize the areas near the holes 64, which have different states depending on the processing state of each hole and the state of the substrate, and to appropriately set the sharpness calculation unit 54.

[0072] In the embodiments described above, an example was shown in which a laser processing device was used as the processing device for drilling holes in the substrate. However, the present invention is not limited to this, and can also be applied to a device that drills holes using a drill. Furthermore, the contents described in the embodiments described above may be combined in any way. [Explanation of symbols]

[0073] 12: Focus determination device (hole recognition device) / 510: Sharpness calculation means / 520: Focus determination means (sharpness evaluation means)

Claims

1. A sharpness calculation means for calculating the sharpness of a hole image showing holes processed on a circuit board, The system includes a focus determination means that determines whether or not the hole image is in focus based on the sharpness calculated by the sharpness calculation means, A focus determination device characterized by the following features.

2. When a hole image captured at a first distance between the camera's focal point and the substrate is defined as the first hole image, a hole image captured at a second distance different by a predetermined distance from the first distance is defined as the second hole image, the sharpness of the first hole image calculated by the sharpness calculation means is defined as the first sharpness, and the sharpness of the second hole image calculated by the sharpness calculation means is defined as the second sharpness, The focus determination means determines that the first hole image is in focus when the second sharpness is less than the first sharpness. The focusing determination device according to feature 1.

3. The sharpness calculation means sets a sharpness calculation unit at a position outside the hole region in the hole image where the hole exists, and calculates the sharpness of the sharpness calculation unit as the sharpness of the hole image. The focusing determination device according to feature 1.

4. The sharpness calculation means sets the sharpness calculation unit at a position outside the area around the hole where the hole processing has had an effect on the surface of the substrate in the hole image. The focus determination device according to claim 3.

5. The system includes a trained model that has been trained to infer the region surrounding the hole, and a means for identifying the hole region using the trained model. The sharpness calculation means sets the sharpness calculation unit at a position outside the area around the hole identified by the identification means in the hole image. The focusing determination device according to feature 4.

6. The sharpness calculation means sets a sharpness calculation unit inside the hole in the hole image and calculates the sharpness of the sharpness calculation unit as the sharpness of the hole image. The focusing determination device according to feature 1.

7. Camera and, A focusing mechanism configured to change the distance between the camera's focal point and the substrate, The device comprises the focusing determination device described in claim 1, Until the focus determination device determines that the image captured by the camera is in focus, the distance between the camera's focal point and the substrate is varied by a predetermined distance, and the camera captures images of the substrate that has been drilled. An automatic focusing system characterized by the following features.

8. A drilling means for drilling holes in a circuit board, The system comprises the automatic focusing system described in claim 7, A substrate processing system characterized by the following features.

Images