Inspection device

Abstract

Provided is an inspection device capable of obtaining accurate coordinates of a chip provided on a substrate. Specifically, this inspection device comprises: a stage on which a target substrate is placed; a camera that images the target substrate to obtain a captured image; and a controller that calculates a target measurement value of target coordinates of a chip with respect to a target origin of the target substrate on the basis of the distance between an image reference position and the chip in the direction in which pixels are arranged and a relative movement set value between the stage and the camera. The controller corrects the target measurement value by using a projective transformation matrix to calculate a target correction value. The controller calculates an alignment measurement value of the alignment coordinates with respect to an alignment origin of an alignment substrate on the basis of the distance between the image reference position and an alignment mark in the direction in which the pixels are arranged, in a state in which the alignment origin is aligned with the image reference position. The controller calculates a parameter of the projective transformation matrix on the basis of the alignment measurement value and a known alignment actual measurement value.

Description

Inspection Equipment

[0001] The present disclosure relates to an inspection apparatus.

[0002] 2. Description of the Related Art As disclosed in Patent Document 1, an inspection device is known that inspects a board based on an image obtained by capturing an image of the board with a camera.

[0003] Japanese Patent Application Publication No. 7-325046

[0004] One type of inspection device involves placing a substrate on a stage on which multiple chips are arranged, capturing an image of the substrate placed on the stage with a camera, and measuring the coordinates of each chip on the substrate based on the captured image.

[0005] However, in such an inspection device, the measured coordinate values ​​of each chip on the substrate obtained based on the captured image may deviate from the true values ​​due to various error factors.

[0006] The present disclosure has been made in view of the above points, and an object thereof is to provide an inspection apparatus that can obtain accurate coordinates of a chip provided on a substrate.

[0007] The inspection apparatus according to the present disclosure includes a stage on which a target substrate having a plurality of chips arranged side by side is placed, a camera that images the target substrate placed on the stage to obtain a captured image divided into a plurality of pixels, and a controller that calculates target measurement values ​​of target coordinates of the chip relative to a target origin provided on the target substrate based on a distance in a direction in which the pixels are arranged between an image reference position in the captured image and the chip shown in the captured image, and a relative movement setting value between the stage and the camera, and the controller corrects the target measurement values ​​related to the target coordinates using a projection transformation matrix to calculate target correction values ​​related to the target coordinates, and an alignment substrate corresponding to the target substrate has a plurality of chips arranged side by side. a plurality of alignment marks corresponding to the chips are provided, and an alignment origin corresponding to the target origin is provided on the alignment substrate, and the controller calculates alignment measurement values ​​of the alignment coordinates of the alignment marks relative to the alignment origin based on the distance in the pixel alignment direction between the image reference position and the alignment mark shown in the captured image when the alignment origin is aligned with the image reference position, the controller acquires known alignment measurement values ​​of the alignment coordinates, and the controller calculates parameters of the projection transformation matrix based on the alignment measurement values ​​and the alignment measurement values.

[0008] In the inspection device according to the present disclosure, the target measurement values ​​of the target coordinates of the chip on the target substrate obtained based on the captured image may deviate from the true values ​​due to various error factors.

[0009] Therefore, in the inspection device according to the present disclosure, a projective transformation matrix is ​​used to convert the target measurement value into a target correction value, thereby bringing the value closer to the true value.

[0010] In particular, the inspection device according to the present disclosure uses the alignment mark of the alignment substrate with the alignment origin aligned with the image reference position to determine the parameters of the projective transformation matrix. The parameters of the projective transformation matrix are then calculated based on the alignment measurement values ​​and known actual alignment measurement values. This allows for a highly accurate projective transformation matrix to be obtained.

[0011] As described above, it is possible to provide an inspection device that can obtain accurate target coordinates of a chip provided on a target substrate.

[0012] In one embodiment, the target substrate has a plurality of chips arranged side by side in a first direction and a second direction intersecting the first direction, and the controller calculates first object measurement values ​​of first object coordinates of the chip in the first direction relative to the target origin based on a horizontal distance between the image reference position and the chip shown in the captured image in the horizontal direction in which the pixels are aligned, and a first relative movement set value between the stage and the camera in the first direction, and the controller calculates second object measurement values ​​of second object coordinates of the chip in the second direction relative to the target origin based on a vertical distance between the image reference position and the chip shown in the captured image in the vertical direction in which the pixels are aligned, and a second relative movement set value between the stage and the camera in the second direction, and the controller corrects the first object measurement values ​​related to the first object coordinates by the projective transformation matrix to calculate first object correction values ​​related to the first object coordinates, and corrects the second object measurement values ​​related to the second object coordinates to calculate second object correction values ​​related to the second object coordinates, and the controller a first alignment measurement value of a first alignment coordinate of the alignment mark in the first direction relative to the alignment origin based on the horizontal distance in the horizontal direction in which the pixels are aligned between the image reference position and the alignment mark shown in the captured image when the alignment origin is aligned with the image reference position, and the controller calculates a first alignment measurement value of a first alignment coordinate of the alignment mark in the first direction relative to the alignment origin based on the horizontal distance in the horizontal direction in which the pixels are aligned between the image reference position and the alignment mark shown in the captured image when the alignment origin is aligned with the image reference position; Based on the distance, a second alignment measurement value of a second alignment coordinate in the second direction of the alignment mark relative to the alignment origin is calculated, and the controller acquires a known first alignment measurement value related to the first alignment coordinate and a known second alignment measurement value related to the second alignment coordinate, and the controller calculates the parameters of the projection transformation matrix based on the first alignment measurement value, the first alignment measurement value, the second alignment measurement value, and the second alignment measurement value.

[0013] With this configuration, even when the stage and the camera move relatively in the first and second directions, accurate first and second target coordinates of the chip provided on the target substrate can be obtained.

[0014] In one embodiment, the controller uses the homography matrix to correct for shifts in the object measurements due to distortion of the camera to obtain the corrected object values.

[0015] According to this configuration, even if there is an error due to camera distortion, the target measurement value can be converted into the target correction value using the projective transformation matrix, and can be made closer to the true value.

[0016] In addition, the inspection device according to the present disclosure includes a stage on which a target substrate having a plurality of chips arranged side by side is placed, a camera that captures an image of the target substrate placed on the stage to obtain an image, and a controller that calculates measurement values ​​of target coordinates of the chip based on the image, and the controller corrects the measurement values ​​of the target coordinates and calculates correction values ​​for the target coordinates using a machine learning model that uses the measurement values ​​of the target coordinates and actual measurement values ​​of the target coordinates as training data.

[0017] In the inspection device according to the present disclosure, the measurement values ​​relating to the target coordinates of the chip on the target substrate obtained based on the captured image may deviate from the true values ​​due to various error factors.

[0018] Therefore, in the inspection device according to the present disclosure, a machine learning model is used to convert the measurement value into a corrected value, which approaches the true value.

[0019] In particular, in the inspection device according to the present disclosure, the measured values ​​and actual measured values ​​are used as training data in the machine learning model, thereby making it possible to obtain a highly accurate machine learning model.

[0020] As described above, it is possible to provide an inspection device that can obtain accurate target coordinates of a chip provided on a target substrate.

[0021] In one embodiment, the captured image is divided into a plurality of pixels, and the controller calculates the measurement value relating to the target coordinates of the chip relative to a target origin provided on the target substrate based on the distance in the direction in which the pixels are aligned between the image reference position in the captured image and the chip reflected in the captured image, and the relative movement setting value between the stage and the camera.

[0022] With this configuration, measurement values ​​relating to the target coordinates of the chip relative to the target origin provided on the target substrate can be easily calculated based on the distance between the image reference position and the chip in the direction in which the pixels are aligned and the relative movement setting value between the stage and the camera.

[0023] In one embodiment, an alignment substrate corresponding to the target substrate is provided with a plurality of alignment marks corresponding to a plurality of the chips, and an alignment origin corresponding to the target origin is provided on the alignment substrate, and the controller calculates alignment measurement values ​​of the alignment coordinates of the alignment marks based on the distance in the direction in which the pixels are aligned between the image reference position and the alignment mark reflected in the captured image when the alignment origin is aligned with the image reference position, and the controller acquires actual alignment measurement values ​​related to the alignment coordinates, and the machine learning model uses the alignment measurement values ​​related to the alignment coordinates as training data as the measurement values ​​related to the target coordinates, and uses the actual alignment measurement values ​​related to the alignment coordinates as training data as the actual measurement values ​​related to the target coordinates.

[0024] With this configuration, the data necessary for training the machine learning model can be easily obtained.

[0025] According to the present disclosure, it is possible to provide an inspection device that can obtain accurate coordinates of a chip provided on a substrate.

[0026] FIG. 1 shows an inspection device. FIG. 2 shows a target substrate. FIG. 3 shows an image obtained by a camera. FIG. 4 shows measurement of target coordinates related to a chip. FIG. 5 shows an image of error factors. FIG. 6 shows an alignment substrate. FIG. 7 shows alignment measurement values ​​related to alignment marks. FIG. 8 shows distortion aberration of the camera lens. FIG. 9 shows a machine learning model.

[0027] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present disclosure, its applications, or its uses.

[0028] <First embodiment> (Inspection device) An inspection device 1 according to the first embodiment will be described. FIG. 1 shows the inspection device 1. The inspection device 1 includes a rail 10, a stage 20, a camera 30, and a controller 40. The rail 10 includes a first rail 11 and a second rail 12. The first rail 11 and the second rail 12 extend horizontally. The first rail 11 extends in a front-rear direction x, which is a first direction among the horizontal directions. The second rail 12 extends in a left-right direction y, which is a second direction among the horizontal directions. The front-rear direction x and the left-right direction y intersect (specifically, are perpendicular to) each other. The first rail 11 and the second rail 12 intersect at a reference position.

[0029] The stage 20 is formed, for example, in the shape of a plate with its thickness direction being the up-down direction z. The up-down direction z is perpendicular to the horizontal direction. The up-down direction z is also the vertical direction. The stage 20 extends in the horizontal direction.

[0030] The stage 20 is placed on the upper surface of the rail 10. More specifically, the stage 20 is placed on the upper surface of the first rail 11 and the upper surface of the second rail 12. The stage 20 is moved along the rail 10 by an actuator (not shown). More specifically, the stage 20 moves in the forward / backward direction x along the first rail 11. The stage 20 moves in the left / right direction y along the second rail 12.

[0031] The camera 30 is disposed above the stage 20. The camera 30 and the stage 20 are spaced apart from each other in the vertical direction z. The imaging unit of the camera 30 faces the upper surface of the stage 20. The imaging axis of the camera 30 is perpendicular to the upper surface (horizontal plane) of the stage 20. The camera 30 is fixed by a bracket or the like (not shown). The camera 30 does not move along with the stage 20. Details of the camera 30 will be described later.

[0032] The controller 40 is built into the main body of the inspection device 1. The controller 40 includes, for example, a microcomputer mounted on a control board and a memory device that stores software for operating the microcomputer. The controller 40 controls the actuator to move the stage 20 along the rails 10. The controller 40 performs the calculation processing described below.

[0033] 2 shows the target substrate 50. The target substrate 50 is a substrate to be inspected by the inspection device 1. The target substrate 50 is, for example, a semiconductor substrate. The target substrate 50 has, for example, a rectangular plate shape.

[0034] The target substrate 50 is placed on the upper surface of the stage 20. As the stage 20 moves along the rails 10, the target substrate 50 moves in the front-back direction x and the left-right direction y.

[0035] A plurality of chips 51 are arranged side by side on the target substrate 50. When the target substrate 50 is placed on the stage 20, the plurality of chips 51 are arranged in a matrix on the target substrate 50 in the front-rear direction x and the left-right direction y. The plurality of chips 51 are arranged at equal intervals in the front-rear direction x. The plurality of chips 51 are arranged at equal intervals in the left-right direction y.

[0036] The chip 51 is, for example, an integrated circuit. For ease of understanding, the chip 51 is shown in Fig. 2 as a simple rectangle.

[0037] The target substrate 50 is provided with a target origin 52. The target origin 52 is configured by a corner of the target substrate 50, for example.

[0038] The coordinate of any chip 51 in the front-to-back direction x relative to (based on) the target origin 52 is defined as the first target coordinate X. The distance in the front-to-back direction x from the target origin 52 to the center of any chip 51 is defined as the first target true value Xr. The coordinate of any chip 51 in the left-to-right direction y relative to (based on) the target origin 52 is defined as the second target coordinate Y. The distance in the left-to-right direction y from the target origin 52 to the center of any chip 51 is defined as the second target true value Yr.

[0039] The user does not know the first target true value Xr and the second target true value Yr.

[0040] For simplicity, FIG. 2 illustrates the first object coordinate X and the second object coordinate Y for only one chip 51, but in reality, there are first object coordinate X and second object coordinate Y for all chips 51.

[0041] (Captured Image) Fig. 3 shows a captured image 70 obtained by capturing an image of a chip 51 on a target substrate 50 with the camera 30. For simplicity, Fig. 3 shows only one chip 51. As described above, the camera 30 does not move along with the stage 20. The target substrate 50, which is placed on the upper surface of the stage 20 and moves in the forward / backward direction x and the left / right direction y, passes below the camera 30. The camera 30 captures an image of the chip 51 on the target substrate 50 placed on the stage 20 to obtain the captured image 70.

[0042] The field of view of the captured image 70 is, for example, rectangular. The captured image 70 is partitioned into a plurality of pixels 71. The plurality of pixels 71 are arranged in a matrix in the horizontal direction t and the vertical direction v. The plurality of pixels 71 are arranged at equal intervals in the horizontal direction t. The plurality of pixels 71 are arranged at equal intervals in the vertical direction v. If there are no error factors described below, the horizontal direction t in which the pixels 71 are arranged coincides with the front-to-back direction x, and the vertical direction v in which the pixels 71 are arranged coincides with the left-to-right direction y.

[0043] The captured image 70 has an image reference position 72. The image reference position 72 is, for example, at the center of the captured image 70.

[0044] 3 shows a horizontal distance T and a vertical distance V. The horizontal distance T is the distance between the image reference position 72 in the captured image 70 and the chip 51 of the target substrate 50 shown in the captured image 70 in the horizontal direction t in which the pixels 71 are arranged. The vertical distance V is the distance between the image reference position 72 in the captured image 70 and the chip 51 of the target substrate 50 shown in the captured image 70 in the vertical direction v in which the pixels 71 are arranged.

[0045] 4 shows the measurement of a first object coordinate X and a second object coordinate Y associated with a chip 51 provided on a target substrate 50. The target substrate 50 placed on the stage 20 moves in the front-to-back direction x and the left-to-right direction y. The camera 30 does not move along with the stage 20. The camera 30 captures an image of the chip 51 of the target substrate 50 placed on the stage 20 and moving in the front-to-back direction x and the left-to-right direction y to obtain a captured image 70. The controller 40 measures the first object coordinate X and the second object coordinate Y associated with the chip 51 on the target substrate 50 using the following method.

[0046] Data of the captured image 70 obtained by the camera 30 is input to the controller 40. The controller 40 calculates the horizontal distance T between the image reference position 72 and the chip 51 based on the captured image 70. The horizontal distance T is found based on the number of pixels 71 aligned in the horizontal direction t between the image reference position 72 and the chip 51 and the magnification of the captured image 70 in the horizontal direction t.

[0047] The controller 40 acquires a first movement setting value Lx relative to the stage 20 and the camera 30 in the forward / backward direction x. The first movement setting value Lx is a setting value for the amount of relative movement between the stage 20 and the camera 30 in the forward / backward direction x. In this example, since the stage 20 is movable and the camera 30 is fixed, the first movement setting value Lx is equal to the setting value for the amount of movement of the stage 20 in the forward / backward direction x. For example, the first movement setting value Lx is an input value that a user inputs to the controller 40 in order to move the stage 20 in the forward / backward direction x.

[0048] Here, when the target substrate 50 placed on the stage 20 is in an initial position (see the two-dot chain line in FIG. 4 ) where it has not moved, the target origin 52 of the target substrate 50 is aligned so as to overlap with the image reference position 72 of the captured image 70. If there are no error factors, which will be described later, the first movement set value Lx coincides with the actual movement amount of the target origin 52 in the forward / backward direction x relative to the image reference position 72.

[0049] As described above, the first object coordinate X is the coordinate of the tip 51 in the forward / backward direction x relative to (based on) the object origin 52. The controller 40 calculates the first object measurement value Xm of the first object coordinate X based on the lateral distance T and the first movement set value Lx.

[0050] Specifically, the first target measurement value Xm is obtained by adding the lateral distance T and the first movement set value Lx (Xm=T+Lx).

[0051] Data of the captured image 70 obtained by the camera 30 is input to the controller 40. The controller 40 calculates the vertical distance V between the image reference position 72 and the chip 51 based on the captured image 70. The vertical distance V is found based on the number of pixels 71 aligned in the vertical direction v between the image reference position 72 and the chip 51 and the magnification of the captured image 70 in the vertical direction v.

[0052] The controller 40 acquires a second movement setting value Ly relative to the stage 20 and the camera 30 in the left-right direction y. The second movement setting value Ly is a setting value for the amount of relative movement in the left-right direction y between the stage 20 and the camera 30. In this example, since the stage 20 is movable and the camera 30 is fixed, the second movement setting value Ly is equal to the setting value for the amount of movement of the stage 20 in the left-right direction y. For example, the second movement setting value Ly is an input value that the user inputs to the controller 40 in order to move the stage 20 in the left-right direction y.

[0053] As described above, when the target substrate 50 placed on the stage 20 is in an initial position (see the two-dot chain line in FIG. 4 ) where it has not moved, the target origin 52 of the target substrate 50 is aligned so as to overlap with the image reference position 72 of the captured image 70. If there are no error factors, which will be described later, the second movement set value Ly coincides with the actual amount of movement of the target origin 52 in the left-right direction y relative to the image reference position 72.

[0054] As described above, the second object coordinate Y is the coordinate of the tip 51 in the left-right direction y relative to (based on) the object origin 52. The controller 40 calculates the second object measurement value Ym of the second object coordinate Y based on the vertical distance V and the second movement set value Ly.

[0055] Specifically, the second target measurement value Ym is obtained by adding the vertical distance V and the second movement setting value Ly (Ym=V+Ly).

[0056] 5 shows an image of error factors. If there were no error factors, the first object measurement value Xm would match the first object true value Xr, which is the true value of the first object coordinate X, and the second object measurement value Ym would match the second object true value Yr, which is the true value of the second object coordinate Y.

[0057] However, in reality, due to various error factors, the first object measurement value Xm may deviate from the first object true value Xr (Xm≠Xr), and the second object measurement value Ym may deviate from the second object true value Yr (Ym≠Yr). Examples of error factors include an assembly error between the stage 20 and the camera 30, the imaging axis of the camera 30 not being orthogonal to the horizontal plane of the stage 20, a deviation in the position and angle of the stage 20, a deviation in the position and angle of the camera 30, a difference between the first movement setting value Lx and the second movement setting value Ly input by the user and the actual movement amount of the stage 20, vibration of the stage 20, vibration of the camera 30, a measurement error (distortion aberration) due to distortion of the lens of the camera 30, and an assembly error between the lens of the camera 30 and the image sensor of the camera 30.

[0058] Since the first target measurement value Xm and the second target measurement value Ym contain errors, they deviate from the first target true value Xr and the second target true value Yr. The first target measurement value Xm and the second target measurement value Ym need to be corrected in some way.

[0059] (Projection Transformation Matrix) The controller 40 corrects the first object measurement value Xm associated with the first object coordinate X using the projection transformation matrix R to calculate the first object correction value Xc associated with the first object coordinate X. The controller 40 corrects the second object measurement value Ym associated with the second object coordinate Y to calculate the second object correction value Yc associated with the second object coordinate Y. The projection transformation matrix R is also called a homography transformation matrix, and is expressed by the following equation [Mathematical Expression 1].

[0060]

[0061] The nine parameters a, b, c, d, e, f, g, h, and s in the projection transformation matrix R must be set by the user. The parameters a to h and s are determined by using the alignment substrate 60, which will be described later. The projection transformation matrix R is stored in the memory of the controller 40, for example.

[0062] (Alignment Substrate) FIG. 6 shows the alignment substrate 60. The alignment substrate 60 corresponds to the target substrate 50. The alignment substrate 60 is a substrate for calibration. The alignment substrate 60 is placed on the stage 20, just like the target substrate 50. The alignment substrate 60 has the same specifications as the target substrate 50. The shape and dimensions of the alignment substrate 60 are the same as those of the target substrate 50.

[0063] The alignment substrate 60 is provided with a plurality of alignment marks 61. The plurality of alignment marks 61 on the alignment substrate 60 correspond to the plurality of chips 51 on the target substrate 50. The shapes and positions of the plurality of alignment marks 61 on the alignment substrate 60 are the same as the shapes and positions of the plurality of chips 51 on the target substrate 50. The pitch and number of the alignment marks 61 are the same as the pitch and number of the chips 51.

[0064] When the alignment substrate 60 is placed on the stage 20, the multiple alignment marks 61 are arranged in a matrix on the alignment substrate 60 in the front-to-back direction x and the left-to-right direction y. The multiple alignment marks 61 are arranged at equal intervals in the front-to-back direction x. The multiple alignment marks 61 are arranged at equal intervals in the left-to-right direction y. The alignment marks 61 are, for example, marks that resemble integrated circuits.

[0065] An alignment origin 62 is provided on the alignment substrate 60. The alignment origin 62 on the alignment substrate 60 corresponds to the target origin 52 on the target substrate 50. The position of the alignment origin 62 on the alignment substrate 60 is the same as the position of the target origin 52 on the target substrate 50. The alignment origin 62 is configured at a corner of the alignment substrate 60, for example.

[0066] The coordinate of any alignment mark 61 in the front-to-back direction x relative to (based on) the alignment origin 62 is defined as the first alignment coordinate X'. The distance in the front-to-back direction x from the alignment origin 62 to the center of any alignment mark 61 is defined as the first alignment measurement value Xk'. The coordinate of any alignment mark 61 in the left-to-right direction y relative to (based on) the alignment origin 62 is defined as the second alignment coordinate Y'. The distance in the left-to-right direction y from the alignment origin 62 to the center of any alignment mark 61 is defined as the second alignment measurement value Yk'.

[0067] The first alignment coordinate X' corresponds to the first target coordinate X. The second alignment coordinate Y' corresponds to the second target coordinate Y. The first alignment actual measurement value Xk' corresponds to the first target true value Xr. The second alignment actual measurement value Yk' corresponds to the second target true value Yr.

[0068] The user knows the first alignment actual measurement value Xk' and the second alignment actual measurement value Yk' in advance. The first alignment actual measurement value Xk' and the second alignment actual measurement value Yk' are measured in advance, for example, by a sensor or the like. The first alignment actual measurement value Xk' is known. The second alignment actual measurement value Yk' is known.

[0069] (Alignment Measurement Values) FIG. 7 shows a first alignment measurement value Xm′ and a second alignment measurement value Ym′ related to the alignment mark 61 on the alignment substrate 60 .

[0070] The alignment origin 62 of the alignment substrate 60 is aligned so as to overlap with the image reference position 72 of the captured image 70. At this time, the alignment substrate 60 is stopped by a stopper or the like (not shown), so that the alignment origin 62 is positioned at the image reference position 72.

[0071] Data of the captured image 70 obtained by the camera 30 is input to the controller 40. The controller 40 calculates the lateral distance T between the image reference position 72 and the alignment mark 61 based on the captured image 70.

[0072] The horizontal distance T is the distance in the horizontal direction t in which the pixels 71 are arranged between the image reference position 72 in the captured image 70 and the alignment mark 61 of the alignment substrate 60 shown in the captured image 70. The horizontal distance T is calculated based on the number of pixels 71 arranged in the horizontal direction t between the image reference position 72 and the alignment mark 61, and the magnification of the captured image 70 in the horizontal direction t.

[0073] As described above, the alignment origin 62 of the alignment substrate 60 is aligned with the image reference position 72. The first alignment coordinate X′ is the coordinate of the alignment mark 61 in the front-to-rear direction x relative to the alignment origin 62 (based on the alignment origin 62).

[0074] The controller 40 calculates the first alignment measurement value Xm′ of the first alignment coordinate X′ based on the lateral distance T when the alignment origin 62 is aligned with the image reference position 72 .

[0075] Due to error factors, the first alignment measurement value Xm' may not coincide with the first alignment actual measurement value Xk' (Xm'≠Xk').

[0076] Data of the captured image 70 obtained by the camera 30 is input to the controller 40. The controller 40 calculates the vertical distance V between the image reference position 72 and the alignment mark 61 based on the captured image 70.

[0077] The vertical distance V is the distance in the vertical direction v in which the pixels 71 are lined up between the image reference position 72 in the captured image 70 and the alignment mark 61 of the alignment substrate 60 shown in the captured image 70. The vertical distance V is calculated based on the number of pixels 71 lined up in the vertical direction v between the image reference position 72 and the alignment mark 61, and the magnification of the captured image 70 in the vertical direction v.

[0078] As described above, the alignment origin 62 of the alignment substrate 60 is aligned with the image reference position 72. The second alignment coordinate Y′ is the coordinate of the alignment mark 61 in the left-right direction y relative to the alignment origin 62 (based on the alignment origin 62).

[0079] The controller 40 calculates the second alignment measurement value Ym′ of the second alignment coordinate Y′ based on the vertical distance V when the alignment origin 62 is aligned with the image reference position 72 .

[0080] Due to error factors, the second alignment measurement value Ym' may not match the second alignment actual measurement value Yk' (Ym'≠Yk').

[0081] (Deriving Parameters of Projection Transformation Matrix) The parameters a to h and s of the projection transformation matrix R are derived by the following method.

[0082] The controller 40 acquires a known first alignment actual measurement value Xk' associated with the first alignment coordinate X'. The first alignment actual measurement value Xk' is stored, for example, in the memory of the controller 40. The controller 40 acquires a known second alignment actual measurement value Yk' associated with the second alignment coordinate Y'. The second alignment actual measurement value Yk' is stored, for example, in the memory of the controller 40.

[0083] As shown below, the controller 40 calculates the parameters a to h and s of the projection transformation matrix R based on the first alignment measurement value Xm', the first alignment actual measurement value Xk', the second alignment measurement value Ym', and the second alignment actual measurement value Yk'.

[0084] Specifically, the alignment substrate 60 is used to replace the formula (1) with the formula (2).

[0085]

[0086] The right side of Equation 2 contains a first alignment correction value Xc' and a second alignment correction value Yc'. The first alignment correction value Xc' corresponds to the first target correction value Xc. The second alignment correction value Yc' corresponds to the second target correction value Yc. The first alignment correction value Xc' and the second alignment correction value Yc' contain unknown parameters a to h, and s, and therefore are not yet expressed as specific numerical values.

[0087] The parameters a to h, s of the projection transformation matrix R must be set so that the first alignment correction value Xc' (first target correction value Xc) approaches the first alignment actual measurement value Xk' (first target true value Xr) and the second alignment correction value Yc' (second target correction value Yc) approaches the second alignment actual measurement value Yk' (second target true value Yr).

[0088] The parameters a to h and s of the projection transformation matrix R are found, for example, by the least squares method. Specifically, the parameters a to h and s are found when the value N shown in equation (3) is minimum.

[0089]

[0090] Since the projection transformation matrix R has nine unknown parameters a to h and s, it is sufficient to formulate and solve nine equations under nine conditions (for example, using nine alignment marks 61) so that the value N in equation [3] is minimized.

[0091] (Action and effect of the first embodiment) In the inspection device 1 according to this embodiment, the target measurement values ​​Xm, Ym of the target coordinates X, Y of the chip 51 on the target substrate 50 obtained based on the captured image 70 may deviate from the target true values ​​Xr, Yr, which are true values, due to various error factors.

[0092] Therefore, in the inspection device 1 according to this embodiment, the target measurement values ​​Xm, Ym are converted into target correction values ​​Xc, Yc by using the projection transformation matrix R, and are brought closer to the target true values ​​Xr, Yr, which are the true values.

[0093] In particular, in the inspection device 1 according to this embodiment, when determining the parameters a to h, and s of the projective transformation matrix R, the alignment mark 61 of the alignment substrate 60 is used in a state in which the alignment origin 62 is aligned with the image reference position 72. Then, the parameters a to h, and s of the projective transformation matrix R are calculated based on the alignment measurement values ​​Xm', Ym' and the alignment actual measurement values ​​Xk', Yk'. This makes it possible to obtain a highly accurate projective transformation matrix R.

[0094] As described above, it is possible to provide an inspection device 1 that can obtain accurate target coordinates X, Y of the chip 51 provided on the target substrate 50.

[0095] When deriving the parameters a to h, s of the projection transformation matrix R, by using the alignment measurement values ​​Xk', Yk' as the correct values, a more accurate projection transformation matrix R can be obtained, which is advantageous in bringing the target correction values ​​Xc, Yc closer to the true target values ​​Xr, Yr, which are the true values.

[0096] According to the inspection device 1 of this embodiment, even when the stage 20 and the camera 30 move relatively in the forward / backward direction x and left / right direction y, the accurate first object coordinates X and second object coordinates Y of the chip 51 provided on the target substrate 50 can be obtained.

[0097] (Variation of First Embodiment) The controller 40 may use a projective transformation matrix R to correct deviations in the object measurement values ​​Xm, Ym caused by distortion of the camera 30 to obtain object correction values ​​Xc, Yc. Here, the distortion of the camera 30 may include not only physical distortion but also optical distortion. Examples of optical distortion of the camera 30 include lens focus error, shading, and lens distortion.

[0098] In this case, the values ​​of the parameters a to h and s of the projective transformation matrix R used to correct the distortion of the camera 30 may differ from the values ​​of the parameters a to h and s of the projective transformation matrix R used to correct the positional deviation of the stage 20. The projective transformation matrix R for correcting the distortion of the camera 30 and the projective transformation matrix R for correcting the positional deviation of the stage 20 may be used separately, or a single projective transformation matrix R having both functions may be used.

[0099] As an example, the distortion of the lens 31 of the camera 30 will be described. FIG. 8 shows the distortion of the lens 31 of the camera 30. In the camera 30 using the lens 31, distortion of the lens 31 occurs. Distortion occurs in the radial direction of the lens 31 and in the tangential direction perpendicular to the radial direction. Radial distortion occurs on the projection plane Q because, when a ray of light emitted from the subject P passes through the principal point of the lens 31 at a certain angle of incidence, the exit angle does not match the angle of incidence. Tangential distortion occurs due to misalignment or tilt of the centers of the multiple lenses that make up the lens 31. Distortion can be expressed, for example, by equations [Formula 4] to [Formula 6].

[0100]

[0101]

[0102]

[0103] Here, when an arbitrary point in three-dimensional space is photographed to obtain image coordinates, the coordinates are (Xu, Yu) for an ideal lens 31 with no distortion aberration, and (Xd, Yd) for an actual lens 31 with distortion aberration. r is the distance from the center of the image to the coordinates Xu, Yu. K1 to K5 are coefficients that represent the distortion of the lens 31.

[0104] Of these, K3 and K4 represent distortion in the tangential direction, but since this can often be ignored in practice, they may be simplified as in equations [7] and [8].

[0105]

[0106]

[0107] When an actual lens 31 having distortion is used, the coordinates Xd, Yd of the chip 51 correspond to the target measurement values ​​Xm, Ym. The controller 40 may use a projective transformation matrix R as shown in Equation 9 to correct the deviation of the coordinates Xd, Yd of the chip 51 (corresponding to the target measurement values ​​Xm, Ym) caused by the distortion of the lens 31 of the camera 30 and obtain the target correction values ​​Xc, Yc.

[0108]

[0109] The nine parameters a to h and s may be the same as those in the first embodiment, or may be calculated separately using the method described below.

[0110] For example, multiple patterns are prepared, including known coordinates Xd, Yd of the chip 51 when an actual lens 31 with distortion is used, and known coordinates Xu, Yu of the chip 51 when an ideal lens 31 without distortion is used. The coordinates Xu, Yu of the ideal chip 51 are, so to speak, correct values. The nine parameters a to h, and s can be obtained by formulating multiple relational expressions shown in Equation (10) and solving them.

[0111]

[0112] According to this modified example, even if there is an error due to distortion (particularly distortion aberration) of the camera 30 (particularly the lens 31), the target measurement values ​​Xm, Ym (for example, the coordinates Xd, Yd of the chip 51 when an actual lens 31 with distortion aberration is used) can be converted into target correction values ​​Xc, Yc using the projection transformation matrix R, thereby bringing them closer to the true values.

[0113] Second Embodiment An inspection device 1 according to a second embodiment will be described. In the following description, the same components as those in the above embodiment will be denoted by the same reference numerals, and detailed description thereof will be omitted.

[0114] In the inspection device 1 according to the first embodiment, the controller 40 corrects the target measurement values ​​Xm, Ym relating to the target coordinates X, Y using the projection transformation matrix R to calculate the target correction values ​​Xc, Yc relating to the target coordinates X, Y.

[0115] Unlike the first embodiment, the inspection device 1 according to the second embodiment uses a machine learning model M instead of a homography matrix R. FIG. 9 shows the configuration of the machine learning model M.

[0116] The machine learning model M is stored in the controller 40 (e.g., a personal computer).

[0117] The controller 40 executes the machine learning model M as follows: The controller 40 corrects the target measurement values ​​Xm, Ym related to the target coordinates X, Y using the machine learning model M to calculate the target correction values ​​Xc, Yc related to the target coordinates X, Y. When the machine learning model M is executed, the target measurement values ​​Xm, Ym are input to the machine learning model M, and the machine learning model M calculates and outputs the target correction values ​​Xc, Yc.

[0118] The machine learning model M is a supervised learning model. Specific examples of the machine learning model M include ridge regression, a gradient boosting decision tree (GDBT), and a multilayer perceptron (MLP).

[0119] The machine learning model M learns as follows: The machine learning model M uses known target measurement values ​​Xm, Ym relating to the target coordinates X, Y and known target actual measurement values ​​Xk, Yk, which are known actual measurement values ​​relating to the target coordinates X, Y, as training data.

[0120] The known target measurement values ​​Xm, Ym are obtained by collecting data on the target measurement values ​​Xm, Ym measured by the method described in the first embodiment. It is preferable to collect a large amount of data on the known target measurement values ​​Xm, Ym. Due to various error factors, the known target measurement values ​​Xm, Ym deviate from the target true values ​​Xr, Yr, which are the true values ​​of the target coordinates X, Y.

[0121] The known target actual measurement values ​​Xk, Yk coincide with the target true values ​​Xr, Yr, which are the true values ​​of the target coordinates X, Y. The known target actual measurement values ​​Xk, Yk are, so to speak, correct values. It is preferable to collect a large amount of data on the known target actual measurement values ​​Xk, Yk. The known target actual measurement values ​​Xk, Yk can be obtained, for example, by directly measuring them using a sensor or the like.

[0122] When learning the machine learning model M, known target measurement values ​​Xm, Ym are provided to the input layer of the machine learning model M as training data, and known target actual measurement values ​​Xk, Yk are provided to the output layer of the machine learning model M as training data.

[0123] (Action and effect of the second embodiment) In the inspection device 1 according to this embodiment, the target measurement values ​​Xm, Ym of the target coordinates X, Y of the chip 51 on the target substrate 50 obtained based on the captured image 70 may deviate from the target true values ​​Xr, Yr, which are true values, due to various error factors.

[0124] Therefore, in the inspection device 1 according to this embodiment, the machine learning model M is used to convert the target measurement values ​​Xm, Ym into target correction values ​​Xc, Yc, thereby bringing them closer to the true target values ​​Xr, Yr.

[0125] In particular, in the inspection device 1 according to this embodiment, known target measurement values ​​Xm, Ym and known target actual measurement values ​​Xk, Yk are used as training data in the machine learning model M. This makes it possible to obtain a highly accurate machine learning model M.

[0126] By using the machine learning model M to find the correlation between the target measurement values ​​Xm, Ym and the target actual measurement values ​​Xk, Yk, it is possible to obtain target corrected values ​​Xc, Yc that are close to the true values.

[0127] As described above, it is possible to provide an inspection device 1 that can obtain accurate target coordinates X, Y of the chip 51 provided on the target substrate 50.

[0128] Based on the distances T and V in the directions t and v in which the pixels 71 are arranged between the image reference position 72 and the chip 51, and the movement setting values ​​Lx and Ly, unknown target measurement values ​​Xmi and Ymi relating to the target coordinates X and Y of the chip 51 relative to the target origin 52 provided on the target substrate 50 can be easily calculated.

[0129] (Variation of the Second Embodiment) As described in the first embodiment, the alignment measurement values ​​Xm', Ym' related to the alignment coordinates X', Y' correspond to the target measurement values ​​Xm, Ym related to the target coordinates X, Y. Therefore, the machine learning model M may apply the known alignment measurement values ​​Xm', Ym' related to the alignment coordinates X', Y' to the training data as the known target measurement values ​​Xm, Ym related to the target coordinates X, Y.

[0130] As described in the first embodiment, the alignment measurement values ​​Xk', Yk' related to the alignment coordinates X', Y' coincide with the target true values ​​Xr, Yr related to the target coordinates X, Y. In other words, the alignment measurement values ​​Xk', Yk' related to the alignment coordinates X', Y' coincide with the known target measurement values ​​Xk, Yk related to the target coordinates X, Y. For this reason, the machine learning model M may apply the known alignment measurement values ​​Xk', Yk' related to the alignment coordinates X', Y' as training data for the known target measurement values ​​Xk, Yk related to the target coordinates X, Y.

[0131] According to this modification, the data necessary for training the machine learning model M can be easily obtained.

[0132] <Other Embodiments> Although the present disclosure has been described above with reference to preferred embodiments, these descriptions are not limiting and, of course, various modifications, substitutions, and combinations are possible.

[0133] The controller 40 may be provided outside the main body of the inspection device 1 instead of being built into the main body of the inspection device 1 .

[0134] Instead of the stage 20 being movable and the camera 30 being fixed, the stage 20 may be fixed and the camera 30 may be movable, or both may be movable.

[0135] The target substrate 50 and the alignment substrate 60 are not limited to being rectangular, and may be, for example, circular. The target substrate 50 and the alignment substrate 60 do not have to be semiconductors. The target origin 52 and the alignment origin 62 may be provided, for example, at the center of the target substrate 50 and the alignment substrate 60, rather than at a corner thereof.

[0136] In the first embodiment, the parameters of the projective transformation matrix R may be calculated by, for example, the Levenberg-Marquardt method or the least median method, instead of the least squares method.

[0137] In the first embodiment, the projective transformation matrix R may be stored in an external server rather than being stored in the memory of the controller 40. In this case, the controller 40 accesses the projective transformation matrix R in the external server to input data to the projective transformation matrix R or receive data output from the projective transformation matrix R.

[0138] In the second embodiment, the machine learning model M may be stored in an external server instead of being stored in the memory of the controller 40. In this case, the controller 40 accesses the machine learning model M in the external server to input data to the machine learning model M or receive data output from the machine learning model M.

[0139] The present disclosure is applicable to inspection devices and is therefore extremely useful and has high industrial applicability.

[0140] x: front-back direction (first direction) y: left-right direction (second direction) z: up-down direction X: first object coordinate Y: second object coordinate Xr: first object true value Yr: second object true value Xc: first object corrected value Yc: second object corrected value Xm: first object measured value Ym: second object measured value Xk, Yk: object actual measured value X': first alignment coordinate Y': second alignment coordinate Xk': first alignment actual measured value Yk': second alignment actual measured value Xm': first alignment measured value Ym': second alignment measured value Xc': first alignment corrected value Yc': second alignment corrected value Lx: first movement set value Ly: second movement set value M: machine learning model t: horizontal direction v: vertical direction T: horizontal distance V: vertical distance R: projective transformation matrix a to h, s: parameters 1: inspection device 10: rail 11 First rail 12 Second rail 20 Stage 30 Camera 31 Lens 40 Controller 50 Target substrate 51 Chip 52 Target origin 60 Alignment substrate 61 Alignment mark 62 Alignment origin 70 Captured image 71 Pixel 72 Image reference position

Claims

1. A system comprising: a stage on which a target substrate having a plurality of chips arranged side by side is placed; a camera which images the target substrate placed on the stage to obtain an image divided into a plurality of pixels; and a controller which calculates object measurement values ​​of object coordinates of the chip relative to an object origin arranged on the target substrate based on a distance in a direction in which the pixels are arranged between an image reference position in the image and the chip shown in the image, and a relative movement setting value between the stage and the camera; the controller corrects the object measurement values ​​relating to the object coordinates using a projection transformation matrix to calculate object correction values ​​relating to the object coordinates; an alignment substrate corresponding to the target substrate is provided with a plurality of alignment marks corresponding to the plurality of chips; and an alignment origin corresponding to the object origin is provided on the alignment substrate; an inspection device, wherein the controller calculates alignment measurement values ​​of the alignment coordinates of the alignment mark relative to the alignment origin based on the distance in the pixel alignment direction between the image reference position and the alignment mark shown in the captured image with the alignment origin aligned with the image reference position; the controller acquires known alignment actual measurement values ​​of the alignment coordinates; and the controller calculates parameters of the projection transformation matrix based on the alignment measurement values ​​and the alignment actual measurement values.

2. A plurality of the chips are arranged on the target substrate in a first direction and a second direction intersecting the first direction, and the controller calculates a first object measurement value of a first object coordinate of the chip in the first direction relative to the object origin based on a horizontal distance between the image reference position and the chip shown in the captured image in the horizontal direction in which the pixels are arranged, and a first relative movement setting value between the stage and the camera in the first direction, and the controller calculates a second object measurement value of a second object coordinate of the chip in the second direction relative to the object origin based on a vertical distance between the image reference position and the chip shown in the captured image in the vertical direction in which the pixels are arranged, and a second relative movement setting value between the stage and the camera in the second direction, and the controller corrects the first object measurement value related to the first object coordinates by the projective transformation matrix to calculate a first object correction value related to the first object coordinates, and corrects the second object measurement value related to the second object coordinates to calculate a second object correction value related to the second object coordinates, the controller calculates a first alignment measurement value of a first alignment coordinate of the alignment mark in the first direction relative to the alignment origin based on the horizontal distance in the horizontal direction in which the pixels are aligned between the image reference position and the alignment mark shown in the captured image with the alignment origin aligned at the image reference position; the controller calculates a second alignment measurement value of a second alignment coordinate of the alignment mark in the second direction relative to the alignment origin based on the vertical distance in the vertical direction in which the pixels are aligned between the image reference position and the alignment mark shown in the captured image with the alignment origin aligned at the image reference position; the controller acquires a known first alignment actual measurement value related to the first alignment coordinate and a known second alignment actual measurement value related to the second alignment coordinate; The inspection apparatus according to claim 1 , wherein the controller calculates the parameters of the homography matrix based on a first alignment measurement value, the first alignment actual measurement value, the second alignment measurement value, and the second alignment actual measurement value.

3. The inspection device according to claim 1 or 2, wherein the controller uses the projective transformation matrix to correct for deviations in the object measurements caused by distortion of the camera to obtain the object correction values.

4. An inspection device comprising: a stage on which a target substrate having a plurality of chips arranged side by side is placed; a camera which images the target substrate placed on the stage to obtain an image; and a controller which calculates measurement values ​​of target coordinates of the chip based on the image, wherein the controller corrects the measurement values ​​of the target coordinates and calculates correction values ​​of the target coordinates using a machine learning model which uses the measurement values ​​of the target coordinates and actual measurement values ​​of the target coordinates as training data.

5. The inspection device described in claim 4, wherein the captured image is divided into a plurality of pixels, and the controller calculates the measurement value relating to the target coordinates of the chip relative to a target origin provided on the target substrate based on the distance between an image reference position in the captured image and the chip shown in the captured image in the direction in which the pixels are aligned, and a relative movement setting value between the stage and the camera.

6. The inspection device of claim 5, wherein an alignment substrate corresponding to the target substrate is provided with a plurality of alignment marks corresponding to a plurality of the chips, the alignment substrate is provided with an alignment origin corresponding to the target origin, the controller calculates alignment measurement values ​​of the alignment coordinates of the alignment mark based on the distance in the pixel arrangement direction between the image reference position and the alignment mark shown in the captured image with the alignment origin aligned with the image reference position, the controller acquires alignment actual measurement values ​​related to the alignment coordinates, and the machine learning model uses the alignment measurement values ​​related to the alignment coordinates as teacher data as the measurement values ​​related to the target coordinates, and uses the alignment actual measurement values ​​related to the alignment coordinates as teacher data as the actual measurement values ​​related to the target coordinates.

Images