Focal position deviation amount correction method, focal position deviation amount correction system, and program

Abstract

A focal position deviation amount correction method according to the present invention involves positioning a camera (12) and a component (P1) of which an image is to be captured such that a focal point is on the component (P1). Input data that includes a focus evaluation value for the component (P1) as captured by the camera (12) is inputted into a learning model that, in response to input of the input data, outputs a relative position deviation amount for the focal position of the camera (12) and the component (P1), and the position of at least one of the camera (12) and the component (P1) is controlled on the basis of the relative position deviation amount outputted by the learning model.

Description

Focus position deviation amount correction method, focus position deviation amount correction system, and program

[0001] The present disclosure relates to a processing device that performs processes such as inspection and alignment of components, etc. When inspecting an image of an object, it estimates the focus position of a camera, and corrects the position of the camera or the object according to the deviation amount between the focus position of the camera and the object. The present disclosure relates to a focus position deviation amount correction method and a focus position deviation amount correction system.

[0002] In recent years, when performing processes such as inspecting or aligning an object such as a component, image recognition technology is often used in scenarios that require particularly high precision. For example, when using image recognition technology to align components, the component and the target position are imaged and recognized by a camera, image processing is performed, the amount of positional deviation between the component and the target position is calculated, and based on the calculation result, the component is moved to align the position of the component.

[0003] At that time, if the camera or the camera holding member, or the component or the holding member of the target position thermally expands due to heat, the component and the target position will be imaged at a position deviated from the focus position of the camera, resulting in a large blur of the image and a possible decrease in the recognition accuracy of components, etc. by the camera.

[0004] In addition, for processes that require high precision, imaging in the vicinity of the focus position is necessary, and various proposals have been made regarding the focus position deviation amount correction system and its correction method.

[0005] For example, in Patent Document 1, the objective lens of the camera is moved vertically from a preset search start position (imaging start position) to a preset search end position (imaging end position) relative to the surface on which the object to be imaged is placed, and an image of the object to be imaged is captured at each sampling interval. The focus position of the objective lens is calculated based on the contrast curve obtained from the contrast values ​​of the captured images. A first threshold and a second threshold smaller than the first threshold are set for the obtained contrast curve, and the midpoint coordinates of each threshold are calculated from the four intersection points of the first threshold, the second threshold, and the contrast curve. The intersection point of the extension of the line connecting the midpoint coordinates of each threshold and the contrast curve is set as the focus position. This method makes it possible to set the focus position with high accuracy even when the number of samples is small.

[0006] Japanese Patent Publication No. 2011-28103

[0007] However, the technology described in Patent Document 1 requires the acquisition of an image taken by moving the objective lens toward the search start position from the focus position, and an image taken by moving the objective lens toward the search end position from the focus position. This presents the problem that moving the objective lens and acquiring multiple images takes time.

[0008] On the other hand, increasing the objective lens movement speed to shorten the imaging time increases the amount of lens movement during imaging, causing blurring in the image. When blurring occurs in the image, it becomes impossible to accurately calculate the contrast value, and as a result, the accuracy of calculating the focus position decreases.

[0009] This disclosure aims to solve these problems and provides a focus position misalignment correction method and a focus position misalignment correction system that can correct the amount of focus misalignment at high speed and with high precision even when the camera's focus position changes due to a shift in the positional relationship between the camera, lens, or object in a processing device that aligns objects such as parts.

[0010] To achieve the above objective, one aspect of the focus misalignment correction method according to the present disclosure is a focus misalignment correction method for aligning a camera and an object to be imaged so that the object to be imaged is in focus, wherein the input data, including the focus evaluation value of the object to be imaged captured by the camera, is input to a learning model that outputs a relative position misalignment between the camera's focus position and the object to be imaged, and the camera and the object to be imaged are controlled based on the relative position misalignment output by the learning model.

[0011] Furthermore, one embodiment of the focus position misalignment correction system according to the present disclosure is a focus position misalignment correction system that aligns a camera with an object to be imaged so that the object to be imaged is in focus, comprising: a stage on which the object to be imaged is placed; a camera that images the object to be imaged; and a control device that uses a learning model that receives input data including a focus evaluation value of the object to be imaged captured by the camera and outputs the focus position of the camera and the relative position misalignment amount with the object to be imaged, and controls the position of at least one of the stage and the camera based on the relative position misalignment amount output by the learning model.

[0012] According to this disclosure, even if heat causes a shift in the positional relationship between the camera, lens, or object, and the camera's focus position changes, the amount of focus position shift can be corrected quickly and with high precision.

[0013] A schematic diagram showing the configuration of the focus misalignment correction system according to Embodiment 1. A functional block diagram of the focus misalignment correction system according to Embodiment 1. A diagram showing the operation flow of the focus misalignment correction method according to Embodiment 1. A diagram showing the learning model generation flow of the focus misalignment correction system according to Embodiment 1. A diagram showing the learning data acquisition operation flow of the focus misalignment correction system according to Embodiment 1. A diagram showing an example of operation parameters acquired by the operation parameter acquisition unit in the control device of the focus misalignment correction system according to Embodiment 1. A diagram explaining the method for calculating the focus evaluation value of a component and the position of a component in the focus misalignment correction system according to Embodiment 1. A diagram showing the relationship between the camera stage position and the focus evaluation value of a component calculated in the focus misalignment correction system according to Embodiment 1. A schematic diagram showing the configuration of the focus misalignment correction system according to Embodiment 2.

[0014] The embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are all specific examples of this disclosure. Therefore, the numerical values, shapes, materials, components, arrangement and connection configurations of components, as well as the steps (processes) and their order, shown in the following embodiments are examples and are not intended to limit this disclosure. Accordingly, any components in the following embodiments that are not described in the independent claims representing the highest-level concepts of this disclosure will be described as optional components.

[0015] In this specification and drawings, the X, Y, and Z axes represent the three axes of a three-dimensional Cartesian coordinate system. The X and Y axes are mutually orthogonal and both are orthogonal to the Z axis. The direction of rotation around the Z axis is denoted as θ. In this embodiment, the Z axis direction is the vertical direction. Note that each figure is a schematic diagram and not necessarily a strictly accurate representation. Therefore, the scale and other aspects in each figure do not necessarily match. In each figure, substantially identical components are denoted by the same reference numerals, and redundant explanations are omitted or simplified.

[0016] Furthermore, in this specification, the terms "upper," "up," "down," and "below" do not refer to the upward (vertically upward) and downward (vertically downward) directions in absolute spatial perception, but rather are used as terms defined by relative positional relationships. Moreover, the terms "upper," "up," and "down" apply not only when two components are spaced apart and another component exists between them, but also when two components are placed in contact with each other.

[0017] (Embodiment 1) [Configuration of Focus Position Shift Amount Correction System and Processing Unit] First, the focus position shift amount correction system 1 and processing unit 10 according to Embodiment 1 will be described using Figures 1 and 2. Figure 1 is a schematic diagram showing the configuration of the focus position shift amount correction system 1 according to Embodiment 1. Figure 2 is a functional block diagram of the focus position shift amount correction system 1 according to Embodiment 1.

[0018] As shown in Figure 1, the focus position misalignment correction system 1 includes a processing unit 10 that recognizes an object such as a part and performs a predetermined process, and a control device 20 that controls the processing unit 10.

[0019] The processing unit 10 is, for example, part of a positioning device that accurately positions electronic components on a substrate and places the electronic components on the substrate, and recognizes the position of the substrate which is the target position. As an example, the processing unit 10 is a component mounting machine.

[0020] As shown in Figure 1, the processing unit 10 includes a stage 11 on which the component P1 is placed, a camera 12 that recognizes the component P1, and a camera stage 13 that holds the camera 12.

[0021] Component P1 is marked with an alignment mark M1. The alignment mark M1 is a characteristic shape of component P1, which is the object being imaged. The alignment mark M1 is printed on the surface of component P1, for example. The alignment mark M1 is, for example, one or more shapes. As an example, the shape of the shape that constitutes the alignment mark M1 is a circle, a square, or a cross, but is not limited to these.

[0022] The stage 11 and the camera stage 13 are configured to be relatively movable. Specifically, the stage 11 and the camera stage 13 are configured to be relatively movable at least along the optical axis direction (Z-axis direction in this embodiment) of the camera 12 held by the camera stage 13. Therefore, either the stage 11 or the camera stage 13 may be configured to be movable, or both the stage 11 and the camera stage 13 may be configured to be movable. Furthermore, the stage 11 and the camera stage 13 may be configured to be movable not only in the Z-axis direction but also in the XY direction, or to be rotatable in the θ direction.

[0023] In this embodiment, both the stage 11 and the camera stage 13 are configured to be movable. Specifically, the stage 11 can move horizontally (in the X-axis and Y-axis directions) and rotate around the Z-axis (θ rotation). The camera stage 13 can move in the Z-axis direction. Therefore, although not shown in the figures, the processing unit 10 has a drive unit that can move the stage 11 in the X-axis and Y-axis directions and rotate it around the Z-axis, and a drive unit that can move the camera stage 13 in the Z-axis direction.

[0024] The camera 12 comprises a camera body 12a having an image sensor, an optical system 12b including a lens, an illumination device 12c, and a computing device (not shown) that performs various calculations. The camera body 12a has an image sensor. The lens of the optical system 12b focuses light onto the image sensor of the camera body 12a. The illumination device 12c has a light source and emits white light as illumination light. The illumination light from the illumination device 12c is not limited to white light, but may be monochromatic light such as red light or blue light, or invisible light such as infrared light (IR light).

[0025] The camera stage 13 is a holding member that holds the camera 12. The camera stage 13, which holds the camera 12, moves along the optical axis of the camera 12 to adjust the position of the camera 12 so that it is in focus on the subject part P1.

[0026] Furthermore, if the focus position of the camera 12 relative to the object being imaged can be adjusted, it is not necessary to move the camera 12 itself using a camera stage 13 or the like. For example, the focus position of the camera 12 can be adjusted by adjusting the position of the lens of the camera 12.

[0027] Camera 12 recognizes component P1 by imaging it as the object to be imaged. In this embodiment, camera 12 recognizes component P1 by imaging component P1 located below camera 12. Specifically, camera 12 recognizes component P1 by imaging and recognizing the alignment mark M1 of component P1. When imaging component P1, the illumination device 12c may be turned on to create a light-dark inversion state before imaging component P1.

[0028] When the camera 12 recognizes part P1, the camera stage 13 is moved so that it approximately coincides with the position where the camera 12 is in focus on part P1 (hereinafter referred to as the "focus position" of the camera 12), and part P1 is recognized by imaging the surface of part P1 with the camera 12. Specifically, the alignment mark M1 printed on part P1 is recognized by imaging part P1 with the camera 12. After the camera 12 has recognized part P1, position correction is performed based on the recognition result so that the focus position of the camera 12 and the position of part P1 precisely coincide.

[0029] The control device 20 is composed of a computer system and performs various controls on the processing device 10. As shown in Figure 2, the control device 20 controls the stage 11, camera 12, and camera stage 13 of the processing device 10.

[0030] In this embodiment, the control device 20 includes an operation command unit 21, an operation parameter acquisition unit 22, a data processing unit 23, a learning model generation unit 24, a focus position deviation amount estimation unit 25, an operation control unit 26, and a storage unit 27.

[0031] The operation command unit 21 determines whether or not a focus position shift correction operation is necessary based on the recognition result of the camera 12. If it is determined that a focus position shift correction operation is necessary, the operation parameter acquisition unit 22 acquires operation parameters and sends an operation command for position correction operation of the camera stage 13 to the processing unit 10 based on the amount of positional shift (estimated value) between the focus position of the camera 12 estimated by the focus position shift estimation unit 25 and the position of the camera 12 in the optical axis direction (hereinafter also simply referred to as "optical axis direction") of the component P1. Specifically, the control device 20 is connected to the stage 11 and the camera stage 13, and sends operation commands to the stage 11 to move the stage 11, and sends operation commands to the camera stage 13 to move the camera stage 13.

[0032] The operation parameter acquisition unit 22 acquires operation parameters during the focus misalignment correction operation, which include at least the recognition result of the part P1 by the camera 12. The recognition result of the part P1 included in the operation parameters is, for example, the detection data of the alignment mark M1 of the part P1.

[0033] The data processing unit 23 formats and processes the data of the operation parameters acquired by the operation parameter acquisition unit 22 during the focus position shift amount correction operation.

[0034] The learning model generation unit 24 receives operation parameters during the focus position shift amount correction operation, which include at least the recognition result of the part P1 by the camera 12, as input data, and generates a machine learning-trained learning model that outputs the relative position shift amount (i.e., focus position shift amount) between the focus position of the camera 12 and the position of the part P1 in the optical axis direction. In other words, the learning model generation unit 24 generates a trained learning model.

[0035] The focus position deviation amount estimation unit 25 receives operational parameters during the focus position deviation amount correction operation, which include at least the recognition result of the part P1 by the camera 12, as input data to the learning model generated by the learning model generation unit 24. The unit estimates the mutual position deviation amount between the focus position of the camera 12 and the position of the part P1 in the optical axis direction from the relative position deviation amount of the focus position of the camera 12 output from the learning model.

[0036] The motion control unit 26 performs focus position misalignment correction by controlling the operation of each component of the processing unit 10. Specifically, the motion control unit 26 controls the operation of the stage 11, the camera 12, and the camera stage 13. The control device 20 is also connected to the camera 12 and sends a command to the camera 12 to image component P1.

[0037] The memory unit 27 stores data processed by the data processing unit 23, the learning model generated by the learning model generation unit 24, and the amount of focus deviation estimated by the focus deviation estimation unit 25. The control device 20 may also send operation commands to the processing unit 10 or control the operation of the processing unit 10 according to the production program stored in the memory unit 27.

[0038] [Focus Position Misalignment Correction Method] Next, the focus position misalignment correction method in the focus position misalignment correction system 1 will be explained with reference to Figures 1 and 2, and with reference to Figure 3. Figure 3 is a diagram showing the operation flow of the focus position misalignment correction method according to Embodiment 1.

[0039] The focusing position misalignment correction method according to this embodiment is a method of aligning the camera 12 and the part P1 so that the part P1, which is the object to be imaged, is in focus.

[0040] As shown in Figure 3, in the focus misalignment correction method according to this embodiment, first, the part P1 is imaged by the camera 12 (step SA1). Specifically, as shown in Figure 1, the stage 11 is moved so that the part P1 placed on the stage 11 is positioned below the camera 12. Furthermore, the position of the camera 12 in the Z-axis direction is moved so that the focus position of the camera 12 and the part P1 substantially coincide, and then the surface of the part P1 is imaged by the camera 12. In this embodiment, the alignment mark M1 printed on the part P1 is imaged by the camera 12. There is no particular limit on the number of times the part P1 is imaged.

[0041] Next, the focus evaluation value for component P1 is calculated (step SA2). The focus evaluation value is an evaluation value that indicates the degree of focus match with component P1 in the captured image. The focus evaluation value is highest when the focus matches component P1, and conversely, it decreases as the focus does not match component P1. In this embodiment, it is assumed that the focus evaluation value is high when the degree of focus match is high, but this is not limited to this. For example, the focus evaluation value may be low when the degree of focus match is high. In this case, the point with the lowest focus evaluation value becomes the in-focus position.

[0042] In this embodiment, the focus evaluation value of component P1 can be calculated by performing calculations on the image captured by the camera 12 in step SA1 using the camera 12's processing unit. For example, the focus evaluation value of component P1 can be obtained by detecting the alignment mark M1 from the image captured by the camera 12 in step SA1 and calculating the difference in brightness between the alignment mark M1 and the pixels adjacent to the alignment mark M1. The method for calculating the focus evaluation value is not particularly limited, as long as the degree of focus match with respect to component P1, which is the object being photographed, can be determined from the image captured by the camera 12. For example, the property that high-frequency components are larger in images with higher focus evaluation values ​​can be used to extract high-frequency components from the image captured by the camera 12 and calculate the focus evaluation value from the integrated value of the high-frequency components. Alternatively, the property that brightness dispersion is larger in images with higher focus evaluation values ​​can be used to calculate the focus evaluation value from the brightness dispersion of the image captured by the camera 12.

[0043] In this embodiment, the focus evaluation value of component P1 was detected using a recognition image of the alignment mark M1 provided for detecting the focus evaluation value, but this is not limited to this. For example, the focus evaluation value of component P1 may be detected using a recognition image in which a part of component P1 has been captured. As a part of component P1, a part of the outer casing of component P1 (for example, a corner of the component) may be used, or an electrode or the like that of component P1 may be used.

[0044] In addition, in this embodiment, the focus evaluation value is calculated by the arithmetic unit of the camera 12, but it is not limited to this. For example, if the focus evaluation value can be calculated, the focus evaluation value may be calculated by the control device 20 or other arithmetic units.

[0045] Next, it is determined whether a focus position deviation amount correction operation for the component P1 is necessary according to the focus evaluation value of the component P1 calculated in step SA2 (step SA3). This determination is made by, for example, the operation command unit 21.

[0046] As a method for determining whether a focus position deviation amount correction operation is necessary, the allowable value of the focus evaluation value set in the production parameters is used as a threshold value. If the estimated focus evaluation value is less than or equal to the threshold value, it is determined that a focus position deviation amount correction operation is necessary. If it is greater than or equal to the threshold value, a method of determining that the focus position deviation amount correction operation is unnecessary may be used. A method of performing a focus position deviation amount correction operation according to the number of correction times set in the production parameters may also be used. A method of making the correction operation unnecessary when either one of both conditions is satisfied may also be used.

[0047] For example, when the processing device 10 is a part of a device for aligning an electronic component with a substrate, if the focus position deviation amount becomes large, the alignment accuracy of the electronic component by the processing device 10 decreases. Therefore, when the alignment accuracy of the electronic component by the processing device 10 falls below a preset threshold value, a focus position deviation amount correction operation may be performed.

[0048] Next, when it is determined in step SA3 that a focus position deviation amount correction operation is necessary (YES in step SA3), the process proceeds to a focus position deviation amount estimation operation. Using the learned learning model generated by the learning model generation unit 24 by the control device 20, the focus position deviation amount, that is, the relative position deviation amount between the focus position of the camera 12 and the position of the component P1 in the optical axis direction is estimated. (Step SA4). When input data including the focus evaluation value of the component P1 imaged by the camera 12 is input to the learning model, the relative position deviation amount between the focus position of the camera 12 and the component P1 (that is, the focus position deviation amount) is output.

[0049] Specifically, the operation parameter acquisition unit 22 acquires operation parameters during the focus position shift amount correction operation, including the focus evaluation value of the part P1 imaged by the camera 12. The data processing unit 23 generates input data for the learning model by shaping and / or processing the acquired operation parameters. The focus position shift amount estimation unit 25 inputs the input data generated by the data processing unit 23 into the learning model stored in the storage unit 27, and estimates the relative position shift amount between the camera 12's focus position and the part P1's position in the optical axis direction based on the relative position shift amount between the camera 12's focus position and the part P1's position in the optical axis direction output from the learning model. The operation parameters acquired by the operation parameter acquisition unit 22 are the same as the input data acquired when the learning model is generated, as described later.

[0050] The method for generating the learning model by the learning model generation unit 24 will be described later.

[0051] Next, based on the amount of focus misalignment estimated in step SA4, the system proceeds to the focus misalignment correction operation and executes the focus misalignment correction operation (step SA5).

[0052] Specifically, in the focus misalignment correction operation, the position of at least one of the camera 12 and component P1 is controlled based on the relative misalignment amount output by the learning model. In this embodiment, since the camera 12 is held on the camera stage 13 and component P1 is placed on the stage 11, in the focus misalignment correction operation, the position of at least one of the stage 11 and camera stage 13 is controlled based on the relative misalignment amount between the focus position of the camera 12 and the position of component P1 in the optical axis direction, as output by the learning model.

[0053] More specifically, in step SA4, the operation command unit 21 sends an alignment operation command for part P1 to the camera stage 13 according to the amount of focus misalignment estimated by the learning model, and the operation control unit 26 drives the camera stage 13 in the Z-axis direction to correct the amount of positional misalignment between the camera 12's focus position and part P1 in the optical axis direction.

[0054] In this embodiment, the focus position of the camera 12 and the Z-axis positional misalignment of the part P1 are corrected by driving only the camera stage 13, but this is not limited to this. Specifically, as long as relative positional correction between the camera 12 and the part P1 can be performed, the relative positional correction between the camera 12 and the part P1 may be performed by driving only the stage 11, or the relative positional correction between the camera 12 and the part P1 may be performed by driving both the stage 11 and the camera stage 13. After the focus position misalignment correction operation, the process returns to step SA1.

[0055] On the other hand, if it is determined in step SA3 that the focus misalignment correction operation is unnecessary (NO in step SA3), the focus misalignment correction operation is terminated and the processing unit 10 moves on to the next operation.

[0056] [Generation of Learning Model] Next, the method by which the learning model generation unit 24 in the control device 20 generates a learning model will be explained using Figure 4. Figure 4 is a diagram showing the flow of learning model generation in the focus position deviation amount correction system 1 according to Embodiment 1.

[0057] As shown in Figure 4, first the processing unit 10 performs a learning data acquisition operation (step SL1). The learning data acquisition operation will now be explained using Figure 5. Figure 5 is a diagram showing the flow of the learning data acquisition operation in the focus position deviation amount correction system 1 according to Embodiment 1.

[0058] As shown in Figure 5, first, the stage 11 and camera stage 13 are moved to the focus position calculation start position (step SL101). The focus position calculation start position is a parameter set in advance within the processing unit 10, and in this embodiment, it is the position where the component P1 placed on the stage 11 is within the imaging field range of the camera 12, and is moved in the negative direction from the camera 12's focus position. The negative direction is the direction in which the distance between the camera 12 and the component P1 in the optical axis direction becomes longer than the camera 12's focus position.

[0059] Next, the camera 12 images the component P1 placed on the stage 11 (step SL102). In this embodiment, the camera images the alignment mark M1 printed on the component P1. There are no particular restrictions on the number of times the component P1 is imaged.

[0060] Next, the operation parameter acquisition unit 22 acquires operation parameters during the learning data acquisition operation, including the focus evaluation value for component P1 calculated from the image captured by the camera 12 (step SL103).

[0061] Here, an example of the operation parameters acquired by the operation parameter acquisition unit 22 is shown in Figure 6. As shown in Figure 6, the operation parameters acquired by the operation parameter acquisition unit 22 include input data and output data. The input data is a group of parameters for estimating the amount of positional misalignment in the optical axis direction between the focus position of the camera 12 and the part P1, and the output data is a group of parameters indicating the amount of positional misalignment in the optical axis direction between the focus position of the camera 12 and the part P1.

[0062] The operation parameter acquisition unit 22 acquires operation parameters as input data from the component P1, stage 11, camera 12, camera stage 13, and processing unit 10 during the focus position misalignment correction operation, in order to estimate the amount of positional misalignment in the optical axis direction between the focus position of camera 12 and component P1. Specifically, as shown in Figure 6, the operation parameters include component recognition data for component P1, device operation data of the processing unit 10, and production parameters.

[0063] The part recognition data includes the calculation result of the focus evaluation value for part P1 obtained from the image captured by camera 12, and the part position of part P1 obtained from the image captured by camera 12. The focus evaluation value for part P1 and the part position of part P1 are calculated from the captured image of the alignment mark M1 on part P1.

[0064] Here, we will explain how to calculate the focus evaluation value and the position of part P1 using Figure 7. Figure 7 is a diagram illustrating how to calculate the focus evaluation value and the position of part P1 using the alignment mark M1 from the image captured by the camera 12. As shown in Figure 7, in this embodiment, the shape of the alignment mark M1 is circular.

[0065] The focus evaluation value and the position of part P1 can be obtained by performing edge detection. In this case, first, the camera 12 captures an image including at least a portion of the alignment mark M1, and the camera 12's processing unit performs edge detection calculation on the captured image to perform edge detection of the alignment mark M1. Specifically, an edge detection point group 30 consisting of multiple edge detection points along the circle of the alignment mark M1 is obtained.

[0066] The graph in Figure 7 schematically shows the results when edge detection calculation is performed in the edge detection region 31. As shown in the graph in Figure 7, when edge detection calculation is performed, the edge strength increases at the boundary between the alignment mark M1 and its surrounding area. The edge strength is calculated from the brightness gradient value (the difference between two adjacent pixels) of a pixel in the image and its surrounding pixels. Specifically, when the pixel with the maximum edge strength exceeds a threshold value set as a calculation parameter, that pixel is detected as an edge location. At this time, the XY coordinates of the pixel detected as an edge location are defined as the edge detection point, and the edge strength of the pixel detected as an edge location is defined as the edge intensity. When edge detection calculation is performed in the edge detection region 31, the boundary parts Pl and Pr of the alignment mark M1 are calculated as edge detection points. By performing this edge detection calculation multiple times while changing the position of the edge detection region 31, an edge detection point group 30 can be obtained.

[0067] The focus evaluation value for component P1 is calculated from the edge intensity of each edge detection point that makes up the edge detection point group 30. For example, the focus evaluation value can be calculated from the average value of the edge intensity of each edge detection point, or the focus evaluation value can be calculated from the maximum or minimum value of the edge intensity of each edge detection point.

[0068] Furthermore, the position of part P1 is calculated from the edge detection point cloud 30. Specifically, the center coordinates of the edge detection point cloud 30 are calculated by fitting the edge detection point cloud 30 to the equation of a circle. These center coordinates become the recognized position Pc of part P1. In this embodiment, the recognized position Pc is the representative coordinate of the alignment mark M1. In other words, the recognized position Pc is one of the part positions of part P1.

[0069] The recognition position Pc (representative coordinates) of the alignment mark M1 does not have to be the coordinates of the center of the circle. Also, the recognition position Pc of part P1 may be any position coordinates related to a mark or reference position detected from the image. For example, if the alignment mark M1 is a cross-shaped mark, the center point of the cross mark may be used as the recognition position Pc of part P1, or the intersection of multiple lines detected from multiple feature points (electrodes, etc.) of part P1 may be used as the recognition position Pc of part P1, or a part of the outer edge of part P1 (for example, a corner of the part) may be used as the recognition position Pc of part P1. In the case of a cross mark, edge detection points may be calculated on each side of the cross mark, the obtained group of edge detection points 30 may be fitted to a line, and the recognition position Pc may be calculated from the intersection and midpoint of the line.

[0070] Furthermore, the method for detecting the edges of the alignment mark M1 is not limited to the method described above; it is sufficient to calculate the edge detection points and the edge strength at the edge detection points from the strength of the edges of the alignment mark M1. For example, edge detection points may be detected by performing an arbitrary filtering process on the captured image, or the feature points of the function may be used as edge detection points by fitting the brightness of each pixel to an arbitrary function, or edge detection points may be calculated by pattern matching and the pattern matching rate may be used as the edge strength. Alternatively, edge detection points may be calculated by differentiating the obtained brightness graph to find the gradient, or the optimal edge detection points may be calculated from the brightness information in the captured image using a learning model.

[0071] Furthermore, the method for calculating the focus evaluation value of component P1 does not necessarily have to be calculated from the edge intensity of the edge detection point group 30. It may also be calculated from the difference between the brightness of the alignment mark M1 and the brightness of the surrounding pixels of the alignment mark M1, or from the integrated value of the high-frequency components extracted from the alignment mark M1 and its surrounding region. According to this method, by edge detection calculation alone, it is possible to calculate the focus evaluation value focusing only on component P1 and the position of component P1 from the image captured by the camera 12.

[0072] According to this embodiment, for example, if the processing device 10 is part of a device that aligns electronic components to a substrate, the focus evaluation value can be calculated simultaneously with the calculation of the component position P1 corresponding to the substrate. This minimizes the processing time required for the focus position deviation correction operation. Furthermore, since the data acquired by the operation parameter acquisition unit 22 includes the focus evaluation value of component P1 and the component position of component P1, the operation parameters can be efficiently calculated in the learning model generation operation by using a method that calculates the focus evaluation value through edge detection calculation.

[0073] Returning to Figure 6, the device operation data of the processing unit 10 includes the operating time data of the processing unit 10. The operating time data may include the elapsed time since the processing unit 10 was powered on, the elapsed time since the processing unit 10 started operation (elapsed time since the processing operation started), the number of processing cycles, the time taken for one processing cycle, the time taken for each step of the processing operation, the elapsed time since an arbitrary predetermined condition in the processing operation was met, and the timing at which each step of the processing operation was executed. The input data to the learning model may include at least one of these. Note that the stage 11, camera 12, and camera stage 13 of the processing unit 10 may deform over time. For example, if the focus evaluation value of a component P1 placed on the stage 11 is imaged by the camera 12 immediately after powering on the processing unit 10 and the focus evaluation value is calculated, the calculated focus evaluation value of component P1 may differ even if the position of component P1 on the stage 11 has not changed, compared to if the component P1 placed on the stage 11 is imaged by the camera 12 after a certain period of time has elapsed since powering on and the focus evaluation value is calculated. This is because even temperature changes at levels difficult to measure with a temperature sensor cause thermal expansion or contraction of each component of the processing unit 10. In a processing unit 10 composed of numerous components, these slight thermal expansions or contractions accumulate, causing the stage 11, camera 12, and camera stage 13 of the processing unit 10 to change over time. Therefore, by including the operating time data of the processing unit 10 in the detection data of the processing unit 10's operating status, it is possible to estimate an appropriate amount of positional displacement for each condition when estimating the amount of positional displacement, even when changes occur over time in the stage 11, camera 12, and camera stage 13.

[0074] The method for measuring the amount of misalignment is not particularly limited, as long as it can measure the amount of misalignment caused by changes over time due to temperature changes inside the processing unit 10. For example, it may be a method to estimate the amount of misalignment from time, or a method to measure the amount of misalignment at a certain point on the stage 11, camera 12, and camera stage 13 using a laser displacement meter and estimate the amount of misalignment of part P1 from the change over time of the measurement results, or a method to measure the strain at a certain point on the stage 11, camera 12, and camera stage 13 using a strain gauge and measure the amount of misalignment from the change over time of the measurement results.

[0075] Furthermore, the device operation data of the processing unit 10 may include time-series temperature change data detected at at least one detection point of the processing unit 10. The time-series change data includes temperature data from a certain time ago at the temperature detection point, difference data between the temperature data from a certain time ago and the current temperature data, and the cumulative value of the time and the amount of temperature change since a temperature change occurred at a certain detection point.

[0076] For example, if there is a temperature change in heat-generating parts such as the actuators of stage 11, camera 12, and camera stage 13, the temperature inside the processing unit 10 may rise over time. Therefore, if the component P1 placed on stage 11 is imaged by camera 12 immediately after a temperature change occurs at a certain detection point and the focus evaluation value is calculated, the calculated focus evaluation value for component P1 may differ even if the temperature at the detection point has not changed. This is because each component expands or contracts due to thermal changes in areas where a temperature sensor is not attached or due to temperature changes inside each component constituting the processing unit 10. Therefore, by including temperature change data over time in the detection data of the operating status of the processing unit 10, it is possible to estimate an appropriate amount of focus shift for each condition when estimating the amount of focus shift even if there is a temperature change inside the processing unit 10. In addition, temperature data when the focus evaluation value exceeds a threshold may be used as a reference, and difference data between that and the current temperature data may also be included. By measuring the current temperature change amount, using the state where the focus evaluation value exceeds a threshold, that is, the state where the focus position of the camera 12 and the position in the optical axis direction of the component P1 coincide, it is possible to estimate the amount of positional deviation from the state where the focus position of the camera 12 and the position in the optical axis direction of the component P1 coincide, due to thermal expansion or contraction of each component caused by temperature changes inside each component constituting the processing device 10, thereby improving the accuracy of estimating the amount of focus positional deviation.

[0077] Furthermore, the production parameters include movement conditions such as axis position, axis movement speed, acceleration, axis movement distance, and axis movement direction for each step of the processing operation in the drive unit of the processing unit 10 (stage 11, camera stage 13); imaging conditions such as the shutter speed, gain of the camera body 12a and the output amount (illumination output amount) of the lighting equipment 12c set when the camera 12 images the part P1; and workpiece conditions such as the shape, material, color, refractive index, and shape of the alignment mark M1 of the part P1 (workpiece) targeted by the processing unit 10. These production parameters may change depending on the product type or production system. By including the production parameters of the processing unit 10 in the input data of the operation parameter acquisition unit 22, even if the production parameters change depending on the product type or production system, it is possible to estimate an appropriate amount of focus deviation for each condition when estimating the amount of focus deviation.

[0078] The output data shown in Figure 6 is a parameter related to the amount of focus shift, and it shows the amount of positional shift in the optical axis direction between the focus position of the camera 12 and the part P1 when the part P1 was imaged in step SA1 (part imaging) in Figure 3. In this embodiment, the position of the camera stage 13 in the Z axis direction is obtained in step SL103, and the amount of positional shift in the Z axis direction between the focus position of the camera 12 and the part P1 is calculated from the data obtained in step SL106. The processing in step SL106 will be described later.

[0079] Returning to Figure 5, after the step of acquiring operation parameters during the learning data acquisition operation (step SL103), it is determined whether the position of the camera stage 13 has reached the focus position calculation operation end position (step SL104). This determination is made, for example, by the operation command unit 21. Specifically, the focus position calculation operation end position is a parameter set in advance within the processing unit 10, and is a position moved in the positive direction from the focus position of the camera 12. The positive direction is the direction in which the distance in the Z-axis direction between the camera 12 and the component P1 becomes shorter than the focus position of the camera 12.

[0080] Next, if it is determined in step SL104 that the camera stage 13 has not yet reached the focus position calculation completion position (NO in step SL104), the camera stage 13 moves at a fixed interval (step SL105). Specifically, the camera stage 13 moves in the positive direction at intervals predetermined within the processing unit 10. A shorter movement interval for the camera stage 13 improves the accuracy of the focus position calculation, but it increases the time required to calculate the focus position. Therefore, it is desirable to set the interval appropriately according to the required focus position calculation accuracy.

[0081] After the camera stage 13 moves, the process returns to step SL102 and image component P1 again.

[0082] On the other hand, if it is determined in step SL104 that the camera stage 13 has reached the position where the focus position calculation operation is complete (YES in step SL104), the process proceeds to the output data calculation step (step SL106).

[0083] The output data calculation operation in step SL106 will be explained using Figure 8. Figure 8 is a diagram showing the relationship between the position of the camera stage 13 when the camera 12 images part P1 while the camera stage 13 moves at regular intervals from the focus position calculation start position to the focus position calculation end position, and the calculated focus evaluation value for part P1.

[0084] Figure 8 shows the relationship between the position of the camera stage 13 and the calculated focus evaluation value for part P1 as a focus evaluation value curve 41 obtained based on the focus evaluation value data group 40. As shown in Figure 5, the focus evaluation value data group 40 is a data group plotted in a two-dimensional coordinate system, where the focus evaluation value acquired by the operation parameter acquisition unit 22 in step SL103 from the image of part P1 captured by the camera 12 in step SL102 in the learning data acquisition flow shown in Figure 5 is used as the vertical axis coordinate, and the position of the camera stage 13 (specifically, the axis position of the camera stage 13) when the camera 12 captured the part P1 is used as the horizontal axis coordinate. The focus evaluation value curve 41 is an approximation curve of the focus evaluation value data group 40.

[0085] In Figure 8, Ps indicates the start position of the focus position calculation operation, and Pe indicates the end position of the focus position calculation operation. The focus evaluation value curve 41 is a curve that has a maximum value (local maximum) between the focus position calculation operation start position Ps and the focus position calculation operation end position Pe. The maximum value of the focus evaluation value is calculated from this focus evaluation value curve 41.

[0086] The position of the camera stage 13 where this focus evaluation value takes its maximum value (i.e., the position FP of the camera 12) is the focus position of the camera 12. The difference between the position of the camera stage 13 at each point in the focus evaluation value data group 40 and the focus position is used as the output data in the learning data acquisition flow SL106. For example, if the position of the camera stage 13 is Pa, the difference value is Δd.

[0087] The output data in the learning data acquisition flow SL106 is a parameter related to the amount of focus position shift, and the calculation method is not particularly limited as long as it is possible to calculate the amount of positional shift in the optical axis direction between the focus position of the camera 12 and the part P1 when the part P1 is imaged in step SA1 (part imaging) in Figure 3. Therefore, the maximum value may be searched from the focus evaluation value data group 40, that position may be set as the focus position, and the difference between the focus position and the acquired data may be used as the output data, or the amount of shift from the focus position calculated in advance by another means may be used as the output data.

[0088] Through the above operations, the data necessary for generating the learning model is acquired, specifically, the part recognition data of the processing unit 10, the device operation data, production parameters, and the amount of positional deviation in the optical axis direction between the focus position of the camera 12 and the part P1. In this embodiment, this operation is repeated multiple times to secure a sufficient amount of data before proceeding to the training dataset generation step. Note that while a larger amount of data improves the estimation accuracy of the learning model, it also increases the time required to generate the training dataset, so it is desirable to set the amount of data appropriately according to the required focus position calculation accuracy.

[0089] Furthermore, it is desirable to change the parameters set during the learning data acquisition operation with each iteration. Specifically, it is advisable to acquire data with different acquisition conditions for the data necessary for generating the learning model, such as the X-axis, Y-axis, Z-axis, and θ-axis positions at the start or end position of the focus position calculation operation, the movement interval of the camera stage 13, and the imaging conditions when imaging part P1. This allows for accurate estimation of the focus position deviation amount under various conditions when estimating the focus position deviation amount in the focus position deviation amount correction flow.

[0090] Returning to Figure 4, after step SL1, a training dataset for the learning model is generated (step SL2). Specifically, the data processing unit 23 formats and processes the data set of operating parameters of the processing unit 10 acquired by the operating parameter acquisition unit 22, and stores the input data and output data as a single training dataset in the storage unit 27. Data processing may include, for example, calculating representative values ​​such as maximum, minimum, median, and mean values ​​from multiple detection data, or calculating statistical values ​​such as the difference between two data points, variance, and standard deviation. By performing such preprocessing, the accuracy of the learning model can be maintained or improved while keeping the number of data points in the training dataset low.

[0091] Next, as shown in Figure 4, a learning model is generated using the learning dataset generated in step SL2 (step SL3). Specifically, the learning model generation unit 24 reads the learning dataset stored in the memory unit 27, and generates a learning model using a learning algorithm such as machine learning, with the input data parameters as inputs and the output data parameters as outputs. As learning algorithms, neural networks (including deep learning using multi-layer neural networks), genetic programming, decision trees, Bayesian networks, support vector machines (SVMs), or linear regression can be used.

[0092] Furthermore, the number of learning models generated does not need to be limited to one per processing unit 10. Additionally, learning models may be added or updated at any time, taking into consideration factors such as the number of training datasets of operating parameters acquired by the operation parameter acquisition unit 22, the estimation accuracy of the learning model, and the time required to generate the learning model. For example, multiple learning models may be generated depending on the type of component P1. Furthermore, the learning model may be updated based on the added training dataset after the operation parameter acquisition unit 22 has acquired new operating parameters. By adding or updating learning models in this way, the accuracy of the learning model can be maintained or improved.

[0093] Next, as shown in Figure 4, the learning model generated in step SL3 is stored in the memory unit 27 (step SL4).

[0094] [Effects, etc.] As described above, in the focus position deviation amount correction method according to this embodiment, input data including the focus evaluation value of the part P1 imaged by the camera 12 is input to a learning model that outputs the relative position deviation amount (i.e., focus position deviation amount) between the focus position of the camera 12 and the position of the part P1, and the position of at least one of the camera 12 and the part P1 is controlled based on the relative position deviation amount output by this learning model. In this embodiment, the position of at least one of the camera stage 13 that holds the camera 12 and the stage 11 on which the part P1 is placed is controlled based on the relative position deviation amount output by the learning model.

[0095] Furthermore, the focus position deviation correction system according to this embodiment includes a stage 11 on which a component P1 is placed, a camera 12 that images the component P1, and a control device 20 that receives input data including the focus evaluation value of the component P1 imaged by the camera 12, inputs the above input data to a learning model that outputs the relative position deviation between the focus position of the camera 12 and the position of the component P1, and controls the position of at least one of the stage 11 and the camera 12 based on the relative position deviation output by this learning model.

[0096] As a result, even if the processing device 10 experiences a shift in the positional relationship between the camera 12, the lens, or the object being imaged (target object) due to heat, causing a change in the camera's focus position, the amount of focus position shift can be corrected quickly and with high accuracy.

[0097] Specifically, in the focus position misalignment correction method and focus position misalignment correction system according to this embodiment, the operation parameter acquisition unit 22 of the control device 20 acquires operation parameters during the focus position misalignment correction operation, including the focus evaluation value of component P1, and the data processing unit 23 shapes and processes the operation parameters to generate input data for the learning model. The focus position misalignment estimation unit 25 inputs the input data generated by the data processing unit 23 into the learning model stored in the storage unit 27, and estimates the relative position misalignment between the focus position of the camera 12 and the position of component P1 in the optical axis direction based on the relative position misalignment amount between the focus position of the camera 12 and the position of component P1 in the optical axis direction output from the learning model.

[0098] As a result, during the focus misalignment correction operation, the camera 12 can correct the focus misalignment by imaging the part P1 once. Therefore, compared to the conventional focus misalignment correction method, which involves moving the camera stage 13 to a position where the distance from the lens tip of the camera 12 to the part P1 is longer than the camera 12's focus position, or moving the camera stage 13 to a position where the distance from the lens tip of the camera 12 to the part P1 is shorter than the camera 12's focus position, and then imaging the camera 12 at regular intervals to determine the focus position and correct the position of the camera stage 13, the time required for focus misalignment correction can be shortened. Furthermore, even if the relative positional relationship between the stage 11 and / or the camera 12 and camera stage 13 changes due to heat, the focus position of the camera 12 and the position of the part P1 in the optical axis direction can be quickly corrected, thereby improving the productivity of the processing unit 10.

[0099] Furthermore, in the focus position deviation correction method and focus position deviation correction system according to this embodiment, the input data to the learning model includes at least the calculation result of the focus evaluation value of component P1.

[0100] This allows the learning model to accurately calculate the positional deviation between the camera 12's focus position and the part P1's position in the optical axis direction, based on the correlation between the distance between the camera 12 and part P1 in the optical axis direction and the focus evaluation value of part P1, which the learning model has previously learned.

[0101] Furthermore, since the focus evaluation value of component P1 is determined from the edge strength calculated from the captured image of alignment mark M1, it is desirable that the edge strength at the edge detection point of alignment mark M1 be included in the input data to the learning model.

[0102] By using the edge strength of the alignment mark M1 as input data, the time required to generate the learning model, the time it takes for the learning model to estimate the amount of focus misalignment (i.e., the relative positional misalignment between the focus position of the camera 12 and the position of component P1 in the optical axis direction), and the amount of data memory required for processing can be minimized compared to the case where the captured image of the alignment mark M1 itself is used as input data. This is because, when the captured image of the alignment mark M1 is used as input data, it is necessary to perform calculations for each pixel of the captured image necessary for generating the learning model and estimating the amount of focus misalignment. However, by using the edge strength of the alignment mark M1 as input data, only the data necessary for estimating the amount of focus misalignment can be input to the learning model, thus minimizing the amount of computation. Therefore, according to this embodiment, the amount of focus misalignment can be corrected while maintaining the productivity of the processing unit 10.

[0103] Furthermore, by using the edge strength at the edge detection point of the alignment mark M1 as input data to the learning model, it is not necessary to use noise, background, or other shapes (such as the outline or pattern of part P1) contained in the captured image of the alignment mark M1, or other shapes that the processing unit 10 does not target for focus position shift correction, in the learning process. This prevents overfitting and a decrease in estimation accuracy. As a result, the estimation accuracy of the focus position shift is improved.

[0104] Furthermore, the edge strength at the edge detection point of the alignment mark M1 is determined from the brightness of the captured image of the alignment mark M1. For this reason, the captured image of the alignment mark M1 may include the brightness of any pixel in the captured image of the alignment mark M1, the contrast value of the image calculated from the brightness, and values ​​calculated from a certain region (for example, values ​​such as mean and variance).

[0105] Furthermore, the correlation between the distance between camera 12 and part P1 in the optical axis direction and the focus evaluation value of part P1 changes depending on the recognition position of part P1. For example, the correlation between the distance between camera 12 and part P1 in the optical axis direction and the focus evaluation value of part P1 at the center of camera 12's field of view may differ from the correlation between the distance between camera 12 and part P1 in the optical axis direction and the focus evaluation value of part P1 at the periphery of camera 12's field of view. Therefore, the input data to the learning model may include the part position of part P1 from the recognition results of part P1 by camera 12.

[0106] Furthermore, regarding the focus evaluation value obtained when the camera 12 images component P1, the relationship between the absolute value of the positional displacement between the camera 12's focus position and the component P1's position in the optical axis direction and the focus evaluation value of component P1 roughly matches regardless of the direction of the positional displacement of component P1 relative to the camera 12's focus position. That is, whether the displacement of component P1 in the optical axis direction relative to the camera 12's focus position is positive or negative, if the absolute value of the displacement of component P1 in the optical axis direction relative to the camera 12's focus position matches, the focus evaluation value may roughly match even if the direction of the positional displacement is different. Therefore, if the input data to the learning model consists only of the focus evaluation value, even if the absolute value of the positional displacement between the camera 12's focus position and the component P1's position in the optical axis direction can be accurately estimated, it may not be possible to accurately estimate the direction of the positional displacement. For this reason, it is desirable that the input data to the learning model include the data described below. This makes it possible to identify the direction of the positional displacement. However, the following explanation does not limit the effect of the data input to the learning model.

[0107] For example, regarding the positional misalignment between the camera 12's focus position and part P1, the input data to the learning model may include at least one of the external dimensions of the feature shape (alignment mark M1) of part P1 and the recognition position of that feature shape. These may change depending on the direction of the positional misalignment between the camera 12's focus position and the part P1's position in the optical axis direction. For example, the external dimensions of the alignment mark M1 of part P1 may differ when the camera 12's focus position and the part P1's position in the optical axis direction change in a negative direction relative to the camera 12's focus position due to thermal strain of the processing unit 10, etc., and when the camera 12's focus position and the part P1's position in the optical axis direction change in a positive direction relative to the camera 12's focus position. Therefore, including the external dimensions of the alignment mark M1 of part P1 in the input data to the learning model allows for accurate estimation of the direction of the positional misalignment between the camera 12's focus position and the part P1's position in the optical axis direction.

[0108] Furthermore, the external dimensions of alignment mark M1 may vary depending on the shape of alignment mark M1. For example, if alignment mark M1 is circular, the external dimensions may include the diameter, area, circumference, length of the major axis, length of the minor axis, and degree of circularity. For example, if alignment mark M1 is a cross mark, the external dimensions may include the area, circumference, length of the vertical and horizontal axes of the lines forming the cross, and line width.

[0109] Furthermore, the device operation data of the processing unit 10 may include, for example, at least one of the following as detection data of the focus position shift amount correction operation status detected at at least one detection point of the processing unit 10: temperature, humidity, and displacement strain at the detection point (measurement point). The detected temperature of the processing unit 10 when thermal strain occurs may be higher than the detected temperature of the processing unit 10 when thermal strain occurs in an ideal state where no thermal strain occurs in the processing unit 10. Therefore, by including temperature measurement data of the processing unit 10 in the detection data of the operating status of the processing unit 10, it is possible to estimate an appropriate focus position shift amount even when the positional relationship between the camera 12 and the part P1 changes due to thermal strain of the stage 11 on which the part P1 is placed, the camera 12, the camera stage 13, etc. The temperature detection points may include the vicinity of heat-generating parts such as actuators in the components constituting the processing unit 10, such as the camera 12 (e.g., camera body 12a), the stage 11, and the camera stage 13, the surface of the stage 11, and the surrounding atmosphere of the processing unit 10. Also, the temperature data does not need to be obtained at one point per detection point; it may be obtained at multiple points. Furthermore, temperature data may be acquired at each step of the focus position shift correction operation, or it may be acquired continuously at regular intervals regardless of the step. Alternatively, temperature data may be acquired as image data using thermography or the like.

[0110] Furthermore, the input data to the learning model may include operating parameters obtained from multiple images. For example, the input data to the learning model may include operating parameters of the processing unit 10 obtained by taking images with the camera 12 while the camera stage 13 has been moved a small distance from the starting position of the focus shift amount correction operation. This makes it possible to more accurately estimate the direction of the focus shift from the change in the amount of focus shift when the camera stage 13 is moved a small distance. In this way, including operating parameters obtained from multiple images in the input data to the learning model can improve the accuracy of estimating the amount of focus shift, but it increases the time required for the focus shift amount estimation operation. Therefore, it is desirable to set appropriate conditions based on the required accuracy of the focus shift amount estimation and the time required for the focus shift amount estimation operation.

[0111] Furthermore, the input data for the learning model may include past input data of the processing unit 10, relative positional shift amounts output by the learning model, and results obtained from past focus position shift amount estimation operations of the processing unit 10. For example, the input data for the learning model may include the operating parameters of the processing unit 10 in an ideal state where no thermal strain occurs, specifically the operating parameters of the processing unit 10 when the focus position of the camera 12 and the position of component P1 in the optical axis direction coincide. The input data for the learning model may also include results obtained from the focus position shift amount estimation operation of the processing unit 10 one cycle prior. By including such data, the focus position shift amount and the focus position shift direction can be estimated based on the amount of change from past operating parameters of the processing unit 10, thereby improving the estimation accuracy of the learning model.

[0112] (Embodiment 2) Next, Embodiment 2 will be described with reference to Figure 9. Figure 9 is a schematic diagram showing the configuration of the focus position misalignment amount correction system 1A according to Embodiment 2.

[0113] As shown in Figure 9, the focus position misalignment correction system 1A according to this embodiment comprises a processing device 10A and a control device 20.

[0114] In this embodiment, the processing unit 10A has a head 14 added to the processing unit 10 in the first embodiment, and furthermore, the structure and mounting direction of the camera 12 are different.

[0115] Furthermore, in the processing device 10A of this embodiment, the objects to be imaged by the camera 12 are two parts: part P1 (first part) and part P2 (second part). In this case, the processing device 10A is, for example, a positioning device or alignment device that positions two parts when aligning two parts. As an example, the processing device 10A is a component mounting machine. In this case, the processing device 10A is part of an alignment device that accurately positions an electronic component, which is one of the two parts (first part), onto a substrate, which is the other of the two parts (second part), and places the electronic component on the substrate, and recognizes the position of the substrate which is the target position.

[0116] The head 14 holds the part P2. The head 14 is configured to be movable relative to the stage 11 on which the part P1 is placed. Specifically, the head 14 and the stage 11 can move relative to each other in the X-axis direction, Y-axis direction, Z-axis direction, and θ-axis direction. In this embodiment, the head 14 has a mechanism that allows it to move in the Z-axis direction, and the stage 11 has a mechanism that allows it to move in the X-axis direction, Y-axis direction, and θ-axis direction.

[0117] Camera 12 comprises a camera body 12a, an optical system 12b including a lens, an illumination device 12c, and a computing device (not shown). Camera 12 is an upper and lower field of view recognition camera, and can recognize the upper field of view and the lower field of view by capturing images with the camera body 12a. Possible structures for recognizing the upper and lower fields of view include a structure using two cameras as the camera body 12a, a first camera for recognizing the upper field of view and a second camera for recognizing the lower field of view, or a structure using two mirrors as one of the optical system 12b, one mirror for capturing the upper field of view and one mirror for capturing the lower field of view, and using one camera as the camera body 12a.

[0118] In this embodiment, the processing device 10A is an alignment device that aligns component P1 and component P2. For example, the processing device 10A is a component mounting machine. In this case, when the processing device 10A calculates the amount of misalignment between component P1 and component P2 based on the image captured by the camera 12, it performs focus misalignment correction in order to maintain a constant recognition accuracy by the camera 12.

[0119] Specifically, in this embodiment, first, in step SA1 of Figure 3, parts P1 and P2 are imaged using the camera 12. Specifically, parts P1 and P2 are imaged by the camera 12 from directly above, with the focus adjusted so that it approximately coincides with the alignment mark M1 of part P1 and the alignment mark M2 of part P2.

[0120] Next, similar to step SA2 in Figure 3, the focus evaluation value for each of component P1 and component P2 is calculated, and similar to step SA3 in Figure 3, it is determined whether correction of the amount of focus misalignment is necessary. In this embodiment, for example, from the focus evaluation values ​​calculated for each of component P1 and component P2, it is determined whether correction is necessary only for component P1, only for component P2, for both component P1 and component P2, or whether correction is unnecessary for both component P1 and component P2.

[0121] Next, the amount of focus misalignment between component P1 and component P2 is estimated in the same manner as in step SA4 of Figure 3. The amount of focus misalignment may be estimated by generating separate learning models for component P1 and component P2 and estimating the amount of focus misalignment from different learning models for component P1 and component P2, or by using learning models that output the amount of focus misalignment for component P1 and component P2 respectively and estimating the amount of focus misalignment from a single learning model.

[0122] Next, the focus misalignment correction operation is performed in the same manner as in step SA5 of Figure 3. The correction amount is calculated based on the focus misalignment amount estimated in step SA4 so that both component P1 and component P2 are in focus. For example, if only component P1's focus position is to be corrected, the head 14 is moved in the optical axis direction; if only component P2's focus position is to be corrected, the stage 11 is moved in the optical axis direction; and if both component P1 and component P2's focus position is to be corrected, the camera 12 and stage 11 are moved. Furthermore, if the correction amounts for component P1 and component P2 are different, the focus position is corrected by operating the head 14, stage 11, and camera stage 13 in combination.

[0123] In this embodiment, the part recognition data among the operation parameters acquired by the operation parameter acquisition unit 22 in step SA4 (estimation of focus misalignment amount) in the alignment method of Figure 3 and step SL103 (acquisition of operation parameters) in the learning data acquisition method of Figure 5 preferably includes the focus evaluation values ​​of part P1 and part P2, respectively, and the production parameters preferably include the imaging parameters for part P1 and the imaging parameters for part P2, respectively.

[0124] As described above, in the focus position misalignment correction system 1A and focus position misalignment correction method according to this embodiment, the camera 12 simultaneously or sequentially images parts P1 and P2, and controls the position of either the camera 12 or parts P1 and P2 so that the camera 12 is in focus on parts P1 and P2 based on the output result of the learning model.

[0125] Specifically, the camera 12, which recognizes the vertical field of view, images parts P1 and P2, and simultaneously estimates and corrects the amount of focus misalignment. This reduces the time required to correct the focus misalignment compared to correcting the focus misalignment for parts P1 and P2 separately. Furthermore, by inputting the operation parameters related to part P2 into the learning model that estimates the focus misalignment of part P1, and the operation parameters related to part P1 into the learning model that estimates the focus misalignment of part P2, the mutual positional misalignment amounts can be incorporated into the learning of the learning models. This improves the accuracy of the estimation of the focus misalignment. This is because the shift in focus is caused by the deformation of the frame and members constituting the processing unit 10A due to heat, which affects the stage 11 on which part P1 is placed, the head 14 that holds part P2, and the camera 12, among others. Therefore, parts P1 and P2 do not necessarily need to be imaged by the same camera 12; it is sufficient if they are imaged within the processing unit 10A.

[0126] (Modification) The focus position misalignment correction system and focus position misalignment correction method related to the present disclosure have been described above based on Embodiments 1 and 2, but the present disclosure is not limited to Embodiments 1 and 2.

[0127] For example, in embodiments 1 and 2 described above, the processing devices 10 and 10A were component mounting machines, but are not limited to these. Specifically, the processing device 10A in embodiment 2 may be a device that processes component P1 by pressing it against the precise position of component P2 using P1 as a mold. In this case, the processing device 10A could be, for example, an imprinting device in which component P1 is an imprint mold having a recessed structure and component P2 is the workpiece to be imprinted. Alternatively, the processing device 10A may not be a processing device such as an imprinting device, but an inspection device in which component P1 is an inspection probe, component P2 is an electronic component, and component P1 is precisely positioned on the electrodes of component P2.

[0128] Furthermore, in Embodiment 1, the object to be imaged was one component P1, and in Embodiment 2, the object to be imaged was two components P1 and P2, but this is not limited to these. For example, there may be three or more objects to be imaged.

[0129] Furthermore, the focus position misalignment correction method in the above embodiments 1 and 2 may be implemented as a computer program implemented by a computer, or as a computer-readable recording medium storing the program. For example, the present disclosure may be a program that causes a computer to execute the focus position misalignment correction method.

[0130] Furthermore, this disclosure also includes forms obtained by applying various modifications to each of the above embodiments that a person skilled in the art could conceive, as well as forms realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of this disclosure. In addition, this disclosure also includes any combination of two or more claims from the multiple claims described in the claims of this application, provided that they are not technically contradictory. For example, if the cited claims described in the claims of this application are made into a multi-claim or multi-multi-claim so as to refer to all of the higher-level claims without technically contradictory, then all combinations of claims included in that multi-claim or multi-multi-claim are also included in this disclosure.

[0131] The technology disclosed herein can be used to adjust the focal position of a camera to match the object being imaged.

[0132] 1, 1A Focus position shift amount correction system 10, 10A Processing unit 11 Stage 12 Camera 12a Camera body 12b Optical system 12c Lighting equipment 13 Camera stage 14 Head 20 Control device 21 Operation command unit 22 Operation parameter acquisition unit 23 Data processing unit 24 Learning model generation unit 25 Focus position shift amount estimation unit 26 Operation control unit 27 Storage unit 30 Edge detection point group 31 Edge detection area 40 Focus evaluation value data group 41 Focus evaluation value curve P1, P2 Parts M1, M2 Alignment mark Pc Recognition position Pl, Pr Boundary area

Claims

1. A method for correcting the amount of focus misalignment between a camera and an object to be imaged so that the object is in focus, the method comprising: inputting input data including the focus evaluation value of the object imaged by the camera to a learning model that outputs a relative position misalignment between the focus position of the camera and the position of the object to be imaged, and controlling the position of at least one of the camera and the object to be imaged based on the relative position misalignment output by the learning model.

2. The method for correcting the amount of focus misalignment according to claim 1, wherein the input data includes the strength of the edges in the characteristic shape of the object to be imaged.

3. The method for correcting the amount of focus misalignment according to claim 1, wherein the input data includes brightness in the characteristic shape of the object to be imaged and in the pixels surrounding the characteristic shape.

4. The method for correcting the amount of focus misalignment according to claim 1, wherein the input data includes at least one of the external dimensions of the feature shape of the object to be imaged and the recognition position of the feature shape.

5. The method for correcting the amount of focus misalignment according to claim 1, wherein the input data includes at least one of the following: temperature, humidity, displacement, and strain at one or more measurement points of the processing device having the method for correcting the amount of focus misalignment.

6. The method for correcting the amount of focus misalignment according to claim 1, wherein the input data includes at least one of the following: the time elapsed since power was turned on to the processing device having the method for correcting the amount of focus misalignment; the time elapsed since the processing device having the method for correcting the amount of focus misalignment started operation; and the time elapsed since any predetermined condition was met in the processing device having the method for correcting the amount of focus misalignment.

7. The method for correcting a focus misalignment according to claim 1, wherein the input data includes the axis position, axis movement speed, and acceleration for each step of the focus misalignment correction operation in the drive unit of the processing apparatus having the focus misalignment correction method, the shutter speed, gain, and illumination output amount in the camera of the processing apparatus having the focus misalignment correction method, and data relating to at least one production parameter of the shape, material, color, and refractive index of the object to be imaged.

8. The method for correcting the amount of focus misalignment according to claim 1, wherein the input data includes at least one of the past input data of the processing device having the method for correcting the amount of focus misalignment and the relative misalignment output by the learning model.

9. The method for correcting the amount of focus misalignment according to claim 1, wherein there are two or more objects to be imaged, the camera images the two or more objects simultaneously or sequentially, and the position of either the camera or one of the two or more objects to be imaged is controlled based on the output result of the learning model so that the camera is in focus on the two or more objects to be imaged.

10. A focus misalignment correction system for aligning a camera with an object to be imaged so that the object is in focus, comprising: a stage on which the object to be imaged is placed; a camera for imaging the object to be imaged; and a control device that controls the position of at least one of the stage and the camera based on the relative misalignment amount output by a learning model, which outputs a relative misalignment amount between the camera's focus position and the position of the object to be imaged when input data including a focus evaluation value of the object imaged by the camera is input.

11. A program that causes a computer to execute the focusing position shift amount correction method described in any one of claims 1 to 9.

Images