Alignment system, alignment method, and recording medium

Abstract

An alignment system (1) has an image processing unit (400) that detects an identification object from a captured image obtained by capturing a workpiece (10) having the identification object by different search methods, namely, a rough search and a detailed search, and a motion control unit (200) that controls a driving instrument (111, 112, 113) that moves the workpiece (10) based on actual coordinates of the identification object detected by the image processing unit (400). The detection of the identification object by the image processing unit (400) and the control of the driving instrument (111, 112, 113) by the motion control unit (400) are repeatedly performed. The detailed search is performed in a search range narrower than that of the rough search, and the next search in the case where the identification object is successfully detected by the detailed search is performed without the rough search but with the detailed search.

Description

Technical Field

[0001] This invention relates to alignment systems, alignment methods, and recording media. Background Technology

[0002] In the field of Factory Automation (FA), alignment techniques are used to align the position of a workpiece, which is the object of control, with a reference position. For example, a control system is used to detect a workpiece from an image captured by an imaging device and align the workpiece by aligning its position with a reference position (e.g., Patent Document 1).

[0003] In the actuator control system described in Patent Document 1, a command for driving the actuator is issued based on the data detected by performing a first position detection process and a second position detection process on the captured image. The first position detection process detects the position of the object with a first precision, and the second position detection process detects the position of the object with a higher precision than the first precision.

[0004] The following describes a scenario where, in this actuator control system, the actuator is preliminarily driven based on intermediate data obtained from the first position detection process, and further driven based on the difference between the final data obtained from the second position detection process and the intermediate data. Thus, drive control begins preliminarily before the final data calculation is completed, enabling high-speed control.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-110916 Summary of the Invention

[0006] In the actuator control system described in Patent Document 1, a coarse search is used as the first position detection process, and a fine search is used as the second position detection process. The purpose of the coarse search is to roughly detect the position of the object from a large-scale image, for example, by using pattern matching. On the other hand, the purpose of the fine search is to accurately detect the position of the object from a small-scale image, for example, by using edge detection.

[0007] Fine-grained searching is based on a small-scale search, thus requiring a defined search range. However, coarse-grained searching, used to define the search range, also suffers from increased processing load and time due to the large range of positions and rotation angles of the target pattern. Therefore, alignment control, which issues control commands based on data detected by both coarse and fine-grained searches, suffers from an overall increase in control processing time.

[0008] The present invention is made in view of the above circumstances, and its object is to provide an alignment system, alignment method and procedure capable of positioning at high speed and with high precision.

[0009] To achieve the above objectives, the alignment system of the present invention is characterized by having: an image processing unit that detects the object to be identified by means of a captured image obtained by capturing a workpiece having an object to be identified through different search methods, namely coarse search and detailed search.

[0010] And a motion control unit, which controls the drive instrument that moves the workpiece based on the actual coordinates of the object detected by the image processing unit. The detection of the object by the image processing unit and the control of the drive instrument by the motion control unit are repeatedly executed. A detailed search is performed within a narrower search range than a coarse search; if the object is successfully detected through a detailed search, the next search will perform a detailed search instead of a coarse search.

[0011] The effects of the invention

[0012] According to the present invention, when an object is successfully detected through a detailed search, the next search will not perform a coarse search but a detailed search, thereby reducing the overall processing time and enabling high-speed and high-precision positioning. Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating an example of the overall structure of the alignment system according to Embodiment 1.

[0014] Figure 2 This is a block diagram illustrating a structural example of a motion control unit.

[0015] Figure 3 This is a block diagram representing a structural example of an image processing unit.

[0016] Figure 4 This is a diagram illustrating the alignment control method.

[0017] Figure 5 This is a diagram illustrating the alignment control method.

[0018] Figure 6 This is a block diagram illustrating a functional structure example of the alignment system involved in Implementation 1.

[0019] Figure 7 This is a flowchart illustrating the alignment control process involved in Implementation 1.

[0020] Figure 8 This is a flowchart illustrating the search process involved in Implementation Method 1.

[0021] Figure 9AThis is a diagram representing an example of a search range.

[0022] Figure 9B This is a diagram illustrating an example of a search angle range.

[0023] Figure 10 This is a diagram illustrating the synchronization method based on transmission path delay measurement.

[0024] Figure 11 This diagram illustrates the timing control of the shutter of the shooting device.

[0025] Figure 12 This is a block diagram illustrating a functional structure example of the alignment system involved in Implementation Method 2.

[0026] Figure 13 This is a flowchart illustrating the alignment control process involved in Implementation Method 2.

[0027] Figure 14 This is a flowchart illustrating the search process involved in Implementation Method 2.

[0028] Figure 15 This is a block diagram illustrating the functional structure of the alignment system involved in Variation Example 1.

[0029] Figure 16 This is a flowchart illustrating the alignment control process involved in Variation Example 1.

[0030] Figure 17 This is a flowchart illustrating the search process involved in Variation Example 1.

[0031] Figure 18A This is a diagram illustrating the prediction of the identified object involved in Variation Example 3.

[0032] Figure 18B This is a diagram illustrating the prediction of the identified object involved in Variation Example 3.

[0033] Figure 19A This is a diagram illustrating the coarse search involved in Variation Example 4.

[0034] Figure 19B This is a diagram illustrating the detailed search involved in Variation Example 4.

[0035] Figure 20A This is a diagram illustrating the rough search involved in Variation Example 5.

[0036] Figure 20B This is a diagram illustrating the detailed search involved in Variation Example 5.

[0037] Figure 21 This is a flowchart illustrating the search process involved in Variation Example 6. Detailed Implementation

[0038] (Implementation Method 1)

[0039] Hereinafter, Embodiment 1 for implementing the present invention will be described in detail with reference to the accompanying drawings. Furthermore, the same or corresponding parts in the drawings are labeled with the same reference numerals.

[0040] Figure 1 This is a block diagram illustrating an example of the overall structure of the alignment system 1 according to Embodiment 1. The alignment system 1 includes: an alignment mechanism 100 that moves a workpiece 10, which is the object of control; an action control unit 200 that controls the alignment mechanism 100; an imaging device 300 that captures images of the workpiece 10; an image processing unit 400 that processes the images captured by the imaging device 300; and a setting terminal 500 that performs various settings used in the processing of the image processing unit 400.

[0041] The alignment mechanism 100 is connected to drive instruments 111, 112, and 113, which provide driving forces in various directions to move the workpiece 10. Additionally, drive instruments 111, 112, and 113 are connected to drive control devices 121, 122, and 123, respectively. Based on control signals from the motion control unit 200, drive control devices 121, 122, and 123 respectively drive drive instruments 111, 112, and 113.

[0042] The drive control devices 121, 122, and 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are communicatively connected to each other. The communication unit can be any conventional communication unit such as Ethernet, CameraLink, CoaxPress, or USB (Universal Serial Bus), but to ensure synchronization, an industrial network based on Ethernet such as CC-Link IE / field or CC-Link IE / TSN is preferred.

[0043] The alignment mechanism 100 includes a mounting stage 101 for placing the workpiece 10 and a mechanism for moving the mounting stage 101. For example, the mounting stage can be translated in the horizontal X and Y directions, which are orthogonal to each other, and it can also be rotated in the θ direction, which is a rotational direction on the horizontal plane. In this embodiment, the driving instrument 111, which is coupled to the alignment mechanism 100, translates in the X direction, the driving instrument 112 translates in the Y direction, and the driving instrument 113 rotates in the θ direction.

[0044] The driving instruments 111, 112, and 113 are any driving instruments (actuators) capable of precisely driving the alignment mechanism 100, such as servo motors. The drive control devices 121, 122, and 123 are control devices that control the driving of the driving instruments 111, 112, and 113 respectively based on control signals from the motion control unit 200, such as servo amplifiers.

[0045] The drive instruments 111, 112, and 113 have arbitrary position sensors, either internally or externally, that detect and output the actual changes in position caused by the drive instruments 111, 112, and 113. The position sensor may be, for example, an encoder attached to the actuator. The encoder's output signal is input to the motion control unit 200 via drive control devices 121, 122, and 123. In this embodiment, the case where the drive instruments 111, 112, and 113 are servo motors, the drive control devices 121, 122, and 123 are servo amplifiers, and the position sensor is an encoder will be described.

[0046] The motion control unit 200 is an action controller that provides the drive control devices 121, 122, and 123 with commands related to the actions of the drive instruments 111, 112, and 113. It may include, for example, a PLC (Programmable Logic Controller). Based on information obtained from the drive control devices 121, 122, and 123 and the image processing unit 400, the motion control unit 200 generates control signals that display the commands and outputs them to the drive control devices 121, 122, and 123.

[0047] like Figure 2 As shown, the motion control unit 200 includes a processor 210, a volatile memory 220, a non-volatile memory 230, a clock 240, and a communication interface 250. The processor 210, volatile memory 220, non-volatile memory 230, clock 240, and communication interface 250 are interconnected via bus B1.

[0048] The processor 210, for example, is a CPU (Central Processing Unit), which reads and executes the control program 231 stored in the non-volatile memory 230, thereby functioning as a position information generation unit 211, a movement calculation unit 212, and an instruction unit 213.

[0049] The position information generation unit 211 of the processor 210 generates position information corresponding to the position of the mounting stage 101 on which the workpiece 10 is placed, based on the output signals of the position sensors of the drive instruments 111, 112, and 113. The movement calculation unit 212 calculates the movement of the workpiece 10 in the X, Y, and θ directions based on the position of the identified object detected by the image processing unit 400 from the captured image. The instruction unit 213 outputs a control signal based on the movement calculated by the movement calculation unit 212 to the drive control devices 121, 122, and 123.

[0050] Volatile memory 220 is a working memory capable of high-speed data reading and writing during arithmetic operations performed by processor 210, such as RAM (Random Access Memory). Non-volatile memory 230 stores control program 231 and control data 232. Control program 231 is used to implement various functions of motion control unit 200, and control data 232 includes parameters used during the execution of control program 231, past detection data, and instruction data. Non-volatile memory 230 is, for example, non-volatile semiconductor memory such as EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic disk, or optical disk.

[0051] Clock 240 measures the local time by counting the clock signals of the clock element in motion control unit 200, and then synchronizes it with drive control devices 121, 122, 123, imaging device 300, and image processing unit 400. Thus, motion control unit 200 has time information synchronized with drive control devices 121, 122, 123, imaging device 300, and image processing unit 400.

[0052] The communication interface 250 is the interface used by the motion control unit 200 to communicate with the drive control devices 121, 122, 123, the shooting device 300, and the image processing unit 400. For example, it is an interface that conforms to communication standards such as CC-LinkIE / field and CC-LinkIE / TSN.

[0053] The imaging device 300 is an imaging device that takes pictures of the workpiece 10 from above the alignment mechanism 100 at constant time intervals. For example, it is a camera with sufficient resolution to achieve the alignment accuracy of the alignment system 1. The number of imaging devices 300 is arbitrary, but is determined according to the number and position of the objects to be identified for alignment. Figure 1The case shown has two imaging devices 300. The object to be identified can be any object indicating the position of the workpiece 10, such as an alignment mark attached to the workpiece 10, a corner of the workpiece 10, or a corner of the stage 101.

[0054] The shooting device 300 also has time information synchronized with the drive control devices 121, 122, 123, motion control unit 200, and image processing unit 400, as well as a communication interface for communicating with the drive control devices 121, 122, 123, motion control unit 200, and image processing unit 400.

[0055] The image processing unit 400 performs object detection processing on the captured image obtained from the imaging device 300 using two different search methods: coarse search and detailed search. The coarse search is performed over the entire range of the captured image obtained from the imaging device 300, while the detailed search is performed over a smaller range than the coarse search. When the image processing unit 400 detects an object through the detailed search, it outputs the actual coordinates of the object to the motion control unit 200. Here, the actual coordinates are coordinates obtained from the reference coordinates of the motion calculation unit 212 of the motion control unit 200.

[0056] like Figure 3 As shown, the image processing unit 400 includes a processor 410, a volatile memory 420, a non-volatile memory 430, a clock 440, and a communication interface 450. The processor 410, volatile memory 420, non-volatile memory 430, clock 440, and communication interface 450 are interconnected via bus B2.

[0057] The processor 410, for example, is a CPU (Central Processing Unit), which reads and executes the control program 431 stored in the non-volatile memory 430, thereby functioning as an image acquisition unit 411, a range determination unit 412, and a search unit 413.

[0058] The image acquisition unit 411 of the processor 410 acquires images captured by the imaging device 300. The range determination unit 412 determines the search range for the target object in the captured images acquired by the image acquisition unit 411 based on information including the position information generated by the position information generation unit 211 of the motion control unit 200. The search unit 413 performs a detailed search for the target object in the image within the search range determined by the range determination unit 412, and outputs the actual coordinates of the target object to the motion control unit 200 when the target object is successfully detected.

[0059] Volatile memory 420 is a working memory capable of high-speed data reading and writing during arithmetic operations performed by processor 410, such as RAM (Random Access Memory). Non-volatile memory 430 stores control program 431 and control data 432. Control program 431 is used to implement various functions of image processing unit 400, and control data 432 includes parameters used during the execution of control program 431 and past detection data. Non-volatile memory 430 is, for example, non-volatile semiconductor memory such as EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic disk, or optical disk.

[0060] Clock 440 measures the local time by counting the clock signals of the clock element in image processing unit 400, and then synchronizes it with drive control devices 121, 122, 123, motion control unit 200, and imaging device 300. Thus, image processing unit 400 has time information synchronized with drive control devices 121, 122, 123, motion control unit 200, and imaging device 300.

[0061] The communication interface 450 is the interface used by the image processing unit 400 to communicate with the drive control devices 121, 122, 123, the motion control unit 200, and the shooting device 300. For example, it is an interface that conforms to communication standards such as CC-LinkIE / field and CC-LinkIE / TSN.

[0062] The setting terminal 500 is a terminal, such as a personal computer, on which the application program corresponding to the control program 431 of the image processing unit 400 is installed. The setting terminal 500 has management functions for the image processing unit 400, including the input or modification of parameters stored in the non-volatile memory 430 of the image processing unit 400. The communication interface of the setting terminal 500 for communicating with the image processing unit 400 is any interface corresponding to the interface of the image processing unit 400, such as a USB interface or an RS232C interface.

[0063] The motion control unit 200 calculates the amount of movement based on the difference between the actual coordinates of the object being identified and the target coordinates of the target being aligned, as detected by the image processing unit 400, and outputs control signals based on the amount of movement to the drive control devices 121, 122, and 123.

[0064] The operation of alignment system 1, which has the structure described above, will be explained. First, using... Figure 4 , 5 An example of an alignment method based on the position of the identified object is given. Figure 4 , 5 This diagram illustrates the alignment method when the object to be identified is an alignment mark 1001 or 1002 attached to the end of the workpiece 10.

[0065] The shapes of alignment marks 1001 and 1002 are arbitrary, but preferably they are shapes that can clearly determine the position and rotation angle of the reference point. For example, preferably... Figure 4 , 5 The crosshairs are shown. Furthermore, the number of alignment marks is arbitrary and can be as follows: Figure 4 The image shows one, or you can see it as shown below. Figure 5 As shown, two alignment marks 1001 and 1002 are attached to the diagonal ends of workpiece 10.

[0066] When using one alignment mark 1001, an image of the shooting range 1300 is acquired by one shooting device 300. When using two alignment marks 1001 and 1002 and the two alignment marks 1001 and 1002 cannot be captured within one shooting range 1300, images of shooting ranges 1300 that are staggered are acquired by two shooting devices 300.

[0067] In such Figure 4 When using one alignment mark 1001, the following three pieces of information are used: the coordinates of the reference point of the alignment target mark 1101, the coordinates of the reference point of the alignment mark 1001 on the workpiece 10, and the angle difference between the alignment target mark 1101 and the alignment mark 1001.

[0068] On the other hand, in such Figure 5 When using two alignment marks 1001 and 1002, the following three pieces of information are used: the midpoint coordinates of the reference points of the alignment target marks 1101 and 1102, the midpoint coordinates of the reference points of the alignment marks 1001 and 1102 on the workpiece 10, and the angle difference between the straight line connecting the alignment target marks 1101 and 1102 and the straight line connecting the alignment marks 1001 and 1002.

[0069] exist Figure 4 , 5 In this case, the angle difference Δθ is always the amount of movement along the θ-axis. Therefore, as... Figure 4As shown, the motion control unit 200 calculates the virtual coordinates of the reference point of the virtual mark 1201 after the workpiece 10 has been rotated by an angle Δθ around the rotation center 1200 of the alignment mechanism 100 along the θ-axis. Then, the movement amount ΔX in the X-axis direction and the movement amount ΔY in the Y-axis direction are calculated by the difference between the virtual coordinates and the target coordinates of the alignment target mark 1101. The control signals for achieving the movements Δθ, ΔX, and ΔY obtained in this way are sent by the command unit 213 of the motion control unit 200 to the drive control devices 121, 122, and 123, thereby issuing commands.

[0070] like Figure 4 As shown, the method using a single alignment mark 1001 can be used in situations where the rotation of the alignment mark 1001 can be clearly determined, such as with a crosshair. On the other hand, as... Figure 5 As shown, when using two alignment marks 1001 and 1002, it is not necessary to detect the rotation of the alignment marks themselves. Therefore, even with alignment marks such as circles, the rotation of the workpiece can be detected. Furthermore, by detecting two sufficiently far apart points and calculating the angular difference Δθ in the θ direction, high-precision control is possible.

[0071] Thus, the motion control unit 200 calculates Δθ, ΔX, and ΔY based on the coordinates and angle differences of the reference points of the identified object, i.e., the alignment marks 1001 and 1002. The drive control devices 121, 122, and 123 use these values ​​to control the drive instruments 111, 112, and 113 to move the workpiece 10. However, usually, it is not possible to make the difference between the workpiece and the target coordinates fall within the allowable range in one go, so the detection of the identified object and the control of the drive instruments 111, 112, and 113 are repeatedly executed.

[0072] In this repetitive process, the burden of searching for the object to be identified from the captured images obtained by the imaging device 300 is very heavy. Therefore, the alignment system 1 according to this embodiment limits the range of the search to reduce the processing burden.

[0073] The following uses Figure 6-8 The detailed processing steps are explained below. Figure 6 This is a block diagram illustrating a functional structure example of the alignment system 1 according to this embodiment. Figure 7 This is a flowchart of the alignment control process executed by the motion control unit 200. Figure 8 This is a flowchart of the search process performed by the image processing unit 400.

[0074] First, the motion control unit 200 issues a command to move the workpiece 10 to a pre-set estimated target position. Figure 7(Step S101). Specifically, the motion control unit 200 outputs control signals to the drive control devices 121, 122, and 123 to achieve movement in each direction. Furthermore, through the control of the drive control devices 121, 122, and 123 based on the control signals, the instruments 111, 112, and 113 are driven to move the platform 101 on which the workpiece 10 is placed.

[0075] After the movement is completed, the motion control unit 200 instructs the image processing unit 400 to acquire and search for images (step S102), and generates and outputs position information based on the output of the position sensors of the driving instruments 111, 112, and 113 (step S103). Then, the motion control unit 200 waits until the image processing unit 400 finishes its search (step S104).

[0076] In step S102, the image processing unit 400, which was instructed by the motion control unit 200 to perform the search process for the identified object, is executed. Figure 8 The process is as shown. Here, since it is the first time taking a picture (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 acquires the image captured by the imaging device 300 (step S202). Then, the search unit 413 performs a coarse search (coarse retrieval) of the identified object (step S203).

[0077] For example, a coarse search is performed using pattern matching processing implemented by using a pattern model of a pre-registered alignment mark 1001. The search unit 413 detects alignment marks 1001 when the pattern matching consistency rate is greater than or equal to a pre-defined threshold, and determines the search range for a detailed search based on the position or rotation angle of the reference point of the alignment mark 1001 (step S204). The shape or size of the search range at this time is preset according to the shape or size of the alignment mark 1001, or it can be set by input from the user to the setting terminal 500.

[0078] For example, in Figure 9A In the case of the crosshair alignment mark 1001 shown, the position of the reference point, i.e., the center, and the rotation angle of the crosshair are determined by pattern matching, and the search range 1301 of the size after adding a pre-defined margin to the size of the alignment mark 1001 is determined. Additionally, it is also determined that... Figure 9B The angle shown refers to the search range (search angle range).

[0079] Next, the search unit 413 performs a detailed search (fine retrieval) on the image within the search range determined in step S204 (step S205). As a detailed search, for example, edge detection is performed to detect accurate straight lines or curves, thereby obtaining more accurate actual coordinates of the identified object. Here, the actual coordinates of the identified object include the position (XY coordinates) and rotation angle (θ coordinates) of the object's reference point.

[0080] If the search unit 413 fails to detect the object in the detailed search of step S205 (step S212: No), it returns to step S201 and repeats the processing of steps S202 to S205. On the other hand, if the object is successfully detected in the detailed search of step S205 (step S212: Yes), the motion control unit 200 outputs the actual coordinates of the object obtained in step S205 (step S213).

[0081] Here, return Figure 7 The flowchart shows that after the search processing performed by the image processing unit 400 is completed (step S104: Yes), the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference between the actual coordinates and the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to a threshold (step S106: Yes), the alignment control processing ends. Here, the target coordinates are the coordinates of the aligned target, including the center position (XY coordinates) and the rotation angle (θ coordinates).

[0082] If the difference between the actual coordinates and the target coordinates exceeds a threshold (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107: instruction step). Specifically, control signals are output to the drive control devices 121, 122, and 123 to make the actual coordinates consistent with the target coordinates.

[0083] After the driving instruments 111, 112, and 113 have finished driving, return to step S102. The motion control unit 200 again instructs to take pictures and search (step S102) and outputs position information (step S103).

[0084] Transferred again Figure 8 The flowchart shows that since the image was captured on the second or subsequent time and the previous detection was successful (step S201: No), the image processing unit 400 obtains the location information (step S206). The range determination unit 412 of the image processing unit 400 calculates the predicted coordinates of the object to be identified at that time point (step S207).

[0085] Specifically, the range determination unit 412 determines the correspondence between the position information obtained from the motion control unit 200 during the initial alignment control and the actual coordinates detected during the initial detailed search. For example, when the XY coordinates of the initial position information are represented by (x1, y1) and the XY coordinates of the actual coordinates are represented by (X1, Y1), their relationship is represented by the constant matrix A of the following formula (1).

[0086] [Mathematical Expression 1]

[0087] A = (X1, Y1) / (x1, y1)...(1)

[0088] If the XY coordinates of the position information obtained from the motion control unit 200 during the second alignment control are set to (x2, y2), then the XY coordinates (X2, Y2) of the predicted coordinates used in the second search can be represented by the constant matrix A expressed by Equation (1) using Equation (2). Furthermore, the same applies to the nth search, including the third search and subsequent searches, using Equation (3).

[0089] [Mathematical Expression 2]

[0090] (X2,Y2)=A(x2,y2)…(2)

[0091] (X n Y n )=A(x n y n )…(3)

[0092] Furthermore, the constant matrix A representing the correspondence between position information and actual coordinates can be updated each time based on the position information and actual coordinates obtained in repeated alignment control, or it can be an average of multiple times. Alternatively, the correspondence between position information and actual coordinates can be pre-built through prior calibration. For example, in the case of alignment control using the corner of the stage 101 as the identification object, the error in each execution is small, so such prior calibration is effective.

[0093] Thus, the correspondence between the XY coordinates and position information of the reference point of the object to be identified detected from the captured image is predetermined, and the XY coordinates are predicted based on the newly acquired position information using this correspondence (step S207). Then, a pre-defined search range is determined with the predicted XY coordinates as the center (step S208). Similarly, for the θ coordinate, based on the predetermined correspondence between the actual coordinates detected through detailed search and the position information, the θ coordinate representing the rotation angle of the object to be identified is predicted based on the newly acquired position information (step S207), and a pre-defined search angle range is determined with the predicted θ coordinates as the center (step S208). Furthermore, the search range may not be centered on the predicted coordinates; depending on the alignment conditions or the shape of the object to be identified, the range including the predicted coordinates may also be used as the search range.

[0094] Here, the size of the search range can be set by the user through input to the setting terminal 500, or it can be set automatically. For example, the size of the search range can be a size with a margin added to the shape or size of the object to be identified, which is set automatically or manually based on the moving speed of the driving instruments 111, 112, and 113. Alternatively, the size of the search range can be statistically determined based on past alignment control results, for example, it can be a size with a margin added to the shape and size of the object to be identified, which is the average difference between the predicted coordinates calculated from the position information at corresponding time points in the past and the actual coordinates detected from the captured images.

[0095] That is, if the range determination unit 412 performs the coarse search in step S203 for the second and subsequent times, and if the previous detection did not fail, it will not perform the coarse search in step S203, but will determine the search range centered on the predicted coordinates calculated in step S207 based on the position information output by the position sensor. The processing of determining the search range based on this position information is significantly reduced compared to the processing of determining the search range obtained by the coarse search performed in step S203. Therefore, high-speed control can be achieved compared to the previous alignment control performed after each coarse search.

[0096] Next, the image acquisition unit 411 acquires an image (step S209; image acquisition step), and performs simplified processing on the image within the search range determined in step S208 (step S210). Simplified processing is any processing performed before a detailed search. For example, the search unit 413 can perform a coarse search within a range defined by the entire range larger than the search range determined in step S208, and then determine the search range again. Alternatively, if the predicted XY coordinates are calculated in step S207, and the search range in the XY coordinates is determined in step S208, a coarse search of the θ coordinate can also be performed as simplified processing to determine the search angle range. Furthermore, simplified processing can be omitted.

[0097] Then, the search unit 413 performs a detailed search on the search range determined in step S208 or the range determined in step S210 (step S211: search step). The method for detailed searching is the same as in step S205. If the result of performing detailed searching in step S211 is that no object can be detected (step S212: No), the process returns to step S201. Since the previous detection failed (step S201: Yes), the processing of steps S202 to S205 is performed. Here, restarting from a coarse search of the entire range is to avoid becoming uncontrollable due to repeated detection failures in the detailed search.

[0098] If the object is successfully detected in the detailed search in step S211 (step S212: Yes), the motion control unit 200 outputs the actual coordinates of the object obtained in step S211.

[0099] Return again Figure 7 The flowchart shows that the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference between the actual coordinates and the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to a threshold (step S106: Yes), the alignment process ends.

[0100] If the difference between the actual coordinates and the target coordinates exceeds a threshold (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107). Specifically, control signals are output to the drive control devices 121, 122, and 123 to make the actual coordinates consistent with the target coordinates. Then, the process returns to step S102 to continue.

[0101] As explained above, the alignment system 1 according to this embodiment includes an image processing unit 400 and a motion control unit. The image processing unit 400 detects the object to be identified from an image captured of a workpiece 10 having the object to be identified. The motion control unit controls the driving instruments 111, 112, and 113 that move the workpiece 10 based on the actual coordinates of the object to be identified detected by the image processing unit 400. The image processing unit 400 predetermines the correspondence between the position information output by the position sensors of the driving instruments 111, 112, and 113 and the actual coordinates of the object to be identified detected from the captured image. Using this correspondence, it calculates the predicted coordinates of the object to be identified based on the position information subsequently obtained, determines the search range including the predicted coordinates, and detects the object to be identified based on the image within the search range. As a result, the coarse search processing of the entire range can be omitted, and positioning can be performed at high speed and with high accuracy.

[0102] (Implementation Method 2)

[0103] Hereinafter, Embodiment 2 for implementing the present invention will be described in detail with reference to the accompanying drawings. Furthermore, the same or corresponding parts in the drawings are labeled with the same reference numerals.

[0104] The overall structure of the alignment system 2 and the hardware structure of each component in this embodiment 2 are the same as those in embodiment 1. The difference between the alignment system 2 in this embodiment 2 and that in embodiment 1 is that the image acquisition unit 411 of the image processing unit 400 acquires captured images from the imaging device 300 by issuing a forwarding instruction specifying a forwarding range, and the search unit 413 performs search processing on captured images within the forwarding range.

[0105] In this embodiment, similar to Embodiment 1, the drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are communicatively connected to each other. However, this embodiment seeks higher responsiveness than Embodiment 1, and therefore preferably uses industrial networks such as CC-LinkIE / field or CC-LinkIE / TSN for communication connection.

[0106] Industrial networks such as CC-LinkIE / field and CC-LinkIE / TSN employ the following mechanisms to ensure synchronization: timing is achieved to a certain degree through H / W instruments, and transmission path delay is statistically measured to align the timing between instruments with a precision of μ seconds.

[0107] Synchronization based on transmission path delay measurement is a method that achieves higher accuracy synchronization by utilizing the measured transmission path delay from the master station to the device station. Figure 10This illustrates a synchronization method based on transmission path delay measurement. In this method, synchronization occurs at a synchronization point. The transmission control (MyStatus) frames sent from the master station 40 experience delays as the distance increases.

[0108] Master station 40 calculates the transmission path delay time of each device station 50, 60, and 70 based on the master station time at which it receives response signals from each device station 50, 60, and 70, and sends the calculation to each device station 50, 60, and 70. The synchronization point is the time after a constant time (Tsync) has elapsed after master station 40 sends the transmission control (MyStatus) frame. After receiving the transmission control (MyStatus) frame, each device station 50, 60, and 70 synchronizes after subtracting the transmission path delay time, i.e., the Tps time (Tsync - delay time).

[0109] In this embodiment, the timing control of the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400 is performed using the transmission path delay measurement method, for example, with the motion control unit 200 as the master station 40. That is, the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400 are interconnected through an industrial network (communication line) that measures the transmission path delay time, and thus have synchronized timing information.

[0110] The motion control unit 200, acting as the master station 40, sets a specific moment after the longest transmission path delay time from the indicated transmission time as the synchronization point. Furthermore, the image processing unit 400 determines the forwarding range, including the predicted coordinates at a future specific moment, and forwards the images within the forwarding range of the captured images at the specified moment, as instructed by the imaging device 300.

[0111] In addition, timing control of shutter speed was also considered for the shooting device 300. Figure 11 This diagram illustrates the timing control of the shutter of the imaging device 300. Synchronization time information is obtained using a synchronization point calculation source, and the imaging device 300 automatically determines the shooting timing based on information obtained from the image processing unit 400 or pre-set information. At this time, in order to shoot at a specific future time, the image processing unit 400 can issue a preliminary shooting instruction within a grace period as needed. Furthermore, aligning the scheduled shooting timing with the center of the exposure time range is also useful.

[0112] Figure 12 This is a block diagram illustrating an example of the functional structure of the alignment system 2 according to this embodiment. Figure 13This is a flowchart of the alignment control process executed by the motion control unit 200. Figure 14 This is a flowchart of the search process performed by the image processing unit 400. According to... Figure 13 , 14 The flowchart illustrates the operation of alignment system 2.

[0113] First, the motion control unit 200 issues a command to move the workpiece 10 to a pre-set estimated target position. Figure 13 (Step S101). Specifically, the motion control unit 200 outputs control signals to the drive control devices 121, 122, and 123 to realize movement in each direction. Furthermore, through the control of the drive control devices 121, 122, and 123 based on the control signals, the drive instruments 111, 112, and 113 are driven, and the platform 101 on which the workpiece 10 is placed moves.

[0114] Following the movement command, the motion control unit 200 determines the shooting time of the shooting device 300 as time T. n (n=1) (step S122), the position information is specified to the image processing unit 400. Additionally, the position information generation unit 211 of the motion control unit 200 generates position information based on the output of the position sensors on the drive instruments 111, 112, and 113, and outputs the position information (step S103). Then, the motion control unit 200 waits until the image processing unit 400 finishes its search (step S104).

[0115] The image processing unit 400, which has obtained position information from the motion control unit 200, executes... Figure 14 The process is as shown. Here, since it is the first shot (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 instructs the shooting device 300 on time T. n The image is captured at point (n=1), and the captured image of the entire range is forwarded (step S221). Then, the image acquisition unit 411 acquires the captured image of the entire range (step S222).

[0116] Next, the search unit 413 of the image processing unit 400 performs a coarse search (coarse retrieval) of the captured images across the entire range (step S203). For example, a coarse search is performed using pattern matching processing implemented using a pre-registered pattern model of the alignment mark 1001. The search unit 413 detects the alignment mark 1001 as the object of recognition when the pattern matching consistency rate is greater than or equal to a pre-defined threshold, and determines the search range for detailed search based on the position or rotation angle of the reference point of the alignment mark 1001 (step S204). The shape or size of the search range at this time is predetermined based on the shape or size of the alignment mark 1001, or it can be set by input from the user to the setting terminal 500.

[0117] Next, a detailed search (fine retrieval) is performed on the image within the search range determined in step S204 (step S205). The detailed search, for example, involves edge detection to detect accurate straight lines or curves, thereby obtaining more accurate actual coordinates of the identified object. Here, the actual coordinates of the identified object include the position (XY coordinates) and rotation angle (θ coordinates) of the object's reference point.

[0118] If no object is detected during the detailed search in step S205 (step S212: No), the process returns to step S201 and repeats steps S221, S222, and S203 to S205. On the other hand, if an object is successfully detected during the detailed search in step S205 (step S212: Yes), the motion control unit 200 outputs the actual coordinates of the object obtained in step S205 (step S213).

[0119] return Figure 13 The flowchart shows that after the image processing unit 400 completes its search process (step S104: Yes), the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference between the actual coordinates and the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to a threshold (step S106: Yes), the alignment control process ends. Here, the target coordinates are the coordinates of the aligned target, including the center position (XY coordinates) and the rotation angle (θ coordinates).

[0120] If the difference between the actual coordinates and the target coordinates exceeds a threshold (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107: instruction step). Specifically, control signals are output to the drive control devices 121, 122, and 123 to make the actual coordinates consistent with the target coordinates.

[0121] After issuing a calibration command to the alignment mechanism 100, the motion control unit 200 returns to step S122 and determines the shooting time as time T. n (n=2)(step S122), output position information (step S103), wait until the image processing unit 400 finishes its search processing (step S104).

[0122] Return again Figure 14 The flowchart shows that the next shot is the second shot and thereafter, and the previous detection was successful (step S201: No). Therefore, the image processing unit 400 indicates time T to the shooting device 300. n The image is captured at point (n=2) (step S223). Additionally, the image processing unit 400 obtains position information from the motion control unit 200 (step S206). The range determination unit 412 of the image processing unit 400 predicts the coordinates of the object to be identified at that time point (step S207).

[0123] The method for predicting coordinates is the same as in Embodiment 1, determining the correspondence between the position information obtained from the motion control unit 200 during the initial alignment control and the actual coordinates detected during the initial detailed search. The range determination unit 412 uses the determined correspondence to predict the XY coordinates of the reference point of the identified object based on the position information obtained in step S206 (step S207). Then, the range determination unit 412 determines a pre-defined forwarding range centered on the XY coordinates predicted in step S207 and issues a forwarding instruction to the imaging device 300 for an image with the specified forwarding range (step S226). Furthermore, the forwarding range may not be centered on the predicted XY coordinates; depending on the alignment conditions or the shape of the identified object, the range including the predicted XY coordinates may be used as the forwarding range.

[0124] Here, the size of the forwarding range can be set by the user through input to the setting terminal 500. For example, there is a trade-off between robustness and processing speed, so the forwarding range can be set small while prioritizing productivity by allowing for processing deviations and maximizing the average processing speed. On the other hand, the forwarding range can be set large when jitter characteristics are prioritized over processing speed and processing time is kept constant.

[0125] Alternatively, the forwarding range can be a size with a margin added to the shape or size of the object to be identified. This margin is set automatically or manually based on the moving speed of the driving instruments 111, 112, and 113. Alternatively, the range determination unit 412 can statistically determine the size of the forwarding range based on past alignment control results. For example, the size of the forwarding range can be the size of the object to be identified plus a margin, which is the average difference between the predicted coordinates calculated based on past position information and the actual coordinates of the object to be identified detected from the image.

[0126] Thus, in cases where the image capture is performed for the second time or subsequent times and the previous detection did not fail, the capturing device 300 does not forward the entire range of images. Instead, it only forwards the images within the forwarding range centered on the predicted coordinates based on the position information from the position sensor, obtained in step S226. The data volume of these forwarded images is significantly reduced compared to the data volume of the entire range of images forwarded in step S221. Therefore, compared to forwarding the entire range of images each time, the data forwarding volume can be reduced, the forwarding time is shortened, and high-speed control can be achieved since a coarse search of the entire range is not performed.

[0127] The image acquisition unit 411 acquires the image of the forwarding range forwarded by the forwarding instruction issued in step S226 (step S209; image acquisition step), and the search unit 413 performs simplified processing on the image within the forwarding range (step S210). Simplified processing is arbitrary processing before performing a detailed search; for example, a coarse search can be performed on the image within the forwarding range acquired in step S209 to determine the search range. Specifically, a coarse search of the θ coordinate can be performed as simplified processing to determine the search angle range. Alternatively, simplified processing can be omitted.

[0128] Subsequently, a detailed search is performed on the images within the forwarding range obtained in step S209 or on the search range determined in step S210 (step S211: search step). The method for the detailed search is the same as in step S205. If the result of the detailed search in step S211 is that no object can be detected (step S212: No), the process returns to step S201. Since the previous detection failed (step S201: Yes), steps S221, S222, and S203-205 are executed. Here, restarting from a coarse search of the entire range of images is to avoid becoming uncontrollable due to repeated detection failures in the detailed search.

[0129] If the object is successfully detected in the detailed search in step S211 (step S212: Yes), the motion control unit 200 outputs the actual coordinates of the object obtained in step S211 (step S213).

[0130] Return again Figure 13 The flowchart shows that the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference between the actual coordinates and the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to a threshold (step S106: Yes), the alignment process ends.

[0131] If the difference between the actual coordinates and the target coordinates exceeds a threshold (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107). Specifically, control signals are output to the drive control devices 121, 122, and 123 to make the actual coordinates consistent with the target coordinates. Then, the process returns to step S122 to continue.

[0132] In this way, the range determination unit 412 of the image processing unit 400 determines the forwarding range of the image based on the location information, and the search unit 413 performs a detailed search on the image forwarded from the imaging device 300. As a result, the forwarding time and search time can be significantly shortened.

[0133] In addition to the method of specifying shooting parameters including the forwarding range together with the shooting instruction, the forwarding range from the image processing unit 400 to the shooting device 300 can also be specified to the shooting device 300 in advance, and then the shooting trigger is output to the shooting device 300.

[0134] When the forwarding range is specified together with the former shooting instruction, the timing of the shooting by the shooting device 300 and the acquisition of location information are synchronized by using the aforementioned transmission path delay measurement synchronization method, thereby reducing waiting time and performing appropriate searches within the forwarding range. On the other hand, when the forwarding range is specified in advance for the latter, the shooting timing can be kept consistent with high precision by outputting the shooting trigger only through a dedicated line.

[0135] In addition, Figure 14 The image forwarding of the entire range specified in step S221 and the image forwarding of the forwarding range specified in step S226 can also be forwarded by the imaging device 300 by downscaling the image. The downscaling method can be any conventional method, such as using downsampling or binning. In particular, binning has the effect of simplifying processing and improving the signal-to-noise ratio of pixel signals. It can suppress the reduction in position recognition accuracy compared to the reduction in resolution caused by downscaling, and is therefore preferred.

[0136] Alternatively, the following control can be implemented: when limiting (adjusting) the forwarding range via the forwarding instruction in step S226, if the actual coordinates are far from the target coordinates in the early stages of alignment control, the range can be reduced along with the limitation of the forwarding range; once the distance between the actual coordinates and the target coordinates approaches and falls within a certain range, the reduction can be stopped. Alternatively, when reducing the image across the entire range or within the forwarding range, a model matching the reduced data can be prepared in advance as a pattern model for searching, and the search unit 413 can perform a coarse search or a detailed search using this model.

[0137] Alternatively, images of the entire range can be acquired in parallel with the detailed search (step S211) following the acquisition of images of the forwarding range via the forwarding instruction in step S226. This reduces the time required to acquire images of the entire range when the identified object cannot be detected in the detailed search of step S211 (step S212: No). In this case, forwarding of the entire range of images can be stopped or the forwarded images can be deleted when the identified object is successfully detected via the detailed search.

[0138] As explained above, in the alignment system 2 of this embodiment, the image processing unit 400 pre-determines the correspondence between the position information of the driving instruments 111, 112, and 113 and the actual coordinates of the object to be identified detected from the captured image. Using this correspondence, it calculates the predicted coordinates of the object to be identified based on the position information subsequently acquired, determines the forwarding range of the image containing the predicted coordinates, and performs a detailed search of the image within the forwarding range forwarded from the capturing device 300 to detect the object to be identified. This reduces forwarding time and the size of the forwarded images. Furthermore, it eliminates the need for a coarse search of the entire image range, thus enabling high-speed and high-precision alignment control.

[0139] (Modified Example)

[0140] The above-described embodiments can be modified in various ways. The following describes some variations.

[0141] [Variation Example 1]

[0142] In embodiments 1 and 2, the range determination unit 412 of the image processing unit 400 determines the range containing the predicted coordinates as the search range or the forwarding range. These predicted coordinates are calculated based on position information output by the position sensors of the driving instruments 111, 112, and 113. However, the search range or the forwarding range can also be determined using other methods. This modified example 1 describes other methods for determining the search range or the forwarding range.

[0143] In this variation, the instruction information related to driving the instruments 111, 112, and 113 is used to calculate the predicted coordinates used to determine the search range or forwarding range. Specifically, the range determination unit 412 of the image processing unit calculates the predicted coordinates based on the instruction information and determines the search range or forwarding range that includes the predicted coordinates.

[0144] In this modified example, higher speed control can be achieved by ensuring the synchronous shooting of the imaging device 300, the driving of the driving instruments 111, 112, and 113 based on command information, and the timing control of obtaining the output of the position sensor. Figure 15 This is a block diagram illustrating the functional structure of the alignment system 3 involved in this variation. Figure 16 This is a flowchart of the alignment control process performed by the motion control unit 200 involved in this variation. Figure 17 This is a flowchart of the search process performed by the image processing unit 400.

[0145] Here, Figure 15-17 This is a variation of Implementation 2, illustrating a method for determining the image forwarding range based on instruction information. However, as a variation of Implementation 1, the method for determining the image search range based on instruction information can also be implemented. Hereinafter, according to... Figure 16 , 17 The flowchart illustrates the operation of the alignment system 3 involved in this variation, but the explanation of the same processing as in embodiment 2 is omitted.

[0146] First, the motion control unit 200 issues a command to move the workpiece 10 to a pre-set estimated target position. Figure 16 (Step S101). In order to determine the forwarding range, the instruction information at this time is sent to the image processing unit 400. Then, based on the control signals involved in the instruction information, the drive instruments 111, 112, and 113 start to drive, and the stage 101 on which the workpiece 10 is placed moves.

[0147] Following the movement command, the motion control unit 200 determines the shooting time of the shooting device 300 as a specific future time T. n (n=1) (step S122), the image processing unit 400 is specified. Additionally, the position information generation unit 211 of the motion control unit 200 specifies time T based on the output of the position sensors of the driving instruments 111, 112, and 113. n The location information is recorded (step S123). Then, the motion control unit 200 waits until the image processing unit 400 finishes its search (step S104). The subsequent operation of the image processing unit 400 is the same as in Embodiment 2, executing... Figure 17The processing of steps S221, 222, 203-205, 212, and 213.

[0148] return Figure 16 The flowchart shows that when the search processing of the image processing unit 400 ends (step S104: Yes), the movement calculation unit 212 of the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and the position information generation unit 211 generates the position information based on time T. n The position sensor outputs position information and records it (step S124). The movement calculation unit 212 calculates the difference between the actual coordinates and the target coordinates, and if the difference between the actual coordinates and the target coordinates is less than or equal to a threshold (step S106: Yes), the alignment control process ends.

[0149] If the difference between the actual coordinates and the target coordinates exceeds a threshold (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100, at which time the specific time T recorded in step S124 is referenced. n The location information at the location is used to execute the instruction, and the offset between the location indicated by the instruction and the actual location indicated by the location information is also compensated (step S125).

[0150] After executing the command sent to the alignment mechanism 100 via the command unit 213, the process returns to step S122 and sets the shooting time as time T. n (n=2)(step S122), specify a specific time T n The location information (n=2) is recorded (step S123) and waited until the search processing of the image processing unit 400 is completed (step S104).

[0151] Transferred again Figure 17 The flowchart shows that the next shot is the second shot and thereafter, and the previous detection was successful (step S201: No). Therefore, the image processing unit 400 indicates time T to the shooting device 300. n The image is captured at point (n=2) (step S223). Additionally, the image processing unit 400 obtains time T from the motion control unit 200. n The instruction information at the location is received (step S224). The range determination unit 412 of the image processing unit 400 calculates the predicted coordinates of the object to be identified at that time point (step S225).

[0152] The method for calculating the predicted coordinates performed by the range determination unit 412 is as follows: First, the correspondence between the command information during the initial alignment control and the actual coordinates detected in the initial detailed search is determined, replacing the position information of Embodiment 2. The range determination unit 412 uses the determined correspondence to calculate the predicted coordinates based on the specific time T obtained in step S224. n The predicted coordinates (XY coordinates) of the reference point of the identified object are calculated based on the instruction information related to the predetermined position (step S225). The prediction method using the correspondence relationship is the same as in embodiment 2.

[0153] Then, the range determination unit 412 determines a pre-defined forwarding range centered on the XY coordinates predicted in step S225, and instructs the imaging device 300 to forward the image within the determined forwarding range (step S226). Furthermore, the forwarding range may not be centered on the predicted XY coordinates; depending on the alignment conditions or the shape of the object being identified, the range including the predicted XY coordinates may also be used as the forwarding range.

[0154] The image acquisition unit 411 acquires the image within the forwarding range that was forwarded by the forwarding instruction issued in step S226 (step S209) and performs simplified processing (step S210). Then, the search unit 413 performs a detailed search on the image within the forwarding range acquired in step S209 or on the image that underwent simplified processing in step S210 (step S211). If the result of the detailed search performed in step S211 is that no object can be detected (step S212: No), the process returns to step S201. Since the previous detection failed (step S201: Yes), the processing steps S221, S222, and S203-205 are executed.

[0155] If the identified object is successfully detected in the detailed search in step S211 (step S212: Yes), the motion control unit 200 outputs the actual coordinates of the identified object obtained in step S211 (step S213). The subsequent processing of the motion control unit 200 is the same as in embodiment 2.

[0156] In this way, the range determination unit 412 of the image processing unit 400 determines the forwarding range of the image based on the instruction information of the motion control unit 200, and the search unit 413 performs a detailed search on the image forwarded from the imaging device 300. Therefore, by determining the search range or forwarding range based on instruction information related to a future predetermined position, more efficient timing control and faster alignment control can be achieved.

[0157] [Variation Example 2]

[0158] This variation 2 describes another method for determining the search range or forwarding range. In embodiments 1 and 2, the range determination unit 412 determines the search range or forwarding range centered on the predicted coordinates calculated based on location information, but in this variation, the search range or forwarding range centered on the actual coordinates detected previously is determined.

[0159] Specifically, the range determination unit 412 does not acquire position information or instruction information, but determines the range centered on the reference point of the identified object stored in the non-volatile memory 230 as the forwarding range. This identified object was previously detected through a detailed search performed by the search unit 413 of the image processing unit 400. Alternatively, the range determination unit 412 determines the range centered on the stored reference point of the identified object or the angular range centered on the rotation angle of the identified object as the search range. Furthermore, the forwarding range or search range may not be centered on the actual coordinates detected previously. Depending on the alignment conditions or the shape of the identified object, the range including the actual coordinates detected previously may also be used as the forwarding range or search range.

[0160] The size of the forwarding range or search range can also be determined based on the moving speed of the actual coordinates of the previously detected identification object. Alternatively, it can be determined based on the setting parameters of the alignment control, based on the moving speed of each driving instrument. For example, it can be set to a size determined by multiplying the shooting interval of the shooting device 300 by the maximum moving speed. When the shooting interval is set short and the moving distance between shooting intervals is short, the search range or forwarding range can be sufficiently defined by utilizing the previous detection results, thus simplifying the process compared to Embodiments 1, 2, and Modification 1.

[0161] [Variation Example 3]

[0162] This variation 3 describes another method for determining the search range or forwarding range. In this variation, the range determination unit 412 determines the search range or forwarding range based on the trajectory of the actual coordinates detected in the past through a detailed search performed by the search unit 413. That is, instead of obtaining location information or command information as in embodiments 1 and 2, the predicted coordinates are calculated based on the trajectory of the actual coordinates of the identified object detected in the past and stored in the non-volatile memory 230 through a detailed search performed by the search unit 413 of the image processing unit 400, and the range centered on the predicted coordinates is determined as the search range or forwarding range.

[0163] Specifically, the range determination unit 412 determines the forwarding range as a range centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identified object and having a predetermined size. Alternatively, the range determination unit 412 determines the search range as a range centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identified object and having a predetermined size, or an angular range centered on the θ coordinates predicted based on the trajectory of the θ coordinates. Figure 18A Figure B illustrates the prediction of the identified object. Furthermore, the forwarding range or search range may not be centered on the predicted coordinates; depending on the alignment conditions or the shape of the identified object, the range including the predicted coordinates may also be used as the forwarding range or search range.

[0164] The predicted coordinates (XYθ coordinates) are calculated using the actual coordinates at two or more previous times, predicted by the imaging device 300 during the shooting process. For example, ... Figure 18A As shown, linear prediction is performed based on the XY coordinate data from the previous two times (the first and the time before that) to predict the XY coordinates at the time of shooting. Alternatively, as... Figure 18B As shown, a second-order prediction is performed based on the XY coordinate data from the previous, the time before that, and the time before that, to predict the XY coordinates at the time of shooting.

[0165] The size of the search range or forwarding range can be determined based on the moving speed of the actual coordinates of previously detected objects. Alternatively, it can be determined based on the moving speed of the driving instruments 111, 112, and 113. For example, it can be set to a size determined by multiplying the shooting interval of the imaging device 300 by the maximum moving speed. Linear prediction has lower prediction accuracy than second-order or higher-order prediction, therefore, it is necessary to increase the size of the search range or forwarding range.

[0166] Furthermore, the size of the search range or forwarding range can be determined based on statistical information about the errors of past detection results. For example, it can also be the size of the object being identified, plus a margin, which is the average difference between the predicted coordinates of the trajectory based on actual coordinates at corresponding points in the past and the actual coordinates detected from the captured image.

[0167] According to this variation, when the shooting interval is long and the moving distance between shooting intervals is long, the prediction accuracy can be improved and the search range or forwarding range can be narrowed.

[0168] [Variation Example 4]

[0169] In embodiments 1 and 2, the case where the identification object is the alignment mark 1001, 1002 attached to the workpiece 10 is described, but the processing can also be changed depending on the shape of the identification object. Figure 19A This diagram illustrates the rough search involved in this variation. Figure 19B This is a diagram illustrating the detailed search involved in this variation.

[0170] For example, in workpiece 12 is as follows Figure 19A In the case of an IC (Integrated Circuit) with multiple terminals, where the identification target is the IC's terminals, the search unit 413 performs pattern matching processing using the pattern model 1312 to conduct a coarse search. Figure 8 (Step S203).

[0171] At this time, the search unit 413 determines the search range 1314 of the detailed search based on the position or rotation angle of the pattern 1313 when the pattern matching consistency rate is greater than or equal to a predetermined threshold. Figure 8 (Step S204). In this pattern matching, such as Figure 19A As shown, sometimes multiple patterns with a consistency rate greater than or equal to a predefined threshold are detected. In this case, the pattern with the highest consistency rate 1313 can be determined to determine the search range 1314.

[0172] exist Figure 19B In the example shown, the search unit 413 determines the search range 1314 as the search range for a detailed search. Figure 8 (Step S204) The search range 1314 has a predetermined positional relationship with respect to the pattern 1313 determined by the coarse search. The search unit 413 performs a detailed search on the search range determined in step S204. Figure 8 (Step S205).

[0173] Here, in embodiments 1 and 2, if it is the second or subsequent capture and the previous detection was successful ( Figure 8 Step S201: No), determine the search range including the predicted coordinates calculated based on the location information. Figure 8 Step S208), perform a detailed search. Figure 8 (Step S211). However, in situations where it is possible to... Figure 19A In the case where the IC shown calculates the predicted coordinates with a certain precision, it is also possible to skip the calculation of predicted coordinates based on position information and instead repeat the detailed search using the search range determined in the coarse search in step S203. Furthermore, the process of determining the search range obtained from the coarse search can be performed only if the detection performed through the detailed search fails.

[0174] According to this variation, processing can be simplified based on the shape of the object being identified. Furthermore, the shape of the object can be categorized by setting the application on terminal 500, and a search processing method suitable for that category can be automatically selected.

[0175] [Variation Example 5]

[0176] This variation 5 illustrates the case where the object to be identified presents a further different form. Figure 20A This diagram illustrates the rough search involved in this variation. Figure 20B This diagram illustrates the detailed search involved in this variation. In this variation, a coarse search model registration and a detailed search logic generation are automatically performed by executing a dedicated application program on the processor of the setting terminal 500.

[0177] For example, in workpiece 13 is as follows Figure 20A In the case of the lens shown, the processor of the setting terminal 500 extracts the shape of the lens from the captured image and registers the pattern model 1322. Furthermore, the processor automatically generates detailed search logic based on the shape of the pattern model 1322. For example, if the processor of the setting terminal 500 detects that the shape of the pattern model 1322 includes an arc, such as... Figure 20B As shown, the following logic is automatically generated as the logic for detailed search: search for edges, approximate the detected edges with circles, and use the center of the circle as the reference point 1323 for detection.

[0178] The image processing unit 400 uses a coarse search pattern model and detailed search logic generated by the application program of the setting terminal 500 to perform the search processing described in embodiments 1 and 2. According to this variation, the most suitable coarse search and detailed search can be performed based on the shape of the object being identified. In addition, the pre-setting for performing the search processing of the image processing unit 400 can be automated, reducing the burden on the user to input settings into the setting terminal 500.

[0179] [Variation Example 6]

[0180] In embodiments 1 and 2, a coarse search is performed on the initial captured image, followed by a detailed search within a search range determined by the results of the coarse search. The predicted coordinates of the identified object are calculated based on the results of the detailed search, and this identified object is used to determine the search range or forwarding range for subsequent detailed searches. However, if the prediction accuracy of the identified object's coordinates is sufficiently high, the coarse search of the initial captured image can be omitted through prior calibration of the alignment control. In particular, when alignment control is implemented using the corner of the mounting stage 101 on which the workpiece 10 is placed, the error in each execution is small, thus such prior calibration is effective.

[0181] Figure 21 This is a flowchart of the search process omitting the coarse search of the initial captured image. After the image acquisition unit 411 acquires the initial captured image (step S301), the range determination unit 412 calculates the predicted coordinates of the object to be identified based on the position information using prior calibration information (step S302). The range determination unit 412 determines the search range including the calculated predicted coordinates (step S303), and performs a detailed search within the determined search range (step S304).

[0182] If the actual coordinates of the object are successfully detected in the detailed search in step S304 (step S305: Yes), the actual coordinates are output to the motion control unit 200 (step S309), and the motion control unit 200 uses these actual coordinates to control the drive instruments 111, 112, and 113. After the workpiece 10 moves, a detailed search is performed on the next captured image within the same search range as the previous one. Thus, if the object is successfully detected through a detailed search, a coarse search is not performed; instead, the detailed search is repeated.

[0183] When performing a detailed search repeatedly, only when the detection of the object fails (step S305: No), a coarse search is performed (step S306) to determine the search range (step S307), and a detailed search is performed within the determined search range (step S308), outputting the actual coordinates (step S309). Thus, according to this variation, a detailed search is performed repeatedly based on prior calibration information, and a coarse search is performed only when the object cannot be detected. This simplifies the process and enables high-speed alignment control.

[0184] Furthermore, the hardware structure and flowchart shown in the above embodiments and variations are examples and can be arbitrarily changed and modified. For example, in the above embodiments and variations, the motion control unit 200, the imaging device 300, the image processing unit 400, and the setting terminal 500 are independent structures, but at least two of them can also be integrated into one structure.

[0185] Alternatively, the search processes described in the above embodiments and variations can be arbitrarily combined and executed. Furthermore, it is also possible to enable the selection of any of the search processes involved in the above embodiments and variations via the setting terminal 500. Alternatively, the image processing unit 400 may automatically select any of the search processes involved in the above embodiments and variations based on conditions such as the type of workpiece and the shape of the object being identified.

[0186] Furthermore, the division of functions implemented by the processors in the motion control unit 200 and the image processing unit 400 in the above-described embodiments and variations is just one example, and the division can be arbitrarily changed. Additionally, the functions implemented by the processors of the motion control unit 200 or the image processing unit 400 can be implemented using a conventional computer system without relying on a dedicated system.

[0187] Alternatively, the program for performing the actions described above can be stored in a computer-readable recording medium such as a CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical Disc), or memory card, and then distributed and installed on a computer, thereby constructing a computer capable of performing each function. Furthermore, when functions are implemented through a sharing of responsibilities between the operating system (OS) and applications, or through cooperation between the OS and applications, only the parts other than the OS may be stored on the recording medium.

[0188] This invention can be implemented and modified in various ways without departing from its broad spirit and scope. Furthermore, the above-described embodiments are illustrative of the invention, not limiting its scope. That is, the scope of the invention is defined not by the embodiments, but by the claims. Moreover, various modifications implemented within the scope of the claims and their equivalents are considered to fall within the scope of this invention.

[0189] Explanation of the label

[0190] 1, 2, 3 Alignment system; 10, 12, 13 Workpiece; 11 Alignment target; 40 Master station; 50, 60, 70 Equipment stations; 100 Alignment mechanism; 101 Platform; 111, 112, 113 Driving instruments; 121, 122, 123 Driving control device; 200 Motion control unit; 210 Processor; 211 Position information generation unit; 212 Movement calculation unit; 213 Instruction unit; 220 Volatile memory; 230 Non-volatile memory; 231 Control program; 232 Control data; 240 Clock; 250 Communication interface; 300 Imaging device; 4 00 Image processing unit, 410 Processor, 411 Image acquisition unit, 412 Range determination unit, 413 Search unit, 420 Volatile memory, 430 Non-volatile memory, 431 Control program, 432 Control data, 440 Clock, 450 Communication interface, 500 Setting terminal, 1001, 1002 Alignment marks, 1101, 1102 Alignment target marks, 1200 θ-axis rotation center, 1201 Virtual mark, 1300 Shooting range, 1301, 1314 Search range, 1312, 1322 Pattern model, 1313 Pattern, 1323 Reference point.

Claims

1. An alignment system having: The image processing unit detects the object to be identified by taking a picture of a workpiece with an object to be identified using different search methods, namely coarse search and detailed search. The motion control unit controls the driving instrument that moves the workpiece based on the actual coordinates of the identified object detected by the image processing unit. as well as The terminal is configured to configure the image processing unit. The detection of the object to be recognized by the image processing unit and the control of the driving instrument by the motion control unit are repeatedly executed. The detailed search is performed within a narrower search range than the coarse search. If the identified object is successfully detected through the detailed search, the next search will not perform the coarse search but will instead perform the detailed search. The setting terminal pre-extracts the shape of the workpiece from the captured image and registers the pattern model used in the coarse search. Based on the shape of the pattern model, it generates the search logic for the detailed search. The coarse search includes a pattern matching process that matches the pattern of the workpiece with the pattern model registered through the designated terminal. The detailed search is performed using the search logic of the detailed search generated by the set terminal based on the shape of the pattern model used in the pattern matching process of the coarse search.

2. The alignment system according to claim 1, wherein, When the terminal registers the pattern model used in the coarse search pattern matching process, if it detects that the shape of the pattern model contains an arc, it generates the following search logic as the logic for the detailed search: It searches for edges, approximates the detected edges with circles, and uses the center of the circle as the reference point for detecting the object. The image processing unit performs the detailed search using the search logic generated by the designated terminal.

3. The alignment system according to claim 1, wherein, If the image processing unit successfully detects the object to be identified through the detailed search, it performs the next detailed search within the search range determined by the actual coordinates of the object to be identified obtained through the detailed search.

4. The alignment system according to claim 1, wherein, If the image processing unit successfully detects the object to be identified through the detailed search within the search range determined by the coarse search, it performs the next detailed search on the search range determined by the detailed search.

5. The alignment system according to claim 1, wherein, If the image processing unit is unable to detect the actual coordinates of the object to be identified through the detailed search, the next search will be performed within the search range determined by re-executing the coarse search.

6. The alignment system according to claim 1, wherein, The image processing unit performs the detailed search within a search range determined by prior calibration. If the object to be identified cannot be detected by the detailed search, it performs the detailed search within a search range determined by the coarse search.

7. The alignment system according to claim 1, wherein, The coarse search determines the scope of the detailed search based on the position or rotation angle of the pattern when the consistency rate of the pattern matching process is greater than or equal to a predefined threshold.

8. The alignment system according to claim 1, wherein, The coarse search determines the scope of the detailed search based on the position or rotation angle of the pattern with the highest consistency rate when there are multiple patterns with a consistency rate greater than or equal to a predefined threshold.

9. An alignment method comprising the following steps: The search step involves detecting the identified object from an image obtained by photographing a workpiece with the object to be identified through a coarse or detailed search; and In the instruction step, based on the actual coordinates of the identified object detected in the search step, an instruction is issued to the driving instrument that moves the workpiece. Repeat the search step and the instruction step. If the identified object is successfully detected through the detailed search in the search step, the next search will not perform the coarse search but will instead perform the detailed search. The shape of the workpiece is extracted from the captured image in advance, and the pattern model used in the coarse search is registered. The search logic for the detailed search is generated based on the shape of the pattern model. The coarse search includes a pattern matching process that matches the pattern of the workpiece with a pre-registered pattern model. The detailed search is performed using the search logic generated based on the shape of the pattern model used in the pattern matching process of the coarse search.

10. A recording medium storing a program readable by a computer for enabling a computer to function as a search unit in an alignment system, the alignment system detecting a workpiece having an identification object from an image taken of the workpiece, performing alignment control of the workpiece based on the actual coordinates of the detected identification object, and the computer performing image processing on the image taken in the alignment system. The search unit detects the object to be identified from the captured image through a coarse search or a detailed search. If the object to be identified is successfully detected through the detailed search, the next search will not perform the coarse search but will instead perform the detailed search. The shape of the workpiece is extracted from the captured image in advance, and the pattern model used in the coarse search is registered. The search logic for the detailed search is generated based on the shape of the pattern model. The coarse search includes a pattern matching process that matches the pattern of the workpiece with a pre-registered pattern model. The detailed search is performed using the search logic generated based on the shape of the pattern model used in the pattern matching process of the coarse search.

Images