Alignment system, alignment procedure and program

Abstract

An alignment system (1) comprises an image processor (400) for detecting an identification target in a captured image of a workpiece (10) containing the identification target by means of a coarse search and a fine search, which are distinct from one another, and an operating controller (200) for controlling a drive (111, 112, 113) to move the workpiece (10), the control being based on the actual coordinates of the identification target detected by the image processor (400). The detection of the identification target by the image processor (400) and the control of the drive (111, 112, 113) by the operating controller (200) are performed repeatedly. The fine search is performed in a search area that is smaller than the area used for the coarse search. If the identification target is detected by the fine search, the image processor (400) then performs the fine search as a subsequent search without performing the coarse search.

Description

Technical area

[0001] The present disclosure relates to an alignment system, an alignment method and a program. background

[0002] In the field of factory automation (FA), alignment techniques are used to align a workpiece or a control target to a reference position. For example, a control system detects a workpiece in an image captured by an imaging device and aligns the workpiece to a reference position (as in Patent Literature 1, for example).

[0003] Patent Literature 1 describes an actuator control system in which commands for driving actuators are provided based on data acquired by both a first position detection and a second position detection performed on the captured image. The first position detection detects the position of a target object with a first accuracy, and the second position detection detects the position of the target object with an accuracy greater than the first accuracy.

[0004] This actuator control system drives the actuators in advance based on intermediate data obtained from the first position detection and further drives the actuators based on a difference between the intermediate data and the final data obtained from the second position detection. As described therein, this allows the drive control to be started in advance before the calculation of the final data is completed, thereby increasing the control speed. Citation listPatent literature

[0005] Patent Literature 1: Disclosure of Unexamined Japanese Patent Application No. 2020-110916 Overview of the inventionTechnical problem

[0006] In the actuator control system described in Patent Literature 1, the first position detection includes a coarse search, and the second position detection includes a fine search. The coarse search is performed using, for example, pattern matching to detect the position of the target object in a large area of ​​an image. The fine search is performed using, for example, edge detection to precisely detect the position of the target object in a limited area of ​​an image.

[0007] The fine search is typically performed within a limited area and thus involves a restriction of the search range. The coarse search, which is performed to restrict the search range, can also result in a high search processing load and take a long time if the ranges of the target pattern's position and rotation angle are large. Therefore, the orientation control, which provides control commands based on data acquired by both the coarse search and the fine search, can have a long total processing time.

[0008] Under such circumstances, it is an object of the present disclosure to provide an alignment system, an alignment method and a program that enable fast and precise positioning. Solution to the problem

[0009] To achieve the above object, an alignment system according to one aspect of the present disclosure includes an image processor for detecting an identification target in a captured image of a workpiece including the identification target through a coarse search and a fine search that are different from each other, and an operation controller for controlling a drive for moving the workpiece, wherein the control is performed based on actual coordinates of the identification target detected by the image processor. The detection of the identification target by the image processor and the control of the drive by the operation controller are repeatedly performed. The fine search is performed in a search range smaller than a range for the coarse search. When the identification target is detected through the fine search, the image processor performs the fine search as a next search without performing the coarse search. Advantageous effects of the invention

[0010] The alignment system according to the above aspect of the present disclosure performs the fine search as the next search without performing the coarse search when the identification target is detected by the fine search, thus reducing the overall processing time and enabling fast and precise positioning. Short description of the drawings Fig. 1 is a block diagram of an alignment system according to Embodiment 1, showing the overall structure; Fig. 2 is a block diagram of an operation controller according to an example; Fig. 3 is a block diagram of an image processor according to an example; Fig. 4 is a diagram showing an alignment control method; Fig. 5 is a diagram showing an alignment control method; Fig. 6 is a functional block diagram of the alignment system according to Embodiment 1; Fig. 7 is a flowchart of an alignment control process in Embodiment 1; Fig. 8 is a flowchart of a search process in Embodiment 1; Fig. 9A is a diagram of an example search area; Fig. 9B is a diagram of an example angle search range; Fig. 10 is a diagram showing synchronization based on transmission path delay measurement; Fig. 11 is a diagram showing a timing control for an aperture in an imaging device; Fig. 12 is a functional block diagram of an alignment system according to Embodiment 2; Fig. 13 is a flowchart of an alignment control process in Embodiment 2; Fig. 14 is a flowchart of a search process in Embodiment 2; Fig. 15 is a functional block diagram of an alignment system according to a modification 1; Fig. 16 is a flowchart of an alignment control process in modification 1; Fig. Figure 17 is a flowchart of a search process in modification 1; Fig. 18A is a diagram showing a prediction of an identification target in Modification 3; Fig. 18B is a diagram showing a prediction of an identification target in Modification 3; Fig. Figure 19A is a diagram showing a coarse search in modification 4; Fig. Figure 19B is a diagram showing a fine search in modification 4; Fig. Figure 20A is a diagram showing a coarse search in modification 5; Fig. Figure 20B is a diagram showing a fine search at modification 5; and Fig. Figure 21 is a flowchart of a search process in modification 6. Description of the embodimentsEmbodiment 1

[0011] Hereinafter, Embodiment 1 of the present disclosure will be described in detail with reference to the drawings. Similar reference numerals denote similar or corresponding components in the drawings.

[0012] Fig. 1 is a block diagram of an alignment system 1 according to Embodiment 1, showing the overall structure. The alignment system 1 includes an aligner 100 that moves a workpiece 10 as a control target, an operation controller 200 that controls the aligner 100, imaging devices 300 that capture images of the workpiece 10, an image processor 400 that processes the images captured by the imaging devices 300, and a setting terminal 500 through which various settings are input for processing performed by the image processor 400.

[0013] The aligner 100 is connected to drives 111, 112, and 113, which provide drive forces in the respective directions and move the workpiece 10. The drives 111, 112, and 113 are connected to drive controllers 121, 122, and 123, respectively. The drive controllers 121, 122, and 123 each drive the corresponding drives 111, 112, or 113 based on control signals from the operation controller 200.

[0014] The drive controllers 121, 122, and 123, the operation controller 200, the image processor 400, and the imaging devices 300 are interconnected to communicate with each other. Any known communication means can be used, such as Ethernet (registered trademark), CameraLink, CoaxPress (registered trademark), or USB (Universal Serial Bus, registered trademark). Ethernet-based commercial networks, such as CC-Link IE Field (registered trademark) or CC-Link IE TSN (CC-Link IE Time Sensitive Networking, registered trademark), can be used to achieve synchronization.

[0015] The aligner 100 includes a support table 101 on which the workpiece 10 can be placed, and a mechanism for moving the support table 101. For example, the support table can be moved in an X direction and a Y direction, which are perpendicular to each other in the horizontal direction, and rotated in a θ direction, which is a rotation direction in a horizontal plane. In the present embodiment, of the drives connected to the aligner 100, drive 111 causes movement in the X direction, drive 112 causes movement in the Y direction, and drive 113 causes rotation in the θ direction.

[0016] Drives 111, 112, and 113 are any actuators capable of precisely driving the aligner 100. Drives 111, 112, and 113 are, for example, servomotors. Drive controllers 121, 122, and 123 control the driving of the respective drives 111, 112, and 113 based on control signals from the operation controller 200. Drive controllers 121, 122, and 123 are, for example, servo amplifiers.

[0017] The drives 111, 112, and 113 each include an internal or external position sensor for detecting and outputting the positions actually changed by driving the drives 111, 112, and 113. The position sensor is, for example, a position encoder attached to the corresponding actuator. The output signals from the position encoders are input to the operation controller 200 through the drive controllers 121, 122, and 123. In the present embodiment described below, the drives 111, 112, and 113 are servomotors, the drive controllers 121, 122, and 123 are servo amplifiers, and the position sensors are position encoders.

[0018] The operation controller 200 is a motion controller that provides commands for the operations of the drives 111, 112, and 113 to the drive controllers 121, 122, and 123. The operation controller 200 comprises, for example, a programmable logic controller (PLC). The operation controller 200 generates control signals indicating the commands based on information obtained from the drive controllers 121, 122, and 123 and the image processor 400, and outputs the control signals to the drive controllers 121, 122, and 123.

[0019] The operating control 200 comprises, as in Fig. 2, a processor 210, a volatile memory 220, a non-volatile memory 230, a clock 240, and a communication interface 250. The processor 210, the volatile memory 220, the non-volatile memory 230, the clock 240, and the communication interface 250 are connected to each other via a bus B1 to communicate with each other.

[0020] The processor 210 is, for example, a central processing unit (CPU) that reads and executes a control program 231 stored in the non-volatile memory 230 to function as a position information generator 211, a motion calculator 212, and a command generator 213.

[0021] The position information generator 211 in the processor 210 generates, based on the output signals from the position sensors in the drives 111, 112, and 113, position information corresponding to the position of the support table 101 on which the workpiece 10 is placed. The motion calculator 212 calculates the amount of movement of the workpiece 10 in the X direction, the Y direction, and the θ direction based on the position of an identification target detected by the image processor 200 in captured images. The command generator 230 outputs control signals based on the amount of movement calculated by the motion calculator 212 to the drive controllers 121, 122, and 123.

[0022] The volatile memory 220 is a working memory capable of reading and writing data at high speed during arithmetic operations performed by the processor 210. The volatile memory 220 is, for example, a random access memory (RAM). The non-volatile memory 230 stores the control program 231 for implementing the various functions of the operation controller 200, as well as control data 232, which includes parameters used in the execution of the control program 231, past acquisition data, and command data. Examples of the non-volatile memory 230 include a non-volatile semiconductor memory such as an electrically erasable programmable read-only memory (EEPROM) or a flash memory, a magnetic memory, and an optical disk.

[0023] The clock 240 measures local time by counting a clock signal from a clock element in the operation controller 200. The clock 240 synchronizes with the drive controllers 121, 122, and 123, the imaging devices 300, and the image processor 400. This allows the operation controller 200 to have time information synchronized with the drive controllers 121, 122, and 123, the imaging devices 300, and the image processor 400.

[0024] The communication interface 250 is an interface for communicating the operation controller 200 with the drive controllers 121, 122 and 123, the imaging devices 300 and the image processor 400. The communication interface 250 conforms to communication standards such as CC-Link IE Field and CC-Link IE TSN.

[0025] The imaging devices 300 capture images of the workpiece 10 from above at predetermined intervals relative to the aligner 100. The imaging devices 300 are, for example, cameras with sufficient resolution to achieve the alignment accuracy of the alignment system 1. Although any number of imaging devices 300 may be used, the number of imaging devices 300 is determined appropriately for the number of identification targets used for alignment and the position of the identification targets. Fig. 1, two imaging devices 300 are used. The identification target can be any target that indicates the position of the workpiece 10. The identification target is, for example, an alignment mark on the workpiece 10, a corner of the workpiece 10, or a corner of the support table 101.

[0026] Each imaging device 300 also includes timing information synchronized with the drive controllers 121, 122, and 123, the operation controller 200, and the image processor 400, and a communication interface for communicating with the drive controllers 121, 122, and 123, the operation controller 200, and the image processor 400.

[0027] The image processor 400 detects the identification target in the captured images obtained from the imaging devices 300 by performing a coarse search and a fine search, which are different search methods. The coarse search is performed in the entire area of ​​the captured images obtained from the imaging devices 300, whereas the fine search is performed in a search area limited with respect to the coarse search area. When the identification target is detected by the fine search, the image processor 400 outputs the actual coordinates of the identification target to the operation controller 200. The actual coordinates are coordinates in a reference coordinate system aligned with the motion calculator 212 in the operation controller 200.

[0028] The image processor 400 comprises, as shown in the Fig. 3 shows a processor 410, a volatile memory 420, a non-volatile memory 430, a clock 440, and a communication interface 450. The processor 410, the volatile memory 420, the non-volatile memory 430, the clock 440, and the communication interface 450 are connected to each other via a bus B2 to communicate with each other.

[0029] The processor 410 is, for example, a CPU that reads and executes a control program 431 stored in the non-volatile memory 430 to function as an image reference device 411, an area determiner 412, and a viewfinder 413.

[0030] The image acquisition device 411 in the processor 410 acquires images acquired by the imaging devices 300. The area determiner 412 determines an area for searching for an identification target in each captured image acquired by the image acquisition device 411 based on information including position information generated by the position information generator 211 in the operation controller 200. The searcher 413 performs a fine search for the identification target in a part of the image within the search area determined by the area determiner 412 and outputs the actual coordinates of the identification target to the operation controller 200 when the identification target is detected.

[0031] Volatile memory 420 is a working memory capable of reading and writing data at high speed during arithmetic operations performed by processor 410. Volatile memory 420 is, for example, a RAM. Non-volatile memory 430 stores the control program 431 for implementing the various functions of image processor 400 and control data 432, which includes parameters used in executing control program 431 and past acquisition data. Examples of non-volatile memory 430 include non-volatile semiconductor memory, such as EEPROM or flash memory, magnetic memory, and an optical disk.

[0032] The clock generator 440 measures local time by counting a clock signal from a clock element in the image processor 400. The clock generator 440 synchronizes with the drive controllers 121, 122, and 123, the operation controller 200, and the imaging devices 300. This allows the image processor 400 to have time information synchronized with the drive controllers 121, 122, and 123, the operation controller 200, and the imaging devices 300.

[0033] The communication interface 450 is an interface for communication of the image processor 400 with the drive controllers 121, 122 and 123, the operation controller 200 and the imaging devices 300. The communication interface 450 conforms to communication standards such as CC-Link IE Field and CC-Link IE TSN.

[0034] The setting terminal 500 is, for example, a personal computer in which an application program corresponding to the control program 431 in the image processor 400 is installed. The setting terminal 500 manages the image processor 400, including managing inputs or changes to parameters stored in the non-volatile memory 430 in the image processor 400. The communication interface for communicating between the setting terminal 500 to be connected and the image processor 400 is an interface corresponding to the interface included in the image processor 400. The communication interface is, for example, a USB interface or an interface of the recommended standard RS232C (RS232C).

[0035] The operation controller 200 calculates the amount of movement based on the difference between the actual coordinates of the identification target detected by the image processor 400 and target coordinates for alignment, and outputs control signals based on the amount of movement to the drive controllers 121, 122, and 123.

[0036] An operation of the alignment system 1 having the above structure will be described. An exemplary conventional alignment method based on the position of the identification target will be described with reference to Fig. 4 and Fig. 5. The Fig. 4 and Fig. 5 are diagrams showing an alignment process performed with identification targets, which are alignment marks 1001 and 1002, attached to ends of the workpiece 10.

[0037] Although the alignment marks 1001 and 1002 may have any shape, the alignment marks 1001 and 1002 may, for example, be cross marks as shown in the Fig. 4 and Fig. 5 to allow clear identification of the position of a reference point and the angle of rotation. Any number of alignment marks can be applied. As shown in the Fig. 4, a single alignment mark may be applied, or two alignment marks 1001 and 1002 may be applied at diagonal ends of the workpiece 10, as shown in the Fig. 5 shown.

[0038] When one alignment mark 1001 is used, one imaging device 300 is used to capture an image of an imaging region 1300. When two alignment marks 1001 and 1002 are used, with one imaging region 1300 not covering the two alignment marks 1001 and 1002 during imaging, two imaging devices 300 are used to capture images of the imaging regions 1300 that are offset from each other.

[0039] If an alignment mark 1001 is used, as in the Fig. 4, three pieces of information are used, including the coordinates of a reference point of an alignment target mark 1101, the coordinates of a reference point of the alignment mark 1001 on the workpiece 10, and the angular difference between the alignment target mark 1101 and the alignment mark 1001.

[0040] If the two alignment marks 1001 and 1002 are used as shown in the Fig. 5, three pieces of information are used, including the center coordinates of the reference points of the alignment target marks 1101 and 1102, the center coordinates of the reference points of the alignment marks 1001 and 1002 on the workpiece 10, and the angle difference between a straight line connecting the alignment target marks 1101 and 1102 and a straight line connecting the alignment marks 1001 and 1002.

[0041] Both in the Fig. 4 as well as the Fig. 5, an angular difference Δθ is the magnitude of the movement in the θ-axis. Accordingly, the operating control 200 determines, as shown in the Fig. 4, the imaginary coordinates of a reference point of an imaginary mark 1201 after rotating the workpiece 10 by the angle Δθ about a rotation center 1200 of the θ-axis of the aligner 100. The amount of movement ΔX in the X direction and the amount of movement ΔY in the Y direction are then calculated based on the difference between the imaginary coordinates and the target coordinates of the alignment target mark 1101. The command generator 213 in the operation controller 200 transmits control signals to or provides commands to the drive controllers 121, 122, and 123 to achieve such movement with the amounts Δθ, ΔX, and ΔY determined in the above manner.

[0042] The procedure, as in the Fig. 4, an alignment mark 1001 can be used if the rotation of the alignment mark 1001, such as a cross mark, can be clearly identified. When using the two alignment marks 1001 and 1002, as shown in the Fig. As shown in Figure 5, rotation of the alignment marks cannot be detected. This allows, for example, the use of circular alignment marks to detect the rotation of the workpiece. Additionally, the angular difference Δθ in the θ direction is calculated based on two points that are sufficiently far apart. This enables precise control.

[0043] In this way, the operation controller 200 calculates the values ​​Δθ, ΔX, and ΔY based on the coordinates and angular difference of the reference points of the alignment marks 1001 and 1002, which are the identification targets. The operation controller 200 uses the calculated values ​​to cause the drive controllers 121, 122, and 123 to control the drives 111, 112, and 113, thereby moving the workpiece 10. However, after a single control operation, the difference from the target coordinates is usually not within an allowable range. Therefore, the detection of the identification targets and the control of the drives 111, 112, and 113 are repeatedly performed.

[0044] With such repeatedly performed processing, the processing load for searching for the identification targets in the images captured by the imaging devices 300 is noticeably high. The alignment system 1 according to the present disclosure reduces the processing load by limiting the search range.

[0045] In the following, this process is described with reference to the Fig. 6 to 8 are described in detail. The Fig. Fig. 6 is a functional block diagram showing an alignment system 1 according to the present embodiment. Fig. 7 is a flowchart of an alignment control process performed by the operation controller 200. The Fig. 8 is a flowchart of a search process performed by the image processor 400.

[0046] The operation controller 200 first provides a command to move the workpiece 10 to a preset approximate target position (step S101 in Fig. 7). Specifically, the operation controller 200 outputs control signals to the drive controllers 121, 122, and 123 to implement movement in the respective directions. The drives 111, 112, and 113 are then driven by the drive controllers 121, 122, and 123 based on the control signals to move the support table 101 on which the workpiece 10 is placed.

[0047] After the movement, the operation controller 200 instructs the image processor 400 to acquire captured images and perform a search (step S102), and generates and outputs position information based on the outputs from the position sensors in the actuators 111, 112, and 113 (step S103). The operation controller 200 then waits until the image processor 400 completes the search (step S104).

[0048] The image processor 400, which is instructed by the operation controller 200 to perform the search process for the identification targets in step S102, executes the Fig. 8. In this state, image acquisition is performed for the first time (YES in step S201), and the image acquisition device 411 in the image processor 400 acquires the images acquired by the imaging devices 300 (step S202). The searcher 413 then performs a rough search for the identification targets (step S203).

[0049] The coarse search is performed, for example, with pattern matching using a pre-registered pattern model of the alignment mark 1001. The searcher 413 detects the alignment mark 1001 when the pattern matching rate is greater than or equal to a predetermined threshold, and determines a search area for a fine search based on the position of the reference point of the alignment mark 1001 or the rotation angle of the alignment mark 1001 (step S204). The shape or size of the search area in this step is preset appropriately for the shape or size of the alignment mark 1001 and can be set by user input to the setting terminal 500.

[0050] For example, the searcher identifies 413 for the Fig. 9A, the cross alignment mark 1001 uses the center position as a reference point and the rotation angle of the cross through pattern matching and determines a search area 1301 having a size obtained by adding a predetermined tolerance to the size of the alignment mark 1001. In addition, a search area is determined based on the angle (a search angle range) shown in the Fig. 9B is shown.

[0051] The searcher 413 performs a fine search in the part of the image within the search area determined in step S204 (step S205). The fine search includes, for example, edge detection to accurately detect straight lines or curves to obtain more accurate actual coordinates of the identification target. The actual coordinates of the identification target include the position of the reference point (XY coordinates) and the rotation angle (θ coordinate) of the identification target.

[0052] If the searcher 413 does not detect an identification target during the fine search in step S205 (NO in step S212), the process returns to step S201 and repeats steps S202 to S205. If the identification target is successfully detected during the fine search in step S205 (YES in step S212), the searcher 413 outputs the actual coordinates of the identification target detected in step S205 to the operation controller 200 (step S213).

[0053] It will again refer to the flow chart of the Fig. 7. After the search process with the image processor 400 is completed (YES in step S104), the operation controller 200 obtains the actual coordinates from the image processor 400 (step S105) and calculates the difference from the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (YES in step S106), the alignment control process is terminated. The target coordinates in this example are coordinates of the alignment target and include a center position (XY coordinates) and a rotation angle (θ coordinate).

[0054] If the difference between the actual coordinates and the target coordinates is greater than the threshold value (NO in step S106), the command generator 213 provides a command to correct the position of the aligner 100 (step S107, or providing a command). Specifically, the command generator 213 outputs control signals for adjusting the actual coordinates to the target coordinates to the drive controllers 121, 122, and 123.

[0055] After the actuators 111, 112, and 113 are driven, the process returns to step S102. The operation controller 200 again provides an instruction to perform image acquisition and search (step S101) and outputs position information (step S103).

[0056] It will again refer to the flow chart of the Fig. 8. Since image acquisition is performed for the second or subsequent time and the last acquisition was successful (NO in step S201), the image processor 400 acquires position information (step S206). The area determiner 412 in the image processor 400 calculates predicted coordinates of the identification target at the time (step S207).

[0057] Specifically, the range determiner 412 identifies a relationship between the position information at the first alignment control acquired from the operation controller 200 and the actual coordinates acquired during the first fine search. For example, the relationship between the XY coordinates of the position information at the first control and the XY coordinates of the actual coordinates is represented by a constant matrix A in Formula 1 below, where (x1, y1) are the XY coordinates of the position information at the first control, and (X1, Y1) are the XY coordinates of the actual coordinates. A=(X1,Y1) / (x1,y1)

[0058] The XY coordinates (X2, Y2) of the prediction coordinates to be used in the second search can be represented by Formula 2 below using the constant matrix A represented by Formula 1, where (x2, y2) are the XY coordinates at the second orientation control, which are indicated by the position information acquired from the operation controller 200. The nth time, including the third time or subsequent times, can be represented by Formula 3 in a similar manner. (X2,Y2)=A(x2,y2) (Xn,Yn)=A(xn,yn)

[0059] The constant matrix representing the relationship between the position information and the actual coordinates can be updated each time, or can be averaged over multiple times based on the position information and the actual information acquired during the repeatedly performed alignment control. In another example, the relationship between the position information and the actual coordinates can be pre-constructed through pre-calibration. For example, pre-calibration for alignment control using a corner of the support table 101 as an identification target is effective and has a lower error per process.

[0060] As described above, the relationship between the XY coordinates of the reference point of the identification target detected in the captured images and the position information is predetermined, and the XY coordinates are predicted from the newly acquired position information using this relationship (step S207). Next, a search area with a predetermined size is determined using the predicted XY coordinates as a center (step S208). Similarly, the relationship for the θ coordinate between the actual coordinate detected in the fine search and the position information is predetermined, and the θ coordinate indicating the rotation angle of the identification target is predicted based on the newly acquired position information (step S207). Then, a search angle range with a predetermined angular width is determined using the predicted θ coordinate as a center (step S208).The search area may be other than the area centered on the predicted coordinates and may be an area encompassing the predicted coordinates appropriately for the alignment conditions or the shape of the identification target.

[0061] The size of the search area can be set by user input to the setting terminal 500 or can be set automatically. For example, the size of the search area can be obtained by adding a tolerance, which is determined automatically or manually based on the moving speed of the actuators 111, 112, and 113, to the shape or size of the identification target. In another example, the size of the search area can be statistically determined based on past alignment control results. For example, the size can be obtained by adding the average of the differences between the predicted coordinates calculated based on the position information and the actual coordinates captured in the captured image at past time points that correspond to each other, as a tolerance to the shape or size of the identification target.

[0062] In other words, in a case where image acquisition is performed for the second or subsequent time and the previous acquisition was not unsuccessful, the area determiner 412 determines the search area centered on the predicted coordinates calculated based on the position information based on the outputs of the position sensors in step S207 without performing the coarse search in step S203. This determination of the search area based on the position information uses a noticeably lower processing load than determining the search area through the coarse search performed in step S203. This enables faster control than the conventional alignment control that performs a coarse search each time.

[0063] The image acquisition device 411 acquires an image (step S209, or acquiring a captured image) and performs a simple process on the part of the acquired image corresponding to the search range determined in step S208 (step S210). The simple process is any preprocessing before the fine search. For example, the searcher 413 may perform a coarse search in an area larger than the search range determined in step S208 but smaller than the entire area, and may redetermine the search range. In another example, the simple process may be a coarse search for the θ coordinate to determine a search angle range when the prediction coordinates of the XY coordinates were calculated in step S207 and the search range was determined based on the XY coordinates in step S208. The simple process may be eliminated.

[0064] The searcher 413 then performs a fine search in the search area determined in step S208 or in the area determined in step S210 (step S211, or searching the captured image). The procedure for the fine search is the same as in step S205. If no identification target is detected in the fine search in step S211 (NO in step S212), the process returns to step S201 and performs steps S202 to S205 where the previous detection was unsuccessful (YES in step S201). Restarting with a coarse search in the full area can prevent loss of control resulting from repeated misdetections in the fine search.

[0065] When the identification target is successfully detected in the fine search in step S211 (YES in step S212), the actual coordinates of the identification target obtained in step S211 are output to the operation controller 200.

[0066] It will again refer to the flow chart of the Fig. 7. The operation controller 200 obtains the actual coordinates from the image processor 400 (step S105) and calculates the difference from the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (YES in step S106), the operation controller 200 terminates the alignment process.

[0067] If the difference between the actual coordinates and the target coordinates is greater than the threshold value (NO in step S106), the command generator 213 provides a command to correct the position of the aligner 100 (step S107). Specifically, the command generator 213 outputs signals to adjust the actual coordinates to the target coordinates to the drive controllers 121, 122, and 123. The process then returns to step S102 and continues.

[0068] As described above, the alignment system 1 according to the present embodiment includes the image processor 400 and the operation controller. The image processor 400 detects an identification target in an image of the workpiece 10 that includes the identification target. The operation controller controls the drives 111, 112, and 113, which move the workpiece 10 based on the actual coordinates of the identification target detected by the image processor 400.The image processor 400 determines in advance the relationship between the position information based on the outputs from the position sensors in the actuators 111, 112, and 113 and the actual coordinates of the identification target detected in the captured image. Using this relationship, it calculates the predicted coordinates of the identification target from the position information acquired next. It determines a search area including these predicted coordinates, and detects the identification target in the part of the image corresponding to the search area. This eliminates a rough search across the entire area and enables fast and precise positioning. Embodiment 2

[0069] Embodiment 2 of the present disclosure will be described below with reference to the drawings. Like reference numerals denote the same or corresponding components throughout the drawings.

[0070] The overall structure of an alignment system 2 according to Embodiment 2 and the hardware configuration of each component are the same as in Embodiment 1. The alignment system 2 according to Embodiment 2 differs from the system in Embodiment 1 in that the image acquisition device 411 in the image processor 400 generates a transmission instruction specifying a transmission range to acquire captured images from the imaging devices 300, and further in that the searcher 413 performs the search process in the part of each captured image within the transmission range.

[0071] In the present embodiment, as in Embodiment 1, the drive controllers 121, 122, and 123, the operation controller 200, the image processor 400, and the imaging devices 300 are connected to communicate with each other. However, the structure in the present embodiment uses higher responsiveness than the structure in Embodiment 1 and can use a commercial network such as CC-Link IE Field or CC-Link IE TSN as a communication link.

[0072] To maintain synchronization in the commercial network, such as CC-Link IE Field and CC-Link IE TSN, hardware devices (H / W devices) include a mechanism that achieves a certain degree of punctuality and synchronizes the times between the devices in microseconds by statistically measuring delays across the transmission paths.

[0073] Synchronization schemes based on transmission path delay measurement use measurements that indicate transmission path delays from a master station to a device station to improve synchronization accuracy. Fig. Figure 10 shows a synchronization scheme based on a transmission path delay measurement. In the scheme based on a transmission path delay measurement, synchronization is performed at synchronization points. A transmission control frame (MyStatus) sent by a master station 40 propagates over the distance with a delay.

[0074] The master station 40 calculates the transmission delay time at the device stations 50, 60, and 70 based on the master station times at which the master station 40 receives the response signals from the device stations 50, 60, and 70, and transmits the delay time to the device stations 50, 60, and 70. Each synchronization point is the time at which a predetermined time (Tsync) has elapsed since the master station 40 transmitted the transmission control frame (MyStatus). Each of the device stations 50, 60, and 70 performs synchronization with the elapse of time Tps, calculated by subtracting the transmission path delay time (Tsync delay time), after the time at which the device received a transmission control frame (MyStatus).

[0075] In the present embodiment, this transmission path delay measurement scheme is used to perform the timing control of the drive controllers 121, 122, and 123, the operation controller 200, the imaging devices 300, and the image processor 400, for example, using the operation controller 200 as the master station 40. In other words, the drive controllers 121, 122, and 123, the operation controller 200, the imaging devices 300, and the image processor 400 are connected to communicate with each other via a commercial network (communication line) for which the transmission path delay times are measured, and have synchronized time information.

[0076] The operation controller 200, as the master station 40, sets a specific time after the longest transmission path delay period has elapsed from the time the instruction is sent as the synchronization point. The image processor 400 then determines an area encompassing the prediction coordinates at the specific time in the future as a transmission area, and instructs each imaging device 300 to capture an image at the specific time and transmit the portion of the image within the transmission area.

[0077] The timing for each imaging device 300 can be performed taking into account the shutter speed. Fig. 11 is a diagram showing the timing of the aperture in the imaging device 300. Using the synchronized timing information used to calculate the synchronization point, the imaging device 300 automatically determines the imaging timing based on setting information obtained from the image processor 400 or predetermined setting information. To achieve image acquisition at a specific time in the future, the image processor 400 may appropriately provide a preceding imaging instruction within a transition period, or the timing for reserved imaging may be aligned with the center of the exposure time range.

[0078] The Fig. 12 is a functional block diagram of the alignment system according to the present embodiment. Fig. 13 is a flowchart of an alignment control process performed by the operation controller 200. The Fig. Fig. 14 is a flowchart of a search process performed by the image processor 400. The operation of the alignment system 2 is described with reference to the flowcharts in the Fig. 13 and Fig. 14 described.

[0079] The operation controller 200 first provides a command to move the workpiece 10 to a predetermined approximate target position (step S101 in Fig. 13). Specifically, the operation controller 200 outputs control signals to the drive controllers 121, 122, and 123 to implement movement in the respective directions. The drives 111, 112, and 113 are then driven by the drive controllers 121, 122, and 123 based on the control signals to move the support table 101 on which the workpiece 10 is placed.

[0080] After providing the movement command, the operation controller 200 determines the imaging timing in the imaging devices 300 as time Tn (n=1) (step S122) and specifies the timing to the image processor 400. The position information generator 211 in the operation controller 200 generates position information based on the outputs from the position sensors in the actuators 111, 112, and 113 and outputs the position information (step S103). The operation controller 200 then waits until the image processor 400 completes the search (step S104).

[0081] After obtaining the position information from the operation controller 200, the image processor 400 executes the Fig. 14. In this state, image acquisition is performed for the first time (YES in step S201), so the image acquisition device 411 in the image processor 400 instructs each imaging device 300 to acquire images at time Tn (n=1) and transmit the entire area of ​​the acquired image (step S221). The image acquisition device 411 then acquires the entire area of ​​the acquired image (step S222).

[0082] The searcher 413 in the image processor 400 performs a coarse search for the identification target in the entire area of ​​the captured image (step S203). The coarse search is performed, for example, by pattern matching using a pre-registered pattern model of the alignment mark 1001. The searcher 413 detects the alignment mark 1001 as the identification target when the pattern matching rate is greater than or equal to a predetermined threshold and determines a search area for a fine search based on the position of the reference point of the alignment mark 1001 or the rotation angle of the alignment mark 1001 (step S204). The shape or size of the search area is predetermined in this example based on the shape or size of the alignment mark 1001 and can be set by a user input to the setting terminal 500.

[0083] Then, a fine search is performed on the part of the image within the search area determined in step S204 (step S205). The fine search includes, for example, edge detection to detect precisely straight lines or curves to obtain more accurate actual coordinates of the identification target. The actual coordinates of the identification target include the position of the reference point (XY coordinates) and the rotation angle (θ coordinate) of the identification target.

[0084] If no identification target is detected during the fine search in step S205 (NO in step S212), the process returns to step S201 and repeats steps S221, S222, and S203 to S205. If the identification target is successfully detected during the fine search in step S205 (YES in step S212), the actual coordinates of the identification target detected in step S205 are output to the operation controller 200 (step S213).

[0085] It will again refer to the flow chart of the Fig. 13. After the search process with the image processor 400 is completed (YES in step S104), the operation controller 200 obtains the actual coordinates from the image processor 400 (step S105) and calculates the difference from the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (YES in step S106), the alignment control process is terminated. The target coordinates in this example are coordinates of the alignment target and include a center position (XY coordinates) and a rotation angle (θ coordinate).

[0086] If the difference between the actual coordinates and the target coordinates is greater than the threshold value (NO in step S106), the command generator 213 provides a command to correct the position of the aligner 100 (step S107, or providing a command). Specifically, the command generator 213 outputs control signals for adjusting the actual coordinates to the target coordinates to the drive controllers 121, 122, and 123.

[0087] After providing the correction command to the aligner 100, the operation controller 200 returns to step S122, determines the imaging timing as the timing Tn (n=2) (step S122), outputs the position information (step S103), and waits until the image processor 400 completes the search process (step S104).

[0088] It will again refer to the flow chart of the Fig. 14. Since the next image acquisition is performed for the second or subsequent time and the previous acquisition was successful (NO in step S201), the image processor 400 instructs each imaging device 300 to acquire an image at time Tn (n=2) (step S223). The image processor 400 acquires the position information from the operation controller 200 (step S206). The area determiner 412 in the image processor 400 predicts the coordinates of the identification target at that time (step S207).

[0089] The method for predicting the coordinates includes, as in Embodiment 1, identifying the relationship between the position information at the first alignment control acquired by the operation controller 200 and the actual coordinates acquired during the first fine search. The area determiner 412 uses the identified relationship to predict the XY coordinates of the reference point of the identification target based on the position information acquired in step S206 (step S207). The area determiner 412 then determines a transmission area with a predetermined size centered on the XY coordinates predicted in step S207 and instructs each imaging device 300 to transmit a captured image for which a transmission area is specified (step S226).The transmission range may be other than the range centered on the predicted XY coordinates, and may be a range that reasonably encompasses the predicted XY coordinates for the alignment conditions or the shape of the identification target.

[0090] The size of the transmission range can be adjusted by user input to the setting terminal 500. For example, robustness and processing speed have a trade-off relationship, and thus a smaller transmission range can be set to prioritize throughput by maximizing average processing speed while allowing fluctuation in processing. Conversely, a larger transmission range can be set when fluctuation characteristics are to be prioritized over processing speed to achieve a constant processing time.

[0091] In another example, the size of the transmission area may be obtained by adding a tolerance, which is automatically or manually adjusted based on the movement speeds of the actuators 111, 112, and 113, to the shape or size of the identification target. In another example, the area determiner 412 may statistically determine the size of the transmission area based on previous alignment control results. For example, the area determiner 412 may determine the size of the transmission area by adding the average difference between the predicted coordinates calculated based on past position information and the actual coordinates of the identification target captured in the image as a tolerance to the size of the identification target.

[0092] Therefore, when image acquisition is performed for the second or subsequent time and the previous acquisition is not unsuccessful as in this case, each imaging device 300 transmits the part of the image within the transmission range centered on the predicted coordinates based on the position information from the position sensors, as determined in step S226, without transmitting the entire range of the image. The data volume of the image transmitted in this step is noticeably smaller than the data volume of the image transmitted with the entire range in step S221. The data transmission volume and transmission time can be shortened compared to the case where the entire range of the image is transmitted each time, thereby enabling fast control without a rough search performed in the entire range.

[0093] The image acquisition device 411 acquires the part of the image within the transmission range that will be transmitted as specified by the transmission instruction in step S226 (step S209, or acquiring an image). The searcher 413 performs a simple process on the part of the image within the transmission range (step S210). The simple process is simple preprocessing before the fine search. For example, the coarse search may be performed on the part of the image within the transmission range acquired in step S209 to determine the search range. Specifically, the simple process may be a coarse process for the θ coordinate to determine a search angle range. The simple process may be eliminated.

[0094] Then, a fine search is performed in the part of the image within the transmission range acquired in step S209 or in the search range determined in step S210 (step S211, or scanning the captured image). The procedure for the fine search is the same as in step S205. If no identification target is detected in the fine search in step S211 (NO in step S212), the process returns to step S201 and executes steps S221, S222, and S203 to S205, with the previous detection being successful (YES in step S201). Starting again with a coarse search in the entire area of ​​the image can prevent loss of control resulting from repeated misdetections in the fine search.

[0095] When the identification target is successfully detected in the fine search in step S211 (YES in step S212), the actual coordinates of the identification target detected in step S211 are output to the operation controller 200 (step S213).

[0096] It will again refer to the flow chart of the Fig. 13. The operation controller 200 obtains the actual coordinates from the image processor 400 (step S105) and calculates the difference from the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (YES in step S106), the operation controller 200 terminates the alignment process.

[0097] If the difference between the actual coordinates and the target coordinates is greater than the threshold value (NO in step S106), the command generator 213 provides a command to correct the position of the aligner 100 (step S107). Specifically, the command generator 213 outputs control signals to the drive controllers 121, 122, and 123 to align the actual coordinates with the target coordinates. The process then returns to step S122 and continues.

[0098] As described above, the area determiner 412 in the image processor 400 determines the transmission area of ​​the image based on the position information, and the searcher 413 performs a fine search in the image transmitted from each imaging device 300. This significantly reduces the transmission time and the search time.

[0099] The image processor 400 may specify the transmission range for the imaging devices 300 before issuing an imaging trigger to the imaging devices 300, rather than specifying the imaging parameters including the transmission range together with an imaging instruction.

[0100] When the transmission range is specified along with the imaging command, the above-described synchronization scheme based on transmission path delay measurement is used to synchronize the timing of image acquisition by the imaging devices 300 and the acquisition of position information, thereby reducing waiting time and enabling a search within a suitable transmission range. When the transmission range is specified in advance, the imaging trigger is output alone over the dedicated line, enabling precise synchronization of the image acquisition timing.

[0101] The entire area of ​​the image specified in step S221 and the area of ​​the image specified in step S226 in Fig. The portion of the image within the transmission range specified in Figure 14 may be reduced by each imaging device 300 prior to transmission. A method for such image reduction may be any known method, such as subsampling or binning. In particular, binning has a relatively low processing load and can improve the signal-to-noise ratio (SNR) of pixel signals, thereby preventing a reduction in position detection accuracy when the resolution is reduced by image reduction.

[0102] Regarding the restriction (cropping) of the transmission range specified by the transmission instruction in step S226, image downsizing may be performed along with the transmission range restriction when the actual coordinates and the target coordinates are far apart in an early stage of alignment control. When the actual coordinates and the target coordinates are close enough to be within a predetermined distance range, downsizing may not be performed. When the image size of the entire area or within the transmission range is reduced, a trained data reduction model may be prepared as a sample model for searching, and the searcher 413 may perform a coarse search or a fine search using this model.

[0103] In parallel with the fine search (step S211), after acquiring the portion of the captured image within the transmission range specified by the transmission instruction in step S226, the entire captured image range can also be acquired. This can save the time required to acquire the entire captured image range if no identification target is detected in the fine search in step S211 (NO in step S212). In this case, if the identification target is detected in the fine search, the transmission of the entire captured image range can be stopped, or the transmitted images can be deleted.

[0104] As described above, in the alignment system 2 according to the present embodiment, the image processor 400 identifies in advance the relationship between the position information provided by the drivers 111, 112, and 113 and the actual information of the identification target captured in the captured image. Using this relationship, the image processor 400 calculates the predicted coordinates of the identification target based on the next acquired position information, determines a transmission range of the image including these predicted coordinates, and detects the identification target by finely searching the part of the image transmitted from each imaging device 300 within the transmission range.This reduces the transmission time and the transmission volume of the image to be transmitted and also eliminates a coarse search in the entire area of ​​the image, enabling fast and precise alignment control. Modifications

[0105] The above embodiments can be modified in various ways. Modifications are described below. Modification 1

[0106] In Embodiments 1 and 2, the area determiner 412 in the image processor 400 determines the area including the predicted coordinates calculated based on the position information as the search area or the transmission area based on the outputs from the position sensors in the drivers 111, 112, and 113. The search area or the transmission area may be determined in other ways. In Modification 1, other methods for determining a search area or a transmission area are described.

[0107] In the present modification, command information about driving the drives 111, 112, and 113 is used to calculate predicted coordinates for determining a search area or a transfer area. Specifically, the area determiner 412 in the image processor calculates the predicted coordinates based on the command information and determines a search area or a transfer area including the predicted coordinates.

[0108] In the present modification, the image pickup of the imaging devices 300, the driving of the actuators 111, 112 and 113 based on the command information and the obtaining of the outputs from the position sensors are controlled so as to be synchronized with each other, thereby enabling faster control. Fig. 15 is a functional block diagram of an alignment system 3 according to the present modification. Fig. 16 is a flowchart of an alignment control process performed by the operation controller 200 according to the present modification. Fig. 17 is a flowchart of a search process performed by the image processor 400.

[0109] The structure in the Fig. 15 to 17 represents a modification of Embodiment 2 and determines an image transfer area based on the command information. However, the structure may similarly represent a modification of Embodiment 1 and determine an image search area based on the command information. The operation of the alignment system 3 according to the present modification is described below with reference to the flowcharts in FIGS. Fig. 16 and Fig. 17. Processing identical to embodiment 2 is not described.

[0110] The operation controller 200 first provides a command to move the workpiece 10 to a preset approximate target position (step S101 in Fig. 16). The command information is forwarded to the image processor 400 for use in determining the transfer area. The drives 111, 112, and 113 are then driven by the control signals for the command information to move the support table 101 on which the workpiece 10 is placed.

[0111] After providing the movement command, the operation controller 200 determines the imaging time in the imaging devices 300 as the specific future time Tn (n=1) (step S122) and specifies the time Tn to the image processor 400. The position information generator 211 in the operation controller 200 specifies the position information at the time Tn for shooting based on the outputs from the position sensors in the drivers 111, 112, and 113 (step S123). The operation controller 200 then waits until the image processor 400 completes the search (step S104). The subsequent operation of the image processor 400 is the same as in Embodiment 2. The image processor 400 executes steps S221, S222, S203 to S205, S212, and S213 in the Fig. 17 through.

[0112] Reference will again be made to the flow chart in the Fig. 16. When the image processor 400 terminates the search process (YES in step S104), the motion calculator 212 in the operation controller 200 acquires the actual coordinates from the image processor 400 (step S105), and the position information generator 211 generates and records position information based on the outputs from the position sensors at time Tn (step S124). The motion calculator 212 calculates the difference between the actual coordinates and the target coordinates. If the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (YES in step S106), the operation controller 200 terminates the alignment control process.

[0113] If the difference between the actual coordinates and the target coordinates is greater than the threshold (NO in step S106), the commander 213 provides a command to correct the position of the aligner 100. The commander 213 refers to the position information at the specific time Tn acquired in step S124 and provides a command to also compensate for the deviation between the position specified by the command and the actual position specified by the position information (step S125).

[0114] After providing the command to the aligner 100 with the command generator 213, the operation controller 200 returns to step S122, determines the imaging time as the time Tn (n=2) (step S122), specifies the position information at the time Tn (n=2) for recording (step S123), and waits until the image processor 400 has completed the search process (step S104).

[0115] It will again refer to the flow chart in Fig. 17. If the next image acquisition is performed for the second or subsequent time and the previous acquisition is successful (NO in step S201), the image processor 400 instructs each imaging device 300 to acquire an image at time Tn (n=2) (step S223). The image processor 400 obtains the command information at time Tn from the operation controller 200 (step S224). The area determiner 412 in the image processor 400 calculates the predicted coordinates of the identification target at that time (step S225).

[0116] The method for calculating the predicted coordinates with the range determiner 412 first identifies the relationship between the command information during the first alignment control, instead of the position information as in Embodiment 2, and the actual coordinates acquired during the first fine search. The range determiner 412 uses the identified relationship to calculate the predicted coordinates (XY coordinates) of the reference point of the identification target based on the command information about the planned position at time Tn acquired in step S224 (step S225). The method for predicting the coordinates using the relationship is the same as in Embodiment 2.

[0117] The area determiner 412 then determines a transfer area with a predetermined size centered on the XY coordinates predicted in step S225 and instructs each imaging device 300 to transfer the portion of the image within the determined transfer area (step S226). The transfer area may be other than the area centered on the predicted XY coordinates and may be an area that appropriately encompasses the predicted XY coordinates for the alignment conditions or the shape of the identification target.

[0118] The image acquisition device 411 acquires the part of the image within the transmission range that will be transmitted as specified by the transmission instruction in step S226 (step S209) and performs a simple process (step S210). The searcher 413 then performs a fine search on the part of the image within the transmission range acquired in step S209 or on the image that underwent the simple process in step S210 (step S211). If no identification target was detected in the fine search in step S211 (NO in step S212), the process returns to step S201 and executes steps S221, S222, and S203 to S205 where the previous detection was unsuccessful (YES in step S201).

[0119] If the identification target is successfully detected during the fine search in step S211 (YES in step S212), the actual coordinates of the identification target obtained in step S211 are output to the operation controller 200 (step S213). The subsequent processing performed by the operation controller 200 is the same as in Embodiment 2.

[0120] As described above, the area determiner 412 in the image processor 400 determines the transmission area of ​​the image based on the command information from the operation controller 200. The searcher 413 performs a fine search in the image transmitted from each imaging device 300. This enables more efficient timing and faster alignment control, with the search area or the transmission area being determined based on the command information about the future planned position. Modification 2

[0121] In Modification 2, another method for determining a search area or a transmission area is described. In Embodiments 1 and 2, the area determiner 412 determines a search area or a transmission area centered around the predicted coordinates calculated based on the position information. In the present modification, the search area or the transmission area is determined using the most recently acquired actual coordinates as the center.

[0122] Specifically, the area determiner 412 determines an area centered around the reference point of the identification target, which was last acquired by the finder 413 in the image processor 400 during the fine search and is stored in the non-volatile memory 230, as the transmission area without acquiring the position information or the command information. In another example, the area determiner 412 determines, as the search area, an area centered around the reference point of the stored identification target or the angular range centered around the rotation angle of the identification target. The transmission area or the search area may be other than the area centered around the last acquired coordinates and may be an area that appropriately includes the last acquired actual coordinates for the alignment conditions or the shape of the identification target.

[0123] The size of the search area or the transmission area can be determined based on the movement speed of the actual coordinates of the identification target, which was previously acquired. In another example, the size of the search area or the transmission area can be determined based on the movement speed of each drive based on setting parameters in the alignment control. For example, the size can be determined by multiplying the maximum movement speed by the imaging interval of the imaging devices 300. If the movement distance between imaging intervals is short when short imaging intervals are set, the last acquired result can be used to sufficiently restrict the search area or the transmission area.The structure in this modification can therefore simplify processing compared with the structures in Embodiments 1 and 2 and Modification 1. Modification 3

[0124] In Modification 3, another method for determining a search area or a transmission area will be described. In this modification, the area determiner 412 determines the search area or the transmission area based on the trajectory of the actual coordinates acquired in the past fine searches performed by the searcher 413. In other words, the structure calculates the predicted coordinates based on the trajectory of the actual coordinates of the identification target acquired in the past fine searches performed by the searcher 413 in the image processor 400 and stored in the nonvolatile memory 230 without acquiring the position information or command information as in Embodiments 1 and 2, and determines an area centered on the predicted coordinates as the search area or the transmission area.

[0125] Specifically, the range determiner 412 determines as the transmission range a range with a predetermined size centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identification target. In another example, the range determiner 412 determines as the search range a range with a predetermined size centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identification target, or an angular range centered on the θ coordinate predicted based on the trajectory of the θ coordinate. Fig. 18A and Fig. 18B are diagrams showing the prediction of the identification target. The transmission range or the search range may be other than the range centered on the predicted coordinates, and may be a range that appropriately encompasses the predicted coordinates for the alignment conditions or the shape of the identification target.

[0126] The predicted coordinates (XYθ coordinates) are calculated based on the prediction during image acquisition by the imaging devices 300 using two or more previous times and the actual coordinates at each of the two or more previous times. For example, as shown in the Fig. 18A, the XY coordinates during image acquisition are predicted by linear prediction based on data indicating the XY coordinates for the last two times, or more specifically, the last time and the second to last time. In another example, as shown in the Fig. 18B, the XY coordinates at image acquisition are predicted by a second-order prediction based on data indicating the XY coordinates at the last three times, or more specifically, the last time, the second-to-last time, and the second-to-last time.

[0127] The size of the search area or the transmission area can be determined based on the movement speed of the actual coordinates of the identification target that were previously acquired. In other examples, the size can be determined based on the movement speeds of the drives 111, 112, and 113. For example, the size can be determined by multiplying the maximum movement speed by the imaging interval of the imaging devices 300. A linear prediction has a lower prediction accuracy than a second-order or higher-order prediction and therefore requires a larger search area or a larger transmission area.

[0128] The size of the search area or transmission area can be determined based on statistical information about errors from past detection results. For example, the size can be obtained by adding the average difference between the predicted coordinates based on the trajectory of the actual coordinates and the corresponding actual coordinates acquired in the image at past time points as a tolerance to the shape and size of the identification target.

[0129] When the imaging interval is larger and the moving distance between the imaging intervals is larger, the structure in the present modification may have higher prediction accuracy and may have a smaller search range or transmission range. Modification 4

[0130] In the above-described embodiments 1 and 2, the identification target includes the alignment marks 1001 and 1002 on the workpiece 10, but the processing may be appropriately changed partially in the shape of the identification target. Fig. Figure 19A is a diagram showing a rough search in the present modification. Fig. Figure 19B is a diagram showing a fine search in the present modification.

[0131] For example, for a workpiece 12 which is an integrated circuit (IC) with multiple terminals as identification targets, the finder 413 performs Fig. 19A, a rough search by pattern matching using a pattern model 1312 (step S203 in Fig. 8).

[0132] In the search, the searcher 413 determines a search area 1314 for a fine search based on the position or rotation angle of a pattern 1313 when the pattern matching rate is greater than or equal to a predetermined threshold (step S204 in Fig. 8). In this pattern matching, as in the Fig. As shown in Figure 19A, multiple patterns with a match rate greater than or equal to a predetermined threshold may be detected. In this case, a pattern 1313 with the highest match rate may be identified to determine the search area 1314.

[0133] In the Fig. 19B, the searcher 413 determines the search area 1314 having a predetermined positional relationship with respect to the pattern 1313 identified by the coarse search as the search area for the fine search (step S204 in Fig. 8). The searcher 413 performs a fine search in the search area determined in step S204 (step S205 in Fig. 8).

[0134] In Embodiments 1 and 2, when the image acquisition is performed for the second time or a subsequent time and the previous acquisition was successful (NO in step S201 in Fig. 8), the search area is determined which includes the predicted coordinates calculated based on the position information (step S208 in Fig. 8), and the fine search is performed (step S211 in Fig. 8). However, if the predicted coordinates can be calculated with a predetermined accuracy, as in the Fig. In the IC shown in Figure 19A, the fine search may be repeated using the search area determined during the coarse search in step S203 without calculating the predicted coordinates based on the position information. If detection by the fine search is unsuccessful, the search area for the coarse search may be determined.

[0135] The structure according to the present modification can appropriately simplify the process for the shape of the identification target. An application in the setting terminal 500 can classify the shape of the identification target and automatically select a search method for the classification. Modification 5

[0136] In modification 5 described below, the identification target has a different shape. Fig. Figure 20A is a diagram showing a rough search for the present modification. Fig. 20B is a diagram illustrating a fine search in the present modification. In the present modification, a processor in the setting terminal 500 executes a dedicated application program to automatically register a model for the coarse search and generate logic for a fine search.

[0137] For example, the processor in the setting terminal 500 extracts for a workpiece 13 formed as shown in the Fig. 20A is a lens, a lens outline from a captured image and registers a pattern model 1322. The processor then automatically generates search logic for a fine search based on the shape of the pattern model 1322. For example, if, as in the Fig. 20B, when the processor in the setting terminal 500 detects the shape of the pattern model 1322 including an arc, the processor automatically generates, as logic for the fine search, logic including searching for an edge, approximating the detected edge to a circle, and detecting the center of the circle as a reference point 1323.

[0138] The image processor 400 performs the search process described in Embodiments 1 and 2 using the coarse search pattern model and the fine search logic generated by the application in the setting terminal 500. The structure according to the present modification can appropriately perform an optimal coarse search and an optimal fine search for the shape of the identification target. The structure also enables automatic presetting for the search process with the image processor 400, thus reducing the user's requirement to input settings into the setting terminal 500. Modification 6

[0139] In Embodiments 1 and 2, the coarse search is performed in the first captured image, the fine search is performed in the search area determined based on the result of the coarse search, and the predicted coordinates of the identification target are calculated based on the result of the fine search to determine the search area or the transmission area for the next fine search. However, if the prediction accuracy of the coordinates of the identification target is sufficiently high, the coarse search in the first captured image can be eliminated by performing calibration before alignment control. Such pre-calibration is particularly effective for alignment control using a corner of the support table 101 on which the workpiece 10 is placed, where an error per execution is small.

[0140] The Fig.21 is a flowchart of a search process without the coarse search for the first captured image. After the image acquisition device 411 acquires a captured image for the first time (step S301), the area determiner 412 calculates the predicted coordinates of the identification target based on position information using pre-calibration information (step S302). The search area including the predicted coordinates calculated by the area determiner 412 is determined (step S303), and a coarse search is performed in the determined search area (step S304).

[0141] If the actual coordinates of the identification target are successfully acquired during the fine search in step S304, the actual coordinates are output to the operation controller 200 (step S309), and the operation controller 200 controls the drives 111, 112, and 113 using the actual coordinates. After the workpiece 10 is moved, a fine search is performed for the next captured image in the same search area as the last time. Accordingly, if the actual coordinates are successfully acquired during the fine search, the fine search is repeated without a coarse search.

[0142] If the detection of the identification target is unsuccessful when repeating the fine search (NO in step S305), the coarse search is performed (step S306) to determine a search area (step S307). The fine search is performed in the determined search area (step S308), and the actual coordinates are output (step S309). In the present modification described above, the fine search is repeated based on the pre-calibration information, and the coarse search is performed when the identification target cannot be detected. This simplifies processing and enables rapid alignment control.

[0143] The hardware configurations and flowcharts described in the above embodiments and modifications are merely examples and can be changed and modified as appropriate. For example, in the above embodiments and modifications, the operation controller 200, the imaging devices 300, the image processor 400, and the setting terminal 500 are independent components, but at least two of the components may be implemented integrally with each other.

[0144] The search processes shown in the above embodiments and modifications can be combined as appropriate. The setting terminal 500 can be used to select any of the search processes in the above embodiments and modifications. In another example, the image processor 400 can automatically select any of the search processes of the above embodiments and modifications as appropriate for conditions such as workpiece types and the shapes of the identification target.

[0145] In the above embodiments and modifications, the functions implemented by the processors in the operation controller 200 and the image processor 400 are shared in an exemplary manner, and may be shared in any other suitable manner. The functions implemented by the processors in the operation controller 200 and the image processor 400 may be implemented in a general-purpose computer system rather than a dedicated system.

[0146] A program for performing the operations described in the above embodiments may be stored in a non-volatile computer-readable storage medium, such as a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), an MO disc (Magneto-Optical Disc), or a memory card for distribution, and may be installed on a computer to implement the above functions. In the system in which functions are implementable by the operating system (OS) and an application in a shared manner or through cooperation between the OS and the application, parts executable by the application other than the OS may be stored in a non-volatile storage medium.

[0147] The foregoing describes some exemplary embodiments for illustrative purposes. Although the foregoing discussion has presented specific embodiments, those skilled in the art will recognize that changes may be made in form and detail without departing from the broader spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. This detailed description is accordingly not to be taken in a limiting sense, and the scope of the invention is defined only by the appended claims, along with the full breadth of equivalents encompassed by such claims. List of reference symbols 1, 2, 3 alignment system 10, 12, 13 Workpiece 11 Target 40 master stations 50, 60, 70 fixture station 100 organizers 101 support table 111, 112, 113 Drive 121, 122, 123 Drive control 200 Operational control 210 processor 211 Position information generator 212 movement calculator 213 commanders 220 volatile memory 230 non-volatile memory 231 Control program 232 control data 240 clock generators 250 communication interface 300 imaging device 400 image processor 410 processor 411 Image reference device 412 Area Determiner 413 searchers 420 volatile memory 430 non-volatile memory 431 Control program 432 control data 440 clock generator 450 communication interface 500 setting terminal 1001, 1002 alignment mark 1101, 1102 Alignment target marker 1200 θ -axis turning center 1201 imaginary marking 1300 imaging area 1301, 1314 search area 1312, 1322 model 1313 patterns 1323 Reference point QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2020-110916

[0005]

Claims

[1] Alignment system comprising: an image processor for detecting an identification target in a captured image of a workpiece including the identification target by a coarse search and a fine search which are different from each other; and an operation controller for controlling a drive for moving the workpiece, wherein the control is performed based on actual coordinates of the identification target detected by the image processor, wherein the detection of the identification target by the image processor and the control of the drive by the operation control are carried out repeatedly, and wherein the fine search is performed in a search area smaller than an area for the coarse search, and wherein the image processor performs the fine search as a next search without performing the coarse search when the identification target is detected by the fine search. [2] The alignment system according to claim 1, wherein, when the identification target is detected by the fine search, the image processor determines a search area based on the actual coordinates obtained by the fine search and performs the fine search in the determined search area as the next search. [3] The alignment system according to claim 1, wherein, when the identification target is detected by the fine search performed in a search area determined by the coarse search, the image processor performs, as the next search, the fine search in the search area searched in the fine search. [4] The alignment system according to any one of claims 1 to 3, wherein, when the actual coordinates of the identification target are not detected by the fine search, the image processor performs, as the next search, the coarse search for determining a search area and the fine search within the determined search area. [5] The alignment system according to any one of claims 1 to 4, wherein the image processor performs the fine search in a search area determined by pre-calibration, performs the coarse search to determine a search area when the identification target is not detected by the fine search, and performs the fine search in the determined search area. [6] The alignment system according to any one of claims 1 to 5, wherein the coarse search includes pattern matching for performing a match of a pattern included in the workpiece with a pre-registered pattern model, and when the pattern has a matching rate greater than or equal to a predetermined threshold, a range of the fine search is determined based on a position or a rotation angle of the pattern. [7] The alignment system according to any one of claims 1 to 5, wherein the coarse search includes pattern matching for performing a match of a pattern included in the workpiece with a pre-registered pattern model, and when a plurality of the patterns have a match rate greater than or equal to a predetermined threshold, a range of the fine search is determined based on a position or a rotation angle of a pattern having a highest match rate among the plurality of patterns. [8] Alignment system according to claim 6 or 7, further comprising: a setting terminal for registering the pattern model to be used for pattern matching in the rough search, wherein, when the fine search is a search for an edge of the pattern included in the workpiece, the setting terminal generates, upon registering the pattern model, a search logic for the fine search based on a shape of the pattern model. [9] Alignment process comprising: Detecting an identification target from a captured image of a workpiece comprising the identification target by a coarse search or a fine search; and Providing a command to a drive to move the workpiece, wherein the command is provided based on actual coordinates of the detected identification target, wherein the detection of the identification target and the provision of the command are performed repeatedly, and wherein the fine search is performed as a next search without performing the coarse search when the identification target is detected by the fine search. [10] A program executable by a computer in an alignment system to detect an identification target included in a workpiece in a captured image of the workpiece and to perform alignment control of the workpiece based on actual coordinates of the detected identification target, the computer being configured to perform image processing on the captured image, the program causing the computer to act as: a finder for detecting the identification target in the captured image by a coarse search or a fine search, and for performing the fine search as a next search without performing the coarse search when the identification target is detected by the fine search.

Images