Alignment system, alignment method, and program

Abstract

An alignment system (1) is provided with: an image processing means (400) that detects an object to be recognized from a captured image in which a workpiece (10) having the object to be recognized is captured by means of a rough search and a detailed search, which are different from each other; and an operation control means (200) that controls drive instruments (111, 112, 113) that move the workpiece (10) on the basis of the actual coordinates of the recognition target detected by the image processing means (400). The detection of the recognition object by the image processing unit (400) and the control of the drive instruments (111, 112, 113) by the operation control unit (400) are repeatedly performed. The detailed search is performed within a search range narrower than the range of the rough search, and the detailed search is performed instead of the rough search in the next search when the recognition target is detected by the successful detailed search.

Description

Technical Field

[0001] The present invention relates to an alignment system, an alignment method and a program. Background Art

[0002] In the field of FA (Factory Automation), a positioning technology is used to align the position of a workpiece as a control object with a reference position. For example, there is a control system that detects the workpiece from an image captured by a camera and performs positioning by aligning the position of the workpiece with a reference position (for example, Patent Document 1).

[0003] In the actuator control system described in Patent Document 1, an instruction for driving the actuator is issued based on data detected by performing a first position detection process and a second position detection process on a captured image, wherein the first position detection process detects the position of the target object with a first accuracy, and the second position detection process detects the position of the target object with an accuracy higher than the first accuracy.

[0004] The following situation is described, that is, in this actuator control system, the actuator is preliminarily driven based on the intermediate data obtained by the first position detection process, and the actuator is further driven based on the difference between the final data obtained by the second position detection process and the intermediate data. Thus, the drive control is preliminarily started before the calculation of the final data is completed, so that the control can be accelerated.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-110916 Summary of the invention

[0006] In the actuator control system described in Patent Document 1, a rough search is used as the first position detection process, and a fine search is used as the second position detection process. The purpose of the rough search is to roughly detect the position of the target object from a large-scale image, for example, using pattern matching. On the other hand, the purpose of the fine search is to accurately detect the position of the target object from a small-scale image, for example, using edge detection.

[0007] Fine search is based on a small range search, so the search range needs to be limited, but the rough search for limiting the search range also has the problem that the search processing load becomes large and the processing time becomes long because the range of the position and rotation angle of the target pattern is large. Therefore, the alignment control that issues control commands based on the data detected by executing both the rough search and the fine search has the problem that the overall control processing time becomes long.

[0008] The present invention has been made in view of the above-mentioned circumstances, and an object of the present invention is to provide an alignment system, an alignment method, and a program capable of performing positioning at high speed and high precision.

[0009] In order to achieve the above-mentioned object, the alignment system of the present invention is characterized by comprising: an image processing unit which detects the identification object by using different search methods, namely, a rough search and a detailed search, from a captured image obtained by capturing a workpiece having the identification object;

[0010] and an action control unit that controls a driving device that moves the workpiece based on the actual coordinates of the recognition object detected by the image processing unit. The detection of the recognition object by the image processing unit and the control of the driving device by the action control unit are repeatedly performed. The detailed search is performed within a search range narrower than the range of the rough search, and when the recognition object is successfully detected by the detailed search, the next search is performed without performing the rough search but performing the detailed search.

[0011] Effects of the Invention

[0012] According to the present invention, when the identification target is successfully detected by the detailed search, the next search is not a rough search but a detailed search is performed, so that the overall processing time can be reduced and positioning can be performed at high speed and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing an overall configuration example of the alignment system involved in embodiment 1.

[0014] Figure 2 This is a block diagram showing a configuration example of an action control unit.

[0015] Figure 3 This is a block diagram showing a configuration example of an image processing unit.

[0016] Figure 4 It is a diagram for explaining the alignment control method.

[0017] Figure 5 It is a diagram for explaining the alignment control method.

[0018] Figure 6 This is a block diagram showing an example of the functional structure of the alignment system involved in embodiment 1.

[0019] Figure 7 This is a flowchart showing the alignment control process involved in the first embodiment.

[0020] Figure 8 This is a flowchart showing the search process involved in Implementation Example 1.

[0021] Fig.9AThis is a diagram showing an example of a search range.

[0022] Fig. 9B It is a diagram showing an example of the search angle range.

[0023] Fig.10 This is a diagram showing a synchronization method based on transmission path delay measurement.

[0024] Fig.11 This is a diagram for explaining the timing control of the shutter of the imaging device.

[0025] Fig.12 This is a block diagram showing an example of the functional structure of the alignment system involved in Implementation Example 2.

[0026] Fig.13 This is a flowchart showing the alignment control processing involved in the second embodiment.

[0027] Fig.14 This is a flowchart showing the search process involved in Implementation Example 2.

[0028] Fig.15 This is a block diagram showing an example of the functional structure of the alignment system involved in the first modification.

[0029] Fig.16 : is a flowchart showing the alignment control process according to the first modification.

[0030] Fig.17 This is a flowchart showing the search process involved in Modification Example 1.

[0031] Fig.18A This is a diagram for explaining the prediction of the recognition target according to Modification 3.

[0032] Fig.18B This is a diagram for explaining the prediction of the recognition target according to Modification 3.

[0033] Fig.19A This is a diagram for explaining a rough search according to Modification 4.

[0034] Fig.19B This is a diagram for explaining the detailed search according to Modification 4.

[0035] Fig. 20A This is a diagram for explaining a rough search according to Modification 5.

[0036] Fig. 20B This is a diagram for explaining the detailed search according to Modification 5.

[0037] Fig.21 This is a flowchart showing the search process involved in Modification Example 6. DETAILED DESCRIPTION

[0038] (Implementation Method 1)

[0039] Hereinafter, Embodiment 1 for implementing the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0040] Figure 1 1 is a block diagram showing an overall configuration example of an alignment system 1 according to Embodiment 1. The alignment system 1 includes an alignment mechanism 100 that moves a workpiece 10 as a control object; an action control unit 200 that controls the alignment mechanism 100; an imaging device 300 that images the workpiece 10; an image processing unit 400 that processes the image captured by the imaging device 300; and a setting terminal 500 that performs various settings used for processing in the image processing unit 400.

[0041] The alignment mechanism 100 is connected to driving devices 111, 112, and 113, and the driving devices 111, 112, and 113 provide driving forces in various directions to move the workpiece 10. In addition, the driving devices 111, 112, and 113 are connected to driving control devices 121, 122, and 123, respectively, and based on the control signal from the motion control unit 200, the driving control devices 121, 122, and 123 drive the driving devices 111, 112, and 113, respectively.

[0042] The drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are connected to each other for communication. The communication unit may be any conventional communication unit such as Ethernet (registered trademark), CameraLink, CoaxPress (registered trademark), USB (Universal Serial Bus, registered trademark), etc., but in order to ensure synchronization, an Ethernet-based industrial network such as CC-Link IE / field (registered trademark) and CC-Link IE / TSN (registered trademark) is preferred.

[0043] The alignment mechanism 100 has a stage 101 for placing the workpiece 10, and a mechanism for moving the stage 101. For example, the stage can be translated in the X direction and the Y direction which are horizontal and orthogonal to each other, and can be rotationally moved in the θ direction which is a rotation direction on the horizontal plane. In this embodiment, the driving device 111 coupled to the alignment mechanism 100 is translated in the X direction, the driving device 112 is translated in the Y direction, and the driving device 113 is rotationally moved in the θ direction.

[0044] The driving devices 111, 112, 113 are any driving devices (actuators) capable of precisely driving the alignment mechanism 100, such as servo motors. The driving control devices 121, 122, 123 are control devices that control the driving of the driving devices 111, 112, 113 based on control signals from the motion control unit 200, such as servo amplifiers.

[0045] The driving devices 111, 112, and 113 have any position sensor inside or outside, and detect and output the position actually changed by the driving of the driving devices 111, 112, and 113. The position sensor is, for example, an encoder attached to the actuator. The output signal of the encoder is input to the motion control unit 200 via the driving control devices 121, 122, and 123. In this embodiment, the driving devices 111, 112, and 113 are servo motors, the driving control devices 121, 122, and 123 are servo amplifiers, and the position sensor is an encoder.

[0046] The motion control unit 200 is a motion controller that provides instructions related to the actions of the drive devices 111, 112, 113 to the drive control devices 121, 122, 123, and includes, for example, a PLC (Programmable Logic Controller). The motion control unit 200 generates a control signal indicating the instruction based on information obtained from the drive control devices 121, 122, 123 and the image processing unit 400, and outputs it to the drive control devices 121, 122, 123.

[0047] like Figure 2 As shown, the motion control unit 200 includes a processor 210, a volatile memory 220, a nonvolatile memory 230, a clock 240, and a communication interface 250. The processor 210, the volatile memory 220, the nonvolatile memory 230, the clock 240, and the communication interface 250 are communicatively connected to each other via a bus B1.

[0048] The processor 210 is, for example, a CPU (Central Processing Unit), and functions as a position information generating unit 211 , a movement amount calculating unit 212 , and an instruction unit 213 by reading and executing a control program 231 stored in a nonvolatile memory 230 .

[0049] The position information generating unit 211 of the processor 210 generates position information corresponding to the position of the mounting table 101 on which the workpiece 10 is mounted, based on the output signal of the position sensor of the driving instruments 111, 112, and 113. The movement amount calculating unit 212 calculates the movement amount of the workpiece 10 in the X direction, the Y direction, and the θ direction, based on the position of the recognition object detected from the captured image by the image processing unit 400. The instruction unit 213 outputs a control signal based on the movement amount calculated by the movement amount calculating unit 212 to the drive control devices 121, 122, and 123.

[0050] The volatile memory 220 is a working memory capable of reading and writing data at high speed during the operation processing executed by the processor 210, such as a RAM (Random Access Memory). The non-volatile memory 230 stores a control program 231 and control data 232. The control program 231 is used to realize various functions of the motion control unit 200, and the control data 232 includes parameters used when executing the control program 231, past detection data, and instruction data. The non-volatile memory 230 is, for example, a non-volatile semiconductor memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory, a magnetic disk, or an optical disk.

[0051] The clock 240 measures the local time by counting the clock signal of the clock element of the motion control unit 200, and then synchronizes with the drive control devices 121, 122, 123, the imaging device 300, and the image processing unit 400. Thus, the motion control unit 200 has the time information synchronized with the drive control devices 121, 122, 123, the imaging device 300, and the image processing unit 400.

[0052] The communication interface 250 is an interface used by the motion control unit 200 to communicate with the drive control devices 121 , 122 , 123 , the imaging device 300 , and the image processing unit 400 , and is an interface compliant with communication standards such as CC-Link IE / field and CC-Link IE / TSN.

[0053] The imaging device 300 is an imaging device that images the workpiece 10 at a constant time interval from above the alignment mechanism 100, and is, for example, a camera having a resolution sufficient for achieving the alignment accuracy of the alignment system 1. The number of imaging devices 300 is arbitrary, but is determined according to the number and positions of the recognition objects used for alignment. Figure 1The case where two imaging devices 300 are provided is shown. The recognition target may be any target indicating the position of the workpiece 10 , such as an alignment mark added to the workpiece 10 , a corner end of the workpiece 10 , or a corner end of the stage 101 .

[0054] The photographing device 300 also has timing information synchronized with the drive control devices 121 , 122 , 123 , the motion control unit 200 , and the image processing unit 400 , and a communication interface for communicating with the drive control devices 121 , 122 , 123 , the motion control unit 200 , and the image processing unit 400 .

[0055] The image processing unit 400 performs a process of detecting the recognition object by using different search methods, namely, a rough search and a detailed search, for the captured image obtained from the image capturing device 300. The rough search is performed in the entire range of the captured image obtained from the image capturing device 300, and the detailed search is performed in a search range smaller than the range of the rough search. When the image processing unit 400 detects the recognition object through the detailed search, it outputs the actual coordinates of the recognition object to the motion control unit 200. Here, the actual coordinates are coordinates under the reference coordinates that have been obtained in agreement with the movement amount calculation unit 212 of the motion control unit 200.

[0056] like Figure 3 As shown, the image processing unit 400 includes a processor 410, a volatile memory 420, a nonvolatile memory 430, a clock 440, and a communication interface 450. The processor 410, the volatile memory 420, the nonvolatile memory 430, the clock 440, and the communication interface 450 are communicatively connected to each other via a bus B2.

[0057] The processor 410 is, for example, a CPU (Central Processing Unit), and functions as an image acquisition unit 411 , a range determination unit 412 , and a search unit 413 by reading and executing a control program 431 stored in a nonvolatile memory 430 .

[0058] The image acquisition unit 411 of the processor 410 acquires an image captured by the camera 300. The range determination unit 412 determines a range for searching for the recognition object in the captured image acquired by the image acquisition unit 411 based on information including the position information generated by the position information generation unit 211 of the motion control unit 200. The search unit 413 performs a detailed search for the recognition object for the image within the search range determined by the range determination unit 412, and outputs the actual coordinates of the recognition object to the motion control unit 200 when the recognition object is successfully detected.

[0059] The volatile memory 420 is a working memory capable of reading and writing data at high speed during the operation processing executed by the processor 410, such as a RAM (Random Access Memory). The non-volatile memory 430 stores a control program 431 for realizing various functions of the image processing unit 400 and control data 432, wherein the control program 431 is used to realize various functions of the image processing unit 400, and the control data 432 includes parameters used when executing the control program 431 and past detection data. The non-volatile memory 430 is, for example, a non-volatile semiconductor memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory, a magnetic disk, or an optical disk.

[0060] The clock 440 measures the local time by counting the clock signal of the clock element of the image processing unit 400, and then synchronizes with the drive control devices 121, 122, 123, the action control unit 200, and the imaging device 300. Thus, the image processing unit 400 has the time information synchronized with the drive control devices 121, 122, 123, the action control unit 200, and the imaging device 300.

[0061] The communication interface 450 is an interface used by the image processing unit 400 to communicate with the drive control devices 121 , 122 , 123 , the motion control unit 200 , and the imaging device 300 , and is, for example, an interface that complies with communication standards such as CC-Link IE / field and CC-Link IE / TSN.

[0062] The setting terminal 500 is a terminal in which an application corresponding to the control program 431 of the image processing unit 400 is installed, such as a personal computer. The setting terminal 500 has a management function of the image processing unit 400, and the management function includes inputting or changing parameters stored in the non-volatile memory 430 of the image processing unit 400. The communication interface of the setting terminal 500 for communication connection with the image processing unit 400 is an arbitrary interface corresponding to the interface of the image processing unit 400, such as a USB interface or an RS232C interface.

[0063] The motion control unit 200 calculates the movement amount based on the difference between the actual coordinates of the recognition object detected by the image processing unit 400 and the target coordinates of the aiming target, and outputs a control signal based on the movement amount to the drive control devices 121 , 122 , and 123 .

[0064] The operation of the alignment system 1 having the above-described structure will be described. Figure 4 , 5 An example of an alignment method based on the position of an identification object as a premise will be described. Figure 4 , 5 This is a diagram for explaining an alignment method when the recognition target is the alignment marks 1001 and 1002 added to the end portions of the workpiece 10 .

[0065] The shapes of the alignment marks 1001 and 1002 are arbitrary, but preferably a shape that can clearly identify the position of the reference point and the rotation angle, for example, preferably Figure 4 , 5 In addition, the number of alignment marks is arbitrary and can be as follows Figure 4 The figure shows 1, but it can also be Figure 5 As shown, two alignment marks 1001 and 1002 are added to the diagonal ends of the workpiece 10 .

[0066] When one alignment mark 1001 is used, one imaging device 300 acquires an image of the imaging range 1300. When two alignment marks 1001 and 1002 are used and the two alignment marks 1001 and 1002 cannot be captured in one imaging range 1300, two imaging devices 300 acquire images of the imaging ranges 1300 that are offset from each other.

[0067] In such Figure 4 In the case where one alignment mark 1001 is used, the following three pieces of information are used: coordinates of a reference point of the alignment target mark 1101 , coordinates of a reference point of the alignment mark 1001 on the workpiece 10 , and an angle difference between the alignment target mark 1101 and the alignment mark 1001 .

[0068] On the other hand, in Figure 5 In the case where two alignment marks 1001 and 1002 are used as shown, the following three information are used: the midpoint coordinates of the reference points of the alignment target marks 1101 and 1102, the midpoint coordinates of the reference points of the alignment marks 1001 and 1102 on the workpiece 10, and the angle difference between the straight line connecting the alignment target marks 1101 and 1102 and the straight line connecting the alignment marks 1001 and 1002.

[0069] exist Figure 4 , 5 In the case of , the angle difference Δθ is the displacement of the θ axis. Figure 4As shown, the motion control unit 200 obtains the virtual coordinates of the reference point of the virtual mark 1201 after the workpiece 10 is rotated by an angle Δθ with the θ-axis rotation center 1200 of the alignment mechanism 100 as the center. Then, the movement amount ΔX in the X-axis direction and the movement amount ΔY in the Y-axis direction are calculated by the difference between the virtual coordinates and the target coordinates of the alignment target mark 1101. The command unit 213 of the motion control unit 200 sends a control signal for realizing the movement of Δθ, ΔX, and ΔY obtained in this way to the drive control devices 121, 122, and 123, thereby issuing a command.

[0070] like Figure 4 As shown in FIG. 1 , the method using one alignment mark 1001 can be used when the rotation of the alignment mark 1001 can be clearly determined like a cross mark. Figure 5 As shown, when two alignment marks 1001 and 1002 are used, it is not necessary to detect the rotation of the alignment marks themselves, so even if the alignment marks are circular, the rotation of the workpiece can be detected. In addition, the angle difference Δθ in the θ direction is calculated by detecting two points that are sufficiently far apart, so that high-precision control can be performed.

[0071] In this way, the motion control unit 200 calculates Δθ, ΔX, and ΔY based on the coordinates and angle differences of the reference points of the alignment marks 1001 and 1002, which are the objects to be recognized, and the drive control devices 121, 122, and 123 use these values ​​to control the drive devices 111, 112, and 113 to move the workpiece 10. However, it is often impossible to make the difference from the target coordinates fall within the allowable range at once, so the detection of the object to be recognized and the control of the drive devices 111, 112, and 113 are repeatedly performed.

[0072] In such a repetitive process, the processing load of searching for the identification target from the captured image obtained by the imaging device 300 is very large. Therefore, the alignment system 1 according to the present embodiment limits the range in which the search is performed to reduce the processing load.

[0073] Below, use Figure 6-8 The detailed processing will be described. Figure 6 is a block diagram showing an example of the functional configuration of the alignment system 1 according to the present embodiment. Figure 7 is a flowchart of the alignment control process performed by the motion control unit 200, Figure 8 is a flowchart of a search process performed by the image processing unit 400 .

[0074] First, the motion control unit 200 issues a command to move the workpiece 10 to a preset estimated target position ( Figure 7, step S101). Specifically, the motion control unit 200 outputs control signals for realizing movement in various directions to the drive control devices 121, 122, and 123. Furthermore, by controlling the drive control devices 121, 122, and 123 based on the control signals, the drive instruments 111, 112, and 113 are driven, and the mounting table 101 on which the workpiece 10 is mounted moves.

[0075] After the movement is completed, the motion control unit 200 instructs the image processing unit 400 to obtain and search for captured images (step S102), and generates and outputs position information based on the output of the position sensors of the driving devices 111, 112, and 113 (step S103). Then, the motion control unit 200 waits until the search of the image processing unit 400 is completed (step S104).

[0076] In step S102, the image processing unit 400 instructed by the action control unit 200 to perform the search process of the recognition object executes Figure 8 Here, since it is the first time to shoot (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 acquires the image shot by the image capturing device 300 (step S202). Then, the search unit 413 performs a rough search (rough search) for the recognition object (step S203).

[0077] For example, a rough search is performed by pattern matching processing using a pre-registered pattern model of the alignment mark 1001. The search unit 413 detects the alignment mark 1001 when the coincidence rate of pattern matching is greater than or equal to a predetermined threshold value, and determines a search range for a detailed search based on the position or rotation angle of the reference point of the alignment mark 1001 (step S204). The shape or size of the search range at this time is pre-set according to the shape or size of the alignment mark 1001, or can be set by a user input to the setting terminal 500.

[0078] For example, in Fig.9A In the case of the alignment mark 1001 of the cross shown in the figure, the position of the center, i.e. the reference point, and the rotation angle of the cross are determined by pattern matching, and a search range 1301 of a size obtained by adding a predetermined margin to the size of the alignment mark 1001 is determined. Fig. 9B The angles shown relate to the search range (search angle range).

[0079] Next, the search unit 413 performs a detailed search (fine search) on the image within the search range determined in step S204 (step S205). As a detailed search, for example, by performing edge detection, an accurate straight line or curve is detected, so that the actual coordinates of the recognition object can be obtained more accurately. Here, the actual coordinates of the recognition object include the position (XY coordinates) and the rotation angle (θ coordinate) of the reference point of the recognition object.

[0080] If the search unit 413 cannot detect the recognition object in the detailed search of step S205 (step S212: No), the process returns to step S201 and repeats the processes of steps S202 to S205. On the other hand, if the recognition object is successfully detected in the detailed search of step S205 (step S212: Yes), the actual coordinates of the recognition object obtained in step S205 are output to the motion control unit 200 (step S213).

[0081] Here, return Figure 7 , after the search process performed by the image processing unit 400 is completed (step S104: Yes), the action control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (step S106: Yes), the alignment control process is terminated. Here, the target coordinates are the coordinates of the alignment target, including the center position (XY coordinates) and the rotation angle (θ coordinate).

[0082] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the command unit 213 commands the alignment mechanism 100 to be corrected (step S107; command step). Specifically, the control signal for aligning the actual coordinates with the target coordinates is output to the drive control devices 121, 122, and 123.

[0083] After the driving of the driving devices 111, 112, and 113 is completed, the process returns to step S102. The motion control unit 200 again instructs photographing and searching (step S102), and outputs position information (step S103).

[0084] Transfer again to Figure 8 , since the image is captured for the second time or later and the previous detection was successful (step S201: No), the image processing unit 400 obtains the position information (step S206). The range determination unit 412 of the image processing unit 400 calculates the predicted coordinates of the recognition target at that time (step S207).

[0085] Specifically, the range determination unit 412 determines the correspondence between the position information obtained from the motion control unit 200 during the initial alignment control and the actual coordinates detected during the initial detailed search. For example, when the XY coordinates of the initial position information are represented by (x1, y1) and the XY coordinates of the actual coordinates are represented by (X1, Y1), their relationship is represented by the constant matrix A of the following formula (1).

[0086] [Mathematical formula 1]

[0087] A=(X1,Y1) / (x1,y1)…(1)

[0088] If the XY coordinates of the position information obtained from the motion control unit 200 during the second alignment control are set to (x2, y2), the XY coordinates (X2, Y2) of the predicted coordinates used in the second search can be expressed by the following formula (2) using the constant matrix A expressed by formula (1). In addition, the third and subsequent n-th searches can also be expressed by formula (3).

[0089] [Mathematical formula 2]

[0090] (X2,Y2)=A(x2,y2)…(2)

[0091] (X n , Y n )=A(x n ,y n )…(3)

[0092] In addition, the constant matrix A representing the correspondence between the position information and the actual coordinates may be updated each time based on the position information and the actual coordinates obtained in the repeated alignment control, or an average of multiple times may be used. Alternatively, the correspondence between the position information and the actual coordinates may be pre-established by prior calibration. For example, in the case of alignment control using the corner end of the stage 101 as the identification object, the error each time is performed is small, so such prior calibration is effective.

[0093] In this way, the correspondence between the XY coordinates of the reference point of the identification object detected from the captured image and the position information is predetermined, and the XY coordinates are predicted based on the newly acquired position information using the correspondence (step S207). Then, a search range of a predetermined size is determined with the predicted XY coordinates as the center (step S208). In addition, similarly, for the θ coordinate, on the basis of the predetermined correspondence between the actual coordinates detected by the detailed search and the position information, the θ coordinate representing the rotation angle of the identification object is predicted based on the newly acquired position information (step S207), and a search angle range of a predetermined angle amplitude is determined with the predicted θ coordinates as the center (step S208). In addition, the search range may not be a range centered on the predicted coordinates, and a range containing the predicted coordinates may be used as the search range depending on the alignment conditions or the form of the identification object.

[0094] Here, the size of the search range may be set by a user input to the setting terminal 500, or may be set automatically. For example, the size of the search range may be a size after adding a margin to the shape or size of the recognition object, and the margin is automatically or manually set based on the moving speed of the driving instruments 111, 112, 113. Alternatively, the size of the search range may be statistically determined based on past alignment control results, for example, the size after adding a margin to the shape and size of the recognition object, and the margin is the average value of the difference between the predicted coordinates calculated based on the position information at the corresponding time points in the past and the actual coordinates detected from the captured image.

[0095] That is, the range determination unit 412 does not perform the rough search of step S203 in the second and subsequent detections and when the previous detection has not failed, and determines the search range centered on the predicted coordinates calculated based on the position information output by the position sensor in step S207. The processing for determining the search range based on the position information is significantly reduced in processing volume compared to the processing for determining the search range obtained by the rough search performed in step S203. As a result, high-speed control can be achieved compared to the conventional alignment control performed after each rough search.

[0096] Next, the image acquisition unit 411 acquires an image (step S209; image acquisition step), and performs a simplified process on the image of the search range determined in step S208 in the acquired image (step S210). The simplified process is an arbitrary process before performing a detailed search. For example, the search unit 413 may perform a rough search within a range defined by a whole range larger than the search range determined in step S208, and determine the search range again. Alternatively, when the predicted coordinates of the XY coordinates are calculated in step S207 and the search range in the XY coordinates is determined in step S208, a rough search of the θ coordinates may be performed as a simplified process to determine the search angle range. In addition, the simplified process may be omitted.

[0097] Then, the search unit 413 performs a detailed search on the search range determined in step S208 or the range determined in step S210 (step S211: search step). The method of the detailed search is the same as that of step S205. If the result of the detailed search in step S211 is that the identification object cannot be detected (step S212: No), the process returns to step S201, and since the previous detection failed (step S201: Yes), the process of steps S202 to 205 is performed. Here, the reason for restarting from the rough search of the entire range is to avoid the situation where the detailed search fails repeatedly and becomes uncontrollable.

[0098] When the recognition target is successfully detected in the detailed search of step S211 (step S212 : Yes), the actual coordinates of the recognition target acquired in step S211 are output to the motion control unit 200 .

[0099] Return again Figure 7 According to the flowchart of , the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold (step S106: Yes), the alignment process ends.

[0100] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the command unit 213 executes a command to correct the position of the alignment mechanism 100 (step S107). Specifically, a control signal for aligning the actual coordinates with the target coordinates is output to the drive control devices 121, 122, and 123. Then, the process returns to step S102 to continue.

[0101] As described above, the alignment system 1 according to the present embodiment includes an image processing unit 400 that detects the identification object from an image obtained by photographing the workpiece 10 having the identification object, and an action control unit that controls the driving instruments 111, 112, and 113 that move the workpiece 10 based on the actual coordinates of the identification object detected by the image processing unit 400. The image processing unit 400 predetermines the correspondence between the position information output by the position sensor based on the driving instruments 111, 112, and 113 and the actual coordinates of the identification object detected from the photographed image, calculates the predicted coordinates of the identification object based on the position information obtained next using the correspondence, determines the search range including the predicted coordinates, and detects the identification object based on the image of the search range. As a result, the rough search process of the entire range can be omitted, and positioning can be performed at high speed and with high precision.

[0102] (Implementation Method 2)

[0103] Hereinafter, Embodiment 2 for implementing the present invention will be described in detail with reference to the drawings. In addition, the same reference numerals are attached to the same or corresponding parts in the drawings.

[0104] The overall structure of the alignment system 2 involved in this embodiment 2 and the hardware structure of each component are the same as those in embodiment 1. In the alignment system 2 involved in this embodiment 2, the difference from embodiment 1 is that the image acquisition unit 411 of the image processing unit 400 acquires the captured image from the shooting device 300 by issuing a forwarding instruction specifying a forwarding range, and the search unit 413 performs search processing on the captured image within the forwarding range.

[0105] In this embodiment, the drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are connected to each other for communication, as in Embodiment 1. However, this embodiment seeks higher responsiveness than Embodiment 1, and therefore, it is preferred that they are connected to each other for communication via an industrial network such as CC-Link IE / field or CC-Link IE / TSN.

[0106] Industrial networks such as CC-Link IE / field and CC-Link IE / TSN have a mechanism to ensure synchronization: a certain degree of timing or more is achieved through H / W instruments, and transmission path delays are statistically measured to align the timing between instruments with μ-second accuracy.

[0107] The synchronization method based on transmission path delay measurement is a method that uses the transmission path delay measurement value from the master station to the device station to achieve higher-precision synchronization. Fig.10The synchronization method based on the transmission path delay measurement is shown. In the transmission path delay measurement method, synchronization is performed at a synchronization point. The transmission control (MyStatus) frame sent from the master station 40 is delayed as the distance increases.

[0108] The master station 40 calculates the transmission path delay time of each device station 50, 60, 70 based on the master station time when the response signal is received from each device station 50, 60, 70, and sends it to each device station 50, 60, 70. The synchronization point is the time after a constant time (Tsync) has passed after the master station 40 sends the transmission control (MyStatus) frame. After receiving the transmission control (MyStatus) frame, each device station 50, 60, 70 synchronizes after the time Tps (Tsync-delay time) is subtracted from the transmission path delay time.

[0109] In the present embodiment, the transmission path delay measurement method is used, for example, with the motion control unit 200 as the master station 40 to perform timing control of the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400. That is, the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400 are connected to each other for communication via an industrial network (communication line) for measuring the transmission path delay time, and have synchronized time information.

[0110] The action control unit 200 as the master site 40 sets a specific time after the longest transmission path delay time has passed since the instructed transmission time as a synchronization point. Furthermore, the image processing unit 400 determines a range including the predicted coordinates at the specific time in the future as a forwarding range, instructs the imaging device 300 to capture images at the specific time, and forwards images within the forwarding range among the captured images.

[0111] Furthermore, the imaging device 300 is subjected to timing control that also takes the shutter speed into consideration. Fig.11 This is a diagram for explaining the timing control of the shutter of the camera 300. The camera 300 automatically determines the shooting timing according to the information obtained from the image processing unit 400 or the pre-set information, using the calculation source of the synchronization point, that is, the time information of the synchronization. At this time, in order to shoot at a specific time in the future, the image processing unit 400 can issue a prior shooting instruction within the grace time as needed. In addition, it is also useful to align the timing of the scheduled shooting with the center of the exposure time range.

[0112] Fig.12 is a block diagram showing an example of the functional structure of the alignment system 2 involved in this embodiment. Fig.13is a flowchart of the alignment control process performed by the motion control unit 200, Fig.14 FIG. 4 is a flowchart of the search process performed by the image processing unit 400. Fig.13 , 14 The flowchart of will explain the operation of the alignment system 2.

[0113] First, the motion control unit 200 issues a command to move the workpiece 10 to a preset estimated target position ( Fig.13 , step S101). Specifically, the motion control unit 200 outputs control signals for realizing movement in various directions to the drive control devices 121, 122, and 123. Furthermore, through the control of the drive control devices 121, 122, and 123 based on the control signals, the drive instruments 111, 112, and 113 are driven, and the mounting table 101 on which the workpiece 10 is mounted moves.

[0114] After the movement instruction is issued, the motion control unit 200 determines the shooting time of the imaging device 300 as time T n (n=1) (step S122), and specifies to the image processing unit 400. In addition, the position information generation unit 211 of the motion control unit 200 generates position information based on the output of the position sensor of the driving equipment 111, 112, 113, and outputs the position information (step S103). Then, the motion control unit 200 waits until the search of the image processing unit 400 is completed (step S104).

[0115] The image processing unit 400 that has acquired the position information from the motion control unit 200 executes Fig.14 Here, since it is the first time to shoot (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 instructs the imaging device 300 to set the time T n (n=1) is photographed, and the photographed image of the entire range is forwarded (step S221). Then, the image acquisition unit 411 acquires the photographed image of the entire range (step S222).

[0116] Next, the search unit 413 of the image processing unit 400 performs a rough search (rough retrieval) of the entire range of the captured image for the identification object (step S203). For example, the rough search is performed by pattern matching processing implemented using a pattern model of the pre-registered alignment mark 1001. The search unit 413 detects the alignment mark 1001 as the identification object when the consistency rate of pattern matching is greater than or equal to a predetermined threshold, and determines the search range of the detailed search based on the position or rotation angle of the reference point of the alignment mark 1001 (step S204). The shape or size of the search range at this time is predetermined based on the shape or size of the alignment mark 1001, and can also be set by the user's input to the setting terminal 500.

[0117] Next, a detailed search (fine search) is performed on the image within the search range determined in step S204 (step S205). The detailed search can detect accurate straight lines or curves by performing edge detection, so that more accurate actual coordinates of the recognized object can be obtained. Here, the actual coordinates of the recognized object include the position (XY coordinates) and the rotation angle (θ coordinate) of the reference point of the recognized object.

[0118] If the recognition object cannot be detected in the detailed search of step S205 (step S212: No), the process returns to step S201 and the processes of steps S221, S222, and S203 to S205 are repeated. On the other hand, if the recognition object is successfully detected in the detailed search of step S205 (step S212: Yes), the actual coordinates of the recognition object obtained in step S205 are output to the motion control unit 200 (step S213).

[0119] return Fig.13 , after the search process of the image processing unit 400 is completed (step S104: Yes), the action control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (step S106: Yes), the alignment control process is terminated. Here, the target coordinates are the coordinates of the alignment target, including the center position (XY coordinates) and the rotation angle (θ coordinate).

[0120] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the command unit 213 commands the alignment mechanism 100 to be corrected (step S107; command step). Specifically, the control signal for aligning the actual coordinates with the target coordinates is output to the drive control devices 121, 122, and 123.

[0121] After issuing the calibration command to the alignment mechanism 100, the motion control unit 200 returns to step S122 and determines the photographing time as time T n (n=2) (step S122), output the position information (step S103), and wait until the search process of the image processing unit 400 is completed (step S104).

[0122] Return again Fig.14 , the next shooting is the second time and thereafter and the previous detection is successful (step S201: No), so the image processing unit 400 instructs the shooting device 300 to n (n=2) (step S223). The image processing unit 400 also obtains position information from the motion control unit 200 (step S206). The range determination unit 412 of the image processing unit 400 predicts the coordinates of the recognition target at that time (step S207).

[0123] The method for predicting coordinates is the same as that in Embodiment 1, and the correspondence between the position information obtained from the motion control unit 200 during the initial alignment control and the actual coordinates detected in the initial detailed search is determined. The range determination unit 412 uses the determined correspondence to predict the XY coordinates of the reference point of the identification object based on the position information obtained in step S206 (step S207). Then, the range determination unit 412 determines a forwarding range of a predetermined size with the XY coordinates predicted in step S207 as the center, and issues a forwarding instruction to the shooting device 300 for the captured image with the forwarding range specified (step S226). In addition, the forwarding range may not be a range centered on the predicted XY coordinates, and the range containing the predicted XY coordinates may be used as the forwarding range depending on the alignment conditions or the form of the identification object.

[0124] Here, the size of the forwarding range can be set by the user input to the setting terminal 500. For example, there is a trade-off between robustness and processing speed, so the forwarding range can be set small when productivity is prioritized by allowing processing deviation and maximizing the average processing speed. On the other hand, when jitter characteristics are prioritized over processing speed and the processing time is to be constant, the forwarding range can be set large.

[0125] Alternatively, the size of the forwarding range may be a size obtained by adding a margin to the shape or size of the recognition object, the margin being automatically or manually set based on the moving speed of the driving devices 111, 112, 113. Alternatively, the range determination unit 412 may statistically determine the size of the forwarding range based on past alignment control results, for example, the size obtained by adding a margin to the size of the recognition object may be used as the size of the forwarding range, the margin being the average value of the difference between the predicted coordinates calculated based on past position information and the actual coordinates of the recognition object detected from the image.

[0126] Thus, when the image is taken for the second time or thereafter and the previous detection has not failed, the image capturing device 300 does not forward the image of the entire range, but only forwards the image of the forwarding range centered on the predicted coordinates based on the position information of the position sensor obtained in step S226. The data volume of the image forwarded here is greatly reduced compared to the data volume of the image of the entire range forwarded in step S221. Thus, compared to the case where the image of the entire range is forwarded each time, the forwarding volume of data can be reduced, the forwarding time is also shortened, and since a rough search of the entire range is not performed, high-speed control can be achieved.

[0127] The image acquisition unit 411 acquires the image of the forwarding range forwarded by issuing the forwarding instruction in step S226 (step S209; image acquisition step), and the search unit 413 performs a simple process on the image within the forwarding range (step S210). The simple process is an arbitrary process before performing a detailed search. For example, a rough search can be performed on the image within the forwarding range acquired in step S209 to determine the search range. In particular, a rough search of the θ coordinate can be performed as a simple process to determine the search angle range. In addition, the simple process can also be omitted.

[0128] Thereafter, a detailed search is performed for the images within the forwarding range obtained in step S209 or for the search range determined in step S210 (step S211: search step). The method of the detailed search is the same as that of step S205. If the result of the detailed search in step S211 is that the identification object cannot be detected (step S212: No), the process returns to step S201, and since the previous detection failed (step S201: Yes), the processing of steps S221, S222, and S203 to 205 is performed. Here, the reason for restarting the rough search of the images in the entire range is to avoid the situation where the detailed search fails repeatedly and becomes uncontrollable.

[0129] When the recognition target is successfully detected in the detailed search of step S211 (step S212 : Yes), the actual coordinates of the recognition target acquired in step S211 are output to the motion control unit 200 (step S213 ).

[0130] Return again Fig.13 According to the flowchart of , the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold (step S106: Yes), the alignment process ends.

[0131] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107). Specifically, a control signal for aligning the actual coordinates with the target coordinates is output to the drive control devices 121, 122, and 123. Then, the process returns to step S122 to continue.

[0132] In this way, the range determination unit 412 of the image processing unit 400 determines the image transfer range based on the position information, and the search unit 413 performs a detailed search on the image transferred from the imaging device 300. This can significantly shorten the transfer time and the search time.

[0133] In addition, in addition to the method of specifying the shooting parameters including the forwarding range together with the shooting instruction, the forwarding range from the image processing unit 400 to the shooting device 300 can also be specified by specifying the forwarding range to the shooting device 300 in advance and then outputting the shooting trigger to the shooting device 300.

[0134] In the case of specifying the forwarding range together with the shooting instruction of the former, the shooting of the shooting device 300 and the acquisition of the position information are made consistent by using the synchronization method of the transmission path delay measurement mentioned above, so that the waiting time can be reduced and the search within the appropriate forwarding range can be performed. On the other hand, in the case of the latter forwarding range being specified in advance, only the shooting trigger is output through the dedicated line, so that the shooting timing can be made consistent with high precision.

[0135] In addition, Fig.14 The image forwarding of the entire range specified in step S221 and the image forwarding of the forwarding range specified in step S226 may also be forwarded by the camera 300 by reducing the image. The reduction method may be any conventional method, such as using downsampling or binning. In particular, binning has the effect of being relatively simple to process and improving the S / N ratio (Signal to Noise Ratio) of the pixel signal, and can suppress the reduction in position recognition accuracy compared to the reduction in resolution caused by reduction, so it is preferred.

[0136] The following control may also be performed, that is, when the forwarding range is limited (trimmed) by the forwarding indication in step S226, the actual coordinates and the target coordinates are far apart in the early stage of the alignment control, and the forwarding range is reduced together with the limitation, and after the distance between the actual coordinates and the target coordinates is close to fall within a certain range, the reduction is not performed. In addition, when the image within the entire range or the forwarding range is reduced, a model matching the reduced data is prepared in advance as a pattern model for search, and the search unit 413 performs a rough search or a detailed search using the model.

[0137] In addition, the captured images of the entire range may be acquired in parallel with the detailed search (step S211) after the captured images of the forwarding range are acquired by the forwarding instruction in step S226. The time required to acquire the captured images of the entire range can be reduced when the identification object cannot be detected in the detailed search in step S211 (step S212: No). In this case, the forwarding of the captured images of the entire range may be stopped or the forwarded images may be deleted at the stage when the identification object is successfully detected by the detailed search.

[0138] As described above, in the alignment system 2 according to the present embodiment, the image processing unit 400 predetermines the correspondence between the position information of the driving instruments 111, 112, and 113 and the actual coordinates of the identification object detected from the captured image, calculates the predicted coordinates of the identification object based on the position information to be acquired next using the correspondence, determines the forwarding range of the image including the predicted coordinates, and performs a detailed search of the image within the forwarding range forwarded from the imaging device 300 to detect the identification object. Thus, the forwarding time and the volume of the forwarded image can be reduced, and the rough search of the image of the entire range can be omitted, so that the alignment control can be performed at high speed and with high precision.

[0139] (Variation Example)

[0140] The above-described embodiment can be modified in various ways. Modifications will be described below.

[0141] [Modification 1]

[0142] In the first and second embodiments, the range determination unit 412 of the image processing unit 400 determines the range including the predicted coordinates calculated based on the position information output by the position sensor of the driving device 111, 112, 113 as the search range or the forwarding range. However, the search range or the forwarding range may be determined by other methods. This modification 1 describes other methods for determining the search range or the forwarding range.

[0143] This modification uses command information related to driving the driving devices 111, 112, and 113 to determine the calculation of predicted coordinates used in the search range or the forwarding range. Specifically, the range determination unit 412 of the image processing unit calculates predicted coordinates based on the command information and determines the search range or the forwarding range including the predicted coordinates.

[0144] In this modification, by ensuring synchronization, the timing control of the imaging by the imaging device 300, the driving of the driving devices 111 , 112 , 113 based on the command information, and the acquisition of the position sensor output is performed, thereby achieving higher-speed control. Fig.15 is a block diagram showing an example of the functional structure of the alignment system 3 involved in this modification. Fig.16 is a flowchart of the alignment control process performed by the motion control unit 200 according to this modification example. Fig.17 is a flowchart of the search process executed by the image processing unit 400 .

[0145] Here, Figure 15-17 This is a variation of the second embodiment, which shows a method of determining the forwarding range of the image based on the instruction information. However, as a variation of the first embodiment, a method of determining the search range of the image based on the instruction information can also be implemented. Fig.16 , 17 The flowchart of exemplifies the operation of the alignment system 3 according to this variation, but descriptions of the same processing as that in the second embodiment are omitted.

[0146] First, the motion control unit 200 issues an instruction to move the workpiece 10 to a preset estimated target position ( Fig.16 , step S101). In order to determine the forwarding range, the instruction information at this time is sent to the image processing unit 400. Then, based on the control signal related to the instruction information, the driving devices 111, 112, and 113 start driving, and the mounting table 101 on which the workpiece 10 is mounted moves.

[0147] After the movement instruction, the motion control unit 200 determines the shooting time of the shooting device 300 to be the specific time T in the future. n (n=1) (step S122), the image processing unit 400 is designated. In addition, the position information generation unit 211 of the motion control unit 200 designates the time T based on the output of the position sensor included in the driving devices 111, 112, and 113. n Then, the action control unit 200 waits until the search of the image processing unit 400 ends (step S104). The subsequent actions of the image processing unit 400 are the same as those of the second embodiment, and the execution Fig.17Processing of steps S221, 222, 203-205, 212, and 213.

[0148] return Fig.16 When the search process of the image processing unit 400 is completed (step S104: Yes), the movement amount calculation unit 212 of the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and the position information generation unit 211 generates the position information based on the time T n The position sensor output generates and records the position information (step S124). The movement amount calculation unit 212 calculates the difference between the actual coordinates and the target coordinates, and ends the alignment control process when the difference between the actual coordinates and the target coordinates is less than or equal to the threshold (step S106: Yes).

[0149] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100, referring to the specific time T recorded in step S124. n The position information at the location is used to execute the instruction in which the offset between the position indicated by the instruction and the actual position indicated by the position information is also compensated (step S125).

[0150] After the instruction to the alignment mechanism 100 is executed by the instruction unit 213, the process returns to step S122 and the imaging time is determined to be time T. n (n=2) (step S122), specify a specific time T n The recording of the position information of (n=2) (step S123) is performed, and the search processing of the image processing unit 400 is waited until it is completed (step S104).

[0151] Transfer again to Fig.17 , the next shooting is the second time and thereafter and the previous detection is successful (step S201: No), so the image processing unit 400 instructs the shooting device 300 to n (n=2) (step S223). In addition, the image processing unit 400 obtains the time T from the motion control unit 200. n The range determination unit 412 of the image processing unit 400 calculates the predicted coordinates of the recognition object at that time (step S225).

[0152] The method of calculating the predicted coordinates executed by the range determination unit 412 is to first determine the correspondence between the command information during the initial alignment control and the actual coordinates detected in the initial detailed search instead of the position information in the second embodiment. The range determination unit 412 uses the determined correspondence to calculate the predicted coordinates based on the specific time T obtained in step S224. n The predicted coordinates (XY coordinates) of the reference point of the recognition object are calculated based on the instruction information related to the predetermined position (step S225). The prediction method using the correspondence relationship is the same as that in the second embodiment.

[0153] Then, the range determination unit 412 determines a forwarding range of a predetermined size with the XY coordinates predicted in step S225 as the center, and instructs the camera 300 to forward the image within the determined forwarding range. (Step S226). In addition, the forwarding range may not be a range centered on the predicted XY coordinates, and the range including the predicted XY coordinates may be used as the forwarding range depending on the alignment conditions or the form of the recognition object.

[0154] The image acquisition unit 411 acquires the image of the forwarding range forwarded by issuing the forwarding instruction in step S226 (step S209), and performs a simple process (step S210). Then, the search unit 413 performs a detailed search for the image in the forwarding range acquired in step S209 or the image that has been simply processed in step S210 (step S211). If the result of the detailed search in step S211 is that the recognition object cannot be detected (step S212: No), the process returns to step S201, and since the previous detection failed (step S201: Yes), the process of steps S221, S222, and S203-205 is performed.

[0155] If the recognition target is successfully detected in the detailed search of step S211 (step S212: Yes), the actual coordinates of the recognition target acquired in step S211 are output to the motion control unit 200 (step S213). The subsequent processing of the motion control unit 200 is the same as that of the second embodiment.

[0156] In this way, the range determination unit 412 of the image processing unit 400 determines the forwarding range of the image based on the instruction information of the action control unit 200, and the search unit 413 performs a detailed search on the image forwarded from the imaging device 300. Thus, the search range or the forwarding range is determined based on the instruction information related to the future predetermined position, so that more efficient timing control can be achieved, and higher-speed alignment control can be achieved.

[0157] [Modification 2]

[0158] This variation 2 describes another method for determining the search range or forwarding range. In embodiments 1 and 2, the range determination unit 412 determines the search range or forwarding range centered on the predicted coordinates calculated based on the location information, but in this variation, the search range or forwarding range is determined centered on the actual coordinates detected last time.

[0159] Specifically, the range determination unit 412 determines the range centered on the reference point of the recognition object stored in the nonvolatile memory 230 as the forwarding range, without obtaining the position information or the instruction information. The recognition object was previously detected by the detailed search performed by the search unit 413 of the image processing unit 400. Alternatively, the range determination unit 412 determines the range centered on the reference point of the recognition object stored or the angle range centered on the rotation angle of the recognition object as the search range. In addition, the forwarding range or the search range may not be the range centered on the actual coordinates detected previously, and the range including the actual coordinates detected previously may be used as the forwarding range or the search range according to the alignment conditions or the form of the recognition object.

[0160] The size of the forwarding range or the search range may also be determined based on the moving speed of the actual coordinates of the identified object detected in the past. Alternatively, it may also be determined based on the setting parameters of the alignment control and the moving speed of each driving instrument. For example, it may be set to a size determined by multiplying the shooting interval of the camera 300 by the maximum moving speed. In the case where the moving distance between the shooting intervals is short due to the short shooting interval setting, the search range or the forwarding range can be fully limited by utilizing the previous detection result, thereby simplifying the processing compared with the first and second embodiments and the modified example 1.

[0161] [Variation 3]

[0162] This modification 3 describes another method for determining the search range or the forwarding range. In this modification, the range determination unit 412 determines the search range or the forwarding range based on the trajectory of the actual coordinates detected in the past by the detailed search performed by the search unit 413. That is, instead of acquiring position information or instruction information as in the first and second embodiments, the predicted coordinates are calculated based on the trajectory of the actual coordinates of the recognition object detected in the past by the detailed search performed by the search unit 413 of the image processing unit 400 and stored in the non-volatile memory 230, and the range centered on the predicted coordinates is determined as the search range or the forwarding range.

[0163] Specifically, the range determination unit 412 determines a range with a predetermined size centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identification object as the forwarding range. Alternatively, the range determination unit 412 determines a range with a predetermined size centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identification object, or an angular range centered on the θ coordinate predicted based on the trajectory of the θ coordinate as the search range. Fig.18A , B is a diagram for explaining the prediction of the recognition object. In addition, the forwarding range or search range may not be a range centered on the prediction coordinates, and the range including the prediction coordinates may be used as the forwarding range or search range according to the alignment conditions or the form of the recognition object.

[0164] The calculation of the predicted coordinates (XYθ coordinates) is performed by prediction at the time of shooting by the imaging device 300 using the time at least two times before the previous time and the actual coordinates at each time. Fig.18A As shown in FIG. 1 , linear prediction is performed based on the XY coordinate data of the previous and the previous two times to predict the XY coordinates at the time of shooting. Fig.18B As shown, based on the XY coordinate data of three times, namely, the previous time, the second previous time, and the second previous time, a second order prediction is performed to predict the XY coordinates at the time of shooting.

[0165] The size of the search range or forwarding range can be determined based on the moving speed of the actual coordinates of the identification object detected in the past. Alternatively, it can also be determined based on the moving speed of the driving instruments 111, 112, and 113. For example, it can be set to a size determined by multiplying the shooting interval of the shooting device 300 by the maximum moving speed. Linear prediction has a lower prediction accuracy than second-order prediction or higher-order prediction, so it is necessary to increase the size of the search range or forwarding range.

[0166] In addition, the size of the search range or forwarding range can be determined based on statistical information on errors in past detection results. For example, it can also be the size after adding a margin to the shape and size of the recognition object, where the margin is the average of the differences between the predicted coordinates of the trajectory based on the actual coordinates at the corresponding time points in the past and the actual coordinates detected from the captured image.

[0167] According to this modification, when the imaging interval is long and the moving distance between imaging intervals is long, the prediction accuracy can be improved and the search range or the forwarding range can be narrowed.

[0168] [Variation 4]

[0169] In the first and second embodiments, the case where the recognition target is the alignment marks 1001 and 1002 added to the workpiece 10 is described. However, a part of the processing may be changed according to the form of the recognition target. Fig.19A is a diagram for explaining the rough search involved in this modification example. Fig.19B This is a diagram for explaining the detailed search according to this modification.

[0170] For example, when the workpiece 12 is Fig.19A In the case where the IC (Integrated Circuit) has a plurality of terminals and the identification target is a terminal of the IC, the search unit 413 performs a rough search by performing pattern matching processing using the pattern model 1312 ( Figure 8 , step S203).

[0171] At this time, the search unit 413 determines the search range 1314 ( Figure 8 , step S204). In the pattern matching, Fig.19A As shown, a plurality of patterns having a coincidence rate greater than or equal to a predetermined threshold value may be detected. In this case, the pattern 1313 having the highest coincidence rate may be identified to determine the search range 1314.

[0172] exist Fig.19B In the example shown, the search unit 413 determines the search range 1314 as the search range for the detailed search ( Figure 8 , step S204), the search range 1314 has a predetermined positional relationship with respect to the pattern 1313 determined by the rough search. The search unit 413 performs a detailed search ( Figure 8 , step S205).

[0173] Here, in the first and second embodiments, when it is the second or subsequent shooting and the previous detection is successful ( Figure 8 , step S201: No), determine the search range that includes the predicted coordinates calculated based on the location information ( Figure 8 , step S208), perform a detailed search ( Figure 8 , step S211). However, in the case where Fig.19A In the case where the predicted coordinates are calculated with a certain accuracy as shown in the IC, the predicted coordinates based on the position information may not be calculated, but the detailed search may be repeated using the search range determined in the rough search in step S203. In addition, the search range determined by the rough search may be processed only when the detection by the detailed search fails.

[0174] According to this modification, the processing can be simplified according to the form of the recognition object. In addition, the form of the recognition object can be classified by the application of the setting terminal 500, and the search processing method suitable for the classification can be automatically selected.

[0175] [Variation 5]

[0176] This modification example 5 describes a case where the identification target has a further different form. Fig. 20A is a diagram for explaining the rough search involved in this modification example. Fig. 20B 2 is a diagram for explaining a detailed search according to this modification. In this modification, a dedicated application is executed by the processor of the setting terminal 500, thereby automatically performing registration of a model for a rough search and generation of logic for a detailed search.

[0177] For example, in the case where workpiece 13 is Fig. 20A In the case of the lens shown in FIG. 1 , the processor of the setting terminal 500 extracts the shape of the lens from the captured image and registers the pattern model 1322. Further, the processor automatically generates a search logic for a detailed search based on the shape of the pattern model 1322. For example, in the case where the processor of the setting terminal 500 detects that the shape of the pattern model 1322 includes an arc, as shown in FIG. Fig. 20B As shown, the following logic is automatically generated as the detailed search logic, namely: searching for edges, performing circle approximation on the detected edges, and performing detection using the center of the circle as the reference point 1323.

[0178] The image processing unit 400 uses the rough search pattern model and detailed search logic generated by the application of the setting terminal 500 to perform the search process described in the first and second embodiments. According to this modification, the most appropriate rough search and detailed search can be performed according to the form of the recognition object. In addition, the pre-setting for executing the search process of the image processing unit 400 can be automated, and the burden of the user inputting the setting to the setting terminal 500 can be reduced.

[0179] [Variation 6]

[0180] In the first and second embodiments, a rough search is performed on the first captured image, a detailed search is performed based on the result of the rough search within the search range determined, and the predicted coordinates of the identified object are calculated based on the result of the detailed search. The identified object is used to determine the search range or forwarding range of the next detailed search. However, if the prediction accuracy of the coordinates of the identified object is high enough, the rough search on the first captured image may be omitted by prior calibration of the alignment control. In particular, in the case of alignment control performed using the corner end of the mounting table 101 on which the workpiece 10 is mounted, the error each time is small, so such prior calibration is effective.

[0181] Fig.21 This is a flowchart of the search process when the rough search for the first captured image is omitted. After the image acquisition unit 411 acquires the first captured image (step S301), the range determination unit 412 uses the previous calibration information to calculate the predicted coordinates of the recognition object based on the position information (step S302). The range determination unit 412 determines the search range including the calculated predicted coordinates (step S303), and performs a detailed search for the determined search range (step S304).

[0182] When the actual coordinates of the object to be identified are successfully detected in the detailed search of step S304 (step S305: Yes), the actual coordinates are output to the motion control unit 200 (step S309), and the motion control unit 200 uses the actual coordinates to control the driving devices 111, 112, and 113. After the workpiece 10 is moved, a detailed search is performed for the next captured image in the same search range as the previous search. In this way, when the object is successfully detected by the detailed search, the detailed search is repeatedly performed without performing a rough search.

[0183] When the detailed search is repeated, only when the detection of the identification object fails (step S305: No), the search range is determined (step S307) by performing a rough search (step S306), and a detailed search is performed in the determined search range (step S308), and the actual coordinates are output (step S309). In this way, according to this modification, the detailed search is repeated based on the prior calibration information, and the rough search is performed only when the identification object cannot be detected. As a result, the processing can be simplified and high-speed alignment control can be achieved.

[0184] In addition, the hardware structure and flowchart shown in the above-mentioned embodiment and modified example are examples, and can be changed and modified arbitrarily. For example, in the above-mentioned embodiment and modified example, the action control unit 200, the shooting device 300, the image processing unit 400 and the setting terminal 500 are independent structures, but it can also be a structure in which at least two of them are integrated.

[0185] In addition, the search processes shown in the above-mentioned embodiments and modified examples may be arbitrarily combined and executed. In addition, it may be set that any one of the search processes involved in the above-mentioned embodiments and modified examples can be selected by setting the terminal 500. Alternatively, the image processing unit 400 may automatically select any one of the search processes involved in the above-mentioned embodiments and modified examples according to conditions such as the type of workpiece and the form of the recognition object.

[0186] In addition, the sharing of the functions implemented by the processors in the motion control unit 200 and the image processing unit 400 in the above-mentioned embodiment and the modified example is an example, and the sharing can be arbitrarily changed. In addition, the functions implemented by the processors in the motion control unit 200 or the image processing unit 400 can be implemented using a general computer system without relying on a dedicated system.

[0187] The program for executing the actions of the above-mentioned embodiments may be distributed by storing it in a computer-readable recording medium such as a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), an MO (Magneto Optical Disc), or a memory card, and installing the program in a computer, thereby forming a computer capable of realizing each function. Furthermore, in the case where each function is realized by sharing the OS (operating system) and an application program, or by the collaboration between the OS and the application program, only the part other than the OS may be stored in the recording medium.

[0188] The present invention can be set as various embodiments and variations without departing from the broad spirit and scope of the present invention. In addition, the above-mentioned embodiments are used to illustrate the present invention, rather than to limit the scope of the present invention. That is, the scope of the present invention is not represented by the embodiments, but by the claims. Moreover, various variations implemented within the scope of the claims and within the scope of the meaning of the invention equivalent thereto are deemed to fall within the scope of the present invention.

[0189] Description of the label

[0190] 1, 2, 3 alignment system, 10, 12, 13 workpiece, 11 alignment target, 40 main site, 50, 60, 70 equipment site, 100 alignment mechanism, 101 mounting table, 111, 112, 113 driving instrument, 121, 122, 123 driving control device, 200 motion control unit, 210 processor, 211 position information generating unit, 212 movement amount calculating unit, 213 instruction unit, 220 volatile memory, 230 non-volatile memory, 231 control program, 232 control data, 240 clock, 250 communication interface, 300 imaging device, 4 00 image processing unit, 410 processor, 411 image acquisition unit, 412 range determination unit, 413 search unit, 420 volatile memory, 430 non-volatile memory, 431 control program, 432 control data, 440 clock, 450 communication interface, 500 setting terminal, 1001, 1002 alignment mark, 1101, 1102 alignment target mark, 1200 θ-axis rotation center, 1201 virtual mark, 1300 shooting range, 1301, 1314 search range, 1312, 1322 pattern model, 1313 pattern, 1323 reference point.

Claims

1. An alignment system comprising: an image processing unit that detects the identification object by using different search methods, namely, a rough search and a detailed search, from a captured image obtained by capturing a workpiece having the identification object; and an action control unit that controls a driving device that moves the workpiece based on the actual coordinates of the recognition object detected by the image processing unit, repeatedly performing the detection of the recognition object by the image processing unit and the control of the driving device by the action control unit, The detailed search is performed within a search range narrower than the range of the rough search, and when the identification target is successfully detected by the detailed search, the next search is performed without performing the rough search but performing the detailed search.

2. The alignment system according to claim 1, wherein: When the recognition object is successfully detected through the detailed search, the image processing unit performs the next detailed search on a search range determined based on actual coordinates of the recognition object obtained through the detailed search.

3. The alignment system according to claim 1, wherein: When the recognition target is successfully detected by the detailed search within the search range determined by the rough search, the image processing unit performs the next detailed search on the search range found by the detailed search.

4. An alignment system according to any one of claims 1 to 3, wherein: When the image processing unit cannot detect the actual coordinates of the recognition object through the detailed search, the next search is to perform the detailed search within the search range determined by re-performing the rough search.

5. An alignment system according to any one of claims 1 to 4, wherein: The image processing unit performs the detailed search in a search range determined by a previous calibration, and when the identification target cannot be detected by the detailed search, performs the detailed search in a search range determined by performing the rough search.

6. An alignment system according to any one of claims 1 to 5, wherein: The rough search includes a pattern matching process for matching a pattern of the workpiece with a pre-registered pattern model, and the rough search determines a range of the detailed search based on a position or a rotation angle of the pattern when a coincidence rate is greater than or equal to a predetermined threshold.

7. An alignment system according to any one of claims 1 to 5, wherein: The rough search includes a pattern matching process for matching the pattern of the workpiece with a pre-registered pattern model. When the rough search has multiple patterns when the consistency rate is greater than or equal to a predetermined threshold, the scope of the detailed search is determined based on the position or rotation angle of the pattern with the highest consistency rate.

8. An alignment system according to claim 6 or 7, wherein: The alignment system further includes a setting terminal for registering the pattern model used in the pattern matching process of the rough search. When the detailed search is a search for an edge of a pattern included in the workpiece, the setting terminal generates a search logic for the detailed search based on a shape of the pattern model when registering the pattern model.

9. An alignment method comprising the following steps: A searching step of detecting the identification object by a rough search or a detailed search from a captured image obtained by capturing a workpiece having the identification object; and an instruction step of issuing an instruction to a driving device for moving the workpiece based on the actual coordinates of the identification object detected in the searching step, Repeating the searching step and the commanding step, In the searching step, when the identification target is successfully detected by the detailed search, the next search is performed without performing the rough search but performing the detailed search.

10. A program for causing a computer to function as a search unit in an alignment system, wherein the alignment system detects an identification object from a captured image obtained by capturing a workpiece having the identification object, performs alignment control of the workpiece based on the detected actual coordinates of the identification object, and the computer performs image processing of the captured image in the alignment system, The search unit detects the recognition object from the captured image by a rough search or a detailed search, and when the recognition object is successfully detected by the detailed search, the next search is performed without performing the rough search but performing the detailed search.

Images