Alignment system, alignment method, and program

Abstract

An alignment system (1) is provided with: an imaging device (300) that captures an image of a workpiece (10) having an object to be recognized; an image processing unit (400) that detects an object to be recognized on the basis of a captured image of the workpiece (10) acquired from the imaging device (300); and an operation control unit (200) that controls drive devices (111, 112, 113) that move the workpiece (10) on the basis of the actual coordinates of the recognition object detected by the image processing unit (400). The image processing unit (400) determines the transfer range of the image on the basis of the actual coordinates of the object to be recognized, and detects the object to be recognized on the basis of a captured image within the transfer range acquired from an imaging device by a transfer instruction specifying the transfer range.

Description

Technical Field

[0001] The present invention relates to an alignment system, an alignment method and a program. Background Art

[0002] In the field of FA (Factory Automation), alignment technology for aligning the position of a workpiece as a control object with a reference position is effectively used. For example, there is a control system in which the workpiece has an alignment mark, the mark is detected based on an image of the workpiece captured by a camera, and the position of the mark is aligned with the reference position to perform alignment (for example, Patent Document 1).

[0003] In the control system described in Patent Document 1, a workpiece is placed on a workbench having a moving mechanism, and the position of a characteristic portion on the workpiece is determined based on a captured image. The control system calculates the speed based on information obtained from the moving mechanism and predicts the movement amount of the workpiece, and limits the search range of the captured image by inferring the position of the characteristic portion. It is described that this can reduce the search time of the characteristic portion and can position the workpiece at a high speed.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2019-215633 Summary of the invention

[0005] The control system described in Patent Document 1 obtains the captured images of the entire range forwarded from the camera device, and searches for images within the search range limited by the captured images of the entire range. However, the captured images of the entire range have the following problems: the data capacity is large, the forwarding takes time, and the overall time for alignment control increases. In addition, the data capacity of the captured images of the entire range is large and cannot be stored in the capacity of the high-speed memory, i.e., the cache memory, of the image processing device. Therefore, there is also the following problem: a lot of time is required in the process of limiting the search range from the captured images of the entire range.

[0006] The present invention has been made in view of the above-mentioned situation, and an object of the present invention is to provide an alignment system, an alignment method, and a program capable of performing positioning at high speed and high precision.

[0007] In order to achieve the above-mentioned object, the alignment system of the present invention is characterized in that it has: a photographing device that photographs a workpiece having an identification object; an image processing unit that detects the identification object based on the photographed image of the workpiece obtained from the photographing device; and an action control unit that controls a driving device that moves the workpiece based on the actual coordinates of the identification object detected by the image processing unit. The image processing unit determines the forwarding range of the image based on the actual coordinates of the identification object, and detects the identification object based on the photographed image within the forwarding range obtained from the photographing device through a forwarding instruction that specifies the forwarding range.

[0008] Effects of the Invention

[0009] According to the present invention, by forwarding an image within a forwarding range determined based on actual coordinates of an identification object detected in the past to an image processing unit, the forwarding time can be shortened, thereby enabling high-speed and high-precision positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a block diagram showing an overall configuration example of the alignment system involved in embodiment 1.

[0011] Figure 2 This is a block diagram showing a configuration example of an action control unit.

[0012] Figure 3 This is a block diagram showing a configuration example of an image processing unit.

[0013] Figure 4 This is a diagram for explaining the alignment control method.

[0014] Figure 5 This is a diagram for explaining the alignment control method.

[0015] Figure 6 This is a block diagram showing an example of the functional structure of the alignment system involved in embodiment 1.

[0016] Figure 7 This is a flowchart showing the alignment control process involved in the first embodiment.

[0017] Figure 8 This is a flowchart showing the search process involved in Implementation Example 1.

[0018] Fig. 9A is a diagram showing an example of a search range.

[0019] Fig. 9B It is a diagram showing an example of the search angle range.

[0020] Fig.10 This figure shows a synchronization method achieved by measuring transmission path delay.

[0021] Fig.11 This is a diagram for explaining the timing control of the shutter of the imaging device.

[0022] Fig.12 This is a block diagram showing an example of the functional structure of the alignment system involved in Implementation Example 2.

[0023] Fig.13 This is a flowchart showing the alignment control processing involved in the second embodiment.

[0024] Fig.14 This is a flowchart showing the search process involved in Implementation Example 2.

[0025] Fig.15 This is a block diagram showing an example of the functional structure of the alignment system involved in the first modification.

[0026] Fig.16 This is a flowchart showing the alignment control process according to Modification 1.

[0027] Fig.17 This is a flowchart showing the search process according to Modification 1.

[0028] Fig.18A This is a diagram for explaining the prediction of the recognition target according to the third modification.

[0029] Fig.18B This is a diagram for explaining the prediction of the recognition target according to the third modification.

[0030] Fig.19A This is a diagram for explaining the rough search according to Modification 4.

[0031] Fig.19B This is a diagram for explaining the detailed search related to Modification Example 4.

[0032] Fig. 20A This is a diagram for explaining the rough search according to the fifth modification.

[0033] Fig. 20B This is a diagram for explaining the detailed search involved in Modification Example 5.

[0034] Fig.21 This is a flowchart showing the search process involved in Modification Example 6. DETAILED DESCRIPTION

[0035] (Implementation Method 1)

[0036] Hereinafter, Embodiment 1 for implementing the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0037] Figure 1 1 is a block diagram showing an overall configuration example of an alignment system 1 according to Embodiment 1. The alignment system 1 includes an alignment mechanism 100 that moves a workpiece 10 as a control object; an action control unit 200 that controls the alignment mechanism 100; an imaging device 300 that images the workpiece 10; an image processing unit 400 that processes the image captured by the imaging device 300; and a setting terminal 500 that performs various settings used for processing in the image processing unit 400.

[0038] The alignment mechanism 100 is connected to drive devices 111, 112, and 113 that apply driving force in various directions to move the workpiece 10. In addition, the drive devices 111, 112, and 113 are respectively connected to drive control devices 121, 122, and 123, and the drive control devices 121, 122, and 123 drive the drive devices 111, 112, and 113 based on control signals from the motion control unit 200.

[0039] The drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the camera 300 are connected to each other for communication. The communication unit may be any existing communication unit such as Ethernet (registered trademark), CameraLink, CoaxPress (registered trademark), USB (Universal Serial Bus, registered trademark), etc., but in order to ensure synchronization, an industrial network based on Ethernet such as CC-Link IE / field (registered trademark) and CC-Link IE / TSN (registered trademark) is preferred.

[0040] The alignment mechanism 100 has a stage 101 for placing the workpiece 10, and a mechanism for moving the stage 101. For example, the stage can be translated in the X direction and the Y direction that are orthogonal to each other in the horizontal direction, and can also be rotationally moved in the θ direction, which is a rotational direction on the horizontal plane. In this embodiment, the drive device 111 coupled to the alignment mechanism 100 is translated in the X direction, the drive device 112 is translated in the Y direction, and the drive device 113 is rotationally moved in the θ direction.

[0041] The drive devices 111, 112, 113 are any drive devices (actuators) capable of precisely driving the alignment mechanism 100, such as servo motors. The drive control devices 121, 122, 123 are control devices that control the drive of the drive devices 111, 112, 113 based on control signals from the motion control unit 200, such as servo amplifiers.

[0042] The driving devices 111, 112, and 113 have any position sensor inside or outside, and detect and output the position actually changed by the driving of the driving devices 111, 112, and 113. The position sensor is, for example, an encoder attached to the actuator. The output signal of the encoder is input to the motion control unit 200 via the driving control devices 121, 122, and 123. In this embodiment, the driving devices 111, 112, and 113 are servo motors, the driving control devices 121, 122, and 123 are servo amplifiers, and the position sensor is an encoder.

[0043] The motion control unit 200 is a motion controller that executes instructions related to the actions of the drive devices 111, 112, 113 for the drive control devices 121, 122, 123, and includes, for example, a PLC (Programmable Logic Controller). The motion control unit 200 generates a control signal representing the instruction based on information obtained from the drive control devices 121, 122, 123 and the image processing unit 400, and outputs it to the drive control devices 121, 122, 123.

[0044] The motion control unit 200 is as follows: Figure 2 As shown, there are a processor 210, a volatile memory 220, a nonvolatile memory 230, a clock 240, and a communication interface 250. The processor 210, the volatile memory 220, the nonvolatile memory 230, the clock 240, and the communication interface 250 are communicatively connected to each other via a bus B1.

[0045] The processor 210 is, for example, a CPU (Central Processing Unit), and functions as a position information generating unit 211 , a movement amount calculating unit 212 , and an instruction unit 213 by reading and executing a control program 231 stored in a nonvolatile memory 230 .

[0046] The position information generating unit 211 of the processor 210 generates position information corresponding to the position of the mounting table 101 on which the workpiece 10 is mounted, based on the output signals of the position sensors of the driving devices 111, 112, and 113. The movement amount calculating unit 212 calculates the movement amounts of the workpiece 10 in the X direction, the Y direction, and the θ direction, based on the position of the recognition object detected by the image processing unit 400 from the captured image. The instruction unit 213 outputs a control signal based on the movement amount calculated by the movement amount calculating unit 212 to the drive control devices 121, 122, and 123.

[0047] The volatile memory 220 is a working memory capable of reading and writing data at high speed during the operation processing executed by the processor 210, such as a RAM (Random Access Memory). The non-volatile memory 230 stores a control program 231 for realizing various functions of the motion control unit 200, and control data 232 including parameters used when executing the control program 231, past detection data, and instruction data. The non-volatile memory 230 is, for example, a non-volatile semiconductor memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory, a magnetic disk, or an optical disk.

[0048] The clock 240 measures the local time by counting the clock signal of the clock element of the motion control unit 200, and synchronizes with the drive control devices 121, 122, 123, the imaging device 300, and the image processing unit 400. Thus, the motion control unit 200 has the time information synchronized with the drive control devices 121, 122, 123, the imaging device 300, and the image processing unit 400.

[0049] The communication interface 250 is an interface used by the motion control unit 200 to communicate with the drive control devices 121 , 122 , 123 , the imaging device 300 , and the image processing unit 400 , and is an interface compliant with communication standards such as CC-Link IE / field and CC-Link IE / TSN.

[0050] The imaging device 300 is an imaging device for imaging the workpiece 10 at regular time intervals from above the alignment mechanism 100, and is, for example, a camera having a resolution sufficient to achieve the alignment accuracy of the alignment system 1. The number of imaging devices 300 is arbitrary, but is determined according to the number and positions of the recognition objects used for alignment. Figure 1 The case where two imaging devices 300 are provided is shown. The recognition target may be any target indicating the position of the workpiece 10 , such as an alignment mark attached to the workpiece 10 , a corner end of the workpiece 10 , or a corner end of the stage 101 .

[0051] The photographing device 300 also has timing information synchronized with the drive control devices 121 , 122 , 123 , the motion control unit 200 , and the image processing unit 400 , and a communication interface for communicating with the drive control devices 121 , 122 , 123 , the motion control unit 200 , and the image processing unit 400 .

[0052] The image processing unit 400 performs a process of detecting the recognition object by coarse exploration and detailed exploration of different exploration methods for the captured image obtained from the image capturing device 300. The coarse exploration is performed in the entire range of the captured image obtained from the image capturing device 300, and the detailed exploration is performed in an exploration range narrower than the coarse exploration range. When the image processing unit 400 detects the recognition object by the detailed exploration, it outputs the actual coordinates of the recognition object to the motion control unit 200. Here, the actual coordinates are the coordinates at the reference coordinates matched with the movement amount calculation unit 212 of the motion control unit 200.

[0053] The image processing unit 400 is as follows Figure 3 As shown, there are a processor 410, a volatile memory 420, a nonvolatile memory 430, a clock 440, and a communication interface 450. The processor 410, the volatile memory 420, the nonvolatile memory 430, the clock 440, and the communication interface 450 are communicatively connected to each other via a bus B2.

[0054] The processor 410 is, for example, a CPU (Central Processing Unit), and reads and executes a control program 431 stored in a nonvolatile memory 430 , thereby functioning as an image acquisition unit 411 , a range determination unit 412 , and a search unit 413 .

[0055] The image acquisition unit 411 of the processor 410 acquires an image captured by the image capturing device 300. The range determination unit 412 determines a range for searching for a recognition object in the captured image acquired by the image acquisition unit 411 based on information including the position information generated by the position information generation unit 211 of the motion control unit 200. The exploration unit 413 performs a detailed search for the recognition object with respect to the image within the search range determined by the range determination unit 412, and outputs the actual coordinates of the recognition object to the motion control unit 200 when the recognition object can be detected.

[0056] The volatile memory 420 is a working memory capable of reading and writing data at high speed during the operation processing executed by the processor 410, such as a RAM (Random Access Memory). The non-volatile memory 430 stores a control program 431 for realizing various functions of the image processing unit 400, and control data 432 including parameters used when executing the control program 431 and past detection data. The non-volatile memory 430 is, for example, a non-volatile semiconductor memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory, a magnetic disk, or an optical disk.

[0057] The clock 440 measures the local time by counting the clock signal of the clock element of the image processing unit 400, and synchronizes with the drive control devices 121, 122, 123, the motion control unit 200, and the imaging device 300. Thus, the image processing unit 400 has the time information synchronized with the drive control devices 121, 122, 123, the motion control unit 200, and the imaging device 300.

[0058] The communication interface 450 is an interface used by the image processing unit 400 to communicate with the drive control devices 121 , 122 , 123 , the motion control unit 200 , and the imaging device 300 , and is an interface compliant with communication standards such as CC-Link IE / field and CC-Link IE / TSN.

[0059] The setting terminal 500 is a terminal installed with an application corresponding to the control program 431 of the image processing unit 400, such as a personal computer. The setting terminal 500 has a management function of the image processing unit 400 including inputting or changing parameters stored in the non-volatile memory 430 of the image processing unit 400. The communication interface used by the setting terminal 500 to communicate with the image processing unit 400 is an arbitrary interface corresponding to the interface of the image processing unit 400, such as a USB interface or an RS232C interface.

[0060] The motion control unit 200 calculates a movement amount based on a difference between the actual coordinates of the recognition object detected by the image processing unit 400 and the target coordinates of the aiming target, and outputs a control signal based on the movement amount to the drive control devices 121 , 122 , 123 .

[0061] The operation of the alignment system 1 having the above-described structure will be described. First, regarding an example of an alignment method based on the position of an object to be recognized, the following example uses Figure 4 , 5 Provide explanation. Figure 4 , 5 This is a diagram for explaining an alignment method when the recognition target is alignment marks 1001 and 1002 attached to the end portions of the workpiece 10 .

[0062] The shapes of the alignment marks 1001 and 1002 are arbitrary, but preferably the shapes are such that the positions and rotation angles of the reference points can be clearly identified. For example, Figure 4 , 5 In addition, the number of alignment marks is arbitrary, such as Figure 4 As shown, it can be 1, such as Figure 5 As shown, two alignment marks 1001 and 1002 may be provided at the diagonal ends of the workpiece 10 .

[0063] When one alignment mark 1001 is used, an image of the imaging range 1300 is acquired by one imaging device 300. When two alignment marks 1001 and 1002 are used, and when the two alignment marks 1001 and 1002 cannot be imaged by one imaging range 1300, images of the imaging ranges 1300 that are offset from each other are acquired by two imaging devices 300.

[0064] In such Figure 4 When one alignment mark 1001 is used as shown, three information are used: coordinates of a reference point of the alignment target mark 1101 , coordinates of a reference point of the alignment mark 1001 on the workpiece 10 , and an angle difference between the alignment target mark 1101 and the alignment mark 1001 .

[0065] On the other hand, in Figure 5 In the case of using two alignment marks 1001 and 1002 as shown, three information are used: the midpoint coordinates of the reference points of the alignment target marks 1101 and 1102, the midpoint coordinates of the reference points of the alignment marks 1001 and 1002 on the workpiece 10, and the angle difference between the straight line connecting the alignment target marks 1101 and 1102 and the straight line connecting the alignment marks 1001 and 1002.

[0066] exist Figure 4 , 5 In any case, the angle difference Δθ becomes the movement amount of the θ axis. Figure 4 As shown, the virtual coordinates of the reference point of the virtual mark 1201 after the workpiece 10 is rotated by an angle Δθ around the θ-axis rotation center 1200 of the alignment mechanism 100 are obtained. Then, based on the difference between the virtual coordinates and the target coordinates of the alignment target mark 1101, the movement amount ΔX in the X-axis direction and the movement amount ΔY in the Y-axis direction are calculated. The command unit 213 of the motion control unit 200 sends the control signal for realizing the movement of Δθ, ΔX, and ΔY obtained as described above to the drive control devices 121, 122, and 123, thereby executing the command.

[0067] like Figure 4 As shown in FIG. 1 , the method using one alignment mark 1001, such as a cross mark, can be used in a case where the rotation of the alignment mark 1001 can be clearly determined. Figure 5 As shown, when two alignment marks 1001 and 1002 are used, it is not necessary to detect the rotation of the alignment mark itself, so that even if the alignment mark is circular, the rotation of the workpiece can be detected. In addition, the angle difference Δθ in the θ direction is calculated by detecting two points that are sufficiently separated, so that high-precision control can be performed.

[0068] As described above, the motion control unit 200 calculates Δθ, ΔX, and ΔY based on the coordinates and angle differences of the reference points of the alignment marks 1001 and 1002 as the recognition targets, and the drive control devices 121, 122, and 123 use these values ​​to control the drive devices 111, 112, and 113 to move the workpiece 10. However, usually, there are many cases where the difference from the target coordinates is not within the allowable range after one execution, so the detection of the recognition target and the control of the drive devices 111, 112, and 113 are repeatedly executed.

[0069] In the above-mentioned repeated processing, the processing load of searching for the recognition target based on the captured image obtained by the imaging device 300 is very large. Therefore, the alignment system 1 according to the present embodiment limits the range in which the search is performed to reduce the processing load.

[0070] Next, for detailed processing, use Figure 6 -8 for explanation. Figure 6 is a block diagram showing an example of a functional structure of an alignment system 1 according to the present embodiment. Figure 7 is a flowchart of the alignment control process executed by the motion control unit 200, Figure 8 is a flowchart of the exploration process executed by the image processing unit 400 .

[0071] First, the motion control unit 200 executes a command to move the workpiece 10 to a preset estimated target position ( Figure 7 , step S101). Specifically, the motion control unit 200 outputs control signals for realizing movement in each direction to the drive control devices 121, 122, 123. Furthermore, the drive devices 111, 112, 113 are driven by the control of the drive control devices 121, 122, 123 based on the control signals, and the mounting table 101 on which the workpiece 10 is mounted moves.

[0072] After the movement is completed, the motion control unit 200 instructs the image processing unit 400 to obtain and search for captured images (step S102), and generates and outputs position information based on the output of the position sensors of the drive devices 111, 112, and 113 (step S103). Then, the motion control unit 200 waits until the search of the image processing unit 400 is completed (step S104).

[0073] The image processing unit 400 instructed by the motion control unit 200 to perform the search process of the recognition object in step S102 executes Figure 8Here, since the image is captured for the first time (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 acquires the captured image captured by the image capturing device 300 (step S202). Then, the search unit 413 performs a rough search for the recognition target (rough search) (step S203).

[0074] The rough search is performed, for example, by pattern matching processing using a pattern model of the pre-registered alignment mark 1001. The search unit 413 detects the alignment mark 1001 when the consistency rate of pattern matching is greater than or equal to a predetermined threshold, and determines the search range for the detailed search based on the position or rotation angle of the reference point of the alignment mark 1001 (step S204). The shape or size of the search range at this time can be pre-set according to the shape or size of the alignment mark 1001, and can be set by the user input to the setting terminal 500.

[0075] For example, Fig. 9A In the case of the alignment mark 1001 of the cross shown in FIG. 1 , the position of the center of the reference point and the rotation angle of the cross are determined by pattern matching, and a search range 1301 having a size with a predetermined margin added to the size of the alignment mark 1001 is determined. Fig. 9B The exploration range covered by the angles shown (exploration angle range).

[0076] Next, the exploration unit 413 performs a detailed exploration (fine search) on the image within the exploration range determined in step S204 (step S205). As a detailed exploration, for example, edge detection is performed to detect accurate straight lines or curves, thereby obtaining more accurate actual coordinates of the recognition object. Here, the actual coordinates of the recognition object include the position (XY coordinates) and the rotation angle (θ coordinate) of the reference point of the recognition object.

[0077] If the search unit 413 cannot detect the recognition object through the detailed search of step S205 (step S212: No), it returns to step S201 and repeats the processing of steps S202 to 205. On the other hand, if the recognition object is successfully detected through the detailed search of step S205 (step S212: Yes), the actual coordinates of the recognition object obtained in step S205 are output to the action control unit 200 (step S213).

[0078] Here, return to Figure 7, after the exploration process performed by the image processing unit 400 is completed (step S104: Yes), the action control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (step S106: Yes), the alignment control process is terminated. Here, the target coordinates are the coordinates of the alignment target, including the center position (XY coordinates) and the rotation angle (θ coordinate).

[0079] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106 : No), the command unit 213 commands the alignment mechanism 100 to be corrected (step S107 ; command step). Specifically, a control signal for matching the actual coordinates with the target coordinates is output to the drive control devices 121 , 122 , 123 .

[0080] After the driving of the driving devices 111, 112, and 113 is completed, the process returns to step S102. The action control unit 200 again instructs photographing and exploring (step S102), and outputs the position information (step S103).

[0081] Again, convert to Figure 8 , the shooting is the second or subsequent shooting, and since the previous detection was successful (step S201: No), the image processing unit 400 obtains the position information (step S206). The range determination unit 412 of the image processing unit 400 calculates the predicted coordinates of the recognition object at that time (step S207).

[0082] Specifically, the range determination unit 412 determines the correspondence between the position information obtained from the motion control unit 200 during the initial alignment control and the actual coordinates detected by the initial detailed search. 1 ,y 1 ) indicates that the XY coordinates of the actual coordinates are expressed by (X 1 , Y 1 ), their relationship is represented by the constant matrix A of the following formula (1).

[0083] [Formula 1]

[0084] A=(X 1 , Y 1 ) / (x 1 ,y 1 )…(1)

[0085] If the XY coordinates of the position information obtained from the motion control unit 200 during the second alignment control are set to (x 2 ,y2 ), then the XY coordinates of the predicted coordinates used in the second exploration (X 2 , Y 2 ) can be expressed by the following equation (2) using the constant matrix A expressed by equation (1). In addition, the nth order including the third order and subsequent orders can also be expressed by equation (3) in the same manner.

[0086] [Formula 2]

[0087] (X 2 ,Y 2 )=A(x 2 ,y 2 )…(2)

[0088] (X n ,,Y n )=A(x n ,y n )…(3)

[0089] In addition, the constant matrix A representing the correspondence between the position information and the actual coordinates may be updated each time based on the position information and the actual coordinates obtained in the repeated alignment control, or may be averaged multiple times. Alternatively, the correspondence between the position information and the actual coordinates may be pre-established by prior calibration. For example, in the case of alignment control using the corner end of the stage 101 as the identification object, the error each time is small, so the prior calibration as described above is effective.

[0090] As described above, the correspondence between the XY coordinates and the position information of the reference point of the identification object detected based on the captured image is determined in advance, and the XY coordinates are predicted based on the newly acquired position information using the correspondence (step S207). Then, a predetermined exploration range of size is determined with the predicted XY coordinates as the center (step S208). In addition, with respect to the θ coordinate, similarly, after the correspondence between the actual coordinates and the position information detected by detailed exploration is determined in advance, the θ coordinate representing the rotation angle of the identification object is predicted based on the newly acquired position information (step S207), and the exploration angle range of the predetermined angular width is determined with the predicted θ coordinate as the center (step S208). In addition, the exploration range may not be a range centered on the predicted coordinates, and the range containing the predicted coordinates may be set as the exploration range by aligning the conditions or identifying the object.

[0091] Here, the size of the search range can be set by the user input to the setting terminal 500, or it can be set automatically. For example, the size of the search range can be the size of the margin automatically or manually set based on the moving speed of the drive devices 111, 112, and 113 relative to the shape or size of the recognition object. Alternatively, the size of the search range can be statistically determined based on past alignment control results. For example, it can be the size after the margin is added with the average value of the difference between the predicted coordinates calculated based on the position information and the actual coordinates detected based on the captured image at the corresponding moments in the past relative to the shape and size of the recognition object.

[0092] That is, the range determination unit 412 does not perform the rough search of step S203 in the second and subsequent times and when the previous detection has not failed, and determines the search range centered on the predicted coordinates calculated based on the position information output by the position sensor in step S207. The amount of processing for determining the search range based on the position information is greatly reduced compared to the processing for determining the search range involved in the rough search performed in step S203. As a result, high-speed control can be achieved compared to the conventional alignment control performed by each rough search.

[0093] Next, the image acquisition unit 411 acquires an image (step S209; image acquisition step), and performs a simplified process on the image of the search range determined in step S208 among the acquired images (step S210). The simplified process is an arbitrary process before performing a detailed search. For example, the search unit 413 performs a rough search in a range larger than the search range determined in step S208 but limited to the entire range, and then determines the search range. Alternatively, in the case where the predicted coordinates of the XY coordinates are calculated in step S207 and the search range in the XY coordinates is determined in step S208, as a simplified process, a rough search of the θ coordinates may be performed to determine the search angle range. In addition, the simplified process may be omitted.

[0094] Then, the exploration unit 413 performs a detailed exploration on the exploration range determined in step S208 or the range determined in step S210 (step S211: exploration step). The method of detailed exploration is the same as that of step S205. When the result of the detailed exploration performed in step S211 is that the object cannot be detected (step S212: No), the process returns to step S201, and since the previous detection failed (step S201: Yes), the processing of steps S202 to 205 is performed. Here, the reason for re-performing the rough exploration of the entire range is to avoid repeated detection failures in the detailed exploration, which makes it impossible to control.

[0095] When the detection of the recognition object is successful in the detailed search of step S211 (step S212 : Yes), the actual coordinates of the recognition object acquired in step S211 are output to the motion control unit 200 .

[0096] Return to Figure 7 According to the flowchart of , the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold (step S106: Yes), the alignment process ends.

[0097] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107). Specifically, a control signal for making the actual coordinates consistent with the target coordinates is output to the drive control devices 121, 122, and 123. Then, the process returns to step S102 to continue.

[0098] As described above, the alignment system 1 according to the present embodiment includes: an image processing unit 400 that detects the identification object based on an image obtained by photographing the workpiece 10 having the identification object; and an action control unit that controls the drive devices 111, 112, and 113 that move the workpiece 10 based on the actual coordinates of the identification object detected by the image processing unit 400. The image processing unit 400 predetermines the correspondence between the position information output by the position sensor based on the drive devices 111, 112, and 113 and the actual coordinates of the identification object detected based on the photographed image, calculates the predicted coordinates of the identification object based on the position information obtained next using the correspondence, determines the search range including the predicted coordinates, and detects the identification object based on the image of the search range. As a result, the rough search process of the entire range can be omitted, and positioning can be performed at high speed and high precision.

[0099] (Implementation Method 2)

[0100] Next, Embodiment 2 for implementing the present invention will be described in detail with reference to the drawings. In addition, the same reference numerals are attached to the same or corresponding parts in the drawings.

[0101] The overall structure of the alignment system 2 involved in this embodiment 2 and the hardware structure of each structural part are the same as those in embodiment 1. In the alignment system 2 involved in this embodiment 2, the difference from embodiment 1 is that the image acquisition unit 411 of the image processing unit 400 performs a forwarding instruction to specify a forwarding range, thereby acquiring a captured image from the shooting device 300, and the captured image within the forwarding range is explored and processed by the exploration unit 413.

[0102] In this embodiment, the drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are connected to each other for communication as in Embodiment 1. However, this embodiment requires higher responsiveness than Embodiment 1, and therefore, it is preferred to connect to each other for communication via an industrial network such as CC-Link IE / field or CC-Link IE / TSN.

[0103] Industrial networks such as CC-Link IE / field and CC-Link IE / TSN have a mechanism to achieve a certain degree of timing through H / W devices to ensure synchronization, and to statistically measure transmission path delays to align the timing between devices with μ-second accuracy.

[0104] The synchronization method based on transmission path delay measurement is a method that uses the transmission path delay measurement value from the master station to the device station to achieve higher-precision synchronization. Fig.10 2 shows a synchronization method implemented by transmission path delay measurement. In the transmission path delay measurement method, synchronization is performed using a synchronization point. The transmission control (MyStatus) frame sent from the master station 40 is delayed as the distance increases.

[0105] The master station 40 calculates the transmission path delay time in each device station 50, 60, 70 based on the master station time when the response signal is received from each device station 50, 60, 70, and sends it to each device station 50, 60, 70. The synchronization point is the time after a certain time (Tsync) has passed since the transmission control (MyStatus) frame was sent out by the master station 40. After receiving the transmission control (MyStatus) frame, each device station 50, 60, 70 synchronizes after the time Tps (Tsync-delay time) is subtracted from the transmission path delay time.

[0106] In the present embodiment, the transmission path delay measurement method is used, for example, the motion control unit 200 is used as the master station 40 to perform timing control of the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400. That is, the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400 are connected to each other for communication via an industrial network (communication line) for measuring the transmission path delay time, and have synchronized time information.

[0107] The action control unit 200 as the master site 40 sets the specific time after the longest transmission path delay time from the instructed transmission time as the synchronization point. Furthermore, the image processing unit 400 determines the range including the predicted coordinates of the future specific time as the forwarding range, and instructs the imaging device 300 to capture at the specific time and forward the images within the forwarding range among the captured images.

[0108] Furthermore, the imaging device 300 needs to perform timing control taking the shutter speed into consideration. Fig.11 This is a diagram for explaining the timing control of the shutter of the camera 300. The camera 300 automatically determines the shooting timing according to the setting information obtained from the image processing unit 400 or in advance, using the synchronization time information which is the calculation source of the synchronization point. At this time, in order to shoot at a specific time in the future, the image processing unit 400 can issue a prior shooting instruction within the grace time as needed. In addition, it is also effective to align the timing of the scheduled shooting with the center of the exposure time range.

[0109] Fig.12 is a block diagram showing an example of the functional structure of the alignment system 2 according to the present embodiment. Fig.13 is a flowchart of the alignment control process executed by the motion control unit 200, Fig.14 is a flowchart of the exploration process performed by the image processing unit 400. Fig.13 , 14 The operation of the alignment system 2 is explained with reference to the flowchart of FIG.

[0110] First, the motion control unit 200 executes a command to move the workpiece 10 to a preset estimated target position ( Fig.13 , step S101). Specifically, the motion control unit 200 outputs a control signal for realizing movement in each direction to the drive control devices 121, 122, and 123. Furthermore, by controlling the drive control devices 121, 122, and 123 based on the control signal, the drive devices 111, 112, and 113 are driven, and the mounting table 101 on which the workpiece 10 is mounted moves.

[0111] After the movement instruction is issued, the motion control unit 200 determines the shooting time in the shooting device 300 as the time T n (n=1) (step S122), the image processing unit 400 is designated. In addition, the position information generation unit 211 of the motion control unit 200 generates position information based on the output of the position sensor of the drive device 111, 112, 113, and outputs the position information (step S103). Then, the motion control unit 200 waits until the search of the image processing unit 400 is completed (step S104).

[0112] The image processing unit 400 that obtains the position information from the motion control unit 200 executes Fig.14 Here, since the image is captured for the first time (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 instructs the image capture device 300 to capture the image at the time T n (n=1) and forwarding of the captured image of the entire range (step S221). Then, the image acquisition unit 411 acquires the captured image of the entire range (step S222).

[0113] Next, the exploration unit 413 of the image processing unit 400 performs a rough exploration (rough search) of the recognition object for the entire range of the captured images (step S203). The rough exploration is performed, for example, by pattern matching processing using a pattern model of the pre-registered alignment mark 1001. The exploration unit 413 detects the alignment mark 1001 when the consistency rate of pattern matching is greater than or equal to a predetermined threshold as the recognition object, and determines the exploration range of the detailed exploration based on the position or rotation angle of the reference point of the alignment mark 1001 (step S204). The shape or size of the exploration range at this time can be predetermined based on the shape or size of the alignment mark 1001, and can be set by the user's input to the setting terminal 500.

[0114] Next, a detailed search (fine search) is performed on the image within the search range determined in step S204 (step S205). The detailed search, for example, performs edge detection to detect accurate straight lines or curves, thereby obtaining more accurate actual coordinates of the identified object. Here, the actual coordinates of the identified object include the position (XY coordinates) and the rotation angle (θ coordinate) of the reference point of the identified object.

[0115] If the recognition object cannot be detected in the detailed search of step S205 (step S212: No), the process returns to step S201 and repeats the processes of steps S221, S222, and S203 to 205. On the other hand, if the recognition object is successfully detected in the detailed search of step S205 (step S212: Yes), the actual coordinates of the recognition object obtained in step S205 are output to the motion control unit 200 (step S213).

[0116] Back to Fig.13, after the exploration process of the image processing unit 400 is completed (step S104: Yes), the action control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold value (step S106: Yes), the alignment control process is terminated. Here, the target coordinates are the coordinates of the alignment target, including the center position (XY coordinates) and the rotation angle (θ coordinate).

[0117] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106 : No), the command unit 213 executes a command to correct the position of the alignment mechanism 100 (step S107 ; command step). Specifically, a control signal for aligning the actual coordinates with the target coordinates is output to the drive control devices 121 , 122 , 123 .

[0118] After issuing the calibration instruction to the alignment mechanism 100, the motion control unit 200 returns to step S122 and determines the shooting time as time T n (n=2) (step S122), the position information is output (step S103), and the process waits until the search process of the image processing unit 400 is completed (step S104).

[0119] Return to Fig.14 , the next shooting is the second time and thereafter, and since the previous detection is successful (step S201: No), the image processing unit 400 instructs the shooting device 300 to shoot at time T n (n=2) is captured (step S223). In addition, the image processing unit 400 obtains position information from the motion control unit 200 (step S206). The range determination unit 412 of the image processing unit 400 predicts the coordinates of the recognition target at that time (step S207).

[0120] The method for predicting coordinates is the same as that in Embodiment 1, and the correspondence between the position information obtained from the motion control unit 200 during the initial alignment control and the actual coordinates detected by the initial detailed exploration is determined. The range determination unit 412 uses the determined correspondence to predict the XY coordinates of the reference point of the identification object based on the position information obtained in step S206 (step S207). Then, the range determination unit 412 determines a forwarding range of a predetermined size with the XY coordinates predicted in step S207 as the center, and issues a forwarding instruction to the camera 300 to forward the captured image of the designated forwarding range (step S226). In addition, the forwarding range may not be a range centered on the predicted XY coordinates, and the range containing the predicted XY coordinates may be used as the forwarding range depending on the alignment conditions or the method of identifying the object.

[0121] Here, the size of the forwarding range can be set by the user input to the setting terminal 500. For example, since robustness and processing speed are in a trade-off relationship, the forwarding range can be set small when throughput is given priority by allowing processing fluctuations and maximizing the average processing speed. On the other hand, when jitter characteristics are given priority over processing speed and it is desired to make the processing time constant, the forwarding range can be set large.

[0122] Alternatively, the size of the forwarding range may be a size after adding a margin automatically or manually set based on the moving speed of the drive devices 111, 112, and 113 relative to the shape or size of the recognition object. Alternatively, the range determination unit 412 may statistically determine the size of the forwarding range based on past alignment control results, for example, by adding the average value of the difference between the predicted coordinates calculated based on past position information and the actual coordinates of the recognition object detected from the image to the size of the recognition object as a margin, and setting the size of the forwarding range as the size.

[0123] As described above, when the image is taken for the second time or later and the previous detection has not failed, the image capturing device 300 does not forward the image of the entire range, but only forwards the image of the forwarding range centered on the predicted coordinates based on the position information of the position sensor obtained in step S226. The data capacity of the image forwarded here is greatly reduced compared to the data volume of the image of the entire range forwarded in step S221. As a result, compared with the case where the image of the entire range is forwarded each time, the forwarding capacity of the data can be reduced, and the forwarding time can be shortened. In addition, since the rough search of the entire range is not performed, high-speed control can be achieved.

[0124] The image acquisition unit 411 acquires the image of the forwarding range forwarded by the forwarding instruction in step S226 (step S209; image acquisition step), and the exploration unit 413 performs a simple process on the image within the forwarding range (step S210). The simple process is an arbitrary process before performing a detailed exploration. For example, a rough exploration can be performed on the image within the forwarding range obtained in step S209 to determine the exploration range. In particular, a rough search of the θ coordinate can be performed as a simple process to determine the exploration angle range. In addition, the simple process can also be omitted.

[0125] Then, a detailed search is performed on the image within the forwarding range obtained in step S209 or on the search range determined in step S210 (step S211; search step). The method of detailed search is the same as that of step S205. When the result of the detailed search performed in step S211 is that the object cannot be detected (step S212: No), the process returns to step S201, and since the previous detection failed (step S201: Yes), the processing of steps S221, S222, and S203 to 205 is performed. Here, the reason for re-performing the rough search from the entire range is to avoid the failure of repeated detection in the detailed search, which makes it impossible to control.

[0126] When the detection of the recognition object is successful in the detailed search of step S211 (step S212 : Yes), the actual coordinates of the recognition object acquired in step S211 are output to the motion control unit 200 (step S213 ).

[0127] Return to Fig.13 According to the flowchart of , the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105) and calculates the difference with the target coordinates. When the difference between the actual coordinates and the target coordinates is less than or equal to the threshold (step S106: Yes), the alignment process ends.

[0128] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100 (step S107). Specifically, a control signal for making the actual coordinates consistent with the target coordinates is output to the drive control devices 121, 122, and 123. Then, the process returns to step S122 to continue.

[0129] As described above, the range determination unit 412 of the image processing unit 400 determines the image transfer range based on the position information, and the search unit 413 performs a detailed search on the image transferred from the imaging device 300. This can significantly shorten the transfer time and the search time.

[0130] In addition, in addition to the method of specifying the shooting parameters including the forwarding range together with the shooting instruction, the forwarding range from the image processing unit 400 to the shooting device 300 can be specified in advance for the shooting device 300 and then the shooting trigger can be output to the shooting device 300.

[0131] When the forwarding range is specified together with the shooting instruction of the former, the shooting of the shooting device 300 and the acquisition timing of the position information are aligned using the synchronization method of the transmission path delay measurement described above, thereby reducing the waiting time and performing the search in the appropriate forwarding range. On the other hand, when the forwarding range is specified before the latter, only the shooting trigger is output through the dedicated line, thereby aligning the shooting timing with high precision.

[0132] In addition, through Fig.14 The image of the entire designated range is forwarded in step S221, and the image of the designated forwarding range is forwarded in step S226. The image can be forwarded by reducing the image by the camera 300. The reduction method can be any existing method, such as using subsampling or merging. In particular, merging has the effect of making the processing lighter and improving the S / N ratio (Signal to Noise Ratio) of the pixel signal, and can suppress the reduction in position recognition accuracy caused by the reduction in resolution, so it is preferred.

[0133] When the forwarding range is limited (trimmed) by the forwarding instruction in step S226, the forwarding range is limited and reduced when the actual coordinates and the target coordinates are separated in the early stage of the alignment control, and when the distance between the actual coordinates and the target coordinates is within a certain range and close, control without reduction can be performed. In addition, when reducing the image in the entire range or the forwarding range, a model matching the reduced data can be prepared in advance as a pattern model for exploration, and the exploration unit 413 performs a rough exploration or a detailed exploration using the model.

[0134] In addition, the captured images of the entire range can be acquired in parallel with the detailed exploration (step S211) after the captured images of the forwarding range are acquired by the forwarding instruction in step S226. The time for acquiring the captured images of the entire range can be reduced in the case where the identification object cannot be detected by the detailed exploration in step S211 (step S212: No). In this case, the forwarding of the captured images of the entire range can be stopped or the forwarded images can be deleted at the stage where the identification object can be detected by the detailed exploration.

[0135] As described above, in the alignment system 2 according to the present embodiment, the image processing unit 400 predetermines the correspondence between the position information of the drive devices 111, 112, and 113 and the actual coordinates of the identification object detected from the captured image, calculates the predicted coordinates of the identification object based on the position information obtained next, determines the forwarding range of the image including the predicted coordinates, and detects the identification object by performing a detailed search of the image within the forwarding range forwarded from the imaging device 300. Thus, the forwarding time and the capacity of the forwarded image can be reduced, and the rough search of the image of the entire range can be omitted, so that alignment control can be performed at high speed and with high precision.

[0136] (Variation Example)

[0137] The above-described embodiment can be modified in various ways. Modifications are described below.

[0138] [Modification 1]

[0139] In embodiments 1 and 2, the range determination unit 412 of the image processing unit 400 determines the range including the predicted coordinates calculated based on the position information output by the position sensor of the driving device 111, 112, 113 as the exploration range or the forwarding range. However, the exploration range or the forwarding range may be determined by other methods. This modification 1 describes other methods for determining the exploration range or the forwarding range.

[0140] This modification example uses command information related to driving the driving devices 111, 112, and 113 when calculating predicted coordinates for determining the search range or forwarding range. Specifically, the range determination unit 412 of the image processing unit calculates predicted coordinates based on the command information and determines the search range or forwarding range including the predicted coordinates.

[0141] In this modification, by ensuring synchronization while performing timing control of imaging by the imaging device 300 , driving of the driving devices 111 , 112 , 113 based on command information, and acquisition of the position sensor output, higher-speed control can be performed. Fig.15 is a block diagram showing an example of the functional structure of the alignment system 3 involved in this modification example, Fig.16 is a flowchart of the alignment control process performed by the motion control unit 200 according to this modification. Fig.17 is a flowchart of the search process performed by the image processing unit 400.

[0142] Here, Fig.1517 is a variation of the second embodiment, which shows a method of determining the forwarding range of an image based on instruction information. However, as a variation of the first embodiment, a method of determining the search range of an image based on instruction information can also be implemented in the same manner. Fig.16 , 17 The operation of the alignment system 3 according to this variant is described with reference to the flowchart, but descriptions of the same processing as that in the second embodiment are omitted.

[0143] First, the motion control unit 200 executes a command to move the workpiece 10 to a preset estimated target position ( Fig.16 , step S101). The command information at this time is transmitted to the image processing unit 400 for determining the forwarding range. Then, based on the control signal related to the command information, the driving devices 111, 112, and 113 start driving, and the mounting table 101 on which the workpiece 10 is mounted moves.

[0144] After the movement instruction, the motion control unit 200 determines the shooting time in the shooting device 300 as the specific time T in the future. n (n=1) (step S122), the image processing unit 400 is designated. In addition, the position information generation unit 211 of the motion control unit 200 generates a time T based on the output of the position sensor of the drive device 111, 112, 113. n The action control unit 200 then waits until the search of the image processing unit 400 is completed (step S104). The subsequent actions of the image processing unit 400 are the same as those of the second embodiment, and the image processing unit 400 performs Fig.17 Processing of steps S221, 222, 203-205, 212, and 213.

[0145] Back to Fig.16 When the search process of the image processing unit 400 is completed (step S104: Yes), the movement amount calculation unit 212 of the motion control unit 200 obtains the actual coordinates through the image processing unit 400 (step S105), and the position information generation unit 211 generates the position information based on the time T n The position sensor output generates and records the position information (step S124). The movement amount calculation unit 212 calculates the difference between the actual coordinates and the target coordinates, and ends the alignment control process when the difference between the actual coordinates and the target coordinates is less than or equal to the threshold (step S106: Yes).

[0146] When the difference between the actual coordinates and the target coordinates exceeds the threshold value (step S106: No), the instruction unit 213 executes an instruction to correct the position of the alignment mechanism 100. However, at this time, the specific time T recorded in step S124 is referred to.n The position information is used to execute an instruction to compensate for the deviation between the position indicated by the instruction and the actual position indicated by the position information (step S125).

[0147] After the instruction unit 213 executes the instruction to the alignment mechanism 100, the process returns to step S122 and the imaging time is determined to be time T. n (n=2) (step S122), for a specific time T n The record of the position information of (n=2) is designated (step S123), and the process waits until the search process of the image processing unit 400 is completed (step S104).

[0148] Convert again to Fig.17 , the next shooting is the second time and thereafter, and since the previous detection is successful (step S201: No), the image processing unit 400 instructs the shooting device 300 to shoot at time T n (n=2) (step S223). In addition, the image processing unit 400 obtains the time T from the motion control unit 200. n The range determination unit 412 of the image processing unit 400 calculates the predicted coordinates of the recognition object at that time (step S225).

[0149] The method of calculating the predicted coordinates by the range determination unit 412 first determines the correspondence between the command information during the initial alignment control and the actual coordinates detected by the initial detailed search, instead of the position information of the second embodiment. The range determination unit 412 uses the determined correspondence to calculate the predicted coordinates according to the specific time T obtained in step S224. n The prediction method using the correspondence relationship is the same as that in the second embodiment.

[0150] Then, the range determination unit 412 determines a forwarding range of a predetermined size with the XY coordinates predicted in step S225 as the center, and instructs the camera 300 to forward the image within the determined forwarding range. (Step S226). In addition, the forwarding range may not be a range centered on the predicted XY coordinates, and the range including the predicted XY coordinates may be used as the forwarding range according to the alignment conditions or the method of identifying the object.

[0151] The image acquisition unit 411 acquires the image within the forwarding range forwarded by the forwarding instruction in step S226 (step S209), and performs a simple process (step S210). Then, the exploration unit 413 performs a detailed exploration on the image within the forwarding range acquired in step S209 or the image after the simple process in step S210 (step S211). When the result of the detailed exploration in step S211 is that the recognition object cannot be detected (step S212: No), the process returns to step S201, and since the previous detection failed (step S201: Yes), the process of steps S221, S222, and S203-205 is performed.

[0152] If the recognition object is successfully detected in the detailed search of step S211 (step S212: Yes), the actual coordinates of the recognition object acquired in step S211 are output to the motion control unit 200 (step S213). The subsequent processing of the motion control unit 200 is the same as that of the second embodiment.

[0153] As described above, the range determination unit 412 of the image processing unit 400 determines the forwarding range of the image based on the instruction information of the action control unit 200, and the exploration unit 413 performs detailed exploration on the image forwarded from the imaging device 300. Thus, the exploration range or the forwarding range is determined based on the instruction information related to the future scheduled position, so that more efficient timing control can be achieved, and higher-speed alignment control can be achieved.

[0154] [Modification 2]

[0155] This variant 2 describes another method for determining the exploration range or forwarding range. In embodiments 1 and 2, the range determination unit 412 determines the exploration range or forwarding range centered on the predicted coordinates calculated based on the location information, but in this variant, the exploration range or forwarding range is determined centered on the actual coordinates detected last time.

[0156] Specifically, the range determination unit 412 does not obtain position information or instruction information, and performs the previous detection through detailed exploration implemented by the exploration unit 413 of the image processing unit 400, and determines the range centered on the reference point of the identification object stored in the non-volatile memory 230 as the forwarding range. Alternatively, the range determination unit 412 determines the range centered on the reference point of the identification object stored, or the angle range centered on the rotation angle of the identification object as the exploration range. In addition, the forwarding range or the exploration range may not be the range centered on the actual coordinates detected last time, and the range including the actual coordinates detected last time may be used as the forwarding range or the exploration range according to the alignment conditions or the method of identifying the object.

[0157] The size of the exploration range or forwarding range can be determined based on the moving speed of the actual coordinates of the identification object detected in the past. Alternatively, it can be determined based on the moving speed of each drive device based on the setting parameters of the alignment control. For example, it can be a size determined by multiplying the maximum moving speed by the shooting interval of the shooting device 300. In the case where the moving distance between shooting intervals is short due to the shooting interval being set short, the exploration range or forwarding range can be fully limited by utilizing the previous detection result, so that the processing can be simplified compared with embodiments 1, 2 and variant 1.

[0158] [Variation 3]

[0159] This modification 3 describes another method for determining the search range or the forwarding range. In this modification, the range determination unit 412 determines the search range or the forwarding range based on the trajectory of the actual coordinates detected in the past in the detailed search performed by the exploration unit 413. That is, instead of acquiring position information or instruction information as in the first and second embodiments, in the detailed search performed by the exploration unit 413 of the image processing unit 400, the predicted coordinates are calculated based on the trajectory of the actual coordinates of the recognition object stored in the non-volatile memory 230, and the range centered on the predicted coordinates is determined as the search range or the forwarding range.

[0160] Specifically, the range determination unit 412 determines a range of a predetermined size centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identification object as the forwarding range. Alternatively, the range determination unit 412 determines a range of a predetermined size centered on the XY coordinates predicted based on the trajectory of the XY coordinates of the reference point of the identification object, or an angular range centered on the θ coordinate predicted based on the trajectory of the θ coordinate as the exploration range. Fig.18A , B is a diagram illustrating the prediction of the identification object. In addition, the forwarding range or the exploration range may not be a range centered on the prediction coordinates, and the range including the prediction coordinates may be used as the forwarding range or the exploration range according to the alignment conditions or the method of identifying the object.

[0161] The calculation of the predicted coordinates (XYθ coordinates) is performed by prediction at the time of shooting by the imaging device 300 using the time at least two times before the previous time and the actual coordinates at each time. Fig.18A As shown, linear prediction is performed based on the XY coordinate data of two times, the previous time and the time before that, to predict the XY coordinates at the time of shooting. Fig.18B As shown, two predictions are performed based on the XY coordinate data of three times, namely, the last time, the last time before, and the last time before, to predict the XY coordinates at the time of shooting.

[0162] The size of the exploration range or forwarding range can be determined based on the moving speed of the actual coordinates of the recognition object detected in the past. Alternatively, it can be determined based on the moving speed of the drive devices 111, 112, and 113. For example, it can be determined by multiplying the maximum moving speed by the shooting interval of the shooting device 300. Since the prediction accuracy of linear prediction is poor compared with second-order prediction or higher-order prediction, it is necessary to increase the size of the exploration range or forwarding range.

[0163] In addition, the size of the exploration range or forwarding range can also be determined based on statistical information on errors in past detection results. For example, it can be the size after adding the average value of the difference between the predicted coordinates of the trajectory based on the actual coordinates at the corresponding moments in the past and the actual coordinates detected from the captured image as a margin with respect to the shape and size of the identified object.

[0164] According to this modification, when the imaging interval is long and the moving distance between imaging intervals is long, the prediction accuracy is improved and the search range or the forwarding range can be narrowed.

[0165] [Variation 4]

[0166] In the first and second embodiments, the case where the recognition target is the alignment marks 1001 and 1002 attached to the workpiece 10 is described. However, a part of the processing may be changed according to the form of the recognition target. Fig.19A 2 is a diagram for explaining the rough search involved in this modification example. Fig.19B This is a diagram for explaining the detailed exploration related to this modification example.

[0167] For example, in the case where workpiece 12 is Fig.19A In the case where the IC (Integrated Circuit) has many terminals and the identification target is the terminal of the IC, the search unit 413 performs pattern matching processing using the pattern model 1312, thereby performing a rough search ( Figure 8 , step S203).

[0168] At this time, the exploration unit 413 determines the exploration range 1314 ( Figure 8 , step S204). In the pattern matching, if Fig.19A As shown, sometimes a plurality of patterns having a consistency rate greater than or equal to a predetermined threshold value are detected, but in this case, the pattern 1313 having the highest consistency rate can be identified to determine the search range 1314.

[0169] exist Fig.19BIn the example shown, the search unit 413 determines the search range 1314 having a predetermined positional relationship based on the pattern 1313 determined by the rough search as the search range for the detailed search ( Figure 8 , step S204). The exploration unit 413 performs a detailed exploration on the exploration range determined in step S204 ( Figure 8 , step S205).

[0170] Here, in the first and second embodiments, when it is the second or subsequent shooting and the previous detection is successful ( Figure 8 , step S201: No), determine the search range including the predicted coordinates calculated based on the location information ( Figure 8 , step S208), perform detailed exploration ( Figure 8 , step S211). However, in the case Fig.19A As shown in FIG. 1 , when the predicted coordinates can be calculated with a certain accuracy, the predicted coordinates based on the position information may not be calculated, and the detailed exploration may be repeated using the exploration range determined by the rough exploration in step S203. Furthermore, the determination process of the exploration range involved in the rough exploration may be performed only when the detection by the detailed exploration fails.

[0171] According to this modification, the processing can be simplified according to the method of the recognition object. In addition, the method of the recognition object can be classified by setting the application of terminal 500, and the search processing method suitable for the classification can be automatically selected.

[0172] [Variation 5]

[0173] This modification example 5 describes a case where the identification target is in a further different form. Fig. 20A 2 is a diagram for explaining the rough search involved in this modification example. Fig. 20B This is a diagram for explaining the detailed search according to this modification. In this modification, the processor of the terminal 500 is set to execute a dedicated application program, thereby automatically performing model registration for the rough search and logic generation for the detailed search.

[0174] For example, in workpiece 13, Fig. 20A In the case of the lens shown in FIG. 1 , the processor of the setting terminal 500 extracts the shape of the lens from the captured image and registers the pattern model 1322. Furthermore, the processor automatically generates exploration logic for detailed exploration based on the shape of the pattern model 1322. For example, in the case where the processor of the setting terminal 500 detects that the shape of the pattern model 1322 includes an arc, as shown in FIG. Fig. 20B As shown, as the detailed search logic, the edge is searched, a circle approximation is performed on the detected edge, and the logic for detecting with the center of the circle as the reference point 1323 is automatically generated.

[0175] The image processing unit 400 performs the exploration process described in the first and second embodiments using the rough exploration pattern model and the detailed exploration logic generated by the application of the setting terminal 500. According to this variation, it is possible to perform the optimal rough exploration and detailed exploration according to the method of recognizing the object. In addition, the prior setting for executing the exploration process of the image processing unit 400 can be automated, and the burden of the user's setting input to the setting terminal 500 can be reduced.

[0176] [Variation 6]

[0177] In the first and second embodiments, a rough search is performed on the first captured image, a detailed search is performed on the search range determined based on the result of the rough search, and based on the result of the detailed search, the predicted coordinates of the recognition object used to determine the search range or forwarding range of the next detailed search are calculated. However, in the case where the prediction accuracy of the coordinates of the recognition object is sufficiently high, the rough search on the first captured image can be omitted by the prior calibration of the alignment control. In particular, in the case of the alignment control performed using the corner end of the mounting table 101 on which the workpiece 10 is mounted, the error for each execution is small, so the prior calibration as described above is effective.

[0178] Fig.21 This is a flowchart of the exploration process in the case where the rough exploration for the first captured image is omitted. After the image acquisition unit 411 acquires the first captured image (step S301), the range determination unit 412 uses the previous calibration information and calculates the predicted coordinates of the recognition object based on the position information (step S302). The exploration range including the predicted coordinates calculated by the range determination unit 412 is determined (step S303), and the detailed exploration is performed on the determined exploration range (step S304).

[0179] In the case where the actual coordinates of the recognition object are detected successfully in the detailed search of step S304 (step S305: Yes), the actual coordinates are output to the motion control unit 200 (step S309), and the motion control unit 200 controls the driving devices 111, 112, and 113 using the actual coordinates. After the movement of the workpiece 10, the detailed search is performed for the next captured image with respect to the same search range as the previous search. As described above, in the case where the detection is successful in the detailed search, the rough search is not performed, but the detailed search is repeatedly performed.

[0180] In the repeated detailed exploration, only when the detection of the identification object fails (step S305: No), the rough exploration is performed (step S306), thereby determining the exploration range (step S307), performing the detailed exploration in the determined exploration range (step S308), and outputting the actual coordinates (step S309). As described above, according to this modification, the detailed exploration is repeated based on the prior calibration information, and the rough exploration is performed only when the identification object cannot be detected. As a result, it is possible to achieve high-speed alignment control with simplified processing.

[0181] In addition, the hardware structure and flowchart shown in the above-mentioned embodiment and modified example are examples and can be arbitrarily changed and modified. For example, in the above-mentioned embodiment and modified example, the action control unit 200, the shooting device 300, the image processing unit 400 and the setting terminal 500 are independent structures, but they can also be a structure that integrates at least two of them.

[0182] In addition, the exploration processing shown in the above-mentioned embodiments and modified examples can also be performed in any combination. In addition, any of the exploration processing involved in the above-mentioned embodiments and modified examples can be selected by setting the terminal 500. Alternatively, the image processing unit 400 can automatically select any of the exploration processing involved in the above-mentioned embodiments and modified examples according to conditions such as the type of workpiece and the method of identifying the object.

[0183] In addition, the sharing of the functions implemented by the processors in the motion control unit 200 and the image processing unit 400 in the above-mentioned embodiments and modified examples is an example, and the sharing can be arbitrarily changed. In addition, the functions implemented by the processors in the motion control unit 200 or the image processing unit 400 do not rely on a dedicated system, and can be implemented using a common computer system.

[0184] In addition, the program for executing the actions of the above-mentioned embodiments can be stored in a computer-readable CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical Disc), memory card or other recording medium and distributed, and the program can be installed in a computer to form a computer capable of realizing each function. Moreover, in the case where each function is realized by sharing the OS (Operating System) and applications, or by the collaboration of the OS and applications, only the part other than the OS can be stored in the recording medium.

[0185] The present invention does not depart from the broad spirit and scope of the present invention, and various embodiments and variations can be realized. In addition, the above-mentioned embodiments are used to illustrate the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is not the embodiment, but is shown by the claims. Moreover, various variations implemented within the scope of the claims and the meaning of the invention equivalent thereto are regarded as within the scope of the present invention.

[0186] Description of the label

[0187] 1, 2, 3 alignment system, 10, 12, 13 workpiece, 11 alignment target, 40 main site, 50, 60, 70 equipment site, 100 alignment mechanism, 101 mounting table, 111, 112, 113 driving device, 121, 122, 123 driving control device, 200 motion control unit, 210 processor, 211 position information generating unit, 212 movement amount calculating unit, 213 instruction unit, 220 volatile memory, 230 non-volatile memory, 231 control program, 232 control data, 240 clock, 250 communication interface, 300 imaging device, 4 00 image processing unit, 410 processor, 411 image acquisition unit, 412 range determination unit, 413 exploration unit, 420 volatile memory, 430 non-volatile memory, 431 control program, 432 control data, 440 clock, 450 communication interface, 500 setting terminal, 1001, 1002 alignment mark, 1101, 1102 alignment target mark, 1200 θ-axis rotation center, 1201 virtual mark, 1300 shooting range, 1301, 1314 exploration range, 1312, 1322 pattern model, 1313 pattern, 1323 reference point.

Claims

1. An alignment system comprising: A photographing device for photographing a workpiece having an identification object; an image processing unit that detects the identification object based on the captured image of the workpiece obtained from the capturing device; and an action control unit that controls a driving device that moves the workpiece based on the actual coordinates of the recognition object detected by the image processing unit, The image processing unit determines a forwarding range of the image based on the actual coordinates of the recognition object, and detects the recognition object based on the captured image within the forwarding range acquired from the imaging device by a forwarding instruction designating the forwarding range.

2. The alignment system according to claim 1, wherein: The image processing unit determines a range having a preset size and including a reference point of the recognition target detected last time as the transfer range.

3. The alignment system according to claim 1, wherein: The image processing unit determines, as the transfer range, a range having a preset size and including predicted coordinates predicted based on a trajectory of a reference point of the recognition target detected in the past.

4. The alignment system according to claim 1, wherein: The image processing unit predetermines the correspondence between the instruction information involved in the driving of the driving device and the actual coordinates of the identified object detected based on the captured image, and determines a range with a predetermined size including the predicted coordinates calculated based on the instruction information using the correspondence as the forwarding range.

5. The alignment system of claim 1, wherein: The image processing unit predetermines the correspondence between the position information output by the position sensor of the driving device and the actual coordinates of the identified object detected based on the captured image, and determines a range with a pre-set size including the predicted coordinates calculated based on the position information using the correspondence as the forwarding range.

6. The alignment system of claim 5, wherein: The position information is information generated based on an output of an encoder attached to the drive device.

7. An alignment system according to any one of claims 1 to 6, wherein: The transfer range has a size obtained by adding a margin determined based on a moving speed of the driving device to the shape or size of the identification object.

8. An alignment system according to any one of claims 3 to 6, wherein: The transfer range has a size obtained by adding a margin determined based on a difference between the predicted coordinates and the actual coordinates detected from the captured image at a past corresponding time with respect to the shape or size of the recognition object.

9. An alignment system according to any one of claims 3 to 6, wherein: having time information of synchronization among the photographing device, the driving device, the image processing unit and the action control unit, The image processing unit determines a range including the predicted coordinates at a specific time in the future as the transfer range, and instructs the imaging device to capture images at the specific time and to transfer images within the transfer range among the captured images.

10. The alignment system of claim 9, wherein: The imaging device, the driving apparatus, the image processing unit, and the action control unit are communicatively connected to each other via a communication line for measuring a transmission path delay time, and the specific time is a time after a longest transmission path delay time has elapsed from an instructed transmission time.

11. An alignment method, comprising: An image acquisition step of acquiring an image from an imaging device that images a workpiece having an identification target; an exploration step of exploring the captured image acquired by the image acquisition step to detect the recognition object; and an instruction step of executing an instruction to move the workpiece based on the actual coordinates of the recognition object detected by the searching step, The image acquisition step determines a forwarding range of the image based on the actual coordinates of the recognition object, and acquires the captured image within the forwarding range from the imaging device using a forwarding instruction that designates the forwarding range.

12. A program for causing a computer that performs image processing of the captured image to function as the following part in an alignment system that obtains a captured image from a photographing device that photographs a workpiece having an identification object, detects the identification object based on the captured image, and performs alignment control of the workpiece based on the detected actual coordinates of the identification object: an image acquisition unit that determines a forwarding range of the image based on the actual coordinates of the recognized object, and acquires the captured image within the forwarding range from the imaging device by a forwarding instruction that specifies the forwarding range; and A search unit searches for the identification object from the captured image within the forwarding range.

Images