Position measurement system and lighting condition optimization system

Abstract

A position measurement system that measures the relative position of a three-dimensional form element with respect to a workpiece using the workpiece and the three-dimensional form element provided to the workpiece as objects is provided with: an imaging device that captures an image of the workpiece; an illumination device which is used as a light source for imaging by the imaging device and can change the illumination condition of the workpiece; an image processing unit that finds a relative position using the image of the workpiece; an illumination condition determination unit that determines a suitable illumination condition for the illumination device, the suitable illumination condition being such that a variation in the relative position obtained by the image processing unit, which is caused by a variation in the positional relationship between the imaging device and the workpiece when the image of the workpiece is imaged, becomes small; and a measurement control unit that causes the imaging device to capture an image of the workpiece under appropriate illumination conditions, and causes the image processing unit to determine the relative position using the captured image of the workpiece.

Description

[0001] Related Applications

[0002] This application is based on Japanese application No. 2022-156495 filed on September 29, 2022, and the contents thereof are cited herein. Technical Field

[0003] The invention relates to a position measurement system and a lighting condition optimization system. Background Art

[0004] When an LED element is mounted on a substrate, the LED element needs to be mounted at a correct position on the substrate in order to ensure positional accuracy with respect to a lens that receives light emitted from the LED. Therefore, the LED mounting position is inspected.

[0005] Patent document 1 describes the following: by optically identifying a bracket holding a substrate with LED components mounted thereon, the position of a representative mark formed on the bracket and the mounting position of the LED component mounted on the substrate in the bracket are detected, and the relative position of the mounting position with respect to the representative mark is determined.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2014-179186 Summary of the invention

[0009] Technical problem to be solved by the invention

[0010] However, for a three-dimensional object with rounded corners such as an LED element, if the direction of light irradiating the three-dimensional object changes for photographing, the state of the shadow on the surface of the three-dimensional object changes, and thus the measured value of the position may change. In the case of photographing while conveying the substrate by a conveyor, the positional relationship between the photographing device and the lighting device and the substrate during photographing may deviate, and the direction of light irradiating the three-dimensional object on the substrate may also change, and thus the measured value of the position may change.

[0011] The conventional method described in Patent Document 1 is based on the premise that there is no deviation in the positional relationship between the imaging device and the lighting device and the substrate during imaging, and therefore does not recognize the possibility of the above-mentioned fluctuation in the measured value.

[0012] The object of the present invention is to reduce the variation of the measured value of the relative position of a three-dimensional morphological element such as an LED element possessed by a workpiece such as a substrate with respect to the workpiece even if the positional relationship between the imaging device and the workpiece changes when measuring the relative position of the morphological element with respect to the workpiece.

[0013] Solutions for solving technical problems

[0014] A position measuring system according to an embodiment of the present invention takes a workpiece and a three-dimensional morphological element possessed by the workpiece as an object, and measures the relative position of the morphological element with respect to the workpiece, the position measuring system comprising: a photographing device for photographing the workpiece; a lighting device used as a light source for the photographing device to photograph, and capable of changing the lighting condition for the workpiece; an image processing unit for obtaining the relative position using an image of the workpiece; a lighting condition determination unit for determining an appropriate lighting condition for the lighting device, the appropriate lighting condition reducing a change in the relative position obtained by the image processing unit due to a change in the positional relationship between the photographing device and the workpiece when the image of the workpiece is photographed; and a measurement control unit for causing the photographing device to photograph the workpiece under the appropriate lighting condition, and causing the image processing unit to obtain the relative position using the photographed image of the workpiece.

[0015] According to the above configuration, when measuring the relative position of a three-dimensional morphological element of the workpiece with respect to the workpiece, even if the positional relationship between the imaging device and the workpiece varies, the variation in the measured value of the relative position of the morphological element with respect to the workpiece can be reduced.

[0016] Furthermore, the morphological element may be a one-dimensional morphological element, and the image processing unit may determine the relative position by using an intersection point between a search line and the one-dimensional morphological element as the position of the morphological element.

[0017] Furthermore, the morphological element may be a two-dimensional morphological element, and the image processing unit may obtain the relative position by using a representative position of the two-dimensional morphological element determined in a search area as the position of the morphological element.

[0018] In addition, it may also be that the morphological element is a plurality of one-dimensional morphological elements, and the image processing unit obtains the position of the intersection of a search line among a plurality of search lines and a morphological element among the plurality of one-dimensional morphological elements for a plurality of groups of one search line and one of the morphological elements, and uses a representative position obtained based on the obtained positions of the plurality of intersections as the position of the plurality of one-dimensional morphological elements to obtain the relative position.

[0019] In addition, it can also be set that the suitable lighting condition determined by the lighting condition determination unit is a lighting condition in which the image used in determining the suitable lighting condition is an image of a plurality of the workpieces, and includes a plurality of images with different positional relationships between the photographing device and the workpiece for one workpiece, and further, when the value regarded as the true value of the relative position can be utilized for each of the workpieces, the total of the relative positions of the plurality of workpieces caused by the change from the value regarded as the true value is reduced.

[0020] In addition, it can also be set that the suitable lighting condition determined by the lighting condition determination unit is a lighting condition in which, when the image used in determining the suitable lighting condition is an image of a plurality of the workpieces and includes a plurality of images having different positional relationships between the photographing device and the workpiece for one workpiece, the total of the changes in the relative position of each of the workpieces for the plurality of workpieces becomes smaller.

[0021] In addition, it may be that, when the workpiece has a plurality of the three-dimensional morphological elements, the lighting condition determination unit determines the suitable lighting condition for each of the three-dimensional morphological elements, and the image processing unit calculates the relative position for each of the plurality of the three-dimensional morphological elements possessed by the workpiece using an image of the workpiece photographed under the suitable lighting condition corresponding to the three-dimensional morphological element to be processed.

[0022] Alternatively, the position measuring system may further include a virtual image generating unit, which generates, based on the image of the workpiece photographed under a plurality of lighting conditions, a virtual image of the workpiece that should be obtained when photographing under lighting conditions different from the lighting conditions used for photographing; the image processing unit also calculates the relative position when the image of the workpiece is the virtual image; and the lighting condition determining unit determines the appropriate lighting condition, which reduces a change in the relative position calculated for the virtual image, or for the photographed image of the workpiece and the virtual image.

[0023] The position measurement system may further include a determination unit that determines a deviation of the relative position from a reference position and determines whether the relative position is good or not based on the deviation, wherein the relative position is determined by the image processing unit.

[0024] A lighting condition optimization system according to one embodiment of the present invention takes a workpiece and three-dimensional morphological elements possessed by the workpiece as objects, and optimizes lighting conditions for lighting when photographing the workpiece to measure the relative position of the morphological elements. The lighting condition optimization system comprises: a photographing device for photographing the workpiece; a lighting device used as a light source for the photographing device to photograph, and capable of changing the lighting conditions for the workpiece; an image processing unit for finding the relative position using an image of the workpiece; and a lighting condition determination unit for determining appropriate lighting conditions for the lighting device, wherein the appropriate lighting conditions reduce changes in the relative position found by the image processing unit that are caused by changes in the positional relationship between the photographing device and the workpiece when the image of the workpiece is photographed.

[0025] According to the above structure, it is possible to provide a lighting condition which, when measuring the relative position of a three-dimensional morphological element of a workpiece relative to the workpiece, can reduce the change in the measured value of the relative position of the morphological element relative to the workpiece even if the positional relationship between the imaging device and the workpiece changes.

[0026] A position measuring system according to an embodiment of the present invention takes a workpiece and a three-dimensional morphological element possessed by the workpiece as an object, and measures the relative position of the morphological element with respect to the workpiece, and the position measuring system comprises: a photographing device for photographing the workpiece; a lighting device used as a light source for the photographing device to photograph, and capable of changing the lighting condition for the workpiece; an image processing unit for obtaining the relative position using an image of the workpiece; a lighting condition determination unit for acquiring a predetermined suitable lighting condition of the lighting device, the suitable lighting condition reducing a change in the relative position obtained by the image processing unit due to a change in the positional relationship between the photographing device and the workpiece when the image of the workpiece is photographed; and a measurement control unit for causing the photographing device to photograph the workpiece under the suitable lighting condition, and causing the image processing unit to obtain the relative position using the photographed image of the workpiece.

[0027] According to the above structure, when measuring the relative position of a three-dimensional morphological element of a workpiece relative to the workpiece, by using the determined appropriate lighting conditions, even if the positional relationship between the imaging device and the workpiece changes, the change in the measured value of the relative position of the morphological element relative to the workpiece can be reduced.

[0028] Effects of the Invention

[0029] According to the present invention, when measuring the relative position of a three-dimensional morphological element of a workpiece with respect to the workpiece, even if the positional relationship between the imaging device and the workpiece varies, the variation of the measured value of the relative position of the morphological element with respect to the workpiece can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a diagram schematically showing an example of a position measurement system 1 according to an embodiment of the present invention.

[0031] Figure 2 It is a schematic diagram of the lighting device LS integrated with the camera 102 involved in the embodiment of the present invention.

[0032] Figure 3 It is a diagram showing an XZ cross section of the lighting device LS according to the embodiment of the present invention.

[0033] Figure 4 It is a bottom view of the lighting device LS according to the embodiment of the present invention.

[0034] Figure 5 Schematically shows the hardware configuration of a control device included in the position measurement system 1 according to the embodiment of the present invention.

[0035] Figure 6 This is a diagram schematically showing an example of the functional configuration of the position measurement system 1 according to the embodiment of the present invention.

[0036] Figure 7 It is a diagram showing the operation stages of the position measurement system 1 according to the embodiment of the present invention.

[0037] Figure 8 This is a diagram showing a substrate on which components are mounted as a measurement target in the embodiment of the present invention.

[0038] Fig. 9 It is a diagram for explaining the change of the shaded area in the components according to the embodiment of the present invention.

[0039] Fig.10 It is a diagram for explaining the position measurement of a one-dimensional marker according to the embodiment of the present invention.

[0040] Fig.11 It is a diagram for explaining the position measurement of a two-dimensional marker according to the embodiment of the present invention.

[0041] Fig.12 It is a diagram for explaining the measurement of positions specified by a plurality of one-dimensional markers according to the embodiment of the present invention.

[0042] Fig.13 It is a diagram showing an example of a base light-emitting pattern according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Hereinafter, an embodiment (hereinafter also referred to as "the present embodiment") according to one aspect of the present invention will be described based on the accompanying drawings. However, the embodiment described below is merely an illustration of the present invention in all aspects. Of course, various improvements and modifications can be made without departing from the scope of the present invention. That is, when implementing the present invention, a specific structure corresponding to the embodiment may also be appropriately adopted.

[0044] Figure 1 It is a diagram schematically showing an example of the position measurement system 1 of the present invention.

[0045] The position measurement system 1 measures the relative position of a three-dimensional morphological element of the workpiece 4 with respect to the workpiece by performing image processing on an input image obtained by photographing the workpiece 4, which is a measurement object conveyed on the belt conveyor 2. The workpiece 4 is, for example, a substrate, and an LED element is provided on the workpiece 4. The LED element is an example of a three-dimensional morphological element of the workpiece 4.

[0046] A camera 102 as a photographing device integrated with a lighting device LS is arranged on the upper part of the belt conveyor 2, and a photographing field of view 6 of the camera 102 is configured to include a predetermined area of ​​the belt conveyor 2. Here, as the lighting device LS, for example, a multi-channel lighting device such as an MDMC (Multi Direction Multi Color) lighting device can be cited, and more specifically, a lighting device described in, for example, Japanese patent application 2018-031747 of the present applicant can be cited.

[0047] Figure 2 1 is a schematic diagram of a lighting device LS integrated with a camera 102. Figure 2 As shown in FIG. 1 , the direction of light irradiation from the lighting device LS is set as the Z axis, the left and right direction of the paper is set as the X axis, and the axis perpendicular to the X axis and the Z axis is set as the Y axis. In addition, the side of the irradiated light is set as the bottom, and the side recorded as Figure 2 The "front" position of the lighting device LS is set to the right side, the left side is set to the left, the front side is set to the front, and the back side is set to the back. The lighting device LS is controlled by the processing device 10 via the camera 102. An opening 46 is provided at the top of the lighting device LS so that the camera 102 can photograph the object W from the top of the lighting device LS.

[0048] also, Figure 3 is a diagram showing an XZ cross section of the lighting device LS (from Figure 2 The XZ section observed from the "front" position is indicated), Figure 4 LS is a bottom view of the lighting device. Figure 3 , Figure 4As shown in FIG. 1 , the lighting device LS as a multi-channel lighting device has a plurality of channel lighting units LSi. More specifically, Figure 4 As shown, the lighting device LS has a circular channel lighting unit LSiC in the center and 12 fan-shaped channel lighting units LSi arranged concentrically around the channel lighting unit LSi when viewed from above. Figure 3 As shown, the light irradiated from the channel lighting unit LSiC in the central part irradiates the reflector 47 and is reflected to the bottom. In this case, the lighting device LS is composed of a total of 13 channel lighting units LSi. When each channel lighting unit LSi emits three-color light, the predetermined lighting light emission pattern of a single color and a single channel is 13×3=39 patterns. Furthermore, the data of the image generated by the shooting of the camera 102 is sent to the processing device 100. The shooting based on the camera 102 is performed periodically or event-based.

[0049] In addition, the lighting device LS may be a movable lighting device in addition to a multi-channel lighting device. The movable lighting device is a lighting device in which the lighting unit is separated from the camera 102 and the lighting direction of the workpiece 4 can be changed by mechanically moving the position of the lighting unit. The lighting direction when using the movable lighting device is also an example of the lighting condition.

[0050] The lighting conditions are, for example, (1) the spatial distribution of the light intensity and / or light color in the lighting device, or (2) the light position determined by the position of the lighting device in the movable lighting device. It may also be a combination of (1) and (2).

[0051] The processing device 100 outputs the lighting command value to the lighting device LS, and causes the lighting device LS to emit light in a light emission pattern corresponding to the lighting command value.

[0052] The processing device 100 instructs the camera 102 to perform shooting and input the captured images. The timing of instructing to perform shooting may be the timing when the sensor (not shown) detects that the workpiece 4 has reached a position suitable for shooting. Instead of instructing the processing device 100 to perform shooting, the processing device 100 may sequentially input images captured by the camera 102 at certain time intervals and select an image to be used from the images input by the processing device 100.

[0053] The processing device 100 may also be connected to a display 104 as an output unit for displaying the processing status, measurement results, etc., and an input unit such as a keyboard 106 and a mouse 108. The processing device 100 is connected to the network 8 and can communicate with other devices via the network 8.

[0054] (1. Hardware structure)

[0055] Next, use Figure 5 , an example of the hardware configuration of the processing device 100 included in the position measurement system 1 according to one embodiment of the present disclosure is described. Figure 5 Schematically shows the hardware configuration of the processing device 100 .

[0056] As an example, the processing device 100 may also be implemented using a general-purpose computer configured according to a general-purpose computer architecture. The processing device 100 includes a processor 110, a main memory 112, a camera interface 114, an input interface 116, a display interface 118, a communication interface 120, and a memory 130. These components are typically connected to each other via an internal bus 122 so as to be communicative.

[0057] The processor 110 implements the functions and processes described below by deploying and executing various programs stored in the memory 130 in the main memory 112. The main memory 112 is composed of a volatile memory, and functions as a working memory required for the processor 110 to execute the programs.

[0058] The camera interface 114 is connected to the camera 102 to acquire an image 138 captured by the camera 102. The camera interface 114 may also instruct the camera 102 on a capture timing and the like.

[0059] The input interface 116 is connected to an input unit such as the keyboard 106 and the mouse 108 , and acquires instructions indicating operations of the input unit by the user.

[0060] The display interface 118 outputs various processing results generated by the processor 110 executing the program to the display 104 .

[0061] The communication interface 120 is responsible for processing for communicating with other devices via the network 8 .

[0062] The memory 130 stores programs such as an OS (Operating System) and a measurement program 132 for making the computer function as the processing device 100. The memory 130 may also store a lighting parameter 134 and a plurality of images 138. The lighting parameter 134 is a parameter indicating lighting conditions. Based on the lighting parameter 134, a lighting command value is generated.

[0063] The measurement program 132 stored in the memory 130 may be installed in the processing device 100 via an optical recording medium such as a DVD (Digital Versatile Disc) or a semiconductor recording medium such as a USB (Universal Serial Bus) memory. Alternatively, the measurement program 132 may be downloaded from a server device on the network.

[0064] When a general-purpose computer is used for implementation, a part of the functions involved in the present embodiment may be implemented by calling necessary software modules among the software modules provided by the OS in a predetermined order and / or at a predetermined time and performing processing. That is, the measurement program 132 involved in the present embodiment may not include all software modules for implementing the functions involved in the present embodiment, but may provide necessary functions by cooperating with the OS.

[0065] In addition, the measurement program 132 may be provided by being embedded in a part of other programs. In this case, the measurement program 132 itself does not include the modules included in the other programs combined as described above, but cooperates with the other programs to perform processing. In this way, the measurement program 132 involved in this embodiment may also be provided by being embedded in other programs.

[0066] in addition, Figure 5 An example of using a general-purpose computer to implement the processing device 100 is shown, but the present invention is not limited thereto, and a dedicated circuit (e.g., ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc.) may be used to implement all or part of its functions. Furthermore, an external device connected to a network may be responsible for a part of the processing.

[0067] (2. Functional composition)

[0068] Next, use Figure 6 An example of the functional configuration of the position measurement system 1 according to the embodiment of the present disclosure will be described. Figure 6 The processing device 100 of the position measurement system 1 may include an image processing unit 141, an illumination condition determination unit 142, a measurement control unit 143, a storage unit 144, a determination unit 145, and a virtual image generation unit 146.

[0069] The image processing unit 141, the lighting condition determination unit 142, the measurement control unit 143, the determination unit 145, and the virtual image generation unit 146 in the processing device 100 are implemented by the processor 110 executing the measurement program 132. The processor 110 is a general-purpose processor. It is also possible to use a dedicated circuit (for example, ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc.) to implement all or part of the functions of the processor 110. Furthermore, an external device connected to the network may be responsible for a part of the processing.

[0070] The image processing unit 141 uses the image of the workpiece 4 to obtain the relative position of the three-dimensional morphological elements of the workpiece 4 with respect to the workpiece 4 .

[0071] The lighting condition determination unit 142 determines the appropriate lighting condition of the lighting device LS so as to reduce the change in relative position obtained by the image processing unit 141 due to the change in the positional relationship between the camera 102 and the workpiece 4 when capturing an image of the workpiece 4.

[0072] The measurement control unit 143 causes the camera 102 to image the workpiece 4 under appropriate lighting conditions, and causes the image processing unit 141 to obtain relative positions of three-dimensional morphological elements of the workpiece 4 with respect to the workpiece using the image of the imaged workpiece 4 .

[0073] The storage unit 144 stores the image processed by the image processing unit 141. In addition, the storage unit 144 stores the lighting parameters indicating the appropriate lighting conditions determined by the lighting condition determination unit 142. The storage unit 144 is implemented by the above-mentioned memory 130 that stores the program or data required for the operation of the position measurement system 1. In addition, the processing device 100 may not include the storage unit 144, and may use an external (device) memory instead of the storage unit 144.

[0074] The determination unit 145 determines the deviation of the relative position from the reference position and determines whether the relative position determined by the image processing unit 141 is good or not based on the deviation. If the position measurement system 1 determines the relative position but does not determine whether the relative position is good or not, the determination unit 145 may not be included.

[0075] The virtual image generation unit 146 generates a virtual image of the workpiece that should be obtained when the workpiece is photographed under a lighting condition different from the lighting condition used for the photographing, based on the images of the workpiece photographed under a plurality of lighting conditions. The virtual image can be used to optimize the lighting condition. When the lighting condition is optimized based only on the images of the workpiece actually photographed, the position measurement system 1 may not include the virtual image generation unit 146.

[0076] The position measurement system 1 can also be used as a lighting condition optimization system that determines an appropriate lighting condition. In this case, the measurement control unit 143 may not be included.

[0077] The position measurement system 1 can also be used as a position measurement system that performs measurement under an appropriate lighting condition obtained from the outside by partially changing the structure. In this case, the lighting condition determination unit 142 is replaced with a lighting condition determination unit that obtains a predetermined appropriate lighting condition of the lighting device LS that reduces the change in the relative position obtained by the image processing unit 141 due to the change in the positional relationship between the camera 102 and the workpiece 4 when the image of the workpiece is captured.

[0078] (3. Action example)

[0079] In the following description of the operation example, the meanings of the symbols used in the mathematical formulas are shown in Tables 1 to 3. Bold lowercase letters represent vectors, and bold uppercase letters represent matrices (indicated by non-bold in the text). Other symbols are scalars. In addition, ________ represents the L2 norm with respect to a vector.

[0080] [Table 1]

[0081]

[0082] [Table 2]

[0083] (When using multi-channel lighting)

[0084]

[0085] [Table 3]

[0086] (When using movable lighting)

[0087] significance symbol Number of lighting fixture positioning positions Z Image number m taken at position z gz,m(1≤z≤Z,1≤m≤M) The final position number to be found z

[0088] (Lighting condition determination stage and measurement stage)

[0089] Figure 7 1 is a diagram showing the operation stages of the position measurement system 1. Figure 7 As shown, the operation of the position measurement system 1 is divided into a lighting condition determination phase and a measurement phase.

[0090] In the lighting condition determination phase, the position measurement system 1 operates as follows.

[0091] The image processing unit 141 obtains the relative position of the three-dimensional morphological elements of the workpiece 4 with respect to the workpiece 4 for each image of the workpiece 4 captured under a plurality of lighting conditions.

[0092] The lighting condition determination unit 142 determines the appropriate lighting condition of the lighting device LS so as to reduce the change in relative position obtained by the image processing unit 141 due to the change in the positional relationship between the camera 102 and the workpiece 4 when capturing an image of the workpiece 4.

[0093] The setting of the lighting conditions of the lighting device LS and the control of the shooting of the camera 102 in the lighting condition determination stage can be performed by manual operation or by automatic operation programmed in the processing device 100.

[0094] In the measurement phase, the position measurement system 1 operates as follows.

[0095] The measurement control unit 143 causes the camera 102 to image the workpiece 4 under appropriate lighting conditions, and causes the image processing unit 141 to obtain relative positions of three-dimensional morphological elements of the workpiece 4 with respect to the workpiece using the image of the imaged workpiece 4 .

[0096] When determining whether the relative position is good or not, the determination unit 145 obtains the deviation of the relative position from the reference position, and determines whether the relative position is good or not based on the deviation. The relative position is obtained by the image processing unit. Here, the reference position may be a reference position in design or a statistical value (e.g., an average value) of a plurality of measured relative positions.

[0097] (Factors of changes in position measurement values)

[0098] Figure 8 1 is a diagram showing a substrate on which a component is mounted. The component is an example of a three-dimensional morphological element, and the substrate is an example of a workpiece 4. Substrate position reference marks A, B, and C are printed on the substrate. Substrate position reference mark A indicates the origin of the substrate coordinate system. m (0) The direction connecting the substrate position reference mark A and the substrate position reference mark B is the X direction in the substrate coordinate system. The direction connecting the substrate position reference mark A and the substrate position reference mark C is the Y direction in the substrate coordinate system. In the captured image, the intersection of the left side of the component and the search line set by the position measurement system 1 is the measurement position r. m (1) .

[0099] Fig. 9It is a diagram for explaining the change of the shaded area generated in the component. Fig. 9 It is observed from the Y direction of the substrate Figure 8 FIG. 4 is a partial view of a component. Due to the deviation of the conveying position of the substrate at the time of shooting, the position of the component at the time of the first shooting is the position of the component illustrated as the component at the first conveying position, and the position of the component at the time of the second shooting is the position of the component illustrated as the component at the second conveying position. Due to the deviation of the conveying position, the direction of the light irradiating the rounded corner on the left side of the component changes in the manner shown in the figure. In contrast, the boundary position between the area irradiated with light and the shadow area not irradiated with light also changes. The boundary position of the shadow area is identified on the image as the edge representing the left side of the component. Therefore, due to the deviation of the conveying position, the measured value of the edge position changes. If in Fig. 9 In the example above, if light is irradiated from the upper left direction to the left side of the component to be measured, and no shadow is formed on the portion to be measured, it is expected that the change in the measured value in this case may be reduced. The change in the measured value of the edge position caused by the change in the positional relationship between the lighting device LS and the camera 102 and the substrate is caused not only by the change in the position of the substrate in the conveying direction, but also by the change in the inclination of the surface of the substrate. In addition, the degree to which the measured value changes also depends on the state of the component, such as the shape of the component, whether the component has light transmittance, whether the surface of the component has specular reflectivity, and the relationship between the color of the component and the color of the illumination light.

[0100] (mark)

[0101] Hereinafter, a three-dimensional morphological element whose position is to be measured on an image is described as a mark, and the position of a measurement target point determined corresponding to the three-dimensional morphological element is described as the position of the mark.

[0102] Markers include one-dimensional markers and two-dimensional markers. One-dimensional markers are edges. Edges are parts of an image where the concentration, color, and other attributes of pixels change sharply relative to the surrounding area (a collection of pixels) and are distributed in a linear manner. In order to determine the position of a one-dimensional marker, a search line is used. The position of a one-dimensional marker is the position of the edge on the search line that intersects the edge.

[0103] The two-dimensional mark is a figure. The position of the two-dimensional mark is the position of a specific feature point of the figure. The feature point of the figure is, for example, the centroid of a closed curve figure, the center of a circle or arc, the intersection of two lines, the angle of a broken line, and the end point of a line segment. The position of the two-dimensional mark is measured with the feature point of the selected type as the object.

[0104] In the following description, terms such as "optimization" of lighting conditions and "minimization" of the variance of measured values ​​are used to match the contents of mathematical formulas. However, even if they are not optimal, it is sufficient to determine the lighting conditions or reduce the variance of the measured values ​​within an appropriate range that does not pose a practical obstacle. Achieving "optimization" that only means achieving the best point is not the main purpose of the invention.

[0105] The following is a general procedure for measuring the position of a marker with reference to an image g that is captured or created as a virtual image. Consider the situation where the coordinates r of a marker are determined from the image g. In the problem setting of position measurement, it is assumed that the positional relationship between the workpiece and the camera varies with each transport, so measurement must be performed in the workpiece coordinate system rather than the camera coordinate system. For example, the workpiece position in the camera coordinate system is measured.

[0106] [Mathematical formula 1]

[0107] The unit also exists. If there is a translation deviation in the position of the workpiece,

[0108] [Mathematical formula 2]

[0109] The processing is transformed into the workpiece coordinate system.

[0110] [Mathematical formula 1]

[0111]

[0112] It may also be the position of the workpiece detected from the image g. The coordinate r of the mark may be expressed by a workpiece coordinate system of an orthogonal coordinate system or a workpiece coordinate system of a polar coordinate system.

[0113] The marker position is determined as the position where the predetermined marker likelihood function L is maximized in the search area E (or where the likelihood first exceeds the threshold in a predetermined coordinate search order) using the entire or a part of the image g as the search area E.

[0114] [Mathematical formula 3]

[0115]

[0116] Here, θ is a parameter vector for marker detection. The parameter vector may include values ​​that specify the algorithm for obtaining the likelihood function L, in addition to the values ​​used in the algorithm for obtaining the likelihood function L. For example, a threshold value for the image density on the search line may be set, including a parameter for an algorithm that specifies the position of a point where the image density exceeds the threshold from the bottom to the top as the edge position, or the position of a point where the image density exceeds the threshold from the top to the bottom as the edge position.

[0117] The problem setting of position measurement is that when the same workpiece is used as the measurement object, even if there is a position deviation (transport deviation) of the workpiece relative to the camera, the measured value of the relative position of the three-dimensional morphological element relative to the workpiece will not be biased. In the case of a transport deviation for the same workpiece, for the M workpiece images g captured m (1≤m≤M), the lighting conditions are determined so as to minimize the variance J of the measured values ​​at the positions defined below. At this time, the mark detection parameter θ may be determined at the same time, or the mark detection parameter θ may be a fixed value.

[0118] [Formula 4]

[0119]

[0120] in

[0121]

[0122] (Changes in measurement question settings)

[0123] The variance J of the measured values ​​can also be shown in Table 4.

[0124] [Table 4]

[0125]

[0126] In Table 4, the position and the posture are shown separately, but generally in this specification, the position of a mark (three-dimensional morphological element) also includes the posture of the mark, and the position of a workpiece also includes the posture of the workpiece.

[0127] (Optimization of lighting conditions)

[0128] Consider a problem setting of selecting an appropriate condition from a plurality of candidate lighting conditions. The candidate lighting conditions are given, for example, as follows.

[0129] When the lighting device is a multi-channel (CH) lighting, only Z patterns of the light emission amount of each lighting channel (CH) are prepared, and a pattern suitable for measurement is selected from them.

[0130] When the lighting device is a movable lighting device (=the position or posture of the lighting device can be controlled), when z positions or postures of the lighting device are prepared, a position or posture suitable for measurement is selected from them.

[0131] In this embodiment, conveyance deviation is simulated under each selectable lighting condition, and multiple shots are taken with the workpiece in different positions relative to the camera, and the lighting condition is selected so that the deviation of the position measurement results obtained from each resulting image is minimized.

[0132] Specifically, in an actual inspection environment, M images are captured while conveying the same workpiece under one lighting condition, repeatedly under each of Z lighting conditions. One of the ZM images captured in this way is denoted as g. z，m .

[0133] The best lighting condition z should be found opt The condition for minimizing the deviation J of the measurement position is obtained as follows.

[0134] [Mathematical formula 5]

[0135]

[0136] The optimization problem of the above formula is solved by any optimization method to obtain the best lighting conditions.

[0137] The optimal marker detection parameter θ can also be obtained at the same time opt The optimization problem of lighting conditions and marker detection parameters can be expressed as follows.

[0138] [Mathematical formula 6]

[0139]

[0140] In the lighting condition determination stage, the user can also pre-specify the area including the mark. This reduces the possibility of mistakenly taking the image features that the user does not expect as the measurement object. This area is hereinafter referred to as the search area E. (p) .

[0141] For one-dimensional labeling, the search line of the edge can be regarded as the search area E (p) Alternatively, the search area E may be set to include the measurement target position on the edge. (p) , in the search area E (p) The search line for the edge is automatically set in . For the search area E of the two-dimensional mark (p) Set to include graphics as 2D markers.

[0142] (Example 1) Determining the position on the one-dimensional coordinate axis based on the one-dimensional edge information

[0143] Fig.10 It is a diagram for explaining the position measurement of a one-dimensional marker.

[0144] An example of defining a label likelihood function is described in detail below.

[0145] As the simplest example, consider the problem of searching for the position of a marker on a search line defined on an image. This is equivalent to the case where the marker search area is a line segment. As a marker, it is necessary to have a structure with a spatial frequency (gradient is not zero) at least in the direction of the search line. For example, it is equivalent to a sharp change in brightness or color. If the contour function of the image on the search line is set to The marker position can be determined by simply giving the contour function. The contour function is a function that collects the surrounding image information as the value on the search line. For example, for each point on the search line, the average value or median value for noise removal is taken for the set of pixel values ​​that pass through the point and are in the direction of the search line. Post-processing such as differentiation or frequency filtering can also be applied as needed.

[0146] In this case, as an example, the label likelihood function may be changed as shown in the following table: Here, the position r in the label likelihood function corresponds to the position t on the contour in a one-to-one manner.

[0147] [Table 5]

[0148]

[0149] (Example 2) Determining the Position on the Two-Dimensional Coordinate Axis Based on the Two-Dimensional Mark

[0150] Fig.11 It is a diagram for explaining the position measurement of a two-dimensional marker.

[0151] As an example of a two-dimensional mark, a predetermined arbitrary figure (e.g., a circle or a rectangle) can be cited. The figure can be a partial image identified by some local feature quantity. However, it is necessary that there is a spatial frequency in the dimensional direction of all coordinates to be measured, that is, the Hessian matrix is ​​not degenerate. The position r is a representative position of the figure based on a certain determination method, for example, represented by xy coordinates. Fig.11 The figure is a square, and the representative position is the center of the square. As an example of the label likelihood function in this case, there are changes shown in the following table.

[0152] [Table 6]

[0153]

[0154] As the feature quantity for specifying the marker coordinates, any local feature quantity proposed so far, such as SIFT, SURF, HoG, Haar-Like, etc., can be used. (p) The operation of determining the coordinates to be marked is similar to the search for key points in feature quantities such as SIFT, SURF, and HoG.

[0155] (Example 3) Determining the Position on the Two-Dimensional Coordinate Axis Based on Multiple One-Dimensional Edge Information

[0156] Fig.12 It is a diagram for explaining the measurement of a position specified by a plurality of one-dimensional markers.

[0157] In the case of (Example 1), the marker coordinates are determined on a predetermined search line on the image, but if Fig.12 By determining the directions of two non-parallel linear edges as in (A), the coordinates of the intersection of the edges can be determined. In order to ensure robustness against noise, it is often practical to integrate a large amount of one-dimensional label coordinate information.

[0158] For example, in the application of determining the position of a circular hole on a substrate or the position of a quadrilateral component, Fig.12 (B) like on a circle or like Fig.12 (C) As in the method described above, search lines are arranged at small intervals on the sides of a rectangle in a manner orthogonal to the edge direction, edge coordinates (= one-dimensional marker coordinates) are detected on each search line, outliers are removed from the edge position information, and least squares fitting is performed to determine the position of the measurement object. When such a method is used, the indication of the search area can be specified as an interval shape between double lines surrounding the contour part of the shape to be detected, and line segments, i.e., search lines, with the double lines as their respective terminals, are set between the double lines.

[0159] (Example 4) Optimization including individual deviations

[0160] In addition to the conveying deviation of the workpiece, the individual deviation can also be added to optimize the lighting conditions. Here, the value regarded as the true value of the individual deviation (hereinafter referred to as the "true value") can be measured by other measuring units. The M workpiece sample set is divided into a sample set C with Q conveying deviations of each individual workpiece q , for each workpiece, the true value of the measured value

[0161] [Mathematical formula 7]

[0162]

[0163] It can be measured by other measuring means. At this time, since the deviation from the true value for each workpiece individual needs to be minimized simultaneously, the following evaluation criterion J is minimized.

[0164] [Mathematical formula 8]

[0165]

[0166] As described above, optimization is performed in such a manner as to approach respective true values ​​while including individual deviations of the workpieces.

[0167] (Example 5) Optimization when the true value of individual deviation is unclear

[0168] When the measurement deviation is defined as the variance relative to all workpiece samples including individual / conveying deviations of the workpiece, and assuming that the true value of the dimension to be measured also includes individual deviations, the lighting conditions and inspection parameters that make the individual deviations of the true value of the dimension no longer visible are obtained according to the criterion of minimizing the overall variance.

[0169] In order to avoid this situation, the M workpiece sample sets may be divided into Q sample sets C having the conveying deviation of each individual workpiece. q , the variance of each workpiece is evaluated separately, and finally the value after adding up the variances calculated for all workpieces is defined as the variance to be minimized. At this time, the evaluation criterion J to be minimized is defined as follows.

[0170] [Mathematical formula 9]

[0171]

[0172] As described above, even when the measured true value varies due to individual variations in the workpieces, it is possible to minimize only the conveyance variations.

[0173] (Example 6) Inspection of simultaneous measurement of multiple markers

[0174] In the description so far, the problem setting is handled in which the position of one mark needs to be determined in order to obtain a measurement value. However, there are also situations in which multiple mark positions must be measured in order to obtain one measurement value. For example, the measurement amount is sometimes defined as the distance between marks.

[0175] As a specific example of measuring the distance between two marks affected by lighting conditions in position measurement, in the case of measuring the relative position of an LED element mounted on a substrate as a workpiece relative to the substrate, the feature used as the reference for the position of the substrate is also a three-dimensional (mark) case that is affected by lighting conditions, such as a hole with a rounded cross-sectional shape at the edge. In addition, as another specific example, in the case of two LED elements mounted on a substrate, it is more important to stably measure the distance between the two LED elements than the relative position of each LED element relative to the substrate. This situation occurs when two LED elements are inserted into two holes set in other components at a predetermined interval. In this case, the invention can be applied by regarding the position of one LED element as the reference for the position of the substrate as a workpiece and the position of another LED element as the position of a three-dimensional morphological element. In addition, one LED element itself can also be regarded as a workpiece. In this case, the other LED element as a three-dimensional morphological element is regarded as being indirectly arranged on the one LED element as a workpiece via the substrate.

[0176] Furthermore, when measuring the positions of a plurality of marks, the lighting conditions may be optimized for each mark, and when imaging a specific mark, the mark may be illuminated under the optimized lighting conditions.

[0177] Consider determining the coordinates r of P markers from an image g captured or produced as a virtual image (p) The workpiece coordinate system is set in the image g. For each mark, from the predetermined search area E (p) In the process, a predetermined marker likelihood function L is maximized (or the likelihood first exceeds a threshold in a predetermined coordinate search order) to determine each marker coordinate.

[0178] [Formula 10]

[0179]

[0180] Here, θ (p) is a parameter for detecting the pth mark. For the M workpiece images g captured when there is a conveying deviation of the workpiece m (1≤m≤M), the lighting conditions are determined so as to minimize the variance J of the measured values ​​defined in Table 7. At this time, the mark detection parameters may be determined at the same time, or the mark detection parameters may be fixed values.

[0181] [Table 7]

[0182]

[0183] As an example, let's deal with the problem of measuring the distance between markers. In the case of distance measurement, at least two markers are required, so the following explanation assumes that P = 2. Using the marker position coordinates obtained in the same way as in the case of position measurement, the measured value of the dimension is obtained as |r (2) -r (1) |. Assuming that the object to be measured (e.g., point A and point B) is not completely within the camera's field of view, a mark for alignment is introduced, and point A (p=1) and the mark for alignment (position r (3) ), and point B (p = 2) and the alignment mark (position r (4) ). In the absence of rotational offset, the dimension is measured as |(r (2) -r (4) )-(r (1) -r (3) )| and get it.

[0184] For the M workpiece images g captured under the condition of workpiece conveyance deviation, m (1≤m≤M), the lighting conditions are determined so as to minimize the variance J of the measured values ​​of the dimensions defined below. At this time, the mark detection parameter θ may be determined at the same time, or the mark detection parameter θ may be a fixed value.

[0185] [Mathematical formula 11]

[0186]

[0187] in

[0188]

[0189] (Example 7) Determination of Optimal Lighting Conditions for Position Inspection Based on Lighting Simulation

[0190] Here, a method for optimizing lighting conditions is described when there are so many lighting conditions that it is difficult to capture all lighting conditions in advance. By using the captured images when the lighting conditions are changed, the lighting simulation can be used to estimate with high accuracy, thereby automating the optimization work.

[0191] It is assumed that the linearity of the sensor in the shooting system (linearity between brightness and pixel value) holds. That is, this means that linearity holds in the system including all shooting corrections such as signal processing such as de-mosaicing of the color filter, gamma correction, dark current offset correction, etc. In order to avoid nonlinearity caused by pixel saturation, HDR (High dynamic range) synthesis can also be used in the shooting of the image. At this time, consider the situation of shooting a workpiece illuminated by K multi-channel (CH) lighting, and mark the luminous intensity of each lighting as the following vector.

[0192] u=(u1,u2,…,u k ) T ,u k ≥0

[0193] At this time, if u i =1,u j = 0 (j≠i), let f be an image in which only the i-th illumination is lit at a predetermined intensity and the others are turned off and the image is captured as a column vector arrangement. i , then the captured image g under any lighting condition u can be modeled by image superposition as follows.

[0194] [Mathematical formula 12]

[0195]

[0196] Here, f' is a multi-channel (CH) image (a large column vector) obtained by vertically arranging K images defined below.

[0197] [Mathematical formula 13]

[0198]

[0199] In addition, in the above formula

[0200] [Mathematical formula 14]

[0201]

[0202] is the Kronecker product, e i express

[0203] [Mathematical formula 15]

[0204] The standard basis of , I represents the identity matrix.

[0205] Thus, the optimization of multi-channel (CH) illumination is equivalent to the generation of the eigenvector g based on the inner product operation from the original image f', that is, the projection matrix

[0206] [Formula 16]

[0207]

[0208] The optimal design of the linear projection direction u in .

[0209] In the following, it is generally considered that, in a situation where K lighting fixtures are installed, the lighting emission pattern is changed for one workpiece, and N patterns are photographed to determine the optimal lighting fixture.

[0210] The light emission pattern in the nth (1≤n≤N) shot is defined as follows. Fig.13 A specific example of the light emission pattern is shown.

[0211] [Mathematical formula 17]

[0212] h n =(h 1，n ,h 2,n ,…,h K,n ) T ,h k,n ≥0

[0213] H=[h1,h2,…,h N ]

[0214] By photographing the workpiece using these light patterns, it is possible to reconstruct the photographed image of the workpiece illuminated by any light pattern. In order to obtain all degrees of freedom of the system matrix, it is desirable to determine the light pattern so that rank H = min (N, K), that is, linear independence. In order to effectively utilize all K degrees of freedom of illumination without any surplus, at least N = K, in this case, H needs to be full rank.

[0215] On the other hand, if N < K, the degree of freedom of the number of lightings cannot be fully and effectively utilized, but it is an effective method to shorten the shooting time of the workpiece. As a method for achieving this, various methods are considered, and as a representative method, a method of estimating the LT (Light Transport) matrix using compressed sensing (Compressed Sensing) can be cited.

[0216] In addition, a case where N>K is considered, which is more than necessary in terms of fully and effectively utilizing the degrees of freedom of illumination and results in a useless number of shots, but can be selected to obtain other purposes such as SN ratio and dynamic range.

[0217] Let the vector in which the images captured with these light patterns are arranged as column vectors be f i , a multi-channel (CH) image vector is defined as follows, which is a combination of N images arranged vertically.

[0218] [Mathematical formula 18]

[0219]

[0220] In the above formula

[0221] [Mathematical formula 14]

[0222]

[0223] is the Kronecker product, ei express

[0224] [Mathematical formula 15]

[0225] The brightness u based on each illumination channel (CH) and the captured image g at this time are expressed as follows.

[0226] [Mathematical formula 19]

[0227] u=Hw,w=(w1,w2,…,w N ) T

[0228]

[0229] Here, w i is the image superposition weight. (The command value of the luminous intensity is calculated based on this, but not the value itself) This is generally allowed to take negative values, but since the luminous intensity of the illumination channel (CH) must be non-negative, the following conditions are set.

[0230] Hw≥0

[0231] Here, the inequality sign for vectors and matrices applies to all elements. H = I, that is, when K (= N) images are taken by turning on the lighting one by one, u and w are equivalent. Next, the problem of finding the optimal lighting u at this time is solved by finding the optimal image superposition weight w.

[0232] If lighting simulation is used, conveyance deviations of workpieces that actually occur are recorded in advance as images, and by performing closed work only on a computer, lighting conditions can be optimized for these samples so as to minimize deviations in measurement results.

[0233] In order to obtain images taken under conditions of workpiece conveying deviation, in an actual inspection environment, the same workpiece is conveyed under one lighting condition while being photographed, and a base image set consisting of M images for lighting simulation is obtained. Hereinafter, the M base image set is labeled as f' using the above-mentioned notation. m (1≤m≤M).

[0234] When the image superposition weight corresponding to the lighting condition is set to w, the

[0235] [Mathematical formula 20]

[0236] Replace the captured imageg m(1≤m≤M), the variance of the position measurement value can be evaluated only on a computer. The optimization problem of finding the image superposition weight corresponding to the lighting condition most suitable for position measurement can be expressed as follows.

[0237] [Mathematical formula 21]

[0238]

[0239] When the P marker detection parameters are also optimized, this can be expressed as follows.

[0240] [Mathematical formula 22]

[0241]

[0242] The exposure and gain of the camera are also adjustment factors, but the adjustment state of the exposure and gain does not affect the method of generating the deviation corresponding to the position of the workpiece (transport deviation) of the measurement result of the relative position of the three-dimensional morphological element with respect to the workpiece. In the lighting condition determination stage and the measurement stage, the exposure and gain are determined within the range in which the captured image does not become too bright or too dark.

[0243] Above, the embodiments of the present invention have been described in detail, and the above description is only an illustration of the present invention in all aspects. Of course, various improvements and deformations can be made without departing from the scope of the present invention. In addition, part or all of the above embodiments can also be recorded as the following notes, but are not limited to the following.

[0244] (Note 1)

[0245] A position measuring system (1) is provided for measuring the relative position of a workpiece (4) and a three-dimensional morphological element of the workpiece (4), the relative position of the morphological element relative to the workpiece (4). The position measuring system (1) comprises:

[0246] A photographing device (102) for photographing the workpiece (4);

[0247] an illumination device (LS), used as a light source for the imaging device (102) to perform imaging, and capable of changing the illumination conditions for the workpiece (4);

[0248] An image processing unit (141) calculates the relative position using the image of the workpiece (4);

[0249] a lighting condition determination unit (142) for determining an appropriate lighting condition for the lighting device (LS), wherein the appropriate lighting condition reduces a change in the relative position obtained by the image processing unit (141) due to a change in the positional relationship between the imaging device (102) and the workpiece (4) when capturing an image of the workpiece (4); and

[0250] The measurement control unit (143) causes the imaging device (102) to image the workpiece (4) under the appropriate lighting condition, and causes the image processing unit (141) to calculate the relative position using the image of the workpiece (4) thus imaged.

[0251] (Note 2)

[0252] A position measuring system (1) according to Supplementary Note 1, wherein:

[0253] The morphological element is a one-dimensional morphological element,

[0254] The image processing unit (141) determines the relative position by taking the intersection point between the search line and the one-dimensional morphological element as the position of the morphological element.

[0255] (Note 3)

[0256] A position measuring system (1) according to Supplementary Note 1, wherein:

[0257] The morphological elements are two-dimensional morphological elements,

[0258] The image processing unit (141) determines the relative position by taking the representative position of the two-dimensional morphological element determined in the search area as the position of the morphological element.

[0259] (Note 4)

[0260] A position measuring system (1) according to Supplementary Note 1, wherein:

[0261] The morphological elements are a plurality of one-dimensional morphological elements,

[0262] The image processing unit (141) obtains the position of the intersection of a search line among multiple search lines and a morphological element among the multiple one-dimensional morphological elements for multiple groups of one search line and one morphological element, and uses the representative position obtained based on the obtained positions of the multiple intersection points as the position of the multiple one-dimensional morphological elements to obtain the relative position.

[0263] (Note 5)

[0264] A position measuring system (1) according to Supplementary Note 1, wherein:

[0265] The suitable lighting condition determined by the lighting condition determination unit (142) is a lighting condition in which the image used in determining the suitable lighting condition is an image of a plurality of the workpieces (4), and includes a plurality of images of one workpiece (4) having different positional relationships between the photographing device (102) and the workpiece (4), and in which, for each workpiece (4), a value regarded as the true value of the relative position can be utilized, so that the relative position of each workpiece (4) changes from the value regarded as the true value, and the total of the relative position of the plurality of workpieces (4) becomes smaller.

[0266] (Note 6)

[0267] A position measuring system (1) according to Supplementary Note 1, wherein:

[0268] The suitable lighting condition determined by the lighting condition determination unit (142) is a lighting condition in which, when the image used in determining the suitable lighting condition is an image of a plurality of the workpieces (4), and when the image includes a plurality of images of one workpiece (4) having a positional relationship between the photographing device (102) and the workpiece (4) that is different from each other, the total relative position of each workpiece (4) is reduced.

[0269] (Note 7)

[0270] A position measuring system (1) according to Supplementary Note 1, wherein:

[0271] When the workpiece (4) has a plurality of the three-dimensional morphological elements, the lighting condition determination unit (142) determines the appropriate lighting condition for each of the three-dimensional morphological elements.

[0272] The image processing unit (141) obtains the relative position of each of the plurality of three-dimensional morphological elements possessed by the workpiece (4) using an image of the workpiece (4) photographed under the appropriate lighting conditions corresponding to the three-dimensional morphological element to be processed.

[0273] (Note 8)

[0274] The position measuring system (1) according to any one of Supplementary Notes 1 to 7, wherein:

[0275] The position measurement system (1) further comprises a virtual image generation unit (146) for generating, based on images of the workpiece (4) captured under a plurality of lighting conditions, a virtual image of the workpiece (4) that would be obtained if the workpiece (4) were captured under lighting conditions different from lighting conditions used for the capturing.

[0276] The image processing unit (141) also obtains the relative position when the image of the workpiece (4) is the virtual image.

[0277] The lighting condition determination unit (142) determines the appropriate lighting condition so as to reduce a change in the relative position obtained for the virtual image or for the captured image of the workpiece and the virtual image.

[0278] (Note 9)

[0279] A position measuring system (1) according to Supplementary Note 8, wherein:

[0280] The position measurement system (1) further comprises a determination unit (145) for determining a deviation of the relative position from a reference position and determining whether the relative position is good or not based on the deviation, wherein the relative position is determined by the image processing unit (141).

[0281] (Note 10)

[0282] The position measuring system (1) according to any one of Supplementary Notes 1 to 7, wherein:

[0283] The position measurement system (1) further comprises a determination unit (145) for determining a deviation of the relative position from a reference position and determining whether the relative position is good or not based on the deviation, wherein the relative position is determined by the image processing unit (141).

[0284] (Note 11)

[0285] A lighting condition optimization system (1) takes a workpiece (4) and a three-dimensional morphological element possessed by the workpiece (4) as objects, and optimizes lighting conditions for lighting when photographing the workpiece (4) by measuring the relative position of the morphological element relative to the workpiece (4). The lighting condition optimization system (1) comprises:

[0286] A photographing device (102) for photographing the workpiece (4);

[0287] an illumination device (LS), used as a light source for the imaging device (102) to perform imaging, and capable of changing the illumination conditions for the workpiece (4);

[0288] An image processing unit (141) obtains the relative position using the image of the workpiece (4); and

[0289] A lighting condition determination unit (142) determines suitable lighting conditions for the lighting device (LS), wherein the suitable lighting conditions reduce the change in the relative position obtained by the image processing unit (141) due to the change in the positional relationship between the imaging device (102) and the workpiece (4) when capturing an image of the workpiece (4).

[0290] (Note 12)

[0291] A position measuring system is provided, which takes a workpiece (4) and a three-dimensional morphological element possessed by the workpiece (4) as objects, and measures the relative position of the morphological element with respect to the workpiece (4). The position measuring system (1) comprises:

[0292] A photographing device (102) for photographing the workpiece (4);

[0293] an illumination device (LS), used as a light source for the imaging device (102) to perform imaging, and capable of changing the illumination conditions for the workpiece (4);

[0294] An image processing unit (141) calculates the relative position using the image of the workpiece (4);

[0295] a lighting condition determination unit (142) for acquiring a predetermined suitable lighting condition of the lighting device (LS), wherein the suitable lighting condition reduces the change in the relative position obtained by the image processing unit (141) due to the change in the positional relationship between the imaging device (102) and the workpiece (4) when capturing an image of the workpiece (4); and

[0296] The measurement control unit (143) causes the imaging device (102) to image the workpiece (4) under the appropriate lighting condition, and causes the image processing unit (141) to calculate the relative position using the image of the workpiece (4) thus imaged.

[0297] Description of Reference Numerals

[0298] 1…position measurement system, 2…belt conveyor, 4…workpiece, 6…shooting field of view, 8…host network, 10…PLC, 12…database device, 100…processing device, 102…camera, 104…display, 106…keyboard, 108…mouse, 110…processor, 112…main memory, 114…camera interface, 116…input interface, 118…display interface, 120…communication interface, 122…internal bus, 130…memory, 132…measurement program, 134…illumination parameter, 138…image, 141…image processing unit, 142…illumination condition determination unit, 143…measurement control unit, 144…storage unit, 145…judgment unit, 146…virtual image generation unit, LS…illumination device, LSi…channel illumination unit.

Claims

1. A position measuring system for measuring a relative position of a workpiece and a three-dimensional morphological element of the workpiece, the position measuring system comprising: A photographing device for photographing the workpiece; an illumination device, used as a light source for the photographing device to take pictures, and capable of changing the illumination conditions for the workpiece; An image processing unit that uses the image of the workpiece to obtain the relative position; a lighting condition determination unit that determines an appropriate lighting condition for the lighting device, wherein the appropriate lighting condition reduces a change in the relative position obtained by the image processing unit due to a change in the positional relationship between the imaging device and the workpiece when the image of the workpiece is captured; as well as The measurement control unit causes the imaging device to image the workpiece under the appropriate lighting condition, and causes the image processing unit to obtain the relative position using the image of the workpiece thus imaged.

2. The position measurement system according to claim 1, wherein: The morphological element is a one-dimensional morphological element, The image processing unit determines the relative position by using an intersection point between a search line and the one-dimensional morphological element as a position of the morphological element.

3. The position measurement system according to claim 1, wherein: The morphological elements are two-dimensional morphological elements, The image processing unit obtains the relative position by using the representative position of the two-dimensional morphological element determined in the search area as the position of the morphological element.

4. The position measurement system according to claim 1, wherein: The morphological elements are a plurality of one-dimensional morphological elements, The image processing unit calculates the position of the intersection of a search line among multiple search lines and a morphological element among the multiple one-dimensional morphological elements for multiple groups of one search line and one morphological element, and uses the representative position calculated based on the calculated positions of the multiple intersection points as the position of the multiple one-dimensional morphological elements to calculate the relative position.

5. The position measurement system according to claim 1, wherein: The suitable lighting condition determined by the lighting condition determination unit is a lighting condition in which the image used in determining the suitable lighting condition is an image of a plurality of the workpieces, and includes a plurality of images of one workpiece having mutually different positional relationships between the photographing device and the workpiece, and in which a value regarded as the true value of the relative position can be utilized for each of the workpieces, so that the total of the relative positions for the plurality of workpieces becomes smaller due to the change in the relative positions for each of the workpieces from the value regarded as the true value.

6. The position measurement system according to claim 1, wherein: The suitable lighting condition determined by the lighting condition determination unit is a lighting condition in which, when the image used in determining the suitable lighting condition is an image of a plurality of the workpieces and includes a plurality of images having mutually different positional relationships between the photographing device and the workpiece for one workpiece, the total change in the relative position of each of the workpieces is reduced.

7. The position measurement system according to claim 1, wherein: When the workpiece has a plurality of the three-dimensional morphological elements, the lighting condition determination unit determines the appropriate lighting condition for each of the three-dimensional morphological elements. The image processing unit obtains the relative position of each of the plurality of three-dimensional morphological elements included in the workpiece using an image of the workpiece captured under the appropriate lighting condition corresponding to the three-dimensional morphological element to be processed.

8. The position measurement system according to any one of claims 1 to 7, wherein: The position measurement system further includes a virtual image generating unit that generates, based on the images of the workpiece captured under a plurality of lighting conditions, a virtual image of the workpiece that would be obtained if the workpiece was captured under lighting conditions different from lighting conditions used for capturing the workpiece. The image processing unit also obtains the relative position when the image of the workpiece is the virtual image. The lighting condition determination unit determines the appropriate lighting condition that reduces a change in the relative position obtained for the virtual image or for the captured image of the workpiece and the virtual image.

9. The position measurement system according to claim 8, wherein: The position measurement system further includes a determination unit that determines a deviation amount of the relative position from a reference position and determines whether the relative position is good or not based on the deviation amount, the relative position being determined by the image processing unit.

10. The position measurement system according to any one of claims 1 to 7, wherein: The position measurement system further includes a determination unit that determines a deviation amount of the relative position from a reference position and determines whether the relative position is good or not based on the deviation amount, the relative position being determined by the image processing unit.

11. A lighting condition optimization system, which optimizes lighting conditions for photographing a workpiece and a three-dimensional morphological element of the workpiece as objects, and comprises: A photographing device for photographing the workpiece; an illumination device, used as a light source for the photographing device to take pictures, and capable of changing the illumination conditions for the workpiece; An image processing unit that uses the image of the workpiece to obtain the relative position; as well as The lighting condition determination unit determines an appropriate lighting condition for the lighting device, wherein the appropriate lighting condition reduces the change in the relative position obtained by the image processing unit due to the change in the positional relationship between the imaging device and the workpiece when capturing an image of the workpiece.

12. A position measuring system for measuring a relative position of a workpiece and a three-dimensional morphological element of the workpiece, the position measuring system comprising: A photographing device for photographing the workpiece; an illumination device, used as a light source for the photographing device to take pictures, and capable of changing the illumination conditions for the workpiece; An image processing unit that uses the image of the workpiece to obtain the relative position; a lighting condition determination unit for acquiring a predetermined suitable lighting condition of the lighting device, wherein the suitable lighting condition reduces a change in the relative position obtained by the image processing unit due to a change in the positional relationship between the imaging device and the workpiece when the image of the workpiece is captured; as well as The measurement control unit causes the imaging device to image the workpiece under the appropriate lighting condition, and causes the image processing unit to obtain the relative position using the image of the workpiece thus imaged.

Images