Measurement system, inspection system, measurement device, measurement method, inspection method, and program

Abstract

To provide a technique to compose pieces of data to be inspected that are measured by different imaging systems according to a plurality of measurement principles to allow highly accurate shape measurement.SOLUTION: A measuring system has: first feature point data generation means that generates first feature point data from picked-up first image data including a predetermined part of a measurement object or first shape data generated based on the first image data; second feature point data generation means that generates second feature point data from second image data different from the first image data or second shape data; and calculation means that, based on the first feature point data and the second feature point data, calculates the correspondence relationship of the position of the predetermined part of the measurement object between the first image data and the second image data or between the first shape data and the second shape data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a measurement system, an inspection system, a measurement device, a measurement method, an inspection method, and a program. [Background technology]

[0002] Conventionally, measurements and inspections using images of substrates have been performed in the manufacturing process of various types of substrates. As a measurement device for this purpose, for example, a technology is known that is equipped with multiple different types of measurement principles within a single device, and selects and combines the optimal information from the measurement results of each measurement principle according to the characteristics of the measurement point, thereby improving the accuracy of shape measurement (for example, Patent Document 1).

[0003] In the technology described in Patent Document 1, the substrate appearance inspection device is equipped with two measurement principles: a measurement principle known as the phase shift method and a measurement principle known as the color highlight method, and shape measurement accuracy is improved by adopting the measurement result with the higher reliability of the two types of image data for each pixel.

[0004] Furthermore, in recent years, as various products have become smaller and more precise, the density of components mounted on component-mounted boards has also increased, and as a result, the number of parts that are shaded in the field of view of the imaging device has increased, resulting in an increase in the number of parts that cannot be accurately inspected by visual inspection.In response to this, a technology has become publicly known that uses X-ray CT inspection to inspect parts that cannot be inspected by visual inspection (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-38315 [Patent Document 2] Japanese Patent Application Publication No. 2017-223468 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, the technology described in Patent Document 1 uses multiple measurement principles, but only one imaging system is required to acquire image data for measurement, and the image data for each measurement principle has a corresponding relationship on a pixel-by-pixel basis. This makes it possible to compare the reliability of information on measurement results obtained using different measurement principles on a pixel-by-pixel basis and to create composite data.

[0007] On the other hand, the measurement principle using visible light images such as the phase shift method and color highlight method and the measurement principle using X-rays as in Patent Document 2 (or other measurement principles such as laser scanning) use different imaging systems to image the substrate to be measured, so it is not possible to obtain image data that corresponds to the position of the substrate to be measured.

[0008] In other words, even if the same substrate is imaged, the pixels representing specific positions on the substrate will not correspond to each other in the image data due to differences in the way the substrate is held, differences in the imaging position and angle, and slight deformations that occur in the substrate itself when the substrate is imaged. As a result, as described in Patent Document 1, it has been difficult to select and synthesize optimal information according to the characteristics of the measurement location.

[0009] The present invention has been made in view of the above-mentioned circumstances, and provides a method for detecting a plurality of images using different imaging systems. The object is to provide a technology that enables highly accurate shape measurement by synthesizing data of an inspection object measured according to a measurement principle. [Means for solving the problem]

[0010] In order to achieve the above object, the present invention employs the following configuration.

[0011] A measurement system that measures the shape of at least a part of a measurement object, a first feature data generating means for generating first feature data indicating a shape of a predetermined portion of the measurement object from first image data captured including the shape of at least a portion of the measurement object or first shape data generated based on the first image data; a second feature data generating means for generating second feature data indicating the shape of the predetermined portion of the measurement object from second image data, which includes the shape of at least the part of the measurement object and is acquired through a process different from that of the first image data, or from second shape data generated based on the second image data; and a calculation means for calculating a correspondence relationship between the positions of the predetermined part of the measurement object in the first image data and the second image data, or in the first shape data and the second shape data, based on the first feature point data and the second feature point data.

[0012] Note that "different acquisition processes" as used herein includes cases where images are captured using different imaging means, as well as cases where the timing of image acquisition is different. Furthermore, "calculating correspondence" includes calculating correspondence between pixels representing the same location of a measurement object between multiple images when the scale (magnification) of different image data is different or when the positional relationship of the same location on the image data is different for the same measurement object. Specifically, for example, when the coordinates of pixels representing the same location of a measurement object are different between first image data and second image data having a certain number of pixels (e.g., coordinates (10, 20) in the first image data and coordinates (12, 13) in the second image data), this includes calculating correspondence between the coordinates of these pixels. It also includes calculating correspondence between pixels representing the same location of a measurement object in the first image data and the second image data when only the image size (magnification) is different. In addition, this also includes calculating the correspondence when the resolution (resolution) of the first image data and the second image data is different (for example, when the pixel coordinates of the second image data corresponding to the pixel coordinates (10.3, 20.6) of the first image data are (10, 20)).

[0013] As described above, by having the feature point data generating means and the calculating means, it becomes possible to calculate pixel correspondences between multiple image data acquired by different imaging systems, and to perform processing such as synthesis based on the calculated correspondences.

[0014] The measurement system may further include a first imaging means for imaging the object to be measured and a second imaging means for imaging the object to be measured. Note that the "imaging means" referred to here is not limited to a camera that detects wavelengths in the visible light range, but also includes an X-ray camera that detects X-rays, a photomultiplier sensor used for laser scanning, and the like. With this configuration, image data of the object to be measured can be obtained within the system without obtaining image data from an external source.

[0015] The first imaging means may be a visible light camera, and the second imaging means may be an X-ray camera. With this configuration, it is possible to calculate the correspondence relationship according to the accurate part of the image data having different characteristics.

[0016] The calculation means may also include at least one of an XY calculation unit that calculates the correspondence relationship of positions of the measurement object in the horizontal direction, and a Z calculation unit that calculates the correspondence relationship of positions of the measurement object in the vertical direction.

[0017] In this specification, the direction perpendicular to the horizontal plane of the object to be measured (i.e., the vertical direction) is referred to as the Z axis, the direction perpendicular to this Z axis (i.e., the horizontal direction) that indicates depth is referred to as the Y axis, and the direction that intersects the Y axis at a right angle on the horizontal plane is referred to as the X axis.

[0018] Furthermore, the first feature point data may include any one of coordinate data indicating the shape of the predetermined portion of the measurement object, binary image data indicating the shape of the predetermined portion of the measurement object, multi-valued image data indicating the shape of the predetermined portion of the measurement object, and height inflection point data indicating the three-dimensional shape of the predetermined portion of the measurement object, The second feature point data may include any of coordinate data indicating the shape of the specified portion of the measurement object, binary image data indicating the shape of the specified portion of the measurement object, multi-valued image data indicating the shape of the specified portion of the measurement object, or height inflection point data indicating the three-dimensional shape of the specified portion of the measurement object.

[0019] Here, "binary image data" can be, for example, image data in which pixels corresponding to the shape of a predetermined part are identified from the color, brightness, etc. of the image data, and a flag of 1 is set for those pixels, and a flag of 0 is set for other parts. Also, "multi-value image data" can be image data in which an edge extraction process is performed on a predetermined part.

[0020] The optimal data to use as feature point data varies depending on which part of the object being measured is being shown, so by using the above-mentioned feature point data appropriately depending on the part, it becomes possible to accurately determine the correspondence between different image data.

[0021] The measurement object is a substrate on which components are mounted, The shape of the specified portion of the measurement object may include any of the shape of a wiring pattern on the substrate, the shape of a land on the substrate, the shape of an electrode of the component mounted on the substrate, and the shape of solder on the substrate.

[0022] The shape of the part from which feature point data is to be acquired must be a shape that can be identified by any imaging device that uses a different measurement principle. In this regard, if the shape of the part is as described above, the shape of the part can be identified from image data acquired by, for example, a visible light camera or an X-ray camera, and therefore valid feature point data can be acquired.

[0023] The measurement system may further include a composite shape data generating unit that generates composite shape data of the predetermined portion of the measurement object by combining at least a portion of the first image data and the second image data or the first shape data and the second shape data based on the positional correspondence of the predetermined portion of the measurement object calculated by the calculating unit. By obtaining such composite shape data, it becomes possible to measure the shape of the inspection object with high accuracy.

[0024] Further, the first image data is image data captured by a visible light camera, and the second image data is X-ray image data, The composite shape data generation means may create composite shape data of the specified portion of the object to be measured by preferentially using information from the second image data or the second shape data for a portion of the first image data that is a blind spot of the visible light camera.

[0025] With this configuration, a three-dimensional shape can be generated for the external appearance of the object being measured based on highly accurate information from the visible light camera, and for areas that are in the blind spots of the visible light camera, a three-dimensional shape can be generated using image data information from the X-ray camera, making it possible to obtain three-dimensional shape data that is highly accurate and has no blind spots.

[0026] The present invention can also be applied as an inspection system for a measurement object, which includes the above-mentioned measurement system and has a composite data inspection means for determining whether the measurement object or a part constituting the measurement object is good or bad based on the composite shape data generated by the composite shape data generation means.

[0027] Such a measurement object inspection system can perform inspection based on the composite shape data, making it possible to perform high-precision measurement object inspection while preventing oversights and overlooks. Furthermore, such a composite data inspection means can apply the same inspection points and inspection standards regardless of the measurement principle, making it possible to perform high-precision measurement object inspection while preventing variations in the inspection quality of the measurement object.

[0028] The present invention can also be understood as an apparatus having the above-described configuration.

[0029] In order to achieve the above object, the present invention provides: A method for measuring a shape of at least a part of a measurement object, comprising: a first feature data generating step of generating first feature data indicating the shape of the predetermined portion of the measurement object from first image data captured including the shape of at least the part of the measurement object or first shape data generated based on the first image data; a second feature data generating step of generating second feature data indicating the shape of the predetermined portion of the measurement object from second image data, which includes the shape of at least a portion of the measurement object and is acquired through a process different from that of the first image data, or from second shape data generated based on the second image data; The measurement method may also include a calculation step of calculating a correspondence relationship between the positions of the predetermined part of the measurement object in the first image data and the second image data, or in the first shape data and the second shape data, based on the first feature point data and the second feature point data.

[0030] Moreover, the calculation step an XY calculation step for determining a correspondence relationship between positions of the measurement object in the horizontal direction; and a Z calculation step of determining a correspondence relationship between positions of the object to be measured in the vertical direction.

[0031] The measurement method may further include a composite shape data generation step of combining at least a portion of the first image data and the second image data, or the first shape data and the second shape data, based on the positional correspondence of the specified portion of the measurement object calculated in the calculation step, to generate composite shape data of the specified portion of the measurement object.

[0032] Furthermore, the first image data may be image data captured by a visible light camera, and the second image data may be X-ray image data, and in the synthetic shape data generation step, for areas in the first image data that are blind spots of the visible light camera, information from the second image data or the second shape data may be used preferentially to create synthetic shape data of the specified portion of the measurement object.

[0033] Furthermore, the present invention includes an inspection object data acquisition step of acquiring composite shape data of a predetermined portion of the measurement object generated by the composite shape data generation step; The present invention can also be applied as an inspection method for a measurement object, which includes a composite data inspection step for determining whether the measurement object or a part constituting the measurement object is good or bad based on the composite shape data of the specified portion of the measurement object acquired by the inspection object data acquisition step.

[0034] The present invention can also be understood as a program for causing a computer to execute the above-described method, or a computer-readable recording medium on which such a program is non-transitoryly recorded.

[0035] The present invention can be achieved by combining the above-described configurations and processes as long as no technical contradiction occurs. [Effects of the Invention]

[0036] According to the present invention, it is possible to provide a technique that enables highly accurate shape measurement by combining data of an object to be inspected that has been measured using a plurality of measurement principles with different imaging systems. [Brief explanation of the drawings]

[0037] [Figure 1] FIG. 1 is a schematic diagram showing the general configuration of a measurement device according to the first embodiment. [Figure 2] FIG. 2 is a flowchart showing the flow of processing in the measurement device according to the first embodiment. [Figure 3] FIG. 3 is a block diagram showing a schematic configuration of a high-precision substrate inspection system according to the second embodiment. [Figure 4] FIG. 4 is an explanatory diagram showing an example of feature point data according to the second embodiment. [Figure 5] Fig. 5A is a diagram illustrating an example of appearance feature point data according to embodiment 2. Fig. 5B is a diagram illustrating an example of X-ray feature point data according to embodiment 2. [Figure 6] FIG. 6 is a flowchart showing the flow of processing in the high-precision substrate inspection system according to the second embodiment. [Figure 7] FIG. 7 is a first diagram showing a processing subroutine in the high-precision substrate inspection system according to the second embodiment. [Figure 8] FIG. 8 is a second diagram showing the processing subroutine in the high-precision substrate inspection system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0038] The following describes embodiments of the present invention with reference to the drawings. However, unless otherwise specified, the dimensions, materials, shapes, relative positions, etc. of the components described in each of the following examples are not intended to limit the scope of the present invention.

[0039] <Application example> (Configuration of application examples) The present invention can be applied, for example, as an information processing device having a function of determining the three-dimensional shape of a measurement object based on image data obtained by capturing images of the measurement object using two different measurement principles (imaging means). Fig. 1 is a schematic diagram showing the general configuration of a measurement device 9 including an information processing device 91 according to this application example.

[0040] The information processing device 91 can be configured, for example, by a general-purpose computer, and includes functional units such as a first feature point data generating unit 911, a second feature point data generating unit 912, a correspondence relationship calculating unit 913, and a composite shape data generating unit 914. In addition, although not shown, it may include various input means such as a mouse and a keyboard, output means such as a display, RAM, HDD, etc. It may also include storage means, communication means, etc.

[0041] The first feature data generating unit 911 generates first feature data indicating the shape of a predetermined portion of the measurement object based on first image data obtained by capturing an image of the measurement object. The second feature data generating unit 912 generates second feature data indicating the shape of a predetermined portion of the measurement object based on the first image data obtained by capturing an image of the measurement object. The first image data and second image data may be stored in advance in a storage unit, or may be obtained from an external source via a communication unit, an input unit, or the like.

[0042] The correspondence calculation unit 913 performs processing to calculate the correspondence between the positions of predetermined parts of the measurement object in the first image data and the second image data based on the first feature point data and the second feature point data so that a composite shape data generation unit 914 (described later) can generate composite three-dimensional shape data. Specifically, for example, a reference is generated for converting the coordinates of each pixel in the second image data into the coordinate system of the first image data.

[0043] Since the first image data and the second image data are image data obtained by capturing images of the measurement object using different imaging means, there is no correspondence between the positions (coordinates) on the image data that indicate the same location on the measurement object. For this reason, these image data cannot be combined as is, but the correspondence calculation unit 913 can calculate the correspondence between the two image data to enable pixel alignment.

[0044] The composite shape data generation unit 914 combines at least a portion of the first image data and the second image data based on the correspondence between the positions of the specified part of the measurement object in the first image data and the second image data calculated by the correspondence calculation unit 913, and generates three-dimensional shape data of the specified part of the measurement object.

[0045] The three-dimensional shape data may be generated by, for example, using the information of the image data with higher reliability among the pixels representing a predetermined portion of the measurement object in the first image data and the second image data as the pixels of the composite data. For example, since blind spots vary depending on the imaging angle, information from the other image data can be used for blind spots in each image data. In this way, highly accurate composite three-dimensional shape data can be obtained, generated only from the image information with higher reliability.

[0046] <Embodiment 1> Next, an embodiment of the present invention will be described in more detail with reference to Figures 1 and 2. A measuring device 9 according to this embodiment is an apparatus that captures images of a component mounting board (hereinafter also simply referred to as board) that is the measurement target using two different measurement principles (imaging means) and determines the three-dimensional shape of the board based on the captured image data.

[0047] The first imaging means 92 and the second imaging means 93 are optical cameras that capture images of the measurement target substrate O being transported by transport rollers (not shown) from a predetermined angle. The first imaging means 92 captures an image of the measurement target substrate O from directly above at an angle perpendicular to the substrate surface, and outputs direct-view image data to the information processing device 91. The second imaging means 93 captures an image of the substrate O from an angle oblique to the substrate surface, and outputs oblique-view image data to the information processing device 91.

[0048] The information processing device 91 has the same configuration and functions as those described in the application example above, and therefore a description thereof will be omitted. The measurement object, the first image data, and the second image data in the application example correspond to the substrate O, the direct-view image data, and the oblique-view image data in this embodiment, respectively.

[0049] In this embodiment, the feature point data can be, for example, X and Y coordinate data of four points indicating the ends (vertices) of a predetermined land on the substrate O. In this case, the shape formed by connecting the points is the shape of the land.

[0050] (Processing flow) 2 shows the procedure of processing performed by the measurement device 9 in this embodiment. First, the measurement device 9 captures an image of the substrate O with the first imaging means 92 to obtain direct-view image data (S901), and then captures an image of the substrate O with the second imaging means 93 to obtain oblique-view image data (S902).

[0051] Next, the measurement device 9 generates first feature data from the direct-view image data using the first feature data generation unit 911 (S903), and generates second feature data from the oblique-view image data using the second feature data generation unit 912 (S904).

[0052] Next, the measuring device 9 performs a process of calculating the correspondence relationship between the positions of predetermined parts of the substrate O in the direct-view image data and the oblique-view image data using the correspondence relationship calculation unit 913 based on the first feature point data and the second feature point data generated in steps S903 and S904 (S905). This makes it possible to process these images based on the calculated correspondence relationship between the direct-view image data and the oblique-view image data.

[0053] Furthermore, the measurement device 9 generates three-dimensional shape data of the predetermined portion of the substrate O by combining at least a portion of the direct-view image data and the oblique-view image data based on the correspondence relationship between the positions of the predetermined portion of the substrate O in the direct-view image data and the oblique-view image data calculated in step S905 (S906), and ends the series of routines. Note that by appropriately repeating this routine depending on the location to be measured, desired measurement results can be obtained.

[0054] According to the measuring device 9 of this embodiment, data of the inspection object measured by a plurality of measurement principles using different imaging systems can be synthesized to obtain highly accurate shape measurement results.

[0055] <Embodiment 2> In the measuring device 9 according to the first embodiment, the present invention is applied as a single device having a plurality of optical cameras provided at different angles, but the present invention is not limited to this configuration. Below, other embodiments of the present invention will be described with reference to Figures 3 to 8, taking as an example a system that inspects a substrate using a visual inspection device and an X-ray inspection device.

[0056] (System Configuration) 3 is a block diagram showing an outline of the configuration of a high-precision substrate inspection system 1 according to this embodiment. The high-precision substrate inspection system 1 according to this embodiment is generally configured to include a visual inspection device 10, an X-ray inspection device 20, a data server 30, and a high-precision inspection terminal 40, which are communicatively connected by communication means (not shown). This system inspects substrates based on measurement results of the substrate obtained by the visual inspection device 10 and higher-precision measurement results obtained based on the measurement results of the substrate obtained by the X-ray inspection device 20.

[0057] The visual inspection device 10 is a device that performs visual inspection of component-mounted boards using an inspection method that combines, for example, the so-called phase shift method and color highlight method. The inspection method that combines the phase shift method and color highlight method is already known technology, so a detailed explanation will be omitted, but this type of inspection makes it possible to accurately detect the shape of electrodes that are visible from the outside and the degree of inclination of fillets in the land portion of the board. The phase shift method is a method that detects distortion of a pattern when pattern light is projected onto the surface of an object. The color highlight method is a method of capturing the three-dimensional shape of an object surface as two-dimensional hue information by irradiating the board with light of multiple colors (wavelengths) at different angles of incidence, and capturing an image in such a way that the color characteristics (color of the light source in the direction of specular reflection as seen from the camera) appear on the solder surface according to its normal direction.

[0058] The appearance inspection device 10 generally includes functional units such as an appearance image capturing unit 110, an appearance shape data generating unit 120, and an appearance inspection unit 130, as well as a projector, a light source, and a stage for holding the board (none of which are shown). The appearance image capturing unit 110 captures an image of the board illuminated by light from a projector and a light source (not shown), and outputs an image for appearance inspection. The appearance shape data generating unit 120 measures the appearance shape of the board based on the appearance inspection image and generates appearance shape data. The appearance inspection unit 130 performs an appearance inspection (i.e., pass / fail judgment) of the board (components thereon) by comparing the appearance shape data with an inspection standard. Note that, hereinafter, even when the term "board inspection" is simply used, it also includes inspection of components on the board.

[0059] The appearance inspection unit 130 further generates appearance feature point data from the appearance shape data. The appearance feature point data can be coordinate data obtained from the appearance shape data that indicates the shape of a predetermined portion of the board (for example, a metal portion connected to each component, the same applies below), binary image data that indicates the shape of the predetermined portion of the board, multi-valued image data that indicates the shape of the predetermined portion of the board, or height inflection point data that indicates the three-dimensional shape of the predetermined portion of the board.

[0060] The above-mentioned information on the appearance inspection image, appearance shape data, appearance inspection results, and appearance feature point data is transmitted from the appearance inspection device 10 to the data server 30 and stored in the data server 30.

[0061] The X-ray inspection device 20 is a device that creates three-dimensional shape data of a substrate by, for example, a method such as CT (Computed Tomography) or tomosynthesis, and judges whether the substrate is good or bad based on the three-dimensional data.

[0062] The X-ray inspection apparatus 20 generally includes functional units such as an X-ray image capturing unit 210, an X-ray shape data generating unit 220, and an X-ray inspection unit 230, as well as an X-ray source and a stage for holding a substrate (none of which are shown). The X-ray image capturing unit 210 captures X-rays emitted from an X-ray source (not shown) and transmitted through the substrate, thereby outputting a tomographic image of the substrate (hereinafter referred to as an X-ray image). The X-ray shape data generating unit 220 measures the three-dimensional shape of the substrate based on multiple X-ray images and generates three-dimensional shape data (hereinafter referred to as X-ray shape data). The X-ray inspection unit 230 compares the X-ray shape data with an inspection standard to inspect the three-dimensional shape of the substrate (i.e., determine whether it is good or bad).

[0063] The X-ray inspection unit 230 further generates X-ray feature point data from the X-ray shape data. The X-ray feature point data can be coordinate data indicating the shape of a predetermined portion of the substrate, binary image data indicating the shape of a predetermined portion of the substrate, multi-valued image data indicating the shape of a predetermined portion of the substrate, or height inflection point data indicating the three-dimensional shape of a predetermined portion of the substrate, all of which are obtained from the X-ray shape data. Furthermore, each piece of information about the X-ray image, X-ray shape data, X-ray inspection results, and X-ray feature point data is transmitted from the X-ray inspection apparatus 20 to the data server 30 and stored therein.

[0064] The above-mentioned appearance feature point data and X-ray feature point data are generated for metal parts of the board that can be identified by either the appearance inspection device 10 or the X-ray inspection device 20. Specifically, the data may be, for example, the shape of a wiring pattern, the shape of a land, the shape of an electrode of a component, the shape of a solder, etc.

[0065] The high-precision inspection terminal 40 is, for example, a general-purpose computer. That is, although not shown, it is equipped with a processor such as a CPU or DSP, a storage unit including a main storage unit such as a read-only memory (ROM) or random access memory (RAM) and an auxiliary storage unit such as an EPROM, a hard disk drive (HDD) or removable media, an input unit such as a keyboard or a mouse, and an output unit such as a liquid crystal display. The high-precision inspection terminal 40 may be configured as a single computer or may be configured as multiple computers that cooperate with each other.

[0066] The auxiliary storage unit stores an operating system (OS), various programs, various information related to the object to be inspected, various inspection standards, etc. The programs stored therein are loaded into the working area of ​​the main storage unit and executed, and the components are controlled through the execution of the programs, thereby realizing functional units that fulfill predetermined purposes, as described below. Note that some or all of the functional units may be realized by hardware circuits such as ASICs and FPGAs.

[0067] Next, we will explain each functional unit included in the high-precision inspection terminal 40. The high-precision inspection terminal 40 includes a composite reference generation unit 410, a composite shape data creation unit 420, a composite shape data inspection unit 430, and a composite inspection unit 440.

[0068] The synthesis reference generation unit 410 acquires the appearance feature point data and X-ray feature point data stored in the data server 30, calculates the correspondence between the positions of specific parts of the substrate in the appearance shape data and the X-ray shape data based on each feature point data, and generates a reference for synthesizing these shape data.

[0069] The synthesis reference generation unit 410 includes an XY synthesis reference generation unit 411 and a Z synthesis reference generation unit 412. The XY synthesis reference generation unit 411 generates an XY synthesis reference (e.g., a transformation matrix) for converting the external shape data into X-ray shape data based on the correspondence of the coordinates of the wiring pattern portion of feature point data (binarized image data with flags set on the wiring pattern) as shown in Fig. 4, for example. Specifically, the external shape data and the wiring pattern of the X-ray shape data are compared using a general pattern matching method, and three or four points with a high degree of match can be selected from the edges and corners of the wiring pattern. Then, a matrix for affine transformation can be calculated using a general method if there are three points, or a matrix for projective transformation can be calculated if there are four points.

[0070] The Z-composite reference generating unit 412 generates a Z-composite reference (e.g., a height magnification factor) by comparing the heights of the solder areas in the external shape and the common parts of the solder areas in the X-ray shape, for example, based on feature point data indicating the solder areas. Specifically, as shown in FIG. 5, for example, N points (end points, angle inflection points, etc.) that are easy to identify are found from the shape data of the common parts, and a magnification factor is found to convert the height in the X-ray shape data to the height in the external shape data. Note that the "common parts" here refer to parts that can be identified both externally and by X-ray. For example, a back fillet, which is a blind spot for the leads in the external view, is not a common part.

[0071] Here, an example of a method for determining the height magnification will be described. Fig. 5A is a diagram illustrating appearance feature point data showing a solder area in a predetermined location, and Fig. 5B is a diagram illustrating X-ray feature point data showing a solder area in a predetermined location. If point G1 in the appearance feature point data corresponds to point X1 in the X-ray feature point data, and G2 corresponds to X2, and G3 corresponds to X3, respectively, the height magnification can be determined by calculating (G1 ÷ X1 + G2 ÷ X2 + G3 ÷ X3) ÷ 3.

[0072] The solder areas on the board are identified using color, brightness, etc. when inspected by a visual inspection device and an X-ray inspection device, so the information on the locations is stored in the data server 30 as feature point data (for example, in flag image format), and the Z synthesis reference generation unit 412 generates the feature point data. It is a good idea to read it out.

[0073] The composite shape data creation unit 420 creates composite shape data by combining the external shape data and the X-ray shape data based on the synthesis standard generated by the synthesis standard generation unit 410. Specifically, for example, the following method can be adopted.

[0074] The Z-combining criteria described above are applied to the X-ray shape data, the magnification of the X-ray shape data is adjusted to match that of the external shape data, and a priority is assigned to each pixel of the external shape data and the X-ray shape data. The priority is an indicator of which data should be used when combining the external shape data and the X-ray shape data. For example, the external shape data may contain areas that are unreliable due to shadows of components, etc., so for pixels in such areas, the X-ray shape data information can be prioritized during combination. Specifically, the X-ray shape data information can be prioritized for areas that are not considered solder areas in the external feature point data, such as back fillets and bottom-mounted components, and for areas that are close to surrounding components and are likely to be blind spots in the external inspection system because they are within a certain distance. The external shape data information can be prioritized for other areas.

[0075] Based on the priority thus set, pixels with a higher priority in either the external shape data or the X-ray shape data are overwritten on the other data, thereby making it possible to create composite shape data.

[0076] The composite shape data inspection unit 430 judges the quality of the board based on preset inspection standards and composite shape data. During inspection, not only the composite shape data but also feature point data such as electrode coordinates and component heights may be used to improve inspection accuracy.

[0077] The composite inspection unit 440 makes a final judgment on the quality of the board by combining the results of the visual inspection, X-ray inspection, and composite shape data inspection. For example, if the top priority is to prevent oversights, it can be determined that the board is defective if any of the three types of inspection results in an NG judgment, or if overdetection needs to be reduced, it can be determined that the board is defective if at least one of the visual inspection and X-ray inspection results in an NG judgment and the composite shape data inspection results in an NG judgment. Note that it is also possible to use only the composite shape data inspection as the final judgment result without referring to the results of the visual and X-ray inspections.

[0078] (High-precision inspection process flow) Next, referring to FIG. 6, a process flow for performing high-precision inspection on a substrate to be inspected in the high-precision substrate inspection system 1 of this embodiment will be described. FIG. 6 is a flowchart showing the process flow. As shown in FIG. 6, first, the appearance inspection device 10 performs appearance inspection of the substrate (S101). FIG. 7 is a flowchart showing the specific process in step S101. As shown in FIG. 7, the appearance inspection device 10 acquires an appearance inspection image of the inspection target substrate via the appearance image capturing unit 110 (S201), and generates appearance shape data in the appearance shape data generating unit 120 based on the acquired image (S202). Then, the appearance inspection unit 130 generates appearance feature point data based on the appearance shape data, and determines whether the substrate is good or bad based on a preset inspection standard and the appearance shape data (S204). Then, the inspection results (and the appearance inspection image, appearance shape data, and appearance feature point data) are output to the data server 30 (S205), and the subroutine processing ends.

[0079] After step S101, the X-ray inspection device 20 performs an X-ray inspection of the substrate (S102). Fig. 8 shows a flowchart of the specific processing in step S102. As shown in Fig. 8, the X-ray inspection device 20 acquires an X-ray image of the substrate to be inspected via the X-ray image capturing unit 210 (S301), and based on this, generates an X-ray shape data by the X-ray shape data generating unit 220. (S302). Then, in the X-ray inspection unit 230, X-ray feature point data is generated based on the X-ray shape data, and the quality of the board is determined based on a preset inspection standard and the X-ray shape data (S304). Then, the inspection results (and the X-ray image, X-ray shape data, and X-ray feature point data) are output to the data server 30 (S305), and the processing of the subroutine is terminated.

[0080] Then, after the processing of step S102 is completed, the high-precision inspection terminal 40 executes the processing of loop L1, which will be described below, on the parts to be inspected with high precision. Note that the parts to be inspected with high precision can be determined based on conditions such as "parts that are designated as parts to be inspected in advance by the program" or "parts that have been determined to be defective by either visual inspection or X-ray inspection." These conditions can also be combined to determine "parts that are designated as parts to be inspected in advance by the program and parts that have been determined to be defective by either visual inspection or X-ray inspection," or a condition can be set such as "parts that are designated as parts to be inspected in advance by the program and parts that have been determined to be defective by either visual inspection or X-ray inspection."

[0081] In loop L1, first, the synthesis reference generation unit 410 selects the external shape data and X-ray shape data that include the component to be inspected with high precision, which are stored in the data server 30 (S103), and selects the lands connected to the target component based on the external feature point data and X-ray feature point data (step S104). Note that the correspondence between the position of the component on the board and each piece of shape data may be obtained from the inspection program, or information may be added to each piece of shape data and stored in the data server 30. Furthermore, when selecting lands, coordinate information of multiple locations that indicate the outline shape of the land in each piece of shape data can be used as feature point data.

[0082] Furthermore, the composite reference generation unit 410 sets the area including the lands selected in step S104 as the inspection target area within each shape data (S105). Because the board is not perfectly horizontal when imaged and warping and distortion often occur, it is difficult to accurately calculate the correspondence for the entire shape data. Therefore, by narrowing the area to only the periphery of each land of the target component as described above and setting the inspection area and performing individual processing, it is possible to achieve planar approximation while maintaining accuracy.

[0083] After the process of step S105 is completed, the XY composite reference generation unit 411 generates an XY composite reference (S106), and then the Z composite reference generation unit 412 generates a Z composite reference (S107). Next, the composite shape data creation unit 420 combines the appearance shape data and the X-ray shape data based on the combination references generated in steps S106 and S107 to create composite shape data (S108). Then, using the composite shape data created in step S108 and the inspection reference, the composite shape data inspection unit 430 performs a pass / fail judgment on the board (S109). Furthermore, the composite inspection unit 440 finally judges the pass / fail of the board by combining the inspection results of the appearance inspection, X-ray inspection, and composite shape data inspection (step S110), thereby completing the series of processes in loop L1.

[0084] Then, when the processing of the above loop L1 is completed for all the components to be inspected with high precision, the high precision board inspection system 1 temporarily terminates this routine. Note that the specific processing from step S106 to step S110 has already been explained when explaining each function, so a repeated explanation will be omitted.

[0085] According to the high-precision board inspection system 1 described above, it is possible to inspect a board (components thereon) based on high-precision composite shape data in which X-ray shape data is used to supplement areas where the reliability of external shape data is low, such as solder back fillets and narrow areas where components are densely packed. Therefore, highly accurate test results can be obtained.

[0086] In this embodiment, the appearance image capturing unit 110 corresponds to a first capturing means, the appearance inspection unit 130 corresponds to a first feature point data generating means, the X-ray image capturing unit 210 corresponds to a second capturing means, the X-ray inspection unit 230 corresponds to a second feature point data generating means, and the composite reference generating unit 410 corresponds to a calculating means. Also, the XY composite reference generating unit 411 corresponds to an XY calculating unit, and the Z composite reference generating unit 412 corresponds to a Z calculating unit.

[0087] <Other> The above-described embodiments merely exemplify the present invention, and the present invention is not limited to the specific embodiments described above. Various modifications and combinations of the present invention are possible within the scope of the technical concept. For example, although the above-described embodiments are described as systems including an imaging unit, the present invention can also be applied to systems that do not include an imaging unit, as described as an information processing device in the application examples.

[0088] Furthermore, in the above-mentioned second embodiment, the appearance inspection device 10 was described as an inspection device that uses a combination of the phase shift method and the color highlight method, but it may also be an appearance inspection device that performs inspection using only the phase shift method or only the color highlight method.

[0089] Furthermore, this method is not limited to the combination of a visual inspection device and an X-ray inspection device, but can also be applied to the combination of a laser scan measurement device and an X-ray inspection device. Even with this combination, the inaccurate Z-axis information (height magnification) obtained by X-ray inspection alone can be corrected by the laser scan measurement.

[0090] It is needless to say that the order of steps S102 and S103 may be reversed in the flow of the high-precision substrate inspection process of the above-described embodiment 2. Also, a step of storing the composite shape data created in step S108 and the result of the composite inspection performed in step S109 in the data server 30 may be included.

[0091] Furthermore, in the above-mentioned embodiment 2, a high-precision inspection terminal 40 for high-precision inspection was provided separately from the appearance inspection device 10 and the X-ray inspection device 20. However, without providing a separate high-precision inspection terminal 40, it is also possible to provide each functional unit for high-precision inspection in either the appearance inspection device 10 or the X-ray inspection device 20 and have it perform the processing of the above-mentioned steps S103 to S110.

[0092] In addition, in each of the above examples, the imaging systems for acquiring the first image data and the second image data are configured differently, but the same imaging system may be used to acquire multiple pieces of image data acquired at different times. For example, the measurement target may be imaged multiple times with different exposure times, and the pixel correspondence between these pieces of image data may be calculated.

[0093] <Additional Notes> One aspect of the present invention is A measurement system (1) for measuring the shape of at least a part of a measurement object, a first feature data generating means (130) for generating first feature data indicating a shape of a predetermined portion of the measurement object from first image data captured including the shape of at least a portion of the measurement object or first shape data generated based on the first image data; a second feature data generating means (230) for generating second feature data indicating the shape of the predetermined portion of the measurement object from second image data, the second image data being obtained by a process different from that of the first image data, or second shape data being generated based on the second image data; The first image data and the previous image data are compared based on the first feature point data and the second feature point data. and a calculation means (410) for calculating a correspondence relationship between the position of the predetermined portion of the measurement object in the second image data or the first shape data and the second shape data.

[0094] A measurement device (9) that measures the shape of at least a part of a measurement object, a first feature data generating means (911) for generating first feature data indicating a shape of a predetermined portion of the measurement object from first image data captured including the shape of at least a portion of the measurement object or first shape data generated based on the first image data; a second feature data generating means (912) for generating second feature data indicating the shape of the predetermined portion of the measurement object from second image data, the second image data including the shape of at least a portion of the measurement object and acquired through a process different from that of the first image data, or second shape data generated based on the second image data; and a calculation means (913) for calculating a correspondence relationship between the positions of the predetermined part of the measurement object in the first image data and the second image data, or in the first shape data and the second shape data, based on the first feature point data and the second feature point data.

[0095] Another aspect of the present invention is A method for measuring a shape of at least a part of a measurement object, comprising: a first feature data generating step (S903) of generating first feature data indicating the shape of the predetermined portion of the measurement object from first image data captured including the shape of at least the part of the measurement object or first shape data generated based on the first image data; a second feature data generating step (S904) of generating second feature data indicating the shape of the predetermined portion of the measurement object from second image data, which includes the shape of at least a part of the measurement object and is acquired through a process different from that of the first image data, or from second shape data generated based on the second image data; and a calculation step (S905) of calculating a correspondence relationship between the positions of the predetermined part of the measurement object in the first image data and the second image data, or in the first shape data and the second shape data, based on the first feature point data and the second feature point data. [Explanation of symbols]

[0096] 1. High-precision inspection system 10. Visual inspection equipment 110 External image capturing unit 120: External shape data generation unit 130 Visual Inspection Department 20. X-ray inspection equipment 210 X-ray imaging unit 220 X-ray shape data generation unit 230 X-ray Inspection Department 30 Data Server 40 High-precision inspection terminal 410...Synthesis standard generation section 411...XY synthesis reference generation section 412...Z synthesis standard generation section 420: Synthetic shape data generation unit 430 Composite shape data inspection unit 440···Composite Inspection Department 9. Measurement equipment 92 First imaging means 93... Second imaging means 911: First feature point data generation unit 912...Second feature point data generation unit 913 Correspondence calculation unit 914: Synthetic shape data generation unit

Claims

1. A measurement system that measures the shape of at least a part of a measurement object, a visible light camera and an X-ray camera for capturing images of the measurement object; a first feature data generating means for generating first feature data indicating a shape of a predetermined portion of the measurement object from first shape data generated based on visible light image data captured including the shape of at least the part of the measurement object; a second feature data generating means for generating second feature data indicating the shape of the predetermined portion of the measurement object from second shape data generated based on X-ray image data captured including the shape of at least the part of the measurement object; a calculation means for calculating a correspondence relationship between the positions of the predetermined portion of the measurement object in the first shape data and the second shape data based on the first feature point data and the second feature point data; a composite three-dimensional shape data generating means for generating composite three-dimensional shape data of the predetermined portion of the measurement object by combining at least a part of the first shape data and the second shape data based on the positional correspondence of the predetermined portion of the measurement object calculated by the calculating means; It has The calculation means an XY synthesis reference generation unit that determines a correspondence relationship between positions of the measurement object in a horizontal direction and generates a synthesis reference in the horizontal direction for the first shape data and the second shape data; a Z synthesis reference generation unit that determines a correspondence relationship between positions of the measurement object in a vertical direction and generates a synthesis reference in the vertical direction for the first shape data and the second shape data; A measurement system comprising:

2. The synthesis criterion in the horizontal direction is a transformation matrix that transforms the first shape data into the second shape data, the synthesis criterion in the vertical direction is a height magnification for converting a height in the second shape data into a height in the first shape data; 2. The measurement system according to claim 1, wherein:

3. the first feature point data includes any one of coordinate data indicating the shape of the predetermined portion of the measurement object, binary image data indicating the shape of the predetermined portion of the measurement object, multi-valued image data indicating the shape of the predetermined portion of the measurement object, and height inflection point data indicating the three-dimensional shape of the predetermined portion of the measurement object, The second feature point data includes any one of coordinate data indicating the shape of the predetermined portion of the measurement object, binary image data indicating the shape of the predetermined portion of the measurement object, multi-valued image data indicating the shape of the predetermined portion of the measurement object, and height inflection point data indicating the three-dimensional shape of the predetermined portion of the measurement object.

3. The measurement system according to claim 1 or 2.

4. the measurement object is a substrate on which components are mounted, The shape of the predetermined portion of the measurement object includes any one of the shape of a wiring pattern on the board, the shape of a land on the board, the shape of an electrode of the component mounted on the board, and the shape of a solder on the board.

4. The measurement system according to claim 1, wherein the measurement system comprises: a first electrode;

5. The shape of the predetermined portion of the measurement object includes the shape of a wiring pattern on the substrate, The XY synthesis reference generation unit the wiring patterns in the first shape data and the second shape data are compared by pattern matching, and the transformation matrix is ​​obtained by selecting three or four points with a high degree of match from the ends or corners of the wiring patterns; 5. The measurement system according to claim 4, wherein:

6. The shape of the predetermined portion of the measurement object includes the shape of the solder on the substrate, The Z synthesis reference generation unit Finding N points from the end points or angle inflection points in the common portion of the solder regions in the first shape data and the second shape data, and based on this, finding a height magnification for converting the height in the second shape data to the height in the first shape data.

6. The measurement system according to claim 4 or 5.

7. the composite three-dimensional shape data generation means generates the composite three-dimensional shape data of the predetermined portion of the measurement object by preferentially using information of the second shape data for a portion of the visible light image data that is a blind spot of the visible light camera.

7. The measurement system according to claim 1, wherein the measurement system comprises:

8. A system for inspecting the measurement object, comprising the measurement system according to any one of claims 1 to 7, an inspection system comprising a composite data inspection means for determining whether the measurement object or a component constituting the measurement object is good or bad based on the composite three-dimensional shape data generated by the three-dimensional composite shape data generation means;

9. A measurement device that measures the shape of at least a part of a measurement object, a first feature data generating means for generating first feature data indicating a shape of a predetermined portion of the measurement object from first shape data generated based on visible light image data captured including the shape of at least the part of the measurement object; a second feature data generating means for generating second feature data indicating the shape of the predetermined portion of the measurement object from second shape data generated based on X-ray image data captured including the shape of at least the part of the measurement object; a calculation means for calculating a correspondence relationship between the positions of the predetermined portion of the measurement object in the first shape data and the second shape data based on the first feature point data and the second feature point data; a composite three-dimensional shape data generating means for generating composite three-dimensional shape data of the predetermined portion of the measurement object by combining at least a part of the first shape data and the second shape data based on the positional correspondence of the predetermined portion of the measurement object calculated by the calculating means, The calculation means an XY synthesis reference generation unit that determines a correspondence relationship between positions of the measurement object in a horizontal direction and generates a synthesis reference in the horizontal direction for the first shape data and the second shape data; a Z synthesis reference generation unit that determines a correspondence relationship between positions of the measurement object in a vertical direction and generates a synthesis reference in the vertical direction for the first shape data and the second shape data; A measuring device comprising:

10. A method for measuring a shape of at least a part of a measurement object, comprising: a first feature point data generating step of generating first feature point data indicating a shape of a predetermined portion of the measurement object from first shape data generated based on visible light image data captured including the shape of at least the part of the measurement object; a second feature data generating step of generating second feature data indicating the shape of the predetermined portion of the measurement object from second shape data generated based on X-ray image data captured including the shape of at least the part of the measurement object; a calculating step of calculating a correspondence relationship between the positions of the predetermined portion of the measurement object in the first image data and the second image data, or the first shape data and the second shape data, based on the first feature point data and the second feature point data; a composite three-dimensional shape data generating step of generating composite three-dimensional shape data of the predetermined portion of the measurement object by combining at least a part of the first shape data and the second shape data based on the positional correspondence of the predetermined portion of the measurement object calculated in the calculating step, The calculation step an XY synthesis reference generating step of determining a correspondence relationship between positions of the measurement object in a horizontal direction and generating a synthesis reference in the horizontal direction for the first shape data and the second shape data; a Z synthesis reference generating step of determining a correspondence relationship between positions of the measurement object in a vertical direction and generating a synthesis reference in the vertical direction between the first shape data and the second shape data; A measurement method that includes

11. The synthesis criterion in the horizontal direction is a transformation matrix that transforms the first shape data into the second shape data, the synthesis criterion in the vertical direction is a height magnification for converting a height in the second shape data into a height in the first shape data; The measurement method according to claim 10 ,

12. The measurement object is a substrate on which components are mounted, the shape of the predetermined portion of the measurement object includes the shape of a wiring pattern on the substrate, In the XY synthesis reference generation step, The wiring patterns in the first shape data and the second shape data are compared by pattern matching, and three or more points with high matching scores are selected from the ends or corners of the wiring patterns. finds the transformation matrix by selecting four points, The measurement method according to claim 11 .

13. The measurement object is a substrate on which components are mounted, the shape of the predetermined portion of the measurement object includes the shape of the solder on the substrate; In the Z synthesis reference generation step, Finding N points from the end points or angle inflection points in the common portion of the solder regions in the first shape data and the second shape data, and based on this, finding a height magnification for converting the height in the second shape data to the height in the first shape data.

13. The measuring method according to claim 11 or 12,

14. In the synthetic three-dimensional shape data generating step, for a portion of the visible light image data that is a blind spot of the visible light camera, information on the second shape data is used preferentially to generate synthetic three-dimensional shape data of the predetermined portion of the measurement object. The measurement method according to claim 10 ,

15. an inspection object data acquisition step of acquiring composite three-dimensional shape data of a predetermined portion of the measurement object generated in the composite three-dimensional shape data generation step in the measurement method according to any one of claims 10 to 12; and a composite data inspection step of determining whether the measurement object or a part constituting the measurement object is good or bad based on the composite three-dimensional shape data of the specified portion of the measurement object acquired by the inspection object data acquisition step.

16. A program for causing a computer to execute each step of the measurement method according to any one of claims 10 to 14 or each step of the inspection method according to claim 15.

Images