Apparatus and method for measuring 3D data of devices mounted on PCB using AI-based phase map restoration

Abstract

The present invention relates to a method for measuring three-dimensional data in a three-dimensional data measuring device for measuring three-dimensional data of a device mounted on a PCB, comprising the steps of: acquiring a moiré pattern projection image, which is a moiré pattern image projected onto the PCB; calculating a phase map using the moiré pattern projection image; extracting from the moiré pattern projection image a shadow area, which is an area where a shadow is generated on the PCB due to the device, and a reflection area, which is an area that has a brighter value than the brightness value of the surrounding identical pattern due to the surface material of the device; generating a reconstructed phase map by reconstructing the phase map using deep learning; and converting the reconstructed phase map into a three-dimensional data value to output the three-dimensional data of the device. According to the present invention, it is possible to accurately measure 3D data values ​​for the shadow area and reflection area that occur in a 3D data measurement method using a moiré pattern, thereby having the effect of improving the reliability of 3D AOI.

Description

Technology Field

[0001] The present invention relates to measuring three-dimensional data of a device mounted on a PCB (Printed Circuit Board), and more specifically to an AI (Artificial Intelligence)-based 3D data measurement method applied to an inspection equipment that inspects the appearance of a device mounted on a PCB by synthesizing more accurate and numerous information than 2D inspection through 3D inspection using moiré pattern-based 3D data measurement when conducting an appearance inspection of the device mounted on the PCB. Background Technology

[0003] A Printed Circuit Board (PCB) has a structure in which conductors and insulators are laminated in the form of a substrate. It can mount various components such as semiconductors, capacitors, and resistors, and serves to provide electrical connections between components. Furthermore, devices are components that constitute the PCB, such as diodes, transistors, and capacitors mounted on it, and reflection occurs depending on the material of the device.

[0004] A 3D Autonomous Optical Inspector (AOI) is an inspection device that measures the height of an object and outputs it in a 3D form. Conventional AOIs, as optical inspection equipment, utilized 2D images for dimensional measurements and external defect inspections. However, mounting defects—such as soldering failures—show discrepancies between normal mounting and the 3D data; since these differences are difficult to distinguish through 2D inspection alone, 3D data measurement became necessary. Methods for measuring 3D data included stereo vision using two cameras and methods that estimate 3D data by projecting patterns or lasers. In the case of stereo vision, configuring the inspection system increases hardware costs and bulk. While the laser projection method is robust against disturbances, its inspection area is limited, resulting in significant time consumption when inspecting large areas.

[0005] Figure 1 illustrates a 3D data measurement device using a moiré pattern.

[0006] In FIG. 1, a projector (102) for projecting a moire pattern, a camera (103) for capturing the moire pattern, and a computer (101) for analyzing the moire pattern image captured by the camera to measure 3D data of the device are shown.

[0007] In FIG. 1, the 3D data measuring device is configured with a computer or PC (101) for processing the acquired image, a projector (102) that generates a moiré pattern shifted by 0, 90, 180, and 270 degrees, and a camera (103) for acquiring the image. The stage (104) is a place for positioning an object to be inspected, and measures the 3D data of the element by calculating the distortion of the moiré pattern projected onto the object.

[0008] While using moiré patterns offers the advantage of rapidly inspecting large areas, it is susceptible to light interference. Since moiré patterns are a 3D data measurement method that utilizes light, differences in pixel values ​​exist due to shadows generated by the device's 3D data and reflections caused by the device's material. Consequently, measuring 3D data based on these variations results in noise, making it difficult to obtain accurate 3D data measurements.

[0009] A shadow region refers to an area where a shadow is cast because the pattern projected by the projector does not reach it due to components mounted on the PCB.

[0010] Figure 2 shows an example of a shadow area caused by interference due to the 3D data of the device.

[0011] In FIG. 2, the projector (202) generating the moiré pattern is an embodiment in which only one unidirectional projector is used, and in this case, there is a shadow area in the image (203) obtained from the camera (201). The shadow area has a value of 0, and it is impossible to measure the 3D data of the correct element.

[0012] A reflection region refers to an area of ​​the device's surface material that has a brighter value than the surrounding identical pattern.

[0013] Figure 3 shows an example of a reflection area caused by the material of the element.

[0014] In FIG. 3, the projector (302) generating the moiré pattern is a unidirectional projector, and an embodiment is illustrated in which only one is used, and a reflection area exists in the image (303) obtained from the camera (301). Due to light saturation, there is a difference in brightness value between the reflection area and the area where no reflection occurs, and it is impossible to measure the 3D data of the correct element.

[0015] Typically, hundreds to thousands of components are assembled on a single PCB. Since shadow and reflection areas vary depending on the component material or differences in 3D data, accurate 3D data measurement requires outputting 3D data values ​​by replacing them with correct values ​​using area reconstruction. To achieve this, it is necessary to define the areas to be reconstructed and to develop ideas for reconstruction methods.

[0016] Conventional techniques for restoring regions include a method of substituting with the average of surrounding values; however, this method is only feasible when reflection occurs under proper mounting conditions, making it difficult to predict the restored value for the reflected region in cases of faulty mounting. Furthermore, errors occur when applying the average of surrounding values ​​to shadows cast on the device.

[0017] With the recent advancement of deep learning, research on image restoration using learning has also been proposed. The Generative Adversarial Network (GAN) is a representative image generation network that proceeds with image generation learning by approaching it with the idea of ​​a conflict between a generator and a discriminator. However, this technology has problems such as requiring a lot of time to generate images as the resolution increases, difficulty in generating correct values ​​for image restoration, and the lack of application to 3D data measurement methods using moiré patterns.

[0018] Figure 4 is a configuration diagram of a conventional moiré pattern-based 3D data measurement device.

[0019] Referring to FIG. 4, a conventional moiré pattern-based 3D data measurement device comprises an image acquisition unit (401) that acquires a moiré pattern projection image obtained by generating a moiré pattern from a projector and projecting it onto an object, a phase map calculation unit (402) that calculates a phase map using pixel values ​​of the moiré pattern projection image, a phase map restoration unit (403) that restores the phase map by removing noise using a deep learning-based restoration network, and a phase map-3D data conversion unit (404) that converts the restored phase map into 3D data values ​​and outputs final 3D data.

[0020] In conclusion, conventional technology has the problem that it cannot infer accurate values ​​for 3D data values ​​generated in shadow areas and reflection areas, and cannot be applied to 3D AOI using moiré patterns by using a fast restoration method for a wide area. Prior art literature

[0021] Republic of Korea Registered Patent 10-0424916 The problem to be solved

[0022] The present invention was devised to solve the above-mentioned problems, and aims to provide a 3D data measurement device and method that restores 3D data measurement errors regarding shadow areas and reflection areas when measuring 3D data of a device mounted on a printed circuit board using a moiré pattern.

[0023] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0025] The present invention, for achieving such an objective, relates to a method for measuring three-dimensional data in a three-dimensional data measuring device for measuring three-dimensional data of a device mounted on a PCB, comprising the steps of: acquiring a moiré pattern projection image, which is a moiré pattern image projected onto the PCB; calculating a phase map using the moiré pattern projection image; extracting from the moiré pattern projection image a shadow area, which is an area where a shadow is generated on the PCB due to the device, and a reflection area, which is an area that has a brighter value than the brightness value of the surrounding identical pattern due to the surface material of the device; generating a restored phase map by restoring the phase map using deep learning; and converting the restored phase map into a three-dimensional data value to output the three-dimensional data of the device.

[0026] In the step of generating the above-mentioned restored phase map, the phase map can be restored through an AutoEncoder-based deep learning neural network.

[0027] In the step of generating the above-mentioned reconstructed phase map, the phase map can be reconstructed through a multivariate autoencoder-based reconstruction network.

[0028] In the step of generating the above-mentioned reconstructed phase map, the reconstructed phase map can be generated by inputting the phase map into the reconstruction network, extracting features from the input phase map image in the encoder of the reconstruction network to generate a latent vector, and converting the latent vector back into an image in the decoder of the reconstruction network.

[0029] After generating the above latent vector, shadow areas and reflection areas, which are the regions to be restored, can be added to the above latent vector.

[0030] In the step of outputting the above three-dimensional data, the above-mentioned restored phase map can be converted into a three-dimensional data value using regression analysis.

[0031] A three-dimensional data measuring device for measuring three-dimensional data of a device mounted on a PCB according to the present invention comprises: a projector for projecting a moiré pattern onto the PCB; a camera for photographing the PCB onto which the moiré pattern is projected by the projector; and a computer that acquires a moiré pattern projection image, which is a moiré pattern image projected onto the PCB from the camera, calculates a phase map using the moiré pattern projection image, extracts a shadow area, which is an area where a shadow is generated on the PCB due to the device, and a reflection area, which is an area that has a brighter value than the brightness value of the surrounding identical pattern due to the surface material of the device, generates a restored phase map by restoring the phase map using deep learning, and converts the restored phase map into a three-dimensional data value to output three-dimensional data of the device.

[0032] The above computer can restore the above phase map through an autoencoder-based deep learning neural network.

[0033] The above computer can reconstruct the phase map through a multivariate autoencoder-based reconstruction network.

[0034] The computer can generate a reconstructed phase map by inputting a phase map into the reconstruction network, generating a latent vector by extracting features from the input phase map image in the encoder of the reconstruction network, and converting the latent vector back into an image in the decoder of the reconstruction network.

[0035] After generating the above latent vector, the computer can add a shadow region and a reflection region, which are the regions to be restored, to the above latent vector.

[0036] The computer can convert the restored phase map into a three-dimensional data value using regression analysis. Effects of the invention

[0038] According to the present invention, it is possible to measure correct 3D data values ​​for shadow areas and reflection areas that occur in a 3D data measurement method using a moiré pattern, thereby improving the reliability of 3D AOI.

[0039] In addition, according to the present invention, by proposing a method to remove noise in shadow and reflection regions by improving the multivariate autoencoder structure, it is possible to measure 3D data more accurately than existing methods, and it is simpler than other existing reconstruction networks, and has the effect of enabling fast 3D data measurement by including a phase map-3D data conversion unit in the reconstruction network.

[0040] In addition, according to the present invention, by proposing a method for measuring 3D data using region reconstruction from a measured phase map without information about the device (type, size, 3D data, etc.) while using various types of devices, there is an advantage in that accurate 3D data values ​​of the device can be inferred by region definition and feature extraction methods even if errors in 3D data measurement occur due to mounting defects and reflections. Brief explanation of the drawing

[0042] Figure 1 illustrates a 3D data measurement device using a moiré pattern. Figure 2 shows an example of a shadow area caused by interference due to the 3D data of the device. Figure 3 shows an example of a reflection area caused by the material of the element. Figure 4 is a configuration diagram of a conventional moiré pattern-based 3D data measurement device. FIG. 5 conceptually illustrates a three-dimensional data measurement device for a device mounted on a PCB according to one embodiment of the present invention. FIG. 6 is a flowchart showing a method for measuring three-dimensional data of a device mounted on a PCB according to one embodiment of the present invention. Figure 7 illustrates a moiré pattern and a moiré pattern projection image. Figure 8 illustrates the shadow and reflection regions that occur when measuring three-dimensional data of a device mounted on a PCB using a moiré pattern. FIG. 9 is a flowchart showing the phase map restoration process and the 3D data conversion process in the restoration network in a method for measuring three-dimensional data of a device mounted on a PCB according to one embodiment of the present invention. Specific details for implementing the invention

[0043] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0044] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0045] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0046] Furthermore, in the description referring to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the present invention, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the present invention, such detailed description is omitted.

[0047] The present invention relates to a method for measuring three-dimensional data in a three-dimensional data measuring device for measuring three-dimensional data of a device mounted on a PCB, comprising the steps of: acquiring a moiré pattern projection image, which is a moiré pattern image projected onto the PCB; calculating a phase map using the moiré pattern projection image; extracting from the moiré pattern projection image a shadow area, which is an area where a shadow is generated on the PCB due to the device, and a reflection area, which is an area that has a brighter value than the brightness value of the surrounding identical pattern due to the surface material of the device; generating a restored phase map by restoring the phase map using deep learning; and converting the restored phase map into a three-dimensional data value to output the three-dimensional data of the device.

[0048] In the step of generating the above-mentioned restored phase map, the phase map can be restored through an AutoEncoder-based deep learning neural network.

[0049] In the step of generating the above-mentioned reconstructed phase map, the phase map can be reconstructed through a multivariate autoencoder-based reconstruction network.

[0050] In the step of generating the above-mentioned reconstructed phase map, the reconstructed phase map can be generated by inputting the phase map into the reconstruction network, extracting features from the input phase map image in the encoder of the reconstruction network to generate a latent vector, and converting the latent vector back into an image in the decoder of the reconstruction network.

[0051] After generating the above latent vector, shadow areas and reflection areas, which are the regions to be restored, can be added to the above latent vector.

[0052] In the step of outputting the above three-dimensional data, the above-mentioned restored phase map can be converted into a three-dimensional data value using regression analysis.

[0053] The present invention defines shadow regions and reflection regions based on pixel values ​​of a moiré pattern projection image obtained from a camera, and inputs the shadow regions and reflection regions into a multivariate autoencoder structure to train a reconstruction network. The defined shadow regions and reflection regions specify the regions to be reconstructed and add features for the corresponding values, and a regression analysis method is added so that the output of the reconstruction network becomes a 3D data value, thereby outputting the correct reconstructed 3D data value.

[0054] FIG. 5 conceptually illustrates a three-dimensional data measurement device for a device mounted on a PCB according to one embodiment of the present invention.

[0055] Referring to FIG. 5, a three-dimensional data measuring device according to one embodiment of the present invention includes a projector (110), a camera (120), and a computer (130).

[0056] The projector (110) serves to project a moiré pattern onto the PCB. More specifically, the projector (110) projects a moiré pattern shifted by 0 degrees, a moiré pattern shifted by 90 degrees, a moiré pattern shifted by 180 degrees, and a moiré pattern shifted by 270 degrees.

[0057] The camera (120) serves to photograph the PCB on which the moiré pattern is projected by the projector (110).

[0058] The computer (130) obtains a moiré pattern projection image, which is a moiré pattern image projected onto a PCB from the camera (120). Specifically, the computer (130) obtains a moiré pattern projection image shifted by 0 degrees, a moiré pattern projection image shifted by 90 degrees, a moiré pattern projection image shifted by 180 degrees, and a moiré pattern projection image shifted by 270 degrees.

[0059] Then, the computer (130) calculates a phase map using a moiré pattern projection image. Then, from the moiré pattern projection image, it extracts a shadow area, which is an area where a shadow is generated on the PCB due to the device, and a reflection area, which is an area that has a brighter value than the surrounding same pattern due to the surface material of the device. Then, it generates a restored phase map by restoring the phase map using deep learning, and converts the restored phase map into a 3D data value to output the 3D data of the device.

[0060] In one embodiment of the present invention, the computer (130) can restore a phase map through an autoencoder-based deep learning neural network. At this time, the computer (130) can restore the phase map through a multivariate autoencoder-based restoration network.

[0061] The computer (130) can generate a reconstructed phase map by inputting a phase map into a reconstruction network, generating a latent vector by extracting features from the input phase map image in the encoder of the reconstruction network, and converting the latent vector back into an image in the decoder of the reconstruction network.

[0062] After the computer (130) generates a potential vector, it can add a shadow area and a reflection area, which are the areas to be restored, to the potential vector.

[0063] The computer (130) can convert the restored phase map into three-dimensional data values ​​using regression analysis.

[0065] FIG. 6 is a flowchart showing a method for measuring three-dimensional data of a device mounted on a PCB according to an embodiment of the present invention. In FIG. 6, the entity performing each step is a computer (130).

[0066] Referring to FIG. 6, a method for measuring three-dimensional data of a device mounted on a PCB according to one embodiment of the present invention includes a step of acquiring a moiré pattern projection image (S610), a step of calculating a phase map (S620), a step of extracting shadow areas and reflection areas (S630), a step of restoring a phase map (S640), and a step of converting three-dimensional data (S650).

[0067] The step of acquiring a moiré pattern projection image (S610) is a process of projecting a moiré pattern generated by a projector (110) onto an object and acquiring a moiré pattern projection image from a camera (120).

[0068] Figure 7 illustrates a moiré pattern and a moiré pattern projection image.

[0069] As shown in FIG. 7, four images of the moiré pattern are generated by shifting the moiré pattern to 0 degrees (601), 90 degrees (603), 180 degrees (605), and 270 degrees (607), and a total of four moiré pattern projection images are generated: a 0-degree moiré pattern projection image (602), a 90-degree moiré pattern projection image (604), a 180-degree moiré pattern projection image (606), and a 270-degree moiré pattern projection image (608).

[0070] In the phase map calculation step (S620), the phase map is calculated based on the pixel values ​​of four moiré pattern projection images.

[0071] In the shadow and reflection area extraction step (S630), the shadow and reflection areas are defined based on threshold values, and the shadow and reflection areas are extracted as binary images. The phase map is a value calculated using arctan for the pixel values ​​of the moiré pattern image and has a value between (-π, +π).

[0072] For 3D data measurement, a phase unwrapping process is required for the phase map to have a range from 0 to 2kπ (k is a constant). In the case of a shadow area, the computer (130) determines that it is a shadow area if all four moiré pattern projected images have pixel values ​​less than a first reference value. For example, if the first reference value is 20, the computer (130) determines that it is a shadow area if all four moiré pattern projected images have pixel values ​​less than 20.

[0073] In the case of a reflection area, the computer (130) defines it as a reflection area if the pixel value of at least one of the four moiré pattern projection images is greater than or equal to the second reference value. For example, if the second reference value is 250, the computer (130) defines it as a reflection area if the pixel value of at least one of the four moiré pattern projection images is 250 or greater.

[0074] Figure 8 illustrates the shadow and reflection regions that occur when measuring three-dimensional data of a device mounted on a PCB using a moiré pattern.

[0075] In the example of FIG. 8, the shadow area (702) for image (701) and the reflection area (704) for image (703) are output as a binarized image, wherein the binarized image has a value of 255 if it is a shadow area or a reflection area, and 0 if it is not a shadow area or a reflection area.

[0076] In the phase map restoration step (S640), the computer (130) performs a restoration network to restore the phase map. To this end, the phase map is input into the restoration network, and the process of restoring the phase map by removing noise is performed by adding features for the restoration of shadow and reflection regions to the latent vector responsible for generating the multivariate autoencoder.

[0077] In the 3D data conversion step (S650), the computer (130) converts the restored phase map into 3D data values ​​through regression analysis and outputs the final 3D data.

[0079] FIG. 9 is a flowchart showing the phase map restoration process and the 3D data conversion process in the restoration network in a method for measuring three-dimensional data of a device mounted on a PCB according to one embodiment of the present invention.

[0080] Referring to FIG. 9, the phase map containing noise calculated in the phase map calculation step (S620) is input into the reconstruction network (S910).

[0081] Then, features of the phase map image input from the encoder in the restoration network are extracted to perform dimensionality reduction, and a latent vector is generated through this (S920).

[0082] Then, shadow regions and reflection regions are added to the latent vector (S930). In this way, the present invention learns to enable more reliable phase map reconstruction by specifying the regions to be used for noise reconstruction of the phase map through the method of adding shadow regions and reflection regions to the latent vector.

[0083] In the decoder of the restoration network, a phase map image is generated again using a latent vector with shadow and reflection regions added, thereby generating a restored phase map, which is a phase map restored with noise removed (S940).

[0084] Then, the reconstructed phase map is converted into 3D data values ​​using regression analysis to output the 3D data of the final device (S950).

[0085] The present invention defines shadow and reflection regions by comparing pixel values ​​of a moiré pattern projected image, improves the structure of a reconstruction network based on a multivariate autoencoder, and enables correct 3D data measurement by using regression analysis in a conversion unit that outputs a phase map as a 3D data value.

[0086] In order to remove noise generated in the moiré pattern-based 3D data measurement method of the present invention, shadow areas and reflection areas are defined by comparing pixel values ​​of the moiré pattern projected image, a multivariate autoencoder is used for phase map reconstruction, a phase map with added noise is input to the reconstruction network, and the structure is improved so that reconstruction can be performed well by specifying the area reconstruction in a way that includes the features of the area to be restored (shadow area, reflection area) during the latent vector generation process of the multivariate autoencoder, which is the reconstruction network.

[0087] In one embodiment of the present invention, a method is proposed that includes features of a region to be restored, wherein information of a shadow region and a reflection region, which are binary images, are encoded in the same dimension into a latent vector used for image generation following a Gaussian distribution to add information of the region to be restored, features of the surrounding region to be restored are included, and a restoration network is trained through cross-entropy comparison between the generated image and the ground truth image.

[0088] While existing reconstruction networks output the same dimension as the input to be reconstructed, in one embodiment of the present invention, a method for outputting 3D data of a device in a moiré pattern-based 3D data measurement method proposes a method for constructing a reconstruction network that outputs accurate 3D data from a reconstructed phase map using a regression analysis method, multiplies the same constant value for the same column using the linearity of the pattern, and outputs 3D data values ​​through parameter optimization for the multiplication of the 3D data portion for the noise-free region.

[0089] Although the present invention has been described above using several preferred embodiments, these embodiments are illustrative and not limiting. Those skilled in the art will understand that various changes and modifications can be made without departing from the spirit of the invention and the scope of rights set forth in the appended claims. Explanation of the symbols

[0091] 110 projector 120 camera 130 computers

Claims

Claim 1 A method for measuring three-dimensional data in a three-dimensional data measuring device for measuring three-dimensional data of a device mounted on a PCB, comprising: a step of acquiring a moiré pattern projection image, which is a moiré pattern image projected onto the PCB; a step of calculating a phase map using the moiré pattern projection image; a step of extracting from the moiré pattern projection image a shadow region, which is an area where a shadow is generated on the PCB due to the device, and a reflection region, which is an area having a brighter value than the brightness value of the surrounding identical pattern due to the surface material of the device; and a step of generating a restored phase map by restoring the phase map using deep learning.The method includes the step of converting the reconstructed phase map into a 3D data value to output 3D data of the device, wherein in the step of generating the reconstructed phase map, the phase map is reconstructed through an AutoEncoder-based deep learning neural network, wherein the phase map is reconstructed through a multivariate AutoEncoder-based reconstruction network, wherein in the step of generating the reconstructed phase map, the phase map is input into the reconstruction network, features are extracted from the input phase map image in the encoder of the reconstruction network to generate a latent vector, and the latent vector is converted back into an image in the decoder of the reconstruction network to generate the reconstructed phase map, wherein after generating the latent vector, a shadow region and a reflection region, which are regions to be reconstructed, are added to the latent vector, and in the step of outputting the 3D data, the reconstructed phase map is converted into a 3D data value using regression analysis, and in the step of acquiring the moiré pattern projection image, a moiré pattern projection image shifted by 0 degrees, a moiré pattern projection image shifted by 90 degrees, a moiré pattern projection image shifted by 180 degrees, A 3D data measurement method characterized by acquiring four moiré pattern projection images shifted by 270 degrees, calculating a phase map in the step of calculating a phase map having values ​​between (-π, +π) by calculating an arctan based on the pixel values ​​of the four moiré pattern projection images, and in the step of extracting, performing a phase unwrapping process on the phase map having a range from 0 to 2kπ (where k is a constant), determining a shadow area if the pixel values ​​of all four moiré pattern projection images are less than a predetermined first reference value, and determining a reflection area if the corresponding pixel value of any one of the four moiré pattern projection images is greater than or equal to a predetermined second reference value. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A 3D data measuring device for measuring 3D data of a device mounted on a PCB, comprising: a projector for projecting a Moire pattern onto the PCB; and a camera for photographing the PCB onto which the Moire pattern is projected by the projector. The computer includes a computer that acquires a moiré pattern projection image, which is a moiré pattern image projected onto the PCB from the camera, calculates a phase map using the moiré pattern projection image, extracts a shadow area, which is an area where a shadow is generated on the PCB due to a device, and a reflection area, which is an area that has a brighter value than the brightness value of the surrounding identical pattern due to the surface material of the device, generates a reconstructed phase map by reconstructing the phase map using deep learning, and converts the reconstructed phase map into a 3D data value to output 3D data of the device, wherein the computer reconstructs the phase map through an autoencoder-based deep learning neural network, thereby reconstructing the phase map through a multivariate autoencoder-based reconstruction network, inputs the phase map into the reconstruction network, generates a reconstructed phase map by extracting features from the input phase map image in the encoder of the reconstruction network to generate a latent vector, and converts the latent vector back into an image in the decoder of the reconstruction network, and wherein the computer... After generating a vector, the shadow area and reflection area, which are the regions to be restored, are added to the latent vector, and the computer converts the restored phase map into a 3D data value using regression analysis, and the projector projects a moiré pattern shifted by 0 degrees, a moiré pattern shifted by 90 degrees, a moiré pattern shifted by 180 degrees, and a moiré pattern shifted by 270 degrees, and the computer projects a projected image of a moiré pattern shifted by 0 degrees, a projected image of a moiré pattern shifted by 90 degrees, and a projected image of a moiré pattern shifted by 180 degrees,A 3D data measurement device characterized by acquiring four moiré pattern projection images shifted by 270 degrees, the computer calculating a phase map having values ​​between (-π, +π) by calculating arctan based on the pixel values ​​of the four moiré pattern projection images, the computer performing a phase unwrapping process on the phase map having a range from 0 to 2kπ (where k is a constant), determining a shadow area if the pixel values ​​of all four moiré pattern projection images are less than a predetermined first reference value, and determining a reflection area if the corresponding pixel value of any one of the four moiré pattern projection images is greater than or equal to a predetermined second reference value. Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 delete Claim 12 delete

Images