Uniform filter for compensating for global error using photometric stereo technology

Abstract

In one exemplary embodiment, photometric stereo techniques are used to apply uniform filtering techniques to reduce or eliminate global defects in scanning components. Images of the component are captured using photometric stereo techniques. Then, the uniform filter obtains the height average value of the adjacent points of each point in the assembly; this actually obscurs the image. The blur processing may be repeated multiple times. The blurred image may then be subtracted from the original image, thereby flattening the image, reducing or eliminating global errors. The filtered image can be used for defect detection, and due to reduction of global errors, the defect detection is more accurate.

Description

Technical Field

[0001] The present application relates generally to defect detection and more particularly to a uniform filter that compensates for global errors of an inspected component using photometric stereo techniques. Background Art

[0002] In manufacturing quality control processes, especially in high-volume production, camera-based imaging systems are often implemented to take pictures of parts for automated defect scanning. In fly-capture applications, when the part is not stopped in front of the camera, it is necessary to capture it with a high-speed camera and provide adequate part illumination to ensure that the image is clear and free of blur, sometimes requiring a combination of multiple different light channels. Summary of the invention

[0003] In one aspect, the present application provides a system, comprising:

[0004] Camera;

[0005] lighting equipment, including a plurality of independently controllable light sources;

[0006] Controller; and

[0007] A computer system comprising at least one hardware processor and a non-transitory computer-readable medium storing instructions, which, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:

[0008] capturing, using a camera, a plurality of images of the assembly, each image of the plurality of images being illuminated by a different light source from among a plurality of independently controllable light sources;

[0009] Applying photometric stereo to the plurality of images to estimate the surface height of each point on the component surface relative to a horizontal line in each of the images, and applying photometric stereo to generate a first version of a two-dimensional height map to indicate the surface height of each point on the component surface;

[0010] Blurring the first version of the two-dimensional height map using a blurring technique to generate a first blurred version of the two-dimensional height map;

[0011] subtracting the first blurred version from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface;

[0012] Repeating the blurring and subtraction operations at least once using the depth map to generate a final version of the two-dimensional depth map;

[0013] measuring depth variations of surface features of the component in a final version of the two-dimensional depth map; and

[0014] The depth variation of the component surface feature is used to determine whether the component surface feature is a defect of the component.

[0015] In another aspect, the present application provides a method, comprising:

[0016] capturing, using a camera, a plurality of images of the assembly, each image of the plurality of images being illuminated by a different light source from among a plurality of independently controllable light sources;

[0017] Applying photometric stereo to the plurality of images to estimate the surface height of each point on the component surface relative to a horizontal line in each of the images, and applying photometric stereo to generate a first version of a two-dimensional height map to indicate the surface height of each point on the component surface;

[0018] Blurring the first version of the two-dimensional height map using a blurring technique to generate a first blurred version of the two-dimensional height map;

[0019] subtracting the first blurred version from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface;

[0020] Repeating the blurring and subtraction operations at least once using the depth map to generate a final version of the two-dimensional depth map;

[0021] measuring depth variations of surface features of the component in a final version of the two-dimensional depth map; and

[0022] The depth variation of the component surface feature is used to determine whether the component surface feature is a defect of the component.

[0023] In a third aspect, the present application provides a non-volatile machine-readable storage medium, on which instructions contained are executed by one or more machines to perform operations, including:

[0024] capturing, using a camera, a plurality of images of the assembly, each image of the plurality of images being illuminated by a different light source from among a plurality of independently controllable light sources;

[0025] Applying photometric stereo to the plurality of images to estimate the surface height of each point on the component surface relative to a horizontal line in each of the images, and applying photometric stereo to generate a first version of a two-dimensional height map to indicate the surface height of each point on the component surface;

[0026] Blurring the first version of the two-dimensional height map using a blurring technique to generate a first blurred version of the two-dimensional height map;

[0027] subtracting the first blurred version from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface;

[0028] Repeating the blurring and subtraction operations at least once using the depth map to generate a final version of the two-dimensional depth map;

[0029] measuring depth variations of surface features of the component in a final version of the two-dimensional depth map; and

[0030] The depth variation of the component surface feature is used to determine whether the component surface feature is a defect of the component.

[0031] It can be seen from the technical solution provided by the present application that the filtered image can be used for defect detection, and due to the reduction of global errors, the defect detection will be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic block diagram of an inspection camera system according to an exemplary embodiment.

[0033] Figure 2 A-2H is a schematic diagram illustrating a photometric stereo technique using a mask according to an exemplary embodiment.

[0034] Figure 3 is a schematic diagram illustrating an assembly having an extremely steep slope according to an exemplary embodiment.

[0035] Figure 4A is a one-dimensional plot representing the height of components along the axis.

[0036] Figure 4B is a one-dimensional graph representing the depth of a component along an axis after performing one iteration of a uniform filtering technique according to an exemplary embodiment.

[0037] Figure 4C is a one-dimensional graph representing the depth of a component along the axis after performing two iterations of a uniform filtering technique according to an exemplary embodiment.

[0038] Figure 4D is a one-dimensional graph representing the depth of a component along the axis after performing three iterations of a uniform filtering technique according to an exemplary embodiment.

[0039] Figure 4E is a one-dimensional graph representing component depth along the axis after performing four iterations of a uniform filtering technique according to an exemplary embodiment.

[0040] Figure 4F is a one-dimensional graph representing component depth along the axis after performing five iterations of a uniform filtering technique according to an exemplary embodiment.

[0041] Figure 5A is a one-dimensional plot representing the height of the curved component along the axis.

[0042] Figure 5B is a one-dimensional graph representing the depth of a curved component along an axis after performing one iteration of a uniform filtering technique according to an exemplary embodiment.

[0043] Figure 5C is a one-dimensional graph representing the depth of a curved component along an axis after performing two iterations of a uniform filtering technique according to an exemplary embodiment.

[0044] Figure 5D is a one-dimensional graph representing the depth of a curved component along an axis after performing three iterations of a uniform filtering technique according to an exemplary embodiment.

[0045] Figure 5E is a one-dimensional graph representing the depth of a curved component along an axis after performing four iterations of a uniform filtering technique according to an exemplary embodiment.

[0046] Fig. 5F is a one-dimensional graph representing the depth of a curved component along an axis after performing five iterations of a uniform filtering technique according to an exemplary embodiment.

[0047] Figure 6 is a schematic diagram illustrating a deep convolutional neural network (DCNN) according to an exemplary embodiment.

[0048] Figure 7 is a flowchart illustrating a method for training a DCNN according to an exemplary embodiment.

[0049] Figure 8 is a flow chart illustrating a method for performing photometric stereo technology with uniform filtering according to an exemplary embodiment.

[0050] Fig. 9 is a block diagram illustrating a software architecture in an example embodiment, which can be installed on any of the above-mentioned devices.

[0051] Fig.10 is a schematic diagram illustrating a machine in the form of a computer system in an example embodiment in which a set of instructions may be executed, causing the machine to perform any of the methods discussed herein. DETAILED DESCRIPTION

[0052] Traditional multi-LED channel current sensing methods usually implement current sensing and real-time feedback control for each individual LED channel circuit. However, this implementation requires a current sensing circuit for each LED channel. Therefore, as the number of LED channels increases, the hardware cost will also increase. In addition, when multiple LED channels are working simultaneously, real-time control of all channels will limit the computing power of the central processing unit (CPU). Expensive parallel computing hardware and software development are required to achieve real-time control of multiple LED channels.

[0053] Furthermore, to assist in high-speed flight capture, a very bright light source is required, and the color temperature and brightness of the light source need to be very consistent. Therefore, a specially designed controller is required to provide accurate current control for each LED channel.

[0054] The performance of inspection cameras can be improved by improving the design of lighting devices to increase the lighting in various scenarios. More specifically, instead of using a single light source - which cannot provide enough light to capture high-quality images to determine surface defects on all surface materials of various components or products - a lighting device with multiple light sources is provided. In addition, the controller of the lighting device with multiple light sources can independently control the multiple light sources, thereby allowing the use of different lighting combinations and sequences to maximize the flexibility of the lighting device in providing sufficient light on different products, components, materials and environments.

[0055] However, managing a variety of different lighting combinations and sequences presents technical challenges, including the fact that a single assembly or production line may have many different components, surfaces may have a variety of potential defects, and there may be a variety of different environmental scenarios that may change the ability of the lighting device to produce an image that can be used for analysis. In addition, as the number of controllable light sources on a lighting device increases, the technical challenges become exponentially more difficult.

[0056] One specific technical issue encountered when using flight capture is that light from various light sources can cause distortions or inaccuracies in the defect detection process. For example, a flat piece of metal may appear to have a dome simply due to the distortion of light coming from an odd angle. This is an example of what is known as a global error, as it is an inaccuracy that affects at least a large portion of the component being inspected. This makes defect detection difficult because systems designed to analyze scanned images of the component will have difficulty distinguishing between global errors and actual local defects. In the example of the flat piece of metal, the system will not be able to distinguish whether the domed appearance is due to lighting or other global errors, or an actual defect in the component itself. Therefore, a solution is needed to reduce or eliminate these global errors to achieve more accurate defect detection.

[0057] Global errors may not even be related to lighting effects or some unexpected or “extra” aspect of the image capture process. In some cases, global errors are characteristics of the component itself that make defect detection more difficult, such as an extremely steep slope in a component plane making it difficult to determine the depth of a hole or protrusion in that plane.

[0058] In one example embodiment, uniform filtering techniques are applied using photometric stereo to reduce or eliminate global defects in a scanned component. An image of the component is taken using photometric stereo. A uniform filter then takes the average of the heights of nearby points at each point on the component. This essentially "blurs" the image. The blurring process can be repeated multiple times. The blurred image is then subtracted from the original image, which essentially smooths the image in a way that reduces or eliminates global errors. The filtered image can be used for defect detection. This method of defect detection is more accurate due to the reduction in global errors.

[0059] Figure 1 1 is a block diagram illustrating an inspection camera system 100 in an example embodiment. The inspection camera system 100 may include a lighting device 102, a controller 104, and a computing system 106. The lighting device 102 includes a plurality of different lights, such as a plurality of LED lights that are individually controlled by the controller 104. This means that the lights can be turned on and off independently, thereby allowing the lighting device 102 to turn on some but not all lights at any given time. In some example embodiments, the brightness of each light can also be independently controlled, so that the lights are not just on or off, but can also have custom brightness / darkness levels.

[0060] The lighting device 102 may also include one or more cameras. Each of these cameras may also be independently controlled to take a picture / capture an image when triggered by a signal. Variables such as exposure time may also be independently controlled.

[0061] In an exemplary embodiment, the lighting device 102 is a light dome. The light dome is used to illuminate a target object, such as a metal casting or other product. The light dome includes a housing containing a plurality of light sources, which will be described in more detail below. In some examples, the light source includes a plurality of LEDs or display screens, which are arranged in a flexible manner to illuminate the target object.

[0062] One or more cameras can be mounted to the reticle via a bracket and capture images of the illuminated target object through the hole in the top of the reticle.

[0063] The controller 104 is an electronic component designed to send signals to the lighting device 102 through one or more channels to control the multiple lights on the lighting device 102.

[0064] The computing system 106 includes various software components running on a computing hardware platform. These components include a sequence generation user interface 108. The sequence generation user interface 108 allows a user to create a sequence consisting of capture configuration information in order, and optionally separated by custom delays. This created sequence can be stored on the controller 104, for example, in a storage component in the controller 104. Upon receiving an external trigger signal, the controller 104 will retrieve the sequence and execute the sequence, which means that the capture configurations in the sequence will be used to control the lighting device 102 according to the parameters defined in each capture configuration and separated by custom delays.

[0065] The external trigger signal may be a hardware trigger (e.g., from a programmable logic controller) or a software trigger (e.g., from an industrial personal computer (PC)). In some exemplary embodiments, one or more trigger signals may come from the factory computer 109. After the trigger signal occurs, the sequence is executed to control the lighting device 102 to light the appropriate light at the appropriate time, and also trigger one or more cameras to take pictures / capture images at the appropriate time.

[0066] The controller 104 sends the trigger signal along with the timestamp to the image processing component 110 on the computing system 106. The controller 104 also receives photos from the lighting device 102, timestamps the photos, and sends the timestamped images to the image processing component 110. The image processing component 110 then encodes a data packet including the photo, capture configuration information, timestamp, camera identification, and other information, and stores the data packets in the first shared memory 112.

[0067] The image analysis component 114 then retrieves this data structure (i.e., the data packet stored in the first shared memory 112 and encoded) and decodes it. The capture configuration information is used to retrieve the artificial intelligence model corresponding to the capture configuration. Each capture configuration has its own artificial intelligence model, although some capture configurations may have multiple artificial intelligence models. In an exemplary embodiment, no artificial intelligence model is shared between two or more capture configurations.

[0068] The AI ​​model uses AI to perform one or more image analysis tasks on the image. These tasks may include, for example, creating a mask for the image that makes one or more inferences about one or more defects of a component in the image.

[0069] The image analysis component 114 then encodes the results of these image analysis tasks into another data structure. This data structure may include, for example, inferred masks, network information, and defect types. This data structure is then stored in the second shared memory 116.

[0070] The CPU 118 then retrieves the data structure from the second shared memory 116 and timestamps the data in the data structure using the information obtained from the programmable logic controller (e.g., part identification, inspection identification, inspection preparation, part start, and part end, etc.) It then packages the sorted data into a data packet and stores it in the third shared memory 120. This data packet may include, for example, part identification, inspection identification, camera identification, images, inference masks, other inference post-processing results, and error codes, etc.

[0071] Then, a user interface component 122 is provided, which can access the data packets in the third shared memory 120 and display part of the data to the user through a graphical user interface. Here, the user can specify the inspection mode (such as manual or automatic), and can also add customer-specific settings, such as image display settings, how and whether to upload images to the cloud environment, etc.

[0072] It should be noted that in some exemplary embodiments, the user interface component 122 and the sequence generation user interface 108 can be combined into a single component. In other words, the sequence definition functionality of the user interface 108 can be combined with the output and setting related functionality of the user interface component 122.

[0073] As previously described, photometric stereo techniques can be used in conjunction with uniformity filters to reduce global errors. Thus, the controller 104 controls the lighting device to take multiple different photos of the component when different lights are triggered (and therefore under different lighting conditions). The lighting may vary when the object is illuminated from multiple different angles. The image processing component 110 then merges the resulting images into a single image of the component.

[0074] More specifically, during the merging of the resulting images, one or more algorithms may be used to estimate the surface normals for each pixel on the surface of the object. A three-dimensional representation of the object's shape may then be reconstructed by integrating the surface normals.

[0075] Photometric stereo assumes that the surface of an object follows Lambertian reflectance, which means that the amount of light reflected from the surface is determined by its orientation relative to the light source and the camera. The camera remains in a fixed position while the light source changes. For each pixel in the image, the light intensity in the different images is analyzed. Due to the change in the lighting angle, the light intensity changes depending on the surface orientation of the object at that pixel. Pixels that are brighter in some images and darker in others indicate changes in surface orientation and curvature.

[0076] By analyzing the changes in light intensity, photometric stereo estimates the surface normal for each pixel on the surface of an object. The surface normal is a vector pointing outward from the surface, indicating the direction the surface is facing. The relationship between the surface normal, light direction, and camera detection is described by Lambert's law, which governs how light reflects off a Lambertian surface.

[0077] The estimated surface normals are integrated to reconstruct a 3D representation of the object's shape. The integration process involves accumulating the normals to estimate the surface depth. The result of the integration is a height map, which represents the object's surface geometry relative to some horizontal plane. Therefore, each surface can be assigned such a height map, which is basically a 2D representation of the height of each point along the surface.

[0078] This height map can then be filtered using the uniform filtering technique described above to reduce the global error.

[0079] Figure 2 A-2H is a schematic diagram illustrating a photometric stereo technique using the mask 200 according to an exemplary embodiment. Figure 2 Each of the figures A-2H shows a different light source of the light mask 200 being illuminated at a different time. It should be noted that these figures only show a two-dimensional representation of the light mask 200 and its effects. In reality, the light mask 200 is three-dimensional, so, for example, light is projected from multiple directions simultaneously.

[0080] Here, component 202 is placed in the center of reticle 200. Here, component 202 is a wedge-shaped component. Camera 204 at the top of reticle 200 captures images of component 202 under illumination from different light sources of reticle 200. Thus, for example, Figure 2 In A, light source 206A is illuminated, while other light sources are not illuminated. Figure 2 In B, light source 206B is illuminated, while other light sources are not illuminated.

[0081] Although Figure 2 Not depicted in A-2H, there may be one or more defects in component 202 for which detection is useful. However, various conditions may make these defects difficult to detect. One example is an extremely steep slope, such as a 45 degree tilt, of component 202 itself. The following is an example of a component with such a tilt.

[0082] Figure 3is a schematic diagram illustrating a component 300 having an extremely steep slope according to an exemplary embodiment. Here, the component 300 has a plane 302 that is at a 45 degree angle relative to the bottom plane of the component 300. There is also a depressed area 304, which may represent a depression or partial hole in the plane 302. The depth of this depressed area 304 may be important in determining whether the depressed area 304 is a defect (e.g., whether it should exist or whether the component 300 is defective). However, the 45 degree angle makes it difficult to calculate a measure of the change in the depth of this depressed area 304, and more specifically, after applying the above-mentioned photometric stereo techniques, it becomes difficult to measure the change in the depth of the depressed area 304 that is reflected in the reconstructed representation.

[0083] Although assembly 300 is three-dimensional, the height and depth maps of a surface such as plane 302 are two-dimensional representations of the depth of each point along the surface. For ease of visualization, the height and depth maps of assembly 300 will be presented in a one-dimensional format in this disclosure. More specifically, the height or depth of each point along axis 306 is presented in subsequent figures.

[0084] Figure 4A is a one-dimensional graph representing the height of component 300 along axis 306. It can be seen that the slope of line 400 is 45 degrees, as in Figure 3 The lowered area 304 can also be seen.

[0085] The uniform filtering technique described above is then applied to the height map. This produces a substantially "blurred" version of line 400. This version of line 400 is then subtracted from line 400, resulting in a flat slope. What remains is the depth of the point, which can be assigned to a point in the surface depth map. Figure 4B is a one-dimensional graph showing the depth of component 300 along axis 306 after performing one iteration of a uniform filtering technique according to an exemplary embodiment. Here, line 450 is essentially a flattened version of line 400, making it easier to accurately measure the depth variation of depressed region 304.

[0086] The uniform filtering technique may be applied multiple times to enhance this flattening effect, making it easier to accurately measure the depth variation of the depressed region 304 . Figure 4C is a one-dimensional graph representing the depth of component 300 along axis 306 after performing two iterations of a uniform filtering technique according to an exemplary embodiment. Figure 4D is a one-dimensional graph representing the depth of a component along the axis after performing three iterations of a uniform filtering technique according to an exemplary embodiment. Figure 4E is a one-dimensional graph representing component depth along the axis after performing four iterations of a uniform filtering technique according to an exemplary embodiment. Figure 4Fis a one-dimensional graph representing component depth along the axis after performing five iterations of a uniform filtering technique according to an exemplary embodiment.

[0087] The uniform filtering technique, at each iteration, first applies one of many different types of blurring techniques before subtracting the result of the blurring technique from the latest version of the surface depth map. One such blurring technique is a box area blur. In a box area blur, a square region of points is taken (i.e., a "box" surrounding the point and the point itself), and the heights of all of these points are averaged and the average is assigned to the point of interest. For example, if the first point in the depth map has a height of 9, and the 8 points surrounding the first point in the 2D depth map all have a height of 3, then the first point would be assigned a height of 3.6 ((9+3+3+3+3+3+3+3+3+3) / 9). Each point's assigned height is then subtracted from the earlier version of the depth map. For example, the latest version of the depth map had a height of 9 for the first point, and the box area blur in this iteration assigned the first point a height of 3.6. The result is that the updated version of the depth map assigns the first point a height of 5.4 (9-3.6).

[0088] It should be appreciated that the box area blur is only one of the blurring techniques that can be used with the uniform filtering technique. Other blurring techniques can be used in other exemplary embodiments, and some of them will be described herein.

[0089] In a 3x3 Gaussian blur, each point is assigned a weight according to a Gaussian distribution. For example, the first point can be assigned a weight of 4, the points surrounding the first point in the plane horizontally and vertically are assigned a weight of 2, and the points in the diagonal direction are assigned a weight of 1. The weighted average is then calculated by multiplying the weights by the height of the corresponding point in the depth map, and the sum of these products is divided by 16 (the sum of all weights). The resulting quotient is then assigned to the first point in the depth map.

[0090] In the 5x5 Gaussian blur, not only the immediately adjacent points are considered, but also all points within 2 points of the target point, essentially expanding the "box" from 3x3 to 5x5. These points are also distributed according to the Gaussian distribution, as follows, the weight of the first point is 36:

[0091]

[0092] The result of the above process is that the depth map is smoothed and the global error is reduced. Another advantage is that the changes in the depth measurement itself become more accurate, especially for surface changes on curved surfaces (e.g., raised or recessed parts). For example, if a circular part of a component appears to have a scratch, it may be difficult to know from which plane to measure the depth change of the "scratch" due to the curved surface curving downward. Using the aforementioned photometric stereo and uniform filtering techniques, the surface is smoothed, making the depth change measurement more accurate. This is in Figures 5A-5F Described in .

[0093] Figure 5A 5 is a graph of a one-dimensional representation of the height of a curved component along an axis. It can be seen that the surface 500 is curved. A concave area 502 can also be seen.

[0094] The height map is then subjected to the uniform filtering technique described above. This produces a substantially "blurred" version of the surface 500. This version of the surface 500 is then subtracted from the surface 500 to flatten the surface 500. The remainder obtained is the depth of that point and can be assigned to the point in the depth map for the surface 500. Figure 5B is a graph of a one-dimensional representation of the depth of a curved component along an axis after applying a primary uniform filtering technique according to an exemplary embodiment. Here, surface 500 is a substantially flat version, making it easier to accurately measure the depth variation of recessed area 502.

[0095] The uniform filtering technique may be applied multiple times to enhance this flattening effect, making it easier to accurately measure the depth variation of the recessed region 502 . Figure 5C is a graph of a one-dimensional representation of the depth of a surface component along an axis after applying a twice uniform filtering technique, according to an exemplary embodiment. Figure 5D is a graph of a one-dimensional representation of the depth of a surface component along an axis after applying a cubic uniform filtering technique, according to an exemplary embodiment. Figure 5E is a graph of a one-dimensional representation of the depth of a surface component along an axis after applying a quartic uniform filtering technique according to an exemplary embodiment. Fig. 5F is a graph of a one-dimensional representation of the depth of a surface component along an axis after applying a quintic uniform filtering technique according to an exemplary embodiment.

[0096] Once the height map has been flattened into a depth map by the techniques described above, depth measurement and defect detection operations can be performed. In particular, the depth variation of any feature of interest in the depth map can be measured, and this measurement can be used to detect defects (e.g., if the depth variation is significantly different from the expected depth variation, such as from an ideal version of the component). One or both of these operations may involve the use of a neural network.

[0097] Images from an image data source can be used to train a neural network by adding one or more labels to those images. This may involve converting the images into different formats that accept those labels. The labels that are added may depend on what the neural network is being trained to do, and may specifically correspond to the classification that the neural network is expected to perform. For example, if an image shows an example of a defect in a particular component, it may be labeled as a defect so that the neural network can learn what a defect in that particular component looks like. This is an example of a positive label. Additionally, if an image shows an example of a component without a defect, it may be labeled as a non-defect so that the neural network can learn what a non-defective component looks like. This is an example of a negative label. While these examples are binary (e.g., positive or negative), in reality the labels may have any number of values, depending on the classification that the neural network is performing.

[0098] In an exemplary embodiment, the neural network is a deep convolutional neural network (DCNN). A DCNN is a machine learning model that can effectively infer nonlinear relationships between homogeneous input fields and desired outputs (which can be classification categories or scalars). DCNN maps inputs to outputs by using a series of artificial neurons called convolutional layers. DCNNs can be trained by providing them with a large amount (e.g., greater than 10,000) of sample data and labels. The goal of its training is to minimize the difference (or "loss") between the model output and the desired output.

[0099] DCNN is not only able to learn to classify images or groups of images, but also learn feature hierarchies by defining multiple layers. The reasoning process involves taking a given input, applying a series of mathematical functions called layers, and calculating these functions on the input data. Each layer extracts features from the output of the previous layer, and all layers are trained jointly. Due to its hierarchical structure, it is called a "deep" convolutional neural network.

[0100] In an exemplary embodiment, five different types of layers are used. The first four layers are convolutional layers, nonlinear layers, pooling layers, and classification layers (although classification layers are just a special case of convolution plus "softmax"). These first four layers can be thought of as a stage, and a DCNN can actually be designed to have any number of these stages. Once all stages are completed, a loss layer is used. Figure 6 6 is a diagram of a DCNN 600 in an exemplary embodiment. Two stages 602A and 602B are shown here.

[0101] Convolutional layers 604A and 604B are the core of DCNN 600. Their parameters include a set of learnable filters that have small receptive fields but extend over the full depth of the input data. During the forward propagation of convolutional layers 604A and 604B, each filter is convolved on the features, computing the dot product between the filter entries and the input, and producing a two-dimensional activation map for that filter. As a result, DCNN 600 learns filters that activate when it sees certain specific types of features.

[0102] The feature maps of all filters can be stacked along the depth dimension to form the complete three-dimensional output of convolutional layers 604A and 604B.

[0103] Convolutional layers 604A and 604B apply a mathematical operation called a convolution. For two spatial dimensions and an indefinite number of non-spatial dimensions (called "channels"), a convolution is defined using the * operator as follows:

[0104]

[0105] Convolutional layers 604A and 604B typically have very small support, such as N=1 and M=1, such that g[n,m,d]=0 when |n|>1 or |m|>1.

[0106] It should be noted that the filters used in convolutional layers 604A and 604B can be activated in the first iteration of DCNN 600 and refined before each additional iteration based on the actions taken by other layers in the previous iteration until a certain error term is below a certain threshold. In an exemplary embodiment, this can be achieved by back-propagation, which will be described in more detail later.

[0107] The output of the convolutional layers 604A and 604B is a set of arrays 606A-606C called feature maps. Each feature map 606A-606C may be generated by a different filter and modified according to various functions in each stage. At the output, each feature map 606A-606C represents a specific feature extracted and conditioned at all locations of the input. Figure 6 The example in is a two-stage system, although one of ordinary skill in the art will recognize that more or fewer stages may be used while still consistent with the present disclosure, and in an exemplary embodiment, the number of stages may be dynamically determined at runtime to optimize results.

[0108] The nonlinear layers 608A and 608B make the DCNN 600 more expressive in revealing the nonlinear relationship between the input and output. Many different nonlinearities can be used in the nonlinear layers, including sigmoid, tanh, and rectified linear functions. For the sake of brevity, an example of nonlinearity will be described here: the rectified linear function. The function is defined as follows:

[0109]

[0110] Pooling layers 610A and 610B are used to reduce the spatial dimension of the input image while retaining some information of the input image. In other words, pooling layers 610A and 610B do not actually perform any learning, i.e., they are a fixed, predefined operation that does not change as training progresses. Instead, they are used for the spatial dimension of the problem. In an exemplary embodiment, a decimation approach can be followed, i.e., one is retained for every N samples in the spatial dimension. In another exemplary embodiment, some local statistics can be used for pooling, such as maximum pooling, defined as follows:

[0111]

[0112] Among them, N=M=2.

[0113] When all stages 602A and 602B are complete, a classification layer 612 is used to classify the image based on the output of the final pooling layer 610B. As described above, the classification layer 612 is actually a special convolutional layer containing a filter designed to produce a score from the three-dimensional output of the final pooling layer 610B. The filter applies a classification function whose weights can be refined in the same way as the weights in the filter function of the ordinary convolutional layers 604A and 604B.

[0114] Back propagation involves calculating the gradient of the loss function (defined below) in the loss layer 614 with respect to the plurality of weights in the DCNN 600. The gradient is then fed back to a method for updating the weights for the next iteration of the DCNN 600 training to attempt to minimize the loss function, which uses different sample data (unless it needs to be repeated, e.g., the sample data runs out). Back propagation uses labeled sample data from a batch of sample data that has passed stages 602A and 602B to calculate the gradient of the loss function for the sample group (although, as described below, the loss function can be dynamically modified to exclude certain samples from consideration).

[0115] Back propagation can include two aspects: propagation and weight update. In terms of propagation, the input images of the training mode are forward propagated through DCNN 600 to generate propagated output activations (i.e., the images pass through stages 602A, 602B). Then, by using the target specified by the training mode, the output activations of DCNN 600 are back propagated to generate increments of all outputs.

[0116] In terms of weight updates, for each weight of each filter, the output delta and input activation are multiplied to get the gradient of the weight, and then a ratio of this gradient is subtracted from the weight. This ratio affects the speed and quality of learning. The higher the ratio, the faster the training, but this comes at the cost of reduced accuracy.

[0117] Therefore, these two aspects, including forward propagation and back propagation through stages 602A, 602B, are repeatedly performed until the error rate is below a certain threshold. Examples of back propagation algorithms compatible with DCNN 600 include, for example, gradient descent.

[0118] The use of backpropagation may depend on whether the combined error of classifying images in the batch of labeled sample data exceeds a preset error threshold. If the combined error is too large, backpropagation should be performed to update and hopefully minimize the error for the next iteration, and the next iteration is performed on subsequent batches of labeled sample data until the combined error does not exceed the threshold.

[0119] As described above, the data can be scored for classification. The DCNN 600 outputs a vector that can be compared to the expected output of some loss function, such as the squared error function:

[0120]

[0121] Figure 7700 is a flowchart illustrating a method 700 for training a DCNN 600 according to an exemplary embodiment. In operation 702, a batch of labeled sample data is input into the DCNN 600, and the current model of the DCNN 600 generates an output. For example, this output may be a score for each labeled sample data. In operation 704, the loss layer 614 of the DCNN 600 calculates the error of the batch of sample data. For example, this error may be a combination of individual errors of each single labeled sample data. In operation 706, the weights in the filters in the convolutional layers 604A and 604B (also including the classification layer 612) are updated according to the loss function defined in the loss layer 614 to minimize the loss. In operation 708, it is determined whether the error has been minimized based on a defined validation set, and this defined validation set may include an error threshold. If the error threshold is exceeded, the error is not minimized, and the process returns to operation 702 to process the next batch of labeled sample images. If the error has been minimized (the threshold has not been exceeded), the DCNN has been trained.

[0122] Figure 8 8 is a flow chart illustrating a method 800 for performing a photometric stereo technique with uniform filtering according to an exemplary embodiment. In operation 802, a plurality of images of a component are captured using a camera, each image being illuminated by a different light source from a plurality of independently controllable light sources. In operation 804, photometric stereo is applied to the plurality of images to estimate a surface height of a component surface at each point along the surface relative to a horizontal line in the plurality of images. The application generates a two-dimensional height map indicating a surface height of the component surface at each point along the surface.

[0123] In operation 806, the first version of the two-dimensional height map is blurred using a blurring technique to generate a first blurred version of the two-dimensional height map. In operation 808, the first blurred version is subtracted from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface. In operation 810, the blurring and subtraction operations are repeated at least once using the depth map of the component surface to generate a final version of the two-dimensional depth map.

[0124] In operation 812, the depth variation of the component surface feature in the final version of the two-dimensional depth map is measured. In operation 814, the depth variation of the component surface feature is used to determine whether the feature is a defect of the component.

[0125] Fig. 9 is a block diagram 900 illustrating a software architecture 902 that may be installed on any of the above-described devices. Fig. 9 This is only one non-limiting example of a software architecture, and it is understood that many other architectures may be implemented to facilitate the functionality described herein. In various embodiments, the software architecture 902 may be composed of: Fig.10The hardware implementation in the machine 1000 of the example system 1000 includes a processor 1010, a memory 1030, and an input / output (I / O) component 1050. In this example architecture, the software architecture 902 can be conceptualized as a stack of layers, each of which can provide specific functionality. For example, the software architecture 902 includes an operating system 904, a library 906, a framework 908, and an application 910. In operation, the application 910 calls an application programming interface (API) 912 through the software stack and receives a message 914 in response to the API call 912, consistent with some embodiments.

[0126] In various implementations, the operating system 904 manages hardware resources and provides common services. The operating system 904 includes, for example, a kernel 920, services 922, and drivers 924. The kernel 920 acts as an abstraction layer between the hardware and other software layers, consistent with certain embodiments. For example, the kernel 920 provides functions such as memory management, processor management (e.g., scheduling), component management, network and security settings. Services 922 can provide other common services to other software layers. Drivers 924 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 924 may include display drivers, camera drivers, or Low-power drivers, Flash drivers, Serial communications drivers (for example, Universal Serial Bus (USB) drivers), drivers, audio drivers, power management drivers, etc.

[0127] In some embodiments, the library 906 provides a low-level common infrastructure utilized by the application 910. The library 906 may include a system library 930 (e.g., a C standard library), which may provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 906 may include an API library 932, such as a media library (e.g., supporting various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio coding, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for rendering 2D and 3D in a graphics context of a display), a database library (e.g., SQLite for providing various relational database functions), a network library (e.g., WebKit for providing web browsing functions), etc. The library 906 may also include various other libraries 934, providing many other APIs for the application 910.

[0128] The framework 908 provides a high-level, general-purpose infrastructure that can be utilized by applications 910. For example, the framework 908 provides various graphical user interface functions, advanced resource management, advanced location services, etc. The framework 908 can provide a wide range of other APIs that applications 910 can utilize, some of which may be specific to a particular operating system 904 or platform.

[0129] In one example implementation, applications 910 include a home screen application 950, a contacts application 952, a browser application 954, a book reader application 956, a location application 958, a media application 960, a messaging application 962, a game application 964, and various other applications, such as third-party applications 966. Applications 910 are programs that perform the functions defined in the program. One or more applications 910 can be created using various programming languages ​​and have a variety of structures, such as object-oriented programming languages ​​(such as Objective-C, Java, or C++) or procedural programming languages ​​(such as C or assembly language). In one specific example, third-party applications 966 (e.g., via ANDROID TM or IOS TM Software Development Kit (SDK) applications developed by entities other than the vendor of a specific platform may be run on mobile operating systems such as iOS. TM ANDROID TM , Mobile software on Phone or other mobile operating systems. In this example, third-party application 966 can call API calls 912 provided by operating system 904 to facilitate the functions described herein.

[0130] Fig.10 A schematic diagram of a machine 1000 is illustrated, in the form of a computer system, in which a set of instructions may be executed to cause the machine 1000 to perform any of the methods discussed herein. Specifically, Fig.10 A schematic diagram of a machine 1000 is shown as an example form of a computer system in which instructions 1016 (e.g., software, a program, an application, an applet, an application, or other executable code) cause the machine 1000 to perform any of the methods discussed herein to be executed. For example, the instructions 1016 may cause the machine 1000 to perform Figure 7 and 8 Additionally or alternatively, instruction 1016 may implement Figure 1-8etc. Instructions 1016 convert a general, unprogrammed machine 1000 into a specific machine 1000 that performs the functions described and illustrated in the manner described. In other embodiments, the machine 1000 may operate as a standalone device, or may be coupled (e.g., networked) to other machines. In a network deployment, the machine 1000 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, but is not limited to, a server computer, a client computer, a PC, a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a mobile phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a network device, a network router, a network switch, a network bridge, or any machine capable of executing instructions 1016, whether sequentially or otherwise, indicating the actions that the machine 1000 should take. Further, while a single machine 1000 is shown, the term "machine" shall also be taken to include a collection of one or more machines 1000 that individually or jointly execute instructions 1016 to perform any of the methodologies discussed herein.

[0131] Machine 1000 may include processor 1010, memory 1030, and I / O components 1050, which may be configured to communicate via bus 1002. In an example embodiment, processor 1010 (e.g., a CPU, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), other processors, or any suitable combination) may include, for example, processor 1012 and processor 1014, which may execute instructions 1016. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions 1016 simultaneously. Although Fig.10 Multiple processors 1010 are shown, and the machine 1000 may include a single-core processor 1012, a multi-core processor 1012, multiple single-core processors 1012, 1014, multiple multi-core processors 1012, 1014, or any combination thereof.

[0132] The memory 1030 may include a main memory 1032, a static memory 1034, and a storage unit 1036, each of which may be accessed by the processor 1010 via the bus 1002. The main memory 1032, the static memory 1034, and the storage unit 1036 store instructions 1016, embodying any of the methods or functions described herein. During execution of these instructions by the machine 1000, the instructions 1016 may also reside, in whole or in part, in the main memory 1032, the static memory 1034, the storage unit 1036, in at least one processor 1010 (e.g., in a cache memory of a processor), or in any suitable combination.

[0133] I / O components 1050 may include various components that receive input, provide output, generate output, transmit information, exchange information, capture measurements, etc. The specific I / O components 1050 included in a particular machine will depend on the type of machine. For example, a portable machine such as a cell phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O components 1050 may include many components not described in the specification. Fig.10 1050 is grouped according to function only to simplify the following discussion, and the grouping is not limiting. In various example embodiments, the I / O component 1050 may include an output component 1052 and an input component 1054. The output component 1052 may include a visual component (e.g., a display such as a plasma display panel (PDP), an LED display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The input component 1054 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a point-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or other pointing instrument), a tactile input component (e.g., a physical button, a touch screen or other tactile input component that provides a touch or touch gesture location and / or force), an audio input component (e.g., a microphone), etc.

[0134] In further example embodiments, the I / O component 1050 may include a biometric component 1056, a motion component 1058, an environmental component 1060, or a position component 1062, among many other components. For example, the biometric component 1056 may include components for detecting expressions (e.g., gestures, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identity recognition (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 1058 may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. Environmental component 1060 may include, for example, an illumination sensor component (e.g., a light meter), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor used to detect hazardous gas concentrations to ensure safety or measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to the surrounding physical environment. Position component 1062 may include a position sensor component (e.g., a global positioning system (GPS) receiver component), an altitude sensor component (e.g., a barometer or barometer that detects air pressure to derive altitude), a direction sensor component (e.g., a magnetometer), etc.

[0135] Communication can be implemented using a variety of technologies. I / O components 1050 may include a communication component 1064 that can couple machine 1000 to network 1080 via coupling 1082 or couple machine 1000 to device 1070 via coupling 1072. For example, communication component 1064 may include a network interface component or other suitable device to interface network 1080. In further examples, communication component 1064 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, Components (e.g. Low power consumption), Components and other communication components to provide communication through other modes. Device 1070 can be another machine or any of a variety of peripheral devices (e.g., coupled via USB).

[0136] In addition, the communication component 1064 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 1064 can include a radio frequency identification (RFID) tag reading component, an NFC smart tag detection component, an optical reading component (e.g., an optical sensor that detects a one-dimensional barcode such as a universal product code (UPC) barcode, a multi-dimensional barcode such as a Quick Response (QR) code, an Aztec code, a Data Matrix, a Dataglyph, a Maxi code, a PDF417, an Ultra code, a UCC RSS-2D barcode, and other optical codes), or an acoustic detection component (e.g., a microphone that recognizes a tag audio signal). In addition, various information can be derived through the communication component 1064, such as a location located via an Internet Protocol (IP) geolocation, a location located via a wireless network ... The location of the signal triangulation positioning, the location of the NFC beacon signal indicating a specific location, etc.

[0137] Various memories (e.g., 1030, 1032, 1034 and / or the memory of processor 1010) and / or storage unit 1036 can store one or more instructions 1016 and data structures (e.g., software) that embody or utilize any of the methods or functions described herein. These instructions (e.g., instructions 1016) when executed by processor 1010 result in the execution of various operations of the disclosed embodiments.

[0138] As used herein, the terms "machine storage medium," "device storage medium," and "computer storage medium" mean the same and may be used interchangeably. The terms refer to a single or multiple storage devices and / or media (e.g., centralized or distributed databases and / or associated caches and servers) that store executable instructions and / or data. Thus, these terms shall include, but are not limited to, solid-state memory, optical and magnetic media, including memory internal or external to the processor. Specific examples include nonvolatile memory, such as semiconductor storage devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), field programmable gate arrays (FPGA), and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium," "computer storage medium," and "device storage medium" specifically exclude carrier waves, modulated data signals, and other such media, at least some of which fall within the category of "signal media" discussed below.

[0139]

[00104] In various example embodiments, one or more portions of network 1080 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of a public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, 1080 or a portion thereof may include a wireless or cellular network, and coupling 1082 may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or another type of cellular or wireless coupling. In this example, coupling 1082 may implement various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.

[0140] Instructions 1016 may be transmitted or received via network 1080 using a transmission medium through a network interface device (e.g., a network interface component included in communication component 1064) and utilizing a well-known transmission protocol (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 1016 may also be transmitted or received to device 1070 using a transmission medium via coupling 1072 (e.g., a point-to-point coupling). The terms "transmission medium" and "signal medium" have the same meaning and may be used interchangeably. The terms "transmission medium" and "signal medium" shall include any intangible medium capable of storing, encoding, or carrying instructions 1016 for execution by machine 1000, including digital or analog communication signals or other intangible media to facilitate the communication of such software. Thus, the terms "transmission medium" and "signal medium" shall include any form of modulated data signals, carrier waves, and the like. The term "modulated data signal" refers to a signal whose one or more characteristics are set or changed in some manner to encode information in the signal.

[0141] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same and are used interchangeably. The term definitions include both machine storage media and transmission media. Thus, these terms include both storage devices / medium and carrier / modulated data signals.

Claims

1. A system comprising: Camera; lighting equipment, including a plurality of independently controllable light sources; Controller; as well as A computer system comprising at least one hardware processor and a non-transitory computer-readable medium storing instructions, which, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising: capturing, using a camera, a plurality of images of the assembly, each image of the plurality of images being illuminated by a different light source from among a plurality of independently controllable light sources; Applying photometric stereo to the plurality of images to estimate the surface height of each point on the component surface relative to a horizontal line in each of the images, and applying photometric stereo to generate a first version of a two-dimensional height map to indicate the surface height of each point on the component surface; Blurring the first version of the two-dimensional height map using a blurring technique to generate a first blurred version of the two-dimensional height map; subtracting the first blurred version from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface; Repeating the blurring and subtraction operations at least once using the depth map to generate a final version of the two-dimensional depth map; measuring depth variations of surface features of the component in a final version of the two-dimensional depth map; and The depth variation of the component surface feature is used to determine whether the component surface feature is a defect of the component.

2. The system according to claim 1, characterized in that The blurring technique is box area blurring.

3. The system according to claim 1, characterized in that The blurring technique is Gaussian blurring.

4. The system according to claim 1, characterized in that Measuring the depth variation of the component surface features in the final version of the two-dimensional depth map and / or using the depth variation of the component surface features includes inputting the final version of the two-dimensional height map into a convolutional neural network trained to predict the classification of surface features.

5. The system according to claim 4, characterized in that The convolutional neural network is trained in the following way: accessing a plurality of images from a first image data source, each image of the plurality of images having n color channels; as well as The convolutional neural network is trained using the multiple images and multiple labels, each label of the multiple labels corresponding to a classification.

6. The system according to claim 1, characterized in that The component surface is curved.

7. The system according to claim 1, characterized in that The inclination angle of the component surface relative to the horizontal line is greater than or equal to 45 degrees.

8. A method comprising: capturing, using a camera, a plurality of images of the assembly, each image of the plurality of images being illuminated by a different light source from among a plurality of independently controllable light sources; Applying photometric stereo to the plurality of images to estimate the surface height of each point on the component surface relative to a horizontal line in each of the images, and applying photometric stereo to generate a first version of a two-dimensional height map to indicate the surface height of each point on the component surface; Blurring the first version of the two-dimensional height map using a blurring technique to generate a first blurred version of the two-dimensional height map; subtracting the first blurred version from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface; Repeating the blurring and subtraction operations at least once using the depth map to generate a final version of the two-dimensional depth map; measuring depth variations of surface features of the component in a final version of the two-dimensional depth map; as well as The depth variation of the component surface feature is used to determine whether the component surface feature is a defect of the component.

9. The method according to claim 8, characterized in that The blurring technique is box area blurring.

10. The method according to claim 8, characterized in that The blurring technique is Gaussian blurring.

11. The method according to claim 8, characterized in that Measuring the depth variation of the component surface features in the final version of the two-dimensional depth map and / or using the depth variation of the component surface features includes inputting the final version of the two-dimensional height map into a convolutional neural network trained to predict the classification of surface features.

12. The method according to claim 11, characterized in that The convolutional neural network is trained in the following way: accessing a plurality of images from a first image data source, each image of the plurality of images having n color channels; as well as The convolutional neural network is trained using the multiple images and multiple labels, each label of the multiple labels corresponding to a classification.

13. The method according to claim 8, characterized in that The component surface is curved.

14. The method according to claim 8, characterized in that The inclination angle of the component surface relative to the horizontal line is greater than or equal to 45 degrees.

15. A non-volatile machine-readable storage medium containing instructions that are executed by one or more machines to perform operations comprising: capturing, using a camera, a plurality of images of the assembly, each image of the plurality of images being illuminated by a different light source from among a plurality of independently controllable light sources; Applying photometric stereo to the plurality of images to estimate the surface height of each point on the component surface relative to a horizontal line in each of the images, and applying photometric stereo to generate a first version of a two-dimensional height map to indicate the surface height of each point on the component surface; Blurring the first version of the two-dimensional height map using a blurring technique to generate a first blurred version of the two-dimensional height map; subtracting the first blurred version from the first version of the two-dimensional height map to obtain a two-dimensional depth map of the component surface; Repeating the blurring and subtraction operations at least once using the depth map to generate a final version of the two-dimensional depth map; measuring depth variations of surface features of the component in a final version of the two-dimensional depth map; as well as The depth variation of the component surface feature is used to determine whether the component surface feature is a defect of the component.

16. The non-volatile machine-readable storage medium according to claim 15, wherein: The blurring technique is box area blurring.

17. The non-volatile machine-readable storage medium according to claim 15, wherein: The blurring technique is Gaussian blurring.

18. The non-volatile machine-readable storage medium according to claim 15, wherein: Measuring the depth variation of the component surface features in the final version of the two-dimensional depth map and / or using the depth variation of the component surface features includes inputting the final version of the two-dimensional height map into a convolutional neural network trained to predict the classification of surface features.

19. The non-volatile machine-readable storage medium according to claim 18, wherein: The convolutional neural network is trained in the following way: accessing a plurality of images from a first image data source, each image of the plurality of images having n color channels; as well as The convolutional neural network is trained using the multiple images and multiple labels, each label of the multiple labels corresponding to a classification.

20. The non-volatile machine-readable storage medium of claim 15, wherein: The component surface is curved.

Images