Showerhead detection based on machine vision

Abstract

Disclosed examples relate to machine vision based showerhead detection systems and methods. One example provides a computing system for machine vision based inspection of a showerhead, the computing system including a processor and a memory. The processor is configured to execute the program using part of the memory to receive image data of the showerhead, identify a target feature in the image data of the showerhead, generate a cropped image based on the target feature identified using the image data of the showerhead, generate a magnified image by magnifying the cropped image, and display the magnified image on the showerhead. And performing one or more measurements on the enlarged image.

Description

Background Art

[0001] Processing chambers are used to perform various semiconductor processes on substrates such as silicon (Si) wafers. Exemplary processes include deposition and etching processes. In such processes, a showerhead assembly is utilized to deliver process gases and / or radio frequency (RF) power into the chamber. The showerhead typically includes a large number of holes designed for uniform delivery of process gases. The size of these holes can be on the order of hundreds of microns, depending on the application. Because semiconductor processing typically involves small-scale manipulation of materials, the precise manufacture of these holes and the maintenance of these holes between processes can significantly impact the yield of such processes. Summary of the Invention

[0002] This Summary is provided to introduce a selection of concepts in a simplified form that are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all of the disadvantages noted in any part of this disclosure.

[0003] Disclosed examples relate to a machine vision-based printhead inspection system and method. One example provides a computing system for machine vision-based inspection of printheads, the computing system comprising a processor and memory. The processor is configured to use a portion of the memory to execute a program to receive image data of the printhead, identify a feature of interest in the image data of the printhead, generate a cropped image based on the identified feature of interest using the image data of the printhead, generate an upscaled image by upscaling the cropped image, and perform one or more measurements on the upscaled image.

[0004] In some such examples, the showerhead additionally or alternatively includes a face plate and a back plate joined together.

[0005] In some such examples, generating the cropped image includes generating a plurality of cropped images, generating the magnified image includes generating a plurality of magnified images by magnifying the plurality of cropped images, and performing one or more measurements on the magnified images includes performing one or more measurements on each of the plurality of magnified images.

[0006] In some such examples, a generative adversarial network is additionally or alternatively used to generate multiple upscaled images.

[0007] In some such examples, the image data additionally or alternatively includes multiple initial images, where each initial image includes a different portion of the printhead.

[0008] In some such examples, the target feature additionally or alternatively comprises a showerhead orifice.

[0009] In some such examples, the measurements additionally or alternatively include circularity measurements including a maximum inscribed circle and a minimum circumscribed circle.

[0010] In some such examples, the processor is further configured to additionally or alternatively provide a detection result based on the performed measurements, wherein the detection result describes a burr detected in the showerhead aperture.

[0011] In some such examples, the target feature additionally or alternatively comprises a chamfer of a showerhead orifice.

[0012] In some such examples, the plurality of cropped images additionally or alternatively include images of a majority of the nozzle aperture of the nozzle.

[0013] Another example provides a method for machine vision-based inspection of a printhead. The method includes receiving image data of the printhead, identifying target features in the image data of the printhead, generating a plurality of cropped images based on the target features identified using the image data of the printhead, generating a plurality of magnified images by magnifying the plurality of cropped images, and performing measurements on the plurality of magnified images.

[0014] In some such examples, the showerhead additionally or alternatively includes a face plate and a back plate joined together.

[0015] In some such examples, image data is additionally or alternatively received from a remote device.

[0016] In some such examples, a generative adversarial network is additionally or alternatively used to generate multiple upscaled images.

[0017] In some such examples, the method additionally or alternatively further includes generating a detection result based on the performed measurements and transmitting the detection result to a remote device, wherein the detection result describes a burr detected in the showerhead orifice.

[0018] Another example provides a printhead inspection system for machine vision-based inspection of printheads. The printhead inspection system includes a camera system, the camera system including a camera, a front lighting system, and a platform for mounting the printhead. The printhead inspection system further includes a computing system and a controller configured to control the camera system to acquire image data of the printhead. The controller is further configured to control the computing system to receive the image data of the printhead from the camera system, identify target features in the image data of the printhead, generate multiple cropped images based on the target features identified using the image data of the printhead, generate multiple magnified images by magnifying the multiple cropped images, and perform measurements on the multiple magnified images.

[0019] In some such examples, the showerhead additionally or alternatively includes a face plate and a back plate joined together.

[0020] In some such examples, the camera system additionally or alternatively further includes a telecentric lens.

[0021] In some such examples, a generative adversarial network is additionally or alternatively used to generate multiple upscaled images.

[0022] In some such examples, the controller is additionally or alternatively further configured to control the computing system to generate a detection result based on the performed measurements, wherein the detection result describes a burr detected in the showerhead aperture. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram showing an exemplary computing system for inspection of a spray head.

[0024] Figure 2 An exemplary imaging system for inspection of a spray head is shown.

[0025] Figure 3A and 3B Shows the conversion and analysis of image data during an exemplary printhead inspection.

[0026] Figure 4 Shows an exemplary sprinkler head detection model.

[0027] Figure 5 A flow chart showing an exemplary method for inspecting a spray head.

[0028] Figure 6 A non-limiting embodiment of a computing system that may implement one or more of the above-described methods and processes is schematically shown. DETAILED DESCRIPTION

[0029] The term "atomic layer deposition (ALD)" generally refers to a process in which a film is formed on a substrate as one or more individual layers by sequentially adsorbing precursors onto the substrate and reacting the adsorbed precursors to form film layers. Examples of ALD processes include plasma-enhanced ALD (PEALD), remote plasma-enhanced ALD (RPEALD), and thermal ALD (TALD).

[0030] The terms "backlight" and "backlighting" generally refer to the process of lighting a subject from the back, with the subject between the light source and the camera.

[0031] The term "burr" generally refers to a rough edge or area present on a surface.

[0032] The term "chamfer" generally refers to a transition edge between two surfaces. For example, the chamfer of a showerhead hole generally refers to a border connecting the inner surface of the showerhead hole and the surface of the showerhead faceplate.

[0033] The term "chemical vapor deposition (CVD)" generally refers to a process in which a solid film is formed on a substrate by directing a continuous flow of one or more precursor gases onto the substrate surface under conditions configured to cause chemical conversion of the precursor gases to a solid phase film.

[0034] The term "contour" generally refers to the outline of a curved or irregular shape.

[0035] The terms "cropping" and "cropping" generally refer to removing unwanted portions from content media.

[0036] The term "deposition" generally refers to the formation of a film of material on a substrate.

[0037] The term "etching" and variations thereof generally refer to the removal of material from a substrate.

[0038] The terms "frontlighting" and "front illumination" generally refer to the process of lighting a subject from the front, where the light source is located on the same side of the subject as the camera and projects light onto the subject.

[0039] The term "generative adversarial network" generally refers to a type of machine learning architecture that uses a generative model and a discriminator model to generate new data. Exemplary generative adversarial networks include the Super Resolution Generative Adversarial Network (SRGAN) for upscaling input images and movies.

[0040] The term "least squares circle" (LSC) generally refers to a circle that separates a contour such that the sum of the total area inside and outside the circle is equal.

[0041] The term "machine learning model" generally refers to a computer program that is configured to recognize patterns in data and / or make predictions without being explicitly programmed to do so.

[0042] The term "maximum inscribed circle" (MIC) generally refers to the largest circle that can be inscribed within a contour.

[0043] The term "minimum circumscribed circle" (MCC) generally refers to the smallest circle that encloses a contour.

[0044] The term "minimum zone circle" (MZC) generally refers to the area between the MIC and the MCC.

[0045] The term "pedestal" generally refers to a structure configured to hold a substrate in a processing chamber.

[0046] The term "processing chamber" generally refers to an enclosure in which a chemical and / or physical process is performed on a substrate. Exemplary chemical and / or physical processes include ALD, CVD, and reactive ion etching processes.

[0047] The term "process gas" generally refers to gaseous chemicals used during processes performed in a processing chamber.

[0048] The term "reactive ion etching" (RIE) generally refers to an etching process in which a chemically reactive plasma is used to remove material from a substrate.

[0049] The term "showerhead" generally refers to a process gas outlet comprising a plurality of holes distributed over a certain area. The showerhead may include a face plate configured to face toward a susceptor in a process chamber. The showerhead may also include a back plate configured to face away from the susceptor in the process chamber.

[0050] The term "showerhead aperture" generally refers to an opening in a showerhead for delivering a process gas.

[0051] The term "showerhead pedestal" generally refers to a pedestal that includes one or more process gas outlets configured to expose a substrate surface facing the pedestal to a process gas. Showerhead pedestals can also be used to perform CVD processes on the backside of a substrate.

[0052] The terms "upscaling" and "upscaling" generally refer to converting content media to higher resolution content media. Exemplary content media include audio data, image data, and video data.

[0053] The manufacture and maintenance of showerheads used in processing chambers includes ensuring that the showerheads meet specifications. One aspect includes confirming that the orifices through which the process gases are transported do not deviate from their specifications. Deviations from their specifications can affect the performance of the showerhead and reduce the yield of the process in which the showerhead is implemented. Examples of such processes include various CVD, ALD, and RIE processes. Deviations in the showerhead can occur at various points in the useful life of the showerhead, including during manufacturing, assembly, and operation. For example, different manufacturers using different tools and machinery may produce showerheads that are slightly different from each other. During the assembly / joining process, the showerhead orifice may become deformed. During exposure to etching processes, such as during showerhead cleaning / maintenance, the showerhead orifice may become too large. The ability to distinguish these differences can assist in troubleshooting and determining corrective actions.

[0054] Another issue includes printhead defects that can affect the performance of the printhead. Examples of these defects include printhead holes that contain burrs or are clogged. Such defects can occur within the printhead hole or on the chamfer of the printhead hole. Another example includes cracks in the printhead. Inspection of the printheads to locate these defects and the above-mentioned deviations can be difficult and costly in terms of time and resources. This is because a single printhead can include thousands of holes with microscale diameters. Therefore, quality control methods often include sample inspection, where only a subset of the printheads and / or holes are inspected to infer the quality of the entire batch.

[0055] There are several different methods for printhead inspection, including both destructive and non-destructive methods. Non-destructive inspection methods can involve the use of contact or optical coordinate measuring machines (CMMs). Contact CMM inspection is time consuming. Therefore, it is difficult to perform contact CMM inspection on every hole of every printhead in a cost-effective manner. Optical CMMs use backlighting technology and are therefore incompatible with assembled, bonded printheads. The inability to perform final inspection after the assembly of the printhead prevents confirmation that the hole has not been deformed or offset during the bonding process. Typically, measurements of the bonded printhead are performed by destructively cutting open the printhead for inspection. Such methods can be used to provide sample data points to infer the quality of a given batch of printheads. However, given the cost associated with implementing defective printheads, the confidence level of such quality assurance methods may not be satisfactory for general use cases.

[0056] In view of the above observations, an example of machine vision-based nozzle inspection is provided. In some aspects, a detection model is implemented using machine vision technology in combination with machine learning to employ an inspection process capable of detecting defects in each hole on the nozzle. The model can be implemented as an online automatic optical inspection system using the above-mentioned technology. Such an inspection system can provide a high-speed and efficient process for performing a complete inspection of the nozzle after bonding in a non-destructive manner. In some implementations, a complete inspection of the nozzle can be performed in minutes compared to previous methods that may take hours. The disclosed examples can provide a more consistent quality control process for an entire batch of nozzles and can provide more accurate inspections than using manual methods. The disclosed examples can also allow the process in which the nozzle is implemented to have lower yield loss.

[0057] The inspection model can be applied to inspect every hole in the nozzle while maintaining efficiency in terms of speed and resources. Different types of analysis can be performed in a more efficient manner than previous methods. The inspection model can be applied to determine the presence of various defects, such as whether the hole is too large after use in the etching process, whether the hole is blocked, whether the nozzle has cracks, etc. Such a model enables operators to make certain assessments that would be difficult to make when using previous sampling methods. For example, an effective method for inspecting nozzles after bonding enables operators to quickly compare nozzles to determine the one that is most suitable for a given application, such as selecting the nozzle with the fewest defects for a more critical application. The ability to quickly inspect nozzles can also prevent the premature scrapping of functional nozzles. This can reduce the life cycle cost of implementing the nozzles. Another advantage of the inspection model includes the ability to detect defects and flaws in the nozzle panel before the panel and backplane are assembled and bonded. This allows operators to avoid wasting time and resources on bonding defective nozzle components.

[0058] Reference Figure 1 , which depicts a schematic diagram of an exemplary computing system 100 for inspection of a sprinkler. Although the description herein describes inspection processes for a sprinkler, it should be understood that such processes can be applied to similar components. Examples include an unengaged sprinkler faceplate and a sprinkler base. The computing system 100 includes a computing device 102, which further includes a processor 104 (e.g., one or more central processing units, or "CPUs"), volatile memory 105, non-volatile memory 106, and input / output (I / O) modules 107. The different components are operably coupled to each other. The non-volatile memory 106 stores a sprinkler inspection program 108, which contains instructions for the various software modules described herein to be executed by the processor 104.

[0059] Upon execution by processor 104, the instructions stored in printhead detection program 108 cause processor 104 to initiate a printhead detection process. The printhead detection process includes extracting image data 112 from imaging system 113. Image data 112 may include one or more images of the printhead. In some implementations, image data 112 includes a single image of the entire printhead. In other implementations, image data 112 includes multiple images, each image mapping a different portion of the printhead.

[0060] The type of imaging system 113 from which the image data 112 is extracted may depend on the application. In some implementations, the imaging system 113 is a remote camera device. For example, a cloud-based inspection system may be implemented in which image data acquired at a facility containing sprinklers is transmitted to a remote system (e.g., computing system 100), which provides inspection results in response. In other implementations, the imaging system 113 is locally connected to the computing system 100 as part of an online inspection system. The imaging system 113 may be implemented using various types of cameras, including but not limited to charge coupled device (CCD) cameras. In some implementations, the imaging system 113 is a handheld or other mobile camera. Examples include smartphone cameras and wearable device cameras, such as cameras integrated into head-mounted devices.

[0061] The printhead inspection program 108 includes an image cropping module 114 that receives the extracted image data 112 as input and generates a plurality of cropped images 116 corresponding to target features on the printhead. For example, the image cropping module 114 may receive image data 112 comprising an initial image of the printhead (or a portion of the printhead) and generate cropped images 116 of each of the orifices of the printhead. Various machine vision techniques may be used to perform the image cropping process. Exemplary techniques include feature detection / extraction algorithms for isolating target features. In some implementations, the image cropping module 114 applies an edge detection algorithm to the image data 112 to identify the orifices in the printhead. In some implementations, the image cropping module 114 applies a circular Hough transform (CHT) algorithm. The identified orifices may then be cropped to generate the plurality of cropped images 116. Such techniques may also be used to identify other target features, including, but not limited to, the chamfers of the printhead orifices. As will be readily appreciated, different feature extraction techniques may be applied depending on the target features. For example, an edge detection algorithm may be applied to identify cracks on the surface of the showerhead.

[0062] Additionally or alternatively, a machine learning model may be applied to identify target features. As an example, a machine learning model such as a feedforward neural network (e.g., a convolutional neural network) may be used to identify target features, such as a nozzle aperture and the chamfer of a nozzle aperture. Such a neural network may be trained using labeled training data containing images of nozzles with target features and nozzles without such features. Furthermore, the model may be trained to identify multiple different target features. Such a neural network may be trained using any suitable training method. As an example, a stochastic gradient descent algorithm with backpropagation may be used to train a feedforward neural network. Any suitable loss function may be optimized in such a process. Examples include mean squared error and mean absolute error. Other types of machine learning models may be used. For example, an unsupervised machine learning model may be used to identify target features. Exemplary training methods for such machine learning models include using unlabeled training data containing images of nozzles, different target features of nozzles, and / or different defects of interest in nozzles. Algorithms employed by unsupervised machine learning models attempt to identify patterns within the unlabeled training data to classify data within different groups (e.g., different target features).

[0063] The printhead inspection program 108 further includes an image upscaling module 118 for generating a plurality of magnified images 120 using the plurality of cropped images 116. After the cropping process, the plurality of cropped images 116 have a relatively low resolution. For example, cropping thousands of individual holes from an initial image of one or more printheads may produce cropped images with a resolution significantly smaller than the resolution of the initial image(s). In some implementations, the cropped images have a resolution of less than or equal to 128×128 pixels. In further implementations, the cropped images have a resolution of 32×32 pixels. To discern information from images of this quality, the upscaling module 118 may be implemented to generate images with higher resolution and finer detail. The upscaling images may have any suitable resolution. In some implementations, the upscaling images have a resolution of at least 128×128 pixels. Different types of upscaling techniques may be implemented. In some implementations, the upscaling module 118 utilizes a machine learning model to perform the upscaling. For example, a generative adversarial network, such as a super-resolution generative adversarial network, may be implemented to upscaling the plurality of cropped images 116. Other upscaling techniques such as interpolation-based upscaling techniques may also be applied.

[0064] The printhead inspection program 108 also includes a measurement module 122 for determining various measurement fits using the plurality of magnified images 120. The measurement results can be used to determine whether the target feature meets predetermined inspection criteria. In certain implementations, measurements are performed to determine various dimensions, including but not limited to the area, roundness, center, and diameter of the printhead holes. Based on these measurement results, other details such as defects and errors can be determined. For example, the measurement results can be used to identify surface cracks, porosity, missing holes, holes with roundness errors, holes with position errors, and holes with defects such as burrs and blockages. The measurement results can also be used to display the distribution of deviations between the desired holes and the holes as manufactured.

[0065] The measurement module 122 can perform various measurements by applying a fitting algorithm to each magnified image 120. Different fitting algorithms can be applied depending on the target feature. In an application targeting the detection of holes in a printhead, the measurement module 122 can be implemented to perform multiple roundness measurements on each magnified image 120 to determine whether the hole deviates from predetermined specifications. The roundness measurements can then be used to determine the presence of certain defects, such as burrs and blockages. An exemplary technique for performing such measurements includes using an edge detection algorithm (e.g., a Canny edge detector) to identify the outline of the hole. Various roundness measurements can be defined from the hole outline. Roundness measurements may include, but are not limited to, the maximum inscribed circle, the minimum circumscribed circle, the minimum annular circle, and the least squares circle. It will be readily appreciated that different measurements can be performed depending on the target feature and / or defect to be identified. For example, an edge detection algorithm can be applied to generate edge profiles that can be used to identify cracks on the surface of a printhead. In such an implementation, circular profiles corresponding to the edges of the printhead and the printhead hole can be filtered out. The remaining profiles can be detected to identify defects (e.g., cracks) on the surface of the printhead.

[0066] The measurement module 122 outputs a detection result 124 based on the determined measurement value. In a cloud-based detection system, the detection result 124 can be transmitted to an appropriate system, such as the system from which the initial image data 112 was received. The detection result 124 can include data describing various details of a given target feature, such as the diameter of a hole, manufacturing offsets, and defects found. In some implementations, the detection result 124 includes data describing whether the target feature in the plurality of magnified images 120 meets predetermined detection criteria. For example, the detection result 124 may include data indicating whether a given hole in the inspected nozzle meets a roundness criterion. Exemplary criteria may include whether the minimum circumscribed circle of the hole is within a predetermined tolerance. Another example includes whether the minimum annular circle of the hole is below a predetermined threshold.

[0067] In some implementations, the inspection results 124 include data describing the type of defect found in a given printhead hole. This information can be derived from the measurement results. For example, a hole with similar minimum circumscribed circle and least squares circle measurements but a relatively small maximum inscribed circle can indicate a local defect, which in turn can indicate the presence of a burr. A relatively large difference in the minimum circumscribed circle and maximum inscribed circle measurements can indicate a clogged or irregularly shaped hole.

[0068] A machine learning model may also be used to determine the detection result 124. Different machine learning models may be implemented, including supervised and unsupervised models. For example, machine learning models of various feedforward neural networks (e.g., convolutional neural networks) may be used to identify the presence of defects such as cracks in a printhead. In some implementations, an artificial neural network is employed to utilize the measurement results to predict the presence and / or type of defect. Examples of machine learning models are described in more detail above.

[0069] The above reference Figure 1 The exemplary computing system and various software modules described provide a processing pipeline for transmitting and analyzing image data of a printhead to provide an inspection result. Different system configurations can be implemented, depending on the application. For example, in a localized printhead inspection system, a controller can be implemented to control an imaging system 113 to acquire image data 112 of a printhead. The controller can also be configured to control the computing system 102 to execute various steps associated with the printhead inspection program 108 and the software modules of the printhead inspection program 108.

[0070] Various types of camera systems and setups may be used to implement the imaging system 113 described above. Figure 2An exemplary imaging system 200 for inspecting printheads is shown. Imaging system 200 includes a platform 202 for mounting the printhead and associated components. In the depicted example, platform 202 mounts a printhead 204. As shown, the faceplate of printhead 204 faces camera 206, i.e., the printhead aperture faces camera 206. Platform 202 is movable, allowing the printhead 204 to be positioned in three dimensions relative to camera 206 and a light source. In some implementations, platform 202 is fixed in place. Additionally or alternatively, camera 206 can be mounted on a three-axis gantry system. In such a system, platform 202 and / or camera 206 can be positioned to perform a raster scan of the printhead 204 and acquire multiple segmented images. In some implementations, the raster scan is performed automatically. Imaging system 200 can be configured to utilize front illumination. While front illumination may result in lower quality images compared to back illumination, such a system enables non-destructive optical inspection of apertures in post-bonding printheads. Various types of cameras can be used, such as visible light cameras. In some implementations, camera 206 is a CCD camera. In some implementations, camera 206 can include a telecentric lens.

[0071] Reference Figure 3A and 3B , depicting the conversion and analysis of image data in an exemplary printhead inspection process 300 . Figure 3A and 3B The images depicted in the drawings are not to scale and are depicted as such for convenience. For example, a showerhead may include hundreds or thousands of holes whose size relative to the overall size of the showerhead is much smaller than shown in the drawings.

[0072] Figure 3A This figure shows the feature extraction process in the image data processing flow. Extracting target features reduces the amount of image data to be analyzed in subsequent steps. This can speed up the inspection process 300. In the exemplary inspection process 300, the target feature is a nozzle hole. In some implementations, the target feature is the chamfer of the nozzle hole.

[0073] At step 302, the process includes dividing the nozzle to be inspected into a plurality of sections. This then also divides the nozzle hole into different parts. Figure 3A A spray head divided into fifteen segments is shown, but the process can be implemented with any number of segments. In some cases, the number of segments depends on the type and quality of the camera used.

[0074] In step 304, for example, using an imaging system (e.g. Figure 2 ) obtains a plurality of initial images from the nozzle. The plurality of initial images are mapped to the plurality of segments. Step 304 displays an exemplary initial image.

[0075] In step 306, a target feature is identified in the initial image. In the exemplary detection process 300, the target feature is a printhead hole. Various machine vision techniques can be used to identify the printhead hole in the initial image. For example, a circular Hough transform can be applied to detect the location of the printhead hole. Additionally or alternatively, a machine learning model can be applied to identify the target feature. Examples of machine vision techniques and machine learning models are described in more detail above.

[0076] At step 308, the detected nozzle orifices are cropped from the initial image to produce a plurality of cropped images. Step 308 shows exemplary cropped images. Each of the cropped images has a lower resolution than the initial image. Generally speaking, given the diameter of the nozzle orifice, the cropped images may have a much smaller resolution than the initial image. In some implementations, the cropped images have a resolution of at least 32×32 pixels. It will be readily appreciated that the resolution of the cropped images depicting the target features may be any resolution, depending on the imaging system utilized and the resolution of the images acquired from such imaging system. For example, in some implementations, the initial image has a resolution of 4056×3040 pixels and the cropped images have a resolution of 32×32 pixels.

[0077] Reference Figure 3B , depicting the steps of magnifying the cropped image and performing measurements. In step 310, a plurality of cropped images are magnified to generate a plurality of magnified images. The magnification process may be performed using various methods. In certain implementations, a super-resolution generative adversarial network is used to magnify the cropped image. SRGAN may be implemented to magnify a low-resolution image into a high-resolution image. In addition to magnifying the image, SRGAN may also restore finer details and textures. In certain implementations, the implemented SRGAN may achieve magnification up to a factor of four. SRGAN may be implemented in various ways. For example, the implemented SRGAN may be a model trained using high-resolution images of a nozzle orifice as training data. The training process may include magnifying a low-resolution image of the nozzle orifice and comparing the result with a corresponding high-resolution image of the nozzle orifice.

[0078] In the exemplary detection process 300, the SRGAN upscales each cropped image from a resolution of 32×32 pixels to a resolution of 128×128 pixels. Different resolutions can be achieved, depending on the application. It will be readily appreciated that other upscaling techniques can also be utilized. In some implementations, an interpolation-based upscaling technique is utilized. In some implementations, a convolutional neural network (CNN) is used to perform the upscaling. In such cases, the resulting upscaled image may lack finer details than would be achieved using the SRGAN.

[0079] At step 312, several measurements are performed on the magnified image. In the exemplary inspection process 300, machine vision techniques are utilized to determine various roundness measurements that can provide details such as the size of a given printhead hole and the presence or absence of defects. The roundness measurements performed may include MIC, MCC, MZC, and LSC measurements. Step 312 depicts performing MCC, LSC, and MIC measurements on an exemplary magnified image. Based on these measurements, the inspection process 300 may output an inspection result describing details regarding the printhead hole. Additionally or alternatively, a machine learning model may be applied to output an inspection result based on the magnified image and / or the measurements performed.

[0080] Although the exemplary inspection process 300 depicts a process flow for the inspection of a showerhead hole, such a model can be similarly configured for general inspection of showerheads and related components. For example, the inspection process 300 can be similarly applied to the inspection of cracks in showerheads and showerhead bases.

[0081] Figure 4 A generalized sprinkler inspection model 400 is shown schematically depicting the conversion and analysis of image data 402 of sprinklers and related components. The sprinkler inspection model 400 begins with an imaging system 404 that provides the image data 402. Various camera systems and configurations can be used to implement the imaging system 404. In certain implementations, the imaging system 404 is an online image acquisition machine that can be implemented as part of a maintenance and / or manufacturing system. It will be readily appreciated that the type of imaging system 404 used can vary widely and depend on the application. Higher-resolution cameras can result in better error detection, while lower-resolution cameras can be implemented at a lower cost.

[0082] The image data 402 provided by the imaging system 404 can have various formats. In the depicted example, the image data 402 includes multiple initial images 406 of the printhead. The multiple initial images 406 can be segmented images of the printhead, each segmented image mapping a different portion of the printhead. The term "different portion" generally refers to any geometric difference between two portions. Thus, in various examples, the different portions may or may not overlap. In other implementations, the image data 402 includes an image depicting the entire printhead, thereby reducing image acquisition time (given the same imaging system) at the expense of image resolution. Reducing the initial image resolution may result in less accurate error detection, depending on the application. For example, in an application targeting the detection of printhead holes, low image resolution may be insufficient because printhead holes typically have diameters on the order of hundreds of microns. In such cases, it may be worthwhile to spend the extra time acquiring multiple segmented images rather than a single, full image.

[0083] The printhead detection model 400 includes a feature extraction process 408 for cropping an initial image 406 so that the remaining portion corresponds to a target feature. For example, the printhead aperture and / or the chamfer of the printhead aperture can be cropped from the initial image of the printhead. Because the printhead aperture can be many orders of magnitude smaller than the printhead itself, the feature extraction process 408 significantly reduces the amount of image data to be analyzed. The feature extraction process 408 includes detecting target features in each initial image 406. Target features can be identified using various methods, including but not limited to edge detection techniques. For example, a circular Hough transform can be applied to detect circles in an imperfect image for identifying printhead apertures. Once identified, the target features can be cropped to generate a plurality of cropped images 410. In some implementations, the initial image 406 is cropped to contain the identified target features at a predetermined image resolution. For example, the initial image 406 can be cropped to produce a cropped image 410 having a resolution of 32x32 pixels. It will be readily appreciated that the initial image 406 can be cropped to have any other resolution.

[0084] The cropping process produces a cropped image 410 having a lower resolution than the original image 406. Detecting features in a low-resolution image can be difficult and unlikely to yield any meaningful results. Therefore, the model 400 includes an upscaling process 412 for converting the cropped image 410 into a higher-resolution upscaled image 414. The upscaling process 412 can be performed using a super-resolution generative adversarial network. An SRGAN can be implemented to restore finer textures in the cropped image, thereby enabling analysis to be performed in a more meaningful manner. Other upscaling techniques can also be utilized, including but not limited to other machine learning techniques. For example, in some implementations, the upscaling process utilizes an interpolation-based upscaling technique. In other implementations, the upscaling process utilizes a CNN.

[0085] With the increased resolution and finer textures achieved using the aforementioned techniques, features present in the magnified image 414 can be more easily detected. The printhead inspection model 400 includes a measurement fitting process 416 that applies various algorithms to provide measurements for evaluating the characteristics of target features. For example, roundness measurements can be used to detect defects such as irregular shapes, burrs, and blockages in the printhead aperture and / or the chamfers of the printhead aperture. Examples of roundness measurements include MIC, MCC, MZC, and LSC measurements.

[0086] The nozzle inspection model 400 outputs inspection result data 418 based on the measurement fit. The inspection result data 418 may include various details about the target features. For example, in an application targeting nozzle hole inspection, the inspection result may include information about the characteristics of the nozzle hole, such as size and identified defects.

[0087] Figure 5A flow chart of an exemplary method 500 for inspecting a printhead and related components is shown. Examples include a printhead including a bonded rear panel and back plate, an unbonded printhead panel, and a printhead base. Method 500 can be implemented using any suitable hardware. Examples include computing system 100 and computing device 102 as described above. Images can be captured by imaging system 113, imaging system 200, or any other suitable imaging system. At step 502, method 500 includes receiving image data of the printhead. The image data can be received from a variety of sources. In some implementations, the image data is received from a remote device. In other implementations, the image data is received from a local device. For example, the image data can be received from an imaging system that is part of an in-line optical inspection system.

[0088] The imaging system (e.g., imaging system 113, imaging system 200) can be implemented in many ways. In some implementations, the imaging system is implemented as a front light imaging system. The imaging system can also include various components to facilitate imaging of the sprinkler. For example, the imaging system can include a platform for positioning the sprinkler. In some implementations, the platform is a movable platform. The imaging system can include one or more cameras. In some implementations, the platform and / or the camera can be repositioned to obtain images of different sections of the sprinkler. For example, the camera can be mounted on a three-axis gantry system. Different types of cameras can be implemented. In some implementations, the imaging system includes a CCD camera. In some implementations, the imaging system includes a handheld camera.

[0089] The image data may include one or more images. In some implementations, the image data includes a single initial image of the entire printhead. In other implementations, the image data includes multiple initial images, each of which maps to a different portion of the printhead (e.g., step 304 shows an exemplary initial image corresponding to a portion of the printhead).

[0090] At step 504, method 500 includes identifying target features in the image data (e.g., step 306). The nozzle includes various target features, including but not limited to nozzle holes and chamfers of the nozzle holes. Additionally or alternatively, the target features include cracks on the surface of the nozzle. Various techniques can be used to identify the target features. In some cases, the technique utilized may depend on the target features. In some implementations, a circular Hough transform algorithm can be applied to identify the nozzle holes. In some implementations, an edge detection algorithm is implemented.

[0091] At step 506, method 500 includes generating one or more cropped images based on the identified target features. In some examples, multiple cropped images are generated. The multiple cropped images can be generated by cropping the target features from the initial image (e.g., step 308). In some implementations, the multiple cropped images include images of a majority of the nozzle holes of the nozzle. In further implementations, the multiple cropped images include an image of each hole of the nozzle. In some implementations, the images are cropped to a predetermined resolution. The multiple cropped images can have any resolution. In some implementations, the cropped images have a resolution of less than 128×128 pixels. In further implementations, the cropped images have a resolution of 32×32 pixels.

[0092] At step 508, method 500 includes generating one or more upscaled images by upscaling one or more cropped images. In some examples, method 500 includes generating a plurality of upscaled images by upscaling a plurality of cropped images (e.g., step 310). The cropped images can be upscaled using various techniques (e.g., upscaling process 412). In some implementations, an SRGAN or other GAN is implemented to upscale the plurality of cropped images. In addition to upscaling the resolution of the cropped images, the SRGAN can also restore finer details and textures in the images. Other machine learning models, such as CNNs, can also be implemented to perform the upscaling. In some implementations, each of the plurality of cropped images is upscaled using an interpolation-based technique. The upscaled images can have any resolution. In some implementations, the upscaled cropped images have a resolution of at least 128×128 pixels. The resolution of the upscaled cropped images can depend on the technique utilized and the resolution of the plurality of cropped images. For example, an SRGAN capable of upscaling up to four times can be implemented to upscale a cropped image having a resolution of 32×32 pixels to a resolution of 128×128 pixels. In other examples, the cropped image can be upscaled starting from any other suitable starting resolution other than 32×32 pixels. Furthermore, the cropped image can be upsampled to any other suitable resolution other than 128×128 pixels.

[0093] At step 510, method 500 includes performing measurements on the plurality of magnified images. Different types of measurements may be performed, depending on the target feature (e.g., step 312 illustrates an exemplary combination of roundness measurements performed on the magnified images). In certain implementations, measurements are performed to determine various dimensions, including but not limited to the area, roundness, center, and diameter of the nozzle holes. Based on the measurements, other details such as defects and errors may be determined. For example, the measurements may be used to identify surface cracks, porosity, missing holes, holes with roundness errors, holes with position errors, and holes with defects such as burrs and blockages. The measurements may also be used to display the distribution of the offset between the desired holes and the holes as manufactured.

[0094] Measurements can be performed by applying various machine vision-based algorithms to the magnified image. For example, in certain implementations, a Canny edge detector algorithm can be applied to identify the outline of the printhead orifice. Based on the identified outline, various roundness measurements can be performed. Roundness measurements may include, but are not limited to, the maximum inscribed circle, the minimum circumscribed circle, the minimum annular circle, and the least squares circle.

[0095] At step 512, method 500 optionally includes transmitting inspection results. Inspection results may be generated based on various measurements performed on the magnified images. Inspection results may include data describing various details of a given target feature, such as the diameter of a printhead orifice and any defects present. In some implementations, the inspection results include data describing whether the target feature in the multiple magnified images meets predetermined inspection criteria.

[0096] The detection results can be transmitted to a variety of devices. In some implementations, the detection results are transmitted to a remote device. For example, in a cloud-based detection system, image data is received from a remote detection system. The image data is analyzed, and the detection results are transmitted to the remote detection system. In other implementations, the detection results are provided and transmitted to a local device.

[0097] Figure 5 The method 500 described herein provides a general framework within which a printhead inspection system can be implemented. Certain aspects utilize machine vision and machine learning techniques. Leveraging these techniques, a printhead inspection system capable of inspecting every orifice of a printhead can be implemented efficiently and quickly. Such a system is well-suited for implementation as an in-line optical inspection system, resulting in more consistent quality control and reduced yield loss.

[0098] Figure 6 A non-limiting embodiment of a computing system 600 is schematically shown that can perform one or more of the methods and processes described above. The computing system 600 is shown in simplified form. The computing system 600 can take the form of one or more personal computers, workstations, computers integrated with substrate processing tools, and / or network-accessible server computers.

[0099] The computing system 600 includes a logic machine 602 and a storage machine 604. The computing system 600 may optionally include a display subsystem 606, an input subsystem 608, a communication subsystem 610, and / or Figure 6 Other parts not shown in the above reference Figure 1 The controller depicted is an example of computing system 600 .

[0100] Logic engine 602 includes one or more physical devices configured to execute instructions. For example, logic engine 602 may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise achieve a desired result.

[0101] The logic machine 602 may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic machine 602 may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The processors of the logic machine 602 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The various components of the logic machine 602 may optionally be distributed across two or more separate devices that may be remotely located and / or configured for coordinated processing. Various aspects of the logic machine 602 may be virtualized and executed by a remotely accessible, networked computing device configured in a cloud computing configuration.

[0102] The storage 604 includes one or more physical devices configured to store instructions 612 that can be executed by a logic machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of the storage 604 can be transformed—for example, to store different data.

[0103] The storage 604 may include removable and / or internal devices. The storage 604 may include optical storage (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic storage (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.). The storage 604 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location addressable, file addressable, and / or content addressable devices.

[0104] It should be understood that storage 604 comprises one or more physical devices. However, alternatively, aspects of the instructions described herein may be propagated via a communication medium (eg, electromagnetic signals, optical signals, etc.) that is not held for a finite duration by a physical device.

[0105] Aspects of the logic machine 602 and the memory machine 604 may be integrated together into one or more hardware logic components. For example, such hardware logic components may include field programmable gate arrays (FPGAs), program and application specific integrated circuits (PASIC / ASICs), program and application specific standard products (PSSP / ASSPs), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0106] When included, display subsystem 606 can be used to present a visual representation of the data stored by storage machine 604. This visual representation can take the form of a graphical user interface (GUI). Because the methods and processes described herein change the data held by the storage machine, and therefore change the state of the storage machine, the state of display subsystem 606 can also be converted to visually represent the change of the underlying data. Display subsystem 606 can include one or more display devices using almost any type of technology. Such a display device can be combined with the logic machine 602 and / or the storage machine 604 in a shared cabinet (a shared enclosure), or such a display device can be a peripheral display device.

[0107] When included, the input subsystem 608 may include or interact with one or more user input devices (e.g., a keyboard, mouse, or touch screen). In some embodiments, the input subsystem 608 may include or interact with selected natural user input (NUI) components. Such components may be integrated or peripheral, and the conversion and / or processing of input actions may be handled on-board or off-board. Exemplary NUI components may include microphones for voice and / or sound recognition, and infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition.

[0108] When included, the communication subsystem 610 can be configured to communicatively couple the computing system 600 with one or more other computing devices. The communication subsystem 610 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem 610 can be configured to communicate using a wireless telephone network, or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem 610 can allow the computing system 600 to send messages to and / or receive messages from other devices over a network such as the Internet.

[0109] It should be understood that the configuration and / or method described herein are exemplary in nature, and these specific one or more embodiments or examples should not be considered as restrictive, because many variations are possible. The specific routine or method described herein can represent one or more of any number of processing strategies. Therefore, the various actions shown and / or described can be performed in the order shown and / or described, in other orders, in parallel or omitted. Equally, the order of the above-mentioned processing can be changed.

[0110] The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

[0111] The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

Claims

1. A computing system for a printhead based on machine vision detection, the computing system comprising: a processor and a memory, the processor being configured to execute a program using a portion of the memory to: receiving image data of the printhead; identifying target features in the image data of the printhead; generating a cropped image based on the target features identified using the image data of the printhead; generating an enlarged image by enlarging the cropped image; and One or more measurements are performed on the magnified image.

2. The computing system of claim 1, wherein the showerhead comprises a front plate and a back plate joined together.

3. The computing system of claim 1 , wherein generating the cropped image comprises generating a plurality of cropped images, wherein generating the zoomed-in image comprises generating a plurality of zoomed-in images by zooming in on the plurality of cropped images, and wherein performing the one or more measurements on the zoomed-in images comprises performing the one or more measurements on each of the plurality of zoomed-in images. The computing system of claim 1 , wherein the plurality of upscaled images are generated using a generative adversarial network.

5. The computing system of claim 1, wherein the image data comprises a plurality of initial images, wherein each initial image includes a different portion of the printhead. The computing system of claim 1 , wherein the target feature comprises a showerhead orifice.

7. The computing system of claim 6, wherein the measurement comprises a circularity measurement comprising a maximum inscribed circle and a minimum circumscribed circle. 8 . The computing system of claim 6 , wherein the processor is further configured to provide a detection result based on the performed measurements, wherein the detection result describes a burr detected in the showerhead aperture.

9. The computing system of claim 1, wherein the target feature comprises a chamfer of a showerhead hole.

10. The computing system of claim 3, wherein the plurality of cropped images include images of a majority of nozzle apertures of the nozzle.

11. A method for machine vision inspection of a printhead, the method comprising: receiving image data of the printhead; identifying target features in the image data of the printhead; generating a plurality of cropped images based on the target features identified using the image data of the printhead; generating a plurality of enlarged images by enlarging the plurality of cropped images; as well as A plurality of measurements are performed on the plurality of magnified images.

12. The method of claim 11, wherein the showerhead comprises a face plate and a back plate joined together. The method of claim 11 , wherein the image data is received from a remote device. The method of claim 11 , wherein the plurality of upscaled images are generated using a generative adversarial network.

15. The method according to claim 11, further comprising: generating a plurality of detection results based on the measurements performed, wherein the detection results describe burrs detected in the printhead hole; and The detection result is transmitted to a remote device.

16. A nozzle detection system based on machine vision detection for a nozzle, the nozzle detection system comprising: A camera system comprising: camera; front lighting system; and a platform for placing the nozzle; computing systems; and A controller configured to: controlling the camera system to acquire image data of the printhead; as well as controlling the computing system to: receiving the image data of the printhead from the camera system; identifying target features in the image data of the printhead; generating a plurality of cropped images based on the target features identified using the image data of the printhead; generating a plurality of enlarged images by enlarging the plurality of cropped images; as well as A plurality of measurements are performed on the plurality of magnified images. The printhead detection system according to claim 16 , wherein the printhead comprises a front plate and a back plate joined together.

18. The printhead inspection system of claim 16, wherein the camera system further comprises a telecentric lens. The printhead detection system of claim 16 , wherein the plurality of magnified images are generated using a generative adversarial network.

20. The printhead inspection system of claim 16, wherein the controller is further configured to control the computing system to generate a plurality of inspection results based on the measurements performed, wherein the inspection results describe burrs detected in a printhead hole.

Images