Substrate inspection unit and substrate processing apparatus including the same

Abstract

The present application provides a substrate inspection unit and a substrate processing apparatus including the same, which can reduce image data labeling operation time through validation of a training image data set and semi-automatic image labeling, while improving classification accuracy of the data set to improve prediction performance. The substrate inspection unit includes a feature extraction module that extracts features from training data included in each category in a case where a plurality of training data related to image data of a substrate is classified according to a defined category; an effectiveness evaluation module that evaluates effectiveness of the features; a category validation module that validates the defined category; and a data reconstruction module that reconstructs the plurality of training data based on the features judged to have effectiveness and the validated category, wherein the reconstructed training data is used when inspecting the substrate.

Description

Technical Field

[0001] This invention relates to a substrate inspection unit and a substrate processing apparatus including the same. More specifically, it relates to a substrate inspection unit capable of inspecting a substrate using image data and a substrate processing apparatus including the same. Background Technology

[0002] When performing printing processes (e.g., RGB patterning) on ​​transparent substrates to manufacture display devices such as LCD panels, PDP panels, and LED panels, a printing apparatus with an inkjet head unit can be used. Summary of the Invention

[0003] When printing on a substrate using an inkjet head unit, the substrate can be inspected as ink is ejected to prevent the mass production of defective substrates. In this case, a camera module installed in the printing equipment can be used to acquire image data for substrate inspection.

[0004] When inspecting a substrate using image data, a deep learning process utilizing the image data can be performed to improve the reliability of the verification.

[0005] However, in the data training step of deep learning, if the operator does not explicitly define the classification criteria for the classes, or uses an image dataset without corresponding labels for each class to perform training, the learning level of the training model will be reduced, and the final prediction performance may be lower.

[0006] The technical problem to be solved by the present invention is to provide a substrate inspection unit and a substrate processing apparatus including the same, which can reduce the image data labeling operation time by verifying the training image dataset and semi-auto image labeling, while improving the classification accuracy of the dataset and thus improving the prediction performance.

[0007] The technical problems of this invention are not limited to those described above. Those skilled in the art can clearly understand other technical problems not mentioned in the following description.

[0008] An aspect of the substrate inspection unit of the present invention for solving the above-mentioned technical problems includes: a feature extraction module, which extracts features from training data included in each category when multiple training data related to image data of the substrate are classified according to predefined categories; a validity evaluation module, which evaluates the validity of the features; a category verification module, which verifies the predefined categories; and a data reconstruction module, which reconstructs the multiple training data based on the features judged to be valid and the verified categories, wherein the reconstructed training data is used when inspecting the substrate.

[0009] The feature extraction module can extract the features using a pre-learning model, which is the result of learning using data above a certain benchmark related to the image data of the substrate.

[0010] The effectiveness evaluation module can evaluate the effectiveness of the features by applying a dimensionality reduction method to the features.

[0011] The effectiveness evaluation module can use the t-SNE (t-distributed stochastic neighborhood embedding) algorithm as the dimensionality reduction method.

[0012] The category verification module can use non-hierarchical clustering analysis to verify the defined categories.

[0013] The category verification module can use a density-based clustering method to verify the defined categories.

[0014] The category verification module can use clustering analysis based on unsupervised learning to verify the defined categories.

[0015] The substrate inspection unit may further include: a data quantity discrimination module, which determines whether the amount of training data included in each category is above a baseline quantity.

[0016] The feature extraction module can extract the features using a pre-learning model when the amount of training data included in each category is less than the baseline amount. The pre-learning model is the result obtained by learning from data above the baseline amount that is related to the image data of the substrate.

[0017] The substrate inspection unit may further include a data labeling module, which directly classifies the plurality of training data according to the predefined categories.

[0018] The substrate inspection unit may further include: a marker information acquisition module, which is input to or receives information about training data classified by category and information about the predefined categories.

[0019] The plurality of training data can be at least one of image data from a substrate in good condition and image data from a substrate in poor condition.

[0020] The multiple training data may be image data of the substrate from which the substrate processing liquid is sprayed.

[0021] The substrate inspection unit may further include: a data processing module for processing the image data of the substrate when the image data of the substrate is acquired; a reference data detection module for detecting reference data; a data analysis module for comparing and analyzing the image data of the substrate and the reference data; and a substrate discrimination module for determining whether the substrate is good or bad based on the comparison and analysis results between the image data of the substrate and the reference data.

[0022] The benchmark data detection module can determine the category associated with the image data of the substrate within the predefined categories, and detect the benchmark data from the training data included in the determined categories.

[0023] The substrate inspection unit can reconstruct the plurality of training data when the device for processing the substrate is driven at least once.

[0024] Furthermore, another aspect of the substrate inspection unit of the present invention for solving the above-mentioned technical problems includes: a feature extraction module, which extracts features from training data included in each category when multiple training data related to image data of the substrate are classified according to a predefined category; an effectiveness evaluation module, which evaluates the effectiveness of the features; a category verification module, which verifies the predefined category; and a data reconstruction module, which reconstructs the multiple training data based on the features judged to be effective and the verified category, wherein the reconstructed training data is used when inspecting the substrate, the feature extraction module extracts the features using transfer learning of a pre-learning model, the pre-learning model being the result of learning using data above a benchmark amount related to the image data of the substrate, the effectiveness evaluation module evaluates the effectiveness of the features by applying a dimensionality reduction method to the features, the category verification module uses DBSCAN based on unsupervised learning to verify the predefined category, and the multiple training data are image data of the substrate being sprayed with substrate treatment liquid, and at least one of image data in a good state and image data in a bad state.

[0025] Furthermore, one aspect of the substrate processing apparatus of the present invention for solving the above-mentioned technical problems includes: a process processing unit that supports the substrate during substrate processing; an inkjet head unit that ejects substrate processing liquid onto the substrate; a frame unit that provides the inkjet head unit and moves the inkjet head unit above the substrate; and a substrate inspection unit that inspects the substrate.

[0026] The substrate inspection unit includes: a feature extraction module, which extracts features from the training data included in each category when multiple training data related to the image data of the substrate are classified according to predefined categories; a validity evaluation module, which evaluates the validity of the features; a category verification module, which verifies the predefined categories; and a data reconstruction module, which reconstructs the multiple training data based on the features judged to be valid and the verified categories, wherein the reconstructed training data is used when inspecting the substrate.

[0027] The feature extraction module can extract the features using transfer learning from the pre-learning model.

[0028] The pre-learning model is the result obtained by learning from data above a benchmark quantity related to the image data of the substrate.

[0029] The category verification module can use DBSCAN, which is based on unsupervised learning, to verify the defined categories.

[0030] Specific details of other embodiments are included in the detailed description and accompanying drawings. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating the internal structure of a substrate processing apparatus according to an embodiment of the present invention.

[0032] Figure 2 This is a first example diagram schematically illustrating the internal modules related to a first method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention.

[0033] Figure 3 This is a second example diagram schematically illustrating the internal modules related to a first method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention.

[0034] Figure 4 This is a third example diagram schematically illustrating an internal module related to a first method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention.

[0035] Figure 5This is a first example diagram illustrating a first method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention.

[0036] Figure 6 This is a second example diagram illustrating a first method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention.

[0037] Figure 7 This is a third example diagram illustrating a first method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention.

[0038] Figure 8 This is an example diagram schematically illustrating an internal module related to a second method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention.

[0039] Figure 9 This is an example diagram illustrating a second method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention.

[0040] Explanation of reference numerals in the attached figures

[0041] 100: Substrate processing apparatus; 110: Processing unit

[0042] 120: Maintenance Unit; 125: Vision Module

[0043] 130: Rack unit; 140: Inkjet head unit

[0044] 150: Substrate processing liquid supply unit; 160: Control unit

[0045] 200: Substrate inspection unit; 210a: Marking information acquisition module

[0046] 210b: Data labeling module; 220: Feature extraction module

[0047] 230: Validity Evaluation Module; 240: Category Validation Module

[0048] 250: Data Reconstruction Module; 260: Data Volume Determination Module

[0049] 610: Data Processing Module; 620: Benchmark Data Detection Module

[0050] 630: Data Analysis Module; 640: Substrate Identification Module Detailed Implementation

[0051] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The advantages and features of the present invention, as well as methods for achieving these advantages and features, will be explained by referring to the following description in conjunction with the accompanying drawings. Figure 1The invention becomes clear from the detailed description of the embodiments. However, the invention is not limited to the embodiments disclosed below, but can be implemented in many different forms. These embodiments are provided only to make the disclosure of the invention complete and to fully inform those skilled in the art of the scope of the invention, which is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same constituent elements.

[0052] When an element or layer is referred to as "on" or "above" another element or layer, it includes not only that it is directly above another element or layer, but also that other layers or elements are in between. Conversely, when an element is referred to as "directly" on or directly above another element, it indicates that there are no other elements or layers in between.

[0053] To readily describe the relationship between one element or component and another, as shown in the figure, spatial relative terms such as "below," "below," "lower," "above," and "upper" can be used. It should be understood that, in addition to the orientation shown in the figure, spatial relative terms also include terms indicating the different orientations of the elements during use or operation. For example, when the element shown in the figure is flipped, an element described as "below" or "below" of another element may be located "above" of that element. Therefore, the exemplary term "below" can include both "below" and "above" orientations. An element may also be oriented in another direction, thus allowing the spatial relative terms to be interpreted according to orientation.

[0054] Although the terms "first," "second," etc., are used to describe various elements, constituent elements, and / or parts, these elements, constituent elements, and / or parts are obviously not limited by these terms. These terms are only used to distinguish one element, constituent element, and / or part from another element, constituent element, and / or part. Therefore, the first element, first constituent element, or first part mentioned below can obviously also be a second element, second constituent element, or second part within the technical concept of the present invention.

[0055] The terminology used in this specification is for illustrative purposes and is not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other constituent elements, steps, operations, and / or components in addition to those mentioned.

[0056] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in the sense that can be commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be ideally or excessively interpreted unless explicitly defined otherwise.

[0057] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing the invention with reference to the drawings, identical or corresponding constituent elements are given the same reference numerals, regardless of the reference numerals, and repeated descriptions thereof are omitted.

[0058] This invention relates to a substrate inspection unit for inspecting a substrate based on image data of the substrate, and a substrate processing apparatus including the same. The substrate inspection unit will be described in more detail later; here, the substrate processing apparatus will be described first.

[0059] Figure 1 This is a schematic diagram illustrating the internal structure of a substrate processing apparatus according to an embodiment of the present invention.

[0060] The substrate processing apparatus 100 processes a substrate G (e.g., a glass substrate) for manufacturing a display device. This substrate processing apparatus 100 can be configured as an inkjet device that prints the substrate G by ejecting substrate processing liquid onto the substrate G using an inkjet head unit 140.

[0061] The substrate processing apparatus 100 can use ink as a substrate processing liquid. Here, substrate processing liquid refers to a liquid used for printing on the substrate G. The substrate processing liquid can be, for example, quantum dot (QD) ink containing ultrafine semiconductor particles, and the substrate processing apparatus 100 can be configured as, for example, a QD (Quantum Dot) CF (Color Filter) inkjet device. The substrate processing apparatus 100 can perform pixel printing on the substrate G using the substrate processing liquid, and can be configured as a circulating inkjet device to prevent the nozzles from being clogged by the substrate processing liquid.

[0062] according to Figure 1 The substrate processing apparatus 100 may include a process processing unit 110, a maintenance unit 120, a gantry unit 130, an inkjet head unit 140, a substrate processing liquid supply unit 150, a controller 160, and a substrate inspection unit 200.

[0063] The process unit 110 supports the substrate G during the PT operation. Here, the PT operation refers to the printing process performed on the substrate G using a substrate processing solution.

[0064] The process unit 110 can support the substrate G in a non-contact manner. For example, the process unit 110 can support the substrate G by suspending it in the air using air. However, this embodiment is not limited to this. The process unit 110 can also support the substrate G in a contact manner. The process unit 110 can also support the substrate G, for example, using a support member with a mounting surface on its upper part.

[0065] The process unit 110 can move the substrate G while utilizing an air-supported substrate G. The process unit 110 may include, for example, a first (1...) st Stage 111 and Air Hole 112.

[0066] The first unit 111 is a base, and is configured such that the substrate G can be placed on it. The pores 112 can be formed through the upper surface of the first unit 111, and multiple pores can be formed within the printing zone on the first unit 111.

[0067] The vent 112 can spray air upwards (to the third direction 30) towards the first unit 111. The vent 112 can thereby suspend the substrate G mounted on the first unit 111 in the air.

[0068] Although not in Figure 1 As shown, the process unit 110 may also include a clamping part and a guide rail. The clamping part clamps the substrate G as it moves along the length direction (first direction 10) of the first stage 111 to prevent the substrate G from detaching from the first stage 111. When the substrate G moves, the clamping part can move along the guide rail in the same direction as the direction of movement of the substrate G while clamping the substrate G. The clamping part and the guide rail may be located on the outside of the first stage 111.

[0069] The maintenance unit 120 measures the ejection position (i.e., dotting) and whether the substrate processing liquid is ejected on the substrate G. The maintenance unit 120 can measure the ejection position and whether the substrate processing liquid is ejected for each of the multiple nozzles provided in the inkjet head unit 140, and can provide the measurement results obtained in this way to the control unit 160.

[0070] Maintenance unit 120 may include, for example, a second (2) nd Stage 121, Third guide rail (3) rdGuide Rail) 122, First Board (1 st Plate 123, Calibration Board 124, and Vision Module 125.

[0071] The second unit 121, like the first unit 111, is a base and can be arranged parallel to the first unit 111. This second unit 121 may include a maintenance zone on its upper part. The second unit 121 can be set to the same size as the first unit 111, but it can also be set to have a size smaller or larger than the first unit 111.

[0072] The third guide rail 122 guides the movement path of the first plate 123. This third guide rail 122 can be set as at least one line on the second plate 121 along the length direction (first direction 10) of the second plate 121. The third guide rail 122 can be implemented, for example, as an LM guide system (Linear Motor Guide System).

[0073] Although Figure 1 The first plate 123 is not shown, but the maintenance unit 120 may also include a fourth guide rail. The fourth guide rail guides the movement path of the first plate 123 in the same way as the third guide rail 122, and may be set as at least one line on the second plate 121 along the width direction (second direction 20) of the second plate 121.

[0074] The first plate 123 can move along the third guide rail 122 and / or the fourth guide rail on the second stage 121. The first plate 123 can move parallel to the substrate G along the third guide rail 122, and can move closer to or further away from the substrate G along the fourth guide rail.

[0075] The calibration plate 124 is used to measure the ejection position of the substrate processing liquid on the substrate G. This calibration plate 124 may include alignment marks, scales, etc. and be disposed on the first plate 123, and may be disposed along the length direction (first direction 10) of the first plate 123.

[0076] The vision module 125 includes a camera module and acquires image information about the substrate G. The image information about the substrate G acquired by the vision module 125 may include information such as whether substrate processing liquid has been ejected, the ejection location of the substrate processing liquid, the ejection amount of the substrate processing liquid, and the ejection area of ​​the substrate processing liquid. Furthermore, in addition to the image information about the substrate G from which substrate processing liquid has been ejected, the vision module 125 can also acquire and provide information about the calibration plate 124.

[0077] When processing the substrate G, the vision module 125 can acquire image information about the substrate G in real time. The vision module 125 can capture images of the substrate G along its length (first direction 10) to obtain image information; in this case, the vision module 125 may include a line scan camera. Alternatively, the vision module 125 can also capture images of each predetermined-size area of ​​the substrate G to obtain image information. In this case, the vision module 125 may include an area scan camera.

[0078] The vision module 125 can be attached to the lower or side surface of the rack unit 130 to acquire image information of the substrate G from which the substrate processing liquid is ejected. However, this embodiment is not limited to this. The vision module 125 can also be attached to the side surface of the inkjet head unit 140. On the other hand, at least one vision module 125 can be provided in the substrate processing apparatus 100, and can be fixedly or movably provided.

[0079] The frame unit 130 supports the inkjet head unit 140. This frame unit 130 can be disposed above the first unit 111 and the second unit 121 so that the inkjet head unit 140 can spray substrate processing liquid onto the substrate G.

[0080] The rack unit 130 can be mounted on the first unit 111 and the second unit 121 with its length direction being the width direction (second direction 20) of the first unit 111 and the second unit 121. The rack unit 130 can be mounted along the first guide rail (1 st Guide Rail 170a and Second Guide Rail (2) nd Guide Rail 170b can move along the length direction (first direction 10) of the first unit 111 and the second unit 121. On the other hand, the first guide rail 170a and the second guide rail 170b can be arranged outside the first unit 111 and the second unit 121 along the length direction (first direction 10) of the first unit 111 and the second unit 121.

[0081] On the other hand, although Figure 1 Although not shown, the substrate processing apparatus 100 may further include a rack moving unit. The rack moving unit causes the rack unit 130 to slide along the first guide rail 170a and the second guide rail 170b. The rack moving unit may be disposed inside the rack unit 130.

[0082] The inkjet head unit 140 ejects the substrate processing liquid into the substrate G in the form of droplets. This inkjet head unit 140 can be disposed on the side surface or the lower surface of the frame unit 130.

[0083] At least one inkjet head unit 140 may be provided on the rack unit 130. When multiple inkjet head units 140 are provided on the rack unit 130, the multiple inkjet head units 140 may be arranged in a row along the length direction (second direction 20) of the rack unit 130. In addition, the multiple inkjet head units 140 may operate independently or, conversely, operate uniformly.

[0084] The inkjet head unit 140 can move along the length direction (second direction 20) of the frame unit 130 to a desired position on the substrate G. However, this embodiment is not limited thereto. The inkjet head unit 140 can move along the height direction (third direction 30) of the frame unit 130, and can also rotate in a clockwise or counterclockwise direction.

[0085] Alternatively, the inkjet head unit 140 can be fixed to the rack unit 130. In this case, the rack unit 130 can be movably mounted.

[0086] Although Figure 1 Although not shown, the substrate processing apparatus 100 may also include an inkjet head moving unit. The inkjet head moving unit causes the inkjet head unit 140 to move linearly or rotate.

[0087] Although Figure 1 Although not shown, the inkjet head unit 140 may include a nozzle plate, multiple nozzles, piezoelectric elements, etc. The nozzle plate constitutes the main body of the inkjet head unit 140. Multiple nozzles (e.g., 128, 256, etc.) can be arranged in multiple rows and columns at certain intervals on the lower part of such a nozzle plate, and the number of piezoelectric elements can be set in the nozzle plate in a number corresponding to the number of nozzles. With such a configuration, the inkjet head unit 140 can eject substrate processing liquid onto the substrate G through the nozzles according to the operation of the piezoelectric elements.

[0088] On the other hand, the inkjet head unit 140 can also independently control the amount of substrate processing liquid ejected through each nozzle according to the voltage applied to the piezoelectric element.

[0089] The substrate processing liquid supply unit 150 supplies ink to the inkjet head unit 140. This substrate processing liquid supply unit 150 may include a storage tank 150a and a pressure control module 150b.

[0090] Storage tank 150a stores substrate processing liquid, and pressure control module 150b regulates the internal pressure of storage tank 150a. Storage tank 150a can supply an appropriate amount of substrate processing liquid to inkjet head unit 140 based on the pressure provided by pressure control module 150b.

[0091] The control unit 160 controls the overall operation of each unit constituting the substrate processing apparatus 100. The control unit 160 can control the operation of, for example, the vent 112 and clamping part of the process processing unit 110, the vision module 125 of the maintenance unit 120, the rack unit 130, the inkjet head unit 140, and the pressure control module 150b of the substrate processing liquid supply unit 150.

[0092] The control device 160 can be implemented as a computer or server including a process controller, a control program, an input module, an output module (or a display module), a memory module, etc. As described above, the process controller may include a microprocessor that performs control functions on the various components constituting the substrate processing apparatus 100, and the control program may execute various processes on the substrate processing apparatus 100 according to the control of the process controller. The memory module stores programs, i.e., processing recipes, for executing various processes on the substrate processing apparatus 100 based on various data and processing conditions.

[0093] On the other hand, the control unit 160 can also perform maintenance on the inkjet head unit 140. For example, the control unit 160 can correct the ejection position of the substrate processing liquid in each nozzle in the inkjet head unit 140 based on the measurement results of the maintenance unit 120, or it can detect defective nozzles (i.e., nozzles that do not eject substrate processing liquid) from multiple nozzles and perform cleaning operations on the defective nozzles.

[0094] The substrate inspection unit 200 can inspect the substrate G based on image data acquired by the vision module 125. Specifically, when image data of the substrate G is acquired by the vision module 125, the substrate inspection unit 200 processes the image data and then inspects the substrate G by comparing and analyzing the image data and reference data. For this purpose, the substrate inspection unit 200 can be linked with the vision module 125 and can be configured like the control unit 160 as a computer or server including a process controller, control program, input module, output module (or display module), memory module, etc.

[0095] The substrate inspection unit 200 can classify the training data using deep learning. From the classified training data, the substrate inspection unit 200 can detect reference data that will be used as a comparison object with the image data of the substrate G, and then inspect the substrate G by comparing the image data of the substrate G with the reference data.

[0096] In the above text, the training data can be image data of substrates G with various shapes. Furthermore, the training data can include at least one type of image data: image data of superior substrates suitable for manufacturing display devices and image data of defective substrates unsuitable for manufacturing display devices.

[0097] When training data is classified using deep learning, the reliability of verification can be improved when inspecting substrate G using image data. However, as mentioned above, in the data training step of deep learning, if the operator does not explicitly define the classification criteria for the categories, or uses an image dataset without corresponding labels for each category to perform training, the learning level of the training model will be reduced, and the final prediction performance may be degraded.

[0098] In this embodiment, the substrate inspection unit 200 can reduce the image data labeling operation time by verifying the training image dataset and performing semi-auto image labeling, while improving the classification accuracy of the dataset and thus improving prediction performance. This method will be described in detail below.

[0099] Figure 2 This is a first example diagram schematically illustrating the internal modules related to a first method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention. The substrate inspection unit 200 can utilize deep learning to classify image data of the substrate G. In this case, such as... Figure 2 As shown, the substrate inspection unit 200 may include a marking information acquisition module 210a, a feature extraction module 220, a validity evaluation module 230, a category verification module 240, and a data reconstruction module 250.

[0100] When a labeling task is performed on multiple training data, the labeling information acquisition module 210a acquires labeling information related to that labeling task. The labeling information acquisition module 210a can acquire information such as the classification category used during labeling and information about the training data that is classified by category through labeling (i.e., information about the labeled training data) as labeling information.

[0101] Multiple operators can perform manual image labeling to conform to predefined classification categories for multiple training data sets. When multiple operators complete manual image labeling for multiple training data sets, the labeling information acquisition module 210a can obtain the labeling information from the results. In this case, the labeling information acquisition module 210a can receive the labeling information input via an input tool such as a keyboard or touchscreen, or via a wired / wireless communication tool such as WiFi.

[0102] In the case where multiple training data are classified into categories by manual image labeling, the feature extraction module 220 extracts features for the objects from the training data included in each category. Here, a feature refers to the characteristic (attribute value) of the data that needs to be predicted or classified.

[0103] When extracting features from the training data included in each category, the feature extraction module 220 can extract the aforementioned features based on the pre-trained model.

[0104] To improve the quality of deep learning models, a large amount of data is required for learning. Therefore, a model that has been pre-learned using a large amount of data for each category—that is, a pre-learned model—can be applied to feature extraction. When performing feature extraction using a pre-learned model, the feature extraction module 220 can extract features for objects from the training data included in each category through transfer learning based on the pre-learned model.

[0105] When the training data included in each category is a large amount of data, the feature extraction module 220 can perform feature extraction based on the training data included in each category. That is, the feature extraction module 220 may not utilize a pre-learned model. Considering these aspects, such as... Figure 3 As shown, the substrate inspection unit 200 may also include a data volume discrimination module 260. Figure 3 This is a second example diagram schematically illustrating the internal modules related to a first method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention.

[0106] The data volume discrimination module 260 determines whether the training data included in each category is a large amount of data. The data volume discrimination module 260 can perform this discrimination based on whether the amount (or quantity) of training data included in each category is above a benchmark value. Specifically, when the amount of training data included in each category is above the benchmark value, the data volume discrimination module 260 can determine that the training data included in each category is a large amount of data; and when the amount of training data included in each category is less than the benchmark value, the data volume discrimination module 260 can determine that the training data included in each category is not a large amount of data.

[0107] On the other hand, the data volume discrimination module 260 can also determine whether the multiple training data for the labeling task is a large amount of data. In this case, when the multiple training data for the labeling task is a large amount of data, the feature extraction module 220 can also perform feature extraction based on the training data included in each category instead of using a pre-learning model.

[0108] Refer again Figure 2 Please provide an explanation.

[0109] The validity evaluation module 230 evaluates the validity of the features extracted by the feature extraction module 220. This validity evaluation module 230 can evaluate the validity of each feature by applying a dimensionality reduction method to the features. By evaluating the validity of each feature, the validity evaluation module 230 removes features deemed invalid, thereby improving the computational and processing speed of the substrate inspection unit 200 related to the image data classification method.

[0110] When evaluating the effectiveness for each feature, the effectiveness evaluation module 230 can use the t-SNE (t-distributed stochastic neighborhood embedding) algorithm as a dimensionality reduction method. However, this embodiment is not limited to this. The effectiveness evaluation module 230 can also use the UMAP (Uniform Manifold Approximation and Projection) algorithm as a dimensionality reduction method.

[0111] The category verification module 240 performs verification on predefined categories. Here, the predefined categories refer to those used for manual image labeling.

[0112] When performing validation on predefined categories, the category validation module 240 can perform validation on the predefined categories based on the appropriateness of each predefined category classification criterion. In this case, the category validation module 240 can utilize non-hierarchical clustering analysis to perform validation on the predefined categories. Specifically, the category validation module 240 can utilize non-hierarchical clustering analysis based on unsupervised learning to perform validation on the predefined categories.

[0113] When using non-hierarchical clustering analysis, the category verification module 240 can utilize density-based clustering methods to perform verification on predefined categories. For example, the category verification module 240 can use DBSCAN (Density Based Spatial Clustering of Applications with Noise) to perform verification on predefined categories. However, this embodiment is not limited to this. The category verification module 240 can also utilize center-based clustering methods to perform verification on predefined categories. For example, the category verification module 240 can use the k-means clustering algorithm to perform verification on predefined categories.

[0114] After evaluating the effectiveness of features and validating categories, the data reconstruction module 250 performs clustering on multiple training data based on the features deemed valid and the categories being validated. That is, if training data clusters into a category being validated, the data reconstruction module 250 determines that the training data is suitable for that category and retains it; conversely, if training data does not cluster into a valid category, the data reconstruction module 250 determines that the training data is unsuitable for that category and excludes it, thus performing clustering on multiple training data.

[0115] The substrate inspection unit 200 can classify the training data (i.e., image data of the substrate G) by including the marker information acquisition module 210a, feature extraction module 220, validity evaluation module 230, category verification module 240, and data reconstruction module 250 as described above. This method of the substrate inspection unit 200, compared to existing methods where the operator subjectively classifies the training data, can achieve the effect of ensuring consistent training image data.

[0116] Furthermore, in the deep learning framework consisting of the steps of collecting image datasets, training image datasets, validating image datasets, designing model architecture, training the model, evaluating the model, and predicting the test image, the present invention can also achieve the same effect as the above-mentioned methods in the step of training image datasets, unlike the existing methods described above which only implement this in the step of collecting image datasets.

[0117] On the other hand, the substrate inspection unit 200 can perform reclassification of training data if the substrate processing apparatus 100 is run at least once.

[0118] Reference above Figure 2 The substrate inspection unit 200 described is an example of classifying image data of substrate G using deep learning, and in this case, the substrate inspection unit 200 does not perform a labeling operation on multiple training data. However, this embodiment is not limited to this. The substrate inspection unit 200 may also perform a labeling operation on multiple training data. This will be explained below.

[0119] Figure 4 This is a third example diagram schematically illustrating the internal modules related to a first method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention. Figure 4 The substrate inspection unit 200 may include a data marking module 210b, a feature extraction module 220, a validity evaluation module 230, a category verification module 240, and a data reconstruction module 250.

[0120] Already referred to Figure 2 The feature extraction module 220, validity evaluation module 230, category verification module 240, and data reconstruction module 250 are described, and their detailed descriptions will be omitted here.

[0121] The data labeling module 210b classifies multiple training data sets into various categories according to predetermined categories. Figure 2 The substrate inspection unit 200 is compared. Figure 2 The substrate inspection unit 200 shown does not classify multiple training data by category, but... Figure 4 The difference in the substrate inspection unit 200 shown is that it directly classifies multiple training data by category through the data labeling module 210b.

[0122] Next, the image data classification method of the substrate inspection unit 200 will be explained. Figure 5 This is a first example diagram illustrating a first method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention. Hereinafter, reference will be made to... Figure 5 Please provide an explanation.

[0123] First, in the case where multiple training data are manually classified into various categories by multiple operators (S310), the feature extraction module 220 extracts features from the training data included in each category as the object (S320). The feature extraction module 220 can extract features from the training data included in each category through transfer learning based on a pre-learned model.

[0124] Then, the validity evaluation module 230 evaluates the validity of the features extracted by the feature extraction module 220 (S330). The validity evaluation module 230 may use a dimensionality reduction method (e.g., the t-SNE algorithm) to evaluate the validity for each feature.

[0125] Then, the category verification module 240 performs verification on the predefined categories (i.e., the categories used to classify the multiple training data in step S310) (S340). The category verification module 240 can use DBSCAN based on unsupervised learning to perform verification on the predefined categories.

[0126] Then, the data reconstruction module 250 performs clustering on multiple training data based on the features that are deemed valid by the validity evaluation module 230 and the categories verified by the category verification module 240 (step S350).

[0127] On the other hand, as described above, when the training data included in each category is a large amount of data, the feature extraction module 220 can extract features based on the training data included in each category instead of utilizing transfer learning based on a pre-learning model. Therefore, the image data classification method of the substrate inspection unit 200 can also be implemented as follows.

[0128] Figure 6 This is a second example diagram illustrating a first method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention. Hereinafter, referring to... Figure 6 Please provide an explanation.

[0129] First, in the case where multiple training data are manually classified into various categories by multiple operators (S410), the data volume discrimination module 260 determines whether the training data included in each category is a large amount of data (S420).

[0130] Here, when it is determined that the training data included in each category is a large amount of data, the feature extraction module 220 uses the training data included in each category to extract features (S430).

[0131] Conversely, when it is determined that the training data included in each category is not a large amount of data, the feature extraction module 220 uses the data applied to the pre-learning model to extract features (S440). Here, the pre-learning model may consist of image data that is related to the training data included in each category.

[0132] Then, the effectiveness evaluation module 230 evaluates the effectiveness of the features extracted by the feature extraction module 220 (S450).

[0133] Then, the category verification module 240 performs verification on the predefined categories (i.e., the categories used to classify multiple training data in step S410) (S460).

[0134] Then, the data reconstruction module 250 performs clustering on multiple training data based on the features that are deemed valid by the validity evaluation module 230 and the categories that are verified by the category verification module 240 (S470).

[0135] On the other hand, the substrate inspection unit 200 can also directly perform labeling operations on multiple training data. Figure 7 This is a third example diagram illustrating a first method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention. Hereinafter, referring to... Figure 7 Please provide an explanation.

[0136] First, the data labeling module 210b classifies multiple training data into various categories according to predetermined categories (S510).

[0137] Then, the feature extraction module 220 extracts features for the object using the training data included in each category (S520).

[0138] Then, the effectiveness evaluation module 230 evaluates the effectiveness of the features extracted by the feature extraction module 220 (S530).

[0139] Then, the category verification module 240 performs verification on the predefined categories (i.e., the categories used to classify multiple training data in step S510) (S540).

[0140] Then, the data reconstruction module 250 performs clustering on multiple training data based on the features that are deemed valid by the validity evaluation module 230 and the categories verified by the category verification module 240 (S550).

[0141] Above, refer to Figures 2 to 7 The image data classification method of the substrate inspection unit 200 has been described. The substrate inspection method utilizing the image data of the substrate inspection unit 200 will be described below.

[0142] Figure 8 This is an example diagram schematically illustrating internal modules related to a second method constituting a substrate inspection unit of a substrate processing apparatus according to an embodiment of the present invention. Furthermore, Figure 9 This is an example diagram illustrating a second method of a substrate inspection unit constituting a substrate processing apparatus according to an embodiment of the present invention. Referring below... Figure 8 and Figure 9 Please provide an explanation.

[0143] First, when the image data of the substrate G is acquired by the vision module 125 (S710), the data processing module 610 of the substrate inspection unit 200 processes the image data (S720). In the above, the image data of the substrate G acquired by the vision module 125 may be the image data of the substrate G on which the substrate processing liquid is sprayed out by the inkjet head unit 140.

[0144] Then, the reference data detection module 620 of the substrate inspection unit 200 detects the reference data (S730) from multiple training data that are classified according to the image data classification method, and uses them as the comparison object with the image data of the substrate G. The reference data detection module 620 can determine the category that is related to the image data of the substrate G in a predefined category, and then detect the reference data from the training data that are classified into the corresponding category.

[0145] Step S730 can be executed after step S720, or it can be executed simultaneously with step S720. Alternatively, step S730 can be executed before step S720. On the other hand, in this embodiment, the image data classification method can be executed before step S730, and then step S730 can be executed.

[0146] Then, the data analysis module 630 of the substrate inspection unit 200 compares and analyzes the image data and reference data of the substrate G (S740).

[0147] Then, the substrate discrimination module 640 of the substrate inspection unit 200 determines whether the substrate G is in a good or bad state based on the above comparison and analysis results (S750). For example, when pixel printing is performed on the substrate G, the substrate discrimination module 640 can determine that the substrate G is in a good state if it is determined that there are no linear defects or area spots caused by impact errors on the substrate G, and can determine that the substrate G is in a bad state if it is determined that there are such linear defects or area spots on the substrate G.

[0148] The present invention described above relates to a method performed by a substrate inspection unit 200, and to a semi-auto image labeling algorithm based on unsupervised learning technology. The algorithm relates to a semi-auto labeling algorithm capable of improving prediction accuracy by validating and supplementing training image data manually labeled by an operator in the initial steps of applying deep learning.

[0149] Typically, the training data used in deep learning algorithms is constructed by pre-defining classification categories in accordance with the objective and then labeling them. Because the training data is constructed in this pre-designed manner, it is used as input data for deep learning algorithms without additional validation tasks.

[0150] However, in the manufacturing field, when performing good / bad image detection based on deep learning, additional validation operations are required on the training data. This is because it is difficult to predict in advance how the good / bad image data will be generated before the device is started. Therefore, unlike typical training data composition, the problem of pre-defining and labeling categories arises.

[0151] Furthermore, in the image data collection step, N operators define categories and perform manual labeling, and then the categories are finally merged to form the training data. Since this step individually reflects the opinions of the N operators, a validation task is required to verify whether the training data conforms to the categories.

[0152] The prediction accuracy of deep learning algorithms is greatly affected by the composition of the training data, the model architecture, and the number of training iterations. Among these, the composition of the training data can be considered the most influential factor.

[0153] The purpose of the deep learning framework proposed in this invention is to improve classification prediction accuracy by manually labeling, validating, and reconstructing training data (Semi-auto Labeling) for N operators to conform to predefined category criteria. In this invention, by thereby improving the reliability of the training data, more accurate prediction and classification of various equipment defects can be achieved.

[0154] The deep learning framework proposed in this invention is implemented as follows. As described in the section on the shortcomings of the prior art, in the manufacturing field, it is necessary to verify the training data and reconstruct the training data by re-performing labeling, thereby improving prediction accuracy. In this invention, the series of processes for verifying and reconstructing training image data in this way is defined as semi-auto image labeling.

[0155] First, based on predefined classification categories, N operators perform manual image labeling.

[0156] Then, for each category in the training data that was first manually labeled with images, features are extracted through transfer learning based on a pre-learning model specific to general image data.

[0157] The effectiveness of the features is then confirmed by reducing the dimensionality of the features extracted by the t-SNE algorithm, thereby improving the computational processing speed.

[0158] Then, DBSCAN, based on unsupervised learning, is used to verify whether the predefined classification criteria for each category are appropriate.

[0159] Clustered images are retained because they fit into their respective categories, while unclustered images are excluded because they don't fit into their categories, thus reconstructing the training image data. This process is repeated for the remaining categories to reconstruct the entire training image data.

[0160] The features of the present invention described above are summarized below.

[0161] First, by performing data consistency verification based on unsupervised learning techniques on training image data manually labeled by N operators in the initial steps of deep learning, and then using training image data that is more accurately classified through semi-automatic image labeling as input data for deep learning, prediction accuracy can be improved.

[0162] Second, semi-automatic image labeling can be achieved by extracting features from training image data through transfer learning, performing feature dimensionality reduction based on the t-SNE algorithm, and then using DBSCAN as unsupervised learning.

[0163] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical concept or essential features. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

Claims

1. A substrate inspection unit, comprising: The feature extraction module extracts features from the training data included in each category, given that multiple training data related to the image data of the substrate are classified according to predefined categories. The effectiveness evaluation module evaluates the effectiveness of the stated features; The category verification module, after the validity evaluation module evaluates the validity of the feature, verifies the predefined category based on the appropriateness of each predefined category classification criterion; as well as The data reconstruction module reconstructs the multiple training data based on the features judged to be valid and the verified categories. The reconstructed training data is used when inspecting the substrate.

2. The substrate inspection unit according to claim 1, wherein, The feature extraction module extracts the features using a pre-learning model, which is the result of learning from data above a certain benchmark related to the image data of the substrate.

3. The substrate inspection unit according to claim 1, wherein, The effectiveness evaluation module evaluates the effectiveness of the features by applying a dimensionality reduction method to the features.

4. The substrate inspection unit according to claim 3, wherein, The effectiveness evaluation module uses the t-SNE algorithm as the dimensionality reduction method.

5. The substrate inspection unit according to claim 1, wherein, The category verification module uses non-hierarchical clustering analysis to verify the defined categories.

6. The substrate inspection unit according to claim 5, wherein, The category verification module uses a density-based clustering method to verify the defined categories.

7. The substrate inspection unit according to claim 1, wherein, The category verification module uses clustering analysis based on unsupervised learning to verify the defined categories.

8. The substrate inspection unit according to claim 1, further comprising: The data volume discrimination module determines whether the amount of training data included in each category is above the baseline amount.

9. The substrate inspection unit according to claim 8, wherein, When the amount of training data included in each category is less than the baseline amount, the feature extraction module extracts the features using a pre-learning model, which is the result of learning using data above the baseline amount related to the image data of the substrate.

10. The substrate inspection unit according to claim 1, further comprising: The data labeling module directly classifies the multiple training data according to the predefined categories.

11. The substrate inspection unit according to claim 1, further comprising: The label information acquisition module is input or receives information about the training data classified by category and information about the predefined categories.

12. The substrate inspection unit according to claim 1, wherein, The plurality of training data are at least one of image data of a substrate in good condition and image data of a substrate in poor condition.

13. The substrate inspection unit according to claim 1, wherein, The multiple training data are image data of the substrate from which the substrate processing liquid is sprayed.

14. The substrate inspection unit according to claim 1, further comprising: The data processing module processes the image data of the substrate when the image data of the substrate is acquired. The benchmark data detection module detects benchmark data. The data analysis module compares and analyzes the image data of the substrate and the reference data; as well as The substrate discrimination module determines whether the substrate is good or bad based on the comparison and analysis results between the image data of the substrate and the reference data.

15. The substrate inspection unit according to claim 14, wherein, The benchmark data detection module determines the category associated with the image data of the substrate within the predefined categories, and detects the benchmark data from the training data included in the determined categories.

16. The substrate inspection unit according to claim 1, wherein, The substrate inspection unit reconstructs the plurality of training data when the device for processing the substrate is driven at least once.

17. A substrate inspection unit, comprising: The feature extraction module extracts features from the training data included in each category, given that multiple training data related to the image data of the substrate are classified according to predefined categories. The effectiveness evaluation module evaluates the effectiveness of the stated features; The category verification module, after the validity evaluation module evaluates the validity of the feature, verifies the predefined category based on the appropriateness of each predefined category classification criterion; as well as The data reconstruction module reconstructs the multiple training data based on the features judged to be valid and the verified categories. Among these methods, reconstructed training data is used when inspecting the substrate. The feature extraction module extracts the features using transfer learning from a pre-learning model, which is the result of learning from a benchmark amount of data related to the image data of the substrate. The effectiveness evaluation module evaluates the effectiveness of the features by applying a dimensionality reduction method to the features. The category verification module uses DBSCAN, based on unsupervised learning, to verify the predefined categories, and The plurality of training data are image data of the substrate being sprayed with substrate processing liquid, and are at least one of image data of a substrate in good condition and image data of a substrate in bad condition.

18. A substrate processing apparatus, comprising: A process unit that supports the substrate during substrate processing; The inkjet head unit sprays substrate processing liquid onto the substrate. A frame unit is provided with the inkjet head unit, and the inkjet head unit is movable above the substrate; as well as The substrate inspection unit inspects the substrate. The substrate inspection unit includes: The feature extraction module extracts features from the training data included in each category, given that multiple training data related to the image data of the substrate are classified according to predefined categories. The effectiveness evaluation module evaluates the effectiveness of the stated features; The category validation module, after the validity evaluation module evaluates the validity of the features, validates the predefined categories based on the appropriateness of each predefined category classification criterion; and The data reconstruction module reconstructs the multiple training data based on the features judged to be valid and the verified categories. The reconstructed training data is used when inspecting the substrate.

19. The substrate processing apparatus according to claim 18, wherein, The feature extraction module extracts the features using transfer learning of a pre-learning model, which is the result of learning from data above a benchmark quantity related to the image data of the substrate.

20. The substrate processing apparatus according to claim 18, wherein, The category verification module uses DBSCAN, an unsupervised learning-based algorithm, to verify the defined categories.

Images