Information processing apparatus, information processing method, and program

Abstract

To provide an information processor capable of highly accurately determining whether or not abnormality is included in a determination object.SOLUTION: An information processing device including a determination processing unit that performs determination regarding an abnormality on a histogram representing a frequency of a value of a determination target having a plurality of values, on the basis of a feature amount representing a degree of asymmetry between a portion where the value is smaller than a mode and a portion where the value is larger than the mode, the feature amount being obtained by including a term of an N-th power of a difference between each of the values and the mode, adjusting the term or a sign of the term, and summing the terms, where N> 0 represents an order.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art In a manufacturing process of a glass substrate, an inspection is carried out for abnormalities occurring in the glass substrate, such as defects occurring in the glass substrate. In defect inspection of glass substrates, the inspection quality required is now so high that even extremely small defects, such as defects of 100 nm or less, are no longer acceptable. As a method for detecting defects in glass substrates, for example, a method using an optical inspection machine is mainly used (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-105930 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, it can be difficult to determine whether or not an image captured using an optical inspection machine contains a defect when it is extremely small, such as a defect of 100 nm or less.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, and a program that can determine with high accuracy whether an object to be determined contains an abnormality. [Means for solving the problem]

[0006] One aspect of the present disclosure is an information processing device that includes a judgment processing unit that makes a judgment regarding an abnormality based on a feature that represents the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, obtained by summing up a histogram that represents the frequency of values ​​of a target to be judged having multiple values, the histogram including a term to the Nth power of the difference between each of the values ​​and a mode, where N>0 represents the degree of order, and adjusting the sign of the term or the term.

[0007] One aspect of the present disclosure is an information processing method in which an information processing device makes a determination regarding an abnormality based on a feature that represents the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, the feature being obtained by summing up a term that is the Nth power of the difference between each value and a mode, where N>0 represents the order, for a histogram that represents the frequency of the values ​​of a target to be determined.

[0008] One aspect of the present disclosure is a program for enabling a computer to: acquire an object to be evaluated having multiple values; calculate, for a histogram representing the frequency of the values ​​of the object to be evaluated, a feature value that represents the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, the feature value being obtained by summing up a term to the Nth power of the difference between each value and the mode, where N>0 represents the degree of order, and adjusting the sign of the term or the term; and perform a judgment regarding an abnormality based on the feature value. [Effects of the Invention]

[0009] According to the information processing device, information processing method, and program of the present disclosure, it is possible to determine with high accuracy whether an abnormality is included in the determination target. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a diagram illustrating an example of the configuration of functional blocks of the information processing device according to the embodiment. [Figure 2]FIG. 10 is a diagram illustrating an example of a procedure of a process performed by the information processing apparatus according to the embodiment. [Figure 3A] FIG. 10 is a diagram illustrating an example of image data conversion according to the embodiment. [Figure 3B] FIG. 10 is a diagram illustrating an example of a luminance histogram of an image according to the embodiment. [Figure 4A] FIG. 10 is a diagram schematically illustrating an example of a pseudo defect image. [Figure 4B] FIG. 10 is a diagram showing an example of a brightness histogram of a pseudo defect image. [Figure 5A] FIG. 10 is a diagram schematically illustrating an example of a defect image in which a large, high-brightness defect exists. [Figure 5B] FIG. 10 is a diagram showing an example of a brightness histogram of a defect image in which a large, high-brightness defect exists; [Figure 6A] FIG. 10 is a diagram schematically illustrating an example of a defect image in which a large, low-brightness defect exists. [Figure 6B] FIG. 10 is a diagram showing an example of a brightness histogram of a defect image in which a large, low-brightness defect is present; [Figure 7A] FIG. 10 is a diagram schematically illustrating an example of a defect image in which a defect of a standard size exists. [Figure 7B] FIG. 10 is a diagram showing an example of a brightness histogram of a defect image in which a defect of a standard size is present; [Figure 8A] 10A and 10B are diagrams illustrating an example of the state of probability moments for a pseudo defect image according to the embodiment. [Figure 8B] 10A and 10B are diagrams illustrating an example of the state of probability moments for a defect image in which a low-brightness large defect exists according to the embodiment; [Figure 8C] FIG. 10 is a diagram schematically illustrating an example of the state of probability moments for a defect image in which a defect of a standard size exists according to the embodiment. [Figure 8D] FIG. 10 is a diagram illustrating an example of quantification using probability moments for a defect image according to an embodiment. [Figure 9A] FIG. 10 is a diagram illustrating an example of a luminance histogram of a defect image according to the embodiment. [Figure 9B]FIG. 10 is a diagram illustrating an example of a luminance histogram in which the most frequent value is normalized to zero according to the embodiment. [Figure 9C] FIG. 10 is a diagram showing an example of a luminance histogram of a background portion after complementation according to the embodiment. [Figure 10] FIG. 10 is a diagram showing a table showing the judgment results according to the embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a brightness histogram of a defect image containing a large, low-brightness defect when the effect of the defect appears on the negative side of the mode. [Figure 12A] FIG. 10 is a diagram illustrating an example of the real part of the Nth moment when the order N is a real number according to the embodiment. [Figure 12B] FIG. 10 is a diagram illustrating an example of an imaginary part of an Nth moment when the order N is a real number according to the embodiment. [Figure 12C] FIG. 10 is a diagram illustrating an example of the norm of an Nth moment when the order N is a real number according to the embodiment. [Figure 13A] 1A and 1B are diagrams illustrating an example of a dark-field defect inspection according to an embodiment. [Figure 13B] FIG. 2 is a diagram schematically illustrating an example of a defect image obtained by a dark-field method according to the embodiment. [Figure 13C] 10A and 10B are diagrams illustrating quantification of a defect signal amount when the order N is 1 according to the embodiment. [Figure 13D] 10A and 10B are diagrams illustrating the quantification of the defect signal amount when the order N is 3 according to the embodiment. [Figure 14A] FIG. 10 is a diagram illustrating an example of an analysis combining probability moments of two orders according to an embodiment. [Figure 14B] FIG. 10 is a diagram showing another example of an analysis combining probability moments of two orders according to the embodiment. [Figure 15] FIG. 2 is a diagram illustrating an example of the configuration of functional blocks of the information processing device according to the embodiment. [Figure 16A] FIG. 10 is a diagram illustrating an example of input and output of a learning model during learning according to an embodiment. [Figure 16B]FIG. 10 is a diagram illustrating an example of input and output of a trained learning model according to an embodiment. [Figure 17] FIG. 10 is a diagram illustrating an example of a learning device and a determination device according to a modified example of the embodiment. [Figure 18] FIG. 2 is a diagram illustrating an example of the hardware configuration of an information processing unit according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following, for the sake of convenience, a symmetrical shape may be exemplified, but in reality, for example, it does not have to be strictly symmetrical, and may also be symmetrical to the extent that it does not cause problems in practical use (i.e., approximately symmetrical). Examples of symmetry to the extent that it does not cause problems in practical use include symmetry with an error that is not considered to be caused by an abnormality that requires detection. For the sake of convenience, in the following description, in addition to general probability moments, the general probability moments extended in this embodiment may also be referred to as probability moments. The image diagrams, histogram diagrams, and numerical examples shown below are for illustrative purposes only and are not necessarily strict or limiting.

[0012] [Information processing device] FIG. 1 is a diagram showing an example of the functional block configuration of an information processing device 11 according to the embodiment. The information processing device 11 is configured using, for example, a computer.

[0013] The information processing device 11 includes an input unit 131, an output unit 132, a communication unit 133, a storage unit 134, a control unit 135, and a determination processing unit. The input unit 131 includes an operation unit 151 . The output unit 132 includes a display unit 152 .

[0014] The input unit 131 receives information from the outside. The input unit 131 has, for example, an operation unit 151 that accepts an operation performed by a user, and inputs information according to the operation accepted by the operation unit 151. The operation unit 151 may have a function to accept operations from a touch panel, or may have a function to accept operations from physical keys. The operation unit 151 may also have a function to accept operations by voice (for example, the user's voice). Furthermore, the input unit 131 may be connected to an external device and receive information output from the external device. The external device may be, for example, a portable recording medium. The recording medium may be called, for example, a storage medium.

[0015] The output unit 132 outputs the information. The output unit 132 has, for example, a display unit 152, and displays (outputs) information on the screen of the display unit 152. The screen may have a touch panel function. The output unit 132 may also be connected to an external device and output information to the external device. The external device may be, for example, a portable recording medium. The output unit 132 may output information in a form other than a display, such as audio output.

[0016] In this embodiment, the input unit 131 and the output unit 132 are shown as separate functional units, but the input unit 131 and the output unit 132 may be configured as a common functional unit (input / output unit). The input / output unit may be configured as a touch panel.

[0017] The communication unit 133 has a function of communicating information with an external device. This communication may be wired or wireless. Here, in this embodiment, the communication unit 133 is shown as a functional unit separate from the input unit 131 and the output unit 132, but the receiving function of the communication unit 133 may be considered to be included in the function of the input unit 131, and the transmitting function of the communication unit 133 may be considered to be included in the function of the output unit 132.

[0018] The storage unit 134 stores information. The storage unit 134 may store any information.

[0019] The control unit 135 performs various controls and processes. In this embodiment, the control unit 135 includes a processor such as a CPU (Central Processing Unit), and executes a predetermined program (control program) by the processor to perform control and processing as defined in the program. The program may be stored in the storage unit 134, for example.

[0020] The determination processing unit 136 makes a predetermined determination by utilizing the quantification of the left-right asymmetry of the histogram. In this embodiment, the determination is a determination of the presence or absence of a predetermined abnormality. In this embodiment, an image analysis algorithm that utilizes quantification of the left-right asymmetry of the histogram is used to make such a determination.

[0021] First, the overall flow of the process will be outlined with reference to FIG. FIG. 2 is a diagram showing an example of a procedure of processing performed by the information processing device 11 according to the embodiment. In this embodiment, the process shown in FIG.

[0022] (Step S1) The determination processing unit 136 acquires the image data (image data). Then, the determination processing unit 136 proceeds to the process of step S2. Here, the image data is a determination target. Instead of a determination target, the image data may be called an inspection target or an abnormality detection target. The image data may be input to the information processing device 11 from outside the information processing device 11, or may be stored in advance in the storage unit 134 of the information processing device 11, for example. The image data is read as numerical data of pixel values ​​(in this embodiment, brightness) arranged on a two-dimensional plane.

[0023] (Step S2) The determination processing unit 136 converts the acquired image data from three-dimensional data to one-dimensional data, and then the determination processing unit 136 proceeds to the process of step S3. In this conversion process, three-dimensional image data (X-axis coordinate value, Y-axis coordinate value, brightness) consisting of coordinate values ​​on two mutually orthogonal axes (for example, X-axis coordinate value, Y-axis coordinate value) and brightness is converted into one-dimensional data consisting of brightness. In other words, brightness data arranged on a two-dimensional plane is converted into brightness data arranged in one dimension.

[0024] (Step S3) The determination processing unit 136 normalizes the histogram of the converted one-dimensional data so that the mode becomes zero (that is, mode=0). Then, the determination processing unit 136 proceeds to the process of step S4. In this embodiment, the normalization process is performed by subtracting the most frequent value from each value of the one-dimensional data to set the most frequent value to zero.

[0025] (Step S4) The determination processing unit 136 extracts information on background areas that are deemed to have no abnormal areas from the histogram of the converted one-dimensional data and performs predetermined interpolation processing. Then, the determination processing unit 136 proceeds to the processing of step S5.

[0026] (Step S5) The determination processing unit 136 calculates (approximately calculates in this embodiment) the standard deviation of the pixel values ​​(luminance in this embodiment) of the background portion after the interpolation processing. Then, the determination processing unit 136 proceeds to the processing of step S6.

[0027] (Step S6) The determination processing unit 136 quantifies the asymmetry of the histogram of the converted one-dimensional data, and acquires the result of the quantification as a feature amount. Then, the determination processing unit 136 proceeds to the process of step S7. In this embodiment, a predetermined evaluation formula is used to calculate a value representing the degree of asymmetry of the histogram, and the evaluation result (calculation result) by the evaluation formula is used as the feature amount. The asymmetry of the histogram is asymmetry in the distribution of pixel values ​​(in this embodiment, luminance). When the pixel values ​​(in this embodiment, luminance) are plotted on the horizontal axis (left-right direction), the asymmetry becomes left-right asymmetry.

[0028] (Step S7) The determination processing unit 136 compares the acquired feature amount with a predetermined threshold value, and determines whether the feature amount is greater than the predetermined threshold value. As a result of this determination, if the determination processing unit 136 determines that the acquired feature amount is greater than the predetermined threshold, the process proceeds to step S8. On the other hand, if the determination processing unit 136 determines that the acquired feature amount is equal to or less than the predetermined threshold value, the process proceeds to step S9. Here, any value may be set as the predetermined threshold value.

[0029] (Step S8) If the determination processing unit 136 determines that the acquired feature amount is greater than a predetermined threshold, it assigns the label 0 to the feature amount. As a result, the label 0 is assigned to the determination target. Then, the determination processing unit 136 proceeds to the processing of step S10.

[0030] (Step S9) If the determination processing unit 136 determines that the acquired feature amount is equal to or less than a predetermined threshold, it assigns the label 1 to the feature amount. As a result, the label 1 is assigned to the object to be determined. Then, the determination processing unit 136 proceeds to the process of step S10.

[0031] (Step S10) The determination processing unit 136 outputs the determination result, and the process of this flow then ends. The determination result may be, for example, a determination result that the feature amount to be determined is abnormal, or a determination result that the feature amount to be determined is not abnormal (for example, normal). The determination result may be, for example, information that associates image data, feature amounts of the image data, and labels that classify the image data. The information may be stored in, for example, the storage unit 134. The manner in which the judgment result is output may be, for example, to display and output the contents of the information of the judgment result on the display unit 152, or to output the information of the judgment result to an external device. The association may be called, for example, linking.

[0032] As described above, in this embodiment, a label such as label 0 or label 1 is assigned to each image to be determined. Here, the label 0 may be, for example, a value of 0, or may be other information. Also, label 1 may be, for example, a value of 1, or may be other information.

[0033] In this embodiment, label 0 indicates an abnormality and label 1 indicates no abnormality, but as another example, these may be reversed. That is, in this embodiment, a case is shown in which a feature value having a property that the larger the feature value, the more likely it is to be abnormal is used, but conversely, a feature value having a property that the smaller the feature value, the more likely it is to be abnormal may also be used. In this embodiment, a binary label is used as an example, but as another example, a ternary or higher value label representing the likelihood of normality or abnormality may be used.

[0034] In the example of FIG. 2, the processing portion including the processing of step S2 and the processing of step S3 is called pre-processing T1. In the example of FIG. 2, the processing portion including steps S4 to S6 is called feature extraction processing T2. In the example of FIG. 2, the processing portion including steps S7 to S9 is called classification processing T3.

[0035] For example, in the information processing device 11, the determination processing unit 136 may include a preprocessing unit that performs preprocessing T1, a feature extraction processing unit that performs feature extraction processing T2, and a classification processing unit that performs classification processing T3. The division of processes into pre-processing T1, feature extraction processing T2, and classification processing T3 is an example, and the process is not necessarily limited to this.

[0036] Next, the process shown in FIG. 2 will be described in more detail. 3A and 3B, the transformation of image data and the luminance histogram of the image are illustrated. FIG. 3A is a diagram showing an example of conversion of image data A1 according to the embodiment. FIG. 3B is a diagram showing an example of a luminance histogram B1 of an image according to the embodiment.

[0037] In FIG. 3A, image data A1 is shown as luminance data arranged on a two-dimensional plane. In the example of Figure 3A, image data A1 has 1024 pixels in the vertical direction and 1024 pixels in the horizontal direction, and has 1048576 (=1024 x 1024) pixels in a two-dimensional plane. Each pixel has a pixel value, which in this example represents luminance. As the brightness value, for example, a value between 0 and 255 may be used, or other values ​​may be used.

[0038] Image data A1, which is luminance data arranged on a two-dimensional plane, is converted into one-dimensional data A2. FIG. 3A shows an example of one-dimensional data A2. In the example of FIG. 3A, the one-dimensional data A2 is data in which 1048576 (=1024×1024) pixel values ​​are arranged. In the example of FIG. 3A, the one-dimensional data A2 corresponds to data in which pixel values ​​are arranged in order from the top row to the bottom row of the image data A1, and in order from left to right in each row. It should be noted that other methods may be used to convert three-dimensional data into one-dimensional data.

[0039] FIG. 3B shows an example of a luminance histogram B1 of one-dimensional data A2 obtained from image data A1. In the graph shown in FIG. 3B, the horizontal axis represents luminance [-] (unitless), and the vertical axis represents frequency [count]. It is also possible to visualize the frequency of occurrence of each brightness by displaying the brightness histogram B1.

[0040] Here, the image to be determined is not limited to this example, and various images may be used. As the image to be determined, for example, an 8-bit image may be used, or an image of other bits may be used. The image to be determined may be, for example, a grayscale image, a black and white image, or a color image. As the image to be determined, for example, an image in the PNG (Portable Network Graphics) format may be used, or an image in another format may be used.

[0041] In this embodiment, an example is given of a case in which an image (captured image) of a glass substrate captured by an imaging unit (e.g., a visible light camera) of an optical inspection machine installed on a production line during a glass substrate manufacturing process is used as the object of judgment. In this embodiment, if it is determined that a defect has occurred on the glass substrate based on an image of the glass substrate, it is determined that there is an abnormality, and if it is determined that there is no defect on the glass substrate, it is determined that there is no abnormality. The defect may also be called a flaw.

[0042] The pseudo defect image will be described with reference to FIGS. 4A and 4B. FIG. 4A is a diagram schematically illustrating an example of the pseudo defect image C11. The pseudo-defect image C11 is an example of an image output from an optical inspection machine, and represents an image in which there is no defect on the glass substrate but which has been erroneously recorded as a defect. FIG. 4B is a diagram showing an example of a brightness histogram D11 of the pseudo defect image C11. In the graph shown in FIG. 4B, the horizontal axis represents luminance [-] (unitless), and the vertical axis represents frequency [count]. In the pseudo defect image C11, the frequency of brightness follows a normal distribution.

[0043] The defect images will be described with reference to FIGS. 5A and 5B, 6A and 6B, and 7A and 7B. FIG. 5A is a diagram schematically illustrating an example of a defect image C1 in which a large, high-brightness defect exists. Defect image C1 is an example of an image output from an optical inspection machine, and represents an image in which a large, high-brightness defect has occurred on a glass substrate. FIG. 5A shows a defect portion E1 in a defect image C1. FIG. 5B is a diagram showing an example of a brightness histogram D1 of a defect image C1 in which a large, high-brightness defect exists. In the graph shown in FIG. 5B, the horizontal axis represents luminance [-] (unitless), and the vertical axis represents frequency [count]. In FIG. 5B, the portion of defect image C1 that is affected by defect portion E1 is shown as defect portion F1. When there is an actual defect (real defect), there are pixels whose luminance frequency does not follow a normal distribution. In this embodiment, a case where there are high-luminance pixels whose luminance frequency does not follow a normal distribution is illustrated as an example.

[0044] FIG. 6A is a diagram schematically illustrating an example of a defect image C2 in which a large, low-brightness defect exists. Defect image C2 is an example of an image output from an optical inspection machine, and represents an image in which a large, low-brightness defect has occurred on a glass substrate. FIG. 6A shows a defect portion E2 in a defect image C2. FIG. 6B is a diagram showing an example of a brightness histogram D2 of a defect image C2 in which a large, low-brightness defect exists. In the graph shown in FIG. 6B, the horizontal axis represents luminance [-] (unitless), and the vertical axis represents frequency [count]. In FIG. 6B, the portion of defect image C2 that is affected by defect portion E2 is shown as defect portion F2. When there is an actual defect (real defect), there are pixels whose luminance frequency does not follow a normal distribution. In this embodiment, a case where there are high-luminance pixels whose luminance frequency does not follow a normal distribution is illustrated as an example.

[0045] FIG. 7A is a diagram schematically illustrating an example of a defect image C3 in which a defect of a standard size exists. Defect image C3 is an example of an image output from an optical inspection machine, and represents an image in which a defect of a standard size occurs on a glass substrate. 7A shows a case where the defect portion in defect image C3 cannot be classified by visual inspection. Note that even defects of a standard size can become visible by increasing the brightness of the image. FIG. 7B is a diagram showing an example of a brightness histogram D3 of a defect image C3 in which a defect of a standard size exists. In the graph shown in FIG. 7B, the horizontal axis represents luminance [-] (unitless), and the vertical axis represents frequency [count]. In FIG. 7B, the portion of defect image C3 that is affected by the defect portion is shown as defect portion F3. When there is an actual defect (real defect), there are pixels whose luminance frequency does not follow a normal distribution. In this embodiment, a case where there are high-luminance pixels whose luminance frequency does not follow a normal distribution is illustrated as an example.

[0046] The image classification method using probability moments will be described with reference to FIGS. 8A, 8B, 8C, and 8D.

[0047] [When degree N is odd] The calculation formula shown in formula (1) is an example of an evaluation formula used in this embodiment.

[0048]

number

[0049] In equation (1), a value (feature amount in this embodiment) representing the evaluation result for one image data is set to z. The right-hand side of equation (1) represents the probability moment around the normalized mode. In equation (1), n ​​is the total number of values ​​(in this embodiment, values ​​representing brightness) possessed by the one-dimensional data obtained from the image data (in this embodiment, the total number of pixels in one image), the variable i represents an integer between 1 and n, mode represents the most frequent value of the values ​​(in this embodiment, the most frequent brightness value for the image data), and σ represents the standard deviation of the background area (frequency of brightness of background noise) that is considered to have no abnormal areas. In equation (1), xi represents the i-th luminance in the one-dimensional data. In equation (1), (xi-mode) on the right side represents the distance between each value xi and the most frequent value mode. In equation (1), N represents the order, which is a positive value. If it is assumed that xi in equation (1) is a value after the most frequent value has already been subtracted from each luminance, then mode in equation (1) becomes zero.

[0050] Here, the case where the degree N is an odd number will be described. When the order N is odd, the signs (positive and negative signs) of the Nth power results are opposite between the side lower than the mode and the side higher than the mode, so when the distribution is symmetrical around the mode, the Nth power results of two points equidistant from the mode cancel each other out. This makes it possible to determine the presence or absence of an abnormality, for example, based on the magnitude relationship between the feature amount z and a predetermined value (threshold value).

[0051] As an example, when the degree N is 9, the calculation formula (1) is expressed as the calculation formula (2).

[0052]

number

[0053] FIG. 8A is a diagram schematically illustrating an example of the state of probability moments for a pseudo defect image according to the embodiment. FIG. 8A shows a luminance histogram D11 similar to the example of FIG. 4B. FIG. 8A also shows the most frequent luminance value G11. Here, the frequency of brightness in the pseudo defect image follows a normal distribution, so if there is no defect, the probability moment is 0. In other words, the brightness histogram D11 is completely symmetrical with respect to the mode G11.

[0054] FIG. 8B is a diagram schematically illustrating an example of the state of probability moments for a defect image in which a low-brightness large defect exists according to the embodiment. FIG. 8B shows a luminance histogram D2 similar to the example of FIG. 6B. FIG. 8B also shows the most frequent luminance value G2. Here, the frequency of the luminance of the defect image deviates from a normal distribution, and when a defect is present, the probability moment diverges to one side (the positive side, which is the right side in the example of FIG. 8B). In the example of FIG. 8B, the distance H1 in the left-right direction between the mode G2 and the defective portion F2 is shown. If there are many pixels with a brightness higher than the most frequent value G2, the odd probability moment will be large. In general, weighted summation is performed according to the distance from the most frequent value.

[0055] FIG. 8C is a diagram schematically illustrating an example of the state of probability moments for a defect image in which a defect of a standard size exists according to the embodiment. FIG. 8C shows a luminance histogram D3 similar to the example of FIG. 7B. FIG. 8C also shows the most frequent luminance value G3. Here, the frequency of the luminance of the defective image deviates from a normal distribution, and when a defect is present, the probability moment diverges to one side (the positive side, which is the right side in the example of FIG. 8C). In the example of FIG. 8C, the horizontal distance H2 between the most frequent value G3 and the defective portion F3, and the horizontal distance H3 between the most frequent value G3 of luminance and the minimum value are shown. If there are many pixels with a brightness higher than the mode G3, the odd probability moment will be large. In general, weighted summation is performed according to the distance from the mode.

[0056] FIG. 8D is a diagram schematically illustrating an example of quantification using probability moments for a defect image according to an embodiment. FIG. 8D shows a luminance histogram D2 similar to the example of FIG. 6B. In FIG. 8D, the brightness histogram D2 is shown divided into a normal portion I1 and an abnormal portion I2. The normal part I1 follows a normal distribution and converges to zero in the calculation of equation (1). On the other hand, the abnormal portion I2 is a portion that deviates from the normal distribution due to a defect, and a value (evaluation result z) corresponding to the abnormal portion I2 is quantified in the calculation of formula (1). In other words, only the abnormal portion I2 is quantified.

[0057] An example of extraction and complementation processing of a background portion that is deemed to have no abnormal portion will be described with reference to FIGS. 9A, 9B, and 9C. In the graphs shown in FIGS. 9A, 9B, and 9C, the horizontal axis represents luminance [-] (unitless), and the vertical axis represents frequency [count].

[0058] FIG. 9A is a diagram showing an example of a luminance histogram D21 of a defect image according to the embodiment. FIG. 9B is a diagram showing an example of a luminance histogram D22 in which the most frequent value is normalized to zero according to the embodiment. The luminance histogram D22 shown in Fig. 9B corresponds to the result of subtracting the most frequent value from each luminance value in the luminance histogram D21 shown in Fig. 9A, which results in the luminance histogram D22 shown in Fig. 9B having a mode value of zero. FIG. 9C is a diagram showing an example of a luminance histogram D23 of the background portion after interpolation according to the embodiment.

[0059] Here, the complementation process (complementation processing) in this embodiment will be described. In this embodiment, even in a defective image, brightness below the most frequent value in the brightness histogram is considered to be free of abnormalities, and is treated as a background portion that is considered to have no abnormal parts.

[0060] In the interpolation process, first, data portions whose brightness is equal to or greater than 0 and equal to or less than the most frequent value are extracted. Next, for each luminance value included in the extracted data portion, the distance (horizontal distance) to the most frequent value is calculated. Then, a data portion having a set of brightness corresponding to the result of adding each distance to the mode is generated, and the generated data portion is integrated with the extracted data portion to generate the result as a brightness histogram D23 of the background portion that is deemed to have no abnormal areas. At this time, only one of the generated data portion and the extracted data portion is used so that the mode does not overlap. As a result, the extracted data portion is placed on the low brightness side (left side) of the mode axis, and the generated data portion is placed on the high brightness side (right side) of the mode axis. The extracted data portion and the generated data portion are bilaterally symmetrical, and the brightness histogram D23 follows a normal distribution.

[0061] In this embodiment, the standard deviation of the frequency of luminance (luminance distribution) of the background portion after interpolation is used as σ in equation (1). The standard deviation of the luminance frequency (luminance distribution) of the background portion after complementation is smaller than, for example, the standard deviation of the luminance frequency (luminance distribution) of the defect image as it is before complementation.

[0062] Here, a case has been described in which there is no effect of defects on the side with lower brightness than the mode value, and there is an effect of defects on the side with higher brightness than the mode value. As another example, if there is no effect of defects on the side with higher brightness than the mode, but there is an effect of defects on the side with lower brightness than the mode, the data portion to the right of the mode in the brightness histogram D22 shown in Figure 9B is considered to be the background portion to be complemented, and a data portion is generated by folding that data portion to the left of the mode, and the two data portions are integrated.

[0063] <Example of probability moment calculation results> In the case of the pseudo defect image C11 in the example of FIGS. 4A and 4B, the calculation result of the probability moment (the value of z in equation (2)) was 205. 5A and 5B, the calculation result of the probability moment (the value of z in equation (2)) was 7.1×(1e9), where (1e9) represents 10 to the 9th power. In the case of the defect image C2 in the example of FIGS. 6A and 6B, the calculation result of the probability moment (the value of z in equation (2)) was 6150. In the case of the defect image C3 in the example of FIGS. 7A and 7B, the calculation result of the probability moment (the value of z in equation (2)) was 11,672.

[0064] FIG. 10 is a diagram showing a table 2011 showing the judgment results according to the embodiment. The evaluation results are those obtained using formula (2). The judgment was made on 30,821 image data, including 17,158 pseudo defect image data and 13,663 defect image data. As shown in Table 2011, of the 17,158 defects whose correct labels were actually pseudo defects, 17,157 were correctly determined to be pseudo defects and 1 was incorrectly determined to be a defect using the probability moment according to this embodiment (probability moment determination). Furthermore, as shown in table 2011, of the 13,663 defects whose correct labels were actually defects, 205 were erroneously determined to be false defects and 13,458 were correctly determined to be defects using the probability moment according to this embodiment (probability moment determination). As a result, the accuracy rate was 99.320%; the false positive rate was 0.006%; and the false negative rate was 1.500%. In this way, the probability moment determination according to this embodiment has sufficiently high accuracy.

[0065] [When degree N is odd] In equation (2), the case where the degree N is 9 is exemplified, but the same applies to the case where the degree N in equation (1) is another odd number.

[0066] [When degree N is even] When the degree N in equation (1) is an even number, we will explain the probability moment when extending the degree N. As a result, the degree N is extended from a positive odd number to a natural number. A probability moment with an even order N cannot determine whether the luminance histogram is skewed high or low (left or right). Therefore, for each pixel that constitutes the image to be judged, if the pixel value (in this embodiment, the brightness value) is lower than the mode, an equation is adopted in which the calculation result of the probability moment is multiplied by (-1).

[0067] Here, the calculation formula shown in formula (3) is an evaluation formula after sign extension, and represents the probability moment around the normalized mode. In equation (3), when the numerator of the fraction on the right side is negative, it becomes a positive value when raised to an even power, so a negative sign (-) is given to the even result. In this way, by extending the sign calculation of the probability moment, it becomes possible to use a positive even number as the order N.

[0068]

number

[0069] When the order N is an even number, the sign (positive or negative sign) of the Nth power result is the same on the side lower than the mode and the side higher than the mode, so by adjusting the sign, when the distribution is symmetrical around the mode, the Nth power results of two points equidistant from the mode cancel each other out. This makes it possible to determine the presence or absence of an abnormality, for example, based on the magnitude relationship between the feature value z and a predetermined value (threshold value).

[0070] FIG. 11 is a diagram showing an example of a luminance histogram D31 of a defect image in which a large, low-luminance defect exists when the effect of the defect (defective portion F31) appears on the negative side of the mode. In the graph shown in FIG. 11, the horizontal axis represents brightness [-] (no unit), and the vertical axis represents frequency [count]. Here, the luminance histogram D31 shown in FIG. 11 has a shape symmetrical (bilaterally symmetrical) between the low and high sides with respect to the luminance histogram D2 shown in FIG. 6B, with the mode value as the axis. Therefore, the sign of z in equation (3) is reversed when the effect of the defect appears on the lower side (left side) of the mode and when the effect of the defect appears on the higher side (right side) of the mode, and the absolute value is the same.

[0071] [If degree N is not a natural number but a real number such as a decimal] When the degree N in equation (1) is a real number such as a decimal, we will explain the probability moment when extending the degree N. As a result, the degree N is extended from a natural number to a positive real number. In order to be able to handle cases where the calculation result of the probability moment is a complex number, an equation that handles the norm of the calculation result of the probability moment is adopted.

[0072] Here, the calculation formula shown in equation (4) is an evaluation formula after sign extension, and represents the probability moment around the normalized mode. The symbol || in equation (4) represents taking the norm of the value therein. In equation (4), when the numerator of the fraction on the right side is negative, the norm is taken and the result becomes a positive value, so a negative sign (-) is given to the result. In this way, by extending the sign calculation of probability moments, it becomes possible to use positive real numbers as the order N.

[0073]

number

[0074] When the degree N is not a natural number but a real number such as a decimal, the norm of the Nth power result is taken, and the sign (positive or negative sign) is adjusted in the same way as when the degree N is an even number. This makes it possible to determine the presence or absence of an abnormality, for example, based on the magnitude relationship between the feature amount z and a predetermined value (threshold value).

[0075] FIG. 12A is a diagram illustrating an example of the real part of an Nth moment (Nth-order probability moment) when the order N is a real number according to the embodiment. In the graph shown in FIG. 12A, the horizontal axis represents the value resulting from subtracting the most frequent value from each luminance, and the vertical axis represents the real part of the value from which the norm is taken. FIG. 12B is a diagram illustrating an example of the imaginary part of the Nth moment when the order N is a real number according to the embodiment. In the graph shown in FIG. 12B, the horizontal axis represents the value resulting from subtracting the most frequent value from each luminance, and the vertical axis represents the imaginary part of the value from which the norm is taken. FIG. 12C is a diagram illustrating an example of the norm of the Nth moment when the order N is a real number according to the embodiment. In the graph shown in FIG. 12C, the horizontal axis represents the value obtained by subtracting the most frequent value from each luminance, and the vertical axis represents the norm.

[0076] FIG. 12A shows a characteristic 2101 when the order N is a decimal number. FIG. 12B shows a characteristic 2102 when the order N is a decimal number similar to that in FIG. 12A. FIG. 12C shows a characteristic 2103 when the order N is a decimal number similar to the case of FIGS. 12A and 12B. Here, in the examples of Figures 12A, 12B, and 12C, the respective characteristics when the order N takes on several different decimal points are summarized and shown as characteristic 2101, characteristic 2102, and characteristic 2103, as shown on the right side of each graph.

[0077] In this way, by setting the order N to a real number, it becomes possible to finely adjust the sensitivity, for example. As a specific example, taking the power of skewness results in a higher sensitivity to left-right symmetry than skewness, but even a slight error that exists even in false defects can lead to a false determination of a non-defective product as defective. In contrast, in this embodiment, the exponent (order N) is expanded to cover all real numbers, making it possible to fine-tune the sensitivity according to business issues.

[0078] The defect inspection using the dark field method will be described with reference to FIGS. 13A, 13B, 13C, and 13D. The determination using the probability moment according to this embodiment is particularly effective when applied to, for example, dark-field defect inspection.

[0079] FIG. 13A is a diagram schematically illustrating an example of a dark-field defect inspection according to the embodiment. In the example of Figure 13A, when light is irradiated onto the object to be determined (in this embodiment, a glass substrate), light from the background part where there is no defect (background light J1) and light from the part where there is a defect (defect light J2) are schematically shown.

[0080] FIG. 13B is a diagram schematically illustrating an example of a defect image C21 obtained by the dark-field method according to the embodiment. The defect image C21 includes a defect portion E21.

[0081] Here, equation (1) is applied to the dark-field defect image C21. When the order N is 9, equation (2) is used. By using such probability moments, the noise signal in the defect image C21 (in this embodiment, the background brightness) can be converged to zero, and only the signal (defect signal) caused by the defect portion can be quantified.

[0082] Furthermore, weighted integration allows for appropriate quantification of defect signal amounts according to the purpose. FIG. 13C is a diagram schematically illustrating quantification of the defect signal amount when the order N is 1 according to the embodiment. FIG. 13D is a diagram schematically illustrating quantification of the defect signal amount when the order N is 3 according to the embodiment. In this way, by using the probability moment, it is possible to appropriately evaluate the defect signal amount of the image to be judged.

[0083] [Example of analysis using combinations of probability moments] 14A and 14B, an example of an analysis using a combination of probability moments is shown. FIG. 14A is a diagram illustrating an example of an analysis combining probability moments of two orders according to an embodiment. In the graph shown in FIG. 14A, the horizontal axis represents the ninth-order probability moment (ninth moment), and the vertical axis represents the third-order probability moment (third moment). The graph shows multiple data points plotting the correspondence between the 9th moment and the 3rd moment for each of multiple pseudo defect images, as well as the characteristics 2201 of a curve fitted to these multiple data points. For example, it is possible to determine that data points on the line of the characteristic 2201 are normal, and that data points not on the line of the characteristic 2201 are abnormal.

[0084] FIG. 14B is a diagram showing another example of an analysis combining probability moments of two orders according to the embodiment. FIG. 14B shows a graph depicting a characteristic 2201 similar to that shown in FIG. 14A. In addition, in the example of Figure 14B, an area is shown with a predetermined width K1 on the side (upper side) where the vertical axis values ​​are high, and a predetermined width K2 on the side (lower side) where the vertical axis values ​​are low, relative to the line of characteristic 2201. For example, it is possible to determine that data points inside the region are normal and data points outside the region are abnormal. In other words, it is possible to regard the inside of a region having an approximate width to the line of characteristic 2201 as a normal region, and the outside of the region as an abnormal region. Here, any width may be used as the predetermined width K1 and the predetermined width K2. Note that the predetermined width K1 and the predetermined width K2 may be the same width or different widths, for example. Furthermore, for data points on the boundary of the region, a rule may be set to determine that they are normal, or a rule may be set to determine that they are abnormal.

[0085] Here, the examples of Figures 14A and 14B show cases where a combination of a probability moment with an order N of 9 and a probability moment with an order N of 3 is used, but combinations of probability moments of other orders may also be used.

[0086] In the example of Figure 14A or the example of Figure 14B, the judgment processing unit 136 has a first calculation function that calculates the probability moment when the order N is a first value (N1) and a second calculation function that calculates the probability moment when the order N is a second value (N2). Here, N1 and N2 are not equal, and N1 and N2 are positive values.

[0087] In this way, the relationship between two probability moments with different orders N (=N1, N2) falls within a certain range when the object to be judged (an image in this embodiment) does not contain an abnormality, so such a certain range is determined in advance using multiple objects (multiple images in this embodiment) that do not contain an abnormality. Then, when actually making a judgment, the relationship between the two probability moments is determined for the object to be judged, and if the relationship falls within the certain range, it is judged to be normal, and conversely, if the relationship does not fall within the certain range (for example, if there is a unique point outside the certain range), it is judged to be abnormal.

[0088] [Example of information processing device using machine learning] 15, 16A, 16B, and 17, a case where a function for obtaining a determination result similar to that of the determination processing unit 136 shown in FIG. 1 is realized by using machine learning will be described.

[0089] <Information processing device> FIG. 15 is a diagram illustrating an example of the functional block configuration of an information processing device 311 according to the embodiment. The information processing device 311 is configured using, for example, a computer.

[0090] The information processing device 311 includes an input unit 131 , an output unit 132 , a communication unit 133 , a storage unit 331 , and a control unit 332 . The input unit 131 includes an operation unit 151 . The output unit 132 includes a display unit 152 . The control unit 332 includes a learning control unit 171 and a determination control unit 172 .

[0091] Here, the input unit 131, the operation unit 151, the output unit 132, the display unit 152, and the communication unit 133 have the same functions as those shown in FIG. 1, for example. In the example of FIG. 15, these processing units are denoted by the same reference numerals as in the example of FIG.

[0092] The storage unit 331 stores information. The storage unit 331 may store any information. In the example of FIG. 15, a learning model Q1 and training data Q2 stored in the storage unit 331 are shown. As the learning model Q1, various learning models may be used, for example, a deep learning model may be used. While machine learning is being performed, learning model Q1 becomes a learning model under training, and after machine learning is completed, learning model Q1 becomes a trained learning model. Furthermore, the training data Q2 may be used for machine learning. However, if the training data Q2 is not used, the training data Q2 does not need to be stored in the storage unit 331.

[0093] The control unit 332 performs various controls and processes. In this embodiment, the control unit 332 includes a processor such as a CPU, and executes a predetermined program (control program) by the processor to perform control and processing defined in the program. The program may be stored in the storage unit 331, for example.

[0094] In the example of FIG. 15, the functions of the control unit 332 include a learning control unit 171 and a determination control unit 172. The learning control unit 171 controls the machine learning process. The determination control unit 172 controls the determination process using the results of machine learning. In the example of FIG. 15, the determination processing unit 351 is configured by the function of the determination control unit 172 making a predetermined determination using the learned learning model Q1.

[0095] <Overview of machine learning training> FIG. 16A is a diagram illustrating an example of input and output of a learning model Q1a during learning according to the embodiment. Here, learning model Q1a represents the state in which learning model Q1 shown in FIG. 15 is undergoing machine learning. In general, the learning control unit 171 updates the parameters of the learning model Q1a based on the output data when predetermined input data is input to the learning model Q1a. At this time, the learning control unit 171 may perform supervised learning using the training data Q2. As another example, unsupervised learning may be performed without using the training data Q2. It should be noted that the schematic diagram shown in FIG. 16A is a diagram for the purpose of general explanation and is not necessarily precise. The learning model under training may be referred to as, for example, a model under training.

[0096] <Outline of judgment using machine learning results> FIG. 16B is a diagram illustrating an example of input and output of the trained learning model Q1b according to the embodiment. Here, the learning model Q1b represents the state in which the learning model Q1 shown in FIG. 15 has undergone machine learning. In general, the determination control unit 172 obtains a predetermined determination result based on output data when predetermined input data is input to the learning model Q1b. The determination result may be, for example, the output data from the learning model Q1b itself, or may be other data obtained based on the output data from the learning model Q1b. It should be noted that the schematic diagram shown in FIG. 16B is a diagram for the purpose of general explanation and is not necessarily precise. A trained learning model may be referred to as, for example, a trained model.

[0097] Hereinafter, for the sake of convenience, the learning model Q1a and the learning model Q1b will not be distinguished from each other, and will be referred to as the learning model Q1. Hereinafter, unless it is explicitly stated that the model is "currently learning," the learning model Q1 represents a learned model (learning model Q1b).

[0098] In this embodiment, the output from the trained learning model Q1 (learning model Q1b in the example of FIG. 16B) will be described as an inference result representing the result of inference by the learning model Q1. Inference results from a machine learning learning model include, for example, classification results or regression results. In this embodiment, the case where the inference result is a classification result will be exemplified. Inference may also be referred to as, for example, prediction, estimation, or conjecture. Furthermore, in this embodiment, a case is shown in which a predetermined judgment is made based on an inference result, but for example, the inference result itself may be used as the result of the judgment, in which case the inference result essentially becomes the result of the judgment.

[0099] In this embodiment, the output (inference result) from the learning model Q1 is the dependent variable, and other items that are thought to affect the dependent variable are the explanatory variables. As a specific example, when the objective variable is the presence or absence of a defect, which indicates whether or not an image contains a defective part (an example of an abnormal part), the explanatory variable is the evaluation result (z value) calculated from the histogram of one-dimensional data obtained from the image. Here, the objective variable and explanatory variables are generally included in the feature quantity of the defect.

[0100] Specifically, the input data to learning model Q1 may include, for example, image data to be judged, one-dimensional data converted from the three-dimensional data that is the image data, or histogram data obtained from the one-dimensional data. In addition, the output data from the learning model Q1 may include, for example, information indicating whether or not the image data contains a defective portion, that is, information indicating the result of determining whether or not there is an abnormality in the image data.

[0101] <Example in which the learning function and the judgment function are provided in separate devices> Here, the example of FIG. 15 illustrates a case where the information processing device 311 has both a learning function for performing machine learning learning and a determination function for making a determination based on the machine learning result. As another example, the learning function and the judgment function may be provided in separate devices.

[0102] <Learning device> FIG. 17 is a diagram illustrating an example of a learning device 211 and a determination device 212 according to a modified example of the embodiment. In the example of FIG. 17, the learning device 211 and the determination device 212 are configured as separate devices. The learning device 211 has a configuration similar to that of the information processing device 311 shown in FIG. 15 except that the determination control unit 172 is not provided. Furthermore, if the learning device 211 does not use the teacher data Q2, it does not need to store the teacher data Q2.

[0103] The learning device 211 performs machine learning learning and provides the learning results (for example, a learned learning model) to the determination device 212. In addition, the transfer of learning results (e.g., a learned learning model) from the learning device 211 to the determination device 212 may be performed, for example, by communication between the learning device 211 and the determination device 212, or may be performed using a portable recording medium, etc. The communication between the learning device 211 and the determination device 212 may be performed via, for example, a server device or a relay device.

[0104] <Judgment device> The determination device 212 has a configuration similar to that of the information processing device 311 shown in FIG. 15 except that the learning control unit 171 is not included. Furthermore, if the determination device 212 does not use the teacher data Q2, the determination device 212 does not need to store the teacher data Q2.

[0105] The determination device 212 acquires a predetermined determination result based on the learning result (for example, a learned learning model) provided by the learning device 211.

[0106] [Example of hardware configuration for each device] Referring to FIG. 18, an example of the hardware configuration of each device is shown. FIG. 18 is a diagram illustrating an example of the hardware configuration of the information processing unit 1001 according to the embodiment. The configuration shown in Fig. 18 may be applied to the information processing device 11 shown in Fig. 1, the information processing device 311 shown in Fig. 15, or the learning device 211 or the determination device 212 shown in Fig. 17. In this case, the functions of each device are realized by the function of the information processing unit 1001 shown in Fig. 18. For example, the configuration of the information processing unit 1001 or a part of the configuration of the information processing unit 1001 may be used in any part of the information processing device 11, the information processing device 311, the learning device 211, or the determination device 212.

[0107] In the example of Figure 18, the information processing unit 1001 includes a processor 1011, an operation device 1012, a display device 1013, a storage device 1014, a memory 1015, an input / output interface 1016, a network interface 1017, and a bus 1021 connecting these. When the information processing unit 1001 is applied to any device, it is not necessarily required that all of the processing units shown in FIG. 18 are provided, and any processing unit not shown in FIG. 18 may be added.

[0108] The processor 1011 is composed of a CPU and the like, and executes a program to perform control and processing defined in the program. The operation device 1012 includes one or more input devices such as a keyboard and a mouse, and receives operations performed by a person (user) or the like. The display device 1013 has a screen and displays and outputs information on the screen.

[0109] The storage device 1014 is a non-volatile storage unit, and is configured, for example, by a hard disk, and stores information. The memory 1015 is a volatile storage unit, and is configured with RAM (Random Access Memory) or the like, and temporarily stores information. As the RAM, for example, a DRAM (Dynamic Random Access Memory) may be used. The storage device 1014 or memory 1015 may store information for a program executed by the processor 1011, for example.

[0110] The input / output interface 1016 is an interface for connecting to an external recording medium or the like. The network interface 1017 is an interface for connecting to an external network.

[0111] Here, the information processing unit 1001 may include one processor or two or more processors as the processor 1011. As an example, the information processing unit 1001 may include multiple CPUs, each of which executes its own control and processing, and these multiple CPUs may work together to realize overall control and processing.

[0112] As described above, the information processing device 11 according to this embodiment can determine with high accuracy whether or not an abnormality is included in the determination target. The information processing device 11 according to this embodiment can also distinguish between, for example, a pseudo defect image and an image in which an actual defect occurs.

[0113] In this embodiment, the case where image data of a glass substrate captured by an optical inspection machine during a manufacturing process is used as the object of judgment is exemplified. In this embodiment, attention is paid to the fact that in an image captured by an optical inspection machine, when there are no abnormalities (e.g., defects), the background brightness has a symmetric distribution shape around the mode, and when there is an abnormal part, the background brightness becomes asymmetric around the mode. Based on this, the technology according to this embodiment was developed. In other words, attention is paid to the fact that a brightness histogram created based on such image data has symmetry around the mode when the image does not contain an abnormal part, but this symmetry is broken when the image contains an abnormal part.

[0114] In this manner, in this embodiment, a value that quantifies the asymmetry of the brightness histogram obtained from an image is used as a feature, and whether or not the image contains an abnormal area is identified based on the feature. In the present embodiment, a case has been exemplified in which a histogram of the luminance of each pixel of an image (luminance histogram) is used, but a histogram of information other than luminance may also be used.

[0115] When an image is used as the object of determination, for example, the accuracy of determination tends to be higher as the number of pixels in the image increases. In other words, if the number of pixels in the image is small, the influence of errors tends to remain. As an example, it is preferable that the number of pixels of the image to be judged is 1 million or more, but this is not necessarily limited to this, and it is sufficient that the number of pixels is sufficient for practical use.

[0116] [Variation of the calculation formula] <Modification of formula (1)> Instead of the equation (1), the equation (5) may be used. That is, even if σ (standard deviation) in formula (1) is omitted, if a practically effective feature (z) can be obtained, formula (5) may be used as the evaluation formula. In this case, for example, the threshold may be adjusted according to formula (5). For example, when the influence of σ (standard deviation) is small, or when the influence of σ (standard deviation) can be known in advance and the threshold can be adjusted, equation (5) may be used.

[0117]

number

[0118] Instead of the equation (1), the equation (6) may be used. That is, even if the (1 / n) part in formula (1) is omitted, if a practically effective feature value (z) can be obtained, formula (6) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (6). For example, when the value of (1 / n) can be known in advance and the threshold can be adjusted, equation (6) may be used.

[0119]

number

[0120] Instead of the equation (1), the equation (7) may be used. That is, even if the σ (standard deviation) and (1 / n) parts in formula (1) are omitted, if a practically effective feature value (z) can be obtained, formula (7) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (7). For example, when the influence of σ (standard deviation) is small, or when the influence of σ (standard deviation) can be grasped in advance, and the value of (1 / n) can be grasped in advance, the threshold may be adjusted and equation (7) may be used.

[0121]

number

[0122] <Modification of formula (3)> Instead of the arithmetic expression shown in equation (3), the arithmetic expression shown in equation (8) may be used. That is, even if σ (standard deviation) in formula (3) is omitted, if a practically effective feature value (z) can be obtained, formula (8) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (8). For example, when the influence of σ (standard deviation) is small, or when the influence of σ (standard deviation) can be known in advance and the threshold can be adjusted, equation (8) may be used.

[0123]

number

[0124] Instead of the equation (3), the equation (9) may be used. That is, even if the (1 / n) part in formula (3) is omitted, if a practically effective feature quantity (z) can be obtained, formula (9) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (9). For example, when the value of (1 / n) can be known in advance and the threshold can be adjusted, equation (6) may be used.

[0125]

number

[0126] Instead of the equation (3), the equation (10) may be used. That is, even if the σ (standard deviation) and (1 / n) parts in formula (3) are omitted, if a practically effective feature value (z) can be obtained, formula (10) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (10). For example, when the influence of σ (standard deviation) is small, or when the influence of σ (standard deviation) can be grasped in advance, and the value of (1 / n) can be grasped in advance, the threshold may be adjusted and equation (10) may be used.

[0127]

number

[0128] <Modification of formula (4)> Instead of the equation (4), the equation (11) may be used. That is, even if σ (standard deviation) in formula (4) is omitted, if a practically effective feature value (z) can be obtained, formula (11) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (11). For example, when the influence of σ (standard deviation) is small, or when the influence of σ (standard deviation) can be known in advance and the threshold can be adjusted, equation (11) may be used.

[0129]

number

[0130] Instead of the equation (4), the equation (12) may be used. That is, even if the (1 / n) part in formula (4) is omitted, if a practically effective feature value (z) can be obtained, formula (12) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (12). For example, when the value of (1 / n) can be known in advance and the threshold can be adjusted, equation (12) may be used.

[0131]

number

[0132] Instead of the equation (4), the equation (13) may be used. That is, even if the σ (standard deviation) and (1 / n) parts in formula (4) are omitted, if a practically effective feature value (z) can be obtained, formula (13) may be used as the evaluation formula. In this case, for example, the threshold value may be adjusted according to formula (13). For example, when the influence of σ (standard deviation) is small, or when the influence of σ (standard deviation) can be grasped in advance, and the value of (1 / n) can be grasped in advance, the threshold may be adjusted and equation (13) may be used.

[0133]

number

[0134] Although the above describes a case where image data is used as the determination target, other data may also be used. For example, time series data from a manufacturing process or the like may be used as the object of judgment. In this case, it is possible to determine whether the time series data contains abnormal values, and to detect abnormalities in the time series data. Such time series data may be data of any physical quantity, and for example, data in which the frequency of values ​​of the physical quantity is symmetrical around the most frequent value when no abnormality occurs is used.

[0135] Furthermore, in the above, the case where the histogram is symmetrical about the mode has been exemplified as being in accordance with a normal distribution, but this is not limitative and the present invention may be applied to, for example, a case where the histogram is in accordance with a Laplace distribution.

[0136] As one configuration example, in the information processing device 11, the determination processing unit 136 determines whether an abnormality exists based on a predetermined feature amount for a histogram that indicates the frequency of a value of a determination target having a plurality of values. The predetermined feature includes a term of the Nth power of the difference between each value and the mode, where N>0 represents the degree, and is obtained by adjusting the sign of that term or the term and summing it up, and is a feature that represents the degree of asymmetry between the part where the value is smaller than the mode and the part where the value is larger than the mode. Therefore, the information processing device 11 can determine with high accuracy whether or not the object to be determined contains an abnormality.

[0137] As one configuration example, in the information processing device 11, when N is an odd number, the determination processing unit 136 calculates z, which is a feature amount, using equation (1) as an arithmetic expression. In formula (1), n ​​represents the total number of values, the variable i represents an integer between 1 and n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a given background area. Therefore, in the information processing device 11, when N is an odd number, it is possible to determine with high accuracy whether or not the determination target includes an abnormality.

[0138] As one configuration example, in the information processing device 11, when N is an even number, the determination processing unit 136 calculates z, which is a feature amount, using equation (3) as an arithmetic expression. In formula (3), n represents the total number of values, the variable i represents an integer between 1 and n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a given background area. Therefore, in the information processing device 11, when N is an even number, it is possible to determine with high accuracy whether or not the object to be determined contains an abnormality.

[0139] As one configuration example, in the information processing device 11, when N is a non-integer (a value that is not an integer), the determination processing unit 136 calculates z, which is a feature amount, using equation (4) as an arithmetic expression. In equation (4), n represents the total number of values, the variable i represents an integer between 1 and n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a given background area. Therefore, in the information processing device 11, when N is a non-integer, it is possible to determine with high accuracy whether or not the determination target includes an abnormality.

[0140] As one configuration example, in the information processing device 11, the object to be determined is an image captured by an optical inspection machine. The value is the brightness. The histogram is a luminance histogram. Therefore, the information processing device 11 can determine with high accuracy whether or not an abnormality exists in an image captured by an optical inspection machine. The abnormality determination process according to this embodiment is suitable for application to the inspection (abnormality detection) of an image captured by an optical inspection machine.

[0141] As one configuration example, in the information processing device 11, the optical inspection machine is a dark-field inspection machine. Therefore, the information processing device 11 can determine with high accuracy whether or not an abnormality exists in an image captured by a dark-field inspection machine. The abnormality determination process according to this embodiment is suitable for application to the inspection (abnormality detection) of an image captured by a dark-field inspection machine.

[0142] As one configuration example, in the information processing device 11, the image is an image of a glass substrate. Therefore, the information processing device 11 can determine with high accuracy whether or not an abnormality occurs on the glass substrate.

[0143] As an example of the configuration, in the information processing device 11, N is a value of 3 or more and 51 or less. Therefore, the information processing device 11 can determine with high accuracy whether or not the object to be determined contains an abnormality. For example, lower-order probability moments are preferable for anomalies such as large, low-brightness defects as in the example of Fig. 6A, whereas higher-order probability moments are preferable for anomalies such as small defects (defects that cannot be classified visually) as in the example of Fig. 7A. If both of these are treated with the same order, it is thought that an order N of about 9 is optimal.

[0144] As one configuration example, in the information processing device 11, the judgment processing unit 136 assumes that N1 and N2 are different values, calculates a first feature which is a feature when N=N1, and a second feature which is a feature when N=N2, and makes a judgment based on the relationship between the first feature and the second feature. Therefore, the information processing device 11 can determine with high accuracy whether or not the object to be determined contains an abnormality, based on the relationship between the two types of feature amounts (first feature amount, second feature amount).

[0145] For example, machine learning may be used. As one configuration example, in the information processing device 311, the determination processing unit 351 makes a determination using a machine learning trained learning model Q1. The learning model Q1 receives input of an image to be judged, or a plurality of values ​​or a histogram of the image to be judged, and outputs an inference result based on the feature amount. Therefore, the information processing device 311 can use the machine learning trained learning model Q1 to determine with high accuracy whether or not an abnormality is included in the determination target.

[0146] A program for implementing the functions of any of the components of any of the above-described devices may be recorded on a computer-readable recording medium and then loaded into a computer system for execution. The term "computer system" as used herein includes hardware such as an operating system or peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and compact discs (CDs) and read-only memories (ROMs), as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory within a computer system that acts as a server or client when a program is transmitted over a network such as the Internet or a communication line such as a telephone line. Such volatile memory may be, for example, RAM. The recording medium may also be, for example, a non-transitory recording medium.

[0147] The above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network such as the Internet or a communication line such as a telephone line. The above program may also be one that realizes part of the above-mentioned functions. Furthermore, the above program may be a so-called differential file that can realize the above-mentioned functions in combination with a program already recorded in a computer system. A differential file may also be called a differential program.

[0148] Furthermore, the functions of any of the components in any of the above-described devices may be implemented by a processor. For example, each process in the embodiments may be implemented by a processor operating based on information such as a program and a computer-readable recording medium storing information such as the program. Here, the functions of each unit of the processor may be implemented by, for example, individual hardware, or may be implemented by integrated hardware. For example, the processor may include hardware, and the hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor may be configured using one or more circuit devices mounted on a circuit board, or one or both of one or more circuit elements. An integrated circuit (IC) or the like may be used as the circuit device, and a resistor or a capacitor may be used as the circuit element.

[0149] Here, the processor may be, for example, a CPU. However, the processor is not limited to a CPU, and various types of processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) may be used. The processor may also be, for example, a hardware circuit such as an ASIC (Application Specific Integrated Circuit). The processor may also be, for example, composed of multiple CPUs, or may be, for example, composed of a hardware circuit such as a multiple ASIC. The processor may also be, for example, composed of a combination of multiple CPUs and a hardware circuit such as a multiple ASIC. The processor may also include, for example, one or more of an amplifier circuit or a filter circuit that processes analog signals.

[0150] The embodiments of this disclosure have been described in detail above with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs within the scope that do not deviate from the gist of this disclosure.

[0151] [Note] (Configuration example 1) to (Configuration example 12) are shown.

[0152] (Configuration example 1) a determination processing unit that performs an abnormality determination based on a feature amount that indicates the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, the feature amount being obtained by summing up a term of the Nth power of the difference between each of the values ​​and a mode, where N>0 indicates an order, and adjusting the sign of the term or the term; Information processing device.

[0153] (Configuration example 2) The determination processing unit When N is an odd number, the feature value z is calculated using equation (1) as an arithmetic formula, In formula (1), n ​​represents the total number of the values, the variable i represents an integer between 1 and n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a predetermined background portion. The information processing device described in (Configuration Example 1).

[0154] (Configuration example 3) The determination processing unit When N is an even number, the feature quantity z is calculated using equation (3) as an arithmetic formula, In formula (3), n represents the total number of the values, the variable i represents an integer between 1 and n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a predetermined background portion. The information processing device described in (Configuration Example 1).

[0155] (Configuration Example 4) The determination processing unit When N is a non-integer, the feature quantity z is calculated using equation (4) as an arithmetic formula, In formula (4), n represents the total number of the values, the variable i represents an integer between 1 and n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a predetermined background portion. The information processing device described in (Configuration Example 1).

[0156] (Configuration Example 5) the object to be determined is an image captured by an optical inspection machine, the value is luminance, the histogram is a luminance histogram; The information processing device according to any one of (Configuration Example 1) to (Configuration Example 4).

[0157] (Configuration Example 6) The optical inspection machine is a dark-field inspection machine. The information processing device according to (Configuration Example 5).

[0158] (Configuration Example 7) The image is an image of a glass substrate. The information processing device according to (Configuration Example 5) or (Configuration Example 6).

[0159] (Configuration Example 8) N is a value between 3 and 51. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 7).

[0160] (Configuration Example 9) the determination processing unit calculates a first feature amount that is the feature amount when N=N1 and a second feature amount that is the feature amount when N=N2, assuming that N1 and N2 are different values, and performs the determination based on the relationship between the first feature amount and the second feature amount. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 8).

[0161] (Configuration Example 10) the determination processing unit makes the determination using a machine learning trained model, The learning model inputs the image when the object to be determined is an image, or the plurality of values ​​of the object to be determined, or the histogram, and outputs an inference result based on the feature amount. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 8).

[0162] It is also possible to provide a method of processing performed by the information processing device described above. (Configuration Example 11) The information processing device a histogram representing the frequency of a plurality of values ​​to be determined, the histogram including a term of the Nth power of the difference between each value and a mode, where N>0 represents the degree of order, and the term or the term's sign is adjusted and the sum is obtained, and the determination of abnormality is performed based on a feature representing the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode; Information processing methods.

[0163] It is also possible to provide a program (computer program) that is executed by a computer that constitutes the above-described information processing device. (Configuration Example 12) On the computer, A function for acquiring a judgment target having multiple values; a function of calculating a feature amount representing the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, the feature amount being obtained by summing up a term of the Nth power of the difference between each value and the mode, where N>0 represents the degree of order, and adjusting the sign of the term or the term; a function of making a determination regarding an abnormality based on the feature amount; A program to achieve this. [Explanation of symbols]

[0164] 11, 311...information processing device, 131...input unit, 132...output unit, 133...communication unit, 134, 331...storage unit, 135, 332...control unit, 136...determination processing unit, 151...operation unit, 152...display unit, 211...learning device, 212...determination device, 1001...information processing unit, 1011...processor, 1012...operation device, 1013...display device, 1014...storage device, 1015...memory, 1016...input / output interface, 1017...network interface, 1021...bus, 2011...table, 2101, 2102, 2103, 22 01...Characteristics, A1...Image data (3D data), A2...1D data, B1...Brightness histogram, C1, C2, C3, C21...Defect image, C11...Pseudo defect image, D1, D2, D3, D11, D21, D22, D23, D31...Brightness histogram, E1, E2, E21, F1, F2, F3, F31...Defect area, G2, G3, G11...Mode, H1, H2, H3...Distance, I1...Normal area, I2...Abnormal area, J1...Background light, J2...Defect light, K1, K2...Specified width, Q1, Q1a, Q1b...Learning model, Q2...Teaching data

Claims

1. a determination processing unit that performs an abnormality determination based on a feature amount that indicates the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, the feature amount being obtained by summing up a term of the Nth power of the difference between each of the values ​​and a mode, where N>0 indicates an order, and adjusting the sign of the term or the term; Information processing device.

2. The determination processing unit When N is an odd number, the feature quantity z is calculated using equation (1) as an arithmetic formula, In formula (1), n ​​represents the total number of the values, the variable i represents an integer of 1 to n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a predetermined background portion. The information processing device according to claim 1 . [Equation 1]

3. The determination processing unit When N is an even number, the feature quantity z is calculated using equation (2) as an arithmetic formula, In formula (2), n represents the total number of the values, the variable i represents an integer of 1 to n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a predetermined background portion. The information processing device according to claim 1 . [Equation 2]

4. The determination processing unit When N is a non-integer, the feature quantity z is calculated using equation (3) as an arithmetic formula, In formula (3), n represents the total number of the values, the variable i represents an integer of 1 to n, xi represents the i-th value, mode represents the mode, and σ represents the standard deviation of a predetermined background portion. The information processing device according to claim 1 . [Equation 3]

5. the object to be determined is an image captured by an optical inspection machine, the value is luminance, the histogram is a luminance histogram; The information processing device according to any one of claims 1 to 4.

6. The optical inspection machine is a dark-field inspection machine. The information processing device according to claim 5 .

7. The image is an image of a glass substrate. The information processing device according to claim 5 .

8. N is a value between 3 and 51, The information processing device according to any one of claims 1 to 4.

9. the determination processing unit calculates a first feature amount that is the feature amount when N=N1 and a second feature amount that is the feature amount when N=N2, assuming that N1 and N2 are different values, and performs the determination based on the relationship between the first feature amount and the second feature amount. The information processing device according to any one of claims 1 to 4.

10. the determination processing unit makes the determination using a machine learning trained model, The learning model inputs the image when the object to be determined is an image, or the plurality of values ​​of the object to be determined, or the histogram, and outputs an inference result based on the feature amount. The information processing device according to any one of claims 1 to 4.

11. The information processing device For a histogram representing the frequency of a plurality of values ​​to be determined, the histogram includes a term of the Nth power of the difference between each value and a mode, where N>0 represents the degree of order, and the term or the term is adjusted and summed up, and the determination of abnormality is performed based on a feature value representing the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode. Information processing methods.

12. On the computer, A function for acquiring a judgment target having multiple values; a function of calculating a feature amount representing the degree of asymmetry between a portion where the value is smaller than the mode and a portion where the value is larger than the mode, the feature amount being obtained by summing up a term of the Nth power of the difference between each value and the mode, where N>0 represents the degree of order, and adjusting the sign of the term or the term; a function of making a determination regarding an abnormality based on the feature amount; A program to achieve this.

Images