Inference system, inference method, data selection system, and data selection method

Abstract

The present invention facilitates inference of the state of a material applied to a substrate by using a roller. An inference system (1) comprises an acquisition unit (11), a classification unit (13), and an inference unit (14). The acquisition unit (11) acquires a captured image obtained by imaging the surface of a roller (3) in a process in which a material (5) is applied to a substrate (4) by using the roller (3). The classification unit (13) outputs, by using a classification model (131) trained through machine learning, aspect information indicating the aspect of the material (5) formed on the surface of the roller (3) indicated by the captured image. The inference unit (14) outputs, on the basis of the aspect information outputted by the classification unit (13), state information pertaining to the application state of the material (5) to the substrate (4).

Description

Estimation system, estimation method, data selection system, and data selection method

[0001] The present disclosure relates to an estimation system, an estimation method, a data selection system, and a data selection method for estimating the state of a material applied to a substrate by a roller.

[0002] Patent Document 1 discloses a coating device that measures the state of a coating film formed by a coating unit that applies a coating liquid.

[0003] Patent Document 2 discloses a coating method for applying a coating liquid to a predetermined thickness on a continuously running web, in which the shape of a coating liquid pool formed between a coating head and the web surface is detected and the coating conditions are controlled based on the detection results.

[0004] JP 2022-149458 A JP 2003-117466 A

[0005] The present disclosure provides an estimation system, an estimation method, a data selection system, and a data selection method that facilitate estimating the state of a material applied to a substrate by a roller.

[0006] An estimation system according to one aspect of the present disclosure includes an acquisition unit, a classification unit, and an estimation unit. The acquisition unit acquires a captured image of a roller surface during a process of applying a material to a substrate using the roller. The classification unit uses a classification model trained by machine learning to output appearance information indicating the appearance of the material formed on the roller surface shown in the captured image. The estimation unit outputs state information regarding the application state of the material on the substrate based on the appearance information output by the classification unit.

[0007] In an estimation method according to one aspect of the present disclosure, during a process of applying a material to a substrate using a roller, an image of the surface of the roller is acquired, and a classification model trained by machine learning is used to output appearance information indicating the appearance of the material formed on the surface of the roller shown in the image, and status information regarding the application status of the material on the substrate is output based on the appearance information.

[0008] A data selection system according to one aspect of the present disclosure includes a data acquisition unit and a data selection unit. The data acquisition unit acquires first training data in which captured images of a roller surface are labeled with the amount of roller pressure on the substrate as a correct answer during a process of applying a material to a substrate using a roller. The data selection unit selects second training data from the first training data to be used for training a classification model based on information output by multiple weak learners trained by machine learning using a portion of the first training data. The multiple weak learners are trained by machine learning to use the captured images as input and output the amount of roller pressure. The classification model is trained by machine learning to output appearance information indicating the appearance of the material formed on the surface of the roller as shown in the captured images.

[0009] In a data selection method according to one aspect of the present disclosure, in a process of applying a material to a substrate using a roller, first training data is acquired in which captured images of the roller surface are labeled with the amount of roller pressure against the substrate as a correct answer, and second training data to be used for training a classification model is selected from the first training data based on information output by multiple weak learners trained by machine learning using a portion of the first training data. The multiple weak learners are trained by machine learning to use the captured images as input and output the amount of roller pressure. The classification model is trained by machine learning to output appearance information indicating the appearance of the material formed on the surface of the roller as shown in the captured images.

[0010] The estimation system, estimation method, data selection system, and data selection method according to one aspect of the present disclosure have the advantage of making it easy to estimate the state of a material applied to a substrate by a roller.

[0011] FIG. 1 is a diagram showing a schematic configuration of an estimation system according to an embodiment. FIG. 2 is an explanatory diagram of the correlation between modal information and the film thickness of a coating film. FIG. 3 is an explanatory diagram of the state of a coating film. FIG. 4 is an explanatory diagram of a first example of image processing. FIG. 5 is an explanatory diagram of an issue in image processing. FIG. 6 is an explanatory diagram of a second example of image processing. FIG. 7 is a flowchart showing an example of operation of an estimation system according to an embodiment. FIG. 8 is an explanatory diagram of an issue regarding the reliability of training data. FIG. 9 is a diagram showing a schematic configuration of a data selection system according to an embodiment. FIG. 10 is a diagram showing an example of output results of each of a plurality of weak learners. FIG. 11 is a flowchart showing an example of operation of a data selection system according to an embodiment. FIG. 12 is a diagram showing a schematic configuration of an estimation system according to a modified example of an embodiment.

[0012] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0013] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, components, arrangement positions and connection forms of the components, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not recited in the independent claims are described as optional components.

[0014] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in all figures, substantially the same components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0015] Furthermore, in this specification, terms indicating the relationship between elements, such as orthogonal, parallel, and the same, terms indicating the shape of elements, such as rectangular and circular, as well as numerical values ​​and numerical ranges, are not expressions that only express a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about a few percent (e.g., about 10%).

[0016] (Embodiment) [1. Configuration] An estimation system 1 according to an embodiment will be described below with reference to Fig. 1. Fig. 1 is a diagram showing a schematic configuration of the estimation system 1 according to an embodiment. Fig. 1(a) is a diagram showing the overall configuration including the estimation system 1 and a roll coater, and Fig. 1(b) is a diagram showing another aspect of the roll coater.

[0017] The estimation system 1 is used when applying a material 5 to a substrate 4 using a roll coater, and is a system for estimating the state of the material 5 applied to the substrate 4 by a roller 3. The substrate 4 is a material that serves as the basis for a product or compound, such as a film, sheet, or substrate. The material 5 is a liquid material that forms a thin film (coating film) on the surface of the substrate 4, such as a metallic paint such as silver paint, or a resin paint.

[0018] In this embodiment, the roll coater includes three pairs of rollers 3 as shown in FIG. 1A . Specifically, the roll coater includes a pair of first rollers 31, a pair of second rollers 32, and a pair of third rollers 33. Each of the pair of first rollers 31 transfers the material 5 to each of the pair of second rollers 32. Each of the pair of second rollers 32 transfers the material 5 transferred from each of the pair of first rollers 31 to each of the pair of third rollers 33. Each of the pair of third rollers 33 applies the material 5 to the surface of the substrate 4 being transported in one direction (the up-and-down direction in FIG. 1A ). Here, the left third roller 33 of the pair of third rollers 33 applies the material 5 to the left side of the substrate 4, and the right third roller 33 applies the material 5 to the right side of the substrate 4.

[0019] In the roll coater, an operator can adjust the thickness of the coating film formed on the surface of the substrate 4 by controlling the amount of pressure applied to the rollers 3 (here, a pair of first rollers 31). For example, the smaller the amount of pressure applied to the rollers 3, the thicker the coating film will be, and the larger the amount of pressure applied to the rollers 3, the thinner the coating film will be. In the embodiment, the amount of pressure applied is expressed in micrometer values.

[0020] The roll coater may include a pair of rollers 3, as shown in Fig. 1(b), for example. In the example shown in Fig. 1(b), the left roller 3 of the pair of rollers 3 applies material 5 to the left surface of the substrate 4, and the right roller 3 applies material 5 to the right surface of the substrate 4. An operator can adjust the thickness of the coating film formed on the surface of the substrate 4 by controlling the amount of pressure applied to the pair of rollers 3.

[0021] Returning to (a) in FIG. 1, the estimation system 1 has a memory and a processor, and various functions are realized by the processor executing programs stored in the memory. The estimation system 1 is, for example, a laptop or desktop personal computer. Note that the estimation system 1 may be any portable information processing terminal such as a smartphone or tablet terminal as long as it has a memory and a processor. The estimation system 1 includes an acquisition unit 11, an image processing unit 12, a classification unit 13, an estimation unit 14, and a storage unit 15.

[0022] The acquisition unit 11 acquires an image P1 (see FIG. 2 , etc., described later) of the surface of the roller 3 during the process of applying the material 5 to the substrate 4 using the roller 3. In the embodiment, the acquisition unit 11 acquires the image P1 captured by the camera 2, which captures an image of the surface of one of the pair of first rollers 31, by communicating with the camera 2. Note that the communication between the acquisition unit 11 and the camera 2 may be wired communication or wireless communication.

[0023] The image processing unit 12 performs image processing to generate an extracted image P2 (see FIG. 4 and the like, which will be described later) that extracts the surface of the roller 3 from the captured image P1 acquired by the acquisition unit 11. The image processing performed by the image processing unit 12 will be described in detail in [2. Image processing], which will be described later.

[0024] The classification unit 13 uses a classification model 131 trained by machine learning to output appearance information indicating the appearance of the material 5 formed on the surface of the roller 3 shown in the captured image P1. In the embodiment, the appearance information is information indicating the pattern drawn by the material 5 on the surface of the roller 3. More specifically, in the embodiment, the appearance information is information indicating the density of a plurality of streaks 51 (see FIG. 2 , etc., described later) drawn by the material 5 on the surface of the roller 3, and is represented by an appearance class, described later.

[0025] Furthermore, in the embodiment, the classification unit 13 uses, as the captured image P1, the extracted image P2 generated by the image processing unit 12. In other words, the classification unit 13 does not use the captured image P1 itself, but rather uses the extracted image P2 generated from the captured image P1, thereby indirectly using the captured image P1.

[0026] The classification model 131 is trained in advance by machine learning so as to receive the captured image P1 as an input and output appearance information. In the embodiment, the classification model 131 is trained in advance by machine learning so as to receive the extracted image P2 generated by the image processing unit 12 as the captured image P1 as an input and output appearance information.

[0027] In the embodiment, the classification model 131 is constructed by training a neural network through supervised learning. Specifically, the classification model 131 performs supervised learning using a large amount of training data prepared by attaching the amount of pressing of the roller 3 against the base material 4 (here, micrometer value) to the captured image P1 (here, extracted image P2) as a correct answer label.

[0028] Here, since there is a correlation between the amount of pressure applied by the roller 3 and the density of the multiple streaks 51 formed by the material 5 on the surface of the roller 3, it is possible to construct the classification model 131 by performing supervised learning as described above. In the embodiment, in order to improve the prediction accuracy of the classification model 131, selection of training data to be used in supervised learning is performed. The selection of training data will be described in detail in [4. Data Selection System] below.

[0029] The estimation unit 14 outputs state information regarding the coating state of the material 5 on the substrate 4 based on the appearance information output by the classification unit 13. In an embodiment, the state information may include information indicating the film thickness of a coating film formed by coating the material 5 on the surface of the substrate 4. In an embodiment, the estimation unit 14 outputs the state information by referring to correlation data indicating the correlation between the appearance information and the coating state of the material 5.

[0030] FIG. 2 is an explanatory diagram of the correlation between appearance information and the film thickness of a coating film. FIG. 2(a) shows an example of a captured image P1 in which the density of the multiple streaks 51 formed on the surface of the roller 3 is relatively high. FIG. 2(b) shows an example of a captured image P1 in which the density of the multiple streaks 51 formed on the surface of the roller 3 is relatively low. FIG. 2(c) shows an example of correlation data previously obtained through experiments. In FIG. 2(c), the vertical axis represents the film thickness of the coating film formed on the surface of the substrate 4, and the horizontal axis represents the appearance class. The appearance class represents the density of the multiple streaks 51 and is expressed as "Class 1," "Class 2," ..., "Class N (N is a natural number)." In the appearance class, "Class 1" represents the highest density of the multiple streaks 51, and "Class N" represents the lowest density of the multiple streaks 51.

[0031] 2(c), there is a correlation between the appearance information (here, the density of the plurality of streaks 51) and the thickness of the coating film such that the greater the density of the plurality of streaks 51, the thinner the thickness of the coating film, and vice versa. Therefore, by referring to the correlation data shown in FIG. 2(c), the estimation unit 14 can estimate the thickness of the coating film based on the appearance information output by the classification unit 13.

[0032] In the embodiment, the state information may include information indicating whether the coating film formed by applying the material 5 to the surface of the substrate 4 is normal or abnormal. In other words, the estimation unit 14 outputs, as the state information, information indicating whether the application state of the material 5 is normal or abnormal.

[0033] 3A and 3B are explanatory diagrams of the state of the coating film. Fig. 3A shows an example of a captured image P1 in which there are no defects in the multiple streaks 51 formed on the surface of the roller 3, i.e., the coating state of the material 5 is normal. Fig. 3B shows an example of a captured image P1 in which there are defects 52 such as uneven coating or foreign matter in the multiple streaks 51 formed on the surface of the roller 3, i.e., the coating state of the material 5 is abnormal.

[0034] In the embodiment, when a captured image P1 such as that shown in (b) of Figure 3 is input, the classification model 131 outputs modal information representing a class that is not classified into any of the above-mentioned "Class 1" to "Class N." Therefore, the estimation unit 14 can estimate that the application state of the material 5 is normal if the modal information represents any of the classes "Class 1" to "Class N," and can estimate that the application state of the material 5 is abnormal if the modal information represents a class that is not classified into any of the classes "Class 1" to "Class N."

[0035] The storage unit 15 is a recording medium that stores various information such as a program executed by the processor, the classification model 131 used in the classification unit 13, and correlation data used in the estimation unit 14. The recording medium is, for example, a hard disk drive, a RAM (Random Access Memory), a ROM (Read Only Memory), or a semiconductor memory. Note that such a recording medium may be volatile or non-volatile.

[0036] [2. Image Processing] Image processing by the image processing unit 12 of the estimation system 1 according to the embodiment will be described below. The image processing is a process for generating an extracted image P2 (here, a second extracted image P22, described later) by extracting the surface of the roller 3 from the captured image P1. Through the image processing, the extracted image P2 becomes an image in which elements other than the surface of the roller 3 have been removed from the captured image P1. Therefore, using the extracted image P2 in the classification unit 13 makes it easier to improve the accuracy of prediction of appearance information by the classification unit 13 compared to when the captured image P1 itself is used in the classification unit 13.

[0037] First and second examples of image processing will be described below. Note that either the first or second example below may be employed as the image processing performed by the image processing unit 12.

[0038] 4 is an explanatory diagram of a first example of image processing. First, in the image processing, the captured image P1 is input to a first detection model 121, thereby detecting the roller 3 in the captured image P1. The first detection model 121 is trained by machine learning to detect a specific object (here, the roller 3) included in the input image.

[0039] Next, in image processing, a first extracted image P21 is generated by extracting a portion of the captured image P1 in which the roller 3 is detected. Specifically, rotation correction is performed on the captured image P1 to match the angle of the roller 3 detected with a reference angle, and then trimming is performed on the captured image P1 after rotation correction to cut out only the portion of the roller 3 detected. The trimmed captured image P1 is then resized to a first reference size, thereby generating the first extracted image P21.

[0040] Next, in image processing, the roller 3 in the first extracted image P21 is detected by inputting the first extracted image P21 into the second detection model 122. Like the first detection model 121, the second detection model 122 is trained by machine learning to detect a specific object (here, the roller 3) included in the input image. Note that the second detection model 122 may be the same model as the first detection model 121.

[0041] Next, in image processing, a second extracted image P22 is generated by extracting the portion of the first extracted image P21 where the roller 3 is detected. Specifically, the first extracted image P21 is trimmed to cut out only the portion of the detected roller 3, and the trimmed first extracted image P21 is resized to a second reference size, thereby generating the second extracted image P22. The generated second extracted image P22 is input to the classification model 131.

[0042] [2-2. Image Processing Issues] Figure 5 is an explanatory diagram of image processing issues. In a first example of image processing, in resizing to correct the cropped first extracted image P21 to the second reference size, the cropped first extracted image P21 is reduced overall. In the example shown in Figure 5, the cropped first extracted image P21 is corrected to the second reference size of width W1 and height H1.

[0043] Here, the first extracted image P21 after trimming includes a plurality of streaks 51, but the plurality of streaks 51 do not have a uniform pattern in the height direction along the rotation direction of the roller 3. Therefore, there is a problem in that simply resizing the first extracted image P21 after trimming generates a second extracted image P22 that includes a plurality of streaks 51 that do not have a uniform pattern in the height direction.

[0044] When such a second extracted image P22 is input to the classification model 131, there is a possibility that the classification model 131 will not be able to output modal information with sufficient prediction accuracy. Furthermore, when machine learning of the classification model 131 is performed using such a second extracted image P22 as training data, there is a possibility that the classification model 131 will have relatively low prediction accuracy.

[0045] [2-3. Second Example] A second example of image processing that can solve the above problem will be described below. FIG. 6 is an explanatory diagram of the second example of image processing. In the second example of image processing, as shown in FIG. 6, first, the trimmed first extracted image P21 is resized to correct its width to width W1. Next, in the second example of image processing, the resized first extracted image P21 is trimmed to cut out a predetermined region having width W1 and height H2, thereby generating a second extracted image P22. Here, the predetermined region is, for example, a region in the middle in the height direction (the direction of rotation of the roller 3), where there is little change in the pattern of the multiple streaks 51.

[0046] Thus, in the second example of image processing, the image processing unit 12 generates an extracted image (second extracted image P22) by extracting a predetermined area in the rotation direction of the roller 3 from the surface of the roller 3 shown in the captured image P1. Therefore, in the second example of image processing, it is easy to generate the second extracted image P22 that includes multiple streaks 51 whose pattern is generally uniform in the height direction, making it easier to solve the above-mentioned problem.

[0047] 3. Operation An example of the operation of the estimation system 1 according to the embodiment will be described below. Fig. 7 is a flowchart showing an example of the operation of the estimation system 1 according to the embodiment.

[0048] First, the acquisition unit 11 of the estimation system 1 acquires the captured image P1 (S11). Here, the acquisition unit 11 communicates with the camera 2 to acquire the captured image P1 captured by the camera 2.

[0049] Next, the image processing unit 12 of the estimation system 1 generates an extracted image P2 by performing image processing on the captured image P1 acquired by the acquisition unit 11 (S12). Here, the image processing unit 12 employs the second example of the image processing described above to generate a second extracted image P22 as the extracted image P2.

[0050] Next, the classification unit 13 of the estimation system 1 outputs appearance information using the classification model 131 (S13). Here, the classification unit 13 inputs the extracted image P2 (second extracted image P22) as the captured image P1 to the classification model 131. As a result, the classification unit 13 (classification model 131) outputs appearance information indicating the appearance of the material 5 formed on the surface of the roller 3 indicated by the extracted image P2.

[0051] The estimation unit 14 of the estimation system 1 then outputs state information based on the aspect information output by the classification unit 13 (S14). Here, the estimation unit 14 outputs information indicating the film thickness of the coating film as state information by referring to the correlation data. The estimation unit 14 also outputs information indicating whether the coating state of the material 5 is normal or abnormal as state information based on the class indicated by the aspect information.

[0052] [4. Data Selection System] In the embodiment, the classification model 131 is trained by machine learning using teacher data (second teacher data) selected by a data selection system 6 (data selection method) described below.

[0053] First, we will explain how we came to select training data using the data selection system 6. Fig. 8 is an explanatory diagram of issues regarding the reliability of training data. Fig. 8(a) is an image diagram showing the correlation between the feature amount assumed by the inventors (here, the density of the multiple streaks 51) and the correct label (here, the μ-meter value), and Fig. 8(b) is an image diagram showing the correlation between the actual feature amount and the correct label.

[0054] 8A, the inventors assumed that there is a one-to-one correspondence between the appearance of the material 5 formed on the surface of the roller 3 (here, the density of the multiple streaks 51 formed by the material 5 on the surface of the roller 3) and the amount of pressing of the roller 3 against the substrate 4, which is a controllable parameter (here, micrometer value).The inventors then considered that by performing machine learning on the captured image P1 (here, the extracted image P2) using a large amount of training data in which the micrometer value is used as the correct answer label, it would be possible to generate a classification model 131 that outputs a micrometer value for the input captured image P1, in other words, that outputs the appearance of the material 5 formed on the surface of the roller 3.

[0055] However, it has been found that in reality, the density of the plurality of streaks 51 is affected not only by the amount of pressing (micrometer value) of the roller 3 against the substrate 4, but also by factors that cannot be controlled or measured, such as the eccentric motion of the roller 3 and the eccentric motion of the shaft of the roller 3. Therefore, in reality, as shown in (b) of Figure 8, there is a many-to-one correspondence between the density of the plurality of streaks 51 and the micrometer value, which poses a problem of low reliability of the training data.

[0056] In view of the above problems, the inventors considered applying a bagging method for ensemble learning using multiple weak learners with relatively low predictive accuracy to select relatively reliable teacher data (second teacher data) from prepared teacher data (first teacher data), and to perform machine learning of the classification model 131 using the selected teacher data.

[0057] [4-1. Overview] A data selection system 6 according to an embodiment will be described below with reference to Fig. 9. Fig. 9 is a diagram showing a schematic configuration of the data selection system 6 according to an embodiment. Fig. 9(a) is a block diagram showing the configuration of the data selection system 6, and Fig. 9(b) is a diagram showing an overview of the operation of the data selection system 6.

[0058] The data selection system 6 has a memory and a processor, and various functions are realized by the processor executing programs stored in the memory. The data selection system 6 is, for example, a laptop or desktop personal computer. Note that the data selection system 6 may be a portable information processing terminal such as a smartphone or tablet terminal as long as it has a memory and a processor. The data selection system 6 may also be realized by the same device as the estimation system 1.

[0059] As shown in FIG. 9A, the data selection system 6 includes a data acquisition unit 61, a data selection unit 62, and a storage unit 63.

[0060] The data acquisition unit 61 acquires first training data in a process of applying the material 5 to the substrate 4 using the roller 3. The first training data is data in which the pressing amount (here, a micrometer value) of the roller 3 against the substrate 4 is attached as a correct answer label to an image P1 capturing an image of the surface of the roller 3.

[0061] In the embodiment, the data acquisition unit 61 acquires a captured image P1 captured by the camera 2 by communicating with the camera 2. The data acquisition unit 61 also acquires the first teacher data by performing image processing on the acquired captured image P1 to obtain an extracted image P2 (here, a second extracted image P22), which is obtained as the captured image P1, and assigning the micrometer value at the time the captured image P1 was acquired to the captured image P1 as a correct label.

[0062] The data selection unit 62 selects second teacher data from the first teacher data based on information output by a plurality of weak learners 621 that have been trained by machine learning using a portion of the first teacher data among the first teacher data. Here, the plurality of weak learners 621 are trained by machine learning to receive the captured image P1 (here, the extracted image P2) as an input and output the amount of pressing of the roller 3 (here, a micrometer value).

[0063] Specifically, as shown in FIG. 9B , the data selection unit 62 randomly selects a portion of the first teacher data from the first teacher data. The data selection unit 62 also randomly divides the selected portion of the first teacher data into a training data set including “train” data and “validation” data, and “test” data. Next, the data selection unit 62 generates a weak learner 621 by performing supervised learning using the training data set. The data selection unit 62 then inputs the “test” data to the generated weak learner 621, thereby obtaining predicted micrometer values ​​corresponding to each of the multiple captured images P1 (here, extracted images P2) included in the “test” data. Thereafter, the data selection unit 62 repeats the above process multiple times (here, X times (X is a natural number)). In other words, the data selection unit 62 generates multiple (here, X) weak learners 621 and obtains information output by each of the multiple generated weak learners 621 (here, predicted values ​​of the micrometer values ​​corresponding to the input captured image P1).

[0064] Next, the data selection unit 62 selects second teacher data from the first teacher data based on the information output by each of the multiple weak learners 621. FIG. 10 is a diagram illustrating an example of the output results of each of the multiple weak learners 621. In FIG. 10, "Image 1," "Image 2," ..., "Image M (M is a natural number)" represent all captured images P1 (here, extracted images P2) included in the first teacher data. Also, in FIG. 10, "trial-1," "trial-2," ..., "trial-x" represent micrometer values ​​output by multiple (here, X) weak learners 621 for each input captured image P1. Note that the captured image P1 input to each weak learner 621 is the captured image P1 included in the "test" data, so each weak learner 621 does not output micrometer values ​​for all captured images P1 included in the first teacher data. 10, "Most frequent prediction" represents, for each captured image P1, the most frequent micrometer value output by each of the multiple weak learners 621. Also, in Fig. 10, "Label" represents the micrometer value as the correct label assigned to each captured image P1.

[0065] 10, the "score" represents the degree of reliability of the first teacher data calculated for each captured image P1 (here, for the extracted image P2) by the data selection unit 62. The "score" is, for example, a value obtained by dividing the number of weak learners 621 that output a micrometer value equal to the mode for each captured image P1 by the number of weak learners 621 to which the captured image P1 was input, expressed as a percentage.

[0066] The data selection unit 62 changes the label of each of the captured images P1 (here, extracted images P2) included in the first teacher data for which a score equal to or greater than a threshold value (e.g., 90) has been calculated to be the most frequent prediction, and selects the captured images P1 as the second teacher data. In this way, the data selection unit 62 selects, from the first teacher data, data that has a relatively strong correlation between the density of the multiple streaks 51 and the micrometer value, in other words, data that is relatively reliable, as the second teacher data.

[0067] The storage unit 63 is a recording medium that stores various information such as the program executed by the processor, the first teacher data acquired by the data acquisition unit 61, and the plurality of weak learners 621 used in the data selection unit 62. The recording medium is, for example, a hard disk drive, RAM, ROM, or semiconductor memory. Note that such a recording medium may be volatile or non-volatile.

[0068] 4-2. Operation An example of the operation of the data selection system 6 according to the embodiment will be described below. Fig. 11 is a flowchart showing an example of the operation of the data selection system 6 according to the embodiment.

[0069] First, the data acquisition unit 61 of the data selection system 6 acquires first teacher data (S21). Here, the data acquisition unit 61 acquires a captured image P1 captured by the camera 2 by communicating with the camera 2. The data acquisition unit 61 also acquires an extracted image P2 (here, a second extracted image P22) obtained by performing image processing on the acquired captured image P1 as the captured image P1, and acquires the first teacher data by attaching the micrometer value at the time the captured image P1 was acquired to the captured image P1 as a correct answer label.

[0070] Next, the data selection unit 62 of the data selection system 6 generates a plurality of weak learners 621 (S22). Here, the data selection unit 62 randomly selects a portion of the first teacher data from the selected portion of the first teacher data. The data selection unit 62 also randomly selects a training dataset from the selected portion of the first teacher data, and performs supervised learning using the selected training dataset to generate the weak learners 621. Thereafter, the data selection unit 62 repeats the above process a plurality of times to generate a plurality of weak learners 621.

[0071] Next, the data selection unit 62 selects second teacher data from the first teacher data using the generated weak learners 621 (S23). Here, the data selection unit 62 calculates scores for all captured images P1 (here, extracted images P2) included in the first teacher data based on information (here, micrometer values) output from each of the weak learners 621. Then, the data selection unit 62 changes the label of each of the captured images P1 for which a score equal to or greater than a threshold is calculated to a most frequent prediction, and selects the captured images P1 as the second teacher data.

[0072] [5. Advantages] The advantages of the estimation system 1 (estimation method) according to the embodiment and the data selection system 6 (data selection method) according to the embodiment will be described below. As described above, the estimation system 1 according to the embodiment estimates the application state of the material 5 on the substrate 4 based on the appearance of the material 5 formed on the surface of the roller 3 shown in the captured image P1. Therefore, the estimation system 1 according to the embodiment has the advantage that the material 5 on the surface of the roller 3 can be viewed as a surface, compared to when the material 5 on the surface of the roller 3 is viewed as a point using, for example, a film thickness meter, and therefore the state of the material 5 applied to the substrate 4 by the roller 3 can be easily estimated.

[0073] For example, when measuring the thickness of a coating film as the state of the material 5 using a film thickness meter, the state of the material 5 on the surface of the roller 3 is measured locally, making it difficult to find coating unevenness or defects. In contrast, the estimation system 1 according to the embodiment can estimate the state of the material 5 on the surface of the roller 3 over a wide area based on the appearance of the material 5 formed on the surface of the roller 3, that is, it is easy to find coating unevenness or defects.

[0074] As described above, the data selection system 6 (data selection method) according to the embodiment selects relatively reliable second teacher data from the first teacher data using a plurality of weak learners 621 with relatively low prediction accuracy. Therefore, the data selection system 6 according to the embodiment has the advantage that it is easy to improve the prediction accuracy of the classification model 131 because machine learning of the classification model 131 can be performed using the relatively reliable second teacher data.

[0075] (Other Embodiments) As described above, the embodiments have been described as examples of the technology according to the present disclosure. However, the technology according to the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made as appropriate. For example, the following modifications are also included in one embodiment of the present disclosure.

[0076] An estimation system 1A according to a modified example of the embodiment will be described below with reference to FIG. 12. FIG. 12 is a diagram showing a schematic configuration of the estimation system 1A according to the modified example. The estimation system 1A according to this modified example differs from the estimation system 1 according to the embodiment in that the estimation system 1A estimates the state of the material 5 based on the detection results of one or more sensors 7. Below, a description of the points common to the estimation system 1 according to the embodiment and the estimation system 1 according to the modified example will be omitted.

[0077] The one or more sensors 7 may include, for example, a temperature sensor. The temperature sensor detects the temperature of the material 5. The one or more sensors 7 may also include, for example, a pressure sensor. The pressure sensor detects the pressure of the material 5. The one or more sensors 7 may also include, for example, a viscosity sensor. The viscosity sensor detects the viscosity of the material 5. In this way, the one or more sensors 7 detect parameters related to the environment in which the material 5 is placed.

[0078] The estimation unit 14 outputs state information by further referring to the detection results of one or more sensors 7, i.e., environmental information indicating the environment in which the material 5 is placed. For example, when estimating the thickness of the coating film by referring to environmental information indicating the temperature of the material 5, the estimation unit 14 refers to correlation data corresponding to the temperature detected by the temperature sensor from multiple correlation data prepared for each temperature of the material 5 that are stored in advance in the storage unit 15. Furthermore, for example, when estimating the thickness of the coating film by referring to environmental information indicating the temperature of the material 5, the estimation unit 14 corrects the correlation data in accordance with the temperature detected by the temperature sensor and refers to the corrected correlation data.

[0079] As described above, in the estimation system 1A according to this modified example, the estimation unit 14 outputs state information by further referring to environmental information indicating the environment in which the material 5 is placed. This has the advantage that it is possible to take into account the influence of, for example, the temperature, pressure, or viscosity of the material 5 on the application state of the material 5 on the substrate 4, making it easier to improve the estimation accuracy of the application state of the material 5 on the substrate 4.

[0080] For example, in the above embodiment, the estimation system 1 includes the image processing unit 12, but the estimation system 1 does not necessarily have to include the image processing unit 12. In other words, the classification unit 13 of the estimation system 1 may be configured to output appearance information using the captured image P1 itself.

[0081] For example, in the above embodiment, the estimation unit 14 has a function of outputting status information indicating whether the coating film is normal or abnormal, but it does not have to have this function.

[0082] For example, in the above embodiment, the classification model 131 is constructed by supervised learning using the second training data selected by the data selection system 6 (data selection method), but this is not limiting. For example, the classification model 131 may be constructed by supervised learning using the first training data.

[0083] For example, the present disclosure can be realized as a program for causing a computer (processor) to execute steps included in the estimation method. Furthermore, the present disclosure can be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the program is recorded. The same applies to steps included in the data selection method.

[0084] For example, when the present disclosure is realized as a program (software), each step is performed by running the program using hardware resources such as a computer's CPU (Central Processing Unit), memory, input / output circuits, etc. In other words, each step is performed by the CPU acquiring data from memory or input / output circuits, etc., performing calculations on the data, and outputting the calculation results to memory or input / output circuits, etc.

[0085] In the above embodiment, the estimation system is implemented by a single device, but the present invention is not limited to this and may be implemented by distributing the system across multiple devices. The same applies to the data selection system.

[0086] Furthermore, in the above-described embodiments, each component included in the estimation system may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. The same applies to each component included in the data selection system.

[0087] Some or all of the functions of the estimation system according to the above embodiments are typically realized as an LSI (Large Scale Integration), which is an integrated circuit. These may be individually integrated into one chip, or some or all of them may be integrated into one chip. Furthermore, the integrated circuit is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may also be used. The same applies to the functions of the data selection system.

[0088] Furthermore, if an integrated circuit technology that can replace LSIs emerges due to advances in semiconductor technology or other derivative technologies, it is natural that each component included in the estimation system may be integrated into an integrated circuit using that technology. The same applies to each component included in the data selection system.

[0089] In addition, this disclosure also includes forms obtained by making various modifications to the embodiments that a person skilled in the art would think of, and forms realized by arbitrarily combining the components and functions in each embodiment within the scope that does not deviate from the intent of this disclosure.

[0090] (Summary) As described above, the estimation system 1, 1A according to the first aspect includes an acquisition unit 11, a classification unit 13, and an estimation unit 14. The acquisition unit 11 acquires a captured image P1 of the surface of the roller 3 during the process of applying the material 5 to the substrate 4 using the roller 3. The classification unit 13 uses a classification model 131 trained by machine learning to output appearance information indicating the appearance of the material 5 formed on the surface of the roller 3 shown in the captured image P1. The estimation unit 14 outputs state information regarding the application state of the material 5 on the substrate 4 based on the appearance information output by the classification unit 13.

[0091] This has the advantage that, compared to viewing the material 5 on the surface of the roller 3 as a point using, for example, a film thickness meter, the material 5 on the surface of the roller 3 can be viewed as a surface, making it easier to estimate the state of the material 5 applied to the substrate 4 by the roller 3.

[0092] In the estimation system 1, 1A according to the second aspect, the estimation unit 14 in the first aspect outputs information indicating the film thickness of the material 5 applied to the substrate 4 as the state information.

[0093] This has the advantage that the material 5 on the surface of the roller 3 can be seen as a plane, making it easier to estimate the film thickness of the material 5 applied to the substrate 4 by the roller 3.

[0094] In addition, in the estimation system 1, 1A according to the third aspect, in the second aspect, the estimation unit 14 outputs state information by referring to correlation data indicating the correlation between the appearance information and the application state of the material 5.

[0095] This has the advantage that an experiment can be conducted in advance to determine the correlation between the appearance information and the application state of the material 5, and the experimental data can be used as correlation data, making it easier to estimate the application state of the material 5.

[0096] In addition, in the estimation system 1, 1A according to the fourth aspect, in any one of the first to third aspects, the estimation unit 14 outputs, as status information, information indicating whether the application status of the material 5 is normal or abnormal.

[0097] This has the advantage that the material 5 on the surface of the roller 3 can be seen as a plane, making it easier to estimate whether the application state of the material 5 is normal or abnormal.

[0098] In addition, in the estimation system 1A according to the fifth aspect, in any one of the first to fourth aspects, the estimation unit 14 outputs state information by further referring to environmental information indicating the environment in which the material 5 is placed.

[0099] This has the advantage that it is possible to take into account the influence of, for example, the temperature, pressure, or viscosity of the material 5 on the application state of the material 5 on the substrate 4, making it easier to improve the accuracy of estimating the application state of the material 5 on the substrate 4.

[0100] Furthermore, the estimation system 1, 1A according to a sixth aspect is any one of the first to fifth aspects, and further includes an image processing unit 12 that executes image processing to generate an extracted image P2 by extracting the surface of the roller 3 from the captured image P1 acquired by the acquisition unit 11. The classification unit 13 uses the extracted image P2 generated by the image processing unit 12 as the captured image P1.

[0101] According to this, the classification unit 13 uses an extracted image P2 in which elements other than the surface of the roller 3 have been removed from the captured image P1, which has the advantage that the accuracy of prediction of appearance information by the classification unit 13 is more likely to be improved compared to when the captured image P1 itself is used in the classification unit 13.

[0102] In addition, in the estimation system 1, 1A relating to the seventh aspect, in the sixth aspect, the image processing unit 12 generates an extracted image P2 by extracting a predetermined area in the rotation direction of the roller 3 from the surface of the roller 3 shown in the captured image P1.

[0103] This has the advantage that it is easy to generate a second extracted image P22 that includes multiple streaks 51 with a generally uniform pattern, making it easier for the classification model 131 to output appearance information with sufficient prediction accuracy.

[0104] In addition, the estimation method according to the eighth aspect involves, in the process of applying material 5 to a substrate 4 using a roller 3, acquiring an image P1 of the surface of the roller 3 (S11), using a classification model 131 trained by machine learning, outputting appearance information indicating the appearance of the material 5 formed on the surface of the roller 3 shown in the image P1 (S13), and outputting state information regarding the application state of the material 5 on the substrate 4 based on the appearance information (S14).

[0105] This has the advantage that, compared to viewing the material 5 on the surface of the roller 3 as a point using, for example, a film thickness meter, the material 5 on the surface of the roller 3 can be viewed as a surface, making it easier to estimate the state of the material 5 applied to the substrate 4 by the roller 3.

[0106] A data selection system 6 according to a ninth aspect includes a data acquisition unit 61 and a data selection unit 62. The data acquisition unit 61 acquires first teacher data. The first teacher data is data in which a captured image P1 of the surface of the roller 3 is captured during a process of applying a material 5 to a substrate 4 using the roller 3 and the amount of pressing of the roller 3 against the substrate 4 is labeled as a correct answer. The data selection unit 62 selects second teacher data from the first teacher data to be used for training the classification model 131 based on information output by multiple weak learners 621 trained by machine learning using a portion of the first teacher data. The multiple weak learners 621 are trained by machine learning to input the captured image P1 and output the amount of pressing of the roller 3. The classification model 131 is trained by machine learning to output appearance information indicating the appearance of the material 5 formed on the surface of the roller 3 shown in the captured image P1.

[0107] This has the advantage that machine learning of the classification model 131 can be performed using second training data that is relatively reliable, making it easier to improve the prediction accuracy of the classification model 131.

[0108] The data selection method according to the tenth aspect acquires first teacher data (S21). The first teacher data is data obtained by attaching the amount of pressing of the roller 3 against the substrate 4 as a correct answer label to an image P1 captured of the surface of the roller 3 during a process of applying a material 5 to the substrate 4 using the roller 3. This data selection method then selects second teacher data to be used for training a classification model 131 from the first teacher data based on information output by multiple weak learners 621 trained by machine learning using a portion of the first teacher data (S23). The multiple weak learners 621 are trained by machine learning to input the captured image P1 and output the amount of pressing of the roller 3. The classification model 131 is trained by machine learning to output appearance information indicating the appearance of the material 5 formed on the surface of the roller 3 shown in the captured image P1.

[0109] This has the advantage that machine learning of the classification model 131 can be performed using second training data that is relatively reliable, making it easier to improve the prediction accuracy of the classification model 131.

[0110] The estimation system, estimation method, data selection system, and data selection method of the present disclosure can be applied to a system for estimating the state of a material applied to a substrate by a roller, etc. In this way, the estimation system, estimation method, data selection system, and data selection method of the present disclosure are industrially useful.

[0111] 1, 1A Estimation system 11 Acquisition unit 12 Image processing unit 13 Classification unit 131 Classification model 14 Estimation unit 15 Memory unit 2 Camera 3 Roller 31 First roller 32 Second roller 33 Third roller 4 Base material 5 Material 51 Streak 52 Defect 6 Data selection system 61 Data acquisition unit 62 Data selection unit 621 Weak learner 63 Memory unit 7 Sensor H1, H2 Height P1 Captured image P2 Extracted image P21 First extracted image P22 Second extracted image W1 Width

Claims

1. In a process of applying a material to a substrate using a roller, an acquisition unit that acquires a captured image of the surface of the roller, a classification unit that outputs aspect information indicating the aspect of the material formed on the surface of the roller shown in the captured image using a classification model learned by machine learning, and an estimation unit that outputs state information regarding the application state of the material on the substrate based on the aspect information output by the classification unit. An estimation system.

2. The estimation system according to claim 1, wherein the estimation unit outputs information indicating the film thickness of the material applied to the substrate as the state information.

3. The estimation system according to claim 2, wherein the estimation unit outputs the state information by referring to correlation data indicating the correlation between the aspect information and the application state of the material.

4. The estimation system according to any one of claims 1 to 3, wherein the estimation unit outputs information indicating whether the application state of the material is normal or abnormal as the state information.

5. The estimation system according to any one of claims 1 to 3, wherein the estimation unit outputs the state information by further referring to environment information indicating the environment where the material is placed.

6. The estimation system according to any one of claims 1 to 3, further comprising an image processing unit that executes image processing to generate an extraction image obtained by extracting the surface of the roller in the captured image acquired by the acquisition unit, and the classification unit uses the extraction image generated by the image processing unit as the captured image.

7. The estimation system according to claim 6, wherein the image processing unit generates the extraction image by extracting a predetermined region in the rotational direction of the roller from the surface of the roller shown in the captured image.

8. In a process of applying a material to a substrate using a roller, a captured image of the surface of the roller is acquired, aspect information indicating the aspect of the material formed on the surface of the roller shown in the captured image is output using a classification model learned by machine learning, and state information regarding the application state of the material on the substrate is output based on the aspect information. An estimation method.

9. In a process of applying a material to a substrate using a roller, a data acquisition unit that acquires first teacher data in which an imaging amount of the roller with respect to the substrate is attached as a correct label to an imaging image obtained by imaging the surface of the roller; and a data selection unit that selects second teacher data used for learning a classification model from the first teacher data based on information output by a plurality of weak learners learned by machine learning using a part of the first teacher data among the first teacher data, wherein the plurality of weak learners are learned by machine learning to output the pushing amount of the roller with respect to the substrate with the imaging image as an input, and the classification model is learned by machine learning to output appearance information indicating the appearance of the material formed on the surface of the roller indicated by the imaging image. Data selection system.

10. In a process of applying a material to a substrate using a roller, first teacher data in which an imaging amount of the roller with respect to the substrate is attached as a correct label to an imaging image obtained by imaging the surface of the roller is acquired, and second teacher data used for learning a classification model is selected from the first teacher data based on information output by a plurality of weak learners learned by machine learning using a part of the first teacher data among the first teacher data, wherein the plurality of weak learners are learned by machine learning to output the pushing amount of the roller with respect to the substrate with the imaging image as an input, and the classification model is learned by machine learning to output appearance information indicating the appearance of the material formed on the surface of the roller indicated by the imaging image. Data selection method.

Images