Information processing apparatus, information processing method, program, and manufacturing management system

Abstract

To support manufacturing based on the correlation of manufacturing process information and a plurality of pieces of sensing information in a manufacturing device having a plurality of sensing devices.SOLUTION: Executing a process of acquiring identification information of an article manufactured by a manufacturing device, manufacturing process information of the article, first sensing information obtained from a first sensing result of the article by a first sensing device, and second sensing information obtained from a second sensing result of the article by a second sensing device, and a process of generating a learned learning model of machine learning based on the manufacturing process information, the first sensing information, and the second sensing information; An information processing device that generates a first learning model that inputs information corresponding to manufacturing process information and outputs prediction results of first sensing information and second sensing information, and a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, a program, and a manufacturing management system. [Background technology]

[0002] 2. Description of the Related Art Defects that occur in products during the manufacturing process are judged using images. In the inspection method described in Patent Document 1, defects detected in an object to be inspected are classified and the size of the defects is determined (see Patent Document 1). In the technology described in Patent Document 1, the results of the appearance inspection in the defect determination unit are fed back to the mass production process (see paragraph 0035 of Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-325711 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional techniques described above, the correlation between manufacturing process information and multiple sensing devices is not fully taken into consideration, and therefore, for example, it is not possible to fully achieve high yields through feedback.

[0005] The present disclosure has been made in consideration of these circumstances, and aims to provide an information processing device, an information processing method, a program, and a manufacturing management system that can support manufacturing based on the correlation between manufacturing process information and multiple sensing information in a manufacturing device having multiple sensing devices. [Means for solving the problem]

[0006] One aspect of the present disclosure is an information processing device that includes an information processing unit, wherein the information processing unit executes a process of acquiring identification information of an item that is a finished product or intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device, and a process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information, and the information processing unit generates at least one of the learning models: a first learning model that inputs information corresponding to the manufacturing process information and outputs a prediction result of the first sensing information and the second sensing information; and a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information.

[0007] One aspect of the present disclosure is an information processing method in which an information processing unit of an information processing device executes a process of acquiring identification information of an item, which is a finished product or intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device, and a process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information, and the information processing unit generates at least one of the learning models: a first learning model that inputs information corresponding to the manufacturing process information and outputs a prediction result of the first sensing information and the second sensing information; and a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing the correlation between the first sensing information and the second sensing information.

[0008] One aspect of the present disclosure is a program for enabling a computer to perform the following functions: acquire identification information of an item, which is a finished product or intermediate product manufactured by a manufacturing apparatus having a first sensing device and a second sensing device; manufacturing process information of the item; first sensing information obtained from a first sensing result of the item by the first sensing device; and second sensing information obtained from a second sensing result of the item by the second sensing device; and generate a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; wherein the program generates at least one of the learning models: a first learning model that inputs information corresponding to the manufacturing process information and outputs a prediction result of the first sensing information and the second sensing information; and a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing the correlation between the first sensing information and the second sensing information.

[0009] One aspect of the present disclosure is a manufacturing management system having an information processing device, wherein the information processing device is equipped with an information processing unit, and the information processing unit executes a process of acquiring identification information of an item that is a finished product or intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device, and a process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information, and the information processing unit generates at least one of the learning models: a first learning model that inputs information corresponding to the manufacturing process information and outputs a prediction result of the first sensing information and the second sensing information; and a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing the correlation between the first sensing information and the second sensing information. [Effects of the Invention]

[0010] According to the information processing device, information processing method, program, and manufacturing management system disclosed herein, manufacturing can be supported based on the correlation between manufacturing process information and multiple pieces of sensing information in a manufacturing device having multiple sensing devices. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a manufacturing management system according to an embodiment. [Figure 2A] FIG. 1 is a diagram illustrating an example of the configuration of a manufacturing apparatus according to an embodiment. [Figure 2B] FIG. 10 is a diagram showing an example of the configuration of a manufacturing apparatus according to a modified example. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of a management device according to the embodiment. [Figure 4A] 10A and 10B are diagrams for explaining an example of a collection process in an external communication device according to an embodiment. [Figure 4B] 10 is a diagram illustrating an example of information processing in an information processing unit of a management device according to an embodiment. FIG. [Figure 4C] 10A and 10B are diagrams for explaining an example of image identification processing in an information processing unit of a management device according to an embodiment. [Figure 4D] 10A and 10B are diagrams for explaining an example of feature amount processing in an information processing unit of a management device according to an embodiment; [Figure 5] FIG. 10 is a diagram illustrating an example of a procedure of a process performed by an information processing unit of the management apparatus according to the embodiment. [Figure 6A] FIG. 10 is a diagram for explaining an example of adding an information processing function when an information processing unit is installed in a cloud according to an embodiment. [Figure 6B] FIG. 10 is a diagram for explaining an example of information processing when a prediction model is installed in a cloud according to an embodiment. [Figure 7A] FIG. 10 is a diagram illustrating an example of input and output of a learning model during learning according to an embodiment. [Figure 7B] FIG. 10 is a diagram illustrating an example of input and output of a trained learning model according to an embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a learning device and a determination device according to a modified example of the embodiment. [Figure 9] FIG. 1 is a diagram illustrating an example of the hardware configuration of an information processing apparatus according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0013] FIG. 1 is a diagram showing an example of the configuration of a manufacturing management system 1 according to an embodiment. The manufacturing management system 1 includes a management device 11, which is an example of an information processing device, a plurality of factories A1 to A3, and a network 31.

[0014] The factory A1 includes a manufacturing device 21, a factory server 22, and an external communication device 23. The manufacturing device 21 is a device that manufactures a predetermined product, and has, for example, a manufacturing line for manufacturing the product. Note that manufacturing may also be called production, etc. Furthermore, a manufacturing line may also be called a production line, etc.

[0015] The factory server 22 manages information related to the manufacturing of products performed by the manufacturing equipment 21, information related to the inspection of the products, etc. The information related to the inspection may include, for example, information related to the inspection conditions, and may also include information related to the inspection results. The external communication device 23 collects necessary information from the factory server 22 and transmits the collected information to the management device 11 via the network 31 . Here, the transmission of information from the external communication device 23 to the management device 11 may be performed spontaneously by the external communication device 23, or may be performed in response to a request from the management device 11, for example. In this embodiment, the factory server 22 and the external communication device 23 are separate entities, but as another example, they may be configured as an integrated entity.

[0016] Here, the configuration of the factory A2 and the configuration of the factory A3 are similar to the configuration of the factory A1, for example. In this embodiment, the manufacturing management system 1 manages the manufacturing of products in a plurality of factories A1 to A3. In this embodiment, the management device 11 performs processing independently for each of the multiple factories A1 to A3 based on information acquired from the external communication device 23 of the corresponding factory. As another example, the management device 11 may combine information acquired from two or more factories and perform processing based on such combined information.

[0017] In the example of Figure 1, the manufacturing management system 1 is shown managing three factories A1 to A3, but in other examples, the number of factories to be managed may be two, or may be four or more. Furthermore, the manufacturing management system 1 may manage only one factory. In this embodiment, for convenience of explanation, a case where the management device 11 manages a factory A1 will be described as an example.

[0018] [Manufacturing equipment] FIG. 2A is a diagram showing an example of the configuration of a manufacturing apparatus 21 according to an embodiment. The manufacturing apparatus 21 includes devices for implementing a series of manufacturing line processes for manufacturing a predetermined product. In this embodiment, the manufacturing apparatus 21 includes an input unit 111, a first process machine B1, a first inspection machine C1, a second process machine B2, a 2-1 inspection machine C2-1 to a 2-3 inspection machine C2-3, a third process machine B3, a third inspection machine C3, and an output unit 112.

[0019] In this embodiment, in the manufacturing line of the manufacturing apparatus 21, the manufacturing steps (also referred to as the manufacturing process in this embodiment) progress from the input section 111 to the output section 112. In this embodiment, for convenience of explanation, the side of the input unit 111 is called the upstream or previous stage, and the side of the output unit 112 is called the downstream or subsequent stage.

[0020] In this embodiment, the inspection machines (first inspection machine C1, second-first inspection machine C2-1 to second-third inspection machine C2-3, and third inspection machine C3) are an example of a sensing device. In addition, in this embodiment, the inspection results obtained by the inspection machine are an example of the sensing results sensed by the sensing device. In addition, in this embodiment, information (inspection result information) obtained from the inspection result acquired by the inspection machine is an example of sensing information obtained from the sensing result sensed by the sensing device.

[0021] The test result information (sensing information) may be the same as the test result (sensing result), or may be different. For example, the test result (sensing result) may represent raw data of the test (sensing) result. Furthermore, for example, the test result information (sensing information) may represent the raw data itself, or may represent other data obtained based on the raw data, or may represent a combination of both of these data.

[0022] The input unit 111 inputs the materials for the product to be manufactured. The first process machine B1 (also referred to as process machine α in the example of Figure 2A) performs a predetermined first manufacturing process (process α) on the material to produce a first item (for convenience of explanation, also referred to as a first intermediate product). The first inspection machine C1 (also referred to as inspection machine α in the example of FIG. 2A) performs a predetermined first inspection (inspection α) on the first intermediate product.

[0023] The second process machine B2 (also referred to as process machine β in the example of Figure 2A) performs a predetermined second manufacturing process (process β) on the inspected first intermediate product to produce a second item (for convenience of explanation, also referred to as a second intermediate product). The 2-1 inspection machine C2-1 (also referred to as inspection machine β1 in the example of FIG. 2A) performs a predetermined 2-1 inspection (inspection β1) on the second intermediate product. The 2-2 inspection machine C2-2 (also referred to as inspection machine β2 in the example of FIG. 2A) performs a predetermined 2-2 inspection (inspection β2) on the second intermediate product. The 2-3 inspection machine C2-3 (also referred to as inspection machine β3 in the example of FIG. 2A) performs a predetermined 2-3 inspection (inspection β3) on the second intermediate product.

[0024] In this embodiment, the 2-1 inspection machine C2-1, the 2-2 inspection machine C2-2, and the 2-3 inspection machine C2-3 are arranged in parallel between the second process machine B2 and the third process machine B3. The inspection target by these inspection machines (2-1 inspection machine C2-1 to 2-3 inspection machine C2-3) is the same item (second intermediate product).

[0025] The third process machine B3 (also referred to as process machine γ in the example of Figure 2A) performs a predetermined third manufacturing process (process γ) on these inspected second intermediate products to produce a third item (for convenience of explanation, also referred to as a product). In this embodiment, the third article is a final product manufactured by a manufacturing line. In this embodiment, the product is formed after a predetermined third manufacturing process (process γ), but the number of manufacturing processes is not limited. A fourth article may be generated by performing a fourth process after the third manufacturing process. The third inspection machine C3 (also referred to as inspection machine γ in the example of FIG. 2A) performs a predetermined third inspection (inspection γ) on the product. The output unit 112 outputs the inspected product to a predetermined location (for example, a storehouse or a location for the next process).

[0026] Here, in the manufacturing apparatus 21, one manufacturing process from among a plurality of different manufacturing processes can be selected as the first manufacturing process of the first process machine B1. Similarly, in the manufacturing apparatus 21, one manufacturing process from among a plurality of different manufacturing processes can be selected as the second manufacturing process of the second process machine B2. Similarly, in the manufacturing apparatus 21, one manufacturing process from among a plurality of different manufacturing processes can be selected as the third manufacturing process of the third process machine B3. Note that selecting one manufacturing process from a plurality of different manufacturing processes may be achieved, for example, by changing the equipment used as the process machine, or by keeping the equipment used as the process machine the same and changing the parameters, etc., used in that equipment during manufacturing.

[0027] Any manufacturing process may be carried out in each of the process machines (first process machine B1 to third process machine B3). As a specific example, a film forming process, a cleaning process, and a processing process may be performed in each of the first process machine B1, the second process machine B2, and the third process machine B3, respectively. In this embodiment, the manufacturing line is a batch-type manufacturing line, and the plurality of inspection machines each perform a predetermined inspection in a batch-type manner. The inspection may be, for example, a highly sensitive inspection for a precise manufacturing process.

[0028] In addition, each inspection machine may have the function of determining whether the item being inspected is normal or not, for example, based on the image information acquired by each inspection machine, whether there is no defect in the image information (i.e., the inspection result is OK) or whether there is a defect in the image information (i.e., the inspection result is NG).

[0029] <Modifications of Manufacturing Equipment> FIG. 2B is a diagram showing an example of the configuration of a manufacturing apparatus 21a according to a modified example. For convenience of explanation, in the example of FIG. 2B, the same devices as those in the example of FIG. 2A are denoted by the same reference numerals.

[0030] The manufacturing apparatus 21a according to the modified example includes a second inspection machine C2a-1 to a fourth inspection machine C2a-3 instead of the 2-1st inspection machine C2-1 to the 2-3rd inspection machine C2-3 in the example of FIG. 2A. In the manufacturing apparatus 21a according to the modified example, a second inspection machine C2a-1, a third inspection machine C2a-2, and a fourth inspection machine C2a-3 are arranged in series between the second process machine B2 and the third process machine B3. In the example (modification) of FIG. 2B, the third inspection machine C3 shown in FIG. 2A is regarded as the fifth inspection machine.

[0031] The second inspection machine C2a-1 (similar to the inspection machine β1 in this example) performs a predetermined second inspection (similar to the inspection β1 in this example) on the second intermediate product. The third inspection machine C2a-2 (similar to the inspection machine β2 in this example) performs a predetermined third inspection (similar to the inspection machine β2 in this example) on the second intermediate product after the inspection. A fourth inspection machine C2a-3 (similar to the inspection machine β3 in this example) performs a predetermined fourth inspection (similar to the inspection machine β3 in this example) on the second intermediate product after the inspection. The third process machine B3 performs a predetermined third manufacturing process (process γ) on the inspected second intermediate product to produce a product.

[0032] In this way, the three inspection machines provided between the second process machine B2 and the third process machine B3 may be arranged in parallel as in the example of Figure 2A, or may be arranged in series as in the example of Figure 2B.

[0033] As another example, a manufacturing apparatus having both two or more inspection machines arranged in parallel and two or more inspection machines arranged in series may be used. The number of process machines provided in the manufacturing apparatus may be any number equal to or greater than 1. Furthermore, various types of manufacturing processes may be performed by each process machine. The number of inspection machines provided in the manufacturing equipment may be any number equal to or greater than 2. Various types of inspections may be performed by each inspection machine.

[0034] [Factory Server] The factory server 22 stores various information related to the manufacturing equipment 21 . In this embodiment, the factory server 22 stores information about the manufacturing processes performed in each of the process machines (first process machine B1 to third process machine B3). This information may be set in the factory server 22 in advance, for example. In addition, in this embodiment, the factory server 22 stores information about the images acquired by each inspection machine (first inspection machine C1, second-first inspection machine C2-1 to second-third inspection machine C2-3, third inspection machine C3) and information about the amount of light acquired by each inspection machine.

[0035] [Management device] FIG. 3 is a diagram illustrating an example of the configuration of the management device 11 according to the embodiment. The management device 11 is configured using, for example, a computer.

[0036] The management device 11 includes an input unit 211 , an output unit 212 , a communication unit 213 , a storage unit 214 , a control unit 215 , and an information processing unit 216 . The input unit 211 includes an operation unit 231 . The output unit 212 includes a display unit 232 . The information processing unit 216 includes an information acquisition processing unit 251 , an analysis processing unit 252 , and a visualization processing unit 253 . The analysis processing unit 252 includes an image identification processing unit 271 and a feature amount processing unit 272 .

[0037] The input unit 211 receives information from the outside. The input unit 211 has, for example, an operation unit 231 that accepts operations performed by an operator (person), and inputs information according to the operations accepted by the operation unit 231. The operation unit 231 may have a function to accept operations from a touch panel, or may have a function to accept operations from physical keys. The operation unit 231 may also have a function to accept operations by voice (for example, the voice of an operator). Furthermore, the input unit 211 may be connected to an external device and receive information output from the external device. The external device may be, for example, a portable recording medium. The recording medium may be called, for example, a storage medium.

[0038] The output unit 212 outputs the information. The output unit 212 has, for example, a display unit 232, and displays (outputs) information on the screen of the display unit 232. The screen may have a touch panel function. The output unit 212 may also be connected to an external device and output information to the external device. The external device may be, for example, a portable recording medium. The output unit 212 may output information in a form other than a display, such as audio output.

[0039] In this embodiment, the input unit 211 and the output unit 212 are shown as separate functional units, but the input unit 211 and the output unit 212 may be configured as a common functional unit (input / output unit). The input / output unit may be configured as a touch panel.

[0040] The communication unit 213 has a function of communicating information with an external device. This communication may be wired or wireless. Here, in this embodiment, the communication unit 213 is shown as a functional unit separate from the input unit 211 and the output unit 212, but the receiving function of the communication unit 213 may be considered to be included in the function of the input unit 211, and the transmitting function of the communication unit 213 may be considered to be included in the function of the output unit 212.

[0041] The storage unit 214 stores information. The storage unit 214 may store any information. In this embodiment, the storage unit 214 stores management information D1, a first learning model E1, a second learning model E2, and an image recognition learning model group W. The management information D1 may be various information.

[0042] The first learning model E1 is a machine learning learning model that makes a predetermined prediction. The second learning model E2 is a machine learning learning model that calculates a predetermined correlation (importance).

[0043] The image discrimination learning model group W includes multiple machine learning learning models, and in this embodiment, includes a first image discrimination learning model W1, a second-first image discrimination learning model W2-1, a second-second image discrimination learning model W2-2, a second-third image discrimination learning model W2-3, and a third image discrimination learning model W3. Each of these multiple learning models is a learning model for discriminating images.

[0044] In this embodiment, the first image discrimination learning model W1, the second-first image discrimination learning model W2-1, the second-second image discrimination learning model W2-2, the second-third image discrimination learning model W2-3, and the third image discrimination learning model W3 correspond to the first inspection machine C1, the second-first inspection machine C2-1, the second-second inspection machine C2-2, the second-third inspection machine C2-3, and the third inspection machine C3, respectively. In this embodiment, a case is shown in which separate image recognition learning models are used for the 2-1 inspection machine C2-1, the 2-2 inspection machine C2-2, and the 2-3 inspection machine C2-3, respectively. However, as another example, a common image recognition learning model may be used for the second inspection machines (2-1 inspection machine C2-1, the 2-2 inspection machine C2-2, and the 2-3 inspection machine C2-3).

[0045] The control unit 215 performs various controls and processes. In this embodiment, the control unit 215 includes a processor such as a CPU (Central Processing Unit), and executes a predetermined program (control program) by the processor to perform control and processing as defined in the program. The program may be stored in the storage unit 214, for example.

[0046] The information processing unit 216 performs various types of information processing. The information acquisition processing unit 251 performs processing to acquire necessary information. The analysis processing unit 252 performs various types of analysis processing. In this embodiment, the analysis includes image identification, calculation of predetermined feature amounts, and judgment using the feature amounts.

[0047] The image identification processing unit 271 performs processing to identify an image. In this embodiment, the image identification processing unit 271 performs a process for identifying the image acquired by each inspection machine. In this embodiment, the image identification processing unit 271 performs image identification processing using a machine learning learning model for each inspection machine. In this embodiment, a learning model included in the image identification learning model group W is used as the learning model.

[0048] The feature amount processing unit 272 performs various processes including calculation of feature amounts. For example, it is possible to find correlation based on the feature amounts. In this embodiment, the feature amount processing unit 272 performs prediction processing using a machine learning learning model. In this embodiment, the first learning model E1 is used as the learning model. In this embodiment, the feature amount processing unit 272 performs a process of calculating the importance using a machine learning learning model. In this embodiment, the second learning model E2 is used as the learning model.

[0049] The visualization processing unit 253 visualizes predetermined information. In this embodiment, the visualization processing unit 253 displays the predetermined information on the display unit 232. The specified information may be, for example, information regarding the results of the prediction processing performed by the feature processing unit 272, or information regarding the results of the importance calculation processing performed by the feature processing unit 272.

[0050] [Example of processing in a manufacturing control system] An example of processing in the manufacturing control system 1 will be described with reference to FIGS. 4A, 4B, 4C, and 4D.

[0051] FIG. 4A is a diagram for explaining an example of the collection process T101 in the external communication device 23 according to the embodiment. In the collection process T101, information is collected and organized. In this embodiment, in the collection process T101, the external communication device 23 performs an acquisition process V101, a storage process V102, and an organization process V103.

[0052] In the acquisition process V101, the external communication device 23 acquires various information from the factory server 22. In the storage process V102, the external communication device 23 classifies the acquired information into manufacturing process information F1, inspection image information F2, inspection information F3, and inspection light amount information F4, and stores (stores) the information.

[0053] The manufacturing process information F1 includes information that identifies the manufacturing process performed in each of the process machines (first process machine B1 to third process machine B3). The manufacturing process information F1 may include information specifying the conditions of the manufacturing process, such as the current conditions during film formation or the model number of the machine used.

[0054] The inspection image information F2 includes information on images acquired by each of the inspection machines (first inspection machine C1, second-first inspection machine C2-1 to second-third inspection machine C2-3, and third inspection machine C3). In this embodiment, for example, a single item may have multiple defects. In this case, information on one image (or two or more images) may be acquired for each defect. For example, if a single item has 100 defects, information on a total of 100 images (or more) may be acquired.

[0055] The inspection information F3 includes information about defects contained in images acquired by each of the inspection machines (first inspection machine C1, second-first inspection machine C2-1 to second-third inspection machine C2-3, and third inspection machine C3). The inspection information F3 may include, for example, information such as the coordinates of each defect or the size of each defect. The inspection information F3 may be generated for each individual defect, or may be generated as information collectively regarding multiple defects. The examination information F3 may be, for example, information in text format, or may be information in another format such as CSV.

[0056] The inspection light intensity information F4 includes information that identifies the amount of light (inspection light intensity) used for inspection (capturing the image) for the images acquired by each inspection machine (first inspection machine C1, second-first inspection machine C2-1 to second-third inspection machine C2-3, third inspection machine C3). In this embodiment, the inspection light intensity may change due to aging of each inspection machine, environmental changes, etc. Therefore, the inspection light intensity may be used as needed for correction (e.g., correction of light intensity) when making a judgment regarding defects detected during inspection.

[0057] In the organizing process V103, the external communication device 23 organizes the stored information. In this embodiment, the external communication device 23 manages information for each product type as product type information F11. The external communication device 23 manages information for each inspection machine as inspection machine-specific information F12. The external communication device 23 manages information for each process (each manufacturing process) as process-specific information F13.

[0058] Here, each piece of information may be linked (associated) with, for example, product identification information or lot identification information. In this embodiment, for example, a glass substrate (glass substrate) is used as the article. In this embodiment, a set of a predetermined number of articles (substrates in this embodiment) is managed as a lot.

[0059] In the series of process machines (first process machine B1 to third process machine B3), each manufacturing process is carried out, for example, for each item or lot as a single unit. Furthermore, in the series of process machines (first process machine B1 to third process machine B3), for example, various combinations of manufacturing processes may be performed, and in this case, inspection results regarding defects are obtained by the respective inspection machines in each of the combinations.

[0060] Similarly, each piece of information may be linked (associated) with identification information of the type of product manufactured by the manufacturing device 21. In this embodiment, the product type identification information specifies the quality required for that product type. In other words, generally, even for the same type of item, the required quality may differ for each product type, and therefore the manufacturing process and the inspection results (detected defect status) may differ for each product type.

[0061] Similarly, each piece of information may be linked (associated) with the identification information of the inspection machine. Similarly, each piece of information may be linked (associated) with identification information of the process.

[0062] Note that each process may be, for example, a series of manufacturing processes (in the example of FIG. 2A, the manufacturing processes in all of the first process machine B1 to the third process machine) that are the same, or may be a series of manufacturing processes in which some of the manufacturing processes are the same. The part of the manufacturing processes may be any number of manufacturing processes greater than or equal to one, and in the example of FIG. 2A, may be any one manufacturing process, or may be any two manufacturing processes.

[0063] FIG. 4B is a diagram for explaining an example of information processing T1 in the information processing unit 216 of the management device 11 according to the embodiment. In the information processing T1, the information processing unit 216 performs an analysis processing T11 and a visualization processing T12. In the analysis process T11, the analysis processing unit 252 performs a storage process V1, an image identification process V2, a feature amount process V3, and a storage process V4. In the visualization process T12, the visualization processing unit 253 performs a prediction result display process V11 and an importance display process V12.

[0064] In the storage process V1, the analysis processing unit 252 receives the manufacturing process information F1a, the inspection image information F2a, the inspection information F3a, and the inspection light amount information F4a from the external communication device 23 and stores (stores) them. Here, the manufacturing process information F1a, inspection image information F2a, inspection information F3a, and inspection light intensity information F4a may be, for example, the same information as the manufacturing process information F1, inspection image information F2, inspection information F3, and inspection light intensity information F4 in the example of Figure 4A, or may have the same essential content but a different format, etc.

[0065] FIG. 4C is a diagram for explaining an example of image identification processing V2 in the information processing unit 216 of the management device 11 according to the embodiment. In this embodiment, the image identification process V2 is performed for each of the first inspection machine C1, the second-first inspection machine C2-1, the second-second inspection machine C2-2, the second-third inspection machine C2-3, and the third inspection machine C3. As another example, the image identification process V2 may be performed collectively for the second inspection machines (the 2-1st inspection machine C2-1 to the 2-3rd inspection machine C2-3).

[0066] In the learning process in the image discrimination process V2, the image discrimination processing unit 271 performs a learning data selection process V21 and an image discrimination learning process V22. In the learning data selection process V21, the image identification processing unit 271 performs a process of selecting information to be used as learning data for machine learning from the inspection image information F2a, the inspection information F3a, and the inspection light intensity information F4a. Then, the image identification processing unit 271 stores the information selected as learning data as learning data I1. Here, the learning data I1 may include, for example, selected inspection image information for the relevant inspection machine, inspection information related to the inspection image information, and inspection light intensity information at the time the inspection image information was acquired.

[0067] In the image identification learning process V22, the image identification processing unit 271 uses the learning data I1 to perform learning (machine learning) of a learning model corresponding to the corresponding inspection machine. In this embodiment, the learning model inputs image data (inspection image information) and outputs information on a determination result (inference result) regarding defects shown in the image of the image data. Various information may be used as information on the judgment results regarding defects, such as one or more of information indicating the presence or absence of a defect, information indicating the possibility (likelihood) of a defect, information indicating the type (kind) of defect, or information indicating the amount (e.g., number) of defects. Various types of defects may be used, such as bubbles, scratches, etc. Furthermore, the defect types may be classified according to the size of the defect, such as large, medium, or small. Here, in the image identification learning process V22, the image identification processing unit 271 may use the learning data I1 as training data.

[0068] In the example of Figure 4C, the first image discrimination learning model W1, which is a learning model corresponding to the first inspection machine C1, the second-first image discrimination learning model W2-1, which is a learning model corresponding to the second-first inspection machine C2-1, the second-second image discrimination learning model W2-2, which is a learning model corresponding to the second-second inspection machine C2-2, the second-third image discrimination learning model W2-3, which is a learning model corresponding to the second-third inspection machine C2-3, and the third image discrimination learning model W3, which is a learning model corresponding to the third inspection machine C3, are shown.

[0069] <Decision processing for image recognition> In the judgment process for image identification in image identification process V2, the image identification processing unit 271 inputs the image data (inspection image information) to be judged into a trained learning model corresponding to the relevant inspection machine, and obtains the information output from the learning model as judgment result information (also referred to as defect judgment information in this embodiment). Here, in the example of Figure 4C, the trained learning model (image discrimination learning model) used for image discrimination is the learning model corresponding to the relevant inspection machine from among the trained first image discrimination learning model W1, the trained second-first image discrimination learning model W2-1, the trained second-second image discrimination learning model W2-2, the trained second-third image discrimination learning model W2-3, and the trained third image discrimination learning model W3.

[0070] Then, the image identification processing unit 271 stores the acquired defect determination information. Here, in the example of Figure 4C, the defect judgment information obtained by the first image discrimination learning model W1 is shown as first defect judgment information G1, the defect judgment information obtained by the 2-1 image discrimination learning model W2-1 is shown as 2-1 defect judgment information G2-1, the defect judgment information obtained by the 2-2 image discrimination learning model W2-2 is shown as 2-2 defect judgment information G2-2, the defect judgment information obtained by the 2-3 image discrimination learning model W2-3 is shown as 2-3 defect judgment information G2-3, and the defect judgment information obtained by the third image discrimination learning model W3 is shown as third defect judgment information G3.

[0071] The function of the learning data selection process V21, the function of the image recognition learning process V22, and the function of the judgment process for image recognition may be provided separately for each inspection machine, or a common function may be provided for multiple inspection machines, and this function may perform processing for each of the multiple inspection machines by time sharing or the like.

[0072] In this embodiment, the image identification processing unit 271 may identify the type of defect (defect type) for image data (image information) in which a defect has occurred in an article, and may trend the identification result. In other words, the image identification processing unit 271 may acquire the defect occurrence status (for example, the number of defects) in chronological order for each defect type. The image identification processing unit 271 may then investigate the correlation between such defect time series information, manufacturing process information, and inspection machine light amount information. Furthermore, the image identification processing unit 271 may use information about each defect as a response variable, and may calculate the importance of the manufacturing process information as a numerical value.

[0073] FIG. 4D is a diagram for explaining an example of feature amount processing V3 in the information processing unit 216 of the management device 11 according to the embodiment. In the feature amount process V3, the feature amount processing unit 272 performs unit-by-unit linking and aggregation process V31, defect-specific regression process V32, and correlation analysis process V33.

[0074] In the unit-by-unit linking and aggregation process V31, the feature processing unit 272 performs a linking process (corresponding process) for each predetermined unit among the information contained in the first defect judgment information G1, the information contained in the second-first defect judgment information G2-1, the information contained in the second-second defect judgment information G2-2, the information contained in the second-third defect judgment information G2-3, the information contained in the third defect judgment information G3, and the information contained in the manufacturing process information F1a, and aggregates the linked information. In this embodiment, the predetermined unit is, for example, a unit for monitoring defects, and specifically, it may be a unit of one article or a unit of a lot. In this embodiment, one lot represents a set of a predetermined number of articles (for example, 10 articles).

[0075] <Regression processing by defect> In the defect-specific regression processing V32, the feature processing unit 272 predicts defects that are inferred to occur for each defect type based on the information of the aggregation results of the unit-by-unit linking and aggregation processing V31, and obtains information on the results of the prediction (prediction information H1). In this embodiment, the feature processing unit 272 makes such predictions using the trained first learning model E1.

[0076] The first learning model E1 receives input of information on a series of manufacturing processes shown in FIG. 2A, for example, and outputs, for each defect type, information on defects that are inferred to occur as prediction result information. Here, "by defect type" may refer to, for example, both by inspection machine and by defect type. Note that if each inspection machine is configured to detect different types of defects, "by inspection machine" and "by defect type" have essentially the same meaning. Furthermore, as the prediction information H1, for each type of defect, one or more of information indicating the presence or absence of a defect, information indicating the possibility (likelihood) of a defect, or information indicating the amount (e.g., number) of defects may be used.

[0077] In this embodiment, the prediction information H1 is information resulting from predicting the state of defects for each defect type that are inferred to occur in each inspection machine for the information on the series of manufacturing processes shown in Fig. 2A. In other words, by assuming what manufacturing processes will be used as a series of manufacturing processes, it is possible to know the predicted state of occurrence of defects for each defect type in that case. Note that the feature processing unit 272 may, for example, perform learning of the first learning model E1 in advance.

[0078] As another example, the first learning model E1 may be configured to input information about a part of the series of manufacturing processes shown in Figure 2A, and output information on the results of predicting the defect conditions for each defect type that is inferred to occur in each inspection machine. Here, the part of the manufacturing process may be one or more arbitrary manufacturing processes.

[0079] <Correlation analysis processing> In the correlation analysis process V33, the feature amount processing unit 272 acquires importance information (importance information H2) for each defect type and each manufacturing process based on the information of the aggregation result of the unit-based linking and aggregation process V31. The importance is based on, for example, the correlation between the defect status for each defect type and the manufacturing process. In this embodiment, the feature processing unit 272 makes such predictions using the trained second learning model E2.

[0080] The second learning model E2 receives input of information on the series of manufacturing processes shown in FIG. 2A, for example, and outputs the importance of each manufacturing process for each defect type as inference result information. Here, "each manufacturing process" may be, for example, each process machine and each selectable manufacturing process, or, as another example, when it does not depend on the selectable manufacturing processes for each process machine, "each manufacturing process" here may be considered to have substantially the same meaning as "each process machine."

[0081] In this embodiment, the importance information H2 is information resulting from inferring the degree of correlation (importance) between each manufacturing process and each defect type for the information on the series of manufacturing processes shown in Fig. 2A. In other words, by assuming what manufacturing processes will be used as a series of manufacturing processes, it is possible to know how important the manufacturing process of each process machine is for each defect type in that case. Note that the feature processing unit 272 may, for example, perform learning of the second learning model E2 in advance.

[0082] As another example, the second learning model E2 may be configured to input information about, for example, some of the manufacturing processes in the series of manufacturing processes shown in Figure 2A, and output the importance of each manufacturing process for each type of defect as inference result information. Here, the part of the manufacturing process may be one or more arbitrary manufacturing processes.

[0083] <Prediction result display process> In the example of FIG. 4B, in the prediction result display process V11, the visualization processing unit 253 performs a process of displaying the content of the prediction information H1 or content based thereon on the display unit 232. As a specific example, such display content may be the amount (e.g., number) of defects predicted to occur for each defect type for a combination of manufacturing processes performed in each process machine (first process machine B1, second process machine B2, and third process machine B3). As an example, if the first inspection machine C1, the 2-1 inspection machine C2-1, the 2-2 inspection machine C2-2, the 2-3 inspection machine C2-3, and the third inspection machine C3 each detect defects of a different defect type, each defect type may be the first inspection machine C1, the 2-1 inspection machine C2-1, the 2-2 inspection machine C2-2, the 2-3 inspection machine C2-3, and the third inspection machine C3, respectively. As another example, if the second inspection machines (the 2-1 inspection machine C2-1 to the 2-3 inspection machine C2-3) are compatible with the same defect type, they may be collectively considered to be each defect type.

[0084] <Importance display processing> In the example of FIG. 4B, in the importance display process V12, the visualization processing unit 253 performs a process of displaying the content of the importance information H2 or content based thereon on the display unit 232. As a specific example, such display content may be the degree to which each manufacturing process correlates (importance) with each defect type for a combination of manufacturing processes performed in each process machine (first process machine B1, second process machine B2, and third process machine B3).

[0085] As an example, the visualization processing unit 253 may display information such as the importance of the number of defects of a specific defect type (e.g., bubble-like defects) detected by the first inspection machine C1 for a certain manufacturing process being 50% and the importance of another certain manufacturing process being 30%. Furthermore, for example, when importance is determined for each of a plurality of manufacturing processes for a certain defect type, the visualization processing unit 253 may display a predetermined number of importance levels from the highest importance level to the lowest importance level as the top (a predetermined number from the top) importance levels.

[0086] Here, we have shown a case where multi-valued information is used as the importance, but as another example, importance consisting of binary information indicating whether or not there is a correlation may be used. Note that when binary information is used, it is possible that even if there is some influence, it may be considered to have no influence, or even if there is not a significant influence, it may be considered to have an influence. As an example, the visualization processing unit 253 may display content indicating that a certain manufacturing process has a correlation with the number of defects of a specific defect type (e.g., scratch-like defects) detected by the third inspection machine C3, or may display content indicating that a certain other manufacturing process has no correlation. Whether or not there is a correlation may be determined based on whether or not the so-called R2 value exceeds a predetermined threshold value, for example. The method of displaying the importance is arbitrary, and for example, a display such as a scatter plot matrix may be used.

[0087] FIG. 5 is a diagram showing an example of a procedure of processing performed by the information processing unit 216 of the management device 11 according to the embodiment.

[0088] (Step S1) In the management device 11, the information acquisition processing unit 251 acquires the information collection result from the external communication device 23. Then, in the management device 11, the process proceeds to step S2.

[0089] (Step S2) In the management device 11, the analysis processing unit 252 performs a predetermined analysis process (the process of steps S21 to S22). Then, in the management device 11, the process proceeds to the process of step S3.

[0090] (Step S21) Here, in this analysis process, the image identification processing unit 271 performs the image identification process V2. Then, the management device 11 proceeds to the process of step S22. The image identification process V2 is performed for each inspection machine. (Step S22) In the analysis process, the feature amount processing unit 272 performs feature amount processing V3, and the management device 11 then ends the analysis process. In the feature amount process V3, for example, one or both of the defect-specific regression process V32 and the correlation analysis process V33 are performed.

[0091] (Step S3) In the management device 11, the visualization processing unit 253 displays the results of the feature amount processing V3. Then, in the management device 11, the process proceeds to step S4. In the visualization process T12, for example, one or both of a prediction result display process V11 and an importance display process V12 are performed.

[0092] (Step S4) In the management device 11, the information processing unit 216 may reflect predetermined control on the manufacturing device 21 in the factory A1 based on the result of the analysis by the analysis processing unit 252. The predetermined control may be, for example, control to change a predetermined parameter of a predetermined manufacturing process in the manufacturing device 21 of the factory A1.

[0093] <Example of real-time monitoring> The management device 11 may monitor the manufacturing devices 21 in the factory A1 in real time. The management device 11 may acquire necessary information in real time when a product is manufactured in the manufacturing equipment 21 of the factory A1, make a predetermined judgment regarding defects, and control the manufacturing equipment 21 based on the judgment results. The judgment may be, for example, a judgment for each defect type. The judgment may be, for example, a judgment predicting a future situation. At this time, the management device 11 may investigate the correlation between time-series information on the defect-related determination results and time-series information on the manufacturing process, for example. Furthermore, the management device 11 may investigate the correlation between, for example, time series information on the judgment results regarding defects, time series information on the manufacturing process, and time series information on the light intensity of the inspection machine.

[0094] <Example of non-real-time monitoring> The management device 11 may monitor the manufacturing devices 21 in the factory A1 in non-real time. After a product is manufactured in the manufacturing equipment 21 of the factory A1, the management device 11 may acquire necessary information in non-real time, make a predetermined judgment regarding defects, and control the manufacturing equipment 21 based on the judgment result. The judgment may be, for example, a judgment for each defect type.

[0095] FIG. 6A is a diagram for explaining an example of adding an information processing function when an information processing unit 216 is installed in the cloud according to the embodiment. It should be noted that FIG. 6A is a schematic diagram for the purpose of explanation and is not necessarily strict.

[0096] FIG. 6A shows a schematic configuration T201 relating to the first inspection machine C1, a schematic configuration T202 relating to the second inspection machines (2-1 inspection machine C2-1 to 2-3 inspection machine C2-3), and a schematic configuration T203 relating to the third inspection machine C3. Here, since these schematic configurations T201 to T203 are similar, the schematic configuration T201 will be described as a representative. In addition, in the example of FIG. 6A, for simplification, the second inspection machines (2-1 inspection machine C2-1 to 2-3 inspection machine C2-3) are shown together, but each of the 2-1 inspection machine C2-1 to 2-3 inspection machine C2-3 may have a separate configuration.

[0097] Predetermined information about the image acquired by the first inspection machine C1 is accumulated in a cloud storage T212 via a network T211. Here, the predetermined information includes, for example, image information and its identification information (ID), and may further include other information.

[0098] Next, the cloud computing resource T213 uses the predetermined information to make a predetermined judgment using a predetermined judgment model T214, which is a learning model of machine learning. Then, the cloud computing resource T213 notifies the first test machine C1 of the result of the determination (determination result T215) or control information based on the result via the network T211.

[0099] In this case, the cloud computing resource T213 can, for example, add an image processing function T231 to the determination model T214, or update an existing image processing function T231. In this case, it is also possible to add a new decision model T232 to the cloud computing resource T213, for example.

[0100] In this way, by configuring AI (Artificial Intelligence) judgments to be performed in the cloud, it is possible to add, update, delete, and verify functions and performance, add new judgment processes, update the contents of existing judgment processes, or add the latest judgment models (e.g., learning models), etc. Also, it is possible to immediately learn using the latest data, or to verify and immediately reflect a new learning model, for example.

[0101] When the management device 11 is provided in a cloud, the management device 11 may be used as the cloud storage T212 and the cloud computing resource T213. Also, for example, various types of information may be stored in a cloud database instead of in the storage unit 214 of the management device 11, and the management device 11 may acquire necessary information from the database.

[0102] FIG. 6B is a diagram for explaining an example of information processing when a prediction model T315 is installed in the cloud according to the embodiment. It should be noted that FIG. 6B is a schematic diagram for the purpose of explanation and is not necessarily strict.

[0103] FIG. 6B shows a schematic configuration T301 relating to a series of manufacturing processes and an inspection machine in the manufacturing equipment. Predetermined information is accumulated in a cloud storage T312 via a network T311. Here, the predetermined information includes, for example, information about the manufacturing process and information about the inspection results by the inspection machine, and may also include other information.

[0104] Next, the cloud computing resource T313 performs a predetermined determination using the predetermined information and obtains the result of the determination (determination result T314). Based on the determination result T314, the cloud computing resource T313 performs a predetermined prediction (determination) using a predetermined prediction model T315, which is a machine learning learning model. Then, the cloud computing resource T313 notifies the manufacturing equipment of the results of the prediction via the network T311.

[0105] In this case, the cloud computing resource T313 may, for example, determine the control content to be recommended for the manufacturing process by comparing the judgment result by the prediction model T315 with a predetermined target value T316, and reflect the determined control content in the manufacturing equipment 21. In this case, for example, predictions regarding inspection results for the manufacturing process can be made as a feedforward (FF) process. In this case, the degree of correlation (for example, importance) of the manufacturing process for each defect type can be predicted as a feedback (FB) process.

[0106] In this way, by configuring the AI ​​to make judgments in the cloud, feedforward processing and feedback processing can be performed for each manufacturing process based on the results of the judgments. Also, for example, it is possible to immediately learn using the latest data, or to verify and immediately reflect new learning models.

[0107] When the management device 11 is provided in a cloud, the management device 11 may be used as the cloud storage T312 and the cloud computing resource T313. Also, for example, various types of information may be stored in a cloud database instead of in the storage unit 214 of the management device 11, and the management device 11 may acquire necessary information from the database.

[0108] [Machine learning configuration example] An example of the machine learning configuration will be described with reference to FIGS. 7A, 7B, and 8. The configuration of the learning model described below may be applied to, for example, individual learning models in this embodiment (first learning model E1, second learning model E2, and each learning model of the image recognition learning model group W).

[0109] <Overview of machine learning training> FIG. 7A is a diagram illustrating an example of input and output of a learning model M1a during learning according to the embodiment. Here, the learning model M1a represents a state in which a predetermined learning model is undergoing machine learning. In general, the learning control function of the management device 11 (for example, the function of the information processing unit 216) updates the parameters of the learning model M1a based on the output data when predetermined input data is input to the learning model M1a. At this time, the learning control function may perform supervised learning using teacher data. As another example, unsupervised learning may be performed without using training data. It should be noted that the schematic diagram shown in FIG. 7A is a diagram for the purpose of general explanation and is not necessarily precise. The learning model under training may be referred to as, for example, a model under training.

[0110] <Outline of judgment using machine learning results> FIG. 7B is a diagram illustrating an example of input and output of a trained learning model M1b according to the embodiment. Here, the learning model M1b represents a state in which a predetermined learning model has been machine-learned. In general, the determination control function of the management device 11 (for example, the function of the information processing unit 216) acquires a predetermined determination result based on output data when predetermined input data is input to the learning model M1b. The determination result may be, for example, the output data from the learning model M1b itself, or may be other data obtained based on the output data from the learning model M1b. It should be noted that the schematic diagram shown in FIG. 7B is a diagram for the purpose of general explanation and is not necessarily precise. A trained learning model may be referred to as, for example, a trained model.

[0111] Hereinafter, for the sake of convenience, the learning model M1a and the learning model M1b will not be distinguished from each other, and will be referred to as the learning model M1.

[0112] Here, the output from the trained learning model M1 (learning model M1b in the example of FIG. 7B) may be called an inference result, which represents the result of inference by the learning model M1. Inference results from a machine learning learning model include, for example, classification results or regression results. In this embodiment, the inference result may be applied when it is a classification result, or may be applied when it is a regression result. Inference may also be referred to as, for example, prediction, estimation, or conjecture. Furthermore, in this embodiment, a case is shown in which a predetermined judgment is made based on an inference result, but for example, the inference result itself may be used as the result of the judgment, in which case the inference result essentially becomes the result of the judgment.

[0113] In this embodiment, the output (inference result) from the learning model M1 is the dependent variable, and other items that are thought to affect the dependent variable are the explanatory variables. Generally, the feature quantities of a defect include a response variable and an explanatory variable.

[0114] <Example in which the learning function and the judgment function are provided in separate devices> Here, the example of FIG. 3 illustrates a case where the management device 11 (an example of an information processing device) has both a learning function for performing machine learning learning and a determination function for making a determination based on the machine learning result. As another example, the learning function and the judgment function may be provided in separate devices.

[0115] <Learning device> FIG. 8 is a diagram illustrating an example of a learning device 311 and a determination device 312 according to a modified example of the embodiment. In the example of FIG. 8, the learning device 311 and the determination device 312 are configured as separate devices. The learning device 311 has a function to control the learning of a machine learning learning model, but does not have a function to control the making of judgments using a learned learning model. If the learning device 311 does not use teacher data, the learning device 311 does not need to store the teacher data.

[0116] The learning device 311 performs machine learning learning and provides the learning results (for example, a learned learning model) to the determination device 312. In addition, the transfer of learning results (e.g., a learned learning model) from the learning device 311 to the determination device 312 may be performed, for example, by communication between the learning device 311 and the determination device 312, or may be performed using a portable recording medium, etc. The communication between the learning device 311 and the determination device 312 may be performed via, for example, a server device or a relay device.

[0117] <Judgment device> The determination device 312 has a function to control the use of a trained learning model to make a determination, but does not have a function to control the training of a machine learning learning model. If the determination device 312 does not use teacher data, the determination device 312 does not need to store the teacher data. The determination device 312 obtains a predetermined determination result based on the learning result (for example, a learned learning model) provided by the learning device 311.

[0118] [Example of hardware configuration for each device] An example of the hardware configuration of each device is shown with reference to FIG. FIG. 9 is a diagram illustrating an example of the hardware configuration of an information processing apparatus 1001 according to the embodiment. The configuration shown in Fig. 9 may be applied to each device such as the management device 11 shown in Fig. 3 or the learning device 311 or the determination device 312 shown in Fig. 8. In this case, the functions of each device are realized by the functions of the information processing device 1001 shown in Fig. 9. For example, the configuration of the information processing device 1001 or a part of the configuration of the information processing device 1001 may be used in any part of each device, such as the management device 11 shown in Figure 3, or the learning device 311 or determination device 312 shown in Figure 8.

[0119] In the example of Figure 9, the information processing device 1001 includes a processor 1011, an operation device 1012, a display device 1013, a storage device 1014, a memory 1015, an input / output interface 1016, a network interface 1017, and a bus 1021 connecting these. When the information processing device 1001 is applied to any device, it does not necessarily have to have all the processing units shown in FIG. 9, and any processing unit not shown in FIG. 9 may be added.

[0120] The processor 1011 is composed of a CPU and the like, and executes a program to perform control and processing defined in the program. The operation device 1012 includes one or more input devices such as a keyboard and a mouse, and receives operations performed by an operator (person) or the like. The display device 1013 has a screen and displays and outputs information on the screen.

[0121] The storage device 1014 is a non-volatile storage unit, and is configured, for example, by a hard disk, and stores information. The memory 1015 is a volatile storage unit, and is configured with RAM (Random Access Memory) or the like, and temporarily stores information. As the RAM, for example, a DRAM (Dynamic Random Access Memory) may be used. The storage device 1014 or memory 1015 may store information for a program executed by the processor 1011, for example.

[0122] The input / output interface 1016 is an interface for connecting to an external recording medium or the like. The network interface 1017 is an interface for connecting to an external network.

[0123] Here, the information processing device 1001 may include one processor or two or more processors as the processor 1011. As an example, the information processing device 1001 may include multiple CPUs, each of which executes its own control and processing, and these multiple CPUs may work together to realize overall control and processing.

[0124] As described above, the management device 11 according to this embodiment can support manufacturing based on the correlation between manufacturing process information and multiple pieces of inspection result information in a manufacturing apparatus 21 having multiple inspection machines, thereby improving manufacturing efficiency. For example, the management device 11 according to this embodiment can predict defects that may occur in manufactured products according to the manufacturing process, and adjust the manufacturing process (for example, change parameters) based on the prediction results so as to reduce defects that may occur in the products. Note that such processing may be performed for each defect type, for example.

[0125] For example, in the management device 11 according to this embodiment, by determining the correlation of the manufacturing process for each defect type that occurs in manufactured items, the manufacturing process can be modified (e.g., parameters can be modified) based on the correlation to reduce defects that occur in the items.

[0126] The management device 11 according to this embodiment is a cloud-based device that has machine learning learning and inference functions, and can use these functions to support the operations of the factories A1 to A3. The management device 11 according to this embodiment can support operations by linking defect determination information obtained from images acquired by an inspection device (sensing device) with manufacturing process information.

[0127] For example, with conventional technology, when multiple sensing devices are used, the information obtained by each sensing device is stored only in a standalone manner, making it impossible to utilize big data for AI. Furthermore, for example, with conventional technology, while improvements in the functions and performance of process inspections are required on a daily basis, it is difficult to add functions and performance that are not included in the initial design, making it difficult to introduce new functions and performance.As a specific example, with conventional technology, it is difficult to design new manufacturing processes and new inspection machines while taking into consideration the coordination between pre- and post-inspection processes and the manufacturing process. Furthermore, for example, in conventional technology, the manufacturing process or inspection equipment may differ for each product type, which makes it complicated to link the inspections in the pre- and post-processes with the manufacturing process, making it difficult to analyze the correlation between the inspection results and operations.

[0128] In contrast, in this embodiment, it is possible to utilize AI that uses big data, for example, for analysis that links pre- and post-process inspections with the manufacturing process. Furthermore, in this embodiment, for example, it is possible to design in consideration of the link between pre- and post-process inspection and manufacturing processes, and it becomes easier to introduce new functions and new performance.

[0129] In the above, the case where the management device 11 according to the present embodiment is provided in the cloud has been described. However, as another example, a device having the same functions as the management device 11 according to the present embodiment may be provided on-premise. In this case, the device may be installed, for example, in a factory according to the present embodiment (e.g., factory A1, etc.). In this case, as an example, the functions of the management device 11 and the information organizing function of the external communication device 23 may be incorporated into the factory server 22 shown in FIG. 1, and the external communication function of the external communication device 23 may not be provided.

[0130] Here, in this embodiment, a case has been shown in which a glass substrate is used as the object to be inspected, but as another example, the object to be inspected may be a substrate other than glass, or may be an object other than a substrate. Furthermore, in this embodiment, the case where image information is acquired by each inspection machine is shown, but as another example, other information may be acquired by each inspection machine, for example, vibration information or sound (e.g., voice) information may be acquired. In this way, inspection by an inspection machine is not necessarily limited to optical inspection, and inspection that detects physical quantities other than light may also be used.

[0131] Furthermore, in this embodiment, multiple product types have been described together, but for example, if the required product quality differs for each product type, a configuration in which the manufacturing process, manufacturing process parameters, or learning model differs for each product type may be used.

[0132] <Configuration example> As an example of configuration, the information processing device has the following configuration. The information processing device includes an information processing unit. The information processing unit executes a process to acquire identification information of an item that is a finished product or intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device. The information processing unit also executes a process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information. The information processing unit generates at least one of a first learning model and a second learning model as the learning model. The first learning model receives information corresponding to the manufacturing process information as input and outputs prediction results of the first sensing information and the second sensing information. The second learning model receives information corresponding to the manufacturing process information as input and outputs a value representing the correlation between the first sensing information and the second sensing information.

[0133] Therefore, the information processing device can support manufacturing based on the correlation between manufacturing process information and multiple pieces of sensing information in a manufacturing device having multiple sensing devices. For example, the first learning model can input future manufacturing process information and predict future first sensing information and second sensing information. For example, in the second learning model, past manufacturing process information can be input and correlations between the past first sensing information and second sensing information can be characterized. The sensing information (first sensing information and second sensing information) may be, for example, information indicating the degree of occurrence of defects (for example, the number, etc.).

[0134] Here, in each of the first learning model and the second learning model, the input to the learning model is, for example, manufacturing process information of at least one process machine, and may be manufacturing process information of a plurality of process machines. Furthermore, as an input to the first learning model, sensing information obtained in a process prior to the location to be predicted (for example, the location of the sensing device) may be used in addition to the manufacturing process information.

[0135] Furthermore, the output from the first learning model may be, for example, first sensing information and second sensing information. Furthermore, the output from the second learning model may be, for example, a value representing a correlation between one or more pieces of manufacturing process information and the first sensing information and a value representing a correlation between one or more pieces of manufacturing process information and the second sensing information. These correlation-representing values ​​may be called, for example, a correlation degree, an influence degree, an importance degree, a correlation value, or a correlation coefficient. The first learning model and the second learning model may each be generated by performing machine learning using information including, for example, manufacturing process information, first sensing information, and second sensing information.

[0136] Here, the information processing device may be configured as, for example, a cloud device. In this case, the identification information, the manufacturing process information, the first sensing information, and the second sensing information may be uploaded from the manufacturing apparatus to the information processing apparatus. In this way, in a configuration in which an information processing device is installed in the cloud, it is easy to add or update the functions and performance of the information processing device, for example, and this reduces the need for work to be limited by the specifications of the computer used as the information processing device.

[0137] Furthermore, in a manufacturing device or an information processing device, the identification information, the manufacturing process information, the first sensing information, and the second sensing information may be linked (associated) with one another. Here, when the information processing device is installed in the cloud and such linking is performed, for example, the identification information, manufacturing process information, first sensing information, and second sensing information may be linked after being uploaded to the cloud, or the identification information, manufacturing process information, first sensing information, and second sensing information may be linked and then uploaded to the cloud in a linked state.

[0138] Furthermore, the first sensing device and the second sensing device may be arranged in parallel or in series in the direction of travel of the manufacturing line of the manufacturing device. Furthermore, the item may be, for example, a product (e.g., a product in its final shipping form), or an intermediate product before reaching such a product, or may be a product that has been reached via an intermediate product on a manufacturing line.

[0139] Furthermore, the sensing information (first sensing information and second sensing information) may be, for example, primary data or secondary data. As a specific example, the primary data may be data resulting from determining the surface flatness from the sensing results of the sensing devices (first sensing device and second sensing device). As a specific example, the secondary data may be data resulting from determining whether an item is good or bad based on information obtained from the sensing devices (first sensing device and second sensing device), or data resulting from classifying (determining) the type of defect.

[0140] The manufacturing apparatus 21 according to this embodiment is an example of a manufacturing apparatus. Moreover, the management device 11 according to the present embodiment is an example of an information processing device. The information processing unit 216 according to this embodiment is an example of an information processing unit. Furthermore, two different inspection machines among the multiple inspection machines (first inspection machine C1, 2-1 inspection machine C2-1 to 2-3 inspection machine C2-3, and third inspection machine C3) according to this embodiment are examples of the first sensing device and the second sensing device. Moreover, information on the manufacturing process relating to one or more of the plurality of process machines (first process machine B1 to third process machine B3) according to this embodiment is an example of manufacturing process information. Moreover, the first learning model E1 according to this embodiment is an example of a first learning model. Moreover, the second learning model E2 according to this embodiment is an example of a second learning model.

[0141] In this embodiment, for example, the inspection image information may be an example of sensing information (first sensing information or second sensing information). In this embodiment, for example, one or both of the inspection information and the inspection light amount information may be further included in the sensing information.

[0142] Furthermore, the manufacturing equipment may include, for example, three or more sensing devices. Furthermore, the manufacturing equipment may be configured to perform two or more manufacturing processes. Furthermore, the information representing the correlation may be multi-valued to represent the level of correlation, or may be binary to represent the presence or absence of correlation.

[0143] As one configuration example, in the information processing device, the first sensing device and the second sensing device perform sensing in a batch system. Therefore, since the information processing device is a batch-type sensing device, it is easy to secure time to perform calculations in real time, for example, when manufacturing is performed by a manufacturing device. In particular, a batch-type sensing device has the advantage of being able to accumulate information such as sensing results, but the information from each sensing device tends to become standalone. For this reason, the technology according to this embodiment is highly effective when applied to a batch-type sensing device. Note that this configuration does not necessarily have to be used, and for example, non-real-time processing may be performed by an information processing device. The first and second sensing devices may also perform sensing in a line system that continuously inspects articles traveling on a line.

[0144] In one configuration example, in the information processing device, the first sensing device and the second sensing device acquire image information of an article. The first sensing information and the second sensing information include at least information on the defect type determined based on the image information. Therefore, in the information processing device, for example, the first learning model makes it possible to make a prediction for each defect type, and the second learning model makes it possible to find a correlation between the manufacturing process and each defect type. Here, the defect type may be determined, for example, by AI (e.g., a machine learning learning model) or may be determined by a rule-based method using preset rules. Such an AI function or rule-based function may be provided in a cloud device, or may be additionally provided in the manufacturing device. It should be noted that this configuration does not necessarily have to be used, and for example, sensing information other than the defect type may be used.

[0145] In one configuration example, in the information processing device, the image information includes information on setting values ​​of one or both of the first sensing device and the second sensing device. Therefore, in the information processing device, for example, the image information and the setting value are linked, and thus the information on the setting value can be used for analysis. As an example, the information processing device can use the information on the setting value when determining the defect type of a defect shown in an image. Here, for example, it is a preferable example that the setting values ​​of both the first sensing device and the second sensing device are used, but a configuration may be adopted in which any one of the setting values ​​is used. Furthermore, as the set value, for example, the amount of inspection light may be used, or another set value may be used. Note that this configuration does not necessarily have to be used, and for example, a configuration in which the set value is not used may be used.

[0146] As one configuration example, in an information processing apparatus, an information processing section monitors and stores information on defect types over time. Therefore, the information processing device can grasp the correlation of manufacturing process information with respect to the temporal trend (time-series trend) of the defect type. As an example, the information processing device can grasp the chronological change of the correlation ranking of defects for each defect type with respect to the manufacturing process. It should be noted that this configuration does not necessarily have to be used, and for example, time-series correlation ranking does not necessarily have to be used.

[0147] As one configuration example, the information processing device outputs a control target value to manufacturing equipment downstream of a specified location in the manufacturing equipment based on a prediction result output from a first learning model regarding the specified location in the manufacturing equipment. Therefore, the information processing device can control the manufacturing process by, for example, feedforward.

[0148] 2A, the information processing device may output a control target value to a third process machine B3 located downstream of the second process machine B2 based on a prediction result for the second process machine B2 (a prediction result for one or more of the 2-1 inspection machine C2-1 to the 2-3 inspection machine C2-3). Note that the prediction result may include a prediction result for a first process machine B1 located upstream of the second process machine B2 (a prediction result based on the inspection result of the first inspection machine C1). In this example, the locations of the inspection machines (2-1 inspection machine C2-1 to 2-3 inspection machine C2-3) provided for the second process machine B2 are an example of a predetermined location, and the third process machine B3 is an example of a manufacturing facility downstream of the predetermined location. Also, in this example, a target value (e.g., a target value of a predetermined parameter) related to the manufacturing process in the manufacturing facility (in this example, the third process machine B3) is an example of a control target value. It should be noted that this configuration does not necessarily have to be used, and for example, feedforward control does not necessarily have to be performed.

[0149] In one configuration example, the information processing device outputs a control target value to manufacturing equipment upstream of a specified location in the manufacturing equipment based on a prediction result output from a first learning model regarding the specified location in the manufacturing equipment. Therefore, the information processing device can control the manufacturing process, for example, by feedback.

[0150] 2A, the information processing device may output a control target value to the first process machine B1 located upstream of the second process machine B2 based on a prediction result for the second process machine B2 (a prediction result for one or more of the 2-1 inspection machine C2-1 to the 2-3 inspection machine C2-3). Note that the prediction result may include a prediction result for the third process machine B3 located downstream of the second process machine B2 (a prediction result based on the inspection result of the third inspection machine C3). In this example, the location of the inspection machine (the corresponding one of the 2-1 inspection machine C2-1 to the 2-3 inspection machine C2-3) provided for the second process machine B2 is an example of a predetermined location, and the first process machine B1 is an example of a manufacturing facility upstream of the predetermined location. Also, in this example, a target value (for example, a target value of a predetermined parameter) related to the manufacturing process in the manufacturing facility (in this example, the first process machine B1) is an example of a control target value. It should be noted that this configuration does not necessarily have to be used, and for example, feedback control does not have to be performed.

[0151] As one configuration example, in the information processing device, the information processing unit determines a correlation ranking of the manufacturing process information based on the degree of influence on the first sensing information and the second sensing information. Therefore, the information processing device can grasp, for example, the degree to which manufacturing process information affects each piece of sensing information (influence level), as a ranking (correlation ranking). As an example, the information processing device can grasp, for example, the degree to which each manufacturing process affects each piece of sensing information (e.g., defects of each defect type), as a ranking (correlation ranking). Such correlation ranking makes it possible, for example, to easily find parameters to be improved, for example, when improving the manufacturing process, either manually or automatically by a device. It should be noted that this configuration does not necessarily have to be used, and for example, correlation ranking does not necessarily have to be used.

[0152] As one configuration example, in the information processing device, the first learning model and the second learning model are generated for each product type. Therefore, the information processing device can generate an appropriate learning model according to the product quality required for each product type, for example. Here, depending on the required quality, for example, the number of defects that can be tolerated for each defect type or the range of defect sizes (tolerance range) may differ, which may result in differences in the content of the manufacturing process, the inspection machine, the inspection conditions, etc. Therefore, depending on the required quality, the correlation between the manufacturing process and the defect occurrence status may differ, and therefore the appropriate learning model (for example, appropriate values ​​of the parameters in the learning model) may differ. Note that this configuration does not necessarily have to be used; for example, a common learning model may be used regardless of the required quality.

[0153] In one configuration example, in the information processing device, a first sensing device senses a first sensing result for determining whether an article passes a first process. The second sensing device senses the second sensing result to determine whether the second process of the article is satisfactory. Therefore, the information processing device can support manufacturing based on the correlation between a plurality of pieces of manufacturing process information and a plurality of pieces of sensing information in a manufacturing device having a plurality of processes. 2A, the process of the first process machine B1 is an example of a first process, the first inspection machine C1 is an example of a first sensing device, the second process machine B2 is an example of a second process, and the second inspection machine (one or more of the 2-1 inspection machine C2-1 to the 2-3 inspection machine C2-3) is an example of a second sensing device. Also, while the first process machine B1 and the second process machine B2 are illustrated as being in an upstream-downstream relationship, other examples may be applied to the first process machine B1 and the third process machine B3 being in an upstream-downstream relationship, or the second process machine B2 and the third process machine B3 being in an upstream-downstream relationship. It should be noted that this configuration does not necessarily have to be used, and for example, the technology according to this embodiment may be applied to locations that are not in an upstream-downstream relationship.

[0154] In one configuration example, in the information processing device, the first sensing device senses a first sensing result for determining whether the third process of the article is good or bad. The second sensing device also senses the second sensing result to determine whether the article passed the third process. Therefore, the information processing device can support manufacturing based on the correlation between manufacturing process information and multiple pieces of sensing information in a manufacturing device in which multiple sensing devices exist for the same process. 2A, the process of the second process machine B2 is an example of a third process, the 2-1 inspection machine C2-1 is an example of a first sensing device, and the 2-2 inspection machine C2-2 is an example of a second sensing device. Also, although the combination of the 2-1 inspection machine C2-1 and the 2-2 inspection machine C2-2 among the 2-1 inspection machine C2-1 to the 2-3 inspection machine C2-3 is illustrated here, other examples may be applied to the combination of the 2-1 inspection machine C2-1 and the 2-3 inspection machine C2-3, or the combination of the 2-2 inspection machine C2-2 and the 2-3 inspection machine C2-3. Furthermore, while the example shown here is one in which multiple inspection machines (2-1 inspection machine C2-1 to 2-3 inspection machine C2-3) are arranged in parallel for the same process, as in the example of Figure 2A, another example may be one in which multiple inspection machines (2nd inspection machine C2a-1 to 4th inspection machine C2a-3) are arranged in series for the same process, as in the example of Figure 2B. It should be noted that this configuration does not necessarily have to be used, and for example, the technology of this embodiment may be applied to locations that are not related to the same process.

[0155] For example, a processing method (information processing method) performed by an information processing device may be provided. For example, a computer program for realizing the processing performed by the information processing device may be provided.

[0156] For example, a system (manufacturing management system) having a manufacturing device and an information processing device may be provided. In the example of FIG. 1, the manufacturing management system 1 is an example of a manufacturing management system. In the example of FIG. 1, the manufacturing management system 1 manages manufacturing equipment in a plurality of factories A1 to A3, but the manufacturing management system 1 may be configured to manage one manufacturing equipment.

[0157] A program for implementing the functions of any of the components of any of the above-described devices may be recorded on a computer-readable recording medium and then loaded into a computer system for execution. The term "computer system" as used herein includes hardware such as an operating system or peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and compact discs (CDs) and read-only memories (ROMs), as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory within a computer system that acts as a server or client when a program is transmitted over a network such as the Internet or a communication line such as a telephone line. Such volatile memory may be, for example, RAM. The recording medium may also be, for example, a non-transitory recording medium.

[0158] The above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network such as the Internet or a communication line such as a telephone line. The above program may also be one that realizes part of the above-mentioned functions. Furthermore, the above program may be a so-called differential file that can realize the above-mentioned functions in combination with a program already recorded in a computer system. A differential file may also be called a differential program.

[0159] Furthermore, the functions of any of the components in any of the above-described devices may be implemented by a processor. For example, each process in the embodiments may be implemented by a processor operating based on information such as a program and a computer-readable recording medium storing information such as the program. Here, the functions of each unit of the processor may be implemented by, for example, individual hardware, or may be implemented by integrated hardware. For example, the processor may include hardware, and the hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor may be configured using one or more circuit devices mounted on a circuit board, or one or both of one or more circuit elements. An integrated circuit (IC) or the like may be used as the circuit device, and a resistor or a capacitor may be used as the circuit element.

[0160] Here, the processor may be, for example, a CPU. However, the processor is not limited to a CPU, and various types of processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) may be used. The processor may also be, for example, a hardware circuit such as an ASIC (Application Specific Integrated Circuit). The processor may also be, for example, composed of multiple CPUs, or may be, for example, composed of a hardware circuit such as a multiple ASIC. The processor may also be, for example, composed of a combination of multiple CPUs and a hardware circuit such as a multiple ASIC. The processor may also include, for example, one or more of an amplifier circuit or a filter circuit that processes analog signals.

[0161] The embodiments of this disclosure have been described in detail above with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs within the scope that do not deviate from the gist of this disclosure.

[0162] [Note] (Configuration example 1) to (Configuration example 14) are shown.

[0163] (Configuration example 1) An information processing device, An information processing unit is provided, The information processing unit A process of acquiring identification information of an item that is a finished product or an intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; A process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; Run The information processing unit As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; Generate at least one of Information processing device.

[0164] (Configuration example 2) the first sensing device and the second sensing device perform sensing in a batch manner; The information processing device described in (Configuration Example 1).

[0165] (Configuration example 3) the first sensing device and the second sensing device acquire image information of the item; the first sensing information and the second sensing information include at least information on the defect type determined based on the image information; (Configuration Example 1) or the information processing device described in (Configuration Example 1).

[0166] (Configuration example 4) the image information includes information on setting values ​​of one or both of the first sensing device and the second sensing device; The information processing device according to (Configuration Example 3).

[0167] (Configuration Example 5) The information processing unit monitoring and storing information on the defect types over time; The information processing device according to (Configuration Example 3) or (Configuration Example 4).

[0168] (Configuration Example 6) the information processing unit outputs a control target value to a manufacturing facility downstream of the predetermined location in the manufacturing equipment based on the prediction result output from the first learning model regarding the predetermined location in the manufacturing equipment. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 5).

[0169] (Configuration Example 7) the information processing unit outputs a control target value to a manufacturing facility upstream of the predetermined location in the manufacturing equipment based on the prediction result output from the first learning model regarding the predetermined location in the manufacturing equipment. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 6).

[0170] (Configuration Example 8) The information processing unit determining a correlation ranking of the manufacturing process information based on the influence of the first sensing information and the second sensing information; The information processing device according to any one of (Configuration Example 1) to (Configuration Example 7).

[0171] (Configuration Example 9) the first learning model and the second learning model are generated for each product type; The information processing device according to any one of (Configuration Example 1) to (Configuration Example 8).

[0172] (Configuration Example 10) the first sensing device senses the first sensing result for determining whether the first process of the article is good or bad; The second sensing device senses the second sensing result to determine whether the second process of the article is good or bad. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 10).

[0173] (Configuration Example 11) the first sensing device senses the first sensing result to determine whether the third process of the article is good or bad; The second sensing device also senses the second sensing result to determine whether the article passed the third process. The information processing device according to any one of (Configuration Example 1) to (Configuration Example 10).

[0174] It is also possible to provide an information processing method performed in an information processing device. (Configuration Example 12) an information processing unit of the information processing device, A process of acquiring identification information of an item that is a finished product or an intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; A process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; Run The information processing unit As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; Generate at least one of Information processing methods.

[0175] It is also possible to provide a program (computer program) that is executed by a computer that constitutes an information processing device. (Configuration Example 13) On the computer, a function of acquiring identification information of an item that is a finished product or intermediate product manufactured by a manufacturing apparatus having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; a function of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; A program for realizing the above, As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; Generate at least one of program.

[0176] It is also possible to provide a manufacturing management system that includes a manufacturing device and an information processing device. (Configuration Example 14) A manufacturing management system having an information processing device, The information processing device includes: An information processing unit is provided, The information processing unit A process of acquiring identification information of an item that is a finished product or an intermediate product manufactured by a manufacturing device having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; A process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; Run The information processing unit As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that inputs information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; Generate at least one of Manufacturing control system. [Explanation of symbols]

[0177] 1... manufacturing management system, 11... management device (information processing device), 21, 21a... manufacturing device, 22... factory server, 23... external communication device, 31... network, 111, 211... input unit, 112, 212... output unit, 213... communication unit, 214... memory unit, 215... control unit, 216... information processing unit, 231... operation unit, 232... display unit, 251... information acquisition processing unit, 252... analysis processing unit, 253... visualization processing unit, 271... image recognition processing unit, 272... feature processing unit, 311... learning device, 312... judgment device, 1001... information processing device, 1011... processor, 1012... operation operation equipment, 1013...display device, 1014...storage device, 1015...memory, 1016...input / output interface, 1017...network interface, 1021...bus, A1 to A3...factory, B1...first process machine, B2...second process machine, B3...third process machine, C1...first inspection machine, C2-1...second-first inspection machine, C2-2...second-second inspection machine, C2-3...second-third inspection machine, C2a-1...second inspection machine, C2a-2...third inspection machine, C2a-3...fourth inspection machine, C3...third inspection machine, D1...management information, E1...first learning model, E2...second learning model, F1, F 1a...Manufacturing process information, F2, F2a...Inspection image information, F3, F3a...Inspection information, F4, F4a...Inspection light intensity information, F11...Product type information, F12...Inspection machine information, F13...Process information, G1...First defect judgment information, G2-1...Second defect judgment information, G2-2...Second defect judgment information, G2-3...Second defect judgment information, G3...Third defect judgment information, H1...Prediction information, H2...Importance information, I1...Learning data, M1a...Learning model currently being trained, M1b...Learning model that has been trained, T11...Analysis processing, T12...Visualization processing, T101...Collection processing, T201~T2 03, T301...Outline configuration, T211, T311...Network, T212, T312...Cloud storage, T213, T313...Cloud computing resources, T214, T232...Decision model, T215, T314...Decision result, T231...Image processing function, T315...Prediction model, T316...Target value, V1, V4, V102...Storage processing, V2...Image recognition processing, V3...Feature processing, V11...Prediction result display processing, V12...Importance display processing, V21...Learning data selection processing, V22...Image recognition learning processing, V31...Unit-by-unit linking and aggregation processing, V32...Defect-specific regression processing,V33...correlation analysis processing, V101...acquisition processing, V103...organization processing, W...image discrimination learning model group, W1...first image discrimination learning model, W2-1...second-first image discrimination learning model, W2-2...second-second image discrimination learning model, W2-3...second-third image discrimination learning model, W3...third image discrimination learning model,

Claims

1. An information processing device, An information processing unit is provided, The information processing unit A process of acquiring identification information of an item that is a finished product or an intermediate product manufactured by a manufacturing apparatus having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; A process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; Run The information processing unit As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that receives information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; generating at least one of Information processing device.

2. the first sensing device and the second sensing device perform sensing in a batch manner; The information processing device according to claim 1 .

3. the first sensing device and the second sensing device acquire image information of the item; the first sensing information and the second sensing information include at least information on the defect type determined based on the image information; 3. The information processing device according to claim 1.

4. the image information includes information on setting values ​​of one or both of the first sensing device and the second sensing device; The information processing device according to claim 3 .

5. The information processing unit monitoring and storing information on the defect types over time; The information processing device according to claim 3 .

6. the information processing unit outputs a control target value to a manufacturing facility downstream of the predetermined location in the manufacturing apparatus based on the prediction result output from the first learning model regarding the predetermined location in the manufacturing apparatus.

3. The information processing device according to claim 1.

7. the information processing unit outputs a control target value to a manufacturing facility upstream of the predetermined location in the manufacturing equipment based on the prediction result output from the first learning model regarding the predetermined location in the manufacturing equipment.

3. The information processing device according to claim 1.

8. The information processing unit determining a correlation ranking of the manufacturing process information based on the influence of the first sensing information and the second sensing information; 3. The information processing device according to claim 1.

9. the first learning model and the second learning model are generated for each product type; 3. The information processing device according to claim 1.

10. the first sensing device senses the first sensing result for determining whether the first process of the article is good or bad; the second sensing device senses the second sensing result to determine whether the second process of the article is good or bad; 3. The information processing device according to claim 1.

11. the first sensing device senses the first sensing result to determine whether the third process of the article is good or bad; The second sensing device also senses the second sensing result to determine whether the article has passed the third process.

3. The information processing device according to claim 1.

12. an information processing unit of the information processing device, A process of acquiring identification information of an item that is a finished product or an intermediate product manufactured by a manufacturing apparatus having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; A process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; Run The information processing unit As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that receives information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; generating at least one of Information processing methods.

13. On the computer, a function of acquiring identification information of an item that is a finished product or intermediate product manufactured by a manufacturing apparatus having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; a function of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; A program for realizing the above, As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that receives information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; generating at least one of program.

14. A manufacturing management system having an information processing device, The information processing device includes: An information processing unit is provided, The information processing unit A process of acquiring identification information of an item that is a finished product or an intermediate product manufactured by a manufacturing apparatus having a first sensing device and a second sensing device, manufacturing process information of the item, first sensing information obtained from a first sensing result of the item by the first sensing device, and second sensing information obtained from a second sensing result of the item by the second sensing device; A process of generating a machine learning trained learning model based on the manufacturing process information, the first sensing information, and the second sensing information; Run The information processing unit As the learning model, a first learning model that receives information corresponding to the manufacturing process information and outputs prediction results of the first sensing information and the second sensing information; a second learning model that receives information corresponding to the manufacturing process information and outputs a value representing a correlation between the first sensing information and the second sensing information; generating at least one of Manufacturing control system.

Images