Inspection Device and Inspection Method

Abstract

The technology provided by the present invention improves the inspection accuracy of the product processed in the second manufacturing process after the first manufacturing process in the inspection device. An inspection device includes: a first acquisition unit that acquires the first defect probability calculated using the first learning model for the data related to the inspection object processed in the first manufacturing process; a second acquisition unit that acquires the second defect probability calculated using the second learning model for the data related to the inspection object processed in the second manufacturing process after the first manufacturing process; a determination unit that, when the acquired second defect probability is equal to or greater than the second threshold, determines that there is a defect in the inspection object, and when the acquired first defect probability is greater than the predetermined first threshold, changes at least one of the second learning model and the second threshold to a condition that improves the inspection accuracy compared to the case where the first defect probability is equal to or less than the first threshold.

Description

Technical Field

[0001] The present invention relates to an inspection device and an inspection method. Background Art

[0002] A machining system is known that includes: a machining unit including a machining machine that performs machining of a workpiece and an inspection device that inspects the machined workpiece, and a machine learning device connected to the machining unit via a network (for example, Patent Document 1). In this machining system, the determination result of the defect rate of the workpiece based on the inspection device is sent to the machine learning device as inspection result data. The machine learning device uses the inspection result data for learning and changes the machining conditions to minimize the defect rate of the workpiece.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2019-025561. Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] An improvement in the inspection accuracy based on the inspection device is sought.

[0008] Means for Solving the Problems

[0009] The present invention can be implemented in the following manner.

[0010] (1) According to one aspect of the present invention, an inspection device is provided. The inspection device includes: a first acquisition unit that acquires a first defect probability calculated using a first learning model for data related to an inspection object processed in a first manufacturing process; a second acquisition unit that acquires a second defect probability calculated using a second learning model for data related to the inspection object processed in a second manufacturing process after the first manufacturing process; and a determination unit that determines that a defect exists in the inspection object when the acquired second defect probability is equal to or greater than a second threshold, and changes at least one of the second learning model and the second threshold to a condition that improves the inspection accuracy compared to the case where the first defect probability is less than or equal to a first threshold.

[0011] According to the inspection device of this aspect, when the first defect probability is greater than the first threshold, the condition is changed to improve the inspection accuracy. Therefore, the inspection accuracy of the product after processing in the second manufacturing process can be improved.

[0012] (2) In the inspection device of the above-described method, it may also be that, when the obtained first defect probability is greater than the first threshold value, the determination unit does not change the second learning model but changes the second threshold value to a second value that is smaller than a first value used when the first defect probability is equal to or less than the first threshold value. It may also be that, when the obtained first defect probability is equal to or less than the first threshold value, the determination unit does not change the second learning model and the second threshold value.

[0013] According to the inspection device of this method, by making the defect determination condition in the determination unit strict, the inspection accuracy of the second inspection device can be improved.

[0014] (3) In the inspection device of the above-described method, it may also be that, when the obtained first defect probability is greater than the first threshold value, the determination unit does not change the second threshold value but changes the second learning model to a second model that is different from a first model used when the first defect probability is equal to or less than the first threshold value. It may also be that, when the obtained first defect probability is equal to or less than the first threshold value, the determination unit does not change the second learning model and the second threshold value.

[0015] According to the inspection device of this method, by changing the learning model used as the inspection condition to a learning model with high inspection accuracy, the calculation accuracy of the second defect probability can be improved, and the inspection accuracy based on the second inspection device can be improved.

[0016] (4) In the inspection device of the above-described method, it may also be that, when the obtained first defect probability is greater than the first threshold value, after the determination unit determines the presence or absence of a defect in the inspection object using the second defect probability, the determination unit further changes the second learning model to a second model that is different from a first model used when the first defect probability is equal to or less than the first threshold value. The second acquisition unit further acquires the second defect probability calculated using the second learning model changed to the second model. The determination unit further uses the second defect probability to determine the presence or absence of a defect in the inspection object. The second defect probability is calculated using the second learning model changed to the second model. It may also be that, when the obtained first defect probability is equal to or less than the first threshold value, the determination unit does not change the second learning model and the second threshold value.

[0017] According to the inspection device of this method, two inspections are performed, and by changing the learning model used as the inspection condition to a learning model with high inspection accuracy, the calculation accuracy of the second defect probability can be improved, and the inspection accuracy based on the second inspection device can be improved.

[0018] (5) In the inspection device of the above-described manner, it is also possible that the first model is a CNN and the second model is an R-CNN.

[0019] According to the inspection device of this manner, by using a simple method of a known learning model, it is possible to improve the inspection accuracy based on the second inspection device.

[0020] The present invention can also be implemented in various forms other than the inspection device. For example, it can be implemented in the form of an inspection method, a manufacturing method of an inspection device, a control method of an inspection device, a computer program for implementing the control method, a non-transitory recording medium recording the computer program, and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is an explanatory diagram showing an inspection system including the inspection device of the present embodiment.

[0022] Figure 2 It is a block diagram showing the internal functional structure of the first inspection device.

[0023] Figure 3 It is a block diagram showing the internal functional structure of the second inspection device.

[0024] Figure 4 It is a process diagram showing each process of the production line.

[0025] Figure 5 It is a process diagram showing the details of the first inspection process based on the first inspection device.

[0026] Figure 6 It is a process diagram showing the details of the second inspection process based on the second inspection device.

[0027] Figure 7 It is a block diagram showing the internal functional structure of the second inspection device as the second embodiment.

[0028] Figure 8 It is a process diagram showing the details of the second inspection process based on the second inspection device of the second embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] A. First Embodiment:

[0030] Figure 1FIG. 0 is an explanatory diagram showing an inspection system 100 including the inspection apparatus of the present embodiment. The inspection system 100 inspects whether there are defects in an inspection target processed in a process included in a production line LP. The inspection target refers to a product flowing in the production line LP. In the present invention, "processing" means performing a specific operation on a product, including an action that affects the properties of the product. For example, when an action other than product processing such as product conveyance affects the properties of the product, it may be included in "processing".

[0031] In the present embodiment, the inspection system 100 includes two inspection apparatuses using machine learning. Specifically, the inspection system 100 includes a first inspection apparatus 60 and a second inspection apparatus 70 as the inspection apparatus of the present embodiment. The first inspection apparatus 60 inspects a product after being processed based on a first manufacturing process, and the second inspection apparatus 70 inspects a product after being processed based on a second manufacturing process. In the present embodiment, the second inspection apparatus 70 obtains the inspection result based on the first inspection apparatus 60 and changes the defect judgment condition according to the obtained inspection result.

[0032] The production line LP includes, for example, a first manufacturing apparatus 31, a second manufacturing apparatus 32, a first conveying apparatus 41, a second conveying apparatus 42, a first correction process 51, and a second correction process 52. In Figure 1 this example, in the production line LP, the product is processed in the order of the first manufacturing apparatus 31, the first conveying apparatus 41, the second manufacturing apparatus 32, and the second conveying apparatus 42.

[0033] The first manufacturing device 31 is a device that performs processing as a first manufacturing process on a product. The first manufacturing device 31 is, for example, a die-casting machine that forms vehicle parts such as cylinder heads and engine blocks. The second manufacturing device 32 is a device that performs processing as a second manufacturing process that is a subsequent process compared to the first manufacturing process on the product. The second manufacturing device 32 is, for example, a device that performs post-processing and post-treatment on the cast product. As a specific example of the second manufacturing device 32, it is a device that performs machining, such as a numerically-controlled lathe (NC lathe) or a machining center. The second manufacturing device 32 performs cutting, drilling, and tapping on the molded product manufactured by the first manufacturing device 31. The second manufacturing device 32 is not limited to machining that includes cutting and drilling, and can also be various devices that perform post-processing and post-treatment after molding. For example, the second manufacturing device 32 can be various devices that perform processing such as casting finishing that includes removing casting burrs and barrel polishing, finishing that includes shot peening and polishing, joining processing that includes thread fastening and welding, surface treatment that includes painting and electroplating, heat treatment, and impregnation treatment. The first manufacturing process and the second manufacturing process are not limited to processing performed by devices, and can also include processing of manual operations performed by humans.

[0034] In this embodiment, the second manufacturing process is a process associated with the first manufacturing process. "Associated with the first manufacturing process" means that the change in the properties of the product after processing based on the first manufacturing process can affect the properties of the product after processing based on the second manufacturing process, and the relationship between the first manufacturing process and the second manufacturing process is independent of each other. As a specific example of the case where the second manufacturing process is associated with the first manufacturing process, there can be cited the case where the processing content of the second manufacturing process is the same as the processing content of the first manufacturing process, the position of the product processed in the second manufacturing process is the same as the position processed in the first manufacturing process, and the processing of the second manufacturing process overlaps with the processing of the first manufacturing process. In these cases, for example, if the properties of the product after processing based on the first manufacturing process change, even if it is judged as a good product by the inspection device after processing based on the first manufacturing process, it is possible that it is affected by the change in the properties of the product based on the first manufacturing process, and the product after processing based on the second manufacturing process has potential abnormalities. Therefore, when it is expected that an abnormality has occurred in the properties of a product after processing based on the first manufacturing process, if the inspection conditions and the judgment conditions of defects are not changed in any way during the inspection after processing based on the second manufacturing process, the potential abnormality may not be detected. For example, when an inspection device using machine learning is used to inspect a product after processing based on the second manufacturing process, it is necessary to make the inspection device fully learn the potential abnormality. However, it is difficult to screen products with potential abnormalities and it is difficult to make the inspection device fully perform machine learning. In the present invention, the improvement of inspection accuracy includes the improvement of the calculation accuracy of the defect probability calculated using data related to the inspection object and the improvement of the detection accuracy of potential abnormalities.

[0035] The first conveyor device 41 and the second conveyor device 42 convey products, components, workpieces, etc. that are the objects of processing by the first manufacturing device 31 and the second manufacturing device 32. As the first conveyor device 41 and the second conveyor device 42, for example, it may include a belt conveyor, a conveyor that moves on a track, etc. The first conveyor device 41 conveys the product after being processed by the first manufacturing device 31. In the present embodiment, the initial setting of the conveying path of the first conveyor device 41 is the path from the first manufacturing process to the second manufacturing process. The first conveyor device 41 switches the conveying path of the product from the first manufacturing process to the first correction process 51 according to the inspection result of the first inspection device 60 indicating whether there is a defect in the product after being processed by the first manufacturing process. The second conveyor device 42 conveys the product after being processed by the second manufacturing device 32. In the present embodiment, the initial setting of the conveying path of the second conveyor device 42 is the path from the second manufacturing process to the subsequent process. For this, the second conveyor device 42 switches the conveying path of the product to the path from the second manufacturing process to the second correction process 52 according to the inspection result of the second inspection device 70 indicating whether there is a defect in the product after being processed by the second manufacturing process.

[0036] The first correction process 51 is implemented when it is determined that there is a defect in the product after being processed by the first manufacturing device 31. The first correction process 51 removes the defect from the product after being processed by the first manufacturing process and sends it to the second manufacturing device 32 as a normal product. The second correction process 52 is implemented when it is determined that there is a defect in the product after being processed by the second manufacturing device 32. The second correction process 52 removes the defect from the product after being processed by the second manufacturing process and sends it to the subsequent process as a normal product. The removal of the defect based on the first correction process 51 and the second correction process 52 can be performed by a dedicated device or by manual work by an operator.

[0037] Figure 2 It is a block diagram showing the internal functional structure of the first inspection device 60. The first inspection device 60 uses the image data of the inspection object to judge the presence or absence of a defect in the inspection object. The first inspection device 60 has a central processing unit (CPU) 62, a storage device 64, a first inspection data acquisition unit 66, and a first communication unit 68. The CPU 62, the storage device 64, the first inspection data acquisition unit 66, and the first communication unit 68 are connected to each other via a bus 61 and can communicate bidirectionally. The CPU 62 functions as a first inspection unit 624 and a first judgment unit 626 by executing various programs stored in the storage device 64.

[0038] The storage device 64 is, for example, a RAM, a ROM, or a hard disk drive (HDD). In the HDD (or ROM), various programs for implementing the functions provided in this embodiment are stored, and the various programs read from the HDD are loaded onto the RAM and executed by the CPU 62. The storage device 64 stores a first reference value 642 for determining whether there are defects in the inspection target and a first learning model 644. In addition, in the storage device 64, the first defect probability calculated by the first inspection unit 624 is temporarily stored.

[0039] The first inspection data acquisition unit 66 acquires data related to the inspection target for inspection. In the present embodiment, the first inspection data acquisition unit 66 is a camera for photographing the product after being processed by the first manufacturing device 31. The first inspection data acquisition unit 66 is provided on the conveyance path for conveying the inspection target after being processed by the first manufacturing device 31, and photographs the inspection target being conveyed to acquire an image of the inspection target. In the present embodiment, the image acquired by the first inspection data acquisition unit 66 is composed of an RGB input image signal, and the RGB input image signal is composed of respective image signal components represented by R (red), G (green), and B (blue). The input image signal may be, for example, a YUV image signal composed of Y (luminance signal), U (first color difference signal), and V (second color difference signal), or a YCbCr image signal or a YPbPr image signal. Regarding the image, in addition to a color image, it may also be a grayscale image having luminance values with an arbitrary color depth such as 1 bit or 8 bits.

[0040] The first inspection unit 624 uses machine learning to calculate the defect probability of the inspection object. The defect probability refers to the probability that the inspection object has a defect. Machine learning can use any type of machine learning such as supervised learning, unsupervised learning, reinforcement learning, etc. In this embodiment, the first inspection unit 624 uses statistical machine learning and uses the first learning model 644 to inspect the inspection object. The first learning model 644 is an identification model in a neural network. The first learning model 644 is a convolutional neural network (CNN: Convolutional neural network). The first learning model 644 is not limited to CNN, and identification models and generation models in various neural networks such as generative adversarial network (GAN: Generative Adversarial Network), variational autoencoder (VAE: Variational Autoencoder), and autoregressive generative network can also be used. The first learning model 644 uses the images of normal products and defective products after being processed by the first manufacturing device 31 to complete learning in advance. In this embodiment, the first inspection unit 624 inputs the data of the inspection object obtained by the first inspection data acquisition unit 66 into the first learning model 644, and calculates the first defect probability by using, for example, the sigmoid function or the softmax function. The first defect probability refers to the probability that the inspection object after being processed by the first manufacturing process has a defect. Regarding the calculated first defect probability, as will be described later, it is sent to the second inspection device 70 by the first communication unit 68.

[0041] The first determination unit 626 uses the first defect probability calculated by the first inspection unit 624 to determine whether there is a defect in the inspection object. In this embodiment, the first determination unit 626 determines whether there is a defect in the inspection object by comparing the first defect probability with a first reference value 642 determined in advance with an arbitrary value. The first reference value 642 is a determination condition for determining the presence or absence of a defect in the inspection object and is set in advance according to an arbitrary value. In this embodiment, the first reference value 642 is set to 2.0%.

[0042] The first communication unit 68 wirelessly communicates the first defect probability of the inspection target calculated by the first inspection unit 624 to the second inspection device 70. The first communication unit 68 also sends an execution command for switching the conveyance path to the first conveyance device 41 of the production line LP. As the wireless communication, for example, wireless communication via a wireless local area network (LAN) using the 2.4 GHz band or 5 GHz band compliant with the IEEE802.11a standard, wireless communication using a sub-gigahertz band less than 1 GHz band (916.5 MHz to 927.5 MHz), or wireless communication using Bluetooth (registered trademark) can be used. The first communication unit 68 can also be wired-connected to the first conveyance device 41 and the second inspection device 70 through a wired LAN such as Ethernet (registered trademark).

[0043] Figure 3 It is a block diagram showing the internal functional structure of the second inspection device 70. The second inspection device 70 uses the image data of the inspection target to determine the presence or absence of a defect in the inspection target. The second inspection device 70 includes a central processing unit (CPU) 72, a storage device 74, a second inspection data acquisition unit 76, a first acquisition unit 77, and a second communication unit 78. The CPU 72, the storage device 74, the second inspection data acquisition unit 76, the first acquisition unit 77, and the second communication unit 78 are interconnected via a bus 71 and can communicate bidirectionally. The first acquisition unit 77 acquires the first defect probability sent from the first inspection device 60 via the first communication unit 68.

[0044] The CPU 72 functions as a second inspection unit 724 and a second determination unit 726 by executing various programs stored in the storage device 74. The storage device 74 is, for example, a RAM, a ROM, or a hard disk drive (HDD). The HDD (or ROM) stores various programs for implementing the functions provided in the present embodiment, and the various programs read from the HDD are loaded onto the RAM and executed by the CPU 72. The storage device 74 stores a first threshold 741 for determining whether to change the inspection conditions of the second inspection unit 724 or the determination conditions of the second determination unit 726, a second threshold 742 for determining whether there is a defect in the inspection target, and a second learning model 744. In addition, in the storage device 74, the second defect probability calculated by the second inspection unit 724 and the first defect probability acquired by the first acquisition unit 77 are temporarily stored.

[0045] The second inspection data acquisition unit 76 acquires data related to the inspection object for inspection. In the present embodiment, the second inspection data acquisition unit 76 is a camera for photographing the product after being processed by the second manufacturing device 32. The second inspection data acquisition unit 76 is provided on the conveyance path for conveying the inspection object after being processed by the second manufacturing device 32, and photographs the inspection object being conveyed to acquire an image of the inspection object. Since the other structure of the second inspection data acquisition unit 76 is the same as that of the first inspection data acquisition unit 66, the description thereof is omitted.

[0046] The second inspection unit 724 calculates the second defect probability and functions as a second acquisition unit for acquiring the second defect probability. The second defect probability refers to the probability that the inspection object after being processed by the second manufacturing process has a defect. The second defect probability is calculated by using the second learning model 744 for the data related to the inspection object processed in the second manufacturing process. In the present embodiment, the second inspection unit 724 calculates and acquires the second defect probability by using machine learning. Machine learning can use any type of machine learning such as supervised learning, unsupervised learning, and reinforcement learning. In the present embodiment, the second inspection unit 724 uses statistical machine learning. The second learning model 744 is an identification model in a neural network. In the present embodiment, the second learning model 744 is a CNN. Specifically, the second learning model 744 adopts a CNN using VGG16. VGG16 refers to a neural network having 13 convolutional layers and 3 fully connected layers. The second learning model 744 is not limited to a CNN, and an identification model and a generation model in various neural networks such as a generative adversarial network (GAN), a variational autoencoder (VAE), and an autoregressive generative network can also be used. The second learning model 744 is pre-trained using images of normal products and defective products after being processed by the second manufacturing device 32. In the present embodiment, the second inspection unit 724 inputs the data of the inspection object acquired by the second inspection data acquisition unit 76 into the second learning model 744, and calculates the second defect probability by using, for example, a sigmoid function or a softmax function.

[0047] The second determination unit 726 uses the second defect probability to determine whether the inspection object has a defect. In addition, the second determination unit 726 switches the determination condition for determining whether the inspection object has a defect according to the result of the first defect probability obtained by the first acquisition unit 77. In the present embodiment, the second determination unit 726 compares the first defect probability with the first threshold 741 and determines which one of the first value 742A and the second value 742B as the second threshold 742 is used as the determination condition. Specifically, when the obtained first defect probability is greater than the first threshold 741, the second threshold 742 is changed from the first value 742A to the second value 742B.

[0048] The first threshold 741 is a threshold for determining whether the second determination unit 726 changes the determination condition. The first threshold 741 is preset to an arbitrary value smaller than the first reference value 642. In the present embodiment, the first threshold 741 is set to half of the first reference value 642, that is, 1.0%. When the first defect probability is greater than the first threshold 741 and smaller than the first reference value 642, the product after the process of the first manufacturing process is not determined to be defective by the first inspection device 60. However, in this case, affected by the change in the characteristics of the product based on the first manufacturing process, there may be potential abnormalities in the product after the process of the second manufacturing process. The second inspection device 70 of the present embodiment determines whether there may be potential abnormalities in the product after the process of the second manufacturing process, affected by the change in the characteristics of the product based on the first manufacturing process, according to whether the first defect probability is greater than the first threshold 741. The first threshold 741 is not limited to half of the first reference value 642, and can be set in consideration of the influence of the product based on the second manufacturing process. For example, it can also be one-fourth or one-third of the first reference value 642.

[0049] The second threshold 742 is a determination condition for determining the presence or absence of a defect in the inspection object, and is preset according to an arbitrary value. In the present embodiment, in the second threshold 742, the first value 742A used under normal circumstances and the second value 742B used when the second determination unit 726 changes the determination condition are preset. The first value 742A is set to 2.0%. Regarding the second value 742B, from the viewpoint of improving the inspection accuracy, it is set to a value smaller than the first value 742A. In the present embodiment, the second value 742B is set to 0.5%.

[0050] The second determination unit 726 uses the second defect probability calculated by the second inspection unit 724 and the second threshold 742 to determine whether there is a defect in the inspection target. In the present embodiment, the second determination unit 726 determines whether there is a defect in the inspection target by comparing the second defect probability with the second threshold 742 as the first value 742A or comparing the second defect probability with the second threshold 742 as the second value 742B according to the determination condition determined based on the first defect probability.

[0051] The second communication unit 78 sends an execution command for switching the transport path to the second transport device 42 of the production line LP via wireless communication. Since the other structure of the second communication unit 78 is the same as that of the first communication unit 68, the description thereof is omitted.

[0052] Figure 4 It is a process chart showing each process of the production line LP. In step S50, the process for the product is implemented through the first manufacturing process. In the present embodiment, the product as a vehicle component is formed by a die-casting machine as the first manufacturing device 31.

[0053] In step S100, the product is inspected through the first inspection process. In the first inspection process, the calculation of the first defect probability and the determination of the presence or absence of a defect are implemented for the product after the process based on the first manufacturing device 31 by the first inspection device 60. The calculated first defect probability is output to the second inspection device 70 in a state associated with the production number of the product and the like.

[0054] In step S180, it is determined whether to switch the process of the delivery destination of the product according to the inspection result of the product based on the first inspection process. In the present embodiment, when the first inspection device 60 determines that there is a defect in the product in the first inspection process (S180: Yes), the first inspection device 60 sends a command signal for switching the transport path of the first transport device 41 to the first transport device 41. As a result, the product determined to have a defect is sent to the first correction process in step S182. When the first inspection device 60 determines that the product has no defect in the first inspection process (S180: No), the first inspection device 60 does not output a command signal to the first transport device 41. As a result, the product is sent to the second manufacturing process in step S190.

[0055] In step S182, as the first correction process, a process of removing defects from the product is performed. In the present embodiment, as an example of removing defects from the product, removing casting flash of the product after casting, filling casting porosity, etc. may be cited. As a result, the defects are removed from the product processed by the first manufacturing device 31, and the product is sent to the second manufacturing device 32 as a normal product. For example, when it is determined that the product has defects and the product is discarded, step S182 may be omitted.

[0056] In step S190, a process for the product is performed through the second manufacturing process. In the present embodiment, post-processing such as cutting and drilling of the product after casting is performed by a machining device as the second manufacturing device 32.

[0057] In step S200, the product is inspected through the second inspection process. In the second inspection process, for the product processed by the second manufacturing device 32, the calculation of the second defect probability and the determination of the presence or absence of defects are performed based on the second inspection device 70.

[0058] In step S280, based on the inspection result of the product in the second inspection process, a process of determining whether to switch the delivery destination of the product is performed. In the present embodiment, when the second inspection device 70 determines that there are defects in the product in the second inspection process (S280: Yes), the second inspection device 70 sends a command signal for switching the transport path of the second transport device 42 to the second transport device 42. As a result, the product determined to have defects is sent to the second correction process of step S282. When the second inspection device 70 determines that there are no defects in the product in the second inspection process (S280: No), the second inspection device 70 does not output a command signal to the second transport device 42, and this process ends.

[0059] In step S282, as the second correction process, a process of removing defects from the product is performed. In the present embodiment, as an example of removing defects from the product, additional machining of machining may be cited. As a result, the defects are removed from the product processed by the second manufacturing device 32, and the product is sent to the subsequent process as a normal product. For example, when it is determined that there are defects in the product and the product is discarded, step S282 may be omitted.

[0060] Figure 5 It is a process chart showing the details of the first inspection process based on the first inspection device 60. In step S110, the first inspection data acquisition unit 66 acquires data for inspection. In the present embodiment, a camera as the first inspection data acquisition unit 66 captures an image of the product, thereby acquiring image data of the product processed by the first manufacturing device 31.

[0061] In step S120, the first inspection unit 624 calculates a first defect probability by inputting the acquired image data into the first learning model 644. In step S130, the first determination unit 626 determines whether there is a defect in the inspection object by using the first defect probability calculated by the first inspection unit 624 and the first reference value 642 stored in the storage device 64. In the present embodiment, the first reference value 642 is set to 2.0%. When the first defect probability is less than 2.0% (S130: Yes), the process proceeds to step S132, and the first determination unit 626 determines that there is no defect in the inspection object. When the first defect probability is 2.0% or more (S130: No), the process proceeds to step S134, and the first determination unit 626 determines that there is a defect in the inspection object. In step S136, the first determination unit 626 sends an instruction signal for switching the conveyance path to the path for sending to the first correction process to the first conveyance device 41 via the first communication unit 68. When the first determination unit 626 finishes the process of step S132 or step S136, the process proceeds to step S140. In step S140, the first determination unit 626 sends the first defect probability calculated in step S120 to the second inspection device 70 via the first communication unit 68. When the transmission of the first defect probability is completed, the first inspection process ends.

[0062] Figure 6 is a process diagram showing details of the second inspection process based on the second inspection device 70. In step S210, the first acquisition unit 77 acquires the first defect probability transmitted wirelessly from the first inspection unit 624. The first defect probability acquired by the first acquisition unit 77 is the first defect probability associated with the production number etc. of the product inspected by the second inspection device 70 in this process. In step S220, the second determination unit 726 compares the acquired first defect probability with the first threshold value 741 stored in the storage device 74. The first threshold value 741 is set to 1.0%. When the acquired first defect probability is less than or equal to the first threshold value 741 (S220: No), the process proceeds to step S230, without changing the determination condition for determining whether there is a defect in the inspection object, and proceeds to step S232.

[0063] In step S232, the second inspection data acquisition unit 76 acquires data for inspection. In the present embodiment, a camera as the second inspection data acquisition unit 76 captures an image of the product, thereby acquiring image data of the product after being processed by the second manufacturing device 32. In step S234, the second inspection unit 724 calculates a second defect probability by inputting the acquired image data into the second learning model 744.

[0064] In step S240, the second determination unit 726 determines whether there is a defect in the inspection target by using the second defect probability calculated by the second inspection unit 724 and the first value 742A stored in the storage device 74 as the second threshold 742. In the present embodiment, the first value 742A is set to 2.0%. When the second defect probability is less than 2.0% (S240: Yes), the second determination unit 726 proceeds to step S242 and determines that there is no defect in the inspection target.

[0065] When the second defect probability is 2.0% or more (S240: No), the process proceeds to step S244, and the second determination unit 726 determines that there is a defect in the inspection target. In step S246, the second determination unit 726 sends a command signal for switching the conveyance path to the path for sending to the second correction process to the second conveyance device 42 via the second communication unit 78. When the process of step S242 or step S246 ends, the second inspection process ends.

[0066] In step S220, when the obtained first defect probability is greater than the first threshold 741 (S220: Yes), the process proceeds to step S250. In step S250, the second determination unit 726 changes the determination condition for determining the presence or absence of a defect in the inspection target. Specifically, the second threshold 742 is changed from the first value 742A to the second value 742B. In addition, in the present embodiment, regarding the change from the first value 742A to the second value 742B, based on the relationship with step S130, it corresponds to the case where the first defect probability is greater than the first threshold 741 and less than the first reference value 642.

[0067] In step S252, the second inspection data acquisition unit 76 acquires data for inspection. In the present embodiment, a camera as the second inspection data acquisition unit 76 captures an image of the product, thereby acquiring an image of the inspection target after being processed by the second manufacturing device 32. In step S254, the second inspection unit 724 calculates the second defect probability by inputting the acquired image data into the second learning model 744.

[0068] In step S260, the second determination unit 726 determines whether there is a defect in the inspection target by using the second defect probability calculated by the second inspection unit 724 and the second value 742B stored in the storage device 74 as the second threshold 742. In the present embodiment, the second value 742B is set to 0.5%. When the second defect probability is less than 0.5% (S260: Yes), the second determination unit 726 proceeds to step S262 and determines that there is no defect in the inspection target.

[0069] When the second defective probability is 0.5% or more (S260: No), the process proceeds to step S264, and the second determination unit 726 determines that there is a defect in the inspection target. In step S266, the second determination unit 726 sends a command signal for switching the transport path to the path for sending to the second correction process to the second transport device 42 via the second communication unit 78. When the processing of step S262 or step S266 is completed, the second determination unit 726 ends the second inspection process.

[0070] As described above, the second inspection device 70 of the present embodiment includes: a first acquisition unit 77 that acquires the first defective probability of the inspection target processed in the first manufacturing process; a second inspection unit 724 that calculates the second defective probability for the inspection target processed in the second manufacturing process using the second learning model 744; and a second determination unit 726 that determines that there is a defect in the inspection target when the second defective probability is equal to or greater than the second threshold 742. When the acquired first defective probability is greater than the preset first threshold 741, the second determination unit 726 changes the second threshold 742 from the first value 742A to the second value 742B to improve the inspection accuracy. When the product after the processing based on the second manufacturing process may be affected by the first manufacturing process and have potential abnormalities, the determination conditions in the second inspection device 70 are changed to conditions that improve the inspection accuracy. Therefore, the inspection accuracy based on the second inspection device 70 can be improved, and the possibility of detecting potential abnormalities in the product after the processing based on the second manufacturing process can be increased. In addition, the conditions are changed only when the product after the processing based on the second manufacturing process may have potential abnormalities due to the first manufacturing process, so that the impact on productivity can be reduced, and the detection accuracy can be improved efficiently.

[0071] According to the second inspection device 70 of the present embodiment, when the acquired first defective probability is greater than the first threshold 741, the second determination unit 726 changes the second threshold 742 to a second value 742B that is smaller than the first value 742A. When the product after the processing based on the second manufacturing process may have potential abnormalities due to the first manufacturing process, by making the defective determination conditions in the second determination unit 726 strict, it is possible to reduce or avoid the situation where potential abnormalities in the product after the processing based on the second manufacturing process are overlooked by the second inspection device 70, and the inspection accuracy can be improved.

[0072] B. Second Embodiment:

[0073] Figure 7It is a block diagram showing the internal functional structure of the second inspection device 70b as the second embodiment. The second inspection device 70b of the second embodiment changes the inspection conditions in the case where the first defect probability is greater than the first threshold 741. The second inspection device 70b differs from the second inspection device 70 of the first embodiment in that the first model 744A and the second model 744B of the second learning model 744 are stored, and the second determination unit 726b is provided instead of the second determination unit 726. The structure other than this is the same as that of the second inspection device 70 of the first embodiment.

[0074] The second determination unit 726b changes the inspection conditions differently from the second determination unit 726 in the first embodiment that changes the determination conditions. Specifically, in the case where the first defect probability is greater than the first threshold 741, the second determination unit 726b changes the inspection conditions in the inspection based on the second inspection unit 724 from the inspection conditions using only the first model 744A of the second learning model 744 to the inspection conditions using both the first model 744A and the second model 744B of the second learning model 744. In the present embodiment, the first model 744A of the second learning model 744 is a CNN, and for example, a CNN using VGG16 can be adopted. Regarding the second model 744B, it is a learning model that prioritizes the inspection accuracy of the inspection object over the processing time until the calculation of the second defect probability compared to the first model 744A. According to the second model 744B, the accuracy of object detection is improved compared to the first model 744A, the calculation accuracy of the defect probability can be improved, and the inspection accuracy can be improved. In the present embodiment, the second model 744B is an R-CNN (Region-based CNN), and for example, R-CNN YOLOv3 can be adopted. As the R-CNN, it is not limited to R-CNN and R-CNN YOLOv3, and for example, Fast R-CNN, Faster R-CNN, YOLO, etc. can also be adopted. For example, Faster R-CNN is preferably applied to the second model 744B because of its high inspection accuracy. For example, YOLO (You only LookOnce) is preferably applied to the first model 744A because of its fast processing speed. In addition, the first model 744A and the second model 744B of the second learning model 744 are not limited to the CNN model or the R-CNN model, and an SSD (Single ShotDetector) can also be used, and various neural network recognition models and generation models such as a generative adversarial network (GAN), a variational autoencoder (VAE), and an autoregressive generative network can also be used.

[0075] Figure 8It is a process chart showing the details of the second inspection process of the second inspection device 70b based on the second embodiment. In the second inspection process of the second embodiment, it is different in that steps S300 to S356 are provided instead of steps S250 to S266. In step S220, when the obtained first defect probability is greater than the first threshold 741 (S220: Yes), the process proceeds to step S300. In step S300, the second determination unit 726b changes the inspection conditions of the second inspection unit 724. Specifically, the second determination unit 726b changes the inspection conditions from those using only the first model 744A of the second learning model 744 to inspection conditions using two learning models, namely, inspection using the first model 744A of the second learning model 744 and inspection using the second model 744B, to perform inspections respectively.

[0076] In step S310, the second inspection data acquisition unit 76 acquires data for inspection. In the present embodiment, a camera serving as the second inspection data acquisition unit 76 captures an image of the product, thereby acquiring an image of the inspection object after being processed by the second manufacturing device 32. In step S320, the second inspection unit 724 calculates the second defect probability by inputting the acquired image data into the CNN-VGG16 of the first model 744A, which is the second learning model 744.

[0077] In step S340, the second determination unit 726b determines whether there is a defect in the inspection object by using the second defect probability calculated by the second inspection unit 724 and the second threshold 742 stored in the storage device 74. In the present embodiment, the second threshold 742 is set to 2.0%. When the second defect probability is 2.0% or more (S340: No), the process proceeds to step S344, and the second determination unit 726b determines that there is a defect in the inspection object. In step S346, the second determination unit 726b sends an instruction signal for switching the conveying path to the path for sending to the second correction process to the second conveying device 42 via the second communication unit 78.

[0078] When the second defect probability is less than 2.0% (S340: Yes), the second determination unit 726 proceeds to step S342. In step S342, the second inspection unit 724 further calculates the defect probability using the second model 744B according to the changed inspection conditions. Specifically, the new second defect probability is calculated by further inputting the acquired image data into the R-CNN YOLOv3, which is the second model 744B. The new second defect probability refers to the second defect probability calculated using the second model 744B after the calculation using the first model 744A.

[0079] In step S350, the second determination unit 726b determines whether there is a defect in the inspection target, using the new second defect probability calculated by the second inspection unit 724 and the second threshold 742. For example, when the first value 742A and the second value 742B, which are the second thresholds 742 shown in the first embodiment, are stored in the storage device 74, the second value 742B may be used instead of the first value 742A. When the new second defect probability is 2.0% or more (S350: No), the process proceeds to step S354, and the second determination unit 726b determines that there is a defect in the inspection target. In step S356, the second determination unit 726b sends an instruction signal for switching the conveyance path to the path for sending to the second correction process to the second conveyance device 42 via the second communication unit 78. When the new second defect probability is less than 2.0% (S350: Yes), the second determination unit 726b proceeds to step S352 and determines that there is no defect in the inspection target. When the second determination unit 726b finishes the processing of step S352, step S356, and step S346, the second inspection process ends.

[0080] According to the second inspection device 70b of the present embodiment, when the first defect probability is greater than the first threshold 741, after the second determination unit 726b calculates the second defect probability using the first model 744A of the second learning model 744, the first model 744A is further changed to the second model 744B, which is a learning model with high inspection accuracy. The second inspection unit 724 also calculates the new defect probability of the inspection target using the second model 744B. When the product after the processing based on the second manufacturing process may have potential abnormalities due to the first manufacturing process, the second determination unit 726b changes the inspection conditions based on the second inspection unit 724 to the conditions for performing two inspections, namely, the inspection using the first model 744A as the second learning model 744 and the inspection using the second model 744B. Therefore, by changing to the inspection conditions using the second model 744B with high inspection accuracy, the inspection accuracy based on the second inspection device 70b can be improved. In addition, by performing two inspections, the inspection accuracy based on the second inspection device 70b can be further improved.

[0081] According to the second inspection device 70b of the present embodiment, the first model 744A is a CNN, and the second model 744B is an R-CNN. Therefore, the inspection accuracy based on the second inspection device 70b can be improved by a simple method using known learning models.

[0082] C. Other Embodiments:

[0083] (C1) In each of the above-described embodiments, an example is shown in which the first manufacturing apparatus 31 is a die casting machine and the second manufacturing apparatus 32 is an apparatus for post-processing and post-treatment of the product after casting. In contrast, the first manufacturing apparatus 31 and the second manufacturing apparatus 32 are not limited to these examples, and may be various devices such as a processing machine, a welding machine, a molding machine, a painting machine, and the like.

[0084] (C2) The first manufacturing process and the second manufacturing process are not necessarily limited to processes that are continuously connected to each other, and may include a process in which other processing is performed between the first manufacturing process and the second manufacturing process on the premise that the first manufacturing process and the second manufacturing process are correlated with each other. However, the number of processes between the first manufacturing process and the second manufacturing process is preferably the number of processes such that the influence of the processing based on the first manufacturing process on the properties of the product after the processing based on the second manufacturing process as a subsequent process is small, for example, preferably 7 or less, and more preferably 3 or less.

[0085] (C3) In each of the above-described embodiments, the first inspection device 60 and the second inspection device 70 use the image data of the inspection object to determine the presence or absence of defects in the inspection object. The first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 acquire the image of the inspection object as data for inspection. In this regard, the data for inspection is not limited to the image of the product, and may be various parameters of the product such as temperature, size, weight, color, shape, etc. The data for inspection is not limited to the data of the product, and may also be data other than the product on the premise that it affects the defect probability of the inspection object. The data for inspection may be, for example, voltage value, current value, pressure value, temperature, displacement amount, their waveforms, change amounts, processing time, etc., the manufacturing conditions and parameters of the devices included in the production line LP. The first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 are not limited to cameras, and may also be various sensors such as optical sensors, sound sensors, thermal sensors, current sensors, voltage sensors, distance sensors, air pressure sensors, acceleration sensors, rotational speed sensors, humidity sensors, pressure sensors, and magnetic sensors. Each sensor is a sensor that acquires data for calculating the defect probability of the inspection object. For example, when the first manufacturing apparatus 31 and the second manufacturing apparatus 32 have devices other than cameras for photographing the inspection object, and the first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 acquire the data of the inspection object, the first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 may also be communication devices that acquire the inspection data from devices other than the first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 via wireless communication or wired communication.

[0086] (C4) In the above second embodiment, the second determination unit 726b changes the inspection conditions of the second inspection unit 724 to the conditions for performing two inspections, namely, an inspection using the first model 744A as the second learning model 744 and an inspection using the second model 744B. In contrast, when the obtained first defective probability is greater than the first threshold 741, the second determination unit 726b may also change the inspection conditions of the second inspection unit 724 from the inspection conditions using the first model 744A as the second learning model 744 to the inspection conditions using only the second model 744B. In this case, in the example of Figure 8 , steps S320, S340, S344, and S346 can be omitted. According to the second inspection device 70b of this method, by changing to the inspection conditions using the second model 744B with high inspection accuracy, the inspection accuracy based on the second inspection device 70b can be improved. Therefore, the inspection accuracy of the second inspection device 70b can be improved when the product processed by the second manufacturing process may have potential abnormalities due to the first manufacturing process.

[0087] (C5) In the above embodiments, an example is shown in which the second inspection unit 724 also functions as the second acquisition unit. Specifically, an example is shown in which the second inspection unit 724 performs both the calculation and acquisition of the second defective probability. In contrast, when the second defective probability is calculated by a device different from the second inspection device 70, such as the second manufacturing device 32, the second acquisition unit may not perform the calculation of the second defective probability and only acquire the second defective probability from the second manufacturing device 32 via wireless communication or wired communication.

[0088] The control unit and its method described in the present invention can be implemented by a dedicated computer provided with a configured processor and a memory, and the processor and the memory are programmed to execute one or more functions implemented by a computer program. Alternatively, the control unit and its method described in the present invention can be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and method described in the present invention can be implemented by one or more dedicated computers, which are composed of a processor programmed to execute one or more functions and a memory and a processor composed of one or more hardware logic circuits. In addition, the computer program can also be stored as instructions executed by a computer in a non-transitory entity recording medium readable by the computer.

[0089] The present invention is not limited to the above-described embodiments and can be implemented in various structures without departing from its gist. For example, with respect to the technical features in the embodiments corresponding to the technical features in each of the aspects described in the summary of the invention, in order to solve part or all of the above problems or to achieve part or all of the above effects, appropriate substitution or combination can be made. In addition, if the technical feature is not described as an essential content in this specification, it can be appropriately deleted.

[0090] Description of Reference Numerals

[0091] 31…First manufacturing device

[0092] 32…Second manufacturing device

[0093] 41…First conveying device

[0094] 42…Second conveying device

[0095] 51…First correction process

[0096] 52…Second correction process

[0097] 60…First inspection device

[0098] 61…Bus

[0099] 62…CPU

[0100] 64…Storage device

[0101] 66…First inspection data acquisition unit

[0102] 68…First communication unit

[0103] 70, 70b Second inspection device

[0104] 71…Bus

[0105] 72…CPU

[0106] 74…Storage device

[0107] 76…Second inspection data acquisition unit

[0108] 77…First acquisition unit

[0109] 78…Second communication unit

[0110] 100…Inspection system

[0111] 624…First inspection unit

[0112] 626…First judgment unit

[0113] 642…First reference value

[0114] 644…The first learning model

[0115] 724…The second inspection unit

[0116] 726, 726b…The second judgment unit

[0117] 741…The first threshold value

[0118] 742…The second threshold value

[0119] 742A…The first value

[0120] 742B…The second value

[0121] 744…The second learning model

[0122] 744A…The first model

[0123] 744B…The second model

[0124] LP…Production line

Claims

1. An inspection device, comprising: a first acquisition unit that acquires a first defect probability, which is calculated using a first learning model for data related to an inspection object processed in a first manufacturing process; a second acquisition unit that acquires a second defect probability, which is calculated using a second learning model for data related to the inspection object processed in a second manufacturing process after the first manufacturing process and associated with the first manufacturing process; and a determination unit that (a) when the acquired first defect probability is less than or equal to a first threshold and the acquired second defect probability is greater than or equal to a second threshold, determines that a defect exists in the inspection object, where the second defect probability is calculated using a first model of the second learning model; (b) when the acquired first defect probability is greater than the first threshold: (b1) when the acquired second defect probability is greater than or equal to the second threshold, determines that a defect exists in the inspection object, where the second defect probability is calculated using the first model of the second learning model; (b2) when the acquired second defect probability calculated using the first model of the second learning model is less than the second threshold, newly calculates the second defect probability using a second model of the second learning model, where the second model is different from the first model, the second model has a higher accuracy than the first model, and the first model has a faster processing speed than the second model; (b3) when the newly calculated second defect probability using the second model is greater than or equal to the second threshold, determines that a defect exists in the inspection object.

2. The inspection device according to claim 1, wherein the first model is a CNN and the second model is an R-CNN.

3. An inspection method, acquiring a first defect probability, which is calculated using a first learning model for data related to an inspection object processed in a first manufacturing process, acquiring a second defect probability, which is calculated using a second learning model for data related to the inspection object processed in a second manufacturing process after the first manufacturing process and associated with the first manufacturing process, (a) when the acquired first defect probability is less than or equal to a first threshold and the acquired second defect probability is greater than or equal to a second threshold, determines that a defect exists in the inspection object, where the second defect probability is calculated using a first model of the second learning model; (b) when the acquired first defect probability is greater than the first threshold: (b1) when the acquired second defect probability is greater than or equal to the second threshold, determines that a defect exists in the inspection object, where the second defect probability is calculated using the first model of the second learning model; (b2)In the case where the obtained second defect probability calculated by the first model using the second learning model is less than the second threshold value, the second model of the second learning model newly calculates the second defect probability. The second model is different from the first model, the second model has a higher accuracy than the first model, and the first model has a faster processing speed than the second model; (b3)In the case where the obtained second defect probability newly calculated by the second model is greater than or equal to the second threshold value, it is determined that there is a defect in the inspection object.

Images