Inspection method and inspection device

A two-stage inspection method using speed-focused and accuracy-focused models enhances the balance between time and accuracy in inspection processes, optimizing the workflow by reducing redundant inspections and improving overall efficiency.

JP7582130B2Active Publication Date: 2024-11-13TOYOTA JIDOSHA KK
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2021143985
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-09-03
Publication Date
2024-11-13
Estimated Expiration
2041-09-03

AI Technical Summary

Technical Problem

There is a need to improve the balance between inspection time and accuracy in multiple inspection processes, particularly when using learning models, as existing methods often struggle to optimize both efficiently.

Method used

Implement a two-stage inspection process using a first learning model that prioritizes processing speed to identify a provisional defective range, followed by a second model that prioritizes accuracy to inspect the identified range, thereby reducing the scope of the second inspection and enhancing overall efficiency and accuracy.

Benefits of technology

This approach allows for a balanced improvement in inspection time and accuracy across multiple stages by narrowing the inspection range in subsequent processes, leveraging models like SSD for speed and R-CNN for accuracy, thus optimizing the inspection workflow.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582130000001
    Figure 0007582130000001
  • Figure 0007582130000002
    Figure 0007582130000002
  • Figure 0007582130000003
    Figure 0007582130000003
Patent Text Reader

Abstract

To provide a technique capable of improving the balance between inspection time and inspection accuracy by reducing the number of tests, in the multiple inspection process over the first inspection process and the second inspection process, by narrowing down the inspection range in the second inspection process to the tentative defect range extracted in the first inspection process.SOLUTION: The inspection method includes: a first inspection step of extracting a provisional defect range from a predetermined inspection range of an object to be inspected processed in the first manufacturing process; and a second inspection step of inspecting the provisional defect range of the inspection object which is processed in the second manufacturing process after the first manufacturing process.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an inspection method and an inspection apparatus. [Background technology]

[0002] An inspection method using a learning model is known (for example, Patent Document 1). In this inspection method, reference data is converted into comparative reference data according to the conditions of the printed matter by deep learning using a learning model, and the comparative reference data is compared with comparative image data generated using image data obtained by imaging the printed matter as the inspection target, to determine the quality of the printed matter. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-186938 A Summary of the Invention [Problem to be solved by the invention]

[0004] In multiple inspection processes that have the same inspection target, there is a need to improve the balance between inspection time and inspection accuracy. This problem is not limited to inspections that use learning models, but is common to any type of inspection. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to one aspect of the present disclosure, there is provided an inspection method including a first inspection step of extracting a tentative defect range from a predetermined inspection range of an inspection object processed in a first manufacturing step, and a second inspection step of inspecting the tentative defect range of the inspection object processed in a second manufacturing step subsequent to the first manufacturing step. According to the inspection method of this embodiment, the number of inspections in the second inspection process can be reduced by narrowing down the inspection range in the second inspection process to the provisional defective range extracted in the first inspection process. Therefore, the takt time of the second inspection process can be shortened, or the inspection accuracy can be improved while maintaining the processing time of the second inspection process. As a result, the balance between the inspection time and the inspection accuracy can be improved in the multiple inspection processes spanning the first inspection process and the second inspection process. (2) In the inspection method of the above aspect, the first inspection step may include extracting the tentative defect range by using a first learning model for the predetermined inspection range, and the second inspection step may include inspecting the tentative defect range by using a second learning model. According to this form of inspection method, it is possible to improve the balance between inspection time and inspection accuracy in multiple inspection processes using a learning model. (3) In the above-described inspection method, the method may include calculating a defect probability for each location of the inspection object using the first learning model for inspection data previously obtained about the inspection object processed in the first manufacturing process, and extracting, as the provisional defect range, a range including locations where the calculated defect probability is equal to or greater than a predetermined first threshold value. According to the inspection method of this aspect, the inspection range in the second inspection step can be narrowed down by utilizing the inspection result in the first inspection step. (4) In the inspection method of the above embodiment, the first inspection process may include determining that the inspection object is defective if there is a location where the calculated defect probability is equal to or greater than a second threshold value higher than the first threshold value, and determining that the inspection object is not defective if the defect probability is less than the first threshold value at any location within the predetermined inspection range. According to the inspection method of this aspect, in the first inspection step, the provisional defective range can be extracted and the presence or absence of a defect can be inspected. (5) In the inspection method of the above aspect, the first inspection process may include extracting, as the tentative defective range, a range including a location where the calculated defective probability is equal to or greater than the first threshold value and is less than the second threshold value. According to the inspection method of this aspect, it is possible to reduce or prevent a portion determined to be defective in the first inspection process from being inspected redundantly in the second inspection process. (6) In the inspection method of the above aspect, the first learning model may be a learning model that prioritizes processing speed over inspection accuracy. The second learning model may be a learning model that prioritizes inspection accuracy over processing speed. According to the inspection method of this aspect, in the first inspection step, the provisional defective range can be extracted from a wider range, and in the second inspection step, the provisional defective range can be inspected with higher inspection accuracy. (7) In the inspection method of the above aspect, the processing speed of the first learning model may be faster than the processing speed of the second learning model. According to the inspection method of this embodiment, the first inspection step can extract a tentative defective range from a wider range than the second inspection step, and therefore a wider range can be inspected in a plurality of inspection steps spanning the first inspection step and the second inspection step. (8) In the inspection method of the above aspect, the first learning model may be a model using SSD, and the second learning model may be a model using R-CNN. According to the inspection method of this form, a general-purpose learning model can be applied to the first inspection process and the second inspection process. (9) In the above-described inspection method, the second inspection process may include, when the amount of data in the extracted provisional defective range is smaller than a predetermined reference value, inspecting the provisional defective range using the second learning model, and further inspecting a range other than the provisional defective range using a third learning model having a processing speed faster than the second learning model. According to the inspection method of this aspect, in the second inspection step, inspection can be performed on the provisionally defective range and the range other than the provisionally defective range. (10) In the inspection method of the above aspect, the third learning model may be a model using SSD. According to this type of inspection method, a general-purpose learning model can be applied to inspections of ranges other than the provisional defective range. (11) In the inspection method according to the above aspect, the inspection object may be a casting. The first inspection step and the second inspection step may include inspecting the casting using image data obtained by capturing an image of the casting. According to the inspection method of this aspect, it is possible to improve the balance between the inspection time of the multiple inspection steps in the manufacturing process of a casting and the inspection accuracy. The present disclosure can also be realized in various forms other than the inspection method, for example, an inspection system, a manufacturing method for an inspection system, a control method for an inspection system, an inspection device, a manufacturing method for an inspection device, a control method for an inspection device, a computer program for realizing these control methods, a non-transitory recording medium on which the computer program is recorded, etc. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is an explanatory diagram showing an inspection system for performing an inspection method according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an internal functional configuration of a first inspection apparatus. [Diagram 3] FIG. 4 is a block diagram showing the internal functional configuration of a second inspection device. [Figure 4] A process diagram showing each process on the production line. [Diagram 5] FIG. 4 is a process diagram showing details of a first inspection process performed by a first inspection device. [Figure 6] FIG. 4 is a process diagram showing details of a second inspection process performed by a second inspection device. [Figure 7] FIG. 11 is a block diagram showing the internal functional configuration of a second inspection device in a second embodiment. [Figure 8] FIG. 11 is a process diagram showing details of a second inspection process by a second inspection device in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: FIG. 1 is an explanatory diagram showing an inspection system 100 for carrying out the inspection method of this embodiment. The inspection system 100 inspects the presence or absence of defects for inspection objects processed in a process included in a production line LP. The inspection objects are, for example, products flowing through the production line LP, and are vehicle parts in this embodiment. In this disclosure, "processing" means performing a specific operation on the inspection object, and includes various actions that may affect the properties of the inspection object, such as processing of the product. For example, if an action other than processing of the product, such as transporting the product, can affect the properties of the product, the action other than processing of the product can also be included in "processing."

[0009] The inspection system 100 of this embodiment includes two inspection devices that utilize machine learning. Specifically, the inspection system 100 includes a first inspection device 60 and a second inspection device 70. The first inspection device 60 inspects products processed in a first manufacturing process, and the second inspection device 70 inspects products processed in a second manufacturing process that is subsequent to the first manufacturing process.

[0010] The production line LP includes, for example, a first production apparatus 31 and a second production apparatus 32, a first conveying apparatus 41 and a second conveying apparatus 42, and a first correction process 51 and a second correction process 52. In the example of Fig. 1, products flow through the production line LP in the order of the first production apparatus 31, the first conveying apparatus 41, the second production apparatus 32, and the second conveying apparatus 42. Note that the inspection system 100 may be included as a part of the production line LP.

[0011] The first manufacturing equipment 31 is an equipment that performs a first manufacturing process on a product. The first manufacturing equipment 31 is, for example, a die casting machine that forms vehicle parts such as a cylinder head and an engine block. The first manufacturing equipment 31 performs die molding on a molten metal material as the process of the first manufacturing process.

[0012] The second manufacturing apparatus 32 is an apparatus that performs a second manufacturing process on the product. The second manufacturing apparatus 32 is, for example, an NC lathe (numerically-controlled lathe), a machining center, etc. The second manufacturing apparatus 32 performs machining such as cutting, drilling, and screw / tapping on the vehicle parts cast by the first manufacturing apparatus 31 as the second manufacturing process. The second manufacturing apparatus 32 is not limited to machining, and may be various apparatuses that perform post-processing or post-treatment on the product after molding. The second manufacturing apparatus 32 may be various apparatuses that perform, for example, a casting finishing process including removal of casting burrs and barrel polishing, a finishing process including shot blasting and buffing, a joining process including screw fastening and welding, a surface treatment including painting and plating, a heat treatment, an impregnation process, etc. The first manufacturing process and the second manufacturing process are not limited to processes by devices, and may include manual processes by humans.

[0013] In this embodiment, the second manufacturing process is a process that is later than the first manufacturing process and is related to the first manufacturing process. "Related to the first manufacturing process" means that the properties of the product after the first manufacturing process may affect the properties of the product after the second manufacturing process. For example, if the properties of the product after the first manufacturing process change, the properties of the product after the second manufacturing process may change. Examples of when the second manufacturing process is related to the first manufacturing process include when the content of the process in the second manufacturing process is similar to the content of the process in the first manufacturing process, when the position where the product is processed in the second manufacturing process is close to the position where the product is processed in the first manufacturing process, and when the process in the second manufacturing process is performed overlapping with the process in the first manufacturing process.

[0014] When the second manufacturing process is related to the first manufacturing process, the location of a defect in a product processed by the first manufacturing process and the location of a defect in a product processed by the second manufacturing process may be close to each other. In addition, when a potential defect that is not determined to be a defect occurs in the first manufacturing process, the defect may be likely to occur at the location where the defect occurred after the second manufacturing process. Therefore, the location of the defect in a product after the second manufacturing process can be estimated from the detection result of the defect in the product processed by the first manufacturing process. As a result, the range of the inspection after the second manufacturing process can be narrowed down from the inspection result after the first manufacturing process. In the inspection system 100 of this embodiment, the inspection result of the first inspection device 60 is used to obtain a provisional defect range to be inspected in the second inspection process, and the second inspection device 70 performs an inspection on the obtained provisional defect range.

[0015] The first conveying device 41 and the second conveying device 42 are, for example, belt conveyors or conveyors that move on a predetermined track. The first conveying device 41 conveys the product after processing by the first manufacturing device 31. In this embodiment, the conveying path of the first conveying device 41 is preset as an initial setting to be a path from the first manufacturing process to the second manufacturing process. The first conveying device 41 can switch the product conveying path to a path from the first manufacturing process to the first correction process 51 based on the inspection result of the first inspection device 60.

[0016] The second conveying device 42 conveys the product after processing by the second manufacturing device 32. In this embodiment, the conveying path by the second conveying device 42 is preset as an initial setting to a path for conveying from the second manufacturing process to a process subsequent to the second manufacturing process. The second conveying device 42 can switch the product conveying path to a path for conveying from the second manufacturing process to the second correction process 52 based on the inspection results of the second inspection device 70. Note that in this embodiment, the conveying of the product by the first conveying device 41 and the second conveying device 42 is not included in "processing" and does not affect the properties of the product.

[0017] The first correction process 51 is performed when it is determined that the product after processing by the first manufacturing equipment 31 is defective. The first correction process 51 removes the defect from the product after processing by the first manufacturing equipment 31 and sends it to the second manufacturing equipment 32. The second correction process 52 is performed when it is determined that the product after processing by the second manufacturing equipment 32 is defective. The second correction process 52 removes the defect from the product after processing by the second manufacturing equipment 32 and sends it to a subsequent process. The removal of defects by the first correction process 51 and the second correction process 52 may be performed by a dedicated device or manually by an operator.

[0018] 2 is a block diagram showing an internal functional configuration of the first inspection device 60. In this embodiment, the first inspection device 60 judges the presence or absence of defects in the inspection target by using data on the inspection target after processing in the first manufacturing process (hereinafter, also referred to as "first inspection data"). The first inspection device 60 includes a central processing unit (CPU) 62, a storage device 64, a first inspection data acquisition unit 66, and a first communication unit 68. The CPU 62, the storage device 64, the first inspection data acquisition unit 66, and the first communication unit 68 are connected to each other via a bus 61 and can communicate bidirectionally. The CPU 62 functions as a first inspection unit 624 and a first judgment unit 626 by executing various programs stored in the storage device 64.

[0019] The storage device 64 is, for example, a RAM, a ROM, or a hard disk drive (HDD). The HDD (or ROM) stores various programs for implementing the functions provided in this embodiment, and the various programs read from the HDD are expanded on the RAM and executed by the CPU 62. The storage device 64 stores a first threshold value TS1, a second threshold value TS2, and a first learning model 644. The storage device 64 temporarily stores the first defect probability calculated by the first inspection unit 624.

[0020] The first inspection data acquisition unit 66 acquires first inspection data to be used for inspection. In this embodiment, the first inspection data is image data obtained by capturing an image of a predetermined inspection range of a product after processing by the first manufacturing device 31, and the first inspection data acquisition unit 66 is a camera for capturing an image of the product after processing by the first manufacturing device 31. The first inspection data acquisition unit 66 is installed on a conveying path for conveying an inspection target after processing by the first manufacturing device 31, and captures an image of the inspection target during conveying to acquire image data of the inspection target. In this embodiment, the image data acquired by the first inspection data acquisition unit 66 is composed of an RGB input image signal composed of each image signal component represented by R (red), G (green), and B (blue). Note that the input image signal of the image data may be, for example, a YUV image signal composed of Y (luminance signal), U (first color difference signal), and V (second color difference signal), or may be a YCbCr image signal or a YPbPr image signal. The image may be a color image or a grayscale image having a luminance value of any color depth such as 1 bit or 8 bits.

[0021] The inspection range of the first inspection device 60 is set based on the takt time set for the first inspection process. The takt time (Tact-Time) in the inspection process means a time value indicating how long it takes to inspect one product. The inspection range of the first inspection device 60 is determined by the amount or number of first inspection data acquired by the first inspection data acquisition unit 66. The amount or number of first inspection data to be acquired is set to an amount or number of data that allows the first inspection device 60 to complete the inspection within the takt time set for the first inspection process. In this embodiment, the first inspection data acquisition unit 66 captures images of 40 predetermined locations on the product and acquires 40 pieces of image data. In this embodiment, the first inspection device 60 uses the first learning model 644, which has a faster processing speed than the second learning model 744, as described below, to increase the amount of image data acquired and expand the inspection range. For example, when the first inspection device 60 uses the second learning model 744 instead of the first learning model 644, the first inspection device 60 can only inspect 10 pieces of image data within the takt time set for the first inspection process. The number of locations imaged by the first inspection data acquisition unit 66 is not limited to 40, and may be less than 40 or more than 40. The takt time may be set using different time values ​​for the first inspection process and the second inspection process, or may be set using the same time value. In this embodiment, the resolution of the image data acquired by the first inspection data acquisition unit 66 and the resolution of the image data acquired by the second inspection data acquisition unit 76 are the same.

[0022] The first inspection unit 624 calculates a defect probability for each part of the inspection target by using machine learning. The defect probability means the probability that the part of the inspection target has a defect. In this embodiment, the first inspection unit 624 inspects the inspection target by using a first learning model 644 stored in the storage device 64. The first learning model 644 is a model using a convolutional neural network (CNN), and specifically, is a single shot multibox detector (SSD) as a method of object detection by image recognition. The first learning model 644 has been previously trained using images of normal products and images of products having defects after processing by the first manufacturing device 31.

[0023] The processing speed of SSD is faster than the processing speed of other models such as R-CNN (Region Based Convolutional Neural Networks) and Mask R-CNN. SSD is a model that prioritizes processing speed over object detection accuracy, i.e., inspection accuracy. As described later, in this embodiment, the processing speed of the first learning model 644 is set to be faster than the processing speed of the second learning model 744 described later. By configuring in this manner, the inspection range of the first inspection process can be made larger than the inspection range of the second inspection process within the takt time set for the first inspection process. The first learning model 644 is not limited to SSD, and models other than SSD, such as Faster RCNN and Fast RCNN, may be used. In this case, from the viewpoint of increasing the inspection range of the first inspection device 60, it is preferable to use a model with a high processing speed. Whether the processing speed of the first learning model 644 is faster than the processing speed of the second learning model 744 can be determined, for example, by comparing the elapsed time when the same inspection data is inspected by the first learning model 644 and the second learning model 744.

[0024] The first inspection unit 624 inputs the first inspection data obtained in advance by the first inspection data acquisition unit 66 into the first learning model 644, and calculates a first defect probability for each location in the input image data. The first defect probability means the probability that the location to be inspected after processing by the first manufacturing process has a defect. The first defect probability may be calculated for each certain region of the input image data. The calculated first defect probability is output to the first judgment unit 626. The first defect probability may further be transmitted to the second inspection device 70 via the first communication unit 68.

[0025] The first determination unit 626 uses the first defect probability calculated by the first inspection unit 624 to determine whether the inspection target is defective, that is, whether the inspection target is defective. In this embodiment, the first determination unit 626 determines whether the inspection target is defective by comparing the first defect probability with a first threshold value TS1 and a second threshold value TS2 stored in the storage device 64. The second threshold value TS2 is set using a value higher than the first threshold value TS1. The first determination unit 626 determines that the inspection target is defective when there is a portion where the first defect probability is equal to or greater than the second threshold value TS2, and determines that the inspection target is not defective when the first defect probability is less than the first threshold value TS1 at all portions in the predetermined inspection range. In this embodiment, the first threshold value TS1 and the second threshold value TS2 are used not only to determine whether the inspection target is defective, but also to extract a provisional defect range.

[0026] In this embodiment, the first determination unit 626 extracts a provisional defect range in addition to determining whether the inspection target has a defect. The provisional defect range means a range including a portion of the inspection target that is estimated to have a high probability of having a defect after the second manufacturing process. The provisional defect range can also be said to be an inspection range that should be inspected in the second inspection process. The provisional defect range may be, for example, the following (1) and (2). (1) The range includes areas that are not determined to be defective after the first manufacturing process but may have potential defects and are assumed to be highly likely to be defective after processing in the second manufacturing process. (2) A range including areas that cannot be identified as defective with the inspection accuracy of the first inspection process, but that can be detected as areas with a high probability of being defective in the first inspection process, and that may be detected as defective with the inspection accuracy of the second inspection process.

[0027] In this embodiment, the first determination unit 626 extracts, as a provisional defective range, a predetermined range including a portion where the first defective probability is equal to or greater than the first threshold value TS1 and less than the second threshold value TS2 from the image data acquired as the first inspection data. More specifically, the first determination unit 626 extracts, as a provisional defective range, image data including a portion where the first defective probability is equal to or greater than the first threshold value TS1 and less than the second threshold value TS2 from the image data of 40 portions of the product acquired as the first inspection data. That is, in this embodiment, the unit for extracting the provisional defective range is each image data. The unit for extracting the provisional defective range is not limited to each image data, and may be position information within a sheet of image data, such as coordinates or pixel position. In this embodiment, the extracted provisional defective range becomes the range of the second inspection data acquired from the product by the second inspection data acquisition unit 76.

[0028] The first threshold value TS1 and the second threshold value TS2 can be set in advance using any value. From the viewpoint of accurately determining the presence or absence of a defect, the first threshold value TS1 and the second threshold value TS2 can be experimentally obtained by obtaining data showing the correspondence between the presence or absence of a defect in the inspection target and the first defect probability, for example. The first threshold value TS1 is preferably set based on the inspection accuracy of the first inspection device 60, and is preferably set to a value that allows the first inspection device 60 to detect at least a portion that is highly likely to have a defect. The lower the first threshold value TS1, the more preferable it is from the viewpoint of improving the detection accuracy of a portion that is highly likely to have a defect in the first inspection process. However, from the viewpoint of preventing or reducing an unnecessary increase in the number of inspections by the second inspection device 70, the closer the first threshold value TS1 is to the second threshold value TS2, the more preferable it is. In this embodiment, the first threshold value TS1 is set to 1.0%, and the second threshold value TS2 is set to 2.0%. In this embodiment, the first threshold value TS1 is set to 50% of the second threshold value TS2. The first threshold TS1 is not limited to 50% of the second threshold TS2, and may be set to any percentage such as 20%, 25%, 30%, 40%, 60%, 70%, 75%, or 80% of the second threshold TS2.

[0029] The first communication unit 68 transmits the provisional defective range extracted by the first judgment unit 626 to the second inspection device 70 via wireless communication. The first communication unit 68 further transmits an execution command for switching the transport path to the first conveyance device 41 of the manufacturing line LP. As the wireless communication, for example, wireless communication through a wireless local network (LAN) using the 2.4 GHz band or the 5 GHz band conforming to the IEEE802.11a standard, wireless communication using a sub-giga band that is a frequency band less than 1 GHz (916.5 MHz to 927.5 MHz), or wireless communication using Bluetooth (registered trademark) can be used. The first communication unit 68 may be wired to the first conveyance device 41 and the second inspection device 70 via a wired LAN such as Ethernet (registered trademark). The first communication unit 68 is not limited to only transmitting the provisional defective range, and may transmit the first defective probability of the inspection target calculated by the first inspection unit 624 to the second inspection device 70.

[0030] 3 is a block diagram showing the internal functional configuration of the second inspection device 70. In this embodiment, the second inspection device 70 judges the presence or absence of defects in the inspection target by using data on the inspection target after processing in the second manufacturing process (hereinafter, also referred to as "second inspection data"). The inspection range of the second inspection device 70 is set based on the tentative defect range acquired from the first inspection device 60.

[0031] The second inspection device 70 includes a central processing unit (CPU) 72, a storage device 74, a second inspection data acquisition unit 76, a provisional defect range acquisition unit 77, and a second communication unit 78. The CPU 72, the storage device 74, the second inspection data acquisition unit 76, the provisional defect range acquisition unit 77, and the second communication unit 78 are connected to one another via a bus 71 and are capable of bidirectional communication. The provisional defect range acquisition unit 77 acquires the provisional defect range transmitted from the first inspection device 60 via the first communication unit 68.

[0032] The CPU 72 functions as a second inspection unit 724 and a second judgment unit 726 by executing various programs stored in the storage device 74. The storage device 74 is, for example, a RAM, a ROM, or a hard disk drive (HDD). Various programs for realizing the functions provided in this embodiment are stored in the HDD (or ROM), and the various programs read from the HDD are expanded on the RAM and executed by the CPU 72. The storage device 74 stores a third threshold value TS3 and a second learning model 744. The storage device 74 may temporarily store the second defect probability calculated by the second inspection unit 724 and the provisional defect range acquired by the provisional defect range acquisition unit 77.

[0033] The second inspection data acquisition unit 76 acquires second inspection data to be used for inspection. In this embodiment, the second inspection data is image data of the product after processing by the second manufacturing apparatus 32, and the second inspection data acquisition unit 76 is a camera for capturing an image of the product after processing by the second manufacturing apparatus 32. The second inspection data acquisition unit 76 is installed on a transport path for transporting the inspection target after processing by the second manufacturing apparatus 32, and captures an image of the inspection target during transport to acquire image data of the inspection target.

[0034] In this embodiment, the second inspection data acquisition unit 76 is set by the second inspection unit 724 according to the tentative defective range acquired from the tentative defective range acquisition unit 77. The second inspection data acquisition unit 76 acquires image data at a position corresponding to the tentative defective range in accordance with an instruction from the second inspection unit 724. The second inspection data acquisition unit 76 may acquire image data from a predetermined range larger than the tentative defective range, and then extract only image data at a position corresponding to the tentative defective range. Other configurations of the second inspection data acquisition unit 76 are similar to those of the first inspection data acquisition unit 66, and therefore description thereof will be omitted.

[0035] The second inspection unit 724 obtains the second defect probability by using machine learning. The second defect probability means the probability that the part of the inspection target after the processing by the second manufacturing process has a defect. In this embodiment, the second inspection unit 724 inspects the inspection target using the second learning model 744 stored in the storage device 74. The second learning model 744 is a model that prioritizes inspection accuracy over processing speed, and for example, R-CNN can be adopted. The second learning model 744 has a slower processing speed than the first learning model 644 that models SSD, but has a higher inspection accuracy than the first learning model 644. By configuring in this way, the second inspection device 70 can inspect the provisional defect range extracted from the inspection range of the first inspection device 60 with high inspection accuracy, and can efficiently inspect the entire multiple inspection processes. The second learning model 744 has been previously trained using images of normal products and images of products having defects after the processing by the second manufacturing device 32. The second learning model 744 is not limited to R-CNN, and other models with high inspection accuracy, such as Mask R-CNN, can be used.

[0036] The second inspection unit 724 inputs the second inspection data acquired by the second inspection data acquisition unit 76 into the second learning model 744, and calculates a second defect probability for each location in the input image data. The second defect probability may be calculated for each certain region of the input image data. The calculated second defect probability is output to the second judgment unit 726.

[0037] The second determination unit 726 determines whether or not the inspection object is defective by using the second defect probability calculated by the second inspection unit 724. In this embodiment, the second determination unit 726 determines whether or not the inspection object is defective by comparing the second defect probability with a third threshold value TS3 stored in the storage device 74. Specifically, the second determination unit 726 determines that the inspection object is defective when there is a location where the acquired second defect probability is equal to or greater than the third threshold value TS3, and determines that the inspection object is not defective when the second defect probability is less than the third threshold value TS3 at all locations in a predetermined inspection range.

[0038] The third threshold value TS3 can be set in advance using any value. The third threshold value TS3 can be experimentally determined, for example, by obtaining data showing the correspondence between the second defect probability and the presence or absence of defects in the inspection target. In this embodiment, the third threshold value TS3 is set to 2.0%, the same as the second threshold value TS2. However, the third threshold value TS3 may be smaller than the second threshold value TS2 or larger than the second threshold value TS2.

[0039] The second communication unit 78 transmits an execution command for switching the conveying path to the second conveying device 42 of the production line LP via wireless communication. Other configurations of the second communication unit 78 are similar to those of the first communication unit 68, and therefore description thereof will be omitted.

[0040] 4 is a process diagram showing each process of the manufacturing line LP. In step S50, a product is subjected to a first manufacturing process. In this embodiment, a vehicle part is formed from a metal material as a raw material by a die casting machine as the first manufacturing device 31.

[0041] In step S100, as a first inspection process, an inspection by the first inspection process and extraction of a tentative defective range are performed. Specifically, the first inspection device 60 judges the presence or absence of a defect using the first defective probability for the product after processing by the first manufacturing device 31, and extracts a tentative defective range. The extracted tentative defective range is output to the second inspection device 70 in correspondence with the serial number of the product, etc.

[0042] In step S180, it is determined whether or not to switch the process to which the product is to be delivered, depending on the result of the product inspection in the first inspection process. In this embodiment, if the first inspection device 60 determines that the product is defective in the first inspection process (S180: YES), the first inspection device 60 transmits a command signal to the first conveyance device 41 to switch the conveyance path of the first conveyance device 41. As a result, the product determined to be defective is delivered to the first correction process 51 in step S182. If the first inspection device 60 determines that the product is not defective in the first inspection process (S180: NO), the first inspection device 60 does not output a command signal to the first conveyance device 41. As a result, the product is delivered to the second manufacturing process in step S190.

[0043] In step S182, a process of removing defects from the product is performed as the first correction process 51. In this embodiment, examples of removing defects from the product include removing flash from the product after casting and filling porosity. As a result, the product processed by the first manufacturing equipment 31 has defects removed and is sent to the second manufacturing equipment 32. For example, in cases where a product determined to be defective is to be discarded, step S182 may be omitted.

[0044] In step S190, the product is subjected to a second manufacturing process. In this embodiment, a machining device serving as the second manufacturing device 32 performs cutting, drilling, and the like on the cast product.

[0045] In step S200, the product is inspected in a second inspection process. In the second inspection process, the second inspection device 70 calculates a second defect probability and determines whether or not the product is defective for a provisional defect range of the product after the processing by the second manufacturing device 32.

[0046] In step S280, it is determined whether or not to switch the process to which the product is to be discharged, depending on the result of the product inspection in the second inspection process. In this embodiment, if the second inspection device 70 determines that the product is defective in the second inspection process (S280: YES), the second inspection device 70 transmits a command signal to the second conveying device 42 to switch the conveying path of the second conveying device 42. As a result, the product determined to be defective is discharged to the second correction process 52 in step S282. If the second inspection device 70 determines that the product is not defective in the second inspection process (S280: NO), the second inspection device 70 does not output a command signal to the second conveying device 42. As a result, the product is discharged to the process following the second inspection process, and this process ends.

[0047] In step S282, a process of removing defects from the product is performed as the second correction process 52. In this embodiment, an example of removing defects from the product is additional machining. As a result, the product processed by the second manufacturing apparatus 32 has defects removed and is sent to a subsequent process. For example, in a case where the product is discarded when it is determined that the product is defective, step S282 may be omitted.

[0048] Fig. 5 is a process diagram showing details of the first inspection process by the first inspection device 60. The process shown in Fig. 5 is started, for example, when the first inspection data acquisition unit 66 of the first inspection device 60 detects the product dispensed from the first manufacturing device 31 by image detection or the like. The process shown in Fig. 5 may also be started when the first inspection device 60 receives a signal indicating that the first manufacturing device 31 has completed processing.

[0049] In step S110, the first inspection data acquisition unit 66 acquires first inspection data related to the product after processing by the first manufacturing equipment 31. In this embodiment, a camera serving as the first inspection data acquisition unit 66 captures images of 40 locations predetermined as the inspection range, and acquires 40 pieces of image data related to the product after processing by the first manufacturing equipment 31.

[0050] In step S120, the first inspection unit 624 calculates a first defect probability FP1 for each location of the acquired image data by inputting the acquired image data into the first learning model 644. In step S130, the first determination unit 626 compares the first defect probability calculated by the first inspection unit 624 with a first threshold value TS1 and a second threshold value TS2 stored in the storage device 64 to determine whether there is a defect in the inspection target. Note that the processes after step S130 can be repeatedly executed for each of the 40 pieces of acquired image data.

[0051] In step S130, when the first defect probability FP1 is less than the first threshold value TS1 (1.0% in this embodiment) at any location within the inspection range (S130: FP1 < TS1), the process proceeds to step S132, and the first determination unit 626 determines that there is no defect in the inspection target and ends the first inspection process. In this case, the first determination unit 626 may transmit a message indicating that there is no provisional defect range to the second inspection device 70 via the first communication unit 68.

[0052] When there is a location where the first defect probability FP1 is equal to or greater than the second threshold value TS2 (2.0% in this embodiment) (S130: TS2 ≤ FP1), the process proceeds to step S134, and the first determination unit 626 determines that there is a defect in the inspection target. In step S136, the first determination unit 626 transmits a command signal for switching the conveyance path to the path leading to the first correction step 51 to the first conveyance device 41 via the first communication unit 68. In this case, the first determination unit 626 transmits a message indicating that there is no provisional defect range to the second inspection device 70 via the first communication unit 68 and ends the first inspection process.

[0053] When the first defective probability FP1 is equal to or greater than the first threshold value TS1 and less than the second threshold value TS2 (S130: TS1 ≤ FP1 < TS2), the process proceeds to step S140. In step S140, the first determination unit 626 extracts a provisional defective range. Specifically, among the acquired 40 pieces of image data, the imaging positions corresponding to the image data in which a portion having the first defective probability FP1 that is equal to or greater than the first threshold value TS1 and less than the second threshold value TS2 is detected are extracted as the provisional defective range. In step S142, the first determination unit 626 outputs the extracted provisional defective range to the first communication unit 68 and transmits it to the second inspection device 70 via the first communication unit 68. When the transmission of the provisional defective range is completed, the first inspection process ends.

[0054] FIG. 6 is a process diagram showing details of the second inspection process by the second inspection device 70. The process shown in FIG. 6 starts, for example, when the second inspection data acquisition unit 76 of the second inspection device 70 detects a product delivered from the second manufacturing device 32. The process shown in FIG. 6 may also start when the second inspection device 70 receives a signal indicating that the processing has been completed.

[0055] In step S210, the second inspection unit 724 acquires the provisional defective range transmitted from the first communication unit 68 of the first inspection device 60 via the provisional defective range acquisition unit 77. In step S212, the second inspection data acquisition unit 76 acquires second inspection data regarding the product after being processed by the second manufacturing device 32. Specifically, the second inspection unit 724 designates the locations to be imaged by the second inspection data acquisition unit 76 according to the provisional defective range acquired in step S210. The second inspection data acquisition unit 76 images the product according to the locations designated by the second inspection unit 724. As a result, the second inspection data acquisition unit 76 acquires the image data of the imaging positions corresponding to the provisional defective range as the second inspection data.

[0056] In step S220, the second inspection unit 724 calculates a second false positive probability FP2 for each location of the acquired image data by inputting the acquired image data into the second learning model 744. In step S230, the second determination unit 726 compares the second false positive probability FP2 calculated by the second inspection unit 724 with a third threshold value TS3 stored in the storage device 74 to determine whether there is a defect in the inspection target. Note that the processing after step S230 can be repeatedly executed for each of the second inspection data for the acquired number of sheets.

[0057] In step S230, when the second false positive probability FP2 is less than the third threshold value TS3 (2.0% in this embodiment) at any location in the provisional defect range (S230: FP2 < TS3), the process proceeds to step S232, and the second determination unit 726 determines that there is no defect in the inspection target and ends the second inspection process. When there is a location where the second false positive probability FP2 is greater than or equal to the third threshold value TS3 (S230: TS3 ≤ FP2), the process proceeds to step S234, and the second determination unit 726 determines that there is a defect in the inspection target. In step S236, the second determination unit 726 transmits a command signal for switching the conveyance path to the path leading to the second correction process 52 to the second conveyance device 42 via the second communication unit 78 and ends the second inspection process.

[0058] As described above, the inspection method of this embodiment includes a first inspection step of extracting a provisional defective range from a predetermined inspection range of an inspection object processed in a first manufacturing step, and a second inspection step of inspecting the provisional defective range of an inspection object processed in a second manufacturing step subsequent to the first manufacturing step. According to the inspection method of this embodiment, the inspection range in the second inspection step is narrowed down to the provisional defective range extracted in the first inspection step, thereby reducing the number of inspections in the second inspection step. By using the inspection result of the first inspection step, the processing time of the second inspection step can be shortened, and the takt time of the second inspection step can be shortened, or the inspection accuracy can be improved while maintaining the processing time of the second inspection step. Therefore, the balance between the inspection time and the inspection accuracy can be improved in the multiple inspection steps spanning the first inspection step and the second inspection step. In addition, for example, by setting the extraction of the provisional defective range to be prioritized in the first inspection step, the processing speed in the first inspection step can be improved over the detection accuracy, and the inspection range of the first inspection step can be expanded, thereby expanding the overall inspection range in the multiple inspection steps. Furthermore, by improving the inspection accuracy of the second inspection process, it is possible to improve the overall detection accuracy in the multiple inspection processes.

[0059] According to the inspection method of this embodiment, the first inspection process includes extracting a tentative defective range for a predetermined inspection range using the first learning model 644, and the second inspection process includes inspecting the tentative defective range using the second learning model 744. Therefore, in a plurality of inspection processes using the learning model, it is possible to improve the balance between the inspection time and the inspection accuracy.

[0060] According to the inspection method of this embodiment, the first inspection step includes calculating a first defect probability for each location to be inspected using the first learning model 644 for the first inspection data obtained in advance, and extracting a range including a location where the calculated first defect probability is equal to or greater than a first threshold value TS1 as a provisional defect range. By using the first defect probability to extract the provisional defect range, it is possible to determine a potential defect that is not determined to be defective. Therefore, it is possible to use the inspection result of the first inspection step to estimate the location where the defect occurs in the product after the second manufacturing step, and narrow down the inspection range in the second inspection step.

[0061] According to the inspection method of this embodiment, the first inspection step includes judging that the inspection object is defective when there is a portion where the calculated first defect probability is equal to or greater than a second threshold value TS2 that is higher than a first threshold value TS1, and judging that the inspection object is not defective when the first defect probability is less than the first threshold value TS1 at all portions in a predetermined inspection range. Therefore, in the first inspection step, it is possible to extract a tentative defect range and inspect for the presence or absence of defects.

[0062] According to the inspection method of the present embodiment, the first inspection step includes extracting a range including a portion where the calculated defect probability is equal to or greater than the first threshold value and less than the second threshold value as a provisional defect range. Therefore, it is possible to reduce or prevent a portion that is equal to or greater than the second threshold value and that is determined to be defective in the first inspection step from being inspected in the second inspection step.

[0063] According to the inspection method of this embodiment, the first learning model 644 is a learning model that prioritizes processing speed over inspection accuracy, and the second learning model 744 is a learning model that prioritizes inspection accuracy over processing speed. In the first inspection process, a wider range is inspected and a provisional defective range is extracted, and in the second inspection process, inspection can be performed with high inspection accuracy. Therefore, in a plurality of inspection processes spanning the first inspection process and the second inspection process, a wider range can be inspected while reducing or preventing a decrease in inspection accuracy. In addition, since the processing speed of the first inspection process is improved by setting the processing speed of the learning model, for example, the processing speed of the first inspection process can be improved without reducing the resolution of the image data acquired in the first inspection process. In this case, for example, the resolution of the image data of the inspection target acquired in the first inspection process and the resolution of the image data of the inspection target acquired in the second inspection process can be matched, and the effect of using the inspection result of the first inspection process in the second inspection process can be improved.

[0064] According to the inspection method of this embodiment, the processing speed of the first learning model 644 is faster than the processing speed of the second learning model 744. The first inspection step allows the provisional defective range to be extracted in a wide inspection range, while inspecting the provisional defective range with high inspection accuracy. Therefore, in a plurality of inspection steps spanning the first inspection step and the second inspection step, it is possible to reduce or prevent a decrease in inspection accuracy while inspecting a wider range.

[0065] According to the inspection method of this embodiment, the first learning model 644 is a model using SSD, and the second learning model 744 is a model using R-CNN. Therefore, a general-purpose learning model can be applied to the first inspection process and the second inspection process.

[0066] According to the inspection method of the present embodiment, the object of inspection is a casting, and in the first inspection step and the second inspection step, the casting is inspected using image data of the casting. Therefore, it is possible to improve the balance between the inspection time and the inspection accuracy in the multiple inspection steps in the manufacturing process of the casting.

[0067] B. Second embodiment: 7 is a block diagram showing an internal functional configuration of a second inspection device 70b included in an inspection system 100 according to a second embodiment of the present disclosure. The inspection system 100 according to the second embodiment is similar in configuration to the inspection system 100 according to the first embodiment except that the inspection system 100 according to the second embodiment includes a second inspection device 70b instead of the second inspection device 70. The second inspection device 70b switches the inspection range and inspection conditions depending on the amount of data in the provisional defective range acquired from the first inspection device 60.

[0068] The second inspection device 70b differs from the second inspection device 70 in the first embodiment in that, in addition to the third threshold value TS3 and the second learning model 744, a data reference value DS and a third learning model 746 are further stored in the storage device 74. The data reference value DS is a threshold for determining the size of the data amount of the provisional defective range. The data amount of the provisional defective range means the data amount of the second inspection data corresponding to the provisional defective range. The data reference value DS can be set using any value. The data reference value DS is set, for example, using a data amount that allows the inspection to be completed in a time sufficiently shorter than the takt time set for the second inspection process. The data reference value DS is a threshold for determining whether the time required for the inspection of the provisional defective range is sufficiently shorter than the takt time set for the second inspection process.

[0069] In this embodiment, the processing speed of the third learning model 746 is set to be faster than the processing speed of the second learning model 744. As a result, in the second inspection process, the range other than the provisional defective range can be inspected by utilizing the time after the inspection of the provisional defective range within the takt time. Therefore, in the second inspection process, a larger inspection range can be inspected. In this embodiment, the third learning model 746 is set with the same SSD as the first learning model 644. The third learning model 746 is not limited to SSD, and other models such as Faster RCNN and Fast RCNN may be used. In addition, in the case of prioritizing inspection accuracy over enlarging the inspection range in the inspection of the range other than the provisional defective range, models such as R-CNN and Mask R-CNN may be used, and the second learning model 744 may be used instead of the third learning model 746. In this case, the third learning model 746 can be omitted.

[0070] 8 is a process diagram showing details of the second inspection process by the second inspection device 70b in the second embodiment. In step S310, the second inspection unit 724 acquires the tentative defective range transmitted from the first inspection device 60 via the tentative defective range acquisition unit 77. In step S320, the second inspection unit 724 compares the acquired data amount of the tentative defective range with the data reference value DS stored in the storage device 74. If the data amount of the tentative defective range is equal to or greater than the data reference value DS (S320: YES), the process proceeds to step S330, where only the inspection of the tentative defective range is performed. If the data amount of the tentative defective range is less than the data reference value DS (S320: NO), the process proceeds to step S340, where the inspection of the tentative defective range and the inspection of the range other than the tentative defective range are performed.

[0071] In step S330, the second inspection data acquisition unit 76 acquires second inspection data related to the provisional defective range of the product after processing by the second manufacturing equipment 32. Step S330 is similar to step S212 shown in Fig. 6, so the description will be omitted. In step S332, the second inspection unit 724 inputs the acquired image data to the second learning model 744 to calculate a second defective probability FP2. The second judgment unit 726 compares the calculated second defective probability FP2 with a third threshold value TS3 stored in the storage device 74 to judge whether or not the inspection target is defective.

[0072] In step S340, the second inspection data acquisition unit 76 acquires second inspection data of the provisional defective range of the product after processing by the second manufacturing device 32 and the range other than the provisional defective range. The second inspection unit 724 designates a location to be imaged by the second inspection data acquisition unit 76 according to the acquired provisional defective range, and designates the range other than the provisional defective range. The designated range other than the provisional defective range may be set to any range set in advance. The range other than the provisional defective range may be adjusted successively according to the amount of data of the provisional defective range. For example, when the amount of data of the acquired provisional defective range is sufficiently small with respect to the data reference value DS, all the ranges of the preset inspection range may be set as the inspection range other than the provisional defective range, and when the amount of data of the provisional defective range is close to the data reference value DS, only the inspection range with a high inspection priority among the preset inspection ranges may be set as the inspection range other than the provisional defective range. In step S342, similar to step S332, the second judgment unit 726 judges whether or not there is a defect in the provisional defective range.

[0073] In step S350, the second inspection unit 724 changes the inspection conditions. In this embodiment, the second inspection unit 724 changes the learning model for calculating the second defect probability. Specifically, the second inspection unit 724 inputs the acquired image data into the third learning model 746 instead of the second learning model 744, thereby calculating the second defect probability for each image data corresponding to a range other than the provisional defective range. When the inspection accuracy of the range other than the provisional defective range is to be increased, the inspection conditions may not be changed and step S350 may be omitted. In step S352, the calculated second defect probability of each location in the range other than the provisional defective range is compared with the third threshold value TS3 to determine whether or not there is a defect in the inspection target. In step S352, the calculated second defect probability of the range other than the provisional defective range may be compared with a threshold value different from the third threshold value TS3, and an arbitrary threshold value according to the inspection conditions changed by step S350 may be set.

[0074] In step S370, the second determination unit 726 checks whether there is a result of the determination that there is a defect in the inspection results in steps S332, S342, and S352. If it is determined that there is no defect in any of the inspection ranges (S370: NO), the second inspection process ends. If it is determined that there is a defect in at least one of the provisionally defective range and the range other than the provisionally defective range (S370: YES), the process proceeds to step S372, and the second determination unit 726 transmits a command signal to the second conveying device 42 via the second communication unit 78 to switch the conveying path to a path that outputs to the second correction process 52, and the second inspection process ends.

[0075] According to the inspection method of this embodiment, the second inspection process, when the data amount of the extracted tentative defective range is smaller than a predetermined data reference value DS, inspects the tentative defective range using the second learning model 744, and inspects the range other than the tentative defective range using the third learning model 746 having a processing speed faster than the second learning model 744. Therefore, while inspecting the tentative defective range in the second inspection process, it is possible to also inspect the range other than the tentative defective range by utilizing the excess time of the takt time set in the second inspection process.

[0076] According to the inspection method of this embodiment, the third learning model 746 is an SSD. Therefore, a general-purpose learning model can be applied to inspections of ranges other than the provisional defective range.

[0077] C. Other embodiments: (C1) In each of the above embodiments, the first manufacturing equipment 31 is a die-casting machine, and the second manufacturing equipment 32 is an equipment for performing post-processing or post-treatment of a product after casting. However, the first manufacturing equipment 31 and the second manufacturing equipment 32 are not limited to these examples, and may be various equipment such as a processing machine, a welding machine, a molding machine, or a coating machine.

[0078] (C2) The first manufacturing process and the second manufacturing process are not necessarily limited to processes successive to each other, and a process of performing another treatment may be included between the first manufacturing process and the second manufacturing process, provided that the first manufacturing process and the second manufacturing process are linked to each other. However, the number of steps between the first manufacturing process and the second manufacturing process is preferably small enough that the treatment in the first manufacturing process affects the properties of the product after the second manufacturing process, and is preferably, for example, 7 or less, and more preferably 3 or less.

[0079] (C3) In each of the above embodiments, the first inspection device 60 and the second inspection device 70 use image data of the inspection object to determine whether or not the inspection object is defective. The first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 acquire image data of the inspection object as inspection data. In contrast, the inspection data is not limited to image data of the product, and may be data related to various parameters of the product, such as temperature, dimensions, weight, color, and shape. The data used for inspection is not limited to product data, and may be data other than the product, provided that it may affect the probability of the inspection object being defective. The inspection data may be, for example, manufacturing conditions and parameters of the devices included in the manufacturing line LP, such as voltage values, current values, pressure values, temperature, displacement amounts, waveforms and changes therein, and processing time. The first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 are not limited to cameras that acquire image data, and may be various sensors such as light sensors, sound sensors, heat sensors, current sensors, voltage sensors, distance sensors, air pressure sensors, acceleration sensors, rotational speed sensors, humidity sensors, pressure sensors, and magnetic sensors. Each sensor is a sensor that acquires inspection data for calculating the defect probability of the inspection target. In addition, when an apparatus other than the first inspection apparatus 60 and the second inspection apparatus 70 can acquire inspection data, such as when the first manufacturing apparatus 31 and the second manufacturing apparatus 32 are equipped with a camera that acquires image data, the first inspection data acquisition unit 66 and the second inspection data acquisition unit 76 may be a communication device that acquires inspection data from an apparatus other than the first inspection apparatus 60 and the second inspection apparatus 70 via wireless communication or wired communication.

[0080] (C4) In each of the above embodiments, an example has been described in which the first determination unit 626 of the first inspection device 60 extracts the provisional defective range. In contrast, each unit of the second inspection device 70, such as the second inspection unit 724, may extract the provisional defective range. In this case, the first threshold value TS1 and the second threshold value TS2 are stored in the storage device 74 of the second inspection device 70. The second inspection device 70 obtains the first defective probability FP1 from the first inspection device 60 via the first communication unit 68, and compares the obtained first defective probability FP1 with the first threshold value TS1 and the second threshold value TS2 to extract the provisional defective range.

[0081] (C5) In each of the above embodiments, a portion of the first inspection data where the first defect probability is equal to or greater than the first threshold value TS1 and less than the second threshold value TS2 is extracted as a provisional defect range. In contrast, both a portion where the first defect probability is equal to or greater than the first threshold value TS1 and a portion where the first defect probability is equal to or greater than the second threshold value TS2 may be extracted as a provisional defect range. According to the inspection system 100 of this form, a portion determined to be defective by the first inspection device 60 can be re-inspected by the second inspection device 70.

[0082] (C6) In each of the above embodiments, the processing speed of the first inspection process is improved by adopting a model with a high processing speed as the first learning model 644 in the first inspection process. As a result, the inspection range of the first inspection process is made larger than the inspection range of the second inspection process. In response to this, the resolution of the inspection data acquired by the first inspection device 60 may be lowered below the resolution of the inspection data acquired by the second inspection device 70, for example, to improve the processing speed of the first inspection process and expand the inspection range of the first inspection process. In this case, the first learning model 644 and the second learning model 744 may be the same model.

[0083] (C7) If it is determined in the first inspection step that there is no provisional defective range, the second inspection step may be omitted. Alternatively, if it is determined in the first inspection step that there is no provisional defective range, in the second inspection step, for example, a predetermined inspection range may be inspected using the third learning model 746, and an inspection similar to that in the first inspection step may be performed.

[0084] (C8) In each of the above embodiments, the inspection system 100 includes two inspection devices, the first inspection device 60 and the second inspection device 70. However, for example, the inspection system 100 may be replaced with a single inspection device that integrates the functions of the first inspection device 60 and the second inspection device 70. Furthermore, instead of the inspection system 100, the inspection system may be replaced with only the second inspection device 70 that extracts a provisional defect range and inspects the inspection target. In this case, the second inspection device 70 may acquire a first defect probability from the first inspection device 60, extract a provisional defect range from the acquired first defect probability, and perform the inspection.

[0085] (C9) In each of the above embodiments, an example has been shown in which the production line LP includes the first production apparatus 31 and the second production apparatus 32. In contrast, the first production apparatus 31 may be provided in a location other than the production line LP, such as in another factory. In this case, the first inspection apparatus 60 may be provided in the production line LP, or may be provided in a location other than the production line LP, similar to the first production apparatus 31. "Inspection objects processed in the first production process" include inspection objects delivered after being processed by the first production apparatus 31 in a location other than the production line LP.

[0086] (C10) In each of the above embodiments, the first learning model 644 and the second learning model 744 are different models. In contrast, the first learning model 644 and the second learning model 744 may be the same model.

[0087] (C11) In each of the above embodiments, the first inspection device 60 includes a first learning model 644, and the second inspection device 70 includes a second learning model 744. In contrast, the first inspection device 60 and the second inspection device 70 may be any type of inspection device that does not include a learning model.

[0088] The control unit and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, the control unit and the method described in the present disclosure may be realized by a special-purpose computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described in the present disclosure may be realized by one or more special-purpose computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer.

[0089] The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention column can be appropriately replaced or combined to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0090] 31...first manufacturing apparatus, 32...second manufacturing apparatus, 41...first conveying apparatus, 42...second conveying apparatus, 51...first correction process, 52...second correction process, 60...first inspection apparatus, 61...bus, 62...CPU, 64...storage device, 66...first inspection data acquisition unit, 68...first communication unit, 70, 70b...second inspection apparatus, 71...bus, 72...CPU, 74...storage device, 76...second inspection data acquisition unit, 77...tentative defect range acquisition unit, 78...second communication unit, 100...inspection system, 624...first inspection unit, 626...first judgment unit, 644...first learning model, 724...second inspection unit, 726...second judgment unit, 744...second learning model, 746...third learning model, DS...data reference value, LP...production line

Claims

1. 1. A testing method comprising: a first inspection step of extracting a provisional defective range from a predetermined inspection range of the inspection object processed in the first manufacturing step; a second inspection step of inspecting the provisional defective area of ​​the inspection object processed in a second manufacturing step subsequent to the first manufacturing step, The first inspection step includes: calculating a defect probability for each portion of the inspection object using a first learning model for inspection data previously obtained regarding the inspection object processed in the first manufacturing process; extracting a range including a portion in which the calculated defect probability is equal to or greater than a predetermined first threshold as the provisional defect range; If there is a portion where the calculated defect probability is equal to or greater than a second threshold value that is higher than the first threshold value, the inspection object is determined to be defective; determining that the inspection object is free of defects when the defect probability is less than the first threshold value at any point in the predetermined inspection range; The second inspection step includes inspecting the provisional defect range by using a second learning model. Testing method.

2. The inspection method according to claim 1, The first inspection step includes extracting, as the tentative defective range, a range including a portion where the calculated defective probability is equal to or greater than the first threshold value and is less than the second threshold value. Testing method.

3. The inspection method according to claim 1 or 2, The first learning model is a learning model that prioritizes processing speed over inspection accuracy, The second learning model is a learning model that prioritizes inspection accuracy over processing speed. Testing method.

4. The inspection method according to any one of claims 1 to 3, The processing speed of the first learning model is faster than the processing speed of the second learning model. Testing method.

5. The inspection method according to claim 4, The first learning model is a model using SSD, and the second learning model is a model using R-CNN. Testing method.

6. An inspection method comprising: a first inspection step of extracting a provisional defective range from a predetermined inspection range of the inspection object processed in the first manufacturing step; a second inspection step of inspecting the provisional defective area of ​​the inspection object processed in a second manufacturing step subsequent to the first manufacturing step, The first inspection step includes extracting the tentative defect range by using a first learning model for the predetermined inspection range, the second inspection step includes inspecting the provisional defective range by using a second learning model when a data amount of the extracted provisional defective range is smaller than a predetermined reference value, and inspecting a range other than the provisional defective range by using a third learning model having a processing speed faster than that of the second learning model. Testing method.

7. The inspection method according to claim 6 , wherein the third learning model is a model using SSD.

8. The inspection method according to any one of claims 1 to 7, The inspection object is a casting, The first inspection step and the second inspection step include inspecting the casting using image data obtained by capturing an image of the casting. Testing method.

9. An inspection device comprising: a first determination unit that extracts a provisional defective range from a predetermined inspection range of the inspection object processed in the first manufacturing process; a first inspection unit that calculates a defect probability for each portion of the inspection object by using a first learning model for inspection data previously obtained regarding the inspection object processed in the first manufacturing process; a second inspection unit that inspects the provisional defective range of the inspection object processed in a second manufacturing process that is subsequent to the first manufacturing process, using a second learning model; The first determination unit, extracting a range including a portion where the calculated defect probability is equal to or greater than a first threshold value as the provisional defect range; If there is a portion where the calculated defect probability is equal to or greater than a second threshold value that is higher than the first threshold value, the inspection object is determined to be defective; determining that the inspection object is free of defects when the defect probability is less than the first threshold value at any point in the predetermined inspection range; Inspection equipment.

Citation Information

Patent Citations

  • Defect detection method of display device and defect detection device of display device

    JP2013033061A

  • Plate-like body defect inspection method and defect inspection device

    JP2014044063A

  • Inspection device and inspection method

    JP2016080686A

  • Classification device, classification method, classification program, and inspection device

    JP2020042754A

  • Sheet defect inspection device

    JP2020106295A