Quality control system, quality control method, quality control program, and information processing device

Abstract

To provide a quality control system that is easy to improve the accuracy of judgement relating to the quality of a product.SOLUTION: A quality control system includes: database that stores photographed image data relating to a photographed image of a steel pipe in a manufacturing process, identification information added to the steel pipe, and photographing time information of the steel pipe, in association with one another; a quality judgement unit 40 that judges quality of the steel pipe by using the photographed image data; an acquisition unit 52 that, when a defect was found later in a steel pipe in which no defect had found by the quality judgement unit 40, acquires identification information of the steel pipe with the defect found later; a specifying unit 54 that specifies the photographed image data from the database, based on the identification information acquired by the acquisition unit 52 and the photographing time information; and a learning unit 56 that causes the quality judgement unit 40 to learn whether quality is good or bad, by using the photographed image data specified by the specifying unit 54 as teacher data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a quality control system, a quality control method, a quality control program, and an information processing device for products such as steel pipes. [Background technology]

[0002] Patent Document 1, listed below, discloses a technology for inspecting an object based on an image captured by an imaging device such as a camera. The imaging processing device disclosed in Patent Document 1 uses artificial intelligence. The imaging processing device disclosed in Patent Document 1 performs artificial intelligence training (AI training) in advance using good and bad objects before carrying out the actual inspection process (paragraph 0027, etc.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-164272 Summary of the Invention [Problem to be solved by the invention]

[0004] In an inspection device or inspection system that performs AI learning as described in Patent Document 1, generally, if many types of training data can be used for AI learning, the accuracy of judgment (the judgment accuracy of the AI, and the judgment accuracy of the inspection device or inspection system that uses AI) can be improved. However, there is a limit to the amount of training data that can be collected before the inspection device or inspection system starts operating, and it is difficult to achieve high judgment accuracy from the beginning of operation of the inspection device or inspection system.

[0005] Furthermore, even after the start of operation of an inspection device or inspection system, defective products that were not detected by the inspection device or inspection system may be discovered later, for example, by visual inspection by a quality control officer. In such cases, it is possible to improve the accuracy of judgments by adding training data and training the AI. However, to ensure efficient and easy AI training, it is necessary to ensure the quality of the training data. Furthermore, in order to obtain high-quality training data, it is undesirable to include information that may become noise, for example.

[0006] The present invention has been made in consideration of the above-mentioned points, and its purpose is to provide a quality control system, a quality control method, a quality control program, and an information processing device that make it easy to improve the accuracy of judgments regarding product quality. [Means for solving the problem]

[0007] The quality control system according to the present invention is characterized by the following: During the manufacturing process Identification information attached to the product, photographing time information of the product, and During the manufacturing process a database that stores addition time information indicating the time when the identification information was added to the product in association with the product; a quality determination unit that determines the quality of the product using the captured image data; an acquisition unit that acquires, when a defect is subsequently discovered in the product that was not found to be defective by the quality determination unit, the identification information of the product in which a defect is subsequently discovered; The identification information and the shooting time information acquired by the acquisition unit , and further the additional time information an identification unit that identifies the captured image data from the database based on the a learning unit that causes the quality determination unit to learn quality determination using the photographed image data identified by the identification unit as training data, The identification unit identifies the photographed image data from the database by identifying the photographed time information of the product in which a defect was subsequently discovered, based on the added time information. [Effects of the Invention]

[0008] It is possible to provide a quality control system, a quality control method, a quality control program, and an information processing device that can easily improve the accuracy of judgments regarding product quality. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram for explaining an overview of a quality control system according to an embodiment of the present invention; [Figure 2] FIG. 1 is a diagram for explaining an overview of a product manufacturing system. [Figure 3] FIG. 2 is a diagram for explaining a manufacturing process of a steel pipe. [Figure 4] FIG. 2 is a diagram illustrating an example of a management record configuration in a management database. [Figure 5] (a) is a diagram illustrating an example of a photographed image of a good product, (b) is a diagram illustrating an example of a photographed image of a defective product, and (c) is a diagram illustrating an example of a photographed image that contains a shadow image to the extent that it is not judged to be a defective product. [Figure 6] FIG. 2 is a diagram for explaining an outline of the functions of a quality control system. [Figure 7] FIG. 1 is a diagram for explaining an outline of a processing procedure in a quality control system. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Overview of Quality Control System 1> An embodiment of the present invention will be described below with reference to the drawings. Fig. 1 shows a simplified configuration of a quality control system 1 according to one embodiment of the present invention. This quality control system 1 includes a product manufacturing system (hereinafter referred to as a "manufacturing system") 10 that manufactures products, a quality determination unit 40 that uses artificial intelligence (AI) to determine whether a product is good or defective, and an optimization unit 50 that improves the determination accuracy of the quality determination unit 40.

[0011] The manufacturing system 10, the quality determination unit 40, and the optimization unit 50 can communicate via a communication network CN as shown in FIG. 1. Examples of the communication network CN include the Internet, a LAN, a WAN, a public telephone line, a base station, a mobile communication network, and a network interconnected via a gateway (including a so-called cloud). Although not shown, the manufacturing system 10, the quality determination unit 40, and the optimization unit 50 each have a communication I / F unit. The manufacturing system 10, the quality determination unit 40, and the optimization unit 50 communicate data via the communication I / F unit in accordance with a predetermined communication protocol. The configurations of the manufacturing system 10, the quality determination unit 40, and the optimization unit 50 are described below.

[0012] <Manufacturing System 10> First, in this embodiment, a steel pipe will be described as an example of a product. Fig. 2 schematically shows a partial configuration of a manufacturing system 10. The manufacturing system 10 manufactures a steel pipe 14 from a steel strip 12.

[0013] In the manufacturing system 10, a strip-shaped steel strip 12 wound in a roll (coil) is drawn out onto a conveying path 13. The drawn steel strip 12 undergoes a predetermined pre-treatment (continuous process), is cut to a predetermined length, and is formed into a cylindrical shape to become a steel pipe 14. Here, the material to be made into the steel pipe 14 in the continuous process may be the steel strip 12, or may be an iron or non-ferrous element or composite. Furthermore, the material used here may be coated with an organic or inorganic substance.

[0014] On the conveying path 13, there are arranged devices used in the continuous process, such as a forming machine 16, a welding machine 17, a photographing device 18, a marking device 19, a reading device 20, and a cutting machine 21. Of these, the forming machine 16 rolls up the strip-shaped steel strip material 12 moving on the conveying path 13 and forms it into a tubular shape, as shown in Figures 2 and 3. In Figure 3, the steel strip material 12 is rolled up so that the edges (side edges) along the longitudinal direction of the steel strip material 12 are butted together.

[0015] Although not shown, the forming machine 16 may be, for example, a machine that forms a steel pipe (spiral steel pipe) by spirally rolling the steel strip 12 into a tubular shape. The conveying path 13 may have a total length of, for example, about 100 to 120 m. The length of the conveying path 13 is not limited to this length and can be changed in various ways.

[0016] The welding machine 17 welds the edges of the tubular steel strip 12. Although not shown in Fig. 2, other pre-processing devices may be added, such as a heating device for heating the steel strip 12 and a bead cutting machine for cutting the bead formed by welding.

[0017] The photographing device 18 photographs the steel pipes 14 flowing on the conveying path 13 from above. Although not shown, the photographing device 18 is equipped with a camera, lighting equipment, etc., and these cameras and lighting equipment are installed at fixed positions above the conveying path 13. The photographing device 18 can photograph the steel pipes 14 on the conveying path 13 using the camera at a fixed position, distance, angle, magnification, and range (field of view).

[0018] Here, the camera 18 may be one that takes moving images or one that takes still images at predetermined timings. The camera position may be directly above the conveying path 13 (or steel pipe 14) (directly above the center of the conveying path 13 in the width direction) or diagonally above. Here, "diagonally above" means a position where the optical axis (viewing axis) of the camera is directed from a position directly above the conveying path 13 to the upstream side (or downstream side) of the conveying path 13 and diagonally downward.

[0019] The photographing range of the photographing device 18 may be adjusted so that the entire steel pipe 14 that would make up one piece of steel is captured in a single image, or may be adjusted so that the entire steel pipe 14 that would make up one piece of steel is photographed in multiple separate images.

[0020] The location of the photographing device 18 is not limited to the downstream side of the welding machine 17, and the photographing device 18 can be provided in other positions. For example, the photographing device 18 can be incorporated into the marking device 19 to photograph the steel pipe 14 during marking. The photographing device 18 can also be incorporated into the cutting machine 21 described below to photograph the steel pipe 14 during cutting.

[0021] As will be described later, the images (photographed images) taken by the photographing device 18 are used as target data for AI judgment by the pass / fail judgment unit 40 (FIG. 1) and as teacher data (which may also be referred to as "learning data") for subsequent optimization of the pass / fail judgment unit 40. For this reason, in order to improve the accuracy of AI judgment, it is desirable that the photographed images be as high-resolution as possible within the allowable range.

[0022] Furthermore, when the steel pipe 14 is photographed, information on the time when the photograph was taken (photographing time information, which is information on the time of photographing) is linked to the data of the photographed image (photographed image data) and stored (saved). The use of this photographing time information will be described later.

[0023] Next, the marking device 19 sequentially marks identification information on the steel pipes 14 that have passed through the photographing device 18. The identification information added to the steel strip material 12 by marking is unique information with different content. The identification information indicates different content for each steel pipe 14.

[0024] As the identification information, various information including a serial number can be used as long as it has identifiability and uniqueness. In this embodiment, the identification information includes at least lot number information (lot number information). The lot number here is information that can identify each steel pipe 14. When the identification information is marked on the steel pipe 14, the time information of the time stamp (added time information) is linked to the identification information and stored (saved). The use of this added time information will be described later.

[0025] As the marking device 19 that adds the identification information to the steel pipe 14, for example, a laser marker or an engraving machine that stamps a large number of dots can be used. In addition, as the marking device 19, for example, a device that prints with ink or attaches a label can also be used. However, by using a laser marker or an engraving machine for marking, the durability and weather resistance of the identification information are improved compared to when printing with ink or attaching a label.

[0026] Furthermore, the display format of the identification information may be symbols, numbers, etc. displayed as they are, or may be designed as a two-dimensional barcode, a QR code (registered trademark), a dot pattern, or the like.

[0027] Here, the marking device 19 can reciprocate a marking head (not shown) on an axis arranged parallel to the conveying path 13. The marking device 19 performs a non-contact marking process on a predetermined position of the steel strip 12 while moving the head (not shown) so that the relative speed with respect to the steel strip 12 moving on the conveying path 13 becomes zero.

[0028] The "predetermined position of the steel strip material 12" referred to here can be determined as a position at a certain distance from one corner of one end of the axial direction of the steel pipe 14 (such as one end downstream in the conveying direction) as a reference position (origin). The size of the area to which the identification information is added by marking can be, for example, about several centimeters square.

[0029] For example, if the marking capability of the marking device 19, such as the marking speed, cannot keep up with the moving speed of the steel strip 12 on the conveying path 13, multiple marking devices 19 may be installed. This ensures that the marking capability can keep up with the moving speed of the steel strip 12, even if the moving speed is high.

[0030] The identification information marked on the steel pipe 14 is read by a reading device 20 arranged after the marking device 19. The reading device 20 is equipped with a sensor (not shown) capable of reading identification information, and reads the identification information attached to the steel pipe 14 in a non-contact manner. The steel pipe 14 may or may not be subjected to a surface treatment such as plating on its outer circumferential surface. As the reading device 20, for example, a camera, a barcode reader, etc. can be used.

[0031] The identification information read by the reading device 20 is transmitted to a manufacturing management device 23 and registered in a manufacturing management database (hereinafter referred to as "database") 24. The manufacturing management device 23 comprehensively controls various devices involved in the continuous process, such as the forming machine 16, welding machine 17, photographing device 18, marking device 19, reading device 20, and cutting machine 21. In addition to the identification information, various types of manufacturing management information are registered in the database 24, and this manufacturing management information is used for various purposes related to the manufacture and shipment of the steel pipe 14. The information registered in the database 24 will be described later.

[0032] The cutting machine 21 cuts the steel strip material 12 that has been subjected to a predetermined processing to a predetermined length. In this way, steel pipes 14 are produced one by one from the continuous steel strip material 12. The cutting machine 21 can produce steel pipes 14 of a variety of lengths according to input instructions. For example, if the length of the shortest steel pipe 14 is l (lowercase letter L), the cutting machine 21 can cut the steel pipe 14 to lengths that are integer multiples of l (l, 2l, 3l, 4l, 5l, etc.). Furthermore, the length of the steel pipe 14 is not limited to integer multiples of the reference length l.

[0033] 4 shows an example of the configuration of management records in the database 24. The database 24 is made up of a large number of management records. In the database 24, one management record is provided corresponding to one piece of identification information. One management record is made up of fields such as "photographed image file name," "photographing time information," "identification information," "timestamp," "product information," "quality information," and "shipping information."

[0034] Among the various fields, the "photographed image file name" field is filled in with the file name of the image captured by the photographing device 18 (photographed image file name). The photographed image file name is the file name of a still image. If the photographing device 18 is for capturing moving images, photographed image data relating to still images at regular time intervals, for example, is extracted from the moving image, and each photographed image data is assigned a consecutive photographed image file name.

[0035] Furthermore, if the photographing device 18 is one that photographs still images, each still image obtained by photographing is given a consecutive photographed image file name. Whether the photographing device 18 is one that photographs moving images or one that photographs still images, the images photographed by the photographing device 18 are used for AI judgment by the pass / fail judgment unit 40 and for subsequent optimization of the pass / fail judgment unit 40 by the optimization unit 50. The AI ​​judgment by the pass / fail judgment unit 40 and the optimization of the pass / fail judgment unit 40 will be described later.

[0036] Here, the captured image file name may include the above-mentioned shooting time information. Furthermore, the shooting time information can include the year, month, day, hour, minute, and even second. In FIG. 4, numerical values ​​such as "210326150006" and "210326150004" are registered as "shooting time information." These numerical values ​​indicate that the captured image data of the corresponding "captured image file name" was captured at 15:00:06 on March 26, 2021, or 15:00:04 on March 26, 2021, respectively.

[0037] In this embodiment, the photographing device 18 photographs at intervals of two seconds or more. Therefore, by including the year, month, day, hour, minute, and second in the "photographing time information," different photographing time information can be assigned without overlapping. However, this is not limited thereto, and for example, the photographing time information may include information on decimal points related to seconds. In this way, even if photographed image data is acquired at short time intervals of less than one second, different photographing time information can be assigned without overlapping.

[0038] Next, the above-mentioned identification information is entered in the "identification information" field. As mentioned above, one (one type of) "identification information" corresponds to each steel pipe 14. Furthermore, as mentioned above, information on the time when the identification information marking was performed (added time information) is entered in the "timestamp" field. The added time information indicated by this timestamp includes the year, month, day, hour, minute, and second.

[0039] In Figure 4, numbers such as "210326150016" and "210326150014" are registered as additional time information indicated by the "time stamp." These numbers indicate that the photographed image data of the corresponding "Photographed Image File Name" is of steel pipe 14 that was marked at 15:00:14 on March 26, 2021, or 15:00:16 on March 26, 2021, respectively.

[0040] In this embodiment, the added time information indicated by the "time stamp" includes the year, month, day, hour, minute, and second, so that different added time information can be added without overlapping. However, this is not limited to this, and for example, the time information may include information on decimal points related to seconds. In this way, even if marking is performed at short time intervals of less than one second, different added time information can be added without overlapping.

[0041] Next, in the "product information" field, process information relating to each manufacturing process, or product information (such as manufacturing instruction number) showing details of the steel pipe 14 that has been manufactured through the manufacturing process, is entered. In addition, in the "quality information" field, quality information of the steel pipe 14 (such as information on the material of the product and the product group identification number) is entered. Furthermore, in the "shipping information" field, shipping information when the steel pipe 14 is shipped (such as information on the product group identification number and product bundle identification number) is entered.

[0042] <Good / bad judgement unit 40> Next, we will explain the quality determination unit 40 mentioned above. The quality determination unit 40 has the function of using artificial intelligence (AI) to determine the quality of the steel pipe 14 manufactured by the manufacturing system 10, and inspecting the steel pipe 14. Although not shown in the figure, the quality determination unit 40 can be a general computer device equipped with a control unit, a memory unit, a communication I / F (communication interface) unit, etc.

[0043] The quality determination unit 40 may be configured with one computer device or multiple computers with distributed functions. Furthermore, the various processes executed by the quality determination unit 40 may be executed in a distributed manner by one or multiple processors. Furthermore, the quality determination unit 40 may include an AI processing unit (AI analysis engine) that is hardware for AI-based determination processing.

[0044] AI learning is performed in the quality judgment unit 40. The quality judgment unit 40 judges the quality of the steel pipe 14 based on judgment criteria obtained by AI learning, and distinguishes between steel pipes 14 that have no defects and steel pipes 14 that have defects. For AI learning, photographed images of the steel pipes 14 are used as training data.

[0045] Figures 5(a) and (b) show schematic examples of captured images of a steel pipe 14. Figure 5(a) shows an example of a captured image of a non-defective product, and Figure 5(b) shows an example of a captured image of a defective product. In the defective steel pipe 14 shown in Figure 5(b), pinholes have occurred as shown in black, and a pinhole image 46 appears in the captured image. In contrast, in the non-defective steel pipe 14 shown in Figure 5(a), the pinhole image 46 does not appear.

[0046] In the AI ​​learning of the quality judgment unit 40, photographed image data in which the pinhole image 46 appears is used as training data for defective products, and photographed image data in which the pinhole image 46 does not appear is used as training data for non-defective products. After the quality judgment system using the quality judgment unit 40 starts operating, the image analysis results in determining that an image in which the pinhole image 46 appears is an image of a defective product.

[0047] Here, AI learning may be performed using only teacher data of defective products, without using teacher data of non-defective products. Furthermore, various AI learning methods can be adopted as long as they enable discrimination between non-defective products and defective products. Examples of AI learning methods that can be adopted include statistical machine learning and deep learning. In this embodiment, supervised learning using teacher data is performed.

[0048] In AI learning, training data is given as the results or correct answers, and regression models and classification models are constructed. Typical supervised learning analysis methods include regression analysis and decision trees. In regression analysis, the relationship between the dependent variable and the explanatory variables is quantitatively analyzed, and predictions are made based on the analysis results. In decision trees, criteria (boundaries) for classification are learned, and data is classified in unknown situations.

[0049] The aforementioned deep learning is based on neural networks. Of the three layers in a neural network (input layer, intermediate layer, and output layer), the intermediate layer that performs weighting and transformation can be one layer. It is also possible to have two or more intermediate layers to make it deeper. Neural networks and deep learning enable tasks such as image recognition to be performed more efficiently.

[0050] <Optimization Unit 50> Next, we will explain the optimization unit 50 mentioned above. The optimization unit 50 has the function of causing the quality judgment unit 40 to perform AI learning and giving judgment capabilities to the quality judgment unit 40. Although not shown in the figure, the optimization unit 50 can be a general computer device equipped with a control unit, a storage unit, a communication I / F (communication interface) unit, etc.

[0051] The optimization unit 50 may be configured as a single computer device or multiple computers with distributed functions. Furthermore, the various processes executed by the optimization unit 50 may be executed in a distributed manner by one or multiple processors.

[0052] The optimization unit 50 causes the quality determination unit 40 to perform AI learning before the manufacturing system 10 is put into operation. In addition, the optimization unit 50 has a function of causing the quality determination unit 40 to perform AI learning after the manufacturing system 10 is put into operation, thereby updating the AI ​​function of the quality determination unit 40.

[0053] With regard to post-facto AI learning, the optimization unit 50 uses training data related to defective products discovered after the manufacturing system 10 is put into operation, and causes the quality judgment unit 40 to update information for detecting defective products. The process for causing the quality judgment unit 40 to perform post-facto AI learning in this way is performed using the AI ​​functions of the optimization unit 50, based on information input from outside (defective product classification and lot number information, which will be described later). The optimization unit 50 automatically performs post-facto optimization of the quality control system 1.

[0054] 1, the optimization unit 50 may include an acquisition unit 52, an identification unit 54, and a learning unit 56. Details will be described later, but when a defect is subsequently discovered in a steel pipe 14 in which no defect was found by the quality determination unit 40, the acquisition unit 52 acquires identification information of the steel pipe 14 in which a defect was subsequently discovered.

[0055] The identifying unit 54 identifies the photographed image data from the database 24 based on the identification information acquired by the acquiring unit 52 and photographing time information relating to the time when the steel pipe 14 was photographed. The learning unit 56 uses the photographed image data identified by the identifying unit 54 as training data and causes the quality judgment unit 40 to learn how to judge quality.

[0056] The acquisition unit 52, the identification unit 54, and the learning unit 56 are configured using a control unit, a memory unit, a communication I / F, etc. provided in the optimization unit 50. The acquisition unit 52, the identification unit 54, and the learning unit 56 are conceptual functional units categorized as functions that the control unit of the optimization unit 50 executes by using a predetermined memory area of ​​the memory unit, a communication I / F, etc., in accordance with a control program stored in the memory unit.

[0057] <Automatic optimization function of quality control system 1> 6 shows a schematic diagram of the automatic optimization function in the quality control system 1. The left side of the figure shows a block of the manufacturing system 10, and the center of the figure shows a block of the system software (quality control program, designated by reference numeral 60) employed in the quality control system 1. Furthermore, the right side of the figure shows a block of the optimization system (designated by reference numeral 70) having the optimization unit 50.

[0058] Here, the term "system software" is used to mean software for comprehensively managing the AI ​​judgment by the quality judgment unit 40 and updates to the functions of the quality judgment unit 40. This system software can be, for example, software that can comprehensively control the manufacturing system 10, the quality judgment unit 40, and the optimization unit 50.

[0059] By running the system software, it is possible to display a dashboard (display on the operation panel) on various terminals, import necessary information from the manufacturing system 10, and have the quality judgment unit 40 perform AI learning. The system software is equipped with functions that enable the import of information and operation input for using AI.

[0060] Under control via the system software, the manufacturing system 10, the quality judgment unit 40, and the optimization unit 50 work together to improve the judgment accuracy of the quality judgment unit 40. Furthermore, application software (application programs) can be added to the system software as needed for specific purposes.

[0061] 6, the manufacturing system 10, the system software block 60, and the optimization system block 70 are shown separately. However, functionally, it is also possible to show at least a part of the manufacturing system 10 or at least a part of the optimization system block 70 as being included in the system software block 60.

[0062] The functions of the quality control system 1 can be broadly divided into a quality judgment function executed by the quality judgment unit 40 and an optimization function executed by the optimization unit 50. In Fig. 6, the flow of information related to the quality judgment function is represented by a solid line, and the flow of information related to the optimization function is represented by a dashed dotted line.

[0063] Regarding the quality determination function represented by the solid line, as shown in the upper left part of Fig. 6, the data of the captured image (captured image data) and lot number information are stored in the database 24 together with added time information in the form of a time stamp from the manufacturing system 10. These processes correspond to steps S10 to S30 in Fig. 7.

[0064] As mentioned above, the photographed image data referred to here is image data photographed and acquired by the photographing device 18 in the manufacturing system 10. As mentioned above, the lot number information is also information contained in the identification information read by the reading device 20 in the manufacturing system 10. Furthermore, as mentioned above, the added time information is also information about the time when the identification information was marked on the steel pipe 14.

[0065] 6, the database 24 is shown in the system software block 60, not in the manufacturing system 10 block. The database 24 may be configured as a single hardware storage device or multiple storage devices. Although not shown, a database for captured images (image database) may be provided separately from the database 24. The captured image data, lot number information, and time stamp (added time information) may be stored in this image database as information linked to each other.

[0066] In the pass / fail judgment function, the pass / fail judgment unit 40 performs image analysis on the captured image (step S40 in FIG. 7). FIGS. 5(a) to 5(c) show schematic examples of captured images. As mentioned above, FIG. 5(a) is a captured image of a non-defective steel pipe 14 in which no defects are visible. FIG. 5(b) is a captured image of a defective steel pipe 14 in which a defective portion (e.g., a portion where a pinhole image 46 appears) is shown shaded in black.

[0067] Furthermore, in FIG. 5(c), although the image is not as clear as the defective portion (the portion where pinhole image 46 appears) shown in FIG. 5(b), a shadow image 48 of a shaded portion (e.g., a portion where plating defects have occurred) is shown, indicated by diagonal lines. Plating defects are defects that occur when, for example, the zinc film is not formed to a uniform thickness. Plating defects appear, for example, as stripes when zinc plating is performed by so-called hot dip galvanization. The image analyzed by the quality determination unit 40 may be subjected to predetermined processing (e.g., binarization processing, gradation processing, etc.).

[0068] If the photographed image shows a defective part (here, a pinhole image 46) as shown in FIG. 5(b), the quality determination unit 40 determines that a defective product has been produced. In the quality determination function, when the quality determination unit 40 determines that a defective product has been produced, an alert is issued to the manufacturing system 10. Note that by providing the quality determination unit 40 with an image analysis engine that has a function (specific function) specialized for AI determination by image analysis, the AI ​​determination can be made faster.

[0069] Here, the level of AI learning of the quality judgment unit 40 at a certain stage is assumed to be such that if a defective part (here, pinhole image 46) as shown in Figure 5(b) is detected, a judgment is made that a defect has occurred, but even if a shadow image 48 as shown in Figure 5(c) is detected, a judgment that a defect has occurred is not made. In a situation where the quality judgment unit 40 has such judgment ability, if a plating defect occurs on the steel pipe 14, a judgment that a defective product has occurred is not made, and an alert is not issued for the steel pipe 14 with the plating defect.

[0070] A steel pipe 14 having a plating defect as described above is discovered after the fact, for example, through an inspection by a quality control officer in the manufacturing system 10 (step S70 in FIG. 7). In the quality control system 1, a person who discovers a defective product after the fact (defect discoverer, not shown) registers the type of defect discovered (defect classification) and the lot number of the corresponding steel pipe 14 in the system software via a terminal device or the like at hand, as shown by the flow of information in the dashed dotted line in the lower left of FIG.

[0071] Examples of the terminal device include a computer connected to the manufacturing system 10, a mobile terminal (including a smartphone or tablet terminal), or a control operation panel. Further examples of the terminal device include a device equipped with a camera or barcode reader that reads identification information (including lot numbers), such as the reading device 20, and a voice input device (such as a microphone) that allows a defect finder to input the defect discovery by voice. Furthermore, the system software may be configured such that the functions required for the terminal device are provided from a server device (not shown). The server device may be configured as a single computer device or multiple computers with distributed functions. Furthermore, the computer device that constitutes the quality determination unit 40 may have the functions of a server device.

[0072] The defective product classification information entered by the defect finder is sent to the optimization unit 50 via the system software and is used for AI learning to improve the judgment accuracy in the pass / fail judgment unit 40, as will be described later. The lot number information is used to search for photographed image data of defective products from the large number of photographed image data stored in the database 24. The defective product classification information and lot number information sent to the optimization unit 50 are acquired by the acquisition unit 52.

[0073] The captured image data is searched for by calculating (back-calculating) the capture time using the timestamp corresponding to the lot number, the conveying speed (line speed) of the conveying path 13, and information on the camera distance. More specifically, if the conveying speed is V [m / min], the camera distance as the capture distance is L [m], the timestamp value (indicating the time when the identification information was added) is T1, and the capture time is T2, the capture time T2 can be calculated using the following formula. T2=T1-L / V

[0074] Here, the conveying speed V [m / min] is the conveying speed of the conveying path 13 at the time corresponding to the timestamp value (the time when the identification information was added) T1. For example, information on the conveying speed set for the manufacturing system 10 is stored in the database 24. When the captured image data is searched, information on the conveying speed set for the period including the time when the identification information was added T1 is read from the database 24 and used to calculate the capture time T2.

[0075] The camera distance L [m] is the distance between the photographing position of the photographing device 18 (the photographing position on the conveying path 13) and the position (marking position) where the identification information is marked on the steel pipe 14 by the marking device 19. This camera distance L [m] is also the movement distance of the steel pipe 14 along the conveying path of the conveying path 13. Furthermore, when the photographing device 18 and the marking device 19 are integrated into a single device, the camera distance L [m] is the distance between the photographing position and the marking position within that device.

[0076] The camera distance L [m] can also be stored in the database 24. If the camera distance L [m] is changed, the information on the camera distance L [m] that was set during the period including the time T1 at which the identification information was added is read from the database 24 and used to calculate the shooting time T2.

[0077] As shown in the above formula, the photographing time T2 can be obtained by subtracting the time it took for the camera to travel (move) the camera distance L at the conveying speed V from the addition time T1, which is the value of the timestamp. For example, if the conveying speed V=60 [m / min], the camera distance L=10 [m], and the timestamp addition time is 15:24:30, it can be determined that the photograph of the steel pipe 14 was taken at 15:24:20.

[0078] Based on the photography time thus obtained, the identification unit 54 calculates the photography time T2 based on the timestamp value (the time at which the identification information was added) T1, and searches for photographed image data of defective products from the photographed image data in the database 24. The identification unit 54 identifies and extracts one photographed image data linked to the corresponding photography time information (step S60 in FIG. 7). The extracted photographed image data is sent to the optimization unit 50. The learning unit 56 of the optimization unit 50 uses the received photographed image data as training data, links it to the aforementioned defective product classification, and causes the quality judgment unit 40 to perform AI learning (step S70 in FIG. 7).

[0079] As a result, the quality determining unit 40 will determine, through image analysis, even defective products of a classification that had not previously been determined to be defective (here, defects that appear as shadow images 48). Alternatively, the quality determining unit 40 will determine an increased proportion of steel pipes 14 that appear as shadow images 48 as defective.

[0080] By performing this kind of ex-post AI learning each time a defective product is discovered in the manufacturing system 10, the AI ​​function of the quality judgment unit 40 is updated. The judgment accuracy of the quality judgment unit 40 is continuously improved, and the quality control capability of the manufacturing system 10 is gradually improved.

[0081] In this embodiment, to avoid complicating the explanation, only two types of defective product classifications (pinholes and plating defects) are exemplified. However, in reality, several tens of types (approximately 30 to 50 types) of defective product classifications are defined. Furthermore, the conveying speed V is not limited to 60 [m / min] and can be changed to various values. For example, the conveying speed V may be set to any value between 40 and 120 [m / min] (such as 100 [m / min] or 120 [m / min]). Furthermore, in FIG. 6, "good / bad" judgment, "alert," "feedback to manufacturing site," and "AI optimization" are enclosed by solid line blocks and indicate the respective processes.

[0082] <Effects of the invention related to quality control system 1> According to the quality control system 1 of this embodiment described above, even if there is a type of defect that cannot be determined as a defective product at a certain point in time in the quality determination unit 40, if an oversight is discovered, new training data is searched for. Then, using the searched training data, the optimization unit 50 updates the function of the quality determination unit 40 to the latest state. When an oversight of a defective product is discovered, the defect classification and lot number are manually registered, but thereafter, photographed image data corresponding to the defective product is automatically searched for.

[0083] Therefore, it is possible to eliminate human judgment as much as possible and sequentially perform optimization of the quality judgment section 40 in an unmanned manner. It is also easy to automatically improve the judgment accuracy of the quality judgment section 40. Furthermore, it is possible to efficiently perform learning of the artificial intelligence in the quality judgment section 40 with less labor.

[0084] In the past, photographed image data was not searched using the methods described above. Therefore, when a defective product that had escaped inspection was discovered, new training data for the defective product had to be created. Then, a quality control officer or the like had to photograph the defective part of the defective product and use the photographed image data as training data to train the quality judgment unit 40 on AI.

[0085] Furthermore, in AI learning, the quality of the training data can be standardized and made uniform by reproducing the same lighting conditions and the same shooting angle as the images captured by the imaging device 18 on the conveyance path 13 of the manufacturing system 10 and taking images under the same shooting conditions. By making the quality of the training data uniform, it becomes easier to obtain highly accurate judgment results from the quality judgment unit 40. However, for defective products discovered after the fact, it is not easy to reproduce the shooting conditions of the imaging device 18 and take images, as it requires a lot of time and effort.

[0086] Furthermore, the timing at which defects are discovered after the fact varies and is not uniform, for example, after the product is completed, during preparation for shipment, before shipment, after delivery to the customer, etc. For this reason, it is difficult to arrange the photographing device 18 in advance to anticipate the occurrence of defective products after the fact. Also, defective products may be discarded before they are photographed, so it is not always possible to photograph defective products after the fact.

[0087] On the other hand, when a defective product is discovered after the fact, it is possible to search for existing photographed image data in the database 24 and use it as training data, rather than taking new photographs. In this case, however, the worker must look at the screen and search the database for the photographed image data of the corresponding defective product based on the quality information and shipping information mentioned above.

[0088] Furthermore, when there are multiple pieces of photographed image data corresponding to one steel pipe 14, the defective portion does not necessarily appear in all of the photographed image data. For this reason, the worker had to visually check the screen to find the photographed image data that shows the defective portion from the multiple pieces of photographed image data corresponding to one steel pipe 14.

[0089] However, according to the quality control system 1 of this embodiment, as described above, the manufacturing system 10 continuously photographs the steel pipes 14 during production. Furthermore, the acquired photographed image data is stored in the database 24, linked to photographing time information, lot numbers, and time stamps (added time information). If a defective product is discovered after the fact, although it is necessary to input the lot number, etc., the photographed images are then automatically found, and the found photographed image data is used as training data.

[0090] Therefore, when performing post-event AI learning, there is no need to manually search for captured image data, making it easy to search for captured image data. Furthermore, there is no need to take new photographs to create captured image data, and there is no need for the time-consuming task of reproducing the photographing conditions and taking photographs. Furthermore, the quality judgment unit 40 can be updated as automatically as possible, making it easier to improve judgment accuracy.

[0091] Furthermore, even if the identification information displayed on the steel pipe 14 does not include information that identifies one piece of photographed image data, one piece of image data can be identified from the identification information. Therefore, one piece of image data can be identified without increasing the amount of information in the identification information. The reason for this is as follows.

[0092] The identification information represents information necessary for product manufacturing and sales in a limited area with a limited amount of information. As mentioned above, the size of the area to which the identification information is added by marking can be, for example, several centimeters square. Furthermore, if the identification information directly includes shooting time information, the amount of information represented by the identification information increases.

[0093] Therefore, the amount of information that can be displayed by the identification information is restricted by the information on the shooting time, and is reduced by the amount of the information on the shooting time. Alternatively, as the amount of information displayed by the identification information increases, the range of the display area for the identification information must be expanded. However, in this embodiment, the shooting time is identified based on the identification information, so one image data can be identified without increasing the amount of information in the identification information.

[0094] In the quality control system 1 of this embodiment, the AI ​​of the pass / fail judgment unit 40 can be referred to as the "local-side AI" or the "on-site-side AI." The AI ​​of the optimization unit 50 can be referred to as the "learning-side AI." Furthermore, when only the optimization unit 50 is connected to the cloud, the AI ​​of the optimization unit 50 can be referred to as the "cloud-side AI."

[0095] 6, the information flow indicated by the solid line can be referred to as the "information flow under normal circumstances." The information flow indicated by the dashed-dotted line can be referred to as the "information flow when a defective product cannot be determined based on the information flow indicated by the solid line (when a defective product is discovered after the fact)." According to the quality control system 1 of this embodiment, it is possible to find defective products and update the judgment ability of the quality judgment unit 40 using image data captured in the same environment in both the solid line and dashed-dotted line cases.

[0096] As described above, improving the judgment accuracy of the quality judgment unit 40 is also important for improving manufacturing yield. For example, if a good product is mistakenly judged to be defective, some products will not be shipped even though they should be. The yield can be calculated, for example, as (product weight) ÷ (input weight). Therefore, if some products are not shipped even though they should be shipped, the "product weight" will be unnecessarily reduced, and the yield value will be correspondingly smaller. Therefore, improving the judgment accuracy of the quality judgment unit 40 is important for improving productivity.

[0097] Furthermore, according to the quality control system 1 of this embodiment, the number of captured image data items (number of images) that can be stored in the database 24 is determined by the relationship between the amount of data per piece of captured image data and the capacity of the database 24. However, the data capacity of the database 24 can be increased as needed. By increasing the data capacity of the database 24, it is possible to continue storing captured image data permanently.

[0098] The database 24 can be configured from a plurality of storage devices (including those distributed in remote locations). The quality control system 1 can also be understood as an information processing device for quality control.

[0099] <Modifications of the quality control system> In the above example, once the transport speed V [m / min], the camera distance L [m] as the shooting distance, and the timestamp value T1 (indicating the time when the identification information was added) are determined, one piece of captured image data is identified. However, even if the various parameters V, L, and T1 are determined, there may be a situation where multiple pieces of image data are identified instead of one piece of captured image data.

[0100] For example, if photographing is performed by the photographing device 18 at a time interval less than the smallest unit of the value indicating the photographing time information, the same value of photographing time information will be assigned to multiple pieces of photographed image data. Specifically, for example, assume that the transport speed V is 1 [m / sec], the interval between photographs taken by the photographing device 18 (the time interval between acquisition of photographed image data) is 0.5 seconds, and the smallest unit of the value of the photographing time information is seconds. In this case, the time interval between acquisition of photographed image data is less than 1 second, which is the smallest unit of the value indicating the photographing time information. Therefore, the same value of photographing time information will be assigned to two different pieces of photographed image data.

[0101] Furthermore, even if the time interval for acquiring photographed image data by the photographing device 18 is one second or more, if multiple photographing devices 18 are installed or if one photographing device 18 is equipped with multiple cameras, the same value of photographing time information will be assigned to the multiple photographed image data. When multiple photographing devices 18 are installed, examples of the installation locations include before or after the marking device 19.

[0102] In cases where the same photographing time information is assigned to multiple pieces of photographed image data as in these cases, the optimization unit 50 can display the multiple pieces of photographed image data to be targeted on a display device, and the worker can visually select the photographed image data to be used as training data, and the selection results can be registered in the system software via a terminal device. The selection results can be registered in the same way as the registration of defective product classifications and lot numbers by the defect finder described above.

[0103] <Inventions included in Quality Control System 1> The quality control system 1 described above includes, for example, the following inventions. (1) A database (such as a manufacturing management database 24) that stores photographed image data relating to photographed images of a product (such as a steel pipe 14) during the manufacturing process, identification information (such as lot number information) attached to the product, and photographed time information of the product, in association with the photographed image data; a quality determination unit (such as quality determination unit 40) that determines the quality of the product using the captured image data; an acquisition unit (such as the acquisition unit 52 of the optimization unit 50) for acquiring the identification information of a product in which a defect has been subsequently discovered, when the defect has subsequently been discovered in the product in which a defect has not been found by the quality determination unit; an identifying unit (such as the identifying unit 54 of the optimization unit 50) that identifies the captured image data from the database based on the identification information and the image capture time information acquired by the acquisition unit; A quality control system comprising a learning unit (such as the learning unit 56 of the optimization unit 50) that uses the captured image data identified by the identification unit as training data to cause the pass / fail judgment unit to learn how to judge pass / fail. (2) A quality control system as described in (1) above, in which the identification unit calculates the shooting time (T2) indicated by the shooting time information based on the conveying speed (V [m / min]) of the product, the addition time information (T1) relating to the addition of the identification information, and the shooting distance (L [m]) which is the distance from the position where the product was photographed to the position where the identification information was added, and finds the photographed image data. (3) A first step (steps S10 to S30) of storing in a database photographed image data (step S10 in FIG. 7) of a product in the manufacturing process, identification information added to the product (step S20), and photographed time information of the product in association with each other; A second step (step S40) of determining whether the product is good or bad using the captured image data; If a defect is subsequently discovered in a product in which no defect was found in the second step (step S50), the identification information of the product in which the defect was subsequently discovered is acquired; Based on the acquired identification information and the photographing time information, photographed image data is identified from the database (step S60); A quality control method comprising a third step (steps S50 to S70) of learning to judge pass / fail (step S70) using the identified photographed image data as training data. (4) A quality control method as described in (3) above, in which the photographed image data is identified by calculating the photographed time indicated by the photographed time information based on the conveying speed of the product, the addition time information relating to the addition of the identification information, and the photographed distance, which is the distance from the position where the photograph of the product was taken to the position where the identification information was added (step S60). (5) storing in a database, in association with each other, photographed image data relating to photographed images of products in the manufacturing process, identification information attached to the products, and photographed time information of the products; causing a quality determination unit to determine the quality of the product using the photographed image data; when a defect is subsequently discovered in a product in which a defect was not found by the quality determining unit, cause the acquiring unit to acquire the identification information of the product in which a defect is subsequently discovered; causing an identification unit to identify photographed image data from the database based on the identification information and the photographing time information acquired by the acquisition unit; a quality control program that causes a learning unit to cause the quality judgment unit to learn how to judge quality using the captured image data identified by the identification unit as training data; (6) a database in which photographed image data relating to photographed images of products in the manufacturing process, identification information attached to the products, and photographed time information of the products are linked and stored; an acquisition unit that acquires the identification information of a product in which a defect is found; an identifying unit that identifies photographed image data from the database based on the identification information and the photographing time information acquired by the acquiring unit; and a generation unit that generates a learning model for determining whether the product is good or bad, using the captured image data identified by the identification unit as training data. (7) An information processing device as described in (6) above, in which the identification unit calculates the shooting time indicated by the shooting time information based on the conveying speed of the product, the addition time information related to the addition of the identification information, and the shooting distance, which is the distance from the position where the product was photographed to the position where the identification information was added, and identifies the photographed image data. (8) The above (1) can also be explained as follows. a product manufacturing system (such as the product manufacturing system 10) that acquires photographed image data while transporting a product (such as a steel pipe 14), adds identification information (including lot number information, for example) to the product, and stores at least the photographed image data, the identification information, and photographed time information of the product in a database (such as a manufacturing management database) in association with each other; a quality determination unit (such as quality determination unit 40) that performs image analysis using the captured image data to inspect the product; and an optimization unit (optimization unit 50, etc.) that, when a defect is later discovered in a product in which no defect was found by the quality judgment unit, searches the database for photographed image data of the product in which a defect was later discovered based on the identification information and the photographing time information, and causes the quality judgment unit to perform learning for image analysis using the photographed image data that has been found as training data. In addition, the above (3), (5), and (6) can also be changed to match the description in (8). (9) In (2) above, "the specifying unit" can also be replaced with "the optimizing unit."

[0104] It should be noted that the above-described embodiment is merely an example of a specific implementation of the present invention, and the technical scope of the present invention should not be construed as being limited thereby. In other words, the present invention can be implemented in various forms without departing from the gist or main characteristics thereof.

[0105] For example, in the above-described embodiment, the steel pipe 14 formed by cutting the steel strip 12 has been described as an example of the product, but the present invention is not limited to this. In addition to the steel pipe 14, other examples of the product include channel steel, equal-leg angle steel, U-shaped steel sheet piles, straight steel sheet piles, round bars, flat steel, square steel, and slit coils. Furthermore, the present invention is not limited to these products, and can be applied to various products obtained by performing secondary processing on products that have been primarily processed from raw materials. [Explanation of symbols]

[0106] 1. Quality Control System 10 Product Manufacturing System 13 Transport path 14 Steel pipe 18 Imaging equipment 19 Marking device 20 Reading device 21 Cutting machine 23 Manufacturing control equipment 24 Manufacturing Management Database 40 Good / bad judgment department 50 Optimization Section 52 Acquisition Department 54 Specific part 56 Learning Department

Claims

1. a database that stores, in association with each other, photographed image data relating to photographed images of products in the manufacturing process, identification information added to the products in the manufacturing process, photographed time information of the products, and addition time information indicating the time when the identification information was added to the products in the manufacturing process; a quality determination unit that determines the quality of the product using the captured image data; an acquisition unit that acquires, when a defect is subsequently discovered in the product that was not found to be defective by the quality determination unit, the identification information of the product in which a defect is subsequently discovered; an identifying unit that identifies the photographed image data from the database based on the identification information, the photographing time information, and the additional time information acquired by the acquiring unit; a learning unit that causes the quality determination unit to learn quality determination using the photographed image data identified by the identification unit as training data, A quality control system in which the identification unit identifies the photographed image data from the database based on the added time information and identifies the photographed time information of the product in which a defect was later discovered.

2. The quality control system described in claim 1, wherein the identification unit calculates the shooting time indicated by the shooting time information based on the product's conveying speed, the added time information, and the shooting distance, which is the distance from the position where the product was photographed to the position where the identification information was added, and searches for the photographed image data.

3. a first step of linking and storing in a database photographed image data relating to a photographed image of a product in the manufacturing process, identification information added to the product in the manufacturing process, photographed time information of the product, and addition time information indicating the time when the identification information was added to the product in the manufacturing process; a second step of determining whether the product is good or bad using the captured image data; If a defect is subsequently discovered in a product in which a defect was not found in the second step, the identification information of the product in which the defect was subsequently discovered is acquired; Identifying the photographed image data from the database based on the acquired identification information, the photographing time information, and the additional time information; a third step of performing learning for determining pass / fail using the identified photographed image data as training data, In the third step, the photographed image data is identified from the database by identifying the photographed time information of the product in which a defect was subsequently discovered based on the added time information.

4. The quality control method described in claim 3, wherein the photographed image data is identified by calculating the photographing time indicated by the photographing time information based on the conveying speed of the product, the added time information, and the photographing distance, which is the distance from the position where the photograph of the product was taken to the position where the identification information was added.

5. storing in a database, in association with each other, photographed image data relating to photographed images of products in the manufacturing process, identification information added to the products in the manufacturing process, photographed time information of the products, and addition time information indicating the time when the identification information was added to the products in the manufacturing process; causing a quality determination unit to determine the quality of the product using the photographed image data; when a defect is subsequently discovered in a product in which a defect was not found by the quality determining unit, cause the acquiring unit to acquire the identification information of the product in which a defect is subsequently discovered; causing an identification unit to identify photographed image data from the database based on the identification information, the photographing time information, and the additional time information acquired by the acquisition unit; a quality control program that causes a learning unit to cause the quality judgment unit to learn quality judgment using the photographed image data identified by the identification unit as teacher data, A quality control program that, when causing the identification unit to identify the photographed image data from the database, identifies the photographed time information of the product in which a defect was later discovered based on the added time information.

6. Photographed image data relating to photographed images of products in the manufacturing process, identification information added to the products in the manufacturing process, a database in which photographing time information of the product and adding time information indicating the time when the identification information was added to the product during the manufacturing process are stored in association with each other; an acquisition unit that acquires the identification information of a product in which a defect is found; an identifying unit that identifies photographed image data from the database based on the identification information, the photographing time information, and the additional time information acquired by the acquiring unit; a generation unit that generates a learning model for determining whether the product is good or bad using the photographed image data identified by the identification unit as training data, The identification unit identifies the photographed image data from the database by identifying the photographed time information of the product in which a defect was later discovered, based on the added time information.

7. The information processing device described in claim 6, wherein the identification unit calculates the shooting time indicated by the shooting time information based on the conveying speed of the product, the added time information, and the shooting distance, which is the distance from the position where the product was photographed to the position where the identification information was added, and identifies the photographed image data.

Images