Manufacturing line abnormality detection device, method, and program, as well as manufacturing device and inspection device

Abstract

Provided are: a device, a method, and a program for detecting signs of an abnormality in a production line which make it possible to detect signs of an abnormality in a production line; a production device; and an inspection device. A processor in the device (30) for detecting signs of an abnormality in a production line acquires images captured by a radiography device (20) of each product (P) produced by a production line (10), and acquires singularity information relating to the singularities of the products (P) on the basis of the acquired images. The processor stores information affecting the determination of whether a defect is present in the products (P) within the singularity information as information relating to defects in memory, and designates information not affecting the determination of whether a defect is present in the products as information not relating to defects in the memory. The processor calculates a line evaluation value indicating the soundness of the production line (10) on the basis of the information relating to defects and the information not relating to defects, and detects signs of an abnormality in the production line (10) on the basis of the calculated line evaluation value.

Description

[Technical Field]

[0001] The present invention relates to an apparatus, method and program for detecting signs of abnormality in a manufacturing line, as well as a manufacturing apparatus and an inspection apparatus, and more particularly to a technique for predicting signs of abnormality in a manufacturing line from products manufactured by the manufacturing line. [Background technology]

[0002] At manufacturing sites for metal parts, inspections are conducted for defects before shipment. However, inspecting the machines that manufacture the products for defects is difficult every time due to the inspection costs and the need for continuous production. Therefore, inspections must be conducted only after regular maintenance is performed or when a defect occurs, making it difficult to detect abnormalities early on.

[0003] In response to this, a production management method for a manufacturing line has been proposed (Patent Document 1), which includes a manufacturing process in which a plurality of parts are assembled to form a product, and an inspection process in which each part or product is inspected as an inspection target, in which the inspection process photographs the inspection target according to preset rules, processes the images obtained by photographing, and further judges the quality of the product by comparing the processed results with a pass / fail judgment value, and also includes a management process in which the image processing results are sequentially accumulated, their contents are judged, and this judgment result is reflected in the manufacturing process.

[0004] Specifically, the production management method for a manufacturing line described in Patent Document 1 involves irradiating light from a lighting device onto the outer surface of the product, and receiving the light reflected from the outer surface with a camera, thereby photographing the outer surface of the product with a camera.

[0005] An upper limit level for detecting mainly bright scratches among scratches that occur on the surface of the object to be inspected, and a lower limit level for detecting black marks and surface roughness that occur on the surface of the object to be inspected are set in association with the brightness level of the video signal obtained from the camera, and the production line production control method sequentially stores information about the video signal obtained by shooting as digital data in a storage medium, and when the level of the video signal exceeds the standard value and shows a tendency to approach the upper limit level, or when the level of the video signal falls below the standard value and shows a tendency to approach the lower limit level, the assembly method in the manufacturing process is changed based on this information, thereby preventing abnormalities from occurring in the next product to be manufactured. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-15930 Summary of the Invention [Problem to be solved by the invention]

[0007] Patent Document 1 describes a method in which information relating to video signals obtained by photographing a product or its components is sequentially stored as digital data in a storage medium, and when the level of the video signal exceeds a standard value and shows a tendency to approach an upper limit level, or when the level of the video signal falls below a standard value and shows a tendency to approach a lower limit level, the assembly method in the manufacturing process is changed based on this information to prevent abnormalities from occurring in the next product to be manufactured.However, there is no description of detecting signs of abnormalities in the manufacturing line by taking into account video signal levels that do not constitute defects (information that does not affect the determination of whether or not a product has a defect).

[0008] The present invention has been made in consideration of the above circumstances, and aims to provide an abnormality detection device, method, and program for a production line, as well as a manufacturing device and an inspection device, that can detect abnormal signs on a production line early and without delay, and that can inspect products. [Means for solving the problem]

[0009] In order to achieve the above object, the invention according to a first aspect is an abnormality sign detection device for a production line, comprising a processor, a photographing device, a first memory, and a second memory, wherein the processor performs a photographing process of photographing products to be inspected manufactured on the production line one by one using the photographing device, a singularity information acquisition process of acquiring singularity information related to singularities of the products based on the images acquired by photographing, a singularity information storage process of saving the acquired singularity information in the first memory, and saving, in the second memory, information of the singularity information that affects the determination of the presence or absence of a defect in the product as defect-related information and information that does not affect the determination of the presence or absence of a defect in the product as defect-non-related information. a product evaluation value calculation process that calculates a product evaluation value that indicates the soundness of the product based on the defect-related information and the defect-unrelated information; a line evaluation value calculation process that calculates a line evaluation value that indicates the soundness of the production line based on the defect-related information and the defect-unrelated information; an abnormal sign detection process that detects an abnormal sign of the production line based on the line evaluation value calculated by the line evaluation value calculation process; and an output process that outputs feedback information including the detection results of the abnormal sign of the production line and the detection results of the product defects.

[0010] According to a first aspect of the present invention, products manufactured on a production line are photographed one by one using an imaging device, singularity information related to the singularities of each product is acquired based on the captured images, and the acquired singularity information is stored in a first memory. Furthermore, of the singularity information stored in the first memory, information that affects the determination of the presence or absence of a product defect is designated as defect-related information, and information that does not affect the determination of the presence or absence of a product defect is designated as defect-non-related information and stored in a second memory. A product evaluation value indicating the soundness of the product is calculated based on the defect-related information, and the presence or absence of a product defect is detected based on the calculated product evaluation value. Furthermore, a line evaluation value indicating the soundness of the production line is calculated based on the defect-related information and defect-non-related information (classified singularity information) stored in the second memory, and an abnormality sign in the production line is detected based on the calculated line evaluation value. This allows for early and prompt detection of an abnormality sign in the production line by taking into account various types of singularity information constituting the inspection history of products manufactured on the production line (including singularity information that does not affect the determination of the presence or absence of a product defect). In addition, by outputting feedback information including the results of detecting abnormal signs on the production line and the results of detecting defects in the products, appropriate countermeasures can be taken automatically or by the user for the production line, and products can be inspected.

[0011] In the production line anomaly sign detection device according to the second aspect of the present invention, the processor preferably performs a defect predictor value calculation process for calculating a defect predictor value based on the defect non-related information and a notification process for notifying the defect predictor value, thereby making it possible to predict the possibility of a defect occurring in the future or the timing of a defect occurrence when products are continuously manufactured on the production line.

[0012] In the production line abnormality sign detection device according to the third aspect of the present invention, the imaging device is preferably a radiographic imaging device, an ultrasonic imaging device, or an infrared imaging device, which makes it possible to image singular points inside the product and perform non-destructive inspection of the product.

[0013] In the abnormality detection device for a production line according to the fourth aspect of the present invention, the singularity information is preferably one or more of singularity type information, occurrence position information, size information, and shape information, and the storage process preferably classifies the singularity information into defect-related information and defect-unrelated information based on one or more of the singularity type information, occurrence position information, size information, and shape information, and stores the information in the second memory.

[0014] In the production line abnormality sign detection device according to the fifth aspect of the present invention, it is preferable that the line evaluation value calculation process calculates a line evaluation value indicating the healthiness of the production line based on at least one piece of defect-related information stored in the second memory and at least two or more pieces of non-defect-related information stored in the second memory.

[0015] In the production line abnormality sign detection device according to the sixth aspect of the present invention, it is preferable that the line evaluation value calculation process calculates a line evaluation value indicating the healthiness of the production line based on two or more pieces of defect-related information stored in the second memory and two or more pieces of defect-unrelated information stored in the second memory.

[0016] In the seventh aspect of the present invention, in the abnormality detection device for a production line, it is preferable that the line evaluation value calculation process calculates a line evaluation value indicating the healthiness of the production line based on defect-related information and non-defect-related information corresponding to multiple products stored in the second memory.

[0017] In the production line anomaly detection device according to the eighth aspect of the present invention, the multiple products are preferably a group of products manufactured within a certain period of time, a group of a certain number of products manufactured in chronological order, or a group of products in one lot, which is a unit for managing products. Note that the group of products manufactured within a certain period of time or the group of a certain number of products may have some products overlapping with other product groups.

[0018] In the production line abnormality sign detection device according to the ninth aspect of the present invention, the line evaluation value calculation process preferably counts the defect-related information and the defect-non-related information and calculates the line evaluation value based on the count value. The line evaluation value may be the count value itself or may be calculated as an average value obtained by dividing the count value by the number of target products.

[0019] In the production line abnormality sign detection device according to the tenth aspect of the present invention, the line evaluation value calculation process preferably weights the defect-related information and the defect-non-related information when counting the defect-related information and the defect-non-related information. Note that different weights may be assigned to the specific point information classified as defect-related information, and different weights may be assigned to the specific point information classified as defect-non-related information.

[0020] In the production line abnormality sign detection device according to the eleventh aspect of the present invention, the line evaluation value calculation process preferably counts the defect-related information by weighting the defect-related information according to the type of the defect-related information, which includes, for example, singular point type information, occurrence position information, size information, shape information, and degree of association with the defect.

[0021] In the production line abnormality sign detection device according to a twelfth aspect of the present invention, the abnormality sign detection process preferably compares two or more line evaluation values ​​and detects abnormality signs in the production line based on the comparison results. By comparing line evaluation values ​​calculated from the inspection histories of two or more products manufactured at different times, it is possible to extract feedback information, such as that products manufactured at a certain manufacturing time generally contain more defect-related information than products manufactured at other manufacturing times, and to detect and notify abnormality signs in the production line.

[0022] In a thirteenth aspect of the present invention, the production line abnormality detection device preferably includes a plurality of manufacturing processes, the singularity information being one or more of singularity type information, occurrence position information, size information, and shape information, and further includes a third memory for storing a first correspondence table that associates specific singularity information included in the singularity information with a specific manufacturing process associated with the specific singularity information among the plurality of manufacturing processes on the production line, the line evaluation value calculation process preferably counts the defect-related information and the defect-non-related information for each of the plurality of manufacturing processes according to the first correspondence table and calculates the counted value for each manufacturing process as a process evaluation value indicating the soundness of each manufacturing process, and the abnormal sign detection process preferably detects an abnormal sign for each manufacturing process on the production line based on the process evaluation value calculated for each manufacturing process. This makes it possible to identify manufacturing processes among the plurality of manufacturing processes that may be deteriorating and take appropriate countermeasures.

[0023] In a fourteenth aspect of the present invention, the abnormality detection device for a production line according to the present invention further includes a fourth memory storing a second correspondence table that associates specific singularity information included in the singularity information with specific environmental information related to the specific singularity information among multiple pieces of environmental information that indicate the manufacturing environment of the production line. The processor preferably performs the following steps: acquires the specific singularity information from the singularity information stored in the first memory; and, upon acquisition, acquires specific environmental information related to the acquired specific singularity information according to the second correspondence table. The output step preferably outputs feedback information including the specific environmental information. In this manner, upon acquisition of the specific singularity information, the processor acquires the specific environmental information related to the specific singularity information from the fourth memory and outputs the feedback information including the specific environmental information. Therefore, by adjusting the production line environment according to the specific environmental information, the occurrence of product defects caused by the environment can be reduced.

[0024] In a fifteenth aspect of the present invention, the abnormality detection device for a production line further comprises a fifth memory that stores quality information indicating the quality of each product group and additional information related to the quality information in association with each other, and the processor performs a process of acquiring quality information regarding the quality of the product group based on each inspection history of the inspection history group corresponding to the product group, and a process of acquiring additional information corresponding to the quality information from the fifth memory based on the acquired quality information, and the output process preferably outputs the additional information acquired corresponding to the product group. The additional information acquired corresponding to the product group may be information such as the content of post-processing of the products and their intended use, depending on the quality of the product group.

[0025] In a sixteenth aspect of the present invention, in the production line anomaly detection device, the singularity information includes location information and size information of singularities in the product, and the processor performs the following steps: acquiring minute singularity information indicating minute singularities with a singularity size smaller than a threshold based on the singularity information acquired by the singularity information acquisition process; and generating, based on the acquired minute singularity information, emphasis information for highlighting the minute singularities, the emphasis information visibly displaying an area that includes the minute singularity and is larger than the singularity size of the minute singularity, and information corresponding to the number of the minute singularities. Preferably, the output process superimposes the emphasis information and the information corresponding to the number of the minute singularities on the image and displays it on the display. Because minute singularities are difficult to see even when their areas are emphasized, the processor generates emphasis information visibly displaying an area larger than the singularity size of the minute singularity, and information corresponding to the number of the minute singularities, and superimposes this information on the image and displays it on the display, making it easier to check the location of the minute singularities, etc.

[0026] In the abnormality detection device for a production line according to the 17th aspect of the present invention, the highlighting information is mask information that fills in an area larger than the singularity size of the micro singularity with at least one of a specific color and brightness, or frame information that surrounds the area, and it is preferable that the information corresponding to the number of micro singularities is text information indicating the number, or at least one of the color and brightness of the highlighting information according to the number.

[0027] In the anomaly detection device for a production line according to an eighteenth aspect of the present invention, the singularity information acquisition process preferably extracts image features and acquires a defect probability of the singularity information for each pixel of the image, and the output process preferably adds a color corresponding to the defect probability to the pixel corresponding to the singularity information and displays it on the display. By adding a color corresponding to the defect probability to the pixel corresponding to the singularity information, the singularity area can be displayed as a gradation or heat map corresponding to the defect probability. The color corresponding to the defect probability of the singularity information includes at least one of hue and saturation.

[0028] A nineteenth aspect of the invention is a manufacturing apparatus including a production line for manufacturing products, a processor, a photographing device, a first memory, and a second memory, wherein the processor performs a photographing process for photographing products manufactured on the production line one by one using the photographing device, a singularity information acquisition process for acquiring singularity information related to singularities of the products based on images acquired by photographing, a singularity information storage process for saving the acquired singularity information in the first memory, and a process for saving, in the second memory, information of the singularity information that affects the determination of the presence or absence of a defect in the product as defect-related information and information that does not affect the determination of the presence or absence of a defect in the product as defect-unrelated information. a product evaluation value calculation process that calculates a product evaluation value that indicates the soundness of the product based on the defect-related information; a defect detection process that detects the presence or absence of a defect in the product based on the product evaluation value calculated by the product evaluation value calculation process; a line evaluation value calculation process that calculates a line evaluation value that indicates the soundness of the production line based on the defect-related information and the defect-unrelated information; an abnormal sign detection process that detects an abnormal sign in the production line based on the calculated line evaluation value; and an output process that outputs feedback information including the detection results of the abnormal sign in the production line and the detection results of the product defects.

[0029] A twentieth aspect of the invention is an inspection device including a processor, a photographing device, a first memory, and a second memory, wherein the processor performs a photographing process of photographing products to be inspected manufactured on a production line one by one using the photographing device, a singularity information acquisition process of acquiring singularity information related to singularities of the products based on images acquired by photographing, a singularity information storage process of saving the acquired singularity information in the first memory, and a process of saving, among the singularity information saved in the first memory, information that affects the determination of the presence or absence of a defect in the product as defect-related information in the second memory, and information that does not affect the determination of the presence or absence of a defect in the product as defect-unrelated information in the second memory. a product evaluation value calculation process that calculates a product evaluation value that indicates the soundness of the product based on the defect-related information; a defect detection process that detects the presence or absence of a defect in the product based on the product evaluation value calculated by the product evaluation value calculation process; a line evaluation value calculation process that calculates a line evaluation value that indicates the soundness of the production line based on the defect-related information and the defect-unrelated information; an abnormal sign detection process that detects an abnormal sign in the production line based on the line evaluation value calculated by the line evaluation value calculation process; and an output process that outputs feedback information including the detection results of the abnormal sign in the production line and the detection results of the product defects.

[0030] A twenty-first aspect of the invention is a method for detecting abnormal signs in a production line, in which a processor performs processing of the following steps to detect abnormal signs in the production line, the method including the steps of: photographing products to be inspected manufactured by the production line one by one using an imaging device; acquiring singularity information related to singularities in the product based on the image acquired by photographing; storing the acquired singularity information in a first memory; storing, from the singularity information stored in the first memory, information that affects the determination of whether or not there is a defect in the product as defect-related information in a second memory, and storing, in the second memory, information that does not affect the determination of whether or not there is a defect in the product as defect-non-related information; calculating a product evaluation value that indicates soundness of the product based on the defect-related information; detecting whether or not there is a defect in the product based on the product evaluation value; calculating a line evaluation value that indicates the soundness of the production line based on the defect-related information and the defect-non-related information; detecting abnormal signs in the production line based on the line evaluation value; and outputting feedback information including the detection results of the abnormal signs in the production line and the detection results of the product defects.

[0031] In the method for detecting an abnormality sign in a production line according to the 23rd aspect of the present invention, the imaging device is preferably a radiographic device, an ultrasonic imaging device, or an infrared imaging device, in order to perform non-destructive testing of the product.

[0032] A twenty-fourth aspect of the invention is a program for detecting signs of abnormality in a production line, which program causes a computer to execute the method for detecting signs of abnormality in a production line according to either the twenty-first or twenty-third aspect. [Effects of the Invention]

[0033] According to the present invention, abnormal signs in a production line can be detected early and without delay, and feedback information including the detection results and the detection results of product defects can be obtained. [Brief explanation of the drawings]

[0034] [Figure 1]FIG. 1 is a schematic diagram showing the configuration of a manufacturing apparatus according to the present invention. [Figure 2] FIG. 2 is a diagram showing an example of a radiation imaging device for imaging a product. [Figure 3] FIG. 3 is a block diagram showing an embodiment of the hardware configuration of an abnormality sign detection device for a manufacturing line according to the present invention. [Figure 4] FIG. 4 is a functional block diagram showing an embodiment of an abnormality sign detection device for a manufacturing line according to the present invention. [Figure 5] FIG. 5 is a chart showing an example of an inspection history stored in the memory. [Figure 6] FIG. 6 is a diagram showing a first example of a first correspondence table that associates specific singularity information included in the singularity information with a specific manufacturing process associated with the specific singularity information among multiple manufacturing processes in a manufacturing line. [Figure 7] FIG. 7 is a diagram showing a second example of the first correspondence table in which specific singularity information included in the singularity information is associated with a specific manufacturing process related to the specific singularity information among multiple manufacturing processes in the manufacturing line. [Figure 8] FIG. 8 is a chart showing an example of count values ​​(process evaluation values ​​for each manufacturing process) when the singular points of each product in a product group are counted for each manufacturing process. [Figure 9] FIG. 9 is a diagram showing an example of a second correspondence table in which specific singularity information is associated with related specific environmental information. [Figure 10] FIG. 10 is a diagram showing an example of a correspondence table in which quality information indicating the quality of each product group is associated with additional information related to the quality information. [Figure 11] FIG. 11 is a diagram showing an example of an image of a product including a minute singular point. [Figure 12] FIG. 12 is a diagram showing another example of an image of a product including a minute singular point. [Figure 13] FIG. 13 is a flowchart showing an embodiment of a method for detecting a sign of abnormality in a production line according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0035] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an abnormality detection device, method, program, manufacturing device, and inspection device for a manufacturing line according to the present invention will now be described with reference to the accompanying drawings.

[0036] [Manufacturing equipment configuration] FIG. 1 is a schematic diagram showing the configuration of a manufacturing apparatus according to the present invention.

[0037] The manufacturing apparatus shown in FIG. 1 is made up of a manufacturing line 10 and an abnormality sign detection device 30 for the manufacturing line.

[0038] The production line 10 comprises a plurality of production processes, production process 1 to production process N, and produces a product P made of metal parts through each of the production processes 1 to N. For example, production process 1 produces part A, production process 2 produces part B and joins parts A and B, and production process N produces part C and joins parts B and C, thereby producing the product P, the object to be produced by this production line 10.

[0039] The abnormality sign detection device 30 for a manufacturing line according to the present invention is configured to include a radiation imaging device (imaging device) 20, and functions as an inspection device that performs non-destructive inspection of a product P using images captured by the radiation imaging device 20.

[0040] The products P manufactured by the manufacturing line 10 are photographed one by one by the radiation imaging device 20.

[0041] FIG. 2 is a diagram showing an example of a radiation imaging device for imaging a product.

[0042] The radiographic imaging device 20 shown in Figure 2 is an X-ray imaging device that uses X-rays as radiation, and is equipped with an X-ray source 22 and an imaging plate 24, with the imaging plate 24 positioned opposite the X-ray source 22 across the product P.

[0043] X-rays emitted from the X-ray source 22 pass through the product P and strike the imaging plate 24, and energy information (X-ray image information) corresponding to the incident dose is stored on the imaging plate 24. The X-ray image information stored on the imaging plate 24 is read by the image reading device 40 shown in Fig. 3 and captured as an X-ray fluoroscopic image of the product P by the abnormality sign detection device 30 on the production line.

[0044] The radiographic imaging device 20 is not limited to one that uses the imaging plate 24, but may also use an X-ray flat panel detector, an X-ray line sensor, X-ray film, etc. Furthermore, the position and direction from which an image is taken of a product are determined depending on the product, and multiple locations on a single product may be imaged.

[0045] The production line abnormality sign detection device 30 performs non-destructive inspection of the product P based on an X-ray fluoroscopic image (hereinafter simply referred to as "image") obtained by photographing the product P, and detects abnormal signs for the entire production line 10 or for each of the production processes 1 to N of the production line 10 from the inspection results of the product P.

[0046] [Hardware configuration of anomaly detection device for manufacturing lines] FIG. 3 is a block diagram showing an embodiment of the hardware configuration of an abnormality sign detection device for a manufacturing line according to the present invention.

[0047] The abnormality sign detection device 30 for a manufacturing line shown in FIG. 3 is configured by a personal computer, a workstation, or the like, and includes a processor 32, a memory 34, a display (display unit) 36, an input / output interface 38, an operation unit 39, and the like.

[0048] The processor 32 is composed of a CPU (Central Processing Unit) and other components, and controls each part of the abnormality sign detection device 30 on the production line, and performs various processes based on images of each product P taken by the radiation imaging device 20 to detect abnormal signs on the production line 10.Details of the various processes performed by the processor 32 will be described later.

[0049] The memory 34 includes flash memory, ROM (Read-only Memory), RAM (Random Access Memory), a hard disk drive, etc. The flash memory, ROM, or hard disk drive is a non-volatile memory that stores an operation system, various programs including the production line abnormality sign detection program according to the present invention, etc. The RAM functions as a working area for processing by the processor 32. It also temporarily stores the structure condition prediction program stored in the flash memory, etc.

[0050] The memory 34 stores singularity information relating to the singularities of the product, acquired by the processor 32's singularity information acquisition process, as an inspection history for each product, and also stores various correspondence tables, etc., which will be described later.

[0051] Of the singularity information, singularity information that affects the determination of whether or not a product has a defect is classified as defect-related information and stored in memory 34, while singularity information that does not affect the determination of whether or not a product has a defect is classified as defect-unrelated information and stored in memory 34. Details of the singularity information will be described later.

[0052] The processor 32 uses the RAM as a working area and acquires necessary data such as various types of specificity information contained in the inspection history of each product from the memory 34 in accordance with the production line abnormality sign detection program, and controls and processes each part of the production line abnormality sign detection device 30.

[0053] The display unit 36 ​​displays feedback information including the detection results of abnormal signs of the production line 10 detected by the processor 32. The user can check the soundness of the production line 10 from the feedback information displayed by the display unit 36, and can perform appropriate maintenance, preservation, etc. on the production line 10.

[0054] The display unit 36 ​​can also display an image of the product P acquired by the processor 32, and the user can check various peculiarities of the product P while looking at the image displayed on the display unit 36. The display unit 36 ​​is also used as part of a GUI (Graphical User Interface) when receiving user instructions from the operation unit 39.

[0055] The input / output interface 38 includes a connection unit connectable to an external device and a communication unit connectable to a network. Examples of the connection unit connectable to an external device include a Universal Serial Bus (USB) and a High-Definition Multimedia Interface (HDMI) (HDMI is a registered trademark). By connecting an image reading device 40 to the input / output interface 38 and having the image reading device 40 read the X-ray image information stored on the imaging plate 24 of the radiation imaging device 20 shown in FIG. 2, the processor 32 can acquire an X-ray fluoroscopic image of the product P via the input / output interface 38.

[0056] The operation unit 39 includes a keyboard, a pointing device such as a mouse, a keyboard, etc., and functions as part of a GUI that accepts various instructions from the user.

[0057] [Embodiment of an abnormality detection device for a manufacturing line] FIG. 4 is a functional block diagram showing an embodiment of an abnormality sign detection device for a manufacturing line according to the present invention.

[0058] The abnormality sign detection device for a production line shown in FIG. 4 includes an imaging processing unit 51, a singularity information acquisition unit 52, a memory control unit 53, a line evaluation value calculation unit 54, an abnormal sign detection unit 55, an output unit 56, a product evaluation value calculation unit 57, a defect detection unit 58, a defect sign value calculation unit 59, and an alarm unit 60, and the processor 32 of the abnormality sign detection device for a production line 30 shown in FIG. 3 functions as the imaging processing unit 51, the singularity information acquisition unit 52, the memory control unit 53, the line evaluation value calculation unit 54, the abnormal sign detection unit 55, the output unit 56, the product evaluation value calculation unit 57, the defect detection unit 58, the defect sign value calculation unit 59, and the alarm unit 60.

[0059] The photography processing unit 51 is a part that performs the photography process of photographing the products P, which are the inspection targets manufactured by the production line 10, one by one using the radiation photography device 20, either automatically or by inputting photography instructions from the user, and acquires images of the products P photographed by the radiation photography device 20 via the input / output interface 38.

[0060] The singularity information acquisition unit 52 performs a singularity information acquisition process to acquire singularity information related to singularities of the product P based on the image acquired by the photography processing unit 51. The singularity information acquired by the singularity information acquisition unit 52 is one or more of type information, occurrence position information, size information, and shape information of the singularity of the product P. The singularities of the product P also include minute heterogeneous parts that cannot be considered defects, and the singularity information is classified into defect-related information, which is information that affects the determination of whether or not the product P has a defect, and defect-unrelated information, which is information that does not affect the determination of whether or not the product P has a defect.

[0061] Possible information on the type of singularity includes gas holes, bubbles, high-density foreign matter, low-density foreign matter, cracks, and welding defects. When product P is composed of multiple parts, information on the part where the singularity occurred is also included as information on the location of the singularity. Welding defects include cavities inside the weld metal (blowholes, pinholes), poor penetration, undercuts, and overlaps. Size information can be expressed as the area (number of pixels) of the singularity region. In addition, spherical singularities can be expressed as the diameter, and linear singularities can be expressed as the length of the singularity.

[0062] The singularity information acquisition unit 52 can be configured using, for example, AI (Artificial Intelligence), and when an image of the product P is input, it extracts image features and classifies each pixel of the image to determine which type of singularity it belongs to, thereby extracting singularity regions and acquiring singularity information such as the recognition results of the singularity type. Alternatively, a user may visually confirm singularities in the image displayed on the display unit 36 ​​and input singularity type information, singularity occurrence position information, etc. via the operation unit 39, and the singularity information acquisition unit 52 may acquire the singularity information input by user operation.

[0063] The memory control unit 53 performs a singularity information saving process of saving the singularity information related to the singularities of the product P acquired by the singularity information acquiring unit 52 in the memory (first memory) 34 as an inspection history for each product P, and a saving process of saving, out of the singularity information saved in the memory 34, information that affects the determination of the presence or absence of a product defect in the memory (second memory) 34 as defect-related information and saving information that does not affect the determination of the presence or absence of a product defect in the memory (second memory) 34 as defect-non-related information in an identifiable manner, and also performs a reading process of reading the inspection history, etc. saved in the memory 34. Note that the first memory and the second memory are not limited to different storage areas in the same memory 34, but may be physically different memories.

[0064] FIG. 5 is a chart showing an example of an inspection history stored in the memory.

[0065] The inspection history shown in Figure 5 is singularity information (singularity type information, singularity occurrence location information, singularity size information, and singularity shape information) associated with the product number for each product, and may also include other information such as the product lot number, inspection image, inspection date and time, etc.

[0066] In addition, in the example shown in FIG. 5, one piece of singularity information is stored for one product, but if multiple pieces of singularity information (including multiple pieces of singularity information of the same type and / or multiple pieces of singularity information of different types) are acquired for one product, multiple pieces of singularity information are stored.

[0067] Furthermore, in the case of a pinhole with a small diameter, which is one of the minute singular points, it is preferable to also store information on the number of pinholes. Note that when there is one pinhole, it is an isolated pinhole, and when there are multiple pinholes, it is a densely packed pinhole. When there are multiple isolated pinholes or densely packed pinholes, it is preferable to store this information as information on multiple singular points. When there are multiple densely packed pinholes, that area is the location where the singular point occurs.

[0068] Furthermore, products for which no peculiarities are detected may also be saved as inspection history ("no peculiarities").

[0069] The memory control unit 53 stores, in memory, singularity information that affects the determination of whether or not a product has a defect as defect-related information, and stores, in memory in an identifiable manner, singularity information that does not affect the determination of whether or not a product has a defect as defect-non-related information. However, it may also be possible to add flag information indicating defect-related information or defect-non-related information to each piece of singularity information.

[0070] Here, when classifying singularity information into defect-related information or defect-unrelated information, this can be done based on one or a combination of information such as singularity size information, type information, occurrence location information, and number of singularities.

[0071] For example, if the size of a singularity is very large, it can be classified as defect-related information regardless of the type information and location information of the singularity. Furthermore, if the type information of the singularity is a pinhole (such as a small bubble), it can be classified as defect-related information or non-defect-related information based on the number of closely spaced pinholes and / or information on their locations. For example, an isolated pinhole can be classified as non-defect-related information. In the case of closely spaced pinholes, if the number is less than the allowable number or if the pinholes are not located in a critical area where high stress is applied, they can be classified as non-defect-related information.

[0072] Returning to Figure 4, the line evaluation value calculation unit 54 reads out the singularity information classified as defect-related information or defect-unrelated information that constitutes the inspection history stored in the memory 34 from the memory 34 via the memory control unit 53, and performs a line evaluation value calculation process to calculate a line evaluation value that indicates the soundness of the production line 10 based on the read out singularity information.

[0073] The singularity information used to calculate the line evaluation value includes singularity information classified as defect-related information and singularity information classified as defect-non-related information. Therefore, the line evaluation value calculation unit 54 calculates a line evaluation value indicating the soundness of the production line 10 based on the defect-related information stored in the memory 34 (second memory) and the defect-non-related information stored in the memory 34 (second memory).

[0074] The line evaluation value calculation unit 54 performs a line evaluation value calculation process to calculate a line evaluation value indicating the soundness of the production line 10 based on the defect-related information stored in the memory 34 (second memory) and the defect-unrelated information stored in the memory 34 (second memory).

[0075] The line evaluation value calculation unit 54 calculates a line evaluation value indicating the healthiness of the production line 10 based on preferably at least one defect-related information and at least two or more defect-non-related information, more preferably two or more defect-related information and two or more defect-non-related information.

[0076] The line evaluation value calculation unit 54 may calculate a line evaluation value indicating the soundness of the production line 10 based on the defect-unrelated information stored in the memory 34 (second memory).

[0077] Furthermore, the singular point information used to calculate the line evaluation value is singular point information (defect-related information and defect-unrelated information) corresponding to a plurality of products.

[0078] Here, the multiple products are preferably a group of products manufactured within a certain period of time, a group of a certain number of products manufactured in chronological order, or a group of products in one lot, which is a unit for managing products.

[0079] The fixed period can be one week, one month, etc., and can be set by the user as appropriate. Furthermore, the inspection history groups for the product groups manufactured during the fixed period can be set to allow overlapping. For example, if the fixed period is one week, by shifting the week by one day, it is possible to acquire an inspection history group corresponding to the product groups manufactured during the past week every day. In this case, six days' worth of inspection history from the newly acquired inspection history group will overlap with the inspection history from the inspection history group acquired the previous day.

[0080] The fixed number can be set by the user as appropriate depending on the product. The fixed number of inspection histories can be set while allowing overlaps in the same way as above.

[0081] The quantity of products in one lot can be set appropriately by the user, taking into consideration inventory levels, raw material procurement, etc.

[0082] The line evaluation value calculation unit 54 calculates a line evaluation value that indicates the soundness of the production line 10 based on the singularity information, which is individual singularity information in multiple inspection histories corresponding to multiple products and is classified into defect-related information and defect-non-related information. Specifically, the line evaluation value calculation unit 54 can count the total number of defect-related information and defect-non-related information in a product group and calculate the line evaluation value based on the count value. The line evaluation value may be the count value itself or may be calculated as an average value obtained by dividing the count value by the number of target products.

[0083] Furthermore, when counting the total number of pieces of defect-related information and defect-non-related information in a product group, the line evaluation value calculation unit 54 preferably weights the defect-related information and the defect-non-related information and counts them. The weights of the defect-related information and the defect-non-related information are preferably determined according to one or more pieces of singularity information selected from singularity type information, occurrence position information, size information, and shape information.

[0084] Furthermore, when counting the defect-related information, the line evaluation value calculation unit 54 preferably performs counting by weighting according to the type of defect-related information. The types of defect-related information include defect type information, occurrence position information, size information, shape information, or degree information, and the line evaluation value calculation unit 54 preferably performs counting by weighting according to the content of the defect-related information (for example, by weighting a defect with a larger size, or by weighting according to the angle of a sharp part in the shape) in addition to weighting according to the type of defect-related information itself.

[0085] The abnormal sign detection unit 55 performs an abnormal sign detection process to detect an abnormal sign of the production line 10 based on the line evaluation value calculated by the line evaluation value calculation unit 54. The abnormal sign detection unit 55 can obtain a line evaluation value indicating the soundness of the production line 10 for each product group (inspection history group), and therefore can predict changes in the soundness of the production line 10 (detect abnormal signs of the production line 10).

[0086] Furthermore, the abnormal sign detection unit 55 can, for example, compare two or more line evaluation values ​​corresponding to two or more product groups with different manufacturing times, and detect abnormal signs in the production line 10 based on the comparison results. For example, if the line evaluation value for each product group does not fluctuate, it can be determined that there are no abnormal signs in the production line 10, and if the line evaluation value for each product group is on the rise and is approaching a threshold value that indicates an abnormality in the production line 10, it can be determined that there are abnormal signs in the production line 10.

[0087] The output unit 56 performs output processing to output feedback information including the detection result of the abnormal sign of the production line 10 detected by the abnormal sign detection unit 55. The output destination of the feedback information can be the display unit 36, the production line 10, or equipment related to the production line 10, etc.

[0088] The user can check the feedback information displayed on the display unit 36 ​​and determine in advance (before a defective product is manufactured) whether maintenance or inspection work is necessary for the production line 10.

[0089] In addition, by outputting feedback information to the production line 10 or equipment related to the production line 10, it is possible to automatically control the various molds, molten metal temperature, pressure, welding temperature, etc. in the production line 10, and to automatically control equipment related to the production line 10 (for example, the temperature, humidity, etc. of air conditioning equipment, raw material storage equipment, etc.).

[0090] <Inspection Department> The product evaluation value calculation unit 57 and the defect detection unit 58 function as an inspection unit for the product P.

[0091] The product evaluation value calculation unit 57 is a part that performs product evaluation value calculation processing to read, for each product P, the defect-related information constituting the inspection history stored in the memory 34 from the memory 34 via the memory control unit 53, and calculate a product evaluation value indicating the soundness of the product for each product P based on the read defect-related information. Note that it is preferable to calculate the product evaluation value according to the type of defect-related information (defect type information, occurrence position information, size information, shape information, degree, etc.).

[0092] When there are multiple pieces of defect-related information for one product P, it is preferable that the product evaluation value calculation unit 57 counts the defect-related information or performs a weighted count according to the type of defect-related information, and calculates the product evaluation value based on the count value.

[0093] The defect detection unit 58 performs a defect detection process to detect the presence or absence of defects in each product P based on the product evaluation value calculated by the product evaluation value calculation unit 57. Furthermore, the defect detection unit 58 may rank products P that are detected as having no defects based on the product evaluation value.

[0094] The output unit 56 outputs feedback information including the detection results of abnormal signs on the production line, as well as feedback information including the detection results of defects in the product P input from the defect detection unit 58. The feedback information of the detection results of defects in the product P can be used as information for automatically or manually sorting the product P into good products and bad products (defective products).

[0095] <Sign of defect> The defect predictor value calculation unit 59 and the notification unit 60 are parts that predict and notify the possibility of a defect occurring in the product P in the future or the time when a defect will occur if the product P is continuously manufactured on the production line 10.

[0096] The defect predictive value calculation unit 59 reads out the non-defect related information constituting the inspection history stored in the memory 34 for each product P from the memory 34 via the memory control unit 53, and performs a defect predictive value calculation process to calculate a defect predictive value based on the read non-defect related information.

[0097] Even in normal areas of a product (component), it may be possible to detect signs of defects from the characteristics of its texture (reflected pattern) (e.g., the uneven brightness pattern, the degree of contrast, or the density if a noise-like pattern appears).

[0098] Furthermore, even if an unusual point within the tolerance range (a unusual point that does not affect the determination of whether or not a product is defective) is at a level that is not currently considered a defect, observation over time may predict that it will reach a level where it will be evaluated as a defect in the future.

[0099] The defect predictor value calculation unit 59 calculates a defect predictor value such as the number of non-defective products that can be continuously manufactured, or the manufacturing period, if the current manufacturing line 10 continues to manufacture the product P.

[0100] The notification unit 60 is a part that notifies the defect predictive value calculated by the defect predictive value calculation unit 59. By notifying the defect predictive value, it is possible to carry out planned system maintenance in advance and review the manufacturing environment, thereby enabling efficient manufacturing. The defect predictive value may also be notified from the output unit 56.

[0101] <Detecting abnormal signs in each manufacturing process on the production line> The abnormality sign detection device 30 for the production line is equipped with a third memory (memory 34) that stores a first correspondence table that corresponds specific singularity information included in singularity information related to the singularity of the product P with a specific manufacturing process related to the specific singularity information among the multiple manufacturing processes 1 to N of the production line 10.

[0102] FIG. 6 is a diagram showing a first example of a first correspondence table that associates specific singularity information included in the singularity information with a specific manufacturing process associated with the specific singularity information among multiple manufacturing processes in a manufacturing line.

[0103] The first correspondence table of the first example shown in FIG. 6 indicates the relationship between the singularity information (particularly, the component in which the singularity information classified as defect-related information occurs) and the manufacturing process.

[0104] As shown in FIG. 6, for example, when part A of product P is manufactured by manufacturing process 1, singularity information related to part A (for example, specific singularity information in which the singularity occurrence position information is part A) is associated with manufacturing process 1 (specific manufacturing process).

[0105] Similarly, singularity information related to part B of product P (for example, specific singularity information in which the singularity occurrence location information is part B) is associated with manufacturing process 2 (a specific manufacturing process), and singularity information related to part C of product P (for example, specific singularity information in which the singularity occurrence location information is part C) is associated with manufacturing process N (a specific manufacturing process).

[0106] FIG. 7 is a diagram showing a second example of the first correspondence table, which associates specific singularity information included in the singularity information with a specific manufacturing process associated with the specific singularity information among multiple manufacturing processes in the manufacturing line.

[0107] The first correspondence table of the second example shown in FIG. 7 indicates the relationship between the singular point information (type of singular point) and the manufacturing process.

[0108] 7, for example, if manufacturing process 3 includes a casting process and there is a possibility that a pinhole may occur, the pinhole (specific singular point information) is associated with manufacturing process 3 (specific manufacturing process). Similarly, if manufacturing process 4 includes a welding process and there is a possibility that a welding defect may occur, the welding defect (specific singular point information) is associated with manufacturing process 4 (specific manufacturing process) that includes the welding process.

[0109] When counting the defect-related information and non-defect-related information in a product group, the line evaluation value calculation unit 54 counts for each manufacturing process 1 to N of the manufacturing line 10 in accordance with the first correspondence table shown in Figures 6 and 7, and can calculate the count value for each manufacturing process 1 to N as a process evaluation value indicating the soundness of each manufacturing process 1 to N.

[0110] The abnormal sign detection unit 55 is not limited to detecting abnormal signs for the entire production line 10 based on the count value (line evaluation value) obtained by weighting and counting the singular points (defect-related information and defect-unrelated information) of the product group as described above, but can also detect abnormal signs for each of the production processes 1 to N of the production line 10 based on the process evaluation value calculated for each production process.

[0111] Furthermore, the output unit 56 can output feedback information including the detection results of abnormal signs detected by the abnormal sign detection unit 55 in each of the manufacturing processes 1 to N of the manufacturing line 10.

[0112] FIG. 8 is a chart showing an example of count values ​​(process evaluation values ​​for each manufacturing process) when the singular points of each product in a product group are counted for each manufacturing process.

[0113] The product groups are product groups in units of lots, and in the example shown in FIG. 8, the count values ​​of the singular points for each product group of lot I, lot II, and lot III are counted for each manufacturing process 1 to N.

[0114] According to the example shown in Figure 8, the product group of Lot III has a significantly higher count value (15) of singular points related to manufacturing process 2 compared to the other product groups of Lots I and II, which suggests that there is a malfunction (sign of abnormality) in manufacturing process 2 of the production line 10.

[0115] Therefore, when the abnormality prediction detection device 30 for the production line obtains feedback information associated with a defect in a specific manufacturing process, it can notify the user of this and prompt them to inspect the specific manufacturing process on the production line 10.

[0116] Furthermore, the processor 32 can calculate a product evaluation value indicating the soundness of each product in the product group based on the singularity information (especially defect-related information) of each product, and may also calculate a line evaluation value by combining the product evaluation values ​​indicating the soundness of each product in the product group.

[0117] <Acquisition of specific environmental information related to singularity information> The abnormality sign detection device 30 for the production line is equipped with a fourth memory (memory 34) that stores a second correspondence table that corresponds specific singularity information included in singularity information related to the singularity of the product P with specific environmental information related to the specific singularity information among multiple environmental information that indicate the production environment in the production line 10.

[0118] FIG. 9 is a diagram showing an example of a second correspondence table in which specific singularity information is associated with related specific environmental information.

[0119] The second correspondence table shown in FIG. 9 indicates the relationship between specific singularity information (type of singularity) and specific environment information (manufacturing environment).

[0120] For example, in a casting process, gas defects may occur due to CO2 entrainment when molten metal is filled. In the example shown in Fig. 9, gas defects and environmental information (excessive CO2) are associated with each other.

[0121] Furthermore, crack defects may occur due to the influence of humidity and welding temperature. In the example shown in Fig. 9, crack defects are associated with environmental information (humidity / welding temperature range unsuitable).

[0122] When processor 32 acquires specific singularity information based on each inspection history corresponding to a product group, it performs a process of acquiring specific environmental information related to the acquired specific singularity information in accordance with the second correspondence table (Figure 9).

[0123] The output unit 56 can output feedback information including specific environmental information related to specific singularity information.

[0124] When feedback information that can associate product P's singularities with the manufacturing environment is acquired, a control signal can be generated for the environmental control device. For example, if a specific type of singularity occurs frequently and the cause of that type of singularity can be narrowed down generally or based on product characteristics or experience (e.g., as shown in Figure 9, gas defects are likely to occur due to high CO2 content, and crack defects are likely to occur when humidity or welding temperature exceeds a certain range), a control signal can be automatically transmitted to air conditioning equipment, welding temperature, and other external environmental control devices to automatically adjust humidity, welding temperature, raw material management environment, etc., and maintain an optimal environment. Note that the user may also manually adjust the air conditioning equipment, welding temperature, etc.

[0125] <Acquisition of additional information corresponding to the quality of the product group> The abnormality sign detection device 30 for the production line includes a fifth memory (memory 34) that stores quality information indicating the quality of each product group and additional information related to the quality information in association with each other.

[0126] FIG. 10 is a diagram showing an example of a correspondence table in which quality information indicating the quality of each product group is associated with additional information related to the quality information.

[0127] The correspondence table shown in FIG. 10 shows the relationship between the quality information indicating the quality of each product group and the additional information.

[0128] As shown in Figure 10, quality information indicating the quality of a product group is, for example, a rank such as best product, excellent product, or good product, and additional information related to the quality information is the processing method including the joining method of the product in the subsequent process, the intended use of the product, etc.

[0129] The quality information for each product group can be determined by a count value (line evaluation value) obtained by counting the singular points of the product group. The processor 32 may also perform a product evaluation value calculation process for each product in the product group, which calculates a product evaluation value indicating the soundness of the product based on the singular point information (particularly defect-related information) of each product, and calculate the quality information of the product group by summing up the product evaluation values ​​of each product in the product group.

[0130] Furthermore, the additional information related to the quality information is not limited to that shown in FIG. 10, but can be created in advance by the user as a report for a product group according to the quality information of the product and the type of product.

[0131] For top-quality products without even the tiniest scratches, sufficient strength can be guaranteed with the relatively simple "screw joining" method during the post-processing, so the additional information "screw joining" can be added. Also, a report can be added stating that the product group can be used in the originally planned equipment.

[0132] For a group of non-defective products that contain minute scratches that do not reach the level of defects, additional information such as "press fitting or high-temperature shrink fitting" as a joining method that can be used to reinforce strength after the fact is added. Also, if sufficient strength can be guaranteed by using a more complex and strong joining method, a report is added stating that the product can be used for specific purpose X under certain conditions.

[0133] Furthermore, for a group of products that are intermediate between the best quality and good quality, additional information such as "press fitting or high temperature shrink fitting" as the joining method can be added, and a report can be issued stating that the product can be used in the equipment for which it was originally intended.

[0134] The processor 32 performs a process of acquiring quality information relating to the quality of the product group based on each inspection history corresponding to the product group, and then performs a process of acquiring additional information corresponding to the quality information from a correspondence table (see FIG. 10) stored in the memory 34 based on the acquired quality information.

[0135] The output unit 56 outputs the acquired additional information corresponding to the product group, thereby making it possible to provide the destination of the product group with additional information (report) appropriately selected according to the quality of the inspected product group.

[0136] <Highlighting of small singularities> FIG. 11 is a diagram showing an example of an image of a product including a minute singular point.

[0137] If the product or part of the product is a casting, minute singular points (potential gas defects) in which tiny bubbles remain may occur within the product.

[0138] In the image shown in FIG. 11(A), there are nine bubbles clustered together on the left side of the image and three bubbles clustered together on the right side of the image.

[0139] The bubbles shown in Figure 11(A) are shown large, but in reality they are of a size that is difficult to visually perceive. Even if image processing is performed to detect bubble (singular point) areas from the image and the bubble areas are highlighted, some bubbles are of a size that is difficult to visually perceive.

[0140] Therefore, in the case of minute singular points that are difficult to distinguish visually, the size and shape of the detected minute singular points are highlighted, so that the position and number of minute singular points contained in the overall image can be easily recognized visually while viewing the overall image of the photographed product.

[0141] The image shown in FIG. 11(B) is superimposed with highlighting information that makes it easy to visually identify areas where minute singularities occur.

[0142] 11(B) is mask information that fills in areas larger than the size of the bubbles, which are minute singular points, with at least one of a specific color and brightness. The area containing nine closely spaced bubbles and the area containing three closely spaced bubbles are each overlaid with mask information and filled in.

[0143] Furthermore, as information corresponding to the number of minute singularities, character information (numbers) indicating the number are superimposed. Note that the mask information in this example has at least one of color and brightness information corresponding to the number of bubbles within the area of ​​the mask information. In this case, even if character information indicating the number of minute singularities is not superimposed, the number can be grasped from the color, etc.

[0144] The processor 32 performs processing to acquire singularity information indicating minute singularities with a singularity size smaller than a threshold value, based on the singularity information acquired by the singularity information acquisition unit 52. The threshold value is preferably set to a value for determining whether the singularity is a minute singularity with a size that is difficult to perceive visually.

[0145] When processor 32 acquires micro-singularity information indicating a micro-singularity, it is preferable to perform a process of generating highlighting information that highlights the micro-singularity, whereby an area that includes the micro-singularity and is larger than the singularity size of the micro-singularity is visibly displayed, and information corresponding to the number of micro-singularities.

[0146] In the case of tiny singularities (in this example, "bubbles") that occur densely across a surface, it is important to understand which area they occur in and how frequently. Therefore, the detection results for each individual bubble are combined to identify the area containing the dense bubbles.

[0147] The processor 32 can acquire an area including densely packed micro singularities by, for example, performing image processing (dilation) to enlarge the area of ​​the micro singularities. When the area of ​​each micro singularity is enlarged at a predetermined enlargement rate, if the micro singularities are densely packed, the enlarged areas will be connected to each other to form an area including densely packed micro singularities. Note that the predetermined enlargement rate is preferably set so that if the micro singularities are densely packed, the enlarged areas will be connected to each other and isolated micro singularities will not be connected to each other.

[0148] The processor 32 generates highlighting information having an area larger than the singularity size, filled with a color or shade corresponding to the number of minute singularities, and further generates character information (numbers) indicating the number of minute singularities.

[0149] The output unit 56 superimposes the emphasis information and information corresponding to the number of minute singular points on the image and displays them on the display unit 36. In the example shown in Fig. 11(B), the emphasis information displayed on the display unit 36 ​​is area information in which areas larger than the singular point size are filled in, and the information corresponding to the number of minute singular points is color or shading information of the filled in area and character information indicating the number of minute singular points.

[0150] The highlighting information displayed on the display unit 36 ​​may be contour information showing the contour of an area larger than the size of the singular point, or frame information (rectangular frame) surrounding the large area, and the information corresponding to the number of minute singular points may be information on at least one of the color and brightness of the filled-in area, or only text information showing the number of minute singular points.Furthermore, the density of minute singular points may be represented by information on at least one of the color and brightness of the filled-in area.

[0151] FIG. 12 is a diagram showing another example of an image of a product including a minute singular point.

[0152] The image shown in Figure 12(A) is an image of a long, thin product (component) or a weld line, and in this image, there are four closely spaced bubbles on the left side of the image and two isolated bubbles on the right side of the image.

[0153] These microscopic singularities made up of air bubbles are difficult to distinguish visually, but by superimposing highlighting information on the image by filling in areas larger than the size of the air bubbles, as shown in Figure 12(B), and further displaying numbers indicating the number of microscopic singularities adjacent to each highlighting information, the position and number of microscopic singularities can be easily recognized visually.

[0154] The image before and after the superimposed display of the emphasis information and the like may be displayed side by side on the display unit 36, or may be configured so that the user can switch between the displayed images.

[0155] When a user visually checks for singularities in the image displayed on the display unit 36 ​​and inputs singularity information (information on the type of singularity, information on the location where the singularity occurs, etc.) using the operation unit 39, the above-mentioned highlighting of minute singularities is effective, particularly when inputting singularity information for minute singularities.

[0156] <Other examples of displaying singularities> The singularity information acquisition unit 52 can be configured by AI as described above, and the AI ​​can be configured by a singularity detection model such as a convolutional neural network (CNN).

[0157] When an image of product P is input to the singularity detection model, it extracts the image features and classifies each pixel of the image to determine which type of singularity it belongs to, thereby extracting the singularity area and estimating the singularity type. The score (= defect probability) estimated by the singularity detection model can be used as follows to perform display processing on the display.

[0158] (1) For each pixel, the distribution of defects and their candidate defects is displayed on the display as a gradient or heat map by making the color darker as the estimated defect probability increases or by changing the hue according to the defect probability.

[0159] (2) A threshold is set, and areas with a certain or higher defect probability are identified, and then the areas are binarized and displayed on the screen.

[0160] (3) In situations where the user does not need to know the detailed shape of each pixel, an area containing pixels above the threshold is identified within an area larger than one pixel (for example, a 24 x 24 pixel rectangle) and displayed within a highlighted frame (= bounding box).

[0161] (4) The area in (3) is highlighted by displaying an arrow or other mark on the display, or by displaying a blinking highlighted frame.

[0162] (5) A list of areas determined to be above the threshold (including the coordinates of the center of gravity and, if necessary, accompanying information such as the detected size and shape) is output in table format or CSV (Comma Separated Value) format.

[0163] Even if only the pixels corresponding to the singular points are emphasized, even if the singular points are minute and therefore easily overlooked compared to the entire image, highlighting or listing them as described above reduces the possibility of them being overlooked, thereby making inspection more efficient.

[0164] (6) For each highlighted or listed area, you can flag or change the color of the area you have checked. You can also add additional information such as the need for reconfirmation.

[0165] When comprehensive checking work is performed, oversights are prevented and inspections are made more efficient.

[0166] (7) By setting two or more thresholds, the “maximum range of defects” when a low threshold is applied and the “minimum range of defects” when a high threshold is applied can be displayed using a double bounding box or two different colors.

[0167] This allows for a more expressive display of the detected area.

[0168] For example, if there is a singularity region with a low defect probability throughout the region and another singularity region with a high defect probability in the center and a low defect probability in the surrounding area, these two singularity regions may be displayed as singularity regions of exactly the same size when binarized using only a single threshold value.

[0169] On the other hand, by setting two or more thresholds, the former singularity area can be displayed in a manner (color or brightness) that suggests it is doubtful whether it is a defect, and the latter singularity area can be displayed in a manner in which the central part is certain to be a defect and the peripheral part is doubtful whether it is a defect.

[0170] In this way, by setting two or more thresholds, the display format of the two singularity areas can be made different, allowing the user to distinguish between the two and focus on checking the latter, which is certain to be a defect. This gives priority to checking rather than treating all items equally, making work more efficient.

[0171] Alternatively, the singularity detection model could output intermediate classes such as "normal," "abnormal," and "ambiguous as to whether it is a defect" from the start, with the intermediate classes being judged by a human, thereby providing flexibility to the output results and clarifying the areas that humans should check as a priority.

[0172] (8) The area exceeding the threshold is displayed in a way that distinguishes whether it exists isolatedly within the part or the entire image, or whether there are multiple areas concentrated within a certain range.

[0173] (9) When it is determined in (8) that multiple defects are concentrated in a certain area, the extent of that area is specifically identified and displayed.

[0174] (10) Calculate the number, area, or density of multiple singular points existing within the region of (9) and display the numerical values.

[0175] (11) The area is colored with brightness, hue, and saturation corresponding to the values ​​calculated in (10), and the area is displayed on the display in a distinguishable manner.

[0176] For example, there is relatively little need to identify each individual porosity (small air bubble) defect candidate, but rather a relatively high need to understand information about where and how many groups of bubbles are present. Therefore, the above display makes inspection more efficient.

[0177] (12) In the pixel distribution after threshold processing of the defect probability, the area or size of the region exceeding the threshold is measured.

[0178] (13) In addition to (12), the similarity to a specific shape, such as circularity, is calculated to determine whether the area exceeding the threshold is "circular" or "flat."

[0179] (14) Based on (12) and / or (13), the degree of abnormality and severity for each region of interest are estimated.

[0180] For example, if there is prior information that a gas defect will appear in a shape close to a circle, the distorted shape may be an unexpected defect, so it is considered to be highly abnormal and a warning is issued.

[0181] In addition, defects with a "crushed shape" or "large size," for example, are considered to be more advanced in damage than other defects, and are therefore more urgently needed and require early detection (i.e., are more serious), prompting inspection.

[0182] (15) The calculated severity or the size of the detected defect area is averaged or summed along a specific area (e.g., a weld line) to sequentially calculate the defect probability for each part or weld location.

[0183] By prioritizing areas that are highly severe or require a person to check again, it becomes easier to respond to abnormalities quickly.

[0184] [Method for detecting abnormalities in manufacturing lines] 13 is a flowchart showing an embodiment of a method for detecting signs of abnormality in a production line according to the present invention. The processing of each step shown in FIG. 13 is performed by the processor 32 of the device 30 for detecting signs of abnormality in a production line shown in FIG.

[0185] 13, the product P manufactured by the manufacturing line 10 is photographed by the radiation imaging device 20 (step S10). The radiation imaging device 20 may be installed on the transport path of the products P manufactured by the manufacturing line 10 and may automatically photograph the products P one by one, or may photograph the products P one by one based on a photographing instruction input from a user.

[0186] The processor 32 acquires an image (X-ray image information) of each product, which is an image taken by the radiation imaging device 20 and is an image of the product P manufactured on the production line 10 (step S12).

[0187] Next, the processor 32 acquires singularity information related to the singularity of the product P based on the image acquired by photographing the product P (step S12). The singularity information can be acquired by AI. The singularity information is one or more of type information, occurrence position information, size information, and shape information of the singularity of the product P.

[0188] The processor 32 stores the singularity information related to the singularity of the product P acquired in step S12 in the memory 34 (first memory) as an inspection history for each product (step S14). In addition, the processor 32 stores, among the singularity information, information that affects the determination of the presence or absence of a defect in the product as defect-related information, and uses the information to determine the presence or absence of a defect in the product. influence The information that is not related to the defect is stored in the memory 34 (second memory) as defect-unrelated information.

[0189] Next, it is determined whether or not the singularity information of one lot of products (plurality of products belonging to the lot number) has been saved in memory 34 (step S16). If the singularity information of one lot of products has not been acquired and saved (if "No"), the process proceeds to step S10, and the processes from step S10 to step S16 are repeated. If the singularity information of one lot of products has been acquired and saved (if "Yes"), the process proceeds to step S18.

[0190] In step S18, processor 32 calculates a line evaluation value indicating the soundness of production line 10 based on the singularity information (defect-related information and defect-non-related information) corresponding to each product group in one lot unit stored in memory 34. The line evaluation value indicating the soundness of production line 10 can be calculated by counting the total number of defect-related information and defect-non-related information in the product group and calculating the count value as the line evaluation value. When counting the total number of defect-related information and defect-non-related information in the product group, it is preferable to weight the defect-related information and defect-non-related information and further to determine the weights of the defect-related information and defect-non-related information according to one or more pieces of singularity information selected from singularity type information, occurrence position information, size information, and shape information.

[0191] The processor 32 detects signs of abnormality in the production line 10 based on the line evaluation value calculated in step S18 (step S20). That is, the processor 32 indirectly detects signs of abnormality in the production line 10 from the quantity of defect-related information and non-defect-related information for the product groups. The processor 32 can also compare the line evaluation values ​​of two or more product groups manufactured at different times and detect signs of abnormality in the production line 10 based on the comparison results.

[0192] The processor 32 outputs feedback information including the detection result of the abnormality sign of the production line 10 detected in step S20 (step S22). The feedback information is output to the display unit 36 ​​that displays the feedback information, or the production line 10 or equipment related to the production line 10.

[0193] The user can check the feedback information displayed on the display unit 36 ​​and determine in advance whether maintenance or inspection work is necessary for the production line 10.

[0194] In addition, by outputting feedback information to the production line 10 or equipment related to the production line 10, it is possible to automatically control the various molds, molten metal temperature, pressure, welding temperature, etc. in the production line 10, and to automatically control the equipment related to the production line 10 (for example, the temperature, humidity, etc. of air conditioning equipment, raw material storage equipment, etc.).

[0195] Figure 13 shows the processing for one lot of products, but the same processing is performed for the next lot of products, making it possible to detect signs of abnormalities in the production line every time one lot of products is manufactured.

[0196] In this example, defect-related information and defect-non-related information corresponding to a product group in a lot unit are used when calculating a line evaluation value indicating the soundness of the production line 10. However, this is not limiting, and multiple inspection histories corresponding to a product group manufactured within a certain period of time or a certain number of product groups manufactured in chronological order can also be used. Furthermore, multiple inspection histories corresponding to a product group manufactured within a certain period of time or a certain number of product groups manufactured in chronological order can be set with overlapping allowed.

[0197] [others] In this embodiment, the products manufactured by the production line are metal products, but the invention is not limited to this and may be non-metal products.

[0198] Furthermore, the imaging device is not limited to a radiation imaging device, and an ultrasonic imaging device or an infrared imaging device may also be used.

[0199] Furthermore, in this embodiment, the hardware structure of a processing unit that executes various processes, such as a CPU (Central Processing Unit), is various processors as follows: The various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0200] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor functions as multiple processing units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a System on Chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0201] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0202] The present invention also includes a production line abnormality sign detection program that, when installed on a computer, causes the computer to function as the production line abnormality sign detection device of the present invention, and a non-volatile storage medium on which the production line abnormality sign detection program is recorded.

[0203] Furthermore, the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0204] 1-N manufacturing process 10 production lines 20 Radiography equipment 22 X-ray source 24 Imaging Plate 30. Anomaly detection device for manufacturing lines 32 processors 34 memory 36 Display section 38 Input / Output Interface 39 Control section 40 Image reader 51 Shooting processing section 52 Singularity information acquisition unit 53 Memory control unit 54 Line evaluation value calculation unit 55 Abnormality sign detection unit 56 Output section 57 Product evaluation value calculation section 58 Defect detection section 59 Defect predictor value calculation unit 60 Information Department A, B, C parts P product S10-S22 steps

Claims

1. An abnormality sign detection device for a production line, comprising a processor, an imaging device, a first memory, and a second memory, The processor: an imaging process of imaging each of the products to be inspected manufactured on the production line using the imaging device; a singularity information acquisition process for acquiring singularity information related to the singularity of the product based on the image acquired by the photographing; a singularity information storage process for storing the acquired singularity information in a first memory; a storing process of storing, in the second memory, information that affects product defects among the singularity information stored in the first memory as defect-related information, and storing, in the second memory, information that does not affect product defects as defect-unrelated information; a product evaluation value calculation process for calculating a product evaluation value indicating the soundness of the product based on the defect-related information; a defect detection process for detecting whether or not there is a defect in the product based on the product evaluation value calculated by the product evaluation value calculation process; a line evaluation value calculation process for calculating a line evaluation value indicating the soundness of the production line based on the defect-related information and the defect-unrelated information; an abnormality sign detection process for detecting an abnormality sign of the production line based on the line evaluation value calculated by the line evaluation value calculation process; and outputting feedback information including the detection results of abnormal signs in the production line and the detection results of defects in the products. An abnormality detection device for manufacturing lines.

2. The processor: a defect predictor value calculation process for calculating a defect predictor value based on the defect non-related information; and performing a notification process of notifying the defect predictive value. The abnormality sign detection device for a manufacturing line according to claim 1.

3. The imaging device is a radiographic device, an ultrasonic imaging device, or an infrared imaging device. The abnormality sign detection device for a manufacturing line according to claim 1 or 2.

4. the singularity information is one or more of type information, occurrence position information, size information, and shape information of the singularity, the storing process classifies the singularity information into the defect-related information and the defect-unrelated information based on one or more of type information, occurrence position information, size information, and shape information of the singularity, and stores the singularity information in the second memory. The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 3.

5. The line evaluation value calculation process includes: calculating a line evaluation value indicating the soundness of the production line based on at least one piece of defect-related information stored in the second memory and at least two pieces of defect-unrelated information stored in the second memory; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 4.

6. The line evaluation value calculation process includes: calculating a line evaluation value indicating the soundness of the production line based on the two or more pieces of defect-related information stored in the second memory and the two or more pieces of defect-unrelated information stored in the second memory; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 4.

7. The line evaluation value calculation process includes: calculating a line evaluation value indicating the soundness of the production line based on the defect-related information and the defect-non-related information corresponding to the plurality of products stored in the second memory; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 4.

8. The plurality of products may be a group of products manufactured within a certain period of time, a group of a certain number of products manufactured in chronological order, or a group of products in one lot, which is a unit for managing products. The abnormality sign detection device for a manufacturing line according to claim 7.

9. the line evaluation value calculation process counts the defect-related information and the defect-non-related information, and calculates the line evaluation value based on the counted values; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 8.

10. the line evaluation value calculation process weights and counts the defect-related information and the defect-non-related information when counting the defect-related information and the defect-non-related information; The abnormality sign detection device for a manufacturing line according to claim 9.

11. the line evaluation value calculation process counts the defect-related information by weighting the defect-related information according to the type of the defect-related information; The abnormality sign detection device for a manufacturing line according to claim 8 or 9.

12. the abnormality sign detection process compares two or more of the line evaluation values ​​and detects an abnormality sign of the production line based on the comparison result; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 11.

13. the manufacturing line includes a plurality of manufacturing processes; the singularity information is one or more of type information, occurrence position information, size information, and shape information of the singularity, a third memory that stores a first correspondence table that associates specific singularity information included in the singularity information with a specific manufacturing process associated with the specific singularity information among the plurality of manufacturing processes of the manufacturing line; the line evaluation value calculation process counts the defect-related information and the defect-non-related information for each of the plurality of manufacturing processes according to the first correspondence table, and calculates the count value for each of the counted manufacturing processes as a process evaluation value indicating the soundness of each manufacturing process; the abnormality sign detection process detects an abnormality sign in each manufacturing process of the manufacturing line based on the process evaluation value calculated for each manufacturing process; The abnormality sign detection device for a manufacturing line according to any one of claims 9 to 11.

14. the singularity information is one or more of type information, occurrence position information, size information, and shape information of the singularity, a fourth memory that stores a second correspondence table that associates specific singularity information included in the singularity information with specific environmental information related to the specific singularity information among a plurality of environmental information indicating a manufacturing environment in the manufacturing line, The processor: A process of acquiring the specific singularity information from the singularity information stored in the first memory; When the specific singularity information is acquired, the specific environmental information related to the acquired specific singularity information is acquired in accordance with the second correspondence table; the output process outputs feedback information including the specific environmental information. The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 13.

15. a fifth memory that stores quality information indicating the quality of each of the product groups and additional information related to the quality information in association with each other; The processor: A process of acquiring the quality information regarding the quality of the product group based on each inspection history of an inspection history group corresponding to the product group; and performing a process of acquiring the additional information corresponding to the quality information from the fifth memory based on the acquired quality information; the output process outputs the additional information acquired corresponding to the product group. The abnormality sign detection device for a manufacturing line according to claim 8.

16. The singularity information includes information on the location and size of the singularity of the product, The processor: a process of acquiring minute singularity information indicating minute singularities having a singularity size smaller than a threshold value based on the singularity information acquired by the singularity information acquisition process; generating emphasis information for highlighting the minute singular points based on the acquired minute singular point information, the emphasis information being for visibly displaying an area that includes the minute singular points and is larger than the singular point size of the minute singular points, and information corresponding to the number of the minute singular points; the output process superimposes the emphasis information and information corresponding to the number of minute singular points on the image and displays the superimposed information on a display; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 15.

17. the highlighting information is mask information for filling an area larger than the singular point size of the minute singular point with at least one of a specific color and brightness, or frame information for surrounding the area; The information corresponding to the number of the minute singular points is character information indicating the number, or at least one of information on the color and brightness of the highlighting information according to the number. The abnormality sign detection device for a manufacturing line according to claim 16.

18. The singularity information acquisition process extracts a feature amount of the image, and acquires a defect probability of the singularity information for each pixel of the image; the output process adds a color corresponding to the defect probability to a pixel corresponding to the singularity information and displays the pixel on a display; The abnormality sign detection device for a manufacturing line according to any one of claims 1 to 17.

19. A manufacturing apparatus including a manufacturing line for manufacturing a product, a processor, an imaging device, a first memory, and a second memory, The processor: an imaging process of imaging the products manufactured on the production line one by one using the imaging device; a singularity information acquisition process for acquiring singularity information related to the singularity of the product based on the image acquired by the photographing; a singularity information storage process for storing the acquired singularity information in a first memory; a storing process of storing, in the second memory, information that affects product defects among the singularity information stored in the first memory as defect-related information, and storing, in the second memory, information that does not affect product defects as defect-unrelated information; a product evaluation value calculation process for calculating a product evaluation value indicating the soundness of the product based on the defect-related information; a defect detection process for detecting whether or not there is a defect in the product based on the product evaluation value calculated by the product evaluation value calculation process; a line evaluation value calculation process for calculating a line evaluation value indicating the soundness of the production line based on the defect-related information and the defect-unrelated information; an abnormality sign detection process for detecting an abnormality sign of the production line based on the calculated line evaluation value; and outputting feedback information including the detection results of abnormal signs in the production line and the detection results of defects in the products. Manufacturing equipment.

20. An inspection device including a processor, an imaging device, a first memory, and a second memory, The processor: an imaging process of imaging each of the products to be inspected manufactured on the production line using the imaging device; a singularity information acquisition process for acquiring singularity information related to the singularity of the product based on the image acquired by the photographing; a singularity information storage process for storing the acquired singularity information in a first memory; a storing process of storing, in the second memory, information that affects product defects among the singularity information stored in the first memory as defect-related information, and storing, in the second memory, information that does not affect product defects as defect-unrelated information; a product evaluation value calculation process for calculating a product evaluation value indicating the soundness of the product based on the defect-related information; a defect detection process for detecting whether or not there is a defect in the product based on the product evaluation value calculated by the product evaluation value calculation process; a line evaluation value calculation process for calculating a line evaluation value indicating the soundness of the production line based on the defect-related information and the defect-unrelated information; an abnormality sign detection process for detecting an abnormality sign of the production line based on the line evaluation value calculated by the line evaluation value calculation process; and outputting feedback information including the detection results of abnormal signs in the production line and the detection results of defects in the products. Inspection equipment.

21. A method for detecting an abnormality sign in a production line, in which a processor detects an abnormality sign in the production line by performing the following steps: a step of photographing the products to be inspected manufactured by the production line one by one using an imaging device; acquiring singularity information related to the singularity of the product based on the image acquired by photographing; storing the acquired singularity information in a first memory; storing, in a second memory, information that affects product defects among the singularity information stored in the first memory as defect-related information, and information that does not affect product defects as defect-unrelated information in the second memory; calculating a product evaluation value indicating the soundness of the product based on the defect-related information; detecting whether or not there is a defect in the product based on the product evaluation value; calculating a line evaluation value indicating the soundness of the production line based on the defect-related information and the defect-unrelated information; detecting an abnormality symptom of the production line based on the line evaluation value; outputting feedback information including the results of detecting abnormal signs in the production line and the results of detecting defects in the products; A method for detecting signs of abnormality in a manufacturing line, including:

22. calculating a defect predictor value based on the defect non-related information; a step of notifying the defect predictor value; The method for detecting an abnormality sign in a manufacturing line according to claim 21, further comprising:

23. The imaging device is a radiographic device, an ultrasonic imaging device, or an infrared imaging device.

23. The method for detecting an abnormality sign in a production line according to claim 21 or 22.

24. 24. A program for detecting an abnormality sign in a production line that causes a computer to execute the method for detecting an abnormality sign in a production line according to any one of claims 21 to 23.

25. A non-transitory computer-readable recording medium having the program according to claim 24 recorded thereon.

26. the abnormality sign detection process compares two or more of the line evaluation values ​​corresponding to two or more of the product groups manufactured at different times, and detects an abnormality sign in the production line based on a comparison result; The abnormality sign detection device for a manufacturing line according to claim 8.

27. The abnormality sign detection process determines that there are no abnormality signs in the production line when the line evaluation value for each product group does not fluctuate, and determines that there are abnormality signs in the production line when the line evaluation value for each product group is on an increasing trend and is approaching a threshold value that is considered to be an abnormality in the production line. The abnormality sign detection device for a manufacturing line according to claim 26.

Images