System and method for inspecting welded sections of piping

Abstract

To improve the accuracy of inspecting welded sections of welded piping.SOLUTION: A system for inspecting welded sections of piping includes: a camera which images piping and a welded section of the piping; an input / output unit which receives an image of the welded section obtained from the camera; a circle detection unit which detects center points of one or more circles formed by ends of a pipe or the welded section, from the image of the welded section received by the input / output unit; a non-welded section estimation unit which estimates a non-welded section region which is not the welded section, based on the center points of the one or more circles; a pre-processing unit which deletes the non-welded section estimated by the non-welded section estimation unit from the image received by the input / output unit to generate a determination image; a determination unit which determines the quality of welding based on the determination image generated by the pre-processing unit; and a registration unit which stores, in a storage, the quality of the welding determined by the determination unit in association with the determination image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a piping weld inspection system and a piping weld inspection method. [Background technology]

[0002] Inspecting the quality of welds that connect pipes in factories and facility construction is a skilled task that requires knowledge and experience. In particular, it is difficult to quantify the shape of the weld bead and the coloring caused by welding burns, so visual sensory testing is generally performed. Weld inspections can be performed using sensing of wavelengths that are different from the visible light to which the human eye is sensitive, such as radiation and ultrasound, but this requires equipment that is more expensive than a camera that is sensitive to visible light.

[0003] In addition, special materials such as magnetic powder and penetrant liquid can be used to make it easier to detect damaged areas, but this makes the inspection procedure complicated and time-consuming.

[0004] In recent years, technology has emerged that uses a deep neural network (DNN), a signal processing technology that mimics the activity of the human brain, to predict appearance quality from images acquired by a camera, automating appearance inspection.

[0005] It is attracting attention as a method that can efficiently conduct visual inspections without using expensive equipment or special materials.

[0006] Patent Document 1 discloses a welding abnormality diagnosis device that includes a camera system that photographs the back side of the workpiece to be welded, a welding part photographing camera image collection unit that collects light-emitting images taken by the camera system when the workpiece is being welded, a welding control information collection unit that collects welding condition setting values, and a welding condition diagnosis unit that calculates welding light feature amounts for diagnosing the welding condition of the workpiece based on the light-emitting image and the welding condition setting values, and judges the quality of the welding based on the welding light feature amounts and upper and lower limit values ​​for welding abnormality judgment. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] WO2021 / 111545 publication Summary of the Invention [Problem to be solved by the invention]

[0008] In Patent Document 1, the pass / fail of the welding quality is judged based on information that needs to be measured during welding work, such as the welding light and welding condition setting values, etc. In other words, there is a problem that after the welding work is completed, it is not possible to capture the welding light with a camera or obtain the welding condition setting values ​​for controlling the welding device.

[0009] This method is difficult to apply when the welding operation and the resulting welded product inspection operation are separated.

[0010] In addition, pre-processing, which removes areas that are not subject to feature extraction and increases the accuracy of the inspection, assumes the presence of welding light, and therefore requires photographing the welded area during the welding process.

[0011] Therefore, when the welding work and the inspection work of the welded product are separate, in order to judge the pass / fail of the weld quality, it is necessary to extract feature amounts from measurable information after the welding work is completed and judge the pass / fail of the welded product based on the feature amounts. To make this judgment with high accuracy, it is necessary to remove areas that are not welded parts from the product image and extract feature amounts only from the welded parts.

[0012] An object of the present invention is to provide a means for inspecting welds in welded pipes, which obtains an image of the weld using a camera that captures images using visible light, without using special materials or expensive equipment, and extracts features useful for determining the pass / fail of the welded product from the area of ​​the weld in the image. [Means for solving the problem]

[0013] The above objective is achieved by a piping weld inspection system including a camera that photographs the welds between pipes, an input / output unit that receives an image of the weld obtained from the camera, a circle detection unit that detects the center points of one or more circles formed by the weld or the end of the pipe from the image of the weld received by the input / output unit, a non-weld estimation unit that estimates non-weld areas that are not welds based on the center points of the one or more circles, a pre-processing unit that deletes the non-weld areas estimated by the non-weld estimation unit from the image received by the input / output unit to create a judgment image, a judgment unit that judges the quality of the welding based on the judgment image created by the pre-processing unit, and a registration unit that associates the quality of the welding judged by the judgment unit with the judgment image and stores them in storage. Effect of the Invention

[0014] The inspection accuracy of welds in welded pipes can be improved. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of an example of an operator terminal according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram illustrating an example of an inspector terminal according to an embodiment of the present invention. [Figure 4] 10 is a flowchart illustrating a process of a worker terminal according to an embodiment of the present invention. [Diagram 5] 10 is a flowchart illustrating a process of an inspection unit according to an embodiment of the present invention. [Figure 6] 11 is a flowchart illustrating a process of a pre-processing unit according to an embodiment of the present invention. [Figure 7] 1 is an example of an input image in an embodiment of the present invention. [Figure 8] 1 is an example of an image after preprocessing in an embodiment of the present invention. [Figure 9] 11 is a graph showing an example of evaluation values ​​in an embodiment of the present invention. [Figure 10]13 is an example of a guide display according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the attached drawings. The attached drawings show specific embodiments according to the principles of the present invention, but these are for understanding the present invention and are not to be used to interpret the present invention in a limited manner. In principle, the same reference numbers are used for common components in each drawing.

[0017] When the welding quality is high, in an image captured so that the welded area is visible, the brightness and color contrast between the welded area and its surrounding area is low, making it difficult to detect the welded area.

[0018] On the other hand, the end of the pipe, which may appear simultaneously with the weld, has a relatively high contrast and is easy to detect. In that case, the weld and the end of the pipe each have a circular shape in the image, and their centers are close to each other. By focusing on this, the center of the circle formed by the weld can be obtained directly or indirectly in a stable manner by detecting the circular shape of the weld or the end of the pipe.

[0019] By removing the outer area of ​​the ring defined based on the detected center point, the proportion of the weld in the entire image increases, making it possible to extract features that are useful for determining whether the welding quality is acceptable or not. EXAMPLES

[0020] Fig. 1 is an example of a system configuration diagram in an embodiment of the present invention. It goes without saying that the present invention may not only be configured as a system, but also as an apparatus in which all or part of the components shown in Fig. 1 are mounted in a housing.

[0021] The piping welded joint inspection system of the embodiment includes an operator terminal 1, a LAN (Local Area Network) 4, and a storage 3 for storing images of welded joints, including an inspector terminal 2. The operator terminal 1 is, for example, a tablet-type electronic terminal, and includes an input device 5, a camera 6, and a display unit 7. A plurality of operator terminals 1 and inspector terminals 2 may be connected.

[0022] The input device 5 is, for example, a touch panel, and instructs image input from the camera 6. Hereinafter, the input device 5 may be referred to as a touch panel. The camera 6 captures an image showing the welded part inside the pipe. The display unit 7 is, for example, a display, and displays position information of the pipe to be inspected and images being captured by the camera 6. Hereinafter, the display unit 7 may be referred to as a display.

[0023] The LAN 4 connects the operator terminal 1, the inspector terminal 2, and a storage 3 such as a cloud in which welding images are stored. The inspector terminal 2 is, for example, a PC, and includes an input unit, a display unit, a storage unit, a CPU, and a memory, and mainly displays inspection information of piping welds acquired by the operator terminal 1.

[0024] 2 is a block diagram of an example of an operator terminal according to an embodiment of the present invention, which includes a CPU (Central Processing Unit) 20, a memory 23, various input / output units 22, and an external storage device.

[0025] The external storage device 21 is, for example, a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and includes an external storage device 21 for storing programs and judgment results related to the pipe welded portion inspection system performed by the worker terminal 1.

[0026] The CPU 20 loads a program stored in the external storage device 21 into the memory 23 and executes it.

[0027] The memory 23 also temporarily stores the programs executed by the CPU 20 and data used by the programs during processing. The programs include a circle detection unit 24 that detects pipe ends and welds from images of welds captured by a camera, a non-weld estimation unit 25 that estimates non-welds based on the detected circles, and a pre-processing unit 26 that deletes the estimated non-welds from the images.

[0028] There is an inspection unit 29 that photographs the welds in the sample pipes based on the weld images created by the pre-processing unit 26, which serve as the criterion for judging whether the weld quality in the inspection is pass or fail, and predicts a score (quality score) that expresses the weld quality in absolute values ​​from the obtained images, and a registration unit 31 that stores the results in the external memory device 21.

[0029] When taking an image, an image taking request confirmation unit 28 accepts an image taking request, an emphasis image generation unit 27 confirms that the weld is shown in an appropriate position in the image, and the taken image is output to the display unit 7. These processing units are all realized as software modules, but may also be realized as hardware.

[0030] 3 is an example of a functional block diagram of an inspector terminal in an embodiment of the present invention. Inspector terminal 2 includes a registration unit 31 and an inspection unit 29, and is connected to a display unit 8 that outputs images. The inspector terminal 2 may be the same as the worker terminal 1.

[0031] The inspector terminal 2 is a terminal for visually confirming the results of the automatic pass / fail judgment of the welds on the product by the worker terminal 1, and for approving the results or for visually judging welds that could not be automatically judged pass / fail by the inspection section of the worker terminal 1.

[0032] Therefore, the final inspection may be performed by an inspector logging in to the system with inspector authority using the same terminal as the worker's terminal.

[0033] The inspection unit 29 can re-inspect the welding image registered by the worker. In this case, the inspector can perform a more advanced inspection by adjusting the parameters required for the inspection and adjusting the image.

[0034] The registration unit 31 approves and registers the automatic inspection results at the worker terminal, and stores the results of the inspector's judgment on the welding images that could not be judged at the worker terminal.

[0035] 4 is a flow chart showing the process of the operator terminal in the embodiment of the present invention. In the registration unit 31, first, the camera 6 is inserted into the inside of a sample pipe that is to be the criterion for judging whether the welding quality is acceptable or not, and starts acquiring an image with the welded part in the image, and reads the image of the latest frame (S1).

[0036] Next, preprocessing unit 26 estimates areas that are not welds (non-welds) in the read image and performs preprocessing to delete the areas (S2). This is a process for accurately performing a quality score prediction process (S5) described later.

[0037] Next, a mask image is generated by painting it black to remove the non-welded area estimated by non-welded estimation unit 25, and the mask image is displayed on display unit 7 while being superimposed on the latest frame image at that time of the video captured by camera 6. Furthermore, a message is displayed to the worker urging him to request an image to be taken if a welded area is captured in a brightly displayed area, that is, an area not estimated to be a non-welded area (S3).

[0038] This is a process to prevent the quality score prediction process (S5) described later from being performed in a state in which there is an error in the estimation of the non-welded area in the pre-processing (S2). After this process, if a welded area is shown in a brightly displayed area in the image displayed on the display unit 7, the worker requests that an image be taken by pressing a button on the input device 5 or the like.

[0039] Next, it is confirmed whether or not an image has been requested by an operation of the worker (S4). If an image has not been requested, the process returns to reading the image of the latest frame of the video captured by the camera 6 (S1). If an image has been requested, next, feature amounts related to the welding quality are extracted from the image from which the estimated non-welded parts have been deleted, and a welding quality score is predicted based on the feature amounts (S5).

[0040] For prediction, a DNN that has learned the relationship between a large number of pairs of image data from which pre-estimated non-welded areas have been removed and welding quality score data may be used. In this case, non-welded area images are input to the DNN, and a predicted welding quality score is obtained as output. The generation of features related to welding quality and the prediction of the quality score based on the features are performed inside the DNN.

[0041] Finally, the score of the welding quality predicted by the registration unit 31 is stored in the judgment result storage unit 32 of the external storage device 21 (S6). At the same time, it may be stored in the external storage 3 via the network. By doing so, the welding image and the judgment result can be shared with the inspector terminal.

[0042] 5 is an example of a flowchart showing the processing of the inspection unit in the embodiment of the present invention. In the inspection unit 29, first, based on the position information of the pipe displayed on the display unit 7 of the worker terminal 1, the worker specifies the pipe to be inspected, inserts the camera 6 into it, starts acquiring an image with the welded part in the image, and reads the image of the latest frame (S10).

[0043] The next pre-processing (S11) to welding quality score prediction (S14) are common to the processing of registration unit 31. Next, the predicted quality score is compared with the quality score of the sample weld stored in storage 3 to determine whether the quality of the piping weld to be inspected can be determined (S15).

[0044] Specifically, the quality score of the sample is s1, the quality score of the inspection target is s2, and the threshold value for determining whether pass / fail judgment is possible or not is th, and the absolute value of the difference between s1 and s2 is compared with th. If the values ​​are equal or th is larger (|s1-s2|≦th), it is determined that automatic pass / fail judgment by image recognition is not possible (S19). Conversely, if th is smaller (|s1-s2|>th), the quality of the piping weld to be inspected is automatically judged as pass / fail (S16).

[0045] Specifically, if s1 and s2 are equal or s2 is greater (s1≦s2), it is judged as pass (S17). Conversely, if s1 is greater (s1>s2), it is judged as fail (S18). After judging pass or fail, the result is notified to the inspector terminal 2 (S20) and the process ends.

[0046] Here, if th is set to a small value and automatic judgment is permitted when the predicted quality scores of the sample and the inspection target are close, the ratio of samples that can be automatically judged among all the pipe samples to be inspected will increase. On the other hand, if the quality score prediction is incorrect, the relationship between the quality scores of the sample and the inspection target may differ from the actual quality, decreasing the accuracy of the automatic pass / fail judgment.

[0047] When th is set to a large value, the proportion of samples for which pass / fail is automatically judged decreases, but the accuracy of pass / fail judgment for the samples to be inspected increases. In other words, th is a parameter that can change the balance between the proportion of automated pass / fail judgment and the accuracy of automatic pass / fail judgment, and an appropriate value is set according to the target value of the system.

[0048] 6 is a flowchart showing an example of the processing of the pre-processing unit in the embodiment of the present invention. The processing will be described below with reference to FIGS.

[0049] 7 is an example of an input image from a camera in an embodiment of the present invention. An input image 40 input from the camera captures an end 41 of a pipe close to the worker, a welded portion 42, and an end 43 of a pipe far from the worker.

[0050] 8 is an example of an image after preprocessing in an embodiment of the present invention. In preprocessing (S2) of input image 40, regions that are not welds (non-welds) are estimated and deleted. Specifically, a circle detection algorithm such as a Hough transform is first applied to input image 40 in which weld 51 or front end 50 and rear end 52 of a pipe form circles, to detect the center positions and radii of those circles (S30).

[0051] At this time, in order to suppress erroneous detection of circles due to image noise, a smoothing filter such as a Gaussian filter or an edge-preserving smoothing filter such as a bilateral filter may be used in advance. Since it is not clear at this point which of the one or more detected circles is the weld, next, the center of the circle formed by the weld is estimated based on the information of the detected circles (S31).

[0052] Specifically, the average value of the positions of the center points of the detected circles is calculated. Since the circles formed by the weld 51 and the front end 50 and the back end 52 of the pipe are nearly concentric, the average value of the positions of those centers roughly corresponds to the position of the center point of the circle formed by the weld 51.

[0053] Welds 51, particularly those with high weld quality, have low brightness and color contrast with the surrounding areas and may be difficult to detect. On the other hand, the front end 50 and the rear end 52 of the pipe have relatively high contrast and are easy to detect. Therefore, even if the weld 51 cannot be detected by circle detection (S30), the position of the center point of the weld 51 can be estimated by the above estimation process. However, if the weld 51 can be detected, the position of the center point of the weld 51 can be found directly. In that case, it is not necessary to prepare an input image 40 in which the front end 50 of the pipe and the rear end 52 of the pipe are captured.

[0054] Next, a circular ring region containing the welded portion 51 is defined based on the center point of the estimated circle of the welded portion 51 (S32). The center point of the ring may be the center point of the welded portion 51 estimated just before, the outer diameter and inner diameter may be the short side of the input image 40, and the inner diameter may be half that length. Also, the position of the end of the pipe may be estimated from the brightness gradient of the input image 40, and the outer diameter and inner diameter of the ring may be determined based on that position.

[0055] Next, the defined ring is superimposed on the input image 40, and image information of the area outside the ring is deleted (S33). Specifically, the input image 40 is cropped so that the long side of the input image 40 matches the outer diameter of the defined ring. Also, the pixel values ​​of the area outside the outer diameter of the ring and the area inside the inner diameter are set to 0, or a smoothing filter such as a Gaussian filter is applied to those areas. Through the above processing, a preprocessed image 53 is obtained from the input image 40.

[0056] FIG. 9 is a graph showing the evaluation values ​​in the embodiment of the present invention.

[0057] Equation 1 is a formula expressing the average luminance gradient from the center point of weld 42 estimated in pre-processing (S2) in registration unit 31 and inspection unit 29 in the embodiment of the present invention.

[0058]

number

[0059] Equation 2 is a formula expressing the likelihood of an end of a tube in the pre-processing (S2) in the registration unit 31 and the inspection unit 29 in the embodiment of the present invention.

[0060]

number

[0061] The process of determining the annulus (S32) is a process of determining an annular region that includes the welded portion 42 based on the result of estimating the center point of the circle formed by the welded portion (S31). The center point of the annulus may be the center point of the estimated circle formed by the welded portion 42, and it is desirable that the outer diameter is outside the welded portion 42 and the inner diameter is inside the welded portion 42.

[0062] The front end 41 of the pipe is on the outside of the welded portion 42, and the rear end 43 of the pipe is on the inside, and these have a high brightness and color contrast with the surrounding areas. Based on this, the outer diameter is set to the diameter of the circle formed by the front end 41 of the pipe, and the inner diameter is set to the diameter of the circle formed by the rear end 43 of the pipe.

[0063] If the luminance of input image 40 at position (r, θ) (r is distance, θ is deflection angle) relative to the estimated center point of weld 42 is denoted as L(r, θ), then the average luminance gradient G(r) in the interval [0, 2π] of θ is expressed by Equation 1. As shown in Fig. 9, the value of G(r) is expressed, for example, for weld 42 as average luminance gradient 61 of the weld, for front end 50 of the pipe as average luminance gradient 62 of the front end of the pipe, and for rear end 52 of the pipe as average luminance gradient 60 of the rear end of the pipe.

[0064] Based on this characteristic, the length redge of the outer or inner diameter of the ring can be calculated as the value of r where G(r) is maximum in the range of r [rmin, rmax] using Equation 2. For the outer diameter, for example, rmin=3 / 4*R, rmax=R, and for the inner diameter, for example, rmin=0, rmax=1 / 2*R. Through the above process, the outer and inner diameters of the ring are set as the diameters of the front end 50 and the back end 52 of the pipe, and the ring that encompasses the weld can be defined.

[0065] 10 is an example of the guide display (S3) in the embodiment of the present invention. The guide display (S3) is a process of displaying on the display unit 7 an image in which the non-welded area 70 deleted in the pre-processing (S2) of the camera image has been darkened, together with a message 71 urging the user to request an image capture if a welded area is captured in a relatively bright area.

[0066] Specifically, the non-welded area 70 deleted in the pre-processing (S2) is displayed in semi-transparent black and overlaid on the input image 40, and is displayed on the display unit 7. This allows the worker to easily check whether the welded area belongs to a light area or a dark area. Furthermore, by also displaying a message 71, an image is requested to be taken in a state where the welded area is not mistakenly deleted in the pre-processing (S2), and the subsequent process, quality score prediction (S5), can be performed with high accuracy.

[0067] It should be noted that the present invention is not limited to the above-mentioned embodiment, but includes various modifications. For example, the above-mentioned embodiment has been described in detail to provide a better understanding of the present invention, and the present invention is not necessarily limited to the embodiment having all of the configurations described.

[0068] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits, etc. In addition, the above-mentioned configurations, functions, etc. may be realized in software, by a processor interpreting and executing a program that realizes each function.

[0069] Information such as programs, tables, files, etc. that realize each function can be stored in storage devices such as non-volatile semiconductor memory, hard disk drives, and SSDs (Solid State Drives), or in non-transitory computer-readable data storage media such as IC cards, SD cards, and DVDs.

[0070] In addition, the control lines and information lines are those considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0071] 1 worker terminal, 2 inspector terminal, 3 storage, 4 LAN, 5 input device, 6 camera, 7, 8 display unit, 20 CPU, 21 external storage device, 22 input / output unit, 23 memory, 24 circle detection unit, 25 non-welding estimation unit, 26 pre-processing unit, 27 enhanced image generation unit, 28 photography request confirmation unit, 29 inspection unit, 30 display unit, 31 registration unit

Claims

1. A camera that photographs the welds between the pipes, an input / output unit for receiving an image of the weld obtained from the camera; a circle detection unit that detects the center points of one or more circles formed by the weld or the end of the pipe from the image of the weld received by the input / output unit; A non-weld estimation unit that estimates a non-weld region that is not a weld based on the center points of the one or more circles; a pre-processing unit that deletes the non-welded region estimated by the non-welded portion estimation unit from the image received by the input / output unit to create a determination image; a judgment unit that judges whether the welding is good or bad based on the judgment image created by the pre-processing unit; A piping weld inspection system including a registration unit that associates the judgement unit's judgement of the quality of welding with an image of the judgement and stores the associated image in storage.

2. 2. The piping weld inspection system of claim 1, The judgment unit judges the quality of the weld based on a quality score, and adds information indicating that the weld cannot be judged to the image of the weld. The registration unit of the piping weld inspection system stores images of welds that cannot be judged separately from images of other welds.

3. 2. The piping weld inspection system of claim 1, The non-welded area estimation unit estimates at least one of the areas inside and outside the circle formed by the center points of the ends as a non-welded area based on the brightness gradient of the image of the weld.

4. 2. The piping weld inspection system of claim 1, an emphasis image generating unit that generates a weld emphasis image by superimposing the non-weld area on the image of the weld; A piping weld inspection system comprising: an output unit that outputs the weld emphasized image generated by the emphasized image generation unit.

5. 5. The piping weld inspection system of claim 4, A piping weld inspection system comprising: an image capture request confirmation unit that confirms that the weld is within a predetermined range within an image when a request to capture an image of a weld is received; and a display unit that displays a message encouraging the weld to be captured within the predetermined range when the weld is not included in the weld-emphasized image.

6. The camera photographs the pipes and their welds, an input / output unit receives the obtained image of the weld; a circle detection unit detecting center points of one or more circles formed by the weld or the end of the pipe from the received image of the weld; A non-weld estimation unit estimates a non-weld region that is not a weld based on the center points of the one or more circles; A preprocessing unit creates a determination image by deleting the estimated non-welded region from the image received by the input / output unit; The judgment unit judges the quality of the weld based on the judgment image, A piping weld inspection method in which a registration unit associates the determined quality of welding with a determined image and stores the result in storage.

Images