Welding appearance inspection device, and welding appearance inspection method

Abstract

To enable the quality of a welded part of a cylindrical pipe to be easily determined even at a place having spatial restrictions.SOLUTION: A welding appearance inspection device includes an image acquisition part 48 for acquiring a photographed image of a welded part of a cylindrical pipe, a welding quality determination part 41 for calculating determination information of the welding quality of a welded part of respective images on the basis of a plurality of images acquired by the image acquisition part and obtained by photographing the welded part from different directions, and determining the welding quality of the welded part by a plurality of pieces of determination information of the welding quality. Specifically, the welding quality determination part for segmenting an image obtained by photographing a welded part to detect a welded part area, performs attitude correction for correcting a direction and perspective distortion of the detected welded part area, extracts an outline of the welded part area subjected to the attitude correction, corrects the outline on the basis of the image obtained by photographing the welded part to acquire a welded part outline correction image, and acquires the height of the welded part and meandering of the welded part as the determination information of the welding quality from the welded part outline correction image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a welding visual inspection device and a welding visual inspection method for a welded cylindrical pipe. [Background technology]

[0002] In factories, plants, etc., hollow pipes made of stainless steel or the like are used to safely and efficiently transport liquids or gases to a specified location. When laying this pipe, the pipe is transported to the site in sections or with sections welded together, and then welded and installed on site.

[0003] Insufficient welding during installation can cause the pipe to break or leak liquid or gas inside. Even if there are no defects at the time of welding, poor welding can gradually damage the joints due to aging or external pressure or vibration, causing the liquid or gas inside the pipe to leak to the outside. For this reason, it is necessary to inspect the welded points at the site where the pipe is assembled to ensure that the welding is done correctly.

[0004] There are several types of weld inspections, but the simplest method is visual inspection. Visual inspection does not require special equipment, can be applied anytime, anywhere, and inexpensively, and can provide quick results. However, to make stable judgments with little variation, the inspector must have knowledge and experience. Due to the declining birthrate and aging population, it is predicted that the number of inspectors with welding knowledge and experience will decrease in the future. Therefore, there is a demand for automated judgments of weld quality so that stable judgments can be made regardless of who performs the inspection.

[0005] As a technique for visually inspecting welded portions, Patent Document 1 discloses a method for automatically determining the quality of welding by generating a model for determining the quality of welding for each material and shape using a three-dimensional shape measurement sensor including a camera.

[0006] Patent Document 2 also discloses a method for determining the quality of a weld by comprehensively judging results including inspection results of the weld (bead) using one or more types of artificial intelligence. It also discloses that even when the weld to be inspected constitutes a spatial (three-dimensional) curve, it is possible to deal with this by performing a planarization process on the data of the weld bead obtained along the weld line so that it becomes a straight (one-dimensional) shape based on the torsion rate (i.e., the amount indicating how far the planar curve deviates in the three-dimensional direction).

[0007] Furthermore, Patent Document 3 discloses a technique for inspecting a joint by measuring the welding quality of the joint on the outer periphery of a pipe using a photographing device. The technique discloses a technique in which an inspector (acceptor) moves the inspection device along the bead to take photographs. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] WO 2020 / 129617 (paragraphs 0009 and 0038) [Patent Document 2] WO 2021 / 177435 (paragraphs 0005, 0032, 0080, 0095) [Patent Document 3] Special Publication No. 2023-531389 (Paragraph No. 0005, 0042) Summary of the Invention [Problem to be solved by the invention]

[0009] When inspecting welds on cylindrical piping, the welds are located in a circular ring around the piping, and the entire weld cannot be seen from a single viewpoint, so the techniques in Patent Documents 1 and 2 alone are not applicable to circular welds such as those in pipe welding, making it impossible to determine whether the welds are good or bad. Also, when welding and assembly are performed on-site, spatial constraints such as the presence of other piping or structures near the welds can hinder inspection, making the technique in Patent Document 3 inapplicable.

[0010] An object of the present invention is to make it possible to easily determine the quality of a welded portion of a cylindrical pipe even in a location with limited space. [Means for solving the problem]

[0011] In order to solve the above problem, the welding appearance inspection device of the present invention is provided with an image acquisition unit that acquires photographed images of the welded portion of a cylindrical pipe, and a welding quality judgment unit that calculates judgment information on the welding quality of the welded portion of each image based on multiple images of the welded portion photographed from different directions acquired by the image acquisition unit, and judges the quality of the welding quality of the welded portion based on the multiple pieces of welding quality judgment information. [Effects of the Invention]

[0012] According to the present invention, the quality of a welded portion of a cylindrical pipe can be easily determined even in a location with limited space. [Brief explanation of the drawings]

[0013] [Figure 1] 2 is a schematic diagram illustrating an example of a pipe weld shape for which the welding visual inspection device of the embodiment determines whether the welding is good or bad; FIG. [Figure 2] 10A and 10B are diagrams illustrating the photographing direction when photographing a welded portion of a pipe with a camera. [Figure 3] 1 is a diagram showing an example of an image of a welded portion of a pipe for which a welding visual inspection device judges whether the welding is good or bad; [Figure 4] 1 is a configuration diagram of a welding appearance inspection device according to an embodiment; [Figure 5] FIG. 10 is a flow chart illustrating the processing of the welding appearance inspection device. [Figure 6A] FIG. 10 is a diagram showing a case where images are taken from two opposing directions. [Figure 6B] FIG. 10 is a diagram showing a case where images are taken from three directions. [Figure 6C] FIG. 10 is a diagram showing a case where images are taken from four directions. [Figure 6D] FIG. 10 is a diagram showing a case where images are taken from seven directions. [Figure 7] FIG. 10 is a flowchart illustrating in detail a process for calculating judgment information on the quality of welding of a pipe. [Figure 8] FIG. 10 is a diagram showing an example of the results of segmenting a weld. [Figure 9A] FIG. 10 is a diagram showing an image of a welded portion area. [Figure 9B] FIG. 10 is a diagram showing an image of a welded portion area. [Figure 9C] FIG. 10 is a diagram showing a weld contour image. [Figure 9D] FIG. 10 is a diagram showing a corrected image of a welded portion. [Figure 9E] FIG. 10 is a diagram showing a weld contour image. [Figure 10] FIG. 10 is a diagram illustrating a contour correction process. [Figure 11A] 10A and 10B are diagrams illustrating a method for acquiring quality determination information of pipe welding. [Figure 11B] 10A and 10B are diagrams illustrating a method for acquiring quality determination information of pipe welding. [Figure 12A] FIG. 10 is a diagram showing the shooting direction of a video. [Figure 12B] FIG. 10 is a diagram showing the shooting times and shooting operations at each shooting position. [Figure 13] FIG. 10 is a flow chart illustrating the processing of a video welding appearance inspection device. [Figure 14] FIG. 10 is a flow diagram illustrating detailed operations of weld tracking. [Figure 15A] FIG. 10 is a diagram illustrating detailed processing of the weld tracking module. [Figure 15B] FIG. 10 is a diagram illustrating detailed processing of the weld tracking module. [Figure 16] 10A to 10C are diagrams illustrating details of a process for automatically extracting an inspection target image. [Figure 17A] FIG. 10 is a diagram showing an example of a display of a comprehensive judgment result of the quality of welding of a pipe. [Figure 17B] FIG. 10 is a diagram showing an example of a display of a comprehensive judgment result of the quality of welding of a pipe. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The examples are illustrative for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. First Embodiment

[0015] Fig. 1 is a schematic diagram showing an example of a pipe weld shape for which a welding visual inspection device according to an embodiment judges whether the weld is good or bad. Fig. 1 shows a pipe weld shape in which two cylindrical pipes 2 and 3 are welded at a welded portion 1. The welded portion 1 is a portion welded along the circumferential direction of the pipes 2 and 3, and continues to the back side of the schematic diagram in Fig. 1, forming a cylindrical (annular) shape.

[0016] The cylindrical height of weld 1, i.e., the horizontal length of weld 1 (hatched area) in Figure 1, is called the width of weld 1. Also, the width direction, i.e., the piping direction 4 (horizontal in Figure 1; the arrows in piping direction 4 are for illustrative purposes only and do not physically exist in actual piping), is called the width direction of weld 1. When pipes 2 and 3 are placed vertically or diagonally, the width direction of weld 1 becomes the vertical direction and the diagonal direction, respectively.

[0017] Furthermore, when a weld is photographed as a video or still image, the direction corresponding to the piping direction 4 in the image is the direction of the weld width. For the sake of convenience, the piping direction 4 in Figure 1 is indicated by an arrow on one side (the left direction in Figure 1).

[0018] In the following description, processing will be described for cylindrical pipes with a circular cross section, but the present invention also includes cases where the cross section of the pipe is rectangular or elliptical.

[0019] 2, the photographing direction when photographing the welded portion 1 of the pipe in FIG. 1 with a camera will be described. For ease of explanation, in Figure 2, the pipe weld is drawn with a thinner line and the pipe is drawn with a thicker line than in Figure 1. Also, for ease of explanation, five points A, B, C, D, and E are shown on the weld 1, and these five points are located at the center of the width of the weld 1 (point D is located on the back side of the pipe). The five points A to E are actually located on the circumference of a circle on the same plane, and the center is designated as P.

[0020] Hereinafter, the circle in three-dimensional space that passes through the five points A to E will be called "circle P" (centered at point P), and the plane that includes points A to E and point P will be called the "plane of circle P." The plane of circle P is perpendicular to the piping direction 4. In Figure 2, the piping is viewed from an angle, so circle P is drawn as an ellipse.

[0021] The welding visual inspection device of the embodiment judges the quality of welding using multiple images taken of the weld 1 of the piping from different positions and directions around the weld. The position and direction of the image is the direction of point P on the plane of circle P. For example, the image taking direction is the direction from point t1 on the extension of point A of line segment PA toward point P. In addition, images are taken from points t2, t3, t4, t5, etc. in the direction of point P. In other words, each of the multiple images taken includes a different area of ​​the weld 1 of the piping, and is different from images of the same location taken from various directions.

[0022] 3 is a diagram showing an example of an image of a welded portion 1 of a pipe for which a welding visual inspection device judges the quality of the weld. An original welded portion image 5 is a photographed image, for example, taken from near point t1 in FIG.

[0023] As shown in Figure 3, due to restrictions on the photographing position at the inspection location, the following problems may occur in the original weld image 5. (1) The upward direction (vertical direction) of the camera is misaligned with respect to the piping direction 4 (the piping is rotated counterclockwise in the photograph). (2) The photograph is taken from a position slightly offset upward from the plane of circle P (the weld 1 appears to be a downwardly convex arch). (3) The upper end 2B and lower end 3B of the piping are outside the field of view (not visible in the original weld image 5). (4) Objects (31, 32) other than the piping for which the quality of the weld is to be judged, including other piping (33), are visible in the background of the piping.

[0024] FIG. 4 is a configuration diagram of a welding appearance inspection device according to an embodiment. The welding appearance inspection device is composed of a welding quality determination unit 41 in which a CPU 42, a camera input unit 43, a display control unit 44, a user IF 45, and a storage 46 are logically connected via a bus 47, a camera 48 connected to the camera input unit 43 to capture an original image 5 of the weld (Figure 3), and a display unit 49.

[0025] By connecting a small camera via a flexible cable as camera 48, it becomes possible to take pictures even in an environment with spatial constraints (for example, when the shooting location is small). Camera 48 has the function of taking still images and / or videos in combination with welding quality determination unit 41. In this specification, camera 48 and / or camera input unit 43 may be referred to as an image acquisition unit.

[0026] A display unit 49 is also connected to the display control unit 44. The user IF 45 assists the user of the system in operating the system, and corresponds to keyboard input, mouse input, etc. The display unit 49 is a liquid crystal display or the like, and may also function as part of the user IF using a touch panel or the like.

[0027] The welding quality determination unit 41 and the display unit 49 may be integrated into one unit, and the welding appearance inspection device may be realized by, for example, a smartphone or a tablet terminal. Also, the welding quality determination unit 41, the camera 48, and the display unit 49 may all be integrated into one unit.

[0028] Storage 46 stores original welded portion images 5 captured by camera 48, as well as a welding appearance inspection program. CPU 42 executes the welding appearance inspection program in storage 46, thereby realizing the functions of a welding appearance inspection device.

[0029] Furthermore, the welding quality determination unit 41 may be configured as a network server, and the camera 48 and the display unit 49 may be connected to the welding quality determination unit 41 via the network. Furthermore, the welding appearance inspection program may be stored in network storage.

[0030] Next, the processing of the welding appearance inspection device will be described with reference to the flow chart of FIG.

[0031] In step S51, a plurality of images are inputted by the camera input unit 43. The images are a plurality of images (still images) of the welding portion 1 photographed by the camera 48 from around the welding portion 1.

[0032] In step S52, it is determined whether or not processing of all images input in step S51 has been completed. If completed (YES in S52), the process proceeds to step S55, and if not completed (NO in S52), the process proceeds to step S53.

[0033] In step S53, information for determining whether the pipe is welded is calculated for each image input in step S51. The information for determining whether the pipe is welded includes (A) the width of the weld (maximum, minimum, average), (B) the height of the weld (protrusion or depression from the pipe surface), (C) the meandering of the weld (the deviation (maximum) of the position of each weld width center point from the average position of the weld width center point in the direction of the pipe), (D) color information of the weld or the area around the weld, and (E) the presence or absence of defects such as pit holes, cracks, overlaps, and undercuts. The detailed processing of step S53 will be described later with reference to FIG. 7.

[0034] In step S54, the determination information calculated in step S53 is stored in the storage 46. Then, the process returns to step S52.

[0035] In step S55, a comprehensive judgment of the quality of the pipe welding is made based on the judgment information calculated in step S53. Because the judgment information is data for each image, when making a comprehensive judgment, the following process is performed to integrate the information. · Calculate the overall maximum values: (A) maximum, (B) protrusion, (B) depression, (C) Calculate the overall minimum: (A) minimum Calculate the overall average value: (A) average ·Integrate all data: (D) If there is even one item that applies, mark it as defective: (E)

[0036] When processing the maximum value, minimum value, and all data integration, noise removal processing can be applied to remove values ​​that are clearly farther away from the other data. Applying noise removal processing reduces the influence of a single erroneous data on the overall judgment result, increasing the reliability of the pass / fail judgment process.

[0037] Next, the photographing direction of the welded portion 1 in inputting multiple images in step S51 of FIG. 5 will be explained with reference to FIGS. 6A to 6D. Cross section 61 in FIGS. 6A to 6D shows a cross section of the welded portion 1. The white arrows in FIGS. 6A to 6D indicate the photographing direction. FIG. 6A shows the positions of point P and points A to E in FIG. 3 (points B to E are omitted in the other figures). The number of images taken per welded portion can be any number greater than or equal to two, and it is desirable that all parts of the welded portion 1 appear in one of the photographed images.

[0038] Figure 6A shows a case where images are taken from two opposing directions, capturing approximately half of the entire weld from one location. As shown in the figure, if images are taken from two directions, at points A and D, at an angle 62 with point P as the vertex in Figure 6A, it is possible to cover a certain amount of area with two images. However, in this case, it is not possible to inspect the edges (near points C and E in Figure 6A) when taking each image.

[0039] Figure 6B shows the case where images are taken from three directions. If images are taken from three directions as in Figure 6B, there will be no area at the weld that cannot be photographed, but if the photographing position is slightly off, there is a possibility that some areas will not be photographed.

[0040] Figure 6C shows the case of taking images from four directions. By taking images from four directions as in Figure 6C, it is possible to take images of all sides of the weld, and even if the image taking position is slightly shifted, it is possible to take images of all sides stably.

[0041] FIG. 6D is a diagram showing a case where images are taken from seven directions. If images are taken from more than four directions as in FIG. 6D, no surface can be photographed, and one location on the weld can be judged using multiple photographed images, which is expected to improve the judgment accuracy. The number of photographing directions can be varied for each weld. Furthermore, the photographing directions do not need to be point-symmetrical with respect to point P, and the interval between one photographing position and the next (corresponding to angle 63 in FIG. 6D) does not need to be constant.

[0042] Next, the process of calculating the piping welding quality determination information in step S53 of Fig. 5 will be described in detail with reference to Fig. 7. In the process of calculating the piping welding quality determination information, an image of one weld is input and information required to determine the quality of the weld is output.

[0043] In step S71, the weld area in the image is detected by segmentation. Segmentation is a type of deep learning in which the image and shape of the object to be detected, such as a weld, are learned in advance, and the specified object is extracted (as an area) in pixel units from the input image.

[0044] Segmentation is performed by learning using a large number of pairs of images of welds and mask images (e.g., an image in which pixels in welds are set to level 0 and pixels outside the welds are set to level 255) that show the shape of the weld within the image (e.g., the shape of weld 1 in Figure 3).

[0045] Figure 8 shows an example of the results of weld segmentation. In the segmentation result image 6, pixels identified as welds are shown in hatched areas, and the rest are shown in white, with the position and shape of the welds detected as weld area 7.

[0046] Here, the weld image, weld area image, and weld contour image are defined as follows. The weld image is an image of a weld in a pipe, and may be a color image or a monochrome image.

[0047] A weld area image is an image that shows an area within a weld image that corresponds to a weld, and is an image in which the signal values ​​of pixels that correspond to welds and signal values ​​of images that do not correspond to welds can be clearly distinguished. In this description, the weld area image is expressed as a binary image, with weld areas expressed as black (0) and other areas expressed as white (1). In Figures 8, 9A, and 9B, the weld area image is included in the welding image. Also, the part of the weld area image that corresponds to a weld is called the weld area.

[0048] A weld contour image is an image that shows the contour line of a weld region in a weld region image. For example, Figures 9C and 9E belong to weld contour images, and in these weld contour images, the contour part is shown in black and the rest is shown in white. The part of the weld contour image that corresponds to the contour part is called the weld contour.

[0049] Returning to FIG. 7, in step S72, the posture of weld area 7 in segmentation result image 6 in FIG. 8 is corrected. Specifically, weld area 7 is rotated so that the weld width direction is in the upward direction of the screen, thereby obtaining upright weld area image 10 in FIG. 9A. Then, viewpoint distortion caused by the shooting point being off the plane of circle P in FIG. 2 is corrected, thereby obtaining viewpoint-distortion-free weld area image 11 in FIG. 9B.

[0050] When the weld is viewed from a viewpoint that is off the plane of the circle P, it is equivalent to viewing the circle from an oblique angle, so the weld has the shape of the upper or lower half of an ellipse. The ellipse formula that is closest to the outline points of the upper side 91 and lower side 92 of the weld area shown in Figure 9A is ((xp) / a)^2+((yq) / b)^2=1 (Equation 1) The elliptical distortion can be corrected by calculating the parameters a, b, p, and q (in equation 1, x^2 represents the square of x).

[0051] Specifically, the pixels of the boundary (contour) of the weld area at the horizontal position x that exists in the range of the upper side 91 and the lower side 92 of the weld in the horizontal direction are dy=b × sqrt((1-((xp) / a)^2)) (Formula 2) (In Equation 2, sqrt(x) represents the square root of x.) The direction of movement is downward on the screen if the top side 91 or bottom side 92 is convex upward, and upward on the screen if the top side 91 or bottom side 92 is convex downward.

[0052] It should be noted that a center line (not shown in the figure) formed by the midpoints of the corresponding points may be used instead of the upper side 91 and the lower side 92. In this case, it is possible to prevent erroneous correction when the photographing position is on the plane of the circle P but the camera is close to the welded part, causing the welded part to be distorted into a barrel shape, i.e., when the upper side 91 is convex upward and the lower side 92 is convex downward.

[0053] Alternatively, the parameters of the ellipse may be calculated using all of the upper side 91, lower side 92, and center line. In this case, the amount of data increases, which reduces the influence of errors in the position of each side, making it possible to calculate more accurate ellipse parameters. Note that the ellipse parameters (p, q) represent the center of the ellipse, but the value of p can be the value of the horizontal center of the weld area, thereby reducing the amount of calculation.

[0054] Furthermore, since parameter a indicates half the length of the horizontal axis of the ellipse, it can be calculated from the horizontal lengths of the upper side 91 and lower side 92 or the number of pixels in the thickness direction of the pipe on the screen (pipe width), and can be used in conjunction with the reduction in the amount of calculation related to the p value mentioned above to reduce the amount of calculation.

[0055] 7, in step S73, the contour of the weld is extracted from the viewpoint distortion-free weld area image 11 including the posture-corrected weld area, to obtain the viewpoint distortion-free weld contour image 12 shown in Fig. 9C. The contour is calculated by extracting the position where white pixels and black pixels meet on the image.

[0056] In step S74, detailed correction of the weld contour is performed. That is, the contour position of viewpoint-distortion-free weld contour image 12 is corrected by referring to corrected weld image 13 ( FIG. 9D ) obtained by correcting original weld image 5 of the weld in the same orientation as viewpoint-distortion-free weld contour image 12 and using the same viewpoint-distortion removal parameters.

[0057] A specific example of the contour correction process will be described with reference to Fig. 10. Fig. 10 is an enlarged view of the contour line portion of viewpoint distortion-free weld contour image 12 in Fig. 9C, and contour line 103 corresponds to the contour line of viewpoint distortion-free weld contour image 12 in Fig. 9C. The dotted lines in Fig. 10 are edge information of corrected weld image 13 (Fig. 9D), which relatively accurately represents the shape of the weld.

[0058] The viewpoint distortion-free weld contour image 12 is compared with the edge 101 at a position corresponding thereto, and one point (for example, point 104) on the contour line 103 is moved to the position of the corresponding point (for example, 102) on the edge 101, and the contour line 103 of the viewpoint distortion-free weld contour image 12 is corrected so that it matches the edge position in the corrected weld image 13, thereby obtaining the corrected weld contour image 14 shown in Fig. 9E. This contour correction process reflects information about the detailed shape of the weld that cannot be fully observed by segmentation 71 or missing shape information in the viewpoint distortion-free weld contour image 12.

[0059] Returning to FIG. 7, in step S75, information for determining whether the pipe is welded is acquired. The information for determining whether the pipe is welded is (A) the width of the weld, (B) the height of the weld, (C) the meandering of the weld, (D) color information of the weld or the area around the weld, and (E) the presence or absence of defects such as pit holes, cracks, overlaps, and undercuts. This information will be explained using FIGS. 11A and 11B, which are enlarged views of corrected weld contour image 14 (FIG. 9E) and corrected weld image 13 (FIG. 9D). In FIG. 11A, 115 corresponds to the left side of the pipe, and 116 corresponds to the right side of the pipe.

[0060] (A) Width of the weld The width of the weld is determined by measuring the number of pixels in the portion indicated by reference numeral 111 in the corrected weld contour image 14 (FIG. 11A), and calculating the maximum, minimum, and average values ​​within the measurement range. The measurement range is from 115 to 116 of the weld. Note that the measurement range may be narrower than the range from 115 to 116. In other words, the range is from a certain distance to the right of the left side surface 115 of the weld to a certain distance to the left of the right side surface 116 of the weld.

[0061] In this case, the certain distance may be a fixed number of pixels or a ratio (for example, 10%) of the number of pixels of the pipe width 117. Since the angle of the camera relative to the weld surface is small at both ends of the weld and near the side of the pipe, and distortion may be large, narrowing the measurement range has the effect of reducing errors in the weld width and suppressing the influence of noise due to distortion, thereby calculating stable results.

[0062] In this case, the thickness of the cylindrical pipe is registered in advance, and the actual size per pixel is calculated from the number of pixels corresponding to the thickness of the pipe in the original weld image 5. In this way, the width of the weld may be found in actual size.

[0063] (B) Height of the weld The height of the weld is measured by measuring the height of the weld contour on the outside of the left side surface 115 or right side surface 116 of the pipe in the weld contour corrected image 14 (FIG. 11A), i.e., the number of pixels at the part indicated by the symbol 112 or the similar number of pixels on the right side surface 116. To calculate the values ​​for the two locations on the left and right, the maximum value of the two is found.

[0064] If the background of the pipe is the same color as the pipe weld, it may be difficult to distinguish the boundary, or there may be an error, making it impossible to calculate the height of the weld correctly. If this is difficult to distinguish, the height is calculated as 0. In this case, if the height of the weld can be detected on either the left or right side, the correct value will be selected using the maximum value processing described above.

[0065] Furthermore, when the height values ​​for the left and right sides are calculated directly, more information is used in the processing in pipe welding quality assessment unit 55 in Figure 5, and the accuracy of the final weld height result can be improved by removing noise and outliers based on the frequency distribution of the weld height.

[0066] (C) Weld meandering The meandering of the weld is measured by measuring the maximum value of the vertical difference between the center line 118 (dotted line) of the weld contour in the corrected weld contour image 14 (FIG. 11A) and the line 113 at the average position of the center line 118 in the vertical direction of the screen. The center line of the weld is the set of points at the vertical positions of the centers of the top and bottom edges of the weld contour at the same horizontal position on the image. When measuring the meandering of the weld, the top edge or bottom edge of the weld contour, or both, may be used instead of the center line of the weld contour.

[0067] The line 113 at the average position in the vertical direction of the screen of the center line 118 may be either a straight line or a curve. If it is a straight line, it can be calculated simply by finding the average value of the vertical coordinate of the center line 118. If it is a curve, it can be calculated by arranging the vertical coordinate values ​​of the center line 118 in order of their horizontal coordinate values ​​and performing a smoothing process (for example, low-pass filtering).

[0068] (D) Color information of the weld or the area around the weld Color information of the weld or its surroundings is calculated by examining the colors of multiple pixels to be inspected within the weld 106 or the areas surrounding the weld 107 and 108 using the corrected weld image 13 in FIG. 11B. The color is calculated using hue, saturation, and brightness, and these values ​​of multiple pixels are clustered, and the representative cluster value (hue, saturation, and brightness) is calculated as the color of the weld or its surroundings. Red, green, and blue components may be used instead of hue, saturation, and brightness.

[0069] It is also possible to measure the hue, saturation, and brightness (or red, green, and blue components) of representative colors of the weld (e.g., light brown, dark blue, colorless [achromatic], etc.) in advance, and calculate which color each of the multiple pixels to be inspected in the above-mentioned area is closest to. In this case, a person looking at the measurement results can intuitively grasp the color. In addition, since a corresponding similar color is obtained for each of the multiple pixels to be inspected, the frequency of occurrence can be displayed to express partial coloring or an analog positioning between two colors.

[0070] By expressing the calculated number of colors as a ratio with the number of pixels to be inspected as the parameter, the following information can be expressed, for example. 40% brown, 30% blue: Possibly a color between brown and blue (such as reddish purple). 80% achromatic (gray), 5% light brown: There are some areas that are light brown. The present invention also includes superimposing and displaying the distribution of these representative colors on the corrected weld contour image 14. In this case, by looking at the distribution, it becomes possible to visually understand whether the image is partially colored with a different color or is an intermediate color between two colors.

[0071] The pixels to be inspected may be all pixels in the area or sampled pixels. When sampling, they may be pixels whose positions are mechanically determined according to a predetermined pattern (for example, every 10 pixels horizontally and every 3 pixels vertically) (fixed interval method), or the area may be divided into N equal equal parts horizontally and M equal parts vertically, and the (N-1) x (M-1) points at these division positions may be used as inspection points (fixed point number method). Furthermore, an arbitrary coordinate sequence with unequal intervals may be used as the predetermined pattern, or a method that combines the fixed point number method and the fixed interval method may be used, in which inspection points determined using the fixed point number method are added to each of the inspection points at fixed intervals (for example, pixels located ±5 pixels apart horizontally and vertically).

[0072] (E) Presence or absence of defects such as pit holes, cracks, overlaps, undercuts, etc. The presence or absence of defects such as pit holes, cracks, overlaps, undercuts, etc. is determined using the corrected weld image 13 (FIG. 11B) and deep learning trained on images of each state.

[0073] Second Embodiment In the above embodiment, an example was described in which a welding visual inspection device prepares multiple still images of a weld and makes a comprehensive judgment on the quality of the weld of the weld. Next, an example will be described in which a welding visual inspection device makes a comprehensive judgment on the quality of the weld of a piping weld based on a video taken from the surrounding area of ​​the weld. For this reason, the welding visual inspection device is provided with a function to automatically extract a still image of the inspection target from the input video.

[0074] 12A and 12B are diagrams showing the time-dependent changes in the photographing position and direction of the welded portion and the photographing operation. Similar to Figure 6, Figure 12A shows a cross section 61 of the welded portion 1, and indicates that video is being captured in the direction of the white arrows from the capture positions of points a, b, c, d, e, f, g, and h. Figure 12B shows the capture time and capture operation at each capture position. More specifically, the horizontal direction indicates time, and the bold lines indicate where video is being captured. The letters above the bold lines indicate the capture positions, and the triangles below the bold lines indicate the start and end times of capture.

[0075] Figure 12B (1) shows the process of starting from point a in the camera direction 121, shooting video, and then shooting points b, c, d, e, f, g, and h in the direction of each arrow, before returning to point a and ending the shooting.

[0076] Next, the processing of the welding appearance inspection device of the second embodiment will be described with reference to the flow chart of FIG.

[0077] In step S131, the welding location to be inspected is designated on the screen displayed by the camera 48, and video recording is started.

[0078] In step S132, frames showing welding points are input one by one from the moving image that has started to be shot (hereinafter, these frames are referred to as input frames). At this time, if the content of a frame is the same as that of the immediately preceding frame, the frame may be input after a predetermined time has elapsed.

[0079] In step S133, it is determined whether or not the photographer has performed an operation to end shooting. If the operation to end shooting has been performed (end of S133), the process proceeds to step S55; if the operation to end shooting has not been performed (continuation of S133), the process proceeds to step S134.

[0080] In step S134, an image determined to be a weld in the immediately preceding frame (previous frame) is searched for in the input frame, and a weld tracking process is performed to identify the position (coordinates) of the weld in the input frame. If the weld cannot be found, that is, if the camera movement is too large or the target weld has moved off the screen, tracking of the weld is deemed to have failed. The weld tracking process in step S134 will be described in detail with reference to FIG. 14.

[0081] In step S135, it is determined whether or not the weld tracking in step S134 was successful. If it was successful (Success in S135), the process proceeds to step S136; if it was unsuccessful (Failure in S135), the process returns to step S132 and moves on to processing the next frame.

[0082] In step S136, an image of a predetermined area (size) including the weld is extracted from the input frame based on the position (coordinates) of the weld identified in step S134. At this time, if the extracted image is appropriate, the processing result is "extracted image available," and if it is not appropriate, the processing result is "extracted image not available."

[0083] In step S137, it is determined whether the processing result of step S136 is "extracted image exists." If an extracted image exists (YES in S137), the process proceeds to step S53; if an extracted image does not exist (NO in S137), the process returns to step S132 and proceeds to processing of the next frame.

[0084] The processing in steps S53 to S55 is the same as the processing explained in FIG. 5, so details are omitted, but the quality of the welding of the welded portion designated in step S131 is determined. In the second embodiment, once the inspector (photographer) starts video recording, there is no need to perform operations to capture still images, which simplifies the operation and reduces the time required for the recording operation, enabling efficient inspection of the welded joint.

[0085] Next, the detailed processing of the weld tracking module in step S134 of FIG. 13 will be described. Figure 14 is a flow diagram illustrating the detailed processing of the weld tracking module, which will now be described with reference to Figures 15A and 15B.

[0086] In step S141, the image of the previous frame (image 151 in FIG. 15A) processed immediately before and the position of the welding point in that image (coordinates 152 in FIG. 15A) are stored, and an area 153 around the coordinates 152 of the welding point in the previous frame image 151 is extracted. In FIG. 15A, area 153 is a square range of pixels centered on coordinates 152.

[0087] In step S142, a region 157 similar to region 153 is detected from the input frame. More specifically, a search region 156 is set around coordinate 155 corresponding to coordinate 152 in the input image (image 154 in FIG. 15B), and a part that is most similar to region 153 in the previous frame (FIG. 15A) is searched for within search region 156. In FIG. 15B, region 157 is determined to be most similar, and coordinate 158 of the center of region 157 is set as a candidate for the welding position in the input image.

[0088] In step S143, the error between the region 157 detected in the similar region detection process in step S142 and the region 153 of the previous frame is calculated. The error may be, for example, the mean square error of the difference in signal values ​​between corresponding pixels in the two regions, or the mean absolute error of the difference in signal values.

[0089] In step S144, it is determined whether the error between area 157 and area 153 obtained in step S143 is smaller than threshold Th. If it is smaller than threshold Th (if the error is small or the similarity is high) (YES in S144), the process proceeds to step S145, and if it is equal to or greater than threshold Th (if the error is large or the similarity is low) (NO in S144), the process proceeds to step S146.

[0090] In step S146, the result (return value) of the welding tracking process is set to "tracking failed" and the weld tracking process is terminated.

[0091] In step S145, coordinates 158 of area 157 in Fig. 15B are set as the position of the tracked weld, and the result (return value) of the weld tracking process is set as "tracking successful." Then, the process proceeds to step S147.

[0092] In step S147, for processing the next frame in the weld tracking module, the image 151 of the previous frame held for work and the coordinates 152 of the weld in that image are replaced with the image 154 of the input frame and the coordinates 158 of the newly detected weld, respectively.

[0093] 14 fails, tracking can continue overall based on the following principle. That is, if tracking fails, step S147, which performs the update process, is not executed, and the currently held image and weld position are retained. For example, if the weld temporarily moves out of the captured image, the currently held image and weld position continue to be retained, making it possible to detect similar areas when the weld appears on the screen again.

[0094] If tracking failures continue for a certain period of time or longer, it is likely that the difference between the images held and the images input will become large over time, and so the tracking process will be terminated to prevent false detections. In other words, automatic extraction of inspection target images will not be performed on the video being shot at the time of termination (in the process of step S136 in Figure 13, "No images extracted" is always returned).

[0095] As a result, the video after the point of termination will not be used to calculate the quality of the welds in the pipe. When the tracking process is terminated, it is desirable to notify the photographer by means of a screen or sound to encourage them to take another photograph. In particular, notifying by sound is more effective in encouraging the photographer, who is focusing on the welds in the pipe and the camera position, to take another photograph, which allows for a prompt re-photography and has the effect of shortening the overall inspection time.

[0096] Incidentally, the determination of whether or not to end photography in step S133 in Fig. 13 is premised on an end operation by the photographer. Specifically, this includes an operation of a button on the screen, an operation of a button provided in the system, or an operation by voice or gesture. In addition to an end operation by the photographer, the end determination also includes the following cases (End Determination A) to (End Determination B).

[0097] (End determination A): When a certain period of time has passed since the start of shooting.

[0098] (End judgment 1): When it is judged from the image that the camera has returned to the starting point of shooting. The image at the start of photography is held, and after a certain time has passed, if it is determined that the pattern or mark of the piping or the background of the piping is the same as when photography started, photography is terminated.

[0099] (End determination C): When it is determined that the camera position has returned to the starting point of shooting. As shown in Fig. 12A, photography is performed from various directions toward the center of the cross section 61 of the pipe (center point P of the weld). The camera or a part that moves in the same way as the camera is provided with an acceleration sensor or a function for measuring position, and photography ends when the camera position returns to the same location as the photography start point (for example, point a in 121 in Fig. 12A) or on the straight line (straight line 122 in Fig. 12A) when point a is viewed from the center point P of the weld.

[0100] (End determination D): When the camera rotates 360 degrees. As shown in Fig. 2A, the images were taken from various directions toward the center of the cross section of the pipe (point P, the center of the weld). If we pay attention to the camera direction (arrows) at each point a to h in Fig. 12A, we can see that the camera direction rotates 360 degrees clockwise between the start and end of the image capture.

[0101] Therefore, a function to detect the camera's attitude (shooting direction), such as a geomagnetic sensor or an acceleration sensor, can be provided on the camera or on a part that moves in the same way as the camera, and the camera's attitude (direction) at the start of shooting can be recorded.After starting video shooting, if it is detected that the camera's attitude is again the same as at the start of shooting, it can be determined that the camera has gone around the weld once and shooting is complete, and the shooting can be stopped.

[0102] In addition to the above-mentioned method using a sensor, methods for detecting the camera's orientation also include a method for estimating the camera's orientation from captured video. That is, a method for estimating the camera's orientation from changes in the position or shape of the video by detecting marks such as patterns, marks, or scratches on the surface of the pipe being photographed, or the shape of the pipe such as a bend.

[0103] For example, if there is a mark at position A in Figure 2, the mark will be near the center of the pipe weld when photographed from the direction marked t1 in Figure 2. On the other hand, if photographed from the direction marked t2 in Figure 2, the mark will be located above the pipe weld (when photographed with the camera oriented so that point A is above point B). In this way, the photographing direction can be estimated from the relative positions of the pipe weld and the mark.

[0104] Alternatively, it is possible to estimate the shooting direction from the movement of the background of the pipe being photographed. When estimating from the background, the background at the start of shooting is recorded, and 360-degree shooting is completed when the background becomes the same as the background recorded at the start of shooting. The shooting direction can be roughly estimated from the movement of the background photographed during that time.

[0105] Estimating the camera's posture from these captured videos makes it possible to use cameras without sensors, reducing camera costs and significantly increasing the number of cameras that can be used, improving convenience.

[0106] It is possible to use the landmark-based method and the background-based method together. By using them together, it becomes possible to compensate for sections where the detection accuracy of each method is reduced (for example, sections where there are no landmarks in the landmark-based method, or sections where the background is uniform and movement cannot be detected in the background-based method), thereby further improving the accuracy of camera pose estimation.

[0107] In the second embodiment described above, it is assumed that a 360-degree video recording of the periphery of the welded portion of the pipe is performed in one video recording. However, in reality, depending on the installation conditions of the pipe, it may not be possible to capture the entire periphery of the pipe in one video recording. In such cases, a method is available in which the recording is divided into multiple recordings, such as pausing the recording and then resuming the recording. In this case, the quality of the welded portion is judged using images extracted from the multiple recorded videos.

[0108] Specifically, divided photography is performed using the method described below. In the following explanation, it is assumed that there is a physical constraint near point d in Figure 12A, and that the camera or photographer cannot pass near point d in the vertical direction of the figure (cannot take continuous photographs). Next, the case of divided photography shown in (2) to (4) in Figure 12B will be explained.

[0109] (Split shooting): How to temporarily suspend shooting. As shown in (2) of Figure 12B, the photographer temporarily suspends the shooting process during shooting, i.e., in the frame input process of step S132 of Figure 13, no frame is input and the process remains stopped, and during that time the camera is moved to another position on the pipe and shooting is resumed again.

[0110] In detail, in Fig. 12B (2), the camera is moved from point a to point d, passing through points b and c, along the lower side of the pipe, and photographed, but the area around point d cannot be added, so it is temporarily put on hold.The camera then passes below, to the right, and above the cross section 61 of the pipe in that order, in the opposite direction to when photographing, and moves to the left side of the cross section of the pipe (the bold dotted line in Fig. 12B (2)), and video photographing is resumed again from point d.

[0111] After video recording resumes, the camera passes through points e, f, g, and h, and then returns to point a. When recording resumes at point d, it resumes from approximately the same angle of view as when recording was suspended, making it possible to continue the weld tracking process even when recording is suspended and resumed, and the process for pipe welding visual inspection in Figure 13 can be applied as is.

[0112] (Split Shooting): A method of temporarily stopping shooting and then shooting again. As shown in (3) of Fig. 12B, the shooting is temporarily interrupted when shooting from point a to point d has been completed, and after moving back to point a, shooting is resumed from point a to points h, g, f, and e, and shooting ends at point e. At this time, in determining whether shooting has ended in step S133 of Fig. 13, not only a determination of whether to continue or end, but also three determinations of whether to continue, interrupt, or end are made.

[0113] If the determination is interrupted in step S133 of Fig. 13, the process returns to step S131 and the target welding location is specified again. As a result, until the end of the imaging determination 133 is determined, the extracted images are treated as images of the same welding location and subjected to the overall determination process (step S55).

[0114] In (Split Shooting G), it is necessary to specify the welding location each time shooting is interrupted and resumed, but unlike (Split Shooting F), it is not necessary to resume from the same position as when it was suspended, which increases the flexibility of shooting and improves the overall efficiency of shooting. For example, in (3) of Figure 12B, the point at which shooting resumes can be set to a point different from point d at the time of suspension (point a in the example), and shooting ends at point e. This is because, for example, there is an obstacle between points d and e, making it difficult to shoot this section. (Split Shooting G) can handle such cases.

[0115] In addition, in both of the above-mentioned (division photography F) and (division photography G), the number of times of suspension or interruption may be two or more.

[0116] In the case of (Split Shooting K), it is acceptable if the shooting locations overlap or are partially missing. That is, as shown in (4) of FIG. 12B, the determination process is executed even if the section from point b to point a after shooting resumes is shot in duplicate, or if the section from point f to point d (missing shooting section) is not shot because shooting is performed at point f.

[0117] In this case, since the quality of the pipe weld is judged based on two or more extracted images of similar positions in the overlapping areas, it is expected that defects will not be overlooked and accuracy will be improved.On the other hand, if there is a section where photography is missing, it will not be possible to detect a defective weld if it is only in that section.

[0118] However, since welding defects are generally caused by the skill of the welding worker, when there is a welding defect, there is a high probability that defects will occur in the entire welded part or in other welded parts, and the probability that the defective section will be only a part of it is relatively low. Furthermore, the probability that the missing image section and this defective section will overlap is even lower.

[0119] Therefore, even if there are sections where photography is missing, the results of the pipe welding visual inspection can be reliable to a certain level. For example, in the past, welding inspections often involved sampling inspections, in which only a portion of the welding was inspected. The inspection according to this embodiment provides more reliable inspection results for the entire series of inspections (inspection of the entire welding area, including welds that were not sampled) than sampling inspections.

[0120] In this way, the pipe welding visual inspection device of the embodiment can be applied even when it is not possible to photograph a portion of the pipe. If it is not possible to photograph a portion of the pipe, this fact can be recorded and displayed together with the inspection results, which makes it possible to take supplementary measures such as visually checking the portion again, thereby further improving the reliability of the inspection.

[0121] The pipe welding visual inspection device of the embodiment may calculate the missing image section by recording the camera's shooting position or camera attitude (direction). More specifically, a means for obtaining the camera's position or attitude is implemented, and the shooting position or camera attitude is recorded for each automatically extracted image.

[0122] Then, at the end of imaging, the missing image sections are calculated from the positions or directions of all automatically extracted images. If a missing image section exists, the photographer is notified of the missing image section and its position or direction at the end of imaging or immediately thereafter, which allows additional imaging of the missing image section and improves the quality of the examination. Furthermore, the missing image section may be presented when the examination results are displayed to alert the person checking the examination results.

[0123] The timing of suspending or suspending photography described in Fig. 12B may be combined with (End Determination C) or (End Determination D) in determining the end of photography in step S133 of Fig. 13. That is, by setting the direction or position up to which photography should be performed at the start of photography, photography is automatically suspended or suspended when the specified direction or position is reached.

[0124] Next, the process of automatically extracting an image to be inspected in step S136 in FIG. 13 will be described in detail with reference to FIG.

[0125] In step S161, an image is cut out from the input frame to have a pre-specified image size so that the weld is at the center based on the position (coordinates) of the weld identified in the weld tracking process in step S134 of FIG.

[0126] In step S162, the image extracted in step S161 is inspected for suitability for inspection to determine whether the welding is good or bad. Details of this inspection method will be described later.

[0127] In step S163, it is determined whether or not the application is appropriate. If the application is appropriate (YES in S163), the process proceeds to step S164; if the application is not appropriate (NO in S163), the process proceeds to step S165.

[0128] In step S164, the image cut out from the cut-out image in step S161 is set as an extracted image, which is the processing result (return value) of the process of automatically extracting an image to be inspected, and "extraction confirmed" is set, and the process ends.

[0129] In step S165, the processing result (return value) of the process of automatically extracting the inspection target image is set to "no extraction", and the process ends.

[0130] The method for inspecting the suitability of the extracted image in step S162 will now be described. In step S162, the image is judged to be suitable when it satisfies all or a combination of some of the conditions described below.

[0131] (Appropriate judgment) If the specified time has not passed since the last image that was judged to be appropriate, or if there is not an interval of at least the specified number of frames, the image is judged to be inappropriate.

[0132] (Appropriate judgment) If the extracted image is blurred, it is deemed inappropriate. Blur is determined based on the presence or absence of high frequency components on the screen. Alternatively, the weld tracking unit 124 calculates the amount of movement on the screen from changes in the coordinates of the weld, and if the amount of movement is greater than a certain amount, the image is deemed blurred.

[0133] (Appropriate judgment) If the position of the weld in the cropped image is not near the center, it is deemed inappropriate. If the weld was photographed in the center of the screen when it was taken, the weld will remain in the center of the screen after cropping. However, if the weld was at the edge of the screen when it was photographed, setting the cropped image so that the weld is in the center will result in the cropped image referencing the area outside the screen of the captured image, so the crop position must be shifted toward the center of the input image so that it does not reference the area outside the screen. In this case, the position of the weld in the cropped image will be off-center. In this case, there is a high possibility that the weld will overlap the edge of the screen, so it is deemed inappropriate.

[0134] (Appropriate Judgment) In the subsequent step S53, segmentation is performed in the process of calculating the pipe welding quality judgment information, and if the weld can be detected correctly, it is judged as correct. If the weld cannot be detected, or the detected weld shape is incorrect (not elongated, divided, etc.), or the detected weld overlaps the edge of the screen, it is judged as incorrect.

[0135] In addition, if segmentation is performed, the segmentation process of step S71 in the detailed processing of the process of calculating the judgment information on the welding quality of the piping in step S53 is not performed, and the results of the segmentation performed with appropriate judgment can be used in the processing within the process of calculating the judgment information on the welding quality of the piping in step S53.

[0136] In the second embodiment described above, an example was shown in which the quality of a weld was determined from a video input from a camera. However, a video file may be input instead of a video input from a camera to determine the quality of the weld. In this case, at the beginning of the process, the position of the first image in the video file where the weld to be inspected is located is specified. The video file may also be composed of two or more files. In this case, the quality of the weld is determined by performing processing similar to that of the divided photography (ka) or divided photography (ki).

[0137] By inputting video files, an inspection method can be realized in which only the photographing of the welded parts is carried out first, and the pass / fail determination process is carried out later as a batch process. Furthermore, if multiple welds are captured in one video file, inputting the same video file and specifying different welds at the beginning of processing when specifying the positions of the welds to be inspected will enable the quality of each weld to be determined, reducing the number of times it takes to capture images. This is particularly effective when there are multiple welds on a single straight pipe.

[0138] Next, examples of screen displays of the results of the overall judgment in the overall judgment of the quality of the welding of the piping in step S55 (FIGS. 5 and 13) are shown in FIGS. 17A and 17B.

[0139] 17A and 17B show the results of a judgment based on the pipe welding quality judgment information, as outline shape (column 172), the judgment results of the weld width and height (column 173), the judgment results of the presence or absence of meandering or pit holes (column 174), and the judgment results of the coloring (column 175), and the overall judgment results are displayed in the rows at the bottom of the screen. Both screen display examples are examples in which an overall judgment was made on one weld using images from four directions.

[0140] In FIG. 17A, the original weld image 5 is displayed in column 171, and by specifying an image in column 178, the corresponding image is displayed and the direction in which it was taken is highlighted. 17B, ​​column 180 indicates the photographing direction relative to pipe cross section 182, and by specifying an image in column 178, the corresponding image is displayed and the photographing direction is highlighted. Also, column 180 can intuitively indicate the missing photographing section between section C and section D.

[0141] The display of the judgment results in Figures 17A and 17B can be applied to both the welding appearance inspection device of the first embodiment and the welding appearance inspection device of the second embodiment. When the welding appearance inspection device of the first embodiment displays Figure 17B, it is sufficient to record the shooting position or camera attitude (direction) in the images taken from each direction.

[0142] More specifically, the camera 48 is configured to be equipped with a GPS receiver to acquire the shooting position, or equipped with a gyro, motion sensor, and geomagnetic sensor to acquire the camera attitude (shooting direction) when shooting, and when the camera input unit 43 inputs an image from the camera 48, it acquires the shooting position or camera attitude (shooting direction) corresponding to the image and records it in association with the image (frame) as the camera position and attitude. The camera attitude (shooting direction) may also be calculated on the welding quality determination unit 41 side, such as the camera input unit 43, from the background of the piping in the acquired image.

[0143] As described above, the camera position and orientation can be estimated by associating the captured image of the weld with the image capture position or capture direction, so this can be taken into account when extracting images for calculating the welding quality assessment information. That is, images are extracted when the camera position and orientation change by a predetermined amount. Furthermore, if the camera position and orientation is greater than a predetermined threshold, there is a possibility that an image may be missing, and an alert may be displayed.

[0144] Furthermore, the present invention is not limited to the above-described examples, and various modifications are included. The above-described examples have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.

[0145] The present disclosure includes the following technical ideas. an input unit to which imaging data of the piping photographed from different directions is input; an extraction unit that extracts, from the imaging data, a first image that is an image of a first region that includes a part of a first welding point of the piping, and a second image that is an image of a second region that is an area that includes a part of the first welding point different from the first region; a welding portion specifying unit that specifies welding portions in the first region and the second region based on the first image and the second image; a determination unit that determines whether the identified welded portion is good or bad, A welding visual inspection device characterized in that the judgment unit judges the quality of the first welded point of the piping based on the quality of the welded portions in the first area and the second area judged by the judgment unit.

[0146] Here, the input unit corresponds to the image acquisition unit, the extraction unit corresponds to the processing unit for tracking the welded portion in step S134 (FIG. 13), the welded portion identification unit corresponds to the processing unit for automatically extracting the image to be inspected in step S136, and the judgment unit corresponds to the processing units for calculating the setting information for the quality of the piping weld in step S53 and for making a comprehensive judgment on the quality of the piping weld in step S55.

[0147] More specifically, the first weld is region 153 (FIGS. 14 and 15A), and the first and second images correspond to frames in which weld tracking was successful. The first and second regions correspond to images cut out from the first and second images in step S161 (FIG. 16). [Explanation of symbols]

[0148] 1 Welded section 2 Piping 3 Piping 4 Piping direction 41 Weld quality judgment section 41 Camera input section 42 CPU 43 Camera input unit (image acquisition unit) 44 Display control unit 45 User Interface 46 Storage 48 Camera (image acquisition unit) 49 Display section

Claims

1. an image acquisition unit that acquires a photographed image of a welded portion of a cylindrical pipe; a welding quality determination unit that calculates determination information on the welding quality of each of the images of the welding portion based on a plurality of images of the welding portion taken from different directions and acquired by the image acquisition unit, and determines the quality of the welding of the piping weld based on the plurality of pieces of welding quality determination information; A welding appearance inspection device comprising:

2. 2. The welding appearance inspection device according to claim 1, The welding quality determination unit Segmenting the image of the weld to detect a weld area; Posture correction is performed to correct the direction of the detected weld area and viewpoint distortion. Extracting the contour of the posture-corrected weld area; correcting the contour based on the captured image of the weld to obtain a corrected weld contour image; From the weld contour correction image, the height of the weld and the meandering of the weld are determined as the information for determining whether the weld is good or bad. Asking for, Welding appearance inspection device.

3. 2. The welding appearance inspection device according to claim 1, The welding quality determination unit extracting an image including the welded portion from frames of the moving image of the welded portion acquired by the image acquisition unit, identifying the welded portion based on the image, and calculating judgment information on the quality of the weld; Welding appearance inspection device.

4. 2. The welding appearance inspection device according to claim 1, The welding quality determination unit a part of the weld of the piping is tracked in frames of the moving image of the weld acquired by the image acquisition unit, an image of a predetermined area including the tracked weld is extracted, the weld is identified based on the image, and information on whether the weld is good or bad is calculated; and the quality of the weld of the weld is determined based on the plurality of pieces of information on whether the weld is good or bad. Welding appearance inspection device.

5. 5. The welding appearance inspection device according to claim 4, The welding quality determination unit When tracking the welded portion, tracking of the welded portion is performed on an image of a frame that is a predetermined time after the immediately preceding selected image. Welding appearance inspection device.

6. 5. The welding appearance inspection device according to claim 4, The welding quality determination unit Acquire the position of the image acquisition unit; When the position of the image acquisition unit returns to the position where image capture started, tracking of the predetermined welded portion of the piping is terminated. Welding appearance inspection device.

7. 5. The welding appearance inspection device according to claim 4, The welding quality determination unit Determine whether the image capture unit is continuing, pausing, or ending the capture; A predetermined weld portion of the pipe is tracked in frames of the moving image captured by the image acquisition unit. Welding appearance inspection device.

8. 4. The welding appearance inspection device according to claim 3, The welding quality determination unit Correlating frames of the moving image of the welding part acquired by the image acquisition unit with the position and orientation of the camera when the frames were captured; Welding appearance inspection device.

9. 9. The welding appearance inspection device according to claim 8, The welding quality determination unit extracting an image including the welded portion from a frame of the moving image of the welded portion acquired by the image acquisition unit based on the position and orientation of the camera; Welding appearance inspection device.

10. 9. The welding appearance inspection device according to claim 8, The welding quality determination unit When displaying the judgment result of the welding quality of the welded portion, an image-capturing missing section is displayed based on the position and orientation of the camera. Welding appearance inspection device.

11. 2. The welding appearance inspection device according to claim 1, The welding quality determination unit determining whether the weld is good or bad at a welded portion of the pipe welded in a circumferential direction; Welding appearance inspection device.

12. 2. The welding appearance inspection device according to claim 1, The welding quality determination unit calculating determination information for welding quality based on a plurality of images of the welded portion at different circumferential positions of the pipe; Welding appearance inspection device.

13. acquiring a motion image of a weld of a cylindrical pipe; tracking a portion of the weld of the piping in frames of the acquired motion image of the weld; extracting an image of a predetermined area that includes the tracked weld; a step of identifying a welded portion based on the image and calculating welding quality determination information; determining whether the welding of the welded portion is good or bad based on the plurality of pieces of welding quality determination information; A welding appearance inspection method including:

Images