MEASUREMENT METHOD, MEASUREMENT APPARATUS, OBJECT MANUFACTURING METHOD, OBJECT QUALITY CONTROL METHOD, OBJECT MANUFACTURING EQUIPMENT, COMPUTING UNIT, IMAGING TERMINAL, IMAGING SYSTEM, AND INFORMATION PROCESSING APPARATUS

Abstract

The measurement method of the present invention includes a first step of converting an image of an object photographed together with a marker plate on which a pattern serving as a length reference for length measurement is drawn, into a front-on image having a predetermined resolution using an image of the pattern in the image, a second step of detecting edges of the object from the front-on image, and a third step of calculating a predetermined physical quantity based on the detected edges.The second step includes a step of selecting a machine learning model that uses the front-on image as input data and the edges in the front-on image as output data according to the type of edge to be detected, a step of detecting edges in the front-on image by inputting the front-on image obtained by the first step into the selected machine learning model, and a step of removing overdetection of edges and / or interpolating undetected edges based on the connectivity of the detected edges.

Description

[Technical field]

[0001] The present invention relates to a measurement method, a measurement device, a manufacturing method for an object, a quality control method for an object, a manufacturing facility for an object, a computing unit, an imaging terminal, an imaging system, and an information processing device. [Background technology]

[0002] In recent years, the API5L standard (see Non-Patent Document 1), which is a standard for steel pipes, has been revised to clarify the description of the shape inspection method of steel pipes. In light of this, it is expected that management of steel pipe shape inspection will become even stricter in the future. Meanwhile, the items for steel pipe shape inspection are diverse, and some of them involve complicated procedures. For example, offset (groove deviation of welded part) must be calculated based on the measurement value of a dial gauge placed on multiple positions on the outer surface of the steel pipe. In addition, the bending of the steel pipe must be calculated based on the float of the steel pipe measured by stretching a water line in the longitudinal direction of the steel pipe, for example. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] AMERICAN PETROLEUM INSTITUTE,API SPECIFICATION 5L FORTY-SIXTH EDITION,APRIL 2018. [Non-Patent Document 2] Xavier Soria, Edgar Riba, and Angel Sappa. “Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection.”, The IEEE Winter Conference on Applications of Computer Vision. 2020, Vol. 1, p. 1912-1921. Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, the items and procedures for the shape inspection of steel pipes are diverse, and therefore the shape inspection is largely performed manually by factory operators. In addition, the shape inspection is performed on multiple surfaces and directions of the steel pipe, such as the outer surface, inner surface, end surface, and length direction. For this reason, the shape inspection takes a lot of time, and it is difficult to perform the shape inspection with high accuracy.

[0005] The present invention has been made to solve the above problems, and an object of the present invention is to provide a measurement method, a measurement device, a calculation unit, an imaging terminal, and an imaging system that can measure a predetermined physical quantity at a predetermined position of an object in a short time and with high accuracy. Another object of the present invention is to provide a manufacturing method for an object that can improve the manufacturing yield of the object. Another object of the present invention is to provide a quality control method for an object that can provide high-quality objects. Another object of the present invention is to provide a manufacturing facility for an object that can improve the manufacturing yield of the object. Another object of the present invention is to provide an information processing device that can improve the manufacturing yield of the object and / or provide high-quality objects. [Means for solving the problem]

[0006] [1] A measurement method according to the present invention is a method for measuring a predetermined physical quantity at a predetermined position of an object, and includes a first step of converting an image of the object taken together with a marker plate on which a pattern serving as a length reference for length measurement is drawn, into a front-facing image having a predetermined resolution by using an image of the pattern in the image, a second step of detecting edges of the object from the front-facing image, and a third step of calculating the predetermined physical quantity based on the detected edges. The second step includes a step of selecting a machine learning model that uses the front-facing image as input data and the edges in the front-facing image as output data according to the type of edge to be detected, a step of detecting the edges in the front-facing image by inputting the front-facing image obtained by the first step into the selected machine learning model, and an edge identification step of removing overdetection of the edges and / or interpolating undetected edges based on connectivity of the detected edges.

[0007] [2] The measurement method according to the present invention, in the above-mentioned invention [1], further includes, before the first step, an imaging step of capturing an image of the object together with the marker plate.

[0008] [3] In the measurement method of the present invention, in the above invention [1] or [2], the edge identification step includes a step of binarizing the edge image, a step of dividing the binarized edge image into a plurality of processing units to generate a plurality of divided binary images, a step of determining whether the edge in each divided binary image is valid or invalid by taking into account the edges in adjacent divided binary images, and a step of interpolating the edges determined to be invalid based on the edges determined to be valid.

[0009] [4] A measuring device according to the present invention is a measuring device that measures a predetermined physical quantity at a predetermined position of an object, and includes a marker plate on which a pattern serving as a length reference for length measurement is drawn, an imaging unit that captures an image of the object together with the marker plate, and a calculation unit that executes the following processes: converting the image into a front-facing image having a predetermined resolution using an image of the pattern in the captured image, detecting edges of the object from the front-facing image, and calculating the predetermined physical quantity based on the detected edges. In the process of detecting the edges, the calculation unit executes the following processes: selecting a machine learning model that uses the front-facing image as input data and the edges in the front-facing image as output data according to the type of edge to be detected, inputting the front-facing image into the selected machine learning model to detect the edges in the front-facing image, and removing overdetection of the edges and / or interpolating undetected edges based on connectivity of the detected edges.

[0010] [5] A method for manufacturing an object according to the present invention includes a manufacturing step of an object, and a measuring step of measuring a predetermined physical quantity at a predetermined position of the object using any one of the above inventions [1] to [3].

[0011] [6] A quality control method for an object according to the present invention includes a measurement step of measuring a predetermined physical quantity at a predetermined position of the object using any one of the above inventions [1] to [3], and a quality control step of performing quality control of the object based on the measurement result of the predetermined physical quantity obtained by the measurement step.

[0012] [7] A manufacturing facility for an object according to the present invention comprises a manufacturing facility for manufacturing an object, and the measuring device of the above invention [4] for measuring a predetermined physical quantity at a predetermined position of the object manufactured by the manufacturing facility.

[0013] [8] A calculation unit according to the present invention is a calculation unit for measuring a predetermined physical quantity at a predetermined position of an object, and executes a process of detecting an edge of the object from a front-facing image calculated from an image of the object taken together with a marker plate on which a pattern serving as a length reference for length measurement is drawn, and a process of calculating the predetermined physical quantity based on the detected edge. In the process of detecting the edge, the calculation unit executes a process of selecting a machine learning model that uses the front-facing image as input data and the edge in the front-facing image as output data according to the type of edge to be detected, a process of detecting the edge in the front-facing image by inputting the front-facing image to the selected machine learning model, and a process of removing overdetection of the edge and / or interpolating undetected edges based on connectivity of the detected edges.

[0014] [9] The calculation unit according to the present invention, in the above-mentioned invention [8], executes a process of converting an image of the object into a front-facing image having a predetermined resolution using the pattern in the image, in order to calculate the front-facing image from the image of the object.

[0015]

[10] The imaging terminal of the present invention is an imaging terminal for measuring a predetermined physical quantity at a predetermined position of an object, and is equipped with an imaging unit that captures an image of the object together with a marker plate on which a pattern serving as a length reference for length measurement is drawn, and a second calculation unit that performs processing on the captured image, and the second calculation unit performs processing to output the captured image to an external first calculation unit which is the calculation unit of the above invention [8] or the above invention [9], and / or is the calculation unit of the above invention [8] or the above invention [9].

[0016]

[11] An imaging system according to the present invention is an imaging system for measuring a predetermined physical quantity at a predetermined position of an object, and comprises a marker plate on which a pattern serving as a length reference for length measurement is drawn, and the imaging terminal of the above invention

[10] for photographing the object on which the marker plate is installed.

[0017]

[12] The information processing device according to the present invention includes a third calculation unit that executes one or more of a process of changing manufacturing conditions of the object and a process of determining the degree of quality of the object, based on a predetermined physical quantity at a predetermined position of the object calculated by an external first or second calculation unit that is the calculation unit of the invention [8] or the invention [9] above, and a plurality of pieces of information about the object from which the physical quantity was calculated.

[0018]

[13] A manufacturing facility for an object according to the present invention comprises a manufacturing facility for manufacturing an object, and an imaging terminal according to the above invention

[10] for measuring a predetermined physical quantity at a predetermined position of the object manufactured by the manufacturing facility. Effect of the Invention

[0019] The measurement method, measurement device, calculation unit, imaging terminal, and imaging system according to the present invention can measure a predetermined physical quantity at a predetermined position of an object in a short time with high accuracy. Furthermore, the manufacturing method of an object according to the present invention can improve the manufacturing yield of the object. Furthermore, the quality control method of an object according to the present invention can provide a high-quality object. Furthermore, the manufacturing equipment of an object according to the present invention can improve the manufacturing yield of the object. Furthermore, the information processing device of the present invention can improve the manufacturing yield of the object and / or provide a high-quality object. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a measurement device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a flowchart showing the flow of a physical quantity measurement process according to the first embodiment of the present invention. [Diagram 3] FIG. 3 is a flowchart showing the flow of the imaging process shown in FIG. [Figure 4] FIG. 4 is a flowchart showing the flow of the projective transformation process shown in FIG. [Diagram 5]FIG. 5 is a schematic diagram showing an example of an image of a welded portion and a marker plate captured by the imaging process. [Figure 6] FIG. 6 is a diagram showing an example of dimensions of the marker plate shown in FIG. [Figure 7] FIG. 7 is a flowchart showing the flow of the detection process shown in FIG. [Figure 8] FIG. 8 is a schematic diagram showing an example of the edge image obtained by the processing in step S32. [Figure 9] FIG. 9 is a flowchart showing the flow of the edge identification process shown in FIG. [Figure 10] FIG. 10 is a diagram for explaining a specific example of the process in step S334 shown in FIG. [Figure 11] FIG. 11 is a diagram for explaining a specific example of the process in step S335 shown in FIG. [Figure 12] FIG. 12 is a diagram for explaining a specific example of the process in step S336 shown in FIG. [Figure 13] FIG. 13 is a diagram for explaining a specific example of the process in step S336 shown in FIG. [Figure 14] FIG. 14 is a diagram showing an example of an edge image obtained as a result of the processes in steps S334 to S336 shown in FIG. [Figure 15] FIG. 15 is a diagram showing an example of the edge image obtained as a result of the processing in step S339. [Figure 16] FIG. 16 is a diagram for explaining the pipe thickness, offset, bead height, inner surface peaking, outer surface peaking, and bead rise angle. [Figure 17] FIG. 17 is a flowchart showing the flow of the measurement process when measuring the pipe thickness. [Figure 18] FIG. 18 is a flowchart showing the flow of the measurement process when measuring the offset and the bead height. [Figure 19-1] FIG. 19-1 is a flowchart showing a flow of a measurement process when measuring the inner surface peaking. [Figure 19-2]FIG. 19-2 is a flowchart showing the flow of the measurement process when measuring outer surface peaking. [Figure 19-3] FIG. 19-3 is a diagram for explaining the measurement process of outer surface peaking. [Figure 20] FIG. 20 is a flowchart showing the flow of the measurement process when measuring the bead rise angle. [Figure 21] FIG. 21 is a schematic diagram showing the configuration of a measurement device according to the second embodiment of the present invention. [Figure 22] FIG. 22 is a block diagram showing a configuration of a main part of the measuring device shown in FIG. [Diagram 23] FIG. 23 is a block diagram showing a configuration of a main part of a modified example of the measuring device shown in FIG. [Figure 24] FIG. 24 is a flowchart showing the flow of processing performed by the imaging terminal and the information processing device in the measuring device shown in FIG. [Diagram 25] FIG. 25 is a block diagram showing the configuration of the main parts of a measurement device according to the third embodiment of the present invention. [Figure 26] FIG. 26 is a flowchart showing a flow of processing performed by the imaging terminal and the information processing device in the measuring device shown in FIG. [Figure 27] FIG. 27 is a block diagram showing the configuration of the main parts of a measurement device according to the fourth embodiment of the present invention. [Figure 28] FIG. 28 is a flowchart showing the flow of processing performed by the imaging terminal and the information processing device in the measuring device shown in FIG. [Figure 29] FIG. 29 is a block diagram showing the configuration of the main parts of a measurement device according to the fifth embodiment of the present invention. [Diagram 30] FIG. 30 is a flowchart showing the flow of processing by the imaging terminal, the first information processing device, and the second information processing device, which is carried out in the measuring device shown in FIG. [Diagram 31] FIG. 31 is a schematic diagram showing a state in which the position where the pipe thickness was measured is superimposed on a photographed image. [Diagram 32] FIG. 32 is a diagram showing the measurement results and error ranges of the pipe thickness in the example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, a measuring device and a measuring method thereof according to first to fifth embodiments of the present invention will be described with reference to the drawings. In the present embodiments, the measuring device measures physical quantities related to the cross-sectional shape of a welded portion of a welded steel pipe, but the present invention is not limited to the present embodiment and can be applied to general processes for measuring physical quantities at a predetermined position of an object that is prone to having edges. As will be described in detail later, examples of physical quantities related to the cross-sectional shape of a welded portion of a welded steel pipe include pipe thickness, offset, bead height, inner surface peaking, outer surface peaking, and bead rise angle.

[0022] [First embodiment] [composition] First, the configuration of a measuring device according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing the configuration of a measuring device according to a first embodiment of the present invention. In this embodiment, the measurement object will be described as a cross section of a welded portion 1c of a welded steel pipe 1. As shown in Fig. 1, a measuring device 20 according to the first embodiment of the present invention is a device that measures physical quantities related to the cross-sectional shape of a welded portion 1c of a welded steel pipe 1, and includes an imaging unit 2, a calculation unit 3, a display unit 4, a memory unit 5, and a marker plate 10.

[0023] The imaging unit 2 is an imaging means for capturing an image of the cross section of the welded steel pipe 1, and is composed of a monochrome camera or a color camera. It is preferable to use a color camera as the imaging unit 2. The imaging unit 2 is preferably designed to ensure a predetermined resolution when capturing an image of the cross section of the welded steel pipe 1. If the installation position of the imaging unit 2 can be changed within a predetermined range, it is preferably designed to ensure a predetermined resolution at all positions within the predetermined range. In addition, the image of the cross section of the welded steel pipe 1 is captured by the imaging unit 2 so as to include a predetermined position for measuring a predetermined physical quantity.

[0024] The calculation unit 3 is a calculation means that performs image processing such as projective transformation on the image data of the cross section of the welded steel pipe 1 captured by the imaging unit 2, and calculates physical quantities related to the cross-sectional shape of the welded part 1c. The calculation unit 3 is composed of a computer equipped with a CPU (Central Processing Unit) or the like.

[0025] The display unit 4 is a display means for displaying information including physical quantities related to the cross-sectional shape of the welded portion 1c calculated by the calculation unit 3. The display unit 4 is configured with a liquid crystal display or the like. In addition to the physical quantities calculated by the calculation unit 3, the display unit 4 may simultaneously display image data of the cross section of the welded steel pipe 1 captured by the imaging unit 2, image data resulting from image processing by the calculation unit 3, and the like.

[0026] The memory unit 5 is a storage means for storing information including physical quantities related to the cross-sectional shape of the welded portion 1c calculated by the calculation unit 3. The memory unit 5 is configured with a hard disk drive or the like. The memory unit 5 may store image data of the cross section of the welded steel pipe 1 captured by the imaging unit 2, image data resulting from image processing by the calculation unit 3, and the like.

[0027] When the imaging unit 2 captures an image of the cross section of the welded steel pipe 1, a marker plate 10 is placed on the cross section of the welded steel pipe 1. The marker plate 10 is for measuring the length of the cross section of the welded steel pipe 1, and a pattern 10a with a known length that serves as a length reference for length measurement is drawn on its surface. As the pattern 10a, it is preferable to use one or more square patterns with one side having a known length, or one or more AR markers (e.g., Aruco), which are markers for three-dimensional surveying, or the like.

[0028] The marker plate 10 is preferably attached to the cross section or surface of the welded steel pipe 1 with a magnet or the like so that it can be easily attached and detached. In order to improve the accuracy of the projective transformation described below, it is preferable that the number of patterns 10a is as large as possible. It is preferable that the welded portion 1c and the patterns 10a are arranged so that they are on the same plane as much as possible. Under normal operating methods, it is preferable that the marker plate 10 has a small structure so that the entire marker plate 10 can be captured when an image of the cross section of the welded steel pipe 1 is captured from the location where the imaging unit 2 is installed.

[0029] In order to realize simple physical quantity measurement, the imaging unit 2, the calculation unit 3, and the display unit 4 may be replaced with an imaging terminal 6 having equivalent functions, as will be described in a second embodiment and the like to be described later.

[0030] [Physical quantity measurement processing] Next, a flow of a physical quantity measurement process according to a first embodiment of the present invention using the measuring device 20 will be described with reference to Fig. 2 to Fig. 20. Fig. 2 is a flowchart showing the flow of the physical quantity measurement process according to the first embodiment of the present invention. The flowchart shown in Fig. 2 starts when an execution command for the physical quantity measurement process is input to the measuring device 20, and the physical quantity measurement process proceeds to step S1.

[0031] In the process of step S1, an operator operates the imaging unit 2 to capture an image of the welded portion 1c, including the bead portion 1a, of the welded steel pipe 1 on which the marker plate 10 is installed (imaging step). The imaging unit 2 inputs image data including the captured images of the welded portion 1c and the marker plate 10 to the calculation unit 3 (imaging process). Details of this imaging process will be described later with reference to Fig. 3. This completes the process of step S1, and the physical quantity measurement process proceeds to the process of step S2.

[0032] In the process of step S2, the calculation unit 3 performs projective transformation of the image data input from the imaging unit 2 into front-facing image data having a predetermined resolution (projective transformation process). Details of this projective transformation process will be described later with reference to Figures 4 to 6. This completes the process of step S2, and the physical quantity measurement process proceeds to the process of step S3.

[0033] In the process of step S3, the calculation unit 3 detects the edge position of the cross section of the welded steel pipe 1 from the front-viewed image data obtained by the process of step S2 (detection process). Details of this detection process will be described later with reference to Figures 7 to 15. This completes the process of step S3, and the physical quantity measurement process proceeds to the process of step S4.

[0034] In the process of step S4, the calculation unit 3 calculates physical quantities related to the cross-sectional shape of the welded portion 1c based on the edge positions detected in the process of step S3 (measurement process). Details of this measurement process will be described later with reference to Figures 16 to 20. This completes the process of step S4, and the physical quantity measurement process proceeds to the process of step S5.

[0035] In the process of step S5, the display unit 4 displays information including the physical quantities related to the cross-sectional shape of the welded portion 1c calculated in the process of step S4. The operator can change the photographing conditions for the image of the welded portion 1c, etc., based on the information displayed on the display unit 4. This completes the process of step S5, and the series of physical quantity measurement processes ends.

[0036] <Image capture processing> Next, the flow of the imaging process in step S1 above will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the imaging process shown in Fig. 2. The flowchart shown in Fig. 3 starts when an execution command for a physical quantity measurement process is input to the measurement device 20, and the imaging process proceeds to step S11.

[0037] In the process of step S11, the operator places the marker plate 10 near the welded portion 1c including the bead portion 1a of the welded steel pipe 1. At this time, it is preferable for the operator to place the marker plate 10 so that the welded portion 1c and the pattern 10a of the marker plate 10 are positioned on the same plane as much as possible. This completes the process of step S11, and the imaging process proceeds to the process of step S12.

[0038] In the process of step S12, an operator operates the imaging unit 2 to capture an image of the welded portion 1c including the bead portion 1a of the welded steel pipe 1 on which the marker plate 10 is installed. At this time, the image of the welded portion 1c is captured so that a predetermined position for measuring a predetermined physical quantity is included in the image. The predetermined physical quantity is a physical quantity that is measured in the subsequent measurement process (step S4). Furthermore, it is preferable that the welded portion 1c and all the patterns 10a drawn on the marker plate 10 are captured at once. The imaging unit 2 inputs image data including the captured images of the welded portion 1c and the marker plate 10 to the calculation unit 3. This completes the process of step S12 and ends the series of imaging processes.

[0039] <Projective transformation processing> Next, the flow of the projective transformation process in step S2 above will be described with reference to Fig. 4 to Fig. 6. Fig. 4 is a flowchart showing the flow of the projective transformation process shown in Fig. 2. The flowchart shown in Fig. 4 starts when the imaging process ends, and the projective transformation process proceeds to step S21.

[0040] In step S21, the calculation unit 3 detects the position of the pattern 10a of the marker plate 10 from the image data input from the imaging unit 2, and detects the coordinates of the pattern 10a on the image (hereinafter referred to as image coordinates). Here, the calculation unit 3 detects the positions of as many patterns 10a as possible from the image data, and lists the image coordinates of the positions (center positions or vertices) of the patterns 10a that have been successfully detected.

[0041] 5 is a schematic diagram showing an example of an image of the welded portion 1c and the marker plate 10 captured by the imaging process. In the example shown in FIG. 5, the marker plate 10 is configured as a rectangular flat plate having a rectangular opening 10b in the center. The multiple patterns 10a are arranged near the four corners of the marker plate 10. The shape of the marker plate 10 and the positions of the patterns 10a are not limited to the shape and positions shown in FIG. 5.

[0042] The image coordinates of the pattern 10a are expressed by the number of pixels along each axis, with the top left vertex of the image as the origin, the right direction of the image as the positive direction of the x-axis, and the bottom direction of the image as the positive direction of the y-axis. For example, if the top left vertex P11 of the top left pattern 10a in FIG. 5 is successfully detected, its image coordinates are determined as follows. That is, starting from the origin at the top left of the image, proceeding x11 pixels along the x-axis direction and y11 pixels along the y-axis direction, the top left vertex P11 is reached, so the calculation unit 3 determines the image coordinates of the top left vertex P11 to be (x11, y11). Similarly, the calculation unit 3 lists the image coordinates (xkn, ykn) of all the successfully detected center positions or vertices Pkn (n=1, 2, . . . , Nk) for each of all the successfully detected patterns k=1, 2, . . . , K.

[0043] When it is desired to obtain many image coordinates from a small number of patterns 10a, the shape of the patterns 10a is preferably a shape having vertices such as a rectangle. On the other hand, when it is desired to facilitate the detection process of the pattern 10a and the image coordinate acquisition process, it is preferable that the shape of the pattern 10a is a point-symmetric shape such as a circle, and the image coordinates of the center position are detected. The pattern 10a may be one for which a detection method and an image coordinate acquisition method have already been established. This completes the process of step S21, and the projective transformation process proceeds to the process of step S22.

[0044] Return to FIG. 4. In the process of step S22, the calculation unit 3 counts the number of vertices Pkn detected in the process of step S21 and judges whether or not projective transformation is possible. In general, for projective transformation, it is necessary to have correspondence between image coordinates and coordinates in the actual three-dimensional space for four or more points. Therefore, if the number of detected vertices Pkn is less than four (step S22: No), the calculation unit 3 judges that projective transformation is not possible, sets a flag indicating that the image transformation has failed, and ends the series of projective transformation processes. If projective transformation is not possible in the projective transformation process, the physical quantity measurement process skips the detection process and proceeds to the measurement process. On the other hand, if the number of detected vertices Pkn is four or more (step S22: Yes), the calculation unit 3 judges that projective transformation is possible, and proceeds to the projective transformation process to the process of step S23.

[0045] In the process of step S23, the calculation unit 3 estimates the parameters of the projective transformation of the captured image data using the image coordinates of the vertices Pkn detected in the process of step S21, and performs projective transformation on the image data. Specifically, first, the calculation unit 3 sets the resolution r of the image data after projective transformation as a parameter before estimating the parameters of the projective transformation. In addition, when performing projective transformation, the calculation unit 3 sets in advance the actual coordinates (hereinafter referred to as "object coordinates") of the vertices Pkn of the pattern 10a. FIG. 6 is a diagram showing an example of the dimensions of the marker plate 10 shown in FIG. 5. The shape of the marker plate 10 and the position of the pattern 10a are not limited to the shape and position shown in FIG. 6.

[0046] As shown in FIG. 6, the object coordinates of the pattern 10a are expressed as a length (for example, in mm) along each axis, with the upper left corner of the marker plate 10 as the origin, the right direction of the marker plate 10 as the positive direction of the X axis, and the lower direction of the marker plate 10 as the positive direction of the Y axis. For example, the object coordinates of the upper left vertex P11 in the upper left pattern 10a in FIG. 6 are determined as follows. That is, starting from the origin at the upper left corner of the marker plate 10, proceeding X11 [mm] along the X axis direction and Y11 [mm] along the Y axis direction, the upper left vertex P11 is reached, so the calculation unit 3 determines the object coordinates of the upper left vertex P11 to be (X11, Y11). Similarly, the calculation unit 3 lists the object coordinates (Xkn, Ykn) of all the center positions or vertices Pkn (n=1, 2, . . . , Nk) that have been successfully detected for each of all the patterns k=1, 2, . . . , K that have been successfully detected.

[0047] From the above, the correspondence between the object coordinates (Xkn, Ykn) and the image coordinates (xkn, ykn) has been clarified for all vertices Pkn that have been successfully detected. Next, the calculation unit 3 calculates the parameters of the projective transformation from these correspondences. Projective transformation is one of the image transformation processes. Projective transformation transforms image data by moving image coordinates (x, y) to different image coordinates (x', y') using the following formulas (1) and (2). In formulas (1) and (2), A, B, C, D, E, F, G, and H are parameters of the projective transformation.

[0048]

number

[0049]

number

[0050] By performing projective transformation using appropriate parameters on the captured image data, the correspondence between one pixel of the projectively transformed image data and the actual length, i.e., the resolution r of the projectively transformed image data, can be determined arbitrarily. By utilizing this, by performing projective transformation on the image data of the photographed weld 1c of the welded steel pipe 1, it is possible to measure the physical quantities related to the cross-sectional shape of the weld on the image.

[0051] To calculate the parameters of the projective transformation, at least four or more "points" arranged on the same plane are photographed, and the correspondence between the object coordinates (X, Y) and the image coordinates (x, y) is obtained. In this embodiment, the center positions and vertices of the pattern 10a are arranged in a predetermined manner, and the correspondence between the object coordinates and the image coordinates is obtained. Therefore, when photographing the marker plate 10 using the imaging unit 2 shown in FIG. 1 or the imaging terminal 6 shown in FIG. 21 described later, it is preferable to photograph not only the welded portion 1c but also four or more center positions or vertices Pkn of the pattern 10a so that they are included in the image data.

[0052] The projective transformation parameters are calculated as follows. First, the calculation unit 3 performs linear transformation on the object coordinates (Xkn, Ykn) of all vertices Pkn using an appropriate coefficient a and constants b and c, and determines the coordinates (xkn', ykn') (in units of, for example, [pix]) where the vertices Pkn should be in the image data after projective transformation. The linear transformation is performed using the following formulas (3) and (4).

[0053]

number

[0054]

number

[0055] The coefficient a is the reciprocal of the resolution r of the image data after the projective transformation. Therefore, it is preferable to determine the resolution r of the image data after the projective transformation in advance and set the value of the coefficient a to the reciprocal of r. The constants b and c are the x-coordinate and y-coordinate of the origin of the object coordinate system in the image data after the projective transformation, respectively. Therefore, it is preferable to determine the coordinates (x0, y0) of the origin of the object coordinates in the image data after the projective transformation in advance and set the values ​​of the constants b and c to b=x0 and c=y0, respectively.

[0056] For example, consider a case where the resolution r of the image data after projective transformation is 0.5 [mm / pix] and the position of the origin of the object coordinates in the image data after projective transformation is (0 [pix], 100 [pix]). In this case, the coefficient a should be set to a = 1 / r = 1 / 0.5 [mm / pix] = 2 [pix / mm], and the constants b and c should be set to 0 [pix] and 100 [pix], respectively.

[0057] Next, the calculation unit 3 calculates the values ​​of parameters A, B, C, D, E, F, G, and H of formulas (1) and (2). The calculation unit 3 determines the values ​​of each parameter so that the image coordinates (xkn, ykn) of the vertex Pkn in the captured image data are transformed into the coordinates (xkn', ykn') where the vertex Pkn should be in the image data after projective transformation, using formulas (1) and (2).

[0058] There are several specific methods for calculating each parameter. For example, if the number of vertices Pkn is exactly four, by substituting the above-mentioned coordinate values ​​into formula (1) and formula (2), the number of formulas and the number of unknowns will match, so that the parameters A, B, C, D, E, F, G, and H can be solved as simultaneous equations. Also, if the number of vertices Pkn is more than four, the values ​​of the parameters A, B, C, D, E, F, G, and H can be determined using a known mathematical method such as the least squares method. The specific method of projective transformation is not limited to the method shown here. This completes the process of step S23, and the projective transformation process proceeds to the process of step S24.

[0059] Returning to FIG. 4, in the process of step S24, the calculation unit 3 performs projective transformation on the image data captured in the imaging process using the projective transformation parameters estimated in the process of step S23. The projective transformation parameters are a combination of parameters A, B, C, D, E, F, G, and H determined in the process of step S23. The image data after projective transformation is generated by replacing the coordinates (x, y) in the image data before conversion with coordinates (x', y') calculated using formula (1) and the above formula (2), while leaving the luminance value or RGB value in the image data before conversion unchanged. This completes the process of step S24, and the series of projective transformation processes ends.

[0060] In the present invention, it is assumed that the welded portion 1c of the welded steel pipe 1, which is the object of measurement, and the marker plate 10 are on the same plane. This assumption is satisfied when the object of measurement is a plane or a curved surface that can be regarded as a plane. Specifically, it is preferable that the curved surface has a variation of about 2 mm or less with respect to a plane parallel to the surface of the marker plate 10. Normally, the variation in a direction parallel to the direction perpendicular to the image plane (the xy plane shown in FIG. 5) (the z direction shown in FIG. 5) cannot be measured from a two-dimensional image. For this reason, when performing three-dimensional measurement of the object of measurement, it is preferable to photograph the object of measurement from multiple directions and obtain multiple images. On the other hand, the present invention omits measurement in a direction perpendicular to the image plane under the assumption that the object of measurement can be regarded as a plane. This enables image correction by projective transformation and simple measurement from a two-dimensional image. Depending on the object of measurement, simpler measurement from a single two-dimensional image is possible.

[0061] <Detection process> Next, the flow of the detection process in step S3 above will be described with reference to Figs. 7 to 15. Fig. 7 is a flowchart showing the flow of the detection process shown in Fig. 2. The flowchart shown in Fig. 7 starts when information on the type of boundary line (edge) to be detected is input after the image conversion process in step S24 above is completed, and the detection process proceeds to step S31. In this embodiment, the boundary line to be detected is specified from among the boundary line between the cross section and the outer surface of the welded steel pipe 1, the boundary line between the cross section and the inner surface of the welded steel pipe 1, and the boundary line of the chamfered portion in the cross section of the welded steel pipe 1, depending on the physical quantity calculated in the subsequent measurement process.

[0062] In the process of step S31, the calculation unit 3 selects one edge detection model to be used in the subsequent process from among the trained edge detection models shown in (1) to (3) below, according to the physical quantity to be calculated in the subsequent measurement process. The edge detection model is a machine learning model in which the image data after projective transformation generated by the projective transformation process is input data, and the image data of an edge image in which the edge part of the welded steel pipe 1 in the image data after projective transformation is emphasized is output data. The edge image is, for example, a binary image in which non-edge parts are white and parts estimated to be edges are black. Alternatively, the edge image is, for example, a monochrome image having 256 gradations in which the value of the probability or reliability of each pixel of the input image being an edge is standardized in the range of 0 to 255, and the pixel value of each pixel of the input image is replaced with the value of the probability or reliability.

[0063] (1) A model for detecting the boundary between the cross section and the outer surface of a welded steel pipe 1 (2) A model for detecting the boundary between the cross section and the inner surface of the welded steel pipe 1 (3) A model for detecting the boundary line of the chamfered part (the line indicating the boundary between the processed surface and the non-processed surface) in the cross section of the welded steel pipe 1

[0064] The edge detection model is generated by machine learning using multiple training data consisting of a set of image data after projective transformation and image data of an edge image extracted from that image data. The edge detection model is an edge detection method using image processing, deep learning, etc., and parameter information related to the edge detection process (e.g., weight information obtained as a result of learning), and is preferably specialized for detecting the edge position of the cross section of the welded steel pipe 1. In particular, when deep learning is used, it is preferable to use one having a network structure specialized for edge detection (e.g., see Non-Patent Document 2).

[0065] Depending on the physical quantity calculated in the subsequent measurement process, there may be edges that do not need to be detected, so the calculation unit 3 may detect a predetermined minimum number of edges using only a part of the edge detection models. For example, when calculating the pipe thickness in the measurement process, all of the edge detection models (1) to (3) above are used. On the other hand, when calculating the offset, bead height, outer surface peaking, etc. in the measurement process, it is sufficient to use only the edge detection model (1) above. Also, when calculating the inner surface peaking, it is sufficient to use only the edge detection model (2) above. Which edge detection model is used is determined according to the information on the type of edge input at the start of the detection process. This completes the process of step S31, and the detection process proceeds to the process of step S32.

[0066] In the process of step S32, the calculation unit 3 inputs the image data after the projective transformation obtained by the image transformation process of step S24 to the edge detection model selected in the process of step S31, thereby extracting an edge image in the image data after the projective transformation. The processes of steps S31 and S32 are repeatedly executed the number of times corresponding to the number of types of boundary lines to be detected. This completes the process of step S32, and the detection process proceeds to the process of step S33.

[0067] In the process of step S33, the calculation unit 3 extracts edge positions from the edge image extracted in the process of step S32 (edge ​​identification process). Details of this edge identification process will be described later with reference to Figures 8 to 15. This completes the process of step S33, and the detection process proceeds to the process of step S34.

[0068] In the process of step S34, the calculation unit 3 determines whether or not all pre-specified edges have been detected. If the result of the determination is that all pre-specified edges have been detected (step S34: Yes), the calculation unit 3 ends the series of detection processes. On the other hand, if all pre-specified edges have not been detected (step S34: No), the calculation unit 3 returns the detection process to the process of step S31.

[0069] (Edge identification processing) Next, the edge identification process in step S33 will be described in detail with reference to Figs. 8 to 15. Fig. 8 shows an example of an edge image obtained by the process in step S32. In Fig. 8, a dashed line 100 indicates an actual edge position, and a line 101 indicates a portion detected as an edge in the process in step S32. In the process in step S32, overdetection and underdetection of edges may occur depending on the properties of the outer surface of the welded steel pipe 1 and the state of the background. Therefore, in the edge identification process, the calculation unit 3 removes overdetection of edges and interpolates underdetection of edges based on the connectivity of the detected edges. In the following description, it is assumed that the edge is projected to cross the image in the x-axis direction in the process in step S24, but the edge may be projected to cross the image in the y-axis direction. If the edge is projected to cross the image in the y-axis direction, the edge position can be extracted by switching the x-axis and y-axis in the following description.

[0070] Fig. 9 is a flowchart showing the flow of the edge identification process shown in Fig. 7. The flowchart shown in Fig. 9 starts when the process of step S32 ends, and the edge identification process proceeds to the process of step S331.

[0071] In the process of step S331, the calculation unit 3 binarizes the edge image generated by the process of step S32. If the edge image is a binary image, this process is unnecessary. If the edge image is a monochrome image having gradations, the calculation unit 3 binarizes the edge image using an appropriate gradation threshold. The binarized image obtained by this process is, for example, an image in which non-edge parts are white and parts estimated to be edges are black. This completes the process of step S331, and the edge identification process proceeds to the process of step S332.

[0072] In the process of step S332, the calculation unit 3 initializes a flag array indicating whether an edge is valid or invalid. More specifically, in the image after projective transformation generated in the process of step S24, the edge is photographed so as to cross the x-axis direction of the image. For this reason, the calculation unit 3 prepares and initializes an array for storing the y coordinate of the edge and an array of flags indicating whether the edge is valid or invalid for each x coordinate. The array for storing the y coordinate of the edge is an array that stores the same number of numerical values ​​as the image width, and records the y coordinate of the edge calculated in the process described below for each x coordinate. The array for storing the y coordinate of the edge is initialized with a missing value. In addition, the flag array indicating whether the edge is valid or invalid is an array that stores the same number of true / false values ​​as the image width, and indicates whether the value of the y coordinate of the edge for each x coordinate is correct or not. If it can be determined that the calculated value of the y coordinate of the edge is correct, the flag is set to valid (true), and if it cannot be determined that it is correct, the flag is set to invalid (false). The flag array indicating whether the edge is valid or invalid is initialized with a value indicating invalid (false). This completes the process of step S332, and the edge identification process proceeds to step S333.

[0073] In the process of step S333, the calculation unit 3 generates a plurality of divided binary images by dividing the binary image (width W, height h) obtained in the process of step S331 into rectangular images with a width of 1 and a height of h. This completes the process of step S333, and the edge identification process proceeds to the process of step S334.

[0074] In the following, the calculation unit 3 executes the processes of steps S334 to S336 in ascending order of x coordinates of the divided binary images generated by the process of step S333. When the execution of the processes of steps S334 to S336 for one divided binary image is completed, the calculation unit 3 extracts a divided binary image from the x coordinate position obtained by incrementing the x coordinate of the divided binary image for which the processing has been completed by 1, and repeats the execution of the processes of steps S334 to S336 again. From the way in which the x axis of the image coordinate system is taken, this process means that the generation of rectangular divided binary images is repeated starting from the left end of the original binary image until the right end is reached.

[0075] In the process of step S334, the calculation unit 3 extracts pixels that can be edges from the divided binary image, and judges whether there is a single or multiple edge candidates in the divided binary image based on the extracted pixels. Specifically, the calculation unit 3 first lists the y coordinates of pixels judged to be edges included in the divided binary image, and generates an edge candidate point y coordinate array {yn} (n=1, 2, . . . , Nx). Nx is the number of pixels judged to be edges in the divided binary image extracted from a certain x coordinate. Then, the calculation unit 3 calculates the difference Δy between the maximum and minimum values ​​of the edge candidate point y coordinate array {yn}, and compares it with a preset threshold value TΔy. This process makes it possible to easily judge whether the entire edge candidate point y coordinate array {yn} constitutes one edge.

[0076] A specific example of the process of step S334 is shown in Fig. 10(a) and (b). In the example shown in Fig. 10(a) and (b), in the divided binary image extracted from the position of x = x1, there is only one black edge candidate (edge ​​candidate E1). In this case, the y coordinates of the pixels determined to be edges are likely to be close to each other. On the other hand, in the divided binary image extracted from the position of x = x2, there are two edge candidates (edge ​​candidates E2, E3). In this case, the y coordinates of the pixels determined to be edges are likely to be scattered over a wide range. Therefore, if the upper limit of the y coordinate width of one edge in the divided binary image is set as the threshold value TΔy, it is possible to determine whether one or more edge candidates exist in the divided binary image. This completes the process of step S334, and the edge identification process proceeds to the process of step S335.

[0077] Return to FIG. 9. In the process of step S335, the calculation unit 3 calculates the y coordinate of the edge in the divided binary image based on the edge candidate point y coordinate array {yn} obtained by the process of step S334. The y coordinate of the edge represents the y coordinate of the edge at a certain x coordinate in the original binary image, and is hereinafter expressed as yx (x=1, 2, . . . , W). In the process of step S335, the process is performed by distinguishing between cases depending on whether there is a single edge candidate in the divided binary image or multiple edge candidates. When there is a single edge candidate in the divided binary image, the calculation unit 3 sets the average value ynm of the edge candidate point y coordinate array {yn} as the y coordinate yx of the edge in the divided binary image. On the other hand, when there are multiple edge candidates in the divided binary image, the calculation unit 3 sets the edge candidate point y coordinate array {yn} that is closest to the y coordinate y(x-1) of the edge in the left adjacent divided binary image, i.e., the divided binary image processed immediately before, as the y coordinate yx of the edge. If the y coordinate y(x-1) of the edge in the divided binary image to the left is a missing value, the calculation unit 3 also sets the y coordinate yn of the edge in the divided binary image to be processed to a missing value.

[0078] A specific example of the processing when there are multiple edge candidates in the segmented binary image is shown in FIGS. 11(a) and 11(b). In the examples shown in FIGS. 11(a) and 11(b), there are multiple edge candidate points P1 to P3 in the segmented binary image extracted from the position x = x3. Among these, the edge candidate point P2, whose y coordinate is close to the y coordinate of the edge obtained from the x coordinate position of x = x3 - 1 which is the left neighbor, is determined as the true edge, and its y coordinate is stored as the y coordinate yn of the edge. On the other hand, the edge candidate points P1 and P3 are determined not to be true edges. The pixels corresponding to the edge candidate points P1 and P3 in the figure are marked with an 'x'. By this processing, when there is a single edge candidate in the segmented binary image, the position of the edge can be selected, and when there are multiple edges, the position of the edge that is considered to be most closely connected to its left neighbor can be selected. Thereby, the processing of step S335 is completed, and the edge identification processing proceeds to the processing of step S336.

[0079] Return to FIG. 9. In the processing of step S336, the arithmetic unit 3 determines whether the y coordinate yx of the edge calculated in the processing of step S335 is reasonable. The processing of step S336, similar to the processing of step S335, performs the processing by classifying according to whether there is a single or multiple edge candidates in the segmented binary image. When there is a single edge candidate in the segmented binary image, the arithmetic unit 3 compares the difference Δys (= |yx - y(x - 1)|) between the y coordinate yx of the edge at the current x coordinate position and the y coordinate y(x - 1) of the adjacent left edge with a preset threshold value TΔys. Then, when Δys < TΔys, the arithmetic unit 3 determines that the y coordinate yx of the edge at the current x coordinate position is valid. On the other hand, when Δys ≥ TΔys, the arithmetic unit 3 determines that the y coordinate yx of the edge at the current x coordinate position is invalid. Specifically, as shown in FIGS. 12(a) and 12(b), the edge candidates extracted from the positions x = x4 and x = x5 are both single, but at the position x = x4 where the difference Δys in the y coordinate from the left neighbor edge is smaller than TΔys, the edge is determined to be connected and valid. On the other hand, at the position x = x5 where the difference Δys in the y coordinate from the left neighbor edge is greater than or equal to TΔys, the edge is determined not to be connected and invalid. The invalid pixels are shown hatched in the figure.

[0080] On the other hand, when there are multiple edge candidates in the divided binary image, the arithmetic unit 3 compares the difference Δym (= |yx - y(x - 1)|) between the y - coordinate yx of the edge at the current x - coordinate position and the y - coordinate y(x - 1) of the left - adjacent edge adjacent to yx with a preset threshold value TΔym. Then, when Δym < TΔym and the y - coordinate y(x - 1) of the left - adjacent edge is valid, the arithmetic unit 3 determines that the y - coordinate yx of the edge at the current x - coordinate position is valid. On the other hand, if not, the arithmetic unit 3 determines that the y - coordinate yx of the edge at the current x - coordinate position is invalid. Specifically, as shown in FIGS. 13(a) and (b), there are multiple edge candidates in the divided binary images extracted from the positions of x = x6 and x = x7. At the position of x = x6 where the difference Δys in the y - coordinate from the left - adjacent edge is smaller than TΔys, the edges are connected and are judged to be valid. On the other hand, at the position of x = x7 where the difference Δys in the y - coordinate from the left - adjacent edge is greater than or equal to TΔys, the edge is not connected and is judged to be invalid. The invalid pixels are shown hatched in the figure. Further, regarding x = x7 + 1, which is the right - adjacent of x = x7, although the difference Δys in the y - coordinate from the obtained left - adjacent edge is smaller than TΔys, it is judged to be invalid because the edge at x = x7 is judged to be invalid.

[0081] Return to FIG. 9. The process of step S336 has the effect of detecting a sudden change in the y coordinate of an edge during the process, and estimating that this is due to an erroneous detection in the process of step S32 and eliminating it. Therefore, the threshold value TΔys and the threshold value TΔym may be set to the maximum amount of variation in the y coordinate allowed for connected edges. The reason why it is not determined whether the edge on the left side is valid when there is a single edge candidate is that if an edge is not detected in a divided binary image of a certain x coordinate and a single edge appears in the divided binary image to the right of the edge candidate, the former edge is always invalid. Conversely, the reason why it is determined whether the edge on the left side is valid when there are multiple edge candidates is that if an edge is not detected in a divided binary image of a certain x coordinate and multiple edges appear in the divided binary image to the right of the edge candidate, it is not possible to determine which edge should be selected, so the determination is withheld until the next single edge candidate appears. This process is intended to expect that, even if the edges detected in step S32 include erroneous detections, a single correct edge candidate will appear in the divided binary image with a relatively high probability. Thus, the process of step S336 is completed, and the edge identification process proceeds to step S337.

[0082] When the above processing of steps S334 to S336 is completed for all divided binary images, the processing results are stored in both the edge y coordinate array and the flag array indicating the valid / invalid determination. FIG. 14 shows a conceptual diagram of the processing results. First, the image is divided into a valid section and an invalid section based on the flag array indicating the valid / invalid determination. When the values ​​of the edge y coordinate array included in the valid section are plotted, it will look like edge 101a in the figure. Edge 101a is an edge that is determined to have been correctly detected among the edges detected by the processing of step S32. On the other hand, the invalid section is a section where no edge was detected in the processing of step S32, or where an edge was detected but the correct edge could not be determined.

[0083] Returning to FIG. 9, in the process of step S337, the calculation unit 3 determines whether or not the processes of steps S334 to S336 have been completed for all divided binary images. If the result of the determination is that the processes of steps S334 to S336 have been completed for all divided binary images (step S337: Yes), the calculation unit 3 advances the edge identification process to the process of step S339. On the other hand, if the processes of steps S334 to S336 have not been completed for all divided binary images (step S337: No), the calculation unit 3 increments the x coordinate of the divided binary image to be processed by 1 in the process of step S338, and then returns the edge identification process to the process of step S333.

[0084] In the process of step S339, the calculation unit 3 interpolates the edges of each invalid section based on the edge y coordinate array {yxn} of the valid section adjacent to the invalid section and the corresponding x coordinate array {xn}. n is the number of edge coordinates in the valid section used in the interpolation process. As a specific interpolation method, for example, it is assumed that the edge shape in the invalid section can be modeled by the quadratic function formula shown in the following formula (5), and a known fitting method (such as the least squares method) may be applied to {xn} and {yxn}.

[0085]

number

[0086] If the result of the interpolation process causes a discontinuity of the edge between the invalid section and the adjacent valid section, the calculation unit 3 may further convert the y coordinate of the edge by linear conversion so that the discontinuity of the edge disappears. Specifically, as a result of the interpolation process using formula (5) for an invalid section having a range of xL≦x≦xR, the coordinates of the left end of the invalid section are (xL, yL) and the coordinates of the right end are (xR, yR). In this case, the coordinates of the right end of the valid section adjacent to the left are (xL-1, yL'), the coordinates of the left end of the valid section adjacent to the right are (xR+1, yR'), and further, yL ≠ yL' or yR ≠ yR'. In this case, the calculation unit 3 can eliminate the discontinuity of the edge between the invalid section and the valid section by linearly converting the y coordinate of the invalid section using formula (6) shown below. This is equivalent to subtracting a straight line connecting the endpoints of the valid sections adjacent to both sides of the invalid section from the edge shape formed by the y coordinate of the invalid section calculated by equation (5).

[0087]

number

[0088] Before the interpolation process, pre-processing may be performed, such as changing an extremely short valid section to an invalid section, or changing an extremely short invalid section to a valid section and changing its y coordinate to an appropriate value (for example, by replacing it with the average value of the edge y coordinates of nearby valid sections). FIG. 15 shows an example of an edge image obtained as a result of the process of step S339. By combining the interpolated edge 101b with the edge 101a obtained by the process up to that point, it is possible to eliminate erroneous detections (overdetection and non-detection) and correctly detect edges. This completes the process of step S339, and the series of edge identification processes ends.

[0089] <Measurement processing> Next, the flow of the measurement process in step S4 above will be described in detail with reference to Figs. 16 to 20. In the measurement process, the calculation unit 3 calculates the physical quantities of the measurement target by image processing. As described above, the image data obtained by photographing the welded portion 1c including the bead portion 1a of the welded steel pipe 1 has been subjected to projective transformation, so the calculation unit 3 can calculate the physical quantities related to the cross-sectional shape of the welded portion from the image. The operation of the calculation unit 3 when measuring the pipe thickness, offset, bead height, inner surface peaking, outer surface peaking, and bead rise angle will be described below.

[0090] As shown in FIG. 16(a), the pipe thickness indicates the thickness of the base material 1b of the welded steel pipe at a position circumferentially away from the center position of the bead portion 1a by a certain distance. As shown in FIG. 16(b), the offset indicates the difference between the height position of one end of the bead portion 1a in the circumferential direction and the height position of the other end of the bead portion 1a in the circumferential direction (the deviation of the height of the groove). As shown in FIG. 16(c), the bead height indicates the difference between the height position of the end of the bead portion 1a in the circumferential direction and the height position of the center of the bead portion 1a (the height of the welded portion). As shown in FIG. 16(d) and (e), the peaking is an index of the sharpness of the butt portion. As shown in FIG. 16(d), the inner surface peaking indicates the size of the gap between the template K that is in contact with the inner surface of the welded steel pipe and the inner surface of the welded steel pipe. As shown in FIG. 16(e), the outer surface peaking indicates the amount of unevenness of the actual outer surface with respect to a virtual circle (dotted line portion) along the outer surface of the welded steel pipe. As shown in FIG. 16(f), the bead rise angle refers to the rise angle of the circumferential end of the bead portion 1a (the angle of the end of the weld reinforcement) with respect to the surface of the base metal 1b of the welded steel pipe.

[0091] When measuring the pipe thickness shown in Fig. 16(a), as shown in Fig. 17, first, the circumferential position of the welded steel pipe where the pipe thickness is to be measured is determined (step S411). Next, the calculation unit 3 identifies the thickness direction of the welded steel pipe at the position determined in step S411 on the image and calculates the pipe thickness (step S412). Then, the calculation unit 3 applies a correction to the calculated pipe thickness (step S413). This makes it possible to measure the pipe thickness.

[0092] When measuring the offset and bead height shown in Fig. 16(b) and (c), first, as shown in Fig. 18, the circumferential position of the welded steel pipe where the offset and bead height are to be measured is determined (step S421). Next, the calculation unit 3 calculates the offset and bead height on the image at the position determined by the processing of step S421 (step S422). Then, the calculation unit 3 applies correction to the calculated offset and bead height (step S423). This makes it possible to measure the offset and bead height.

[0093] When measuring the inner surface peaking shown in FIG. 16(d), as shown in FIG. 19-1, first, the calculation unit 3 reproduces the shape of the template K used to measure the inner surface peaking in the image (step S431). At this time, the calculation unit 3 may add a cross-sectional inclination correction of the template K taking into account the shooting direction of the image. Next, the calculation unit 3 calculates the position of the template K when the template K is applied to the edge of the inner surface of the welded steel pipe detected by the detection process (step S432). Then, the calculation unit 3 calculates the gap between the edge of the inner surface of the welded steel pipe and the template K on the image to calculate the inner surface peaking value (step S433). This makes it possible to measure the inner surface peaking.

[0094] When measuring the outer surface peaking shown in FIG. 16(e), as shown in FIG. 19-2, first, the calculation unit 3 determines the circumferential position (measurement position) of the welded steel pipe where the outer surface peaking is measured (step S431-2). At this time, as shown in FIG. 19-3, the measurement position 19C is determined so as to avoid the bead portion 1a. That is, the measurement position 19C is determined so that the measurement position 19C is not located on the bead portion 1a. Next, as shown in FIG. 19-2, the calculation unit 3 places a virtual circle in the image (step S432-2). The virtual circle is a circle that corresponds to the outer surface edge position when the steel pipe is assumed to be a perfect circle. As shown in FIG. 19-3, the virtual circle L is placed so as to pass through points 19A and 19B. These points 19A and 19B are points that are a predetermined distance d away from the straight line Lc in the radial direction of the steel pipe that passes through the measurement position 19C, and are points on the edge of the outer surface of the welded steel pipe detected by the detection process. At this time, it is advisable for the calculation unit 3 to make a correction for the cross-sectional inclination of the virtual circle L taking into account the photographing direction of the image. Next, the calculation unit 3 measures the distance p between the virtual circle L and the measurement position 19C for the edge of the outer surface of the welded steel pipe detected by the detection process, i.e., the outer surface peaking (step S433-2). For example, if the steel pipe is a perfect circle, the outer surface edge of the welded steel pipe detected by the detection process will coincide with the virtual circle except for the bead portion, so the outer surface peaking will be zero. This makes it possible to measure the outer surface peaking.

[0095] When measuring the bead rise angle shown in Fig. 16(f), as shown in Fig. 20, first, an operator determines the circumferential position of the welded steel pipe where the bead rise angle is to be measured (step S441). Next, the calculation unit 3 searches for a tangent line that is in contact with the outer surface of the welded steel pipe and the edge of the bead portion, starting from the position where the bead rise angle is to be measured (step S442). Then, the calculation unit 3 calculates the angle between the searched tangent lines on the image (step S443). This makes it possible to measure the bead rise angle.

[0096] As is clear from the above description, in the measuring device 20 according to the first embodiment of the present invention, the calculation unit 3 takes an image of the cross section of the welded steel pipe 1 including the marker plate 10, converts the image of the cross section into a frontal image having a predetermined resolution using the image of the marker plate 10, detects the edge of the cross section from the frontal image, and calculates a predetermined physical quantity based on the detected edge. The calculation unit 3 also selects a machine learning model that uses the frontal image as input data and the edge of the cross section in the frontal image as output data corresponding to the type of edge to be detected, inputs the frontal image to the selected machine learning model to detect the edge of the cross section in the frontal image, and removes overdetection of edges and interpolates undetected edges based on the connectivity of the detected edges. This allows the predetermined physical quantity in the cross section of the welded steel pipe 1 to be measured accurately in a short time.

[0097] Second Embodiment Fig. 21 is a schematic diagram showing the configuration of a measurement device according to a second embodiment of the present invention. Fig. 22 is a block diagram showing the configuration of the main parts of the measurement device shown in Fig. 21. As shown in Fig. 21, a measurement device 20 according to the second embodiment of the present invention includes an imaging terminal 6, an information processing device 7, and a marker plate 10. In this embodiment, the imaging terminal 6 is configured by a computer including an imaging unit 61, a second display unit 62, a second input unit 63, a second calculation unit 64, a second communication unit 65, and a second storage unit 66.

[0098] Specifically, the imaging terminal 6 is composed of a smartphone, a tablet terminal, a digital camera with a communication function, or the like. In this embodiment, a case will be described in which the imaging terminal 6 is a smartphone. The imaging unit 61 is composed of a camera or the like built into the smartphone. The second display unit 62 is composed of a display or the like built into the smartphone. The second input unit 63 is composed of a touch panel or the like built into the smartphone. The second calculation unit 64 is composed of a CPU, a GPU, or the like built into the smartphone. The second communication unit 65 is provided in the second calculation unit 64, and is composed of a communication interface or the like built into the smartphone. The second storage unit 66 is composed of a memory or the like built into the smartphone.

[0099] The imaging unit 61, second display unit 62, second calculation unit 64, and second storage unit 66 included in the imaging terminal 6 have functions equivalent to the imaging unit 2, calculation unit 3, display unit 4, and storage unit 5 included in the measuring device 20 of the first embodiment. Since the imaging terminal 6 includes the imaging unit 61, it is not possible to set a virtual device (such as a cloud) on the network. The imaging terminal 6, together with the marker plate 10, constitutes a measuring device that measures the physical quantity of the welded portion 1c.

[0100] The information processing device 7 is configured with a computer. An example of the computer is an all-in-one personal computer including a first display unit 71, a first input unit 72, a first calculation unit 73, a first communication unit 74, and a first storage unit 75.

[0101] The first display unit 71 is composed of a monitor, etc. The first input unit 72 is composed of a keyboard, a mouse, a microphone, etc. The first calculation unit 73 is composed of a CPU, a GPU, etc., and also has a first communication unit 74. The first communication unit 74 is provided in the first calculation unit 73, and is composed of a communication interface, etc. The first storage unit 75 is composed of a hard disk, a solid state drive, etc. A virtual device (cloud, etc.) on a network may be used as the information processing device 7.

[0102] The imaging terminal 6 and the information processing device 7 can communicate with each other via the network 8 by the second communication unit 65 and the first communication unit 74 that are included in each of them. As the network 8, a known network technology can be used. For example, the network 8 includes the Internet, a local area network, or a combination of the Internet and a local area network. The communication method can be wired, wireless, or a combination of wired and wireless. The imaging terminal 6 and the information processing device 7 may be configured to be able to communicate directly with each other via the second communication unit 65 and the first communication unit 74 without going through the network 8. In this case, the communication method can be wired, wireless, or a combination of wired and wireless. As a more specific communication method in this case, a predetermined communication signal via a signal line connecting the communication units and a Universal Serial Bus (hereinafter, abbreviated as USB) can be used as an example of wired communication. On the other hand, an example of wireless communication can be Bluetooth (registered trademark).

[0103] In the measuring device 20 of this embodiment, when the physical quantity of the welded portion 1c is measured, the imaging terminal 6 and the information processing device 7 are not limited to being located in the same facility. For example, the imaging terminal 6 and the information processing device 7 may be located in different facilities of the same company. The imaging terminal 6 may be located in the company's facility and the information processing device 7 in another company's facility. The imaging terminal 6 may be located in the other company's facility and the information processing device 7 in the company's facility. In other words, as long as the imaging terminal 6 and the information processing device 7 can communicate with each other, the respective positions when measuring the physical quantity of the welded portion 1c are not particularly limited.

[0104] In the measuring device 20 of this embodiment, image data captured by the imaging unit 61 and information on the measured physical quantities, etc. may be stored not only in the second storage unit 66 of the imaging terminal 6 but also in the first storage unit 75 of the information processing device 7. That is, the imaging terminal 6 may transmit information including the measured physical quantities, etc. from the second communication unit 65 to the first communication unit 74 of the information processing device 7, and record the information in the first storage unit 75 having a larger storage area than the second storage unit 66.

[0105] The information transmitted from the imaging terminal 6 to the information processing device 7 is predetermined information determined in advance according to the purpose of measurement, etc. Examples of the predetermined information include image data captured by the imaging unit 61. The predetermined information includes parameters during processing in the imaging terminal 6. These parameters include, for example, type, position, angle, size, and identifier, which are information related to the pattern 10a arranged on the marker plate 10. Furthermore, the parameters include the resolution in the projective transformation process described above. It is preferable that the operator can specify these processing parameters by operating the second input unit 63 of the imaging terminal 6.

[0106] The predetermined information includes the processing results in the imaging terminal 6. The processing results include text data related to the parameters of the projective transformation. The predetermined information includes basic information such as the date and time of the image capture, the location, the photographer, information related to the imaging terminal 6, and information related to the welded steel pipe 1. The information related to the imaging terminal 6 includes, for example, the identifier and IP address of the imaging terminal 6. The information related to the welded steel pipe 1 includes information that can identify the welded steel pipe 1, such as the object number, object name, lot number, and specifications, and information that characterizes the welded steel pipe 1. The information related to the welded steel pipe 1 is input by an operator using, for example, the second input unit 63 of the imaging terminal 6.

[0107] 23, the measurement device 20 of this embodiment may include an external storage device 9, which is a data recording means using a large-scale database, in addition to the imaging terminal 6 and the information processing device 7. The external storage device 9 is provided in a facility different from that of the imaging terminal 6 and the information processing device 7. In the measurement device 20 of this embodiment, predetermined information and the like may be recorded in the external storage device 9. A virtual device (such as a cloud) on a network may be used as the external storage device 9.

[0108] Fig. 24 is a flowchart showing a flow of processing by the imaging terminal 6 and the information processing device 7 performed by the measurement device 20 shown in Fig. 21. The imaging process S101, the projective transformation process S102, the detection process S103, the measurement process S104, and the display process S105 shown in Fig. 21 are similar to the imaging process S1, the projective transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown in Fig. 2.

[0109] In the measuring device 20 of the present embodiment, first, the imaging unit 61 of the imaging terminal 6 captures an image of the welded portion 1c including the bead portion 1a and the marker plate 10 at a predetermined position (imaging process S101). Next, the second calculation unit 64 of the imaging terminal 6 performs projective transformation on the captured image data (projective transformation process S102). Next, the second calculation unit 64 of the imaging terminal 6 detects the edge position of the cross section of the welded steel pipe 1 using the image data after projective transformation (detection process S103). Next, the second calculation unit 64 of the imaging terminal 6 measures a predetermined physical quantity related to the cross-sectional shape of the welded portion 1c based on the detected edge position (measurement process S104). Next, the second display unit 62 of the imaging terminal 6 displays information on the measured predetermined physical quantity, etc. (display process S105). After that, the second communication unit 65 of the imaging terminal 6 transmits predetermined information, such as information on the measured physical quantity, to the first communication unit 74 of the information processing device 7 (transmission process S106).

[0110] Next, the first communication unit 74 of the information processing device 7 receives predetermined information from the second communication unit 65 of the imaging terminal 6 (reception process S201). After that, the information processing device 7 records the predetermined information by the first storage unit 75 (recording process S202).

[0111] In the measuring device 20 of this embodiment, by using the imaging terminal 6, it is possible to operate not only as a simple measuring system but also as a measurement support system for an operator on-site. For example, as a result of introducing high portability of the imaging terminal 6 and image conversion processing using the marker plate 10, it becomes possible for an operator to measure a predetermined physical quantity of the welded portion 1c from a position of his / her choice. In this case, by simplifying the user interface, it becomes possible to measure the physical quantity without providing the operator with special training for physical quantity measurement. In addition, by displaying the processing results, etc. in real time on the second display unit 62 of the imaging terminal 6, the operator can search for the orientation of the imaging unit 61 suitable for photographing the cross section of the welded steel pipe 1, and the imaging unit 61 can photograph under optimal conditions, resulting in a more preferable state.

[0112] In the measuring device 20 of this embodiment, when a physical quantity is measured by the imaging terminal 6, the operator can change the imaging conditions of the image data of the welded portion 1c on which the marker plate 10 is provided, taken together with the marker plate 10, based on the information including the displayed physical quantity, and use this as a clue to search for an optimal imaging position. Therefore, it is useful as a measurement support system for an on-site operator. In particular, when the imaging terminal 6 having the second display unit 62 is used, the above-mentioned usefulness is further enhanced. At this time, when the above-mentioned imaging process S101 is executed, the imaging conditions in the imaging process S101 can be changed.

[0113] Third embodiment FIG. 25 is a block diagram showing the main components of a measuring device 20 according to a third embodiment of the present invention. As shown in FIG. 25, the measuring device 20 according to this embodiment includes an imaging terminal 6, an information processing device 7, and a marker plate 10. The imaging terminal 6 is configured by a computer including an imaging unit 61, a second display unit 62, a second input unit 63, a second calculation unit 64, a second communication unit 65, and a second storage unit 66. In this embodiment, a smartphone is used as the imaging terminal 6. The imaging terminal 6, together with the marker plate 10, constitutes a measuring device for measuring the physical quantity of the welded portion 1c. The information processing device 7 is configured by a computer including at least a first calculation unit 73, a first communication unit 74, and a first storage unit 75. The imaging terminal 6 and the information processing device 7 are capable of communicating with each other via a network 8 by the second communication unit 65 and the first communication unit 74 that are included in each of them. The network 8 can use the same technology as the network 8 of the second embodiment described above. Furthermore, the imaging terminal 6 and the information processing device 7 may be configured to be able to communicate directly with each other via the second communication unit 65 and the first communication unit 74, without going through the network 8. In this case, the same technique as that for performing direct communication in the second embodiment described above can be used.

[0114] In the measuring device 20 of this embodiment, when a physical quantity is measured, the imaging terminal 6 and the information processing device 7 are not limited to being located within the same facility. For example, the imaging terminal 6 and the information processing device 7 may be located in different facilities of the same company. The imaging terminal 6 may be located in the company's facility and the information processing device 7 in another company's facility. The imaging terminal 6 may be located in another company's facility and the information processing device 7 in the company's facility. In other words, as long as the imaging terminal 6 and the information processing device 7 can communicate with each other, their respective positions when measuring a physical quantity are not particularly limited.

[0115] Fig. 26 is a flowchart showing a flow of processing by the imaging terminal 6 and the information processing device 7 performed by the measurement device shown in Fig. 24. The imaging process S111, the projective transformation process S212, the detection process S213, the measurement process S214, and the display process S114 shown in Fig. 26 are similar to the imaging process S1, the projective transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown in Fig. 3.

[0116] In the measuring device 20 of this embodiment, first, the imaging unit 61 of the imaging terminal 6 captures an image of the welded portion 1c and the marker plate 10 at a predetermined position (imaging process S111). Next, the second communication unit 65 of the imaging terminal 6 transmits information for projective transformation of the image data and physical quantity measurement to the first communication unit 74 of the information processing device 7 (transmission process S112).

[0117] The information transmitted from the imaging terminal 6 to the information processing device 7 is predetermined information determined in advance in accordance with the processing performed by the information processing device 7. The predetermined information for projective transformation of image data and physical quantity measurement includes image data captured by the imaging unit 61. The predetermined information includes parameters for processing in the information processing device 7. The parameters include type, position, angle, size, and identifier, which are information related to the pattern 10a arranged on the marker plate 10. The parameters include resolution in the projective transformation processing S212. It is preferable that the operator can specify the parameters for these processes by operating the second input unit 63 of the imaging terminal 6.

[0118] The predetermined information includes basic information such as the date and time of the photograph, the location, the photographer, information about the equipment used for the photograph, and information about the welded steel pipe 1. Information about the equipment used for the photograph includes, for example, the equipment identifier and IP address. Information about the welded steel pipe 1 includes, for example, information that can identify the welded steel pipe 1, such as the object number, object name, lot number, and specifications, and information that characterizes the welded steel pipe 1.

[0119] Next, the information processing device 7 receives predetermined information from the second communication unit 65 of the imaging terminal 6 by the first communication unit 74. Next, the first calculation unit 73 of the information processing device 7 performs projective transformation on the captured image data (projective transformation process S212). Next, the first calculation unit 73 of the information processing device 7 detects the edge position of the cross section of the welded steel pipe 1 using the image data after projective transformation (detection process S213). Next, the first calculation unit 73 of the information processing device 7 measures a predetermined physical quantity related to the cross-sectional shape of the welded portion 1c based on the detected edge position (measurement process S214). Next, the first communication unit 74 of the information processing device 7 transmits information related to the measured predetermined physical quantity to the second communication unit 65 of the imaging terminal 6 (transmission process S215). After that, the information processing device 7 records the information received from the imaging terminal 6 and the information calculated by the information processing device 7 in the first storage unit 75 (recording process S216).

[0120] Next, the second communication unit 65 of the imaging terminal 6 receives the information on the measured physical quantity from the first communication unit 74 of the information processing device 7 (reception process S113). Next, after the second calculation unit 64 of the imaging terminal 6 receives the information, the second display unit 62 of the imaging terminal 6 displays the information on the measured physical quantity on the second display unit 62 (display process S114).

[0121] The information on the measured physical quantity transmitted from the information processing device 7 to the imaging terminal 6 includes the following. For example, the information on the measured physical quantity includes processed image data. This processed image data includes values ​​of physical quantities for image data captured by the imaging unit 61 of the imaging terminal 6 or image data obtained by projectively transforming the image data. In this case, the imaging terminal 6 can display the received image data as is on the second display unit 62. Furthermore, the information on the measured physical quantity includes text data related to parameters of the projective transformation. In this case, the imaging terminal 6 can display information obtained by analyzing the received text data on the second display unit 62. Furthermore, it is preferable that the imaging terminal 6 processes the image data previously transmitted to the information processing device 7 to create image data to which values ​​of the measured physical quantities are added, and display the image data on the second display unit 62.

[0122] [Fourth embodiment] FIG. 27 is a block diagram showing the main components of a measuring device 20 according to a fourth embodiment of the present invention. As shown in FIG. 27, the measuring device 20 according to this embodiment includes an imaging terminal 6, an information processing device 7, and a marker plate 10. The imaging terminal 6 is configured by a computer including an imaging unit 61, a second calculation unit 64, a second communication unit 65, and a second storage unit 66. In this embodiment, a digital camera equipped with a communication function is used as the imaging terminal 6. The imaging terminal 6 constitutes an imaging device for measuring the physical quantity of the welded portion 1c together with the marker plate 10. The information processing device 7 is configured by a computer including at least a first display unit 71, a first calculation unit 73, a first communication unit 74, and a first storage unit 75. The imaging terminal 6 and the information processing device 7 can communicate with each other via a network 8 by the second communication unit 65 and the first communication unit 74 included in each of them. The network 8 can use the same technology as the network 8 of the second embodiment described above. Furthermore, the imaging terminal 6 and the information processing device 7 may be configured to be able to communicate directly with each other via the second communication unit 65 and the first communication unit 74, without going through the network 8. In this case, the same technique as in the case of performing direct communication in the second embodiment described above can be used.

[0123] In the measuring device 20 of this embodiment, when a predetermined physical quantity is measured, the imaging terminal 6 and the information processing device 7 are not limited to being located within the same facility. For example, the imaging terminal 6 and the information processing device 7 may be located in different facilities of the same company. The imaging terminal 6 may be located in the company's facility and the information processing device 7 in another company's facility. Also, the imaging terminal 6 may be located in another company's facility and the information processing device 7 in the company's facility. In other words, as long as the imaging terminal 6 and the information processing device 7 can communicate with each other, their respective positions when measuring the physical quantity are not particularly limited.

[0124] Fig. 28 is a flowchart showing a flow of processing by the imaging terminal 6 and the information processing device 7 performed by the measuring device 20 shown in Fig. 27. The imaging process S121, the projective transformation process S222, the detection process S223, the measurement process S224, and the display process S225 shown in Fig. 28 are similar to the imaging process S1, the projective transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown in Fig. 3.

[0125] In the measuring device 20 of this embodiment, first, the imaging unit 61 of the imaging terminal 6 captures an image of the welded portion 1c including the bead portion 1a and the marker plate 10 at a predetermined position (imaging process S121). Next, the second communication unit 65 of the imaging terminal 6 transmits information to the first communication unit 74 of the information processing device 7 (transmission process S122). The information transmitted from the imaging terminal 6 to the information processing device 7 includes image data captured by the imaging unit 61.

[0126] Next, the first communication unit 74 of the information processing device 7 receives information including image data from the second communication unit 65 of the imaging terminal 6 (reception process S221). Next, the first calculation unit 73 of the information processing device 7 performs projective transformation on the received image data (projective transformation process S222). Next, the first calculation unit 73 of the information processing device 7 detects the edge position of the cross section of the welded steel pipe 1 from the image data after projective transformation (detection process S223). Next, the first calculation unit 73 of the information processing device 7 measures a predetermined physical quantity related to the cross-sectional shape of the welded portion 1c based on the detected edge position (measurement process S224). Next, the first display unit 71 of the information processing device 7 displays information related to the measured predetermined physical quantity (display process S225). After that, the information processing device 7 records the information received from the imaging terminal 6 and the information calculated by the first calculation unit 73 in the first storage unit 75 (recording process S226).

[0127] It is preferable that the parameters for processing in the information processing device 7 are set in advance. Specifically, the parameters can be determined in advance as one, selected from a plurality of options, or set to a predetermined value for each process. The parameters include type, position, angle, size, and identifier, which are information about the pattern 10a arranged on the marker plate 10. The parameters also include the resolution in the projective transformation process S222, etc.

[0128] The information processing device 7 is input with basic information when each image is captured by the imaging terminal 6. The basic information includes basic information such as the date and time of the image capture, the location, the photographer, information about the imaging terminal 6, and information about the welded steel pipe 1. The information about the imaging terminal 6 includes information such as the identifier and IP address of the imaging terminal 6. The information about the imaging terminal 6 is stored, for example, in the second storage unit 66 of the imaging terminal 6, and is transmitted from the second communication unit 65 to the first communication unit 74 of the information processing device 7 together with the image data captured by the imaging unit 61. The information about the welded steel pipe 1 includes information that can identify the welded steel pipe 1, such as the object number, object name, lot number, and specification, and information that characterizes the welded steel pipe 1. The information about the welded steel pipe 1 is transmitted to the information processing device 7 from an external device by communication.

[0129] Fifth embodiment 29 is a block diagram showing the main configuration of a measuring device according to a fifth embodiment of the present invention. The measuring device 20 of this embodiment includes an imaging terminal 6, a first information processing device 107, a second information processing device 207, and a marker plate 10. In the following, the pass / fail judgment of a welded steel pipe 1 will be described as an example of the process of judging the quality level of an object performed by the second information processing device 207.

[0130] The imaging terminal 6 is configured by a computer including an imaging unit 61, a second display unit 62, a second input unit 63, a second calculation unit 64, a second communication unit 65, and a second storage unit 66. In this embodiment, a smartphone is used as the imaging terminal 6. The imaging terminal 6, together with the marker plate 10, configures a measuring device for measuring a physical quantity of the welded portion 1c.

[0131] The first information processing device 107 is configured by a computer including at least a first display unit 171, a first input unit 172, a first calculation unit 173, a first communication unit 174, and a first storage unit 175. The first information processing device 107 corresponds to the information processing device 7 included in the measuring device 20 of the second embodiment.

[0132] The second information processing device 207 is configured by a computer including at least a third display unit 271, a third input unit 272, a third calculation unit 273, a third communication unit 274, and a third storage unit 275. The third display unit 271, the third input unit 272, the third calculation unit 273, the third communication unit 274, and the third storage unit 275 have functions equivalent to those of the first display unit 71, the first input unit 172, the first calculation unit 173, the first communication unit 174, and the first storage unit 175.

[0133] The imaging terminal 6, the first information processing device 107, and the second information processing device 207 are capable of communicating with each other via the network 8 by the second communication unit 65, the first communication unit 174, and the third communication unit 274 that are included in each of the imaging terminal 6, the first information processing device 107, and the second information processing device 207. The network 8 can use the same technology as the network 8 in the second embodiment described above. Also, the imaging terminal 6, the first information processing device 107, and the second information processing device 207 may be configured to be able to communicate directly with each other via the second communication unit 65, the first communication unit 174, and the third communication unit 274, without going through the network 8. In this case, the same technology as that used in the case of direct communication in the second embodiment described above can be used.

[0134] In the measuring device 20 of the present embodiment, when performing physical quantity measurement, the imaging terminal 6, the first information processing device 107, and the second information processing device 207 are not limited to being in the same facility. For example, the imaging terminal 6, the first information processing device 107, and the second information processing device 207 may be in different facilities of the same company. The imaging terminal 6 and the first information processing device 107 may be in different facilities of the same company, and the second information processing device 207 may be in a facility of another company. The imaging terminal 6 and the second information processing device 207 may be in the company's facility, and the first information processing device 107 may be in the facility of another company. The imaging terminal 6 and the first information processing device 107 may be in the facility of another company, and the second information processing device 207 may be in the company's facility. The imaging terminal 6 and the second information processing device 207 may be in the facility of another company, and the first information processing device 107 may be in the company's facility. The imaging terminal 6 and the second information processing device 207 may be in the facility of another company, and the first information processing device 107 may be in the company's facility. The imaging terminal 6 may be located in a facility of another company, and the first information processing device 107 and the second information processing device 207 may be located in the facility of the company. In other words, as long as the imaging terminal 6, the first information processing device 107, and the second information processing device 207 can communicate with each other, their respective positions when measuring physical quantities are not particularly limited.

[0135] Fig. 30 is a flowchart showing an example of the processing of the imaging terminal 6, the first information processing device 107, and the second information processing device 207 performed by the measuring device 20 shown in Fig. 29. The imaging process S131, the projective transformation process S232, the detection process S233, the measurement process S234, and the display process S134 shown in Fig. 30 are similar to the imaging process S1, the projective transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown in Fig. 3.

[0136] In the measuring device 20 of this embodiment, first, the imaging unit 61 of the imaging terminal 6 captures an image of the welded portion 1c including the bead portion 1a and the marker plate 10 at a predetermined position (imaging process S131). Next, the second communication unit 65 of the imaging terminal 6 transmits information to the first communication unit 174 of the first information processing device 107 and the third communication unit 274 of the second information processing device 207 (transmission process S132). The first communication unit 174 of the first information processing device 107 receives information from the second communication unit 65 of the imaging terminal 6 (reception process S231). The third communication unit 274 of the second information processing device 207 receives information from the second communication unit 65 of the imaging terminal 6 (reception process S331).

[0137] The information transmitted from the imaging terminal 6 to the first information processing device 107 includes image data captured by the imaging unit 61. The information transmitted from the imaging terminal 6 to the first information processing device 107 includes parameters for processing in the imaging terminal 6. These parameters include type, position, angle, size, and identifier, which are information about the pattern 10a arranged on the marker plate 10. The parameters include the resolution in the projective transformation process S232. It is preferable that the operator can specify these processing parameters by operating the second input unit 63 of the imaging terminal 6.

[0138] The information transmitted from the imaging terminal 6 to the second information processing device 207 includes basic information when each image was captured by the imaging terminal 6. The basic information includes basic information such as the date and time of capture, the location, the photographer, information about the imaging terminal 6, and information about the welded steel pipe 1. The information about the imaging terminal 6 includes information such as the identifier and IP address of the imaging terminal 6 stored in the second storage unit 66. The information about the welded steel pipe 1 includes information that can identify the welded steel pipe 1, such as the object number, object name, lot number, and specification, and information that characterizes the welded steel pipe 1. The information about the welded steel pipe 1 is input by an operator using, for example, the second input unit 63 of the imaging terminal 6. The information transmitted from the imaging terminal 6 to the second information processing device 207 includes, for example, the ID of the image data captured by the imaging terminal 6.

[0139] Next, the first calculation unit 173 of the first information processing device 107 performs projective transformation on the received image data (projective transformation process S232). Next, the first calculation unit 173 of the first information processing device 107 detects the edge position of the cross section of the welded steel pipe 1 from the image data after projective transformation (detection process S233). Next, the first calculation unit 173 of the first information processing device 107 measures a predetermined physical quantity related to the cross-sectional shape of the welded portion 1c based on the detected edge position (measurement process S234). The first communication unit 174 of the first information processing device 107 transmits information related to the measured predetermined physical quantity to the third communication unit 274 of the second information processing device 207 (transmission process S235). Thereafter, the first information processing device 107 records the information related to the measured physical quantity in the first storage unit 175 (recording process S236). The information related to the measured physical quantity includes text data related to the parameters of projective transformation.

[0140] Next, the third communication unit 274 of the second information processing device 207 receives information on the measured physical quantity from the first communication unit 174 of the first information processing device 107 (reception process S332). Next, the third calculation unit 273 of the second information processing device 207 performs a process of determining the quality level of the welded steel pipe 1 having the welded part 1c photographed by the imaging terminal 6 and a process of changing the manufacturing conditions based on the information on the measured physical quantity and a plurality of pieces of information on the welded steel pipe 1 (determination process S333). The plurality of pieces of information on the welded steel pipe 1 are information on the design drawing of the welded steel pipe 1 and the shape after manufacturing, information on the effect of a change in manufacturing conditions on the measured physical quantity, information on a method of determining the operation amount of the operating procedure of the manufacturing equipment, and combinations of these, which are previously held by the second information processing device 207. In addition, the plurality of pieces of information also include the judgment criteria described later. It is preferable that the plurality of pieces of information on the welded steel pipe 1 are stored in the third storage unit 275 provided in the second information processing device 207.

[0141] An example of the process for determining the quality level of the welded steel pipe 1 is a process for comparing information about the measured physical quantities with a judgment standard and judging whether the welded steel pipe 1 passes or fails. As a result of this process, information about the quality level of the welded steel pipe 1 is obtained. More specifically, information about the pass / fail judgment of the welded steel pipe 1 is obtained. An example of the process for changing the manufacturing conditions is a process for determining whether or not the manufacturing conditions of the manufacturing equipment that manufactures the welded steel pipe 1 need to be changed based on information about the measured physical quantities, and a process for calculating a specific method for changing the conditions and an amount of operation. An example of the process for changing the manufacturing conditions is a process for determining commands related to the shipping, moving, disposal, and other operations of the welded steel pipe 1 in stock. As a result of these processes, information about changes in the manufacturing conditions of the object (welded steel pipe 1) is obtained.

[0142] Next, the third communication unit 274 of the second information processing device 207 transmits management information for the welded steel pipe 1 based on the measured physical quantities, including one or more of information on the pass / fail judgment of the welded steel pipe 1 and information on the change in the manufacturing conditions, to the second communication unit 65 of the imaging terminal 6 and a control unit of the manufacturing facility (not shown) (transmission process S334).Then, the second information processing device 207 records the information on the pass / fail judgment of the welded steel pipe 1 in the third storage unit 275 (recording process S335).

[0143] The control unit of the manufacturing facility (not shown) receives management information for the welded steel pipe 1 based on the measured physical quantities, including information related to pass / fail judgment of the welded steel pipe 1, from the third communication unit 274 of the second information processing device 207. Furthermore, based on the received management information for the welded steel pipe 1, the control unit of the manufacturing facility (not shown) executes commands related to changes in the manufacturing facility, and to shipping, moving, discarding, and other operations for the welded steel pipe 1 in stock.

[0144] The management information of the welded steel pipe 1 based on the measured physical quantity includes information on the processing of the stocked welded steel pipe 1 as information on changes in the manufacturing conditions of the welded steel pipe 1. The information on the processing of the stocked welded steel pipe 1 includes information on the number, name, lot number, warehouse in which the welded steel pipe 1 is stored, location, scheduled shipping date, and scheduled shipping destination, etc. Furthermore, the information on the processing of the stocked welded steel pipe 1 includes information on the specifications of the warehouse, the grade of each of the welded steel pipes 1 stored in the warehouse, scheduled shipping date, storage location, etc. And / or, the management information of the welded steel pipe 1 based on the measured physical quantity includes information on changing the manufacturing conditions of the object as information on changes in the manufacturing conditions of the welded steel pipe 1. The information on changing the manufacturing conditions of the object includes information on the manufacturing process and its conditions, the date of implementation of each process, other measurement results during manufacturing, the number, name, lot number, etc. of the manufactured welded steel pipe 1. Furthermore, the information on changing the manufacturing conditions of the object includes information including a predetermined physical quantity and information characterizing the welded steel pipe 1, and information on criteria for changing the conditions of each process, etc.

[0145] The usage method of the management information will be described using examples of inventory management and changes in manufacturing conditions. First, the third calculation unit 273 extracts management information from the third storage unit 275 using part or all of the information on the welded steel pipe 1 as a key. Next, the third calculation unit 273 compares the extracted management information with a predetermined physical quantity measured in the measurement process S234 and information characterizing the welded steel pipe 1. If the comparison result satisfies the judgment criterion, the third calculation unit 273 decides not to perform operations such as moving the inventory or changing the conditions of the manufacturing equipment. On the other hand, if the judgment criterion is not met, the third calculation unit 273 decides to perform some operation on the welded steel pipe 1. For example, in the case of inventory management, the third calculation unit 273 decides to move the position of the welded steel pipe 1 according to a predetermined rule obtained from the information characterizing the welded steel pipe 1. Specific methods of moving include moving to a location that shows an appropriate grade, moving to a location scheduled for disposal, etc. Furthermore, for example, in the case of controlling manufacturing equipment, the third calculation unit 273 extracts conditions of the manufacturing equipment that affect a specified physical quantity from the information characterizing the welded steel pipe 1, and determines the operating procedures and operating quantities of the manufacturing equipment.

[0146] Next, when the information regarding the pass / fail judgment of the welded steel pipe 1 is included in the management information of the welded steel pipe 1, the second communication unit 65 of the imaging terminal 6 receives the information regarding the pass / fail judgment of the welded steel pipe 1 from the third communication unit 274 of the second information processing device 207 (reception process S133). After that, the second display unit 62 of the imaging terminal 6 displays the information regarding the pass / fail judgment of the welded steel pipe 1 (display process S134). The measuring device 20 of this embodiment is not limited to transmitting the information regarding the pass / fail judgment of the welded steel pipe 1 judged by the second information processing device 207 to the imaging terminal 6 in real time after the pass / fail judgment is made. The operator operates the second input unit of the imaging terminal 6 to transmit information such as the ID of the image data corresponding to the welded steel pipe 1 for which the operator wishes to know the pass / fail judgment result to the second information processing device 207, and requests information regarding the pass / fail judgment of the welded steel pipe 1. Then, the second information processing device 207 transmits information regarding the pass / fail judgment of the welded steel pipe 1 corresponding to the ID of the image data recorded in the third storage unit 275 to the imaging terminal 6 based on the information such as the ID of the received image data. This allows the operator to use the imaging terminal 6 to measure the physical quantities of the welded steel pipe 1, and then perform inspection such as determining whether the welded steel pipe 1 is pass / fail, manage inventory, and change manufacturing conditions at any time up until the welded steel pipe 1 is shipped as an object, thereby managing quality and manufacturing.

[0147] The present invention can also be applied as a measurement step included in a manufacturing method of an object, and a physical quantity related to a cross-sectional shape of the object may be measured in a known or existing manufacturing step. According to such a manufacturing method of an object, the manufacturing yield of the object can be improved.

[0148] Furthermore, the present invention can be applied to a quality control method for an object, and the quality of the object may be controlled by measuring a physical quantity related to the cross-sectional shape of the object. Specifically, the present invention can measure a physical quantity related to the cross-sectional shape of the object, and control the quality of the object from the measurement result. In the quality control step, based on the measurement result, it is determined whether the physical quantity satisfies a pre-specified control standard, and the quality of the object is controlled. According to such a quality control method for an object, a high-quality welded steel pipe can be provided.

[0149] [Example] Finally, an embodiment of the present invention will be described. In this embodiment, the cross section of the welded part of a welded steel pipe was selected as the object of measurement of the physical quantity. A marker plate 10 with four AR markers was used. A smartphone was used as the imaging terminal 6, and a notebook PC equipped with a graphic board was used as the information processing device 7. Communication between the imaging terminal 6 and the information processing device 7 was realized by data transfer by a hard disk via a USB port. In the physical quantity measurement process, the pipe thickness was measured by photographing 10 different cross sections of the welded steel pipe. The measurement was performed at one location each on the left and right sides of the welded metal bead. Figure 31 is a schematic diagram showing the state in which the position where the pipe thickness was measured is superimposed on the photographed image. In Figure 31, 11 indicates the outer surface of the welded steel pipe, 12 indicates the inner surface of the welded steel pipe, 13 indicates the cross section of the welded steel pipe, and 14 indicates the welded metal bead. The dashed lines 15L and 15R are lines indicating the pipe thickness obtained as a result of measurement using the measurement method according to the present invention, and coincide with the pipe thickness direction. Figure 32 shows the measurement result of the pipe thickness and the error range. In order to evaluate the error of the measured value of the present invention relative to the measured value of the pipe thickness measured manually, the absolute average of the error was calculated and found to be 0.5 mm. This confirmed that the present invention can accurately measure physical quantities such as pipe thickness.

[0150] Although the embodiments of the present invention have been described above, the present invention is not limited by the descriptions and drawings that form part of the disclosure of the present invention according to the present embodiments. In other words, other embodiments, examples, and operation techniques, etc., made by those skilled in the art based on the present embodiments are all included in the scope of the present invention. [Industrial Applicability]

[0151] According to the present invention, it is possible to provide a measurement method, a measurement device, a calculation unit, an imaging terminal, and an imaging system capable of measuring a predetermined physical quantity at a predetermined position of an object in a short time with high accuracy. Moreover, according to the present invention, it is possible to provide a manufacturing method for an object capable of improving the manufacturing yield of the object. Moreover, according to the present invention, it is possible to provide a quality control method for an object capable of providing high-quality objects. Moreover, according to the present invention, it is possible to provide a manufacturing facility for an object capable of improving the manufacturing yield of the object. Moreover, according to the present invention, it is possible to provide an information processing device capable of improving the manufacturing yield of the object and / or providing high-quality objects. [Explanation of symbols]

[0152] 1. Welded steel pipes 1a Bead section 1b Base material 1c Welding 2. Imaging unit 3 Arithmetic section 4 Display 5 Storage section 6. Imaging Device 7. Information processing equipment 8 Network 9 External storage device 10 Marker Plates 10a Pattern 10b opening 11 Exterior 12. Inside 13 Section 14 Welding Reinforcement 19C Measurement position 20. Measurement Equipment 61 Imaging unit 62 2nd display section 63 Second input section 64 2nd calculation section 65 2nd Communications Department 66 2nd memory section 71 1st display section 72 First input section 73 1st calculation section 74 First Communications Department 75 1st memory section 101a Edge 107 First information processing device 171 1st display section 172 First input section 173 1st calculation section 174 First Communications Department 175 1st memory section 207 Second information processing device 271 Third display 272 Third Input Section 273 Third calculation section 274 Third Communications Department 275 Third memory section L Virtual circle

Claims

1. A method for measuring a predetermined physical quantity at a predetermined position on an object, comprising the steps of: a first step of converting an image of the object captured together with a marker plate on which a pattern serving as a length reference for length measurement is drawn, into a front-on image having a predetermined resolution by using an image of the pattern in the image; a second step of detecting an edge of the object from the front-view image; a third step of calculating the predetermined physical quantity based on the detected edge; Including, The second step comprises: Selecting a machine learning model in which the front-view image is used as input data and the edge in the front-view image is used as output data according to the type of the edge to be detected; Detecting the edge in the front-view image by inputting the front-view image obtained by the first step into a selected machine learning model; an edge identification step for removing overdetection of the edges and / or for interpolating underdetection of the edges based on connectivity of the detected edges; Measurement methods, including

2. The measurement method according to claim 1 , further comprising, prior to the first step, an imaging step of capturing an image of the object together with the marker plate.

3. The edge identification step comprises: binarizing the edge image; Dividing the binarized edge image into a plurality of processing units to generate a plurality of divided binary images; determining whether the edge in each divided binary image is valid or invalid by taking into account the edge in an adjacent divided binary image; interpolating the edges determined to be invalid based on the edges determined to be valid; The measurement method according to claim 1 , comprising:

4. A measurement device that measures a predetermined physical quantity at a predetermined position on an object, A marker plate on which a pattern serving as a length reference for length measurement is drawn; an imaging unit that captures an image of the object together with the marker plate; a calculation unit that executes a process of converting the captured image into a front-facing image having a predetermined resolution by using an image of the pattern in the captured image, a process of detecting an edge of the object from the front-facing image, and a process of calculating the predetermined physical quantity based on the detected edge; Equipped with In the process of detecting the edges, the calculation unit executes a process of selecting a machine learning model in which the front-facing image is used as input data and the edges in the front-facing image are used as output data according to the type of the edge to be detected, a process of detecting the edges in the front-facing image by inputting the front-facing image to the selected machine learning model, and a process of removing overdetection of the edges and / or interpolating undetected edges based on connectivity of the detected edges. Measuring equipment.

5. A manufacturing step of the object; A measurement step of measuring a predetermined physical quantity at a predetermined position of an object by using the measurement method according to any one of claims 1 to 3; A method for manufacturing an object, comprising:

6. A measurement step of measuring a predetermined physical quantity at a predetermined position of an object by using the measurement method according to any one of claims 1 to 3; a quality control step of performing quality control of the object based on a measurement result of a predetermined physical quantity obtained by the measurement step; A method for quality control of an object, comprising:

7. A manufacturing facility for manufacturing the object; The measurement device according to claim 4 , which measures a predetermined physical quantity at a predetermined position of the object manufactured by the manufacturing facility; An object manufacturing facility comprising:

8. A calculation unit for measuring a predetermined physical quantity at a predetermined position of an object, A process of detecting an edge of the object from a frontal image calculated from an image of the object taken together with a marker plate on which a pattern serving as a length reference for length measurement is drawn; A process of calculating the predetermined physical quantity based on the detected edge; In addition to carrying out the above, In the process of detecting edges, A process of selecting a machine learning model in which the front-view image is input data and the edge in the front-view image is output data according to the type of the edge to be detected; A process of detecting the edges in the front-view image by inputting the front-view image to the selected machine learning model; removing overdetection of the edges and / or interpolating underdetection of the edges based on connectivity of the detected edges; A calculation unit that executes the above.

9. The calculation unit according to claim 8 , further comprising: a processing step of converting an image of the object into a front-facing image having a predetermined resolution by using the pattern in the image in order to calculate the front-facing image from the image of the object.

10. An imaging terminal for measuring a predetermined physical quantity at a predetermined position of an object, an imaging unit that captures an image of the object together with a marker plate on which a pattern serving as a length reference for length measurement is drawn; A second calculation unit that executes processing on the captured image; Equipped with The second calculation unit is 10. A process of outputting the captured image to an external first calculation unit which is the calculation unit according to claim 8 or 9. and / or a calculation unit according to claim 8 or 9, Imaging device.

11. 1. An imaging system for measuring a predetermined physical quantity at a predetermined position of an object, comprising: A marker plate on which a pattern serving as a length reference for length measurement is drawn; an imaging terminal according to claim 10, which captures an image of the object on which the marker plate is placed; An imaging system comprising:

12. 10. An information processing device comprising: a third calculation unit that executes one or more of a process of changing manufacturing conditions of the object and a process of determining a degree of quality of the object, based on a predetermined physical quantity at a predetermined position of the object calculated by an external first or second calculation unit that is the calculation unit described in claim 8 or 9, and a plurality of pieces of information regarding the object from which the physical quantity was calculated.

13. A manufacturing facility for manufacturing the object; an imaging terminal according to claim 10 for measuring a predetermined physical quantity at a predetermined position of the object manufactured by the manufacturing facility; An object manufacturing facility comprising:

Images