Measurement method, measurement device, object manufacturing method, object quality management method, object manufacturing apparatus, calculation unit, imaging terminal, imaging system, and information processing device

Abstract

A measurement method according to the present invention comprises: a first step of converting an image of an object, which is captured together with a marking plate on which a pattern serving as a length reference for length measurement is drawn, into a facing image having a predetermined resolution by using an image of a pattern in the image; a second step of detecting the edge of the object from the right-facing image; and a third step of calculating a predetermined physical quantity on the basis of the detected edge, the second step including: a step of selecting a machine learning model in which the facing image is used as input data and the edge in the facing image is used as output data according to the type of the edge to be detected; a step of detecting an edge in the facing image by inputting the facing image obtained in the first step into the selected machine learning model; and a step of removing excessive detection of the edge and / or interpolating non-detection of the edge on the basis of the detected connectivity of the edge.

Description

Technical Field

[0001] The present invention relates to a measuring method, a measuring device, a method for manufacturing an object, a method for managing the quality of an object, equipment for manufacturing an object, a computing unit, a photographing terminal, a photographing system, and an information processing device. Background Art

[0002] In recent years, the API5L standard (see non-patent document 1), which is the standard for steel pipes, has been revised to clarify the description of the shape inspection method for steel pipes. Based on this, it can be expected that the management of the shape inspection of steel pipes will be more stringent in the future. On the other hand, the items for the shape inspection of steel pipes involve many aspects, and there are also items that include complex processes. For example, the offset (groove offset of the weld) must be calculated based on the measurement value of the micrometer by placing the micrometer against multiple positions on the outer surface of the steel pipe. In addition, the bending of the steel pipe must be calculated based on the floating of the steel pipe measured by tensioning a horizontal rope in the longitudinal direction of the steel pipe.

[0003] Non-Patent Literature 1: American Petroleum Institute, API Specification 5L Forty-Sixth Edition, April 2018.

[0004] Non-patent document 2: Xavier Soria, Edgar Riba, and Angel Sappa. "Dense ExtremeInception Network: Towards a Robust CNN Model for Edge Detection.", The IEEEWinter Conference on Applications of Computer Vision.2020, Vol.1, p.1912-1921.

[0005] As such, the shape inspection of steel pipes involves many items and processes, necessitating a significant degree of manual labor by factory operators. Furthermore, shape inspections are performed on multiple surfaces and directions of the steel pipe, including the outer surface, inner surface, end faces, and longitudinal direction. Consequently, shape inspections are time-consuming and difficult to perform with high accuracy. Summary of the Invention

[0006] The present invention is completed to solve the above-mentioned problems, and its purpose is to provide a measurement method, a measuring device, a computing unit, a photographing terminal and a photographing system that can measure a specified physical quantity at a specified position of an object in a short time and with high precision. In addition, another purpose of the present invention is to provide a manufacturing method of an object that can improve the manufacturing yield of an object. In addition, another purpose of the present invention is to provide a quality management method of an object that can provide high-quality objects. In addition, another purpose of the present invention is to provide manufacturing equipment for an object that can improve the manufacturing yield of an object. In addition, another purpose of the present invention is to provide an information processing device that can provide an object with an improved manufacturing yield and / or high-quality objects.

[0007] [1] The measuring method according to the present invention measures a predetermined physical quantity at a predetermined position of an object, and includes: a first step of converting an image of the object obtained by photographing the object together with a marking plate depicting a pattern serving as a length reference for length measurement into a facing image having a predetermined resolution using an image of the pattern in the image; a second step of detecting an edge of the object from the facing image; and a third step of calculating the predetermined physical quantity based on the detected edge, wherein the second step includes: a step of selecting a machine learning model that uses the facing image as input data and the edge in the facing image as output data according to the type of the edge to be detected; a step of detecting the edge in the facing image by inputting the facing image obtained by the first step into the selected machine learning model; and an edge recognition step of removing over-detection of the edge and / or interpolating non-detection of the edge based on the connectivity of the detected edge.

[0008] [2] The measuring method involved in the present invention is completed on the basis of the above invention [1], and includes: a shooting step, before the above first step, shooting the image of the above object together with the above marking plate.

[0009] [3] The measurement method involved in the present invention is completed on the basis of the above invention [1] or the above invention [2], and the above edge recognition step includes: a step of binarizing the image of the above edge; a step of dividing the binarized image of the above edge into multiple processing units to generate multiple segmented binary images; a step of determining whether the above edge in each segmented binary image is valid or invalid by considering the above edge in adjacent segmented binary images; and a step of interpolating the above edge determined to be invalid based on the above edge determined to be valid.

[0010] [4] The measuring device involved in the present invention measures a specified physical quantity at a specified position of an object, wherein the device comprises: a marking plate depicting a pattern serving as a length reference for length measurement; a photographing unit photographing an image of the object together with the marking plate; and a calculation unit executing a process of converting the image into a facing image having a specified resolution using an image of the pattern in the photographed image, a process of detecting an edge of the object from the facing image, and a process of calculating the specified physical quantity based on the detected edge, wherein in the process of detecting the edge, the calculation unit executes a process of selecting a machine learning model that uses the facing image as input data and the edge in the facing image as output data according to the type of the edge to be detected, a process of detecting the edge in the facing image by inputting the facing image into the selected machine learning model, and a process of removing over-detection of the edge and / or interpolating undetected edges based on the connectivity of the detected edges.

[0011] [5] The method for manufacturing an object according to the present invention comprises: a step of manufacturing the object; and a step of measuring a predetermined physical quantity at a predetermined position of the object using any one of the inventions [1] to [3] above.

[0012] [6] The quality management method of an object involved in the present invention includes: a measurement step, using any one of the above inventions [1] to [3] to measure a specified physical quantity at a specified position of the object; and a quality management step, performing quality management of the above object based on the measurement result of the specified physical quantity obtained by the above measurement step.

[0013] [7] The manufacturing equipment of the object involved in the present invention comprises: a manufacturing equipment for manufacturing the object; and a measuring device of the above invention [4] for measuring a specified physical quantity at a specified position of the above object manufactured by the above manufacturing equipment.

[0014] [8] The operation unit involved in the present invention is used to measure a specified physical quantity at a specified position of an object, wherein the following processing is performed: a processing of detecting the edge of the above-mentioned object from a facing image calculated based on an image of the above-mentioned object obtained by photographing it together with a marking plate depicting a pattern serving as a length reference for length measurement; and a processing of calculating the above-mentioned specified physical quantity based on the detected above-mentioned edge, and in the processing of detecting the above-mentioned edge, the following processing is performed: a processing of selecting a machine learning model that uses the above-mentioned facing image as input data and the above-mentioned edge in the above-mentioned facing image as output data according to the type of the above-mentioned edge to be detected; a processing of detecting the above-mentioned edge in the above-mentioned facing image by inputting the above-mentioned facing image into the selected above-mentioned machine learning model; and a processing of removing over-detection of the above-mentioned edge and / or interpolating non-detection of the above-mentioned edge based on the connectivity of the above-mentioned detected edge.

[0015] [9] The operation unit involved in the present invention is completed on the basis of the above-mentioned invention [8], wherein, in order to calculate the above-mentioned facing image based on the image of the above-mentioned object, a process of using the above-mentioned pattern in the above-mentioned image to transform the above-mentioned image into a facing image with a specified resolution is performed.

[0016]

[10] The photographing terminal involved in the present invention is used to measure a specified physical quantity at a specified position of an object, wherein it comprises: a photographing unit that photographs the image of the above-mentioned object together with a marking plate depicting a pattern that serves as a length reference for length measurement; and a second operation unit that processes the above-mentioned photographed image, the above-mentioned second operation unit performs processing to output the above-mentioned photographed image to a first operation unit that is external to the operation unit of the above-mentioned invention [8] or the above-mentioned invention [9], and / or the above-mentioned second operation unit is the operation unit of the above-mentioned invention [8] or the above-mentioned invention [9].

[0017]

[11] The photographing system involved in the present invention is used to measure a specified physical quantity at a specified position of an object, and comprises: a marking plate on which a pattern serving as a length reference for length measurement is drawn; and a photographing terminal of the above-mentioned invention

[10] for photographing the above-mentioned object on which the above-mentioned marking plate is provided.

[0018]

[12] The information processing device involved in the present invention comprises: a third operation unit that performs one or more of the processing of changing the manufacturing conditions of the above-mentioned object and the processing of determining the degree of quality of the above-mentioned object based on the specified physical quantity at the specified position of the object calculated by the operation unit of the above-mentioned invention [8] or the above-mentioned invention [9], that is, the external first or second operation unit and a plurality of information related to the above-mentioned object for which the above-mentioned physical quantity is calculated.

[0019]

[13] The manufacturing equipment of the object involved in the present invention comprises: a manufacturing equipment for manufacturing the object; and a shooting terminal of the above invention

[10] for measuring a specified physical quantity at a specified position of the above object manufactured by the above manufacturing equipment.

[0020] According to the measuring method, measuring device, computing unit, photographing terminal and photographing system involved in the present invention, a specified physical quantity at a specified position of an object can be measured in a short time and with high precision. In addition, according to the manufacturing method of an object involved in the present invention, the manufacturing yield of an object can be improved. In addition, according to the quality management method of an object involved in the present invention, a high-quality object can be provided. In addition, according to the manufacturing equipment of an object involved in the present invention, the manufacturing yield of an object can be improved. In addition, according to the information processing device involved in the present invention, an object with an improved manufacturing yield and / or high quality can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram showing the structure of a measuring device as a first embodiment of the present invention.

[0022] Figure 2 This is a flowchart showing the flow of physical quantity measurement processing as the first embodiment of the present invention.

[0023] Figure 3 Yes Figure 2 Flowchart of the process of shooting processing shown.

[0024] Figure 4 Yes Figure 2 Flowchart showing the flow of projection transformation processing.

[0025] Figure 5 Schematic diagram showing an example of an image of a welded portion and a marking plate captured by imaging processing.

[0026] Figure 6 Yes Figure 5 An example of the dimensions of the marker plate is shown in the diagram.

[0027] Figure 7 Yes Figure 2 Flowchart of the detection processing flow shown.

[0028] Figure 8 3 is a schematic diagram showing an example of an edge image obtained by the process of step S32.

[0029] Figure 9 Yes Figure 7 Flowchart showing the flow of edge recognition processing.

[0030] Figure 10 Is used to illustrate Figure 9 FIG. 1 is a diagram showing a specific example of the process of step S334.

[0031] Figure 11 Is used to illustrate Figure 9 FIG. 1 is a diagram showing a specific example of the process of step S335.

[0032] Figure 12 Is used to illustrate Figure 9 FIG. 1 is a diagram showing a specific example of the process of step S336.

[0033] Figure 13 Is used to illustrate Figure 9 FIG. 1 is a diagram showing a specific example of the process of step S336.

[0034] Figure 14 Yes Figure 9 FIG. 1 is a diagram showing an example of an edge image obtained as a result of the processing of steps S334 to S336.

[0035] Figure 15 339 is a diagram showing an example of an edge image obtained as a result of the processing in step S339.

[0036] Figure 16 This diagram illustrates pipe thickness, offset, weld bead height, inner surface peaking, outer surface peaking, and weld bead rise angle.

[0037] Figure 17 This is a flowchart showing the flow of measurement processing when measuring pipe thickness.

[0038] Figure 18 This is a flowchart showing the flow of measurement processing when measuring offset and weld bead height.

[0039] Figure 19-1 This is a flowchart showing the flow of a measurement process when measuring the inner surface peaking degree.

[0040] Figure 19-2 This is a flowchart showing the flow of a measurement process when measuring the outer surface peaking degree.

[0041] Figure 19-3 It is a diagram for explaining the measurement process of the outer surface peaking degree.

[0042] Figure 20 This is a flowchart showing the flow of the measurement process when measuring the weld bead rise angle.

[0043] Figure 21 It is a schematic diagram showing the structure of a measuring device as a second embodiment of the present invention.

[0044] Figure 22 Yes Figure 21 The block diagram of the main structure of the measuring device shown.

[0045] Figure 23 Yes Figure 21 A block diagram showing the main structure of a modified example of the measuring device shown.

[0046] Figure 24 It means by Figure 21 The flowchart shows the flow of processing by the imaging terminal and the information processing device implemented by the measurement device shown.

[0047] Figure 25 This is a block diagram showing the main configuration of a measuring device according to a third embodiment of the present invention.

[0048] Figure 26 It means by Figure 25 The flowchart shows the flow of processing by the imaging terminal and the information processing device implemented by the measurement device shown.

[0049] Figure 27 This is a block diagram showing the main configuration of a measuring device according to a fourth embodiment of the present invention.

[0050] Figure 28 It means by Figure 27 The flowchart shows the flow of processing by the imaging terminal and the information processing device implemented by the measurement device shown.

[0051] Figure 29 This is a block diagram showing a main configuration of a measuring device according to a fifth embodiment of the present invention.

[0052] Figure 30 It means by Figure 29 The flowchart shows the process flow of the imaging terminal, the first information processing device, and the second information processing device implemented by the measurement device shown.

[0053] Figure 31 This is a schematic diagram showing a state where the position for measuring the pipe thickness is superimposed on a captured image.

[0054] Figure 32 It is a graph showing the measurement results and error range of the pipe thickness in Examples. DETAILED DESCRIPTION

[0055] The following describes, with reference to the accompanying drawings, a measurement device and its measurement method according to the first to fifth embodiments of the present invention. In these embodiments, the measurement device measures physical quantities related to the cross-sectional shape of the welded portion of a welded steel pipe. However, the present invention is not limited to these embodiments and is applicable to any process involving measuring physical quantities at predetermined locations on objects prone to edge formation. While details will be described later, examples of physical quantities related to the cross-sectional shape of the welded portion of a welded steel pipe include pipe thickness, offset, weld bead height, inner surface peaking, outer surface peaking, and weld bead rise angle.

[0056] [First embodiment]

[0057] [structure]

[0058] First, refer to Figure 1 The configuration of a measuring device according to a first embodiment of the present invention will be described. Figure 1 Schematic diagram showing the structure of a measuring device as a first embodiment of the present invention. In this embodiment, the measurement object is described as a cross section of a welded portion 1c of a welded steel pipe 1. Figure 1 As shown, a measuring device 20 as a first embodiment of the present invention is a device for measuring physical quantities related to the cross-sectional shape of a weld portion 1 c of a welded steel pipe 1 , and includes an imaging unit 2 , a computing unit 3 , a display unit 4 , a storage unit 5 , and a marking plate 10 .

[0059] The imaging unit 2 is a device for capturing images of the cross section of the welded steel pipe 1 and is comprised of a monochrome camera and a color camera. A color camera is preferably used as the imaging unit 2. The imaging unit 2 is preferably designed to maintain a specified resolution when capturing images of the cross section of the welded steel pipe 1. If the installation position of the imaging unit 2 can be changed within a specified range, it is preferably designed to maintain the specified resolution at all positions within the specified range. Furthermore, the imaging unit 2 captures images of the cross section of the welded steel pipe 1 at specified positions for measuring specified physical quantities.

[0060] The calculation unit 3 is a computing unit that performs image processing such as projection transformation on the image data of the cross section of the welded steel pipe 1 captured by the imaging unit 2 to calculate physical quantities related to the cross-sectional shape of the welded portion 1 c. The calculation unit 3 is composed of a computer equipped with a CPU (Central Processing Unit).

[0061] The display unit 4 is a display unit that displays information including physical quantities related to the cross-sectional shape of the welded portion 1c calculated by the calculation unit 3. The display unit 4 is composed of a liquid crystal display or the like. In addition to displaying the physical quantities calculated by the calculation unit 3, the display unit 4 can also simultaneously display image data of the cross section of the welded steel pipe 1 captured by the imaging unit 2 and image data resulting from image processing performed by the calculation unit 3.

[0062] The storage unit 5 is a storage device that stores information including physical quantities related to the cross-sectional shape of the welded portion 1c calculated by the calculation unit 3. The storage unit 5 is composed of a hard disk drive or the like. The storage unit 5 may also store image data of the cross section of the welded steel pipe 1 captured by the imaging unit 2, image data resulting from image processing performed by the calculation unit 3, and the like.

[0063] When the imaging unit 2 captures an image of the cross section of the welded steel pipe 1, a marking plate 10 is placed on the cross section of the welded steel pipe 1. The marking plate 10 is used to measure the length of the welded steel pipe 1 within its cross section. A pattern 10a of known length, serving as a reference for length measurement, is depicted on its surface. Pattern 10a is preferably a pattern of one or more squares with known side lengths, or one or more AR markers (e.g., Aruco), which are used as markers for three-dimensional measurement.

[0064] The marking plate 10 is preferably mounted using magnets or the like to facilitate easy attachment and detachment to the cross section and surface of the welded steel pipe 1. To improve the accuracy of the projection transformation described below, the number of patterns 10a is preferably as large as possible. The welded portion 1c and the patterns 10a are preferably arranged on the same plane as much as possible. In typical operations, the marking plate 10 is preferably constructed in a small size so that the entire marking plate 10 can be captured when an image of the cross section of the welded steel pipe 1 is captured from a location where the imaging unit 2 is installed.

[0065] In order to realize simple physical quantity measurement, as described in the second embodiment and the like to be described later, the imaging unit 2 , the computing unit 3 , and the display unit 4 may be replaced with an imaging terminal 6 having equivalent functions.

[0066] [Physical quantity measurement processing]

[0067] Next, refer to Figures 2 to 20 The flow of a physical quantity measurement process according to the first embodiment of the present invention using the above-described measuring device 20 will be described. Figure 2 This is a flowchart showing the flow of physical quantity measurement processing as the first embodiment of the present invention. Figure 2 The flowchart shown starts when a command to execute the physical quantity measurement process is input to the measuring device 20 , and the physical quantity measurement process proceeds to the process of step S1 .

[0068] In the process of step S1, the operator operates the imaging unit 2 to capture an image of the weld portion 1c including the weld bead 1a of the welded steel pipe 1 provided with the marking plate 10 (imaging step). The imaging unit 2 inputs image data including the captured image of the weld portion 1c and the marking plate 10 to the computing unit 3 (imaging process). For details of this imaging process, please refer to Figure 3 This will be described later. Thus, the process of step S1 is completed, and the physical quantity measurement process proceeds to the process of step S2.

[0069] In the process of step S2, the computing unit 3 performs projection transformation on the image data input from the imaging unit 2 into the facing image data having a predetermined resolution (projection transformation process). For details of the projection transformation process, please refer to Figures 4 to 6 This will be described later. Thus, the process of step S2 is completed, and the physical quantity measurement process proceeds to the process of step S3.

[0070] In the process of step S3, the calculation unit 3 detects the edge position of the cross section of the welded steel pipe 1 based on the facing image data obtained by the process of step S2 (detection process). For details of the detection process, please refer to Figures 7 to 15 This will be described later. Thus, the process of step S3 is completed, and the physical quantity measurement process proceeds to the process of step S4.

[0071] In the process of step S4, the calculation unit 3 calculates the physical quantity related to the cross-sectional shape of the welded portion 1c based on the edge position detected by the process of step S3 (measurement process). For details of this measurement process, please refer to Figures 16 to 20 This will be described later. Thus, the process of step S4 is completed, and the physical quantity measurement process proceeds to the process of step S5.

[0072] In step S5, the display unit 4 displays information including the physical quantities related to the cross-sectional shape of the welded portion 1c calculated in step S4. The operator can change the imaging conditions of the welded portion 1c based on the information displayed on the display unit 4. This completes step S5, and the series of physical quantity measurement processes ends.

[0073] <Photo Processing>

[0074] Next, refer to Figure 3 The flow of the imaging process in step S1 described above will be described. Figure 3 Yes Figure 2 Flowchart of the process of shooting processing shown. Figure 3 The flowchart shown starts when a command to execute the physical quantity measurement process is input to the measuring device 20 , and the imaging process proceeds to the process of step S11 .

[0075] In step S11, the operator places the marking plate 10 near the weld portion 1c, including the weld bead 1a, of the welded steel pipe 1. At this point, the operator preferably places the marking plate 10 so that the weld portion 1c and the pattern 10a of the marking plate 10 are as flush as possible. This completes step S11, and the imaging process proceeds to step S12.

[0076] In the process of step S12, the operator operates the imaging unit 2 to capture an image of the weld portion 1c, including the weld bead portion 1a, of the welded steel pipe 1 provided with the marking plate 10. At this time, the image of the weld portion 1c is captured in such a manner that a predetermined position for measuring a predetermined physical quantity is included in the image. The predetermined physical quantity is a physical quantity to be measured in the subsequent measurement process (step S4). Moreover, it is preferred that the weld portion 1c and all the patterns 10a depicted on the marking plate 10 are captured at once. The imaging unit 2 inputs the image data including the captured images of the weld portion 1c and the marking plate 10 into the calculation unit 3. Thus, the process of step S12 is completed, and a series of imaging processes are ended.

[0077] Projection Transformation Processing

[0078] Next, refer to Figures 4 to 6 The flow of the projection transformation process in step S2 described above will be described. Figure 4 Yes Figure 2 Flowchart showing the flow of projection transformation processing. Figure 4 The flowchart shown starts when the imaging process is completed, and the projection transformation process proceeds to the process of step S21.

[0079] In step S21, the computing unit 3 detects the position of the pattern 10a of the marking plate 10 from the image data input by the imaging unit 2, and detects the coordinates of the pattern 10a on the image (hereinafter referred to as image coordinates). Here, the computing unit 3 detects the positions of as many patterns 10a as possible from the image data and lists the image coordinates of the positions (center positions or vertices) of the patterns 10a that have been successfully detected.

[0080] Figure 5 : is a schematic diagram showing an example of an image of the welded portion 1c and the marking plate 10 captured by the imaging process. Figure 5 In the example shown, the marking plate 10 is composed of a rectangular flat plate having a rectangular opening 10b in the center. A plurality of patterns 10a are arranged near the four corners of the marking plate 10. The shape of the marking plate 10 and the positions of the patterns 10a are not limited to Figure 5 The shape and position shown.

[0081] The image coordinates of pattern 10a are expressed as the number of pixels along each axis, with the top left vertex of the image as the origin, the right direction of the image as the positive direction of the x-axis, and the bottom direction of the image as the positive direction of the y-axis. Figure 5 If detection of the upper left vertex P11 of the upper left pattern 10a in the image is successful, its image coordinates are determined as follows. Specifically, if x11 pixels are advanced along the x-axis and y11 pixels along the y-axis from the upper left origin of the image, the upper left vertex P11 is reached. Therefore, the calculation unit 3 sets the image coordinates of the upper left vertex P11 to (x11, y11). Similarly, the calculation unit 3 lists the image coordinates (xkn, ykn) of all successfully detected center positions or vertices Pkn (n=1, 2, ..., Nk) for each of all successfully detected patterns k=1, 2, ..., K.

[0082] To obtain a large number of image coordinates from a small number of patterns 10a, a shape with vertices, such as a rectangle, is preferred as the shape of the pattern 10a. On the other hand, to facilitate the detection and image coordinate acquisition of the pattern 10a, it is preferable to use a point-symmetrical shape, such as a circle, and detect the image coordinates of its center. Patterns for which detection and image coordinate acquisition methods have already been established can also be used as the pattern 10a. This completes step S21, and the projection transformation process proceeds to step S22.

[0083] Return to Figure 4 . In the processing of step S22, the operation unit 3 counts the number of vertices Pkn detected in the processing of step S21 to determine whether the projective transformation can be performed. Generally speaking, in order to perform the projective transformation, it is necessary to obtain the correspondence between the image coordinates and the coordinates in the actual three-dimensional space at more than 4 points. Therefore, when the number of detected vertices Pkn is less than 4 points (step S22: No), the operation unit 3 determines that the projective transformation cannot be performed, and ends a series of projective transformation processes after creating a flag indicating that the image transformation has failed. When the projective transformation cannot be performed in the projective transformation process, the physical quantity measurement process skips the detection process and enters the measurement process. On the other hand, when the number of detected vertices Pkn is more than 4 points (step S22: Yes), the operation unit 3 determines that the projective transformation can be performed, and causes the projective transformation process to enter the process of step S23.

[0084] In step S23, the calculation unit 3 uses the image coordinates of the vertex Pkn detected in step S21 to estimate the parameters for the projective transformation of the captured image data and perform a projective transformation on the image data. Specifically, before estimating the projective transformation parameters, the calculation unit 3 sets the resolution r of the image data after the projective transformation as a parameter. Furthermore, during the projective transformation, the calculation unit 3 pre-sets the actual coordinates (hereinafter referred to as "object coordinates") of the vertex Pkn of the pattern 10a. Figure 6 Yes Figure 5 The shape of the marking plate 10 and the position of the pattern 10a are not limited to Figure 6 The shape and position shown.

[0085] like Figure 6 As shown, the object coordinates of the pattern 10a are expressed as the length along each axis (unit is, for example, [mm]) with the upper left corner of the marking plate 10 as the origin, the right direction of the marking plate 10 as the positive direction of the X axis, and the bottom direction of the marking plate 10 as the positive direction of the Y axis. Figure 6 The object coordinates of the upper left vertex P11 of the upper left pattern 10a in the image are determined as follows. Specifically, if one proceeds X11 mm along the X-axis and Y11 mm along the Y-axis from the upper left origin of the marking plate 10, the upper left vertex P11 is reached. Therefore, the calculation unit 3 sets the object coordinates of the upper left vertex P11 to (X11, Y11). Similarly, the calculation unit 3 lists the object coordinates (Xkn, Ykn) of all successfully detected center positions or vertices Pkn (n = 1, 2, ..., Nk) for each of all successfully detected patterns k = 1, 2, ..., K.

[0086] Based on the above, the correspondence between the object coordinates (Xkn, Ykn) and the image coordinates (xkn, ykn) for all successfully detected vertices Pkn becomes clear. Therefore, next, the operation unit 3 calculates the parameters of the projective transformation based on their correspondence. Projective transformation is one of the image deformation processes. Projective transformation deforms the image data by moving the image coordinates (x, y) to another image coordinates (x', y') using the following mathematical formulas (1) and (2). In mathematical formulas (1) and (2), A, B, C, D, E, F, G, and H are the parameters of the projective transformation.

[0087] [Formula 1]

[0088]

[0089] [Formula 2]

[0090]

[0091] By performing projection transformation based on appropriate parameters on the captured image data, the correspondence between a single pixel of the projected image data and the actual length, i.e., the resolution r of the projected image data, can be arbitrarily determined. By utilizing this, by performing projection transformation on image data obtained by capturing the weld 1c of the welded steel pipe 1, physical quantities related to the cross-sectional shape of the weld can be measured on the image.

[0092] In order to calculate the parameters of the projective transformation, at least four points arranged on the same plane are photographed to obtain the correspondence between their object coordinates (X, Y) and image coordinates (x, y). In this embodiment, the correspondence between the object coordinates and the image coordinates is obtained by making the center position and the vertex of the pattern 10a a predetermined configuration. Therefore, when using Figure 1 The imaging unit 2 shown, Figure 21 When the imaging terminal 6 or the like shown captures the mark plate 10, it is preferable to capture not only the welded portion 1c but also the center position or vertex Pkn of the pattern 10a in four or more image data.

[0093] The parameters for the projective transformation are calculated as follows. First, the calculation unit 3 performs a primary transformation on the object coordinates (Xkn, Ykn) of all vertices Pkn using appropriate coefficients a and constants b and c to determine the coordinates (xkn', ykn') that the vertex Pkn should have in the image data after the projective transformation (units are, for example, [pix]). This primary transformation is performed using the following equations (3) and (4).

[0094] [Formula 3]

[0095] xkn′=aXkn+b…(3)[Formula 4]

[0096] ykn′=aYkn+c…(4)

[0097] The coefficient a corresponds to the reciprocal of the resolution r of the image data after projection transformation. Therefore, it is preferable to set the value of the coefficient a to the reciprocal of r based on the predetermined resolution r of the image data after projection transformation. The constants b and c are the x- and y-coordinates of the origin of the object coordinate system in the image data after projection transformation. Therefore, it is preferable to set the values ​​of the constants b and c to b = x0 and c = y0, respectively, based on the predetermined coordinates (x0, y0) that the origin of the object coordinate system should take in the image data after projection transformation.

[0098] For example, consider a case where the resolution r of the projected image data is set to 0.5 [mm / pix], and the position of the origin of the object coordinates in the projected image data is set to (0 [pix], 100 [pix]). In this case, the coefficient a can be set to a = 1 / r = 1 / 0.5 [mm / pix] = 2 [pix / mm], and the constants b and c can be set to 0 [pix] and 100 [pix], respectively.

[0099] Next, the calculation unit 3 calculates the values ​​of the parameters A, B, C, D, E, F, G, and H of the mathematical formula (1) and the mathematical formula (2). The calculation unit 3 determines the value of each parameter so that the image coordinates (xkn, ykn) of the vertex Pkn in the captured image data are transformed into the coordinates (xkn', ykn') that the vertex Pkn should have in the image data after the projective transformation using the mathematical formula (1) and the mathematical formula (2).

[0100] There are several specific methods for calculating each parameter. For example, if the number of vertices Pkn is exactly 4, if the values ​​of the above coordinates are substituted into mathematical formulas (1) and (2), the number of mathematical formulas matches the number of unknowns, so the parameters A, B, C, D, E, F, G, and H can be solved as simultaneous equations. In addition, if the number of vertices Pkn is greater than 4, the values ​​of the parameters A, B, C, D, E, F, G, and H can be determined using a known mathematical method such as the least squares method. The specific method of projective transformation is not limited to the method shown here. Thus, the processing of step S23 is completed, and the projective transformation processing enters the processing of step S24.

[0101] Return to Figure 4 . In the processing of step S24, the operation unit 3 uses the parameters of the projection transformation inferred in the processing of step S23 to perform projection transformation on the image data captured by the shooting process. The parameters of the projection transformation refer to the combination of parameters A, B, C, D, E, F, G, and H determined in the processing of step S23. The image data after the projection transformation is generated by keeping the brightness value or RGB value in the image data before the transformation unchanged, and replacing the coordinates (x, y) in the image data before the transformation with the coordinates (x', y') calculated using mathematical formula (1) and the above-mentioned mathematical formula (2). Thus, the processing of step S24 is completed, and a series of projection transformation processing ends.

[0102] In the present invention, it is assumed that the welded portion 1c of the welded steel pipe 1 and the marking plate 10, which are the objects of measurement, are located on the same plane. This assumption is satisfied when the object of measurement is a plane or a curved surface that can be considered a plane. Specifically, it is preferable that the variation of the curved surface relative to the plane parallel to the surface of the marking plate 10 is controlled within about 2 mm. Generally, the curved surface is not parallel to the image plane ( Figure 5 The direction perpendicular to the xy plane shown in Figure 5 Fluctuations in directions parallel to the z-direction (shown in the figure) cannot be measured from a two-dimensional image. Therefore, when performing three-dimensional measurement of an object, it is preferable to capture the object from multiple directions and acquire multiple images. On the other hand, assuming the object is a flat surface, the present invention omits measurements perpendicular to the image plane. This enables image correction based on projective transformation and simple measurements from two-dimensional images. Depending on the object being measured, even simpler measurements can be performed from a single two-dimensional image.

[0103] <Detection and processing>

[0104] Next, refer to Figures 7 to 15 The flow of the detection process in step S3 described above will be described. Figure 7 Yes Figure 2 Flowchart of the detection processing flow shown. Figure 7 The flowchart shown starts after the image conversion process in step S24 is completed and information regarding the type of the boundary line (edge) to be detected is input, and the detection process proceeds to step S31. In this embodiment, the boundary line to be detected is specified based on physical quantities calculated in subsequent measurement processing from among the boundary line between the cross section and the outer surface of the welded steel pipe 1, the boundary line between the cross section and the inner surface of the welded steel pipe 1, and the boundary line of the chamfered portion within the cross section of the welded steel pipe 1.

[0105] In the processing of step S31, the calculation unit 3 selects an edge detection model to be used in the subsequent processing from the learned edge detection models shown in the following (1) to (3), in combination with the physical quantities calculated in the subsequent measurement processing. The edge detection model is a machine learning model that takes the image data generated by the projection transformation processing as input data and the image data of the edge image in which the edge portion of the welded steel pipe 1 is emphasized in the projection transformation image data as output data. The edge image refers to, for example, a binary image in which the non-edge portion is white and the portion inferred to be the edge is black. Alternatively, the edge image refers to, for example, a monochrome image with 256 grayscale levels in which the pixel value of each pixel of the input image is replaced by the probability or reliability value of each pixel of the input image being an edge, which is normalized to a range of 0 to 255.

[0106] (1) Model for detecting the boundary line between the cross section and the outer surface of the welded steel pipe 1

[0107] (2) Model for detecting the boundary line between the cross section and the inner surface of the welded steel pipe 1

[0108] (3) Model for detecting the boundary line of the chamfered portion in the cross section of the welded steel pipe 1 (the line indicating the boundary between the processed surface and the non-processed surface)

[0109] The edge detection model is generated through machine learning using a plurality of training data consisting of a set of image data after projection transformation and image data of edge images extracted from the image data. The edge detection model preferably includes edge detection methods using image processing, deep learning, and other methods, as well as parameter information related to the edge detection process (e.g., weight information obtained as a result of learning), information customized for detecting the edge position of the cross section of the welded steel pipe 1. In particular, when using deep learning, it is preferable to utilize a model having a network structure customized for edge detection (e.g., see Non-Patent Document 2).

[0110] Depending on the physical quantities calculated in the subsequent measurement process, there may be edges that do not need to be detected, so the operation unit 3 can use only a part of the above-mentioned edge detection model to detect the specified minimum edge. For example, when calculating the pipe thickness in the measurement process, all of the above-mentioned edge detection models (1) to (3) are used. In contrast, when calculating the offset, weld height, outer surface peaking, etc. in the measurement process, it is sufficient to use only the above-mentioned edge detection model (1). In addition, when calculating the inner surface peaking, it is sufficient to use only the above-mentioned edge detection model (2). Which edge detection model to use is determined based on the information related to the edge category input at the beginning of the detection process. Thus, the processing of step S31 is completed, and the detection process enters the processing of step S32.

[0111] In step S32, the computing unit 3 inputs the projectively transformed image data obtained through the image transformation process in step S24 into the edge detection model selected in step S31, thereby extracting an edge image from the projectively transformed image data. The processes of steps S31 and S32 are repeated a number of times equal to the number of boundary line types to be detected. Step S32 is then completed, and the detection process proceeds to step S33.

[0112] In the process of step S33, the computing unit 3 extracts edge positions from the edge image extracted in the process of step S32 (edge ​​recognition process). For details of the edge recognition process, please refer to Figures 8 to 15 This will be described later. Thus, the process of step S33 is completed, and the detection process proceeds to the process of step S34.

[0113] In step S34, the calculation unit 3 determines whether all pre-specified edges have been detected. If all pre-specified edges have been detected (step S34: Yes), the calculation unit 3 ends the series of detection processes. On the other hand, if not all pre-specified edges have been detected (step S34: No), the calculation unit 3 returns the detection process to step S31.

[0114] (Edge recognition processing)

[0115] Next, refer to Figures 8 to 15 The edge recognition process in step S33 described above will be described in detail. Figure 8 An example of the edge image obtained by the process of step S32 is shown. Figure 8 In the figure, dotted line 100 represents the actual edge position, and line 101 represents the portion detected as an edge in the processing of step S32. In the processing of step S32, depending on the properties of the outer surface of the welded steel pipe 1 and the state of the background, there is a possibility of over-detection or under-detection of edges. Therefore, in the edge recognition processing, the operation unit 3 eliminates over-detection of edges and interpolates under-detection of edges based on the connectivity of the detected edges. In the following description, although the projection transformation is performed in the processing of step S24 so that the edge traverses the image in the x-axis direction, the projection transformation can also be performed so that the edge traverses the image in the y-axis direction. When the projection transformation is performed so that the edge traverses the image in the y-axis direction, the edge position can be extracted by replacing the x-axis and y-axis in the following description.

[0116] Figure 9 Yes Figure 7 Flowchart showing the flow of edge recognition processing. Figure 9 The flowchart shown starts when the process of step S32 ends, and the edge recognition process proceeds to the process of step S331.

[0117] In step S331, the computing unit 3 binarizes the edge image generated by step S32. If the edge image is a binary image, this process is unnecessary. If the edge image is a monochrome image with grayscale, the computing unit 3 binarizes the edge image based on an appropriate grayscale threshold. The resulting binary image is, for example, an image in which non-edge portions are white and portions inferred to be edges are black. Step S331 is then completed, and the edge recognition process proceeds to step S332.

[0118] In step S332, the calculation unit 3 initializes an array of flags indicating the validity or invalidity of edges. Specifically, in the image after the projective transformation generated in step S24, the edges are captured transversely to the image's x-axis. Therefore, the calculation unit 3 prepares and initializes, for each x-coordinate, an array storing the edge's y-coordinates and an array of flags indicating the validity or invalidity of the edges. The array storing the edge's y-coordinates stores a number of values ​​equal to the image width, and for each x-coordinate, records the edge's y-coordinate calculated through the processing described below. The array storing the edge's y-coordinates is initialized with a missing value. Furthermore, the array of flags indicating the validity or invalidity of edges stores a number of true and false values ​​equal to the image width, indicating whether the edge's y-coordinate value for each x-coordinate is correct. If the calculated edge's y-coordinate value is correct, the flag is set to valid (true); if it is not correct, the flag is set to invalid (false). The array of flags indicating the validity / invalidity of edges is initialized with an invalid value (false). Thus, the process of step S332 is completed, and the edge recognition process proceeds to the process of step S333.

[0119] In step S333, the calculation unit 3 generates a plurality of segmented binary images by segmenting the binary image (width W, height h) obtained in step S331 into strip images of width 1 and height h. This completes step S333, and the edge recognition process proceeds to step S334.

[0120] Next, the calculation unit 3 executes the processes of steps S334 through S336 in ascending order of the x-coordinates of the segmented binary images generated by the process of step S333. After completing the processes of steps S334 through S336 for a single segmented binary image, the calculation unit 3 extracts the segmented binary image from the x-coordinate position obtained by incrementing the x-coordinate of the processed segmented binary image by 1 and repeats the processes of steps S334 through S336. Due to the x-axis of the image coordinate system, this process means that the generation of strip-shaped segmented binary images is repeated starting from the left end of the original binary image until the right end is reached.

[0121] In the processing of step S334, the operation unit 3 extracts pixels that may be edges from the segmented binary image and determines whether there is a single or multiple edge candidates in the segmented binary image based on the extracted pixels. Specifically, the operation unit 3 first lists the y coordinates of the pixels determined to be edges within the segmented binary image to generate an edge candidate point y coordinate array {yn} (n = 1, 2, ..., Nx). Nx is the number of pixels in the segmented binary image determined to be edges taken from a certain x coordinate. Then, the operation unit 3 calculates the difference Δy between the maximum and minimum values ​​of the edge candidate point y coordinate array {yn} and compares it with a pre-set threshold value TΔy. Based on this processing, it is possible to easily determine whether the edge candidate point y coordinate array {yn} as a whole constitutes a single edge.

[0122] Figure 10 (a) and (b) show specific examples of the processing of step S334. Figure 10 In the examples shown in (a) and (b), in the segmented binary image extracted from the position x = x1, there is only one black edge candidate (edge ​​candidate E1). In this case, the y coordinates of the pixels determined to be edges are likely to be close to each other. On the other hand, in the segmented binary image extracted from the position x = x2, there are two edge candidates (edge ​​candidates E2 and E3). In this case, the y coordinates of the pixels determined to be edges are likely to be scattered over a wide range. Therefore, if the upper limit of the y coordinate width of an edge in the segmented binary image is set as the threshold value TΔy, it is possible to determine whether there is one or more edge candidates in the segmented binary image. Thus, the processing of step S334 is completed, and the edge recognition processing enters the processing of step S335.

[0123] Return to Figure 9. In the processing of step S335, the operation unit 3 calculates the y coordinate of the edge in the segmented binary image based on the edge candidate point y coordinate array {yn} obtained by the processing of step S334. The y coordinate of the edge represents the y coordinate of the edge in a certain x coordinate of the original binary image, and is therefore denoted as yx (x=1, 2, ..., W) below. In the processing of this step S335, processing is performed separately depending on whether there is a single edge candidate or multiple edge candidates in the segmented binary image. In the case where there is a single edge candidate in the segmented binary image, the operation unit 3 uses the average value ynm of the edge candidate point y coordinate array {yn} as the y coordinate yx of the edge in the segmented binary image. On the other hand, in the case where there are multiple edge candidates in the segmented binary image, the operation unit 3 uses the coordinate in the edge candidate point y coordinate array {yn} that is closest to the y coordinate y(x-1) of the edge in the segmented binary image adjacent to the left, that is, the segmented binary image processed immediately before, as the y coordinate yx of the edge. When the y coordinate y(x-1) of the edge in the left adjacent segmented binary image is a missing value, the calculation unit 3 also sets the y coordinate yn of the edge in the segmented binary image to be processed as a missing value.

[0124] Figure 11 (a) and (b) show specific examples of processing when there are multiple edge candidates in the segmented binary image. Figure 11 In the examples shown in (a) and (b), there are multiple edge candidate points P1 to P3 in the segmented binary image extracted from the position x = x3, but the edge candidate point P2, which has a y coordinate close to the edge obtained from the x coordinate position of its left neighbor at x = x3-1, is determined to be a true edge, and its y coordinate is saved as the edge's y coordinate yn. On the other hand, edge candidate points P1 and P3 are determined not to be true edges. The pixels corresponding to edge candidate points P1 and P3 in the figure are indicated by ×. Through this processing, when there is a single edge candidate in the segmented binary image, its edge position can be selected, and when there are multiple edges, the position of the edge that is considered to be closest to its left neighbor is selected. Thus, the processing of step S335 is completed, and the edge recognition processing enters the processing of step S336.

[0125] Return to Figure 9. In the processing of step S336, the operation unit 3 determines whether the y-coordinate yx of the edge calculated in the processing of step S335 is reasonable. Similar to the processing of step S335, the processing of step S336 is processed according to whether there is a single edge candidate or multiple edge candidates in the segmented binary image. In the case where there is a single edge candidate in the segmented binary image, the operation unit 3 compares the difference Δys (=|yx-y(x-1)|) between the y-coordinate yx of the edge at the current x-coordinate position and the y-coordinate y(x-1) of the adjacent left edge with the preset threshold value TΔys. Then, in the case of Δys<TΔys, the operation unit 3 determines that the y-coordinate yx of the edge at the current x-coordinate position is valid. On the other hand, in the case of not Δys<TΔys, the operation unit 3 determines that the y-coordinate yx of the edge at the current x-coordinate position is invalid. Specifically, if Figure 12 (a) Figure 12 As shown in (b), the edge candidates extracted from positions x = x4 and x = x5 are both single. However, at position x = x4, where the y-coordinate difference Δys with the left edge is less than TΔys, the edge is connected and judged to be valid. On the other hand, at position x = x5, where the y-coordinate difference Δys with the left edge is greater than TΔys, the edge is not connected and is judged to be invalid. Invalid pixels are indicated by diagonal lines in the figure.

[0126] On the other hand, when there are multiple edge candidates in the segmented binary image, the operation unit 3 compares the difference Δym (=|yx-y(x-1)|) between the y coordinate y(x-1) of the left adjacent edge adjacent to the y coordinate yx of the edge at the current x-coordinate position and the preset threshold value TΔym. Then, when Δym<TΔym and the y coordinate y(x-1) of the left adjacent edge is valid, the operation unit 3 determines that the y coordinate yx of the edge at the current x-coordinate position is valid. On the other hand, if this is not the case, the operation unit 3 determines that the y coordinate yx of the edge at the current x-coordinate position is invalid. Specifically, if Figure 13 (a) Figure 13 As shown in (b), there are multiple edge candidates in the segmented binary images extracted from the positions x=x6 and x=x7, but at the position x=x6 where the difference Δys between the y coordinates of the edge to the left is less than TΔys, the edge is connected and judged to be valid. On the other hand, at the position x=x7 where the difference Δys between the y coordinates of the edge to the left is greater than TΔys, the edge is not connected and is judged to be invalid. Invalid pixels are represented by oblique lines in the figure. Moreover, with respect to the right neighbor x=x7+1 of x=x7, although the difference Δys between the y coordinates of the edge to the left is less than TΔys, the edge at x=x7 is judged to be invalid.

[0127] Return to Figure 9 The processing of step S336 has the following effect: when the y coordinate of the edge suddenly changes during the processing, it is detected and inferred that it is the result of an erroneous detection in the processing of step S32, and is eliminated. Therefore, the threshold TΔys and the threshold TΔym can both set the maximum change in the y coordinate allowed for the connected edge. The reason for not determining whether the left edge is valid when there is a single edge candidate is that when no edge is detected in the segmented binary image of a certain x coordinate, and a single edge appears in the segmented binary image of its right neighbor, the former edge is definitely invalid. Conversely, the reason for determining whether the left edge is valid when there are multiple edge candidates is that when no edge is detected in the segmented binary image of a certain x coordinate, and multiple edges appear in the segmented binary image of its right neighbor, it is impossible to decide which edge to select, so the determination is postponed until the next single edge candidate appears. This processing expects that although the edges detected in the processing of step S32 include erroneous detections, a single correct edge candidate will appear in the segmented binary image with a certain high probability. Thus, the process of step S336 is completed, and the edge recognition process proceeds to the process of step S337.

[0128] When the above processing of steps S334 to S336 is completed for all segmented binary images, the processing results are stored in both the edge y-coordinate array and the flag array indicating the validity / invalidity determination. Figure 14 A conceptual diagram showing the processing results. First, the image is divided into valid and invalid intervals based on the flag array indicating valid / invalid determination. Plotting the values ​​of the y-coordinate array of edges included in the valid interval results in edge 101a in the figure. Edge 101a is the edge that was correctly detected in step S32. On the other hand, the invalid interval is the interval where no edge was detected in step S32, or where an edge was detected but the correct edge could not be determined.

[0129] Return to Figure 9 In step S337, the operation unit 3 determines whether the processing of steps S334 to S336 has been completed for all segmented binary images. If the result of the determination is that the processing of steps S334 to S336 has been completed for all segmented binary images (step S337: Yes), the operation unit 3 advances the edge detection processing to step S339. On the other hand, if the processing of steps S334 to S336 has not been completed for all segmented binary images (step S337: No), the operation unit 3 increments the x-coordinate of the segmented binary image to be processed by 1 as step S338, and then returns the edge detection processing to step S333.

[0130] In step S339, the calculation unit 3 interpolates the edges of each invalid interval based on the y-coordinate array {yxn} of the edges of the adjacent valid intervals and the corresponding x-coordinate array {xn}. n is the number of edge coordinates within the valid interval used in the interpolation process. As a specific interpolation method, for example, assuming that the edge shape within the invalid interval can be modeled using the quadratic function shown in the following mathematical formula (5), a known fitting method (such as the least squares method) can be applied to {xn} and {yxn}.

[0131] [Formula 5]

[0132] y=ax 2 +bn+c…(5)

[0133] As a result of the interpolation process, when an edge discontinuity occurs between an invalid interval and an adjacent valid interval, the operation unit 3 may further transform the y coordinate of the edge by linear transformation to eliminate the edge discontinuity. Specifically, as a result of the interpolation process based on mathematical formula (5) on an invalid interval with a range of xL≤x≤xR, the coordinates of the left end of the invalid interval are (xL, yL) and the coordinates of the right end are (xR, yR). At this time, the coordinates of the right end of the valid interval adjacent to the left are (xL-1, yL'), and the coordinates of the left end of the valid interval adjacent to the right are (xR+1, yR'), and yL≠yL' or yR≠yR'. At this time, the operation unit 3 uses the mathematical formula (6) shown below to perform a linear transformation on the y coordinate of the invalid interval, thereby eliminating the edge discontinuity between the invalid interval and the valid interval. This is equivalent to subtracting the straight line connecting the endpoints of the valid sections adjacent to both sides of the invalid section from the edge shape formed by the y coordinate of the invalid section calculated by the mathematical formula (5).

[0134] [Formula 6]

[0135] yxn'=yxn+((xR-xn)×(yL-yL')+(xn-xL)×(yR--yR')) / (xR-xL+1)

[0136] …(6)

[0137] Before interpolation processing, pre-processing such as changing a very short valid interval to an invalid interval, or changing a very short invalid interval to a valid interval and changing its y coordinate to an appropriate value (for example, replacing it with the average value of the edge y coordinates of nearby valid intervals, etc.) may be performed. Figure 15This figure shows an example of an edge image obtained as a result of step S339. By combining interpolated edge 101b with edge 101a obtained through the previous processing, false detections (overdetection and underdetection) can be eliminated and edges accurately detected. This completes step S339, and the series of edge recognition processes ends.

[0138] <Measurement Process>

[0139] Next, refer to Figures 16 to 20 The measurement process flow in step S4 will be described in detail. During the measurement process, the computing unit 3 calculates physical quantities of the measurement target through image processing. As described above, the image data of the weld portion 1c, including the weld bead 1a, captured by the welded steel pipe 1, undergoes projection transformation. This allows the computing unit 3 to calculate physical quantities related to the cross-sectional shape of the weld portion based on the image. The following describes the operation of the computing unit 3 when measuring pipe thickness, offset, weld bead height, inner surface peaking, outer surface peaking, and weld bead rise angle.

[0140] like Figure 16 As shown in (a), the pipe thickness represents the thickness of the base material 1b of the welded steel pipe at a position separated by a predetermined distance from the center position of the weld bead 1a in the circumferential direction. Figure 16 As shown in (b), the offset represents the difference between the height position of one end portion of the weld bead 1a in the circumferential direction and the height position of the other end portion of the weld bead 1a in the circumferential direction (deviation of the groove height). Figure 16 As shown in (c), the weld height represents the difference between the height position of the circumferential end of the weld bead portion 1a and the height position of the center portion of the weld bead portion 1a (the height of the weld portion). Figure 16 (d) Figure 16 As shown in (e), the peak degree is an indicator of the sharpness of the butt joint. Figure 16 As shown in (d), the inner surface peaking degree represents the size of the gap between the template paper K in contact with the inner surface of the welded steel pipe and the inner surface of the welded steel pipe. Figure 16 As shown in (e), the outer surface peaking degree represents the amount of unevenness of the actual outer surface relative to the imaginary circle (dashed line portion) along the outer surface of the welded steel pipe. Figure 16 As shown in (f), the weld bead rise angle represents the rise angle of the circumferential end portion of the weld bead portion 1a relative to the surface of the base material 1b of the welded steel pipe (the angle of the end portion where the weld is elevated).

[0141] In measurement Figure 16 When the pipe thickness is as shown in (a), Figure 17As shown, first, the circumferential position of the welded steel pipe at which the thickness is to be measured is determined (step S411). Next, the computing unit 3 locates the welded steel pipe at the position determined in step S411 on the image in the thickness direction and calculates the pipe thickness (step S412). The computing unit 3 then corrects the calculated pipe thickness (step S413). This allows pipe thickness measurement.

[0142] In measurement Figure 16 (b) Figure 16 When the offset and weld height shown in (c) are Figure 18 As shown, first, the circumferential position of the welded steel pipe at which the offset and weld bead height are to be measured is determined (step S421). Next, the computing unit 3 calculates the offset and weld bead height at the position determined in step S421 on the image (step S422). The computing unit 3 then corrects the calculated offset and weld bead height (step S423). This allows the offset and weld bead height to be measured.

[0143] In measurement Figure 16 When the inner surface peaking degree is shown in (d), as Figure 19-1 As shown, the calculation unit 3 first reproduces the shape of the template paper K used to measure the inner surface peaking within the image (step S431). At this point, the calculation unit 3 applies a cross-sectional tilt correction to the template paper K, taking into account the image capture direction. Next, the calculation unit 3 calculates the position of the template paper K when it is placed against the edge of the inner surface of the welded steel pipe detected by the detection process (step S432). The calculation unit 3 then calculates the inner surface peaking value by calculating the gap between the inner surface edge of the welded steel pipe and the template paper K in the image (step S433). This allows the inner surface peaking to be measured.

[0144] In measurement Figure 16 When the outer surface peaking degree is shown in (e), as Figure 19-2 As shown, first, the calculation unit 3 determines the circumferential position (measurement position) of the welded steel pipe to measure the outer surface peaking degree (step S431-2). Figure 19-3 As shown in FIG. 1 , the measurement position 19C is determined so as to avoid the weld bead 1a. That is, the measurement position 19C is determined so as not to be located on the weld bead 1a. Figure 19-2 As shown, the calculation unit 3 configures an imaginary circle in the image (step S432-2). The imaginary circle is a circle corresponding to the outer edge position when the steel pipe is assumed to be a perfect circle. Figure 19-3As shown, the imaginary circle L is configured to pass through points 19A and 19B. These points 19A and 19B are separated by a predetermined distance d from a straight line Lc in the radial direction of the steel pipe passing through measurement position 19C, and are located on the edge of the outer surface of the welded steel pipe detected by the detection process. At this point, the calculation unit 3 applies a cross-sectional tilt correction to the imaginary circle L that takes into account the image capture direction. Next, the calculation unit 3 measures the distance p between the imaginary circle L and measurement position 19C, i.e., the outer surface peaking, for the outer surface edge of the welded steel pipe detected by the detection process (step S433-2). For example, if the steel pipe is a perfect circle, the outer surface edge of the welded steel pipe detected by the detection process coincides with the imaginary circle except for the weld bead, and thus the outer surface peaking is zero. This allows the outer surface peaking to be measured.

[0145] In measurement Figure 16 When the weld bead rise angle is shown in (f), Figure 20 As shown, the operator first determines the circumferential position of the welded steel pipe at which the weld bead rise angle is to be measured (step S441). Next, the calculation unit 3 searches for tangent lines that touch the outer surface of the welded steel pipe and the edge of the weld bead, starting from the position where the weld bead rise angle is to be measured (step S442). The calculation unit 3 then calculates the angle formed by the searched tangent lines on the image (step S443). This allows the weld bead rise angle to be measured.

[0146] As can be seen from the above description, in the measuring device 20 according to the first embodiment of the present invention, the computing unit 3 captures an image of the cross section of the welded steel pipe 1, including the marking plate 10. Using the image of the marking plate 10, the computing unit 3 converts the cross section image into a facing image having a predetermined resolution. Edges of the cross section are detected from the facing image, and a predetermined physical quantity is calculated based on the detected edges. Furthermore, the computing unit 3 selects a machine learning model corresponding to the type of edge to be detected, which takes the facing image as input data and outputs the edges of the cross section in the facing image. By inputting the facing image into the selected machine learning model, the edges of the cross section in the facing image are detected. Based on the connectivity of the detected edges, overdetected edges are eliminated and undetected edges are interpolated. This allows for the measurement of predetermined physical quantities of the cross section of the welded steel pipe 1 in a short time and with high accuracy.

[0147] [Second embodiment]

[0148] Figure 21 It is a schematic diagram showing the structure of a measuring device as a second embodiment of the present invention. Figure 22 Yes Figure 21 The block diagram of the main structure of the measuring device is shown in FIG. Figure 21As shown, a measuring device 20 according to a second embodiment of the present invention includes an imaging terminal 6, an information processing device 7, and a marking plate 10. In this embodiment, the imaging terminal 6 is composed of a computer including an imaging unit 61, a second display unit 62, a second input unit 63, a second computing unit 64, a second communication unit 65, and a second storage unit 66.

[0149] Specifically, the shooting terminal 6 is composed of a smartphone, a tablet terminal, a digital camera with a communication function, etc. In this embodiment, the description is given in the case of using a smartphone as the shooting terminal 6. The shooting unit 61 is composed of a camera built into the smartphone, etc. The second display unit 62 is composed of a display built into the smartphone, etc. The second input unit 63 is composed of a touch panel built into the smartphone, etc. The second computing unit 64 is composed of a CPU, GPU, etc. built into the smartphone. The second communication unit 65 is provided in the second computing unit 64 and is composed of a communication interface built into the smartphone, etc. The second storage unit 66 is composed of a memory built into the smartphone, etc.

[0150] The imaging terminal 6 includes an imaging unit 61, a second display unit 62, a second computing unit 64, and a second storage unit 66, which have the same functions as the imaging unit 2, computing unit 3, display unit 4, and storage unit 5 of the measuring device 20 of the first embodiment. Because the imaging terminal 6 includes the imaging unit 61, it cannot configure virtual devices on a network (e.g., cloud). The imaging terminal 6, together with the marking plate 10, constitutes a measurement device that measures physical quantities of the welded portion 1c.

[0151] The information processing device 7 is constituted by a computer. An example of the computer is an integrated personal computer including a first display unit 71 , a first input unit 72 , a first calculation unit 73 , a first communication unit 74 , and a first storage unit 75 .

[0152] The first display unit 71 is composed of a monitor, etc. The first input unit 72 is composed of a keyboard, a mouse, a microphone, etc. The first computing unit 73 is composed of a CPU, a GPU, etc., and further includes a first communication unit 74. The first communication unit 74 is provided in the first computing unit 73 and is composed of a communication interface, etc. The first storage unit 75 is composed of a hard disk, a solid-state drive, etc. A virtual device on the network (such as a cloud) may also be used as the information processing device 7.

[0153] The photographing terminal 6 and the information processing device 7 can communicate with each other via the network 8 through the second communication unit 65 and the first communication unit 74 respectively. As the network 8, a well-known network technology can be used. For example, it includes the Internet, a local area network, or a network combining the Internet and a local area network. The communication method can use a wired, wireless, or a method combining wired and wireless. In addition, the photographing terminal 6 and the information processing device 7 can also be configured to communicate directly through the second communication unit 65 and the first communication unit 74 without going through the network 8. In this case, a wired, wireless, or a method combining wired and wireless can also be used. As a more specific communication method at this time, as an example of wired communication, a prescribed communication signal with the help of a signal line connecting the communication units, a universal serial bus (hereinafter referred to as USB), etc. can be used. On the other hand, as an example of wireless communication, Bluetooth (registered trademark) etc. can be used.

[0154] In the measuring device 20 of this embodiment, when measuring the physical quantities of the weld portion 1c, the imaging terminal 6 and the information processing device 7 are not limited to being located in the same facility. For example, the imaging terminal 6 and the information processing device 7 may be located in different facilities of the company. Alternatively, the imaging terminal 6 may be located in the company's facility, and the information processing device 7 may be located in another company's facility. Alternatively, the imaging terminal 6 may be located in another company's facility, and the information processing device 7 may be located in the company's facility. In other words, as long as the imaging terminal 6 and the information processing device 7 can communicate with each other, their respective locations when measuring the physical quantities of the weld portion 1c are not particularly limited.

[0155] In the measuring device 20 of this embodiment, image data captured by the imaging unit 61 and information related to the measured physical quantity may be stored not only in the second storage unit 66 of the imaging terminal 6 but also in the first storage unit 75 of the information processing device 7. That is, the imaging terminal 6 may transmit information including the measured physical quantity from the second communication unit 65 to the first communication unit 74 of the information processing device 7, and store the information in the first storage unit 75 having a storage area larger in capacity than the second storage unit 66.

[0156] The information sent from the photographing terminal 6 to the information processing device 7 is predetermined information based on the purpose of measurement, etc. As an example of the predetermined information, it includes image data captured by the photographing unit 61. As the predetermined information, it includes parameters for the processing of the photographing terminal 6. The parameters include, for example, information related to the pattern 10a configured on the marking plate 10, namely, the type, position, angle, size and identifier. In addition, the parameters include the resolution in the above-mentioned projection transformation processing, etc. It is preferred that the parameters of these processes can be specified by the operator operating the second input unit 63 of the photographing terminal 6.

[0157] The prescribed information includes the processing results of the imaging terminal 6. This processing result includes text data related to the parameters of the projection transformation. The prescribed information includes basic information such as the date and time of the imaging, the location, the photographer, information related to the imaging terminal 6, and information related to the welded steel pipe 1. Information related to the imaging terminal 6 includes, for example, the imaging terminal 6 identifier and IP address. Information related to the welded steel pipe 1 includes information that can identify the welded steel pipe 1, such as the object number, object name, batch number, and specifications, as well as information that characterizes the welded steel pipe 1. Information related to the welded steel pipe 1 is input by the operator using, for example, the second input unit 63 of the imaging terminal 6.

[0158] like Figure 23 As shown, the measurement device 20 of this embodiment may also include an external storage device 9, which is a data recording unit utilizing a large-scale database, independently of the imaging terminal 6 and the information processing device 7. The external storage device 9 is installed in a facility separate from the imaging terminal 6 and the information processing device 7. In the measurement device 20 of this embodiment, predetermined information and the like may also be recorded in the external storage device 9. A virtual device on a network (such as a cloud) may be used as the external storage device 9.

[0159] Figure 24 It means by Figure 21 The flowchart of the process flow of the imaging terminal 6 and the information processing device 7 executed by the measurement device 20 shown. Figure 21 The imaging process S101, projection transformation process S102, detection process S103, measurement process S104 and display process S105 shown are the same as those in FIG. Figure 2 The imaging process S1, the projection transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown are similar.

[0160] In the measuring device 20 of this embodiment, the imaging unit 61 of the imaging terminal 6 first captures an image of the weld 1c, including the weld bead 1a, and the marking plate 10 at a predetermined position (imaging process S101). Next, the second computing unit 64 of the imaging terminal 6 performs projection transformation on the captured image data (projection transformation process S102). Next, the second computing unit 64 of the imaging terminal 6 uses the projected image data to detect the edge position of the cross section of the welded steel pipe 1 (detection process S103). Next, the second computing unit 64 of the imaging terminal 6 measures a predetermined physical quantity related to the cross-sectional shape of the weld 1c based on the detected edge position (measurement process S104). Next, the second display unit 62 of the imaging terminal 6 displays information on the measured predetermined physical quantity, etc. (display process S105). Then, the second communication unit 65 of the imaging terminal 6 transmits predetermined information, such as information related to the measured physical quantity, to the first communication unit 74 of the information processing device 7 (transmission process S106).

[0161] Next, the first communication unit 74 of the information processing device 7 receives the prescribed information from the second communication unit 65 of the imaging terminal 6 (receiving process S201). Then, the information processing device 7 records the prescribed information in the first storage unit 75 (recording process S202).

[0162] The measuring device 20 of this embodiment utilizes the imaging terminal 6, making it possible to operate not only as a simple measurement system but also as a measurement support system for on-site operators. For example, the high degree of mobility of the imaging terminal 6 and the image conversion processing using the marking plate 10 allow the operator to measure a predetermined physical quantity of the weld portion 1c from a free position. In this case, by simplifying the user interface, the operator can measure physical quantities without requiring special training for physical quantity measurement. In addition, the imaging terminal 6 displays processing results in real time on the second display unit 62, allowing the operator to search for the orientation of the imaging unit 61 suitable for photographing the cross section of the welded steel pipe 1, enabling optimal conditions for photographing by the imaging unit 61, resulting in a more preferred state.

[0163] In the measuring device 20 of this embodiment, when performing physical quantity measurements using the imaging terminal 6, the operator can change the imaging conditions for the image data captured together with the welded portion 1c provided with the marking plate 10 based on the displayed information containing the physical quantity, thereby providing a clue for searching for the optimal imaging position. Therefore, it is useful as a measurement support system for on-site operators. This usefulness is particularly enhanced when using the imaging terminal 6 equipped with a second display unit 62. In this case, while executing the aforementioned imaging process S101, the imaging conditions in that imaging process S101 can be changed.

[0164] [Third embodiment]

[0165] Figure 25 1 is a block diagram showing the main structure of a measuring device 20 according to a third embodiment of the present invention. Figure 25As shown, the measuring device 20 of this embodiment includes an imaging terminal 6, an information processing device 7, and a marking plate 10. The imaging terminal 6 is composed of a computer equipped with an imaging unit 61, a second display unit 62, a second input unit 63, a second computing unit 64, a second communication unit 65, and a second storage unit 66. In this embodiment, the description is given using a smartphone as the imaging terminal 6. The imaging terminal 6 and the marking plate 10 together constitute a measuring device for measuring physical quantities of the weld portion 1c. The information processing device 7 is composed of a computer equipped with at least a first computing unit 73, a first communication unit 74, and a first storage unit 75. The imaging terminal 6 and the information processing device 7 can communicate with each other via a network 8 using their respective second communication units 65 and first communication units 74. The network 8 can utilize the same technology as the network 8 of the second embodiment described previously. Alternatively, the imaging terminal 6 and the information processing device 7 can communicate directly via the second communication unit 65 and first communication unit 74, rather than via the network 8. In this case, the same technology as the direct communication method described previously in the second embodiment can also be utilized.

[0166] In the measuring device 20 of this embodiment, when performing physical quantity measurement, the imaging terminal 6 and the information processing device 7 are not limited to being located in the same facility. For example, the imaging terminal 6 and the information processing device 7 may be located in different facilities of the company. Alternatively, the imaging terminal 6 may be located in the company's facility, and the information processing device 7 may be located in another company's facility. Alternatively, the imaging terminal 6 may be located in another company's facility, and the information processing device 7 may be located in the company's facility. In other words, as long as the imaging terminal 6 and the information processing device 7 can communicate with each other, their respective locations during physical quantity measurement are not particularly limited.

[0167] Figure 26 It means by Figure 24 Flowchart showing the flow of processing by the imaging terminal 6 and the information processing device 7 executed by the measurement device shown. Figure 26 The imaging process S111, the projection transformation process S212, the detection process S213, the measurement process S214 and the display process S114 are shown in FIG. Figure 3 The imaging process S1, the projection transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown are the same.

[0168] In the measuring device 20 of this embodiment, the imaging unit 61 of the imaging terminal 6 first captures an image of the welded portion 1c and the marking plate 10 at a predetermined position (imaging process S111). Next, the second communication unit 65 of the imaging terminal 6 transmits information used for projective transformation of the image data and physical quantity measurement to the first communication unit 74 of the information processing device 7 (transmission process S112).

[0169] The information sent from the photographing terminal 6 to the information processing device 7 is predetermined information based on the processing performed in the information processing device 7. As the predetermined information for the projective transformation of image data and the measurement of physical quantities, the image data photographed by the photographing unit 61 is included. The predetermined information includes parameters during the processing of the information processing device 7. The parameters include information related to the pattern 10a configured on the marking plate 10, namely, the type, position, angle, size and identifier. The parameters include the resolution in the projective transformation processing S212. Preferably, the parameters of these processes can be specified by the operator operating the second input unit 63 of the photographing terminal 6.

[0170] The prescribed information includes basic information such as the date and time of the photograph, the location, the photographer, the equipment used for the photograph, and information related to the welded steel pipe 1. Information related to the equipment used for the photograph includes, for example, the equipment identifier and IP address. Information related to the welded steel pipe 1 includes, for example, information that can identify the welded steel pipe 1, such as the object number, object name, batch number, and specifications, and information that characterizes the welded steel pipe 1.

[0171] Next, the information processing device 7 receives specified information from the second communication unit 65 of the imaging terminal 6 through the first communication unit 74. Next, the first operation unit 73 of the information processing device 7 performs projection transformation on the captured image data (projection transformation processing S212). Next, the first operation unit 73 of the information processing device 7 uses the image data after projection transformation to detect the edge position of the cross section of the welded steel pipe 1 (detection processing S213). Next, the first operation unit 73 of the information processing device 7 measures a specified physical quantity related to the cross-sectional shape of the welded portion 1c based on the detected edge position (measurement processing S214). Next, the first communication unit 74 of the information processing device 7 sends information related to the measured specified physical quantity to the second communication unit 65 of the imaging terminal 6 (sending processing S215). Then, the information processing device 7 records the information received from the imaging terminal 6 and the information calculated by the information processing device 7 in the first storage unit 75 (recording processing S216).

[0172] Next, the second communication unit 65 of the imaging terminal 6 receives information related to the measured physical quantity from the first communication unit 74 of the information processing device 7 (reception process S113). Then, after the second computing unit 64 of the imaging terminal 6 receives the information, the second display unit 62 of the imaging terminal 6 displays the information related to the measured physical quantity on the second display unit 62 (display process S114).

[0173] The information related to the measured physical quantity sent from the information processing device 7 to the photographing terminal 6 includes the following. For example, the information related to the measured physical quantity includes processing result image data. The processing result image data refers to image data captured by the photographing unit 61 of the photographing terminal 6 or image data obtained by projecting the image data, including the value of the physical quantity. In this case, the photographing terminal 6 can directly display the received image data on the second display unit 62. In addition, the information related to the measured physical quantity includes text data related to the parameters of the projection transformation. In this case, the photographing terminal 6 can display the information obtained by analyzing the received text data on the second display unit 62. In addition, it is preferred that the photographing terminal 6 processes the image data previously sent to the information processing device 7 to create image data with the value of the measured physical quantity added, and display it on the second display unit 62.

[0174] [Fourth embodiment]

[0175] Figure 27 FIG. 2 is a block diagram showing the main structure of a measuring device 20 according to a fourth embodiment of the present invention. Figure 27 As shown, the measuring device 20 of this embodiment includes an imaging terminal 6, an information processing device 7, and a marking plate 10. The imaging terminal 6 is composed of a computer equipped with an imaging unit 61, a second computing unit 64, a second communication unit 65, and a second storage unit 66. In this embodiment, the description is given using a digital camera equipped with a communication function as the imaging terminal 6. The imaging terminal 6 and the marking plate 10 together constitute an imaging device for measuring physical quantities of the weld portion 1c. The information processing device 7 is composed of a computer equipped with at least a first display unit 71, a first computing unit 73, a first communication unit 74, and a first storage unit 75. The imaging terminal 6 and the information processing device 7 can communicate with each other via a network 8 using the second communication unit 65 and the first communication unit 74 respectively. The network 8 can use the same technology as the network 8 of the second embodiment described previously. Alternatively, the imaging terminal 6 and the information processing device 7 can communicate directly via the second communication unit 65 and the first communication unit 74, rather than via the network 8. In this case, the same technology as the direct communication method described previously in the second embodiment can also be used.

[0176] The measuring device 20 of this embodiment is not limited to the situation where the imaging terminal 6 and the information processing device 7 are located in the same facility when measuring a prescribed physical quantity. For example, the imaging terminal 6 and the information processing device 7 may be located in different facilities of the company. Alternatively, the imaging terminal 6 may be located in the company's facility, and the information processing device 7 may be located in another company's facility. In addition, the imaging terminal 6 may be located in another company's facility, and the information processing device 7 may be located in the company's facility. That is, as long as the imaging terminal 6 and the information processing device 7 can communicate with each other, their respective locations when measuring physical quantities are not particularly limited.

[0177] Figure 28 It means in Figure 27 The flowchart of the process flow of the imaging terminal 6 and the information processing device 7 executed by the measurement device 20 shown. Figure 28 The imaging process S121, projection transformation process S222, detection process S223, measurement process S224 and display process S225 shown are the same as those in FIG. Figure 3 The imaging process S1, the projection transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown are the same.

[0178] In the measurement device 20 of this embodiment, the imaging unit 61 of the imaging terminal 6 first captures an image of the weld portion 1c, including the weld bead 1a, and the marking plate 10 at a predetermined position (imaging process S121). Next, the second communication unit 65 of the imaging terminal 6 transmits information to the first communication unit 74 of the information processing device 7 (transmission process S122). The information transmitted from the imaging terminal 6 to the information processing device 7 includes the image data captured by the imaging unit 61.

[0179] Next, the first communication unit 74 of the information processing device 7 receives information including image data from the second communication unit 65 of the imaging terminal 6 (receiving process S221). Next, the first operation unit 73 of the information processing device 7 performs projection transformation on the received image data (projection transformation process S222). Next, the first operation unit 73 of the information processing device 7 detects the edge position of the cross section of the welded steel pipe 1 based on the image data after projection transformation (detection process S223). Next, the first operation unit 73 of the information processing device 7 measures a predetermined physical quantity related to the cross-sectional shape of the weld 1c based on the detected edge position (measurement process S224). Next, the first display unit 71 of the information processing device 7 displays information related to the measured predetermined physical quantity (display process S225). Then, the information processing device 7 records the information received from the imaging terminal 6 and the information calculated by the first operation unit 73 in the first storage unit 75 (recording process S226).

[0180] It is preferable to pre-set parameters for processing by the information processing device 7. Specifically, the parameters can be pre-set to a single value, selected from multiple options, or set to a specific value for each process. Parameters include information related to the pattern 10a placed on the marking plate 10, namely, type, position, angle, size, and identifier. Parameters also include resolution, etc., in the projection transformation process S222.

[0181] Basic information generated when each image is captured by the imaging terminal 6 is input to the information processing device 7. This basic information includes information such as the date and time of capture, location, photographer, information related to the imaging terminal 6, and information related to the welded steel pipe 1. Information related to the imaging terminal 6 includes information such as the imaging terminal 6's identifier and IP address. Information related to the imaging terminal 6 is stored in, for example, the second storage unit 66 of the imaging terminal 6 and transmitted along with image data captured by the imaging unit 61 from the second communication unit 65 to the first communication unit 74 of the information processing device 7. Information related to the welded steel pipe 1 includes information that identifies the welded steel pipe 1, such as the object number, object name, lot number, and specifications, as well as information that characterizes the welded steel pipe 1. Information related to the welded steel pipe 1 is transmitted to the information processing device 7 from an external device via communication.

[0182] [Fifth embodiment]

[0183] Figure 29 This is a block diagram showing the main components of a measuring device according to a fifth embodiment of the present invention. The measuring device 20 of this embodiment includes an imaging terminal 6, a first information processing device 107, a second information processing device 207, and a marking plate 10. The following describes the process of determining the quality of an object by the second information processing device 207, using the acceptance / failure determination of a welded steel pipe 1 as an example.

[0184] The imaging terminal 6 is composed of a computer including an imaging unit 61, a second display unit 62, a second input unit 63, a second computing unit 64, a second communication unit 65, and a second storage unit 66. In this embodiment, a smartphone is used as the imaging terminal 6. The imaging terminal 6, together with the marking plate 10, constitutes a measuring device for measuring physical quantities of the weld portion 1c.

[0185] The first information processing device 107 is constituted by a computer including at least a first display unit 171, a first input unit 172, a first calculation unit 173, a first communication unit 174, and a first storage unit 175. The first information processing device 107 corresponds to the information processing device 7 included in the measurement device 20 of the second embodiment.

[0186] The second information processing device 207 is configured as a computer including at least a third display unit 271, a third input unit 272, a third computing unit 273, a third communication unit 274, and a third storage unit 275. The third display unit 271, the third input unit 272, the third computing unit 273, the third communication unit 274, and the third storage unit 275 have the same functions as the first display unit 71, the first input unit 172, the first computing unit 173, the first communication unit 174, and the first storage unit 175.

[0187] The imaging terminal 6, the first information processing device 107, and the second information processing device 207 can communicate with each other via the network 8 through the second communication unit 65, the first communication unit 174, and the third communication unit 274 respectively. The network 8 can use the same technology as the network 8 of the second embodiment described previously. In addition, the imaging terminal 6, the first information processing device 107, and the second information processing device 207 can also be configured to communicate directly through the second communication unit 65, the first communication unit 174, and the third communication unit 274 without passing through the network 8. In this case, the same technology as the case of direct communication in the second embodiment described previously can also be used.

[0188] When measuring physical quantities, the measuring device 20 of this embodiment is not limited to having the imaging terminal 6, first information processing device 107, and second information processing device 207 located in the same facility. For example, the imaging terminal 6, first information processing device 107, and second information processing device 207 may be located in different facilities of the company. Alternatively, the imaging terminal 6 and first information processing device 107 may be located in different facilities of the company, while the second information processing device 207 may be located in a facility of another company. Alternatively, the imaging terminal 6 and second information processing device 207 may be located in the company's facility, while the first information processing device 107 may be located in a facility of another company. Alternatively, the imaging terminal 6 and first information processing device 107 may be located in a facility of another company, while the second information processing device 207 may be located in the company's facility. Alternatively, the imaging terminal 6 and second information processing device 207 may be located in a facility of another company, while the first information processing device 107 may be located in the company's facility. Alternatively, the imaging terminal 6 may be located in a facility of another company, while the first information processing device 107 may be located in the company's facility. That is, as long as the imaging terminal 6 , the first information processing device 107 , and the second information processing device 207 can communicate with each other, their respective positions during physical quantity measurement are not particularly limited.

[0189] Figure 30 It means by Figure 29 The flowchart of an example of the processing of the imaging terminal 6, the first information processing device 107, and the second information processing device 20 performed by the measurement device 20 shown. Figure 30 The imaging process S131, the projection transformation process S232, the detection process S233, the measurement process S234 and the display process S134 are shown in FIG. Figure 3 The imaging process S1, the projection transformation process S2, the detection process S3, the measurement process S4, and the display process S5 shown are the same.

[0190] In the measurement device 20 of this embodiment, the imaging unit 61 of the imaging terminal 6 first captures an image of the weld portion 1c, including the weld bead 1a, and the marking plate 10 at a predetermined position (imaging process S131). Next, the second communication unit 65 of the imaging terminal 6 transmits information to the first communication unit 174 of the first information processing device 107 and the third communication unit 274 of the second information processing device 207 (transmission process S132). The first communication unit 174 of the first information processing device 107 receives information from the second communication unit 65 of the imaging terminal 6 (reception process S231). The third communication unit 274 of the second information processing device 207 receives information from the second communication unit 65 of the imaging terminal 6 (reception process S331).

[0191] The information transmitted from the imaging terminal 6 to the first information processing device 107 includes image data captured by the imaging unit 61. The information transmitted from the imaging terminal 6 to the first information processing device 107 includes parameters for processing by the imaging terminal 6. These parameters include information related to the pattern 10a arranged on the marking plate 10, namely, the type, position, angle, size, and identifier. The parameters include, for example, the resolution in the projective transformation process S232. These processing parameters are preferably specified by the operator through the second input unit 63 of the imaging terminal 6.

[0192] The information transmitted from the imaging terminal 6 to the second information processing device 207 includes basic information about each image captured by the imaging terminal 6. This basic information includes information such as the date and time of the capture, the location, the photographer, information related to the imaging terminal 6, and information related to the welded steel pipe 1. Information related to the imaging terminal 6 includes information such as the identifier and IP address of the imaging terminal 6 stored in the second storage unit 66. Information related to the welded steel pipe 1 includes information that can identify the welded steel pipe 1, such as the object number, object name, batch number, and specifications, and information that characterizes the welded steel pipe 1. Information related to the welded steel pipe 1 is input by an operator using the second input unit 63 of the imaging terminal 6, for example. The information transmitted from the imaging terminal 6 to the second information processing device 207 includes, for example, the ID of the image data captured by the imaging terminal 6.

[0193] Next, the first computing unit 173 of the first information processing device 107 performs a projection transformation on the received image data (projection transformation process S232). Next, the first computing unit 173 of the first information processing device 107 detects the edge position of the cross section of the welded steel pipe 1 based on the image data after the projection transformation (detection process S233). Next, the first computing unit 173 of the first information processing device 107 measures a predetermined physical quantity related to the cross-sectional shape of the weld portion 1c based on the detected edge position (measurement process S234). The first communication unit 174 of the first information processing device 107 transmits information related to the measured predetermined physical quantity to the third communication unit 274 of the second information processing device 207 (transmission process S235). Then, the first information processing device 107 records the information related to the measured physical quantity in the first storage unit 175 (recording process S236). The information related to the measured physical quantity includes text data related to the parameters of the projection transformation.

[0194] Next, the third communication unit 274 of the second information processing device 207 receives information related to the measured physical quantity from the first communication unit 174 of the first information processing device 107 (reception process S332). Next, the third computing unit 273 of the second information processing device 207 performs processing to determine the quality of the welded steel pipe 1 having the weld portion 1c captured by the imaging terminal 6, and to change the manufacturing conditions (determination process S333), based on the information related to the measured physical quantity and other information related to the welded steel pipe 1. The multiple pieces of information related to the welded steel pipe 1 include information on the design drawings and post-manufacturing shape of the welded steel pipe 1, information related to the impact of changes in manufacturing conditions on the measured physical quantity, information related to the method for determining the operating quantities in the operating process of the manufacturing equipment, and combinations thereof, which are pre-stored in the second information processing device 207. The determination criteria described below are also included in this multiple pieces of information. Preferably, this multiple piece of information related to the welded steel pipe 1 is stored in the third storage unit 275 of the second information processing device 207.

[0195] As an example of a process for determining the quality of the welded steel pipe 1, a process can be exemplified by comparing information related to measured physical quantities with a determination standard to determine whether the welded steel pipe 1 is acceptable. This process results in information related to the quality of the welded steel pipe 1. More specifically, information related to determining whether the welded steel pipe 1 is acceptable is obtained. Furthermore, as an example of a process for changing manufacturing conditions, a process can be exemplified by determining whether the manufacturing conditions of the manufacturing equipment used to manufacture the welded steel pipe 1 need to be changed based on information related to measured physical quantities, as well as the specific method for changing the conditions and the calculation of the amount of work performed. Furthermore, as an example of a process for changing manufacturing conditions, a process can be exemplified by determining commands related to shipping, moving, discarding, or other operations on the welded steel pipes 1 in inventory. As a result of these processes, information related to changes in the manufacturing conditions of the object (welded steel pipe 1) is obtained.

[0196] Next, the third communication unit 274 of the second information processing device 207 transmits management information about the welded steel pipe 1 based on the measured physical quantities, including at least one of information related to the acceptance test of the welded steel pipe 1 and information related to changes in the manufacturing conditions, to the second communication unit 65 of the imaging terminal 6 and the control unit of the manufacturing equipment (not shown) (transmission process S334). The second information processing device 207 then records the information related to the acceptance test of the welded steel pipe 1 in the third storage unit 275 (recording process S335).

[0197] The control unit of the manufacturing equipment (not shown) receives management information about the welded steel pipe 1 based on measured physical quantities, including information related to the acceptance / failure determination of the welded steel pipe 1, from the third communication unit 274 of the second information processing device 207. Based on the received management information about the welded steel pipe 1, the control unit of the manufacturing equipment (not shown) executes commands related to changes in the manufacturing equipment, shipment, movement, disposal, and other operations related to the welded steel pipe 1 in inventory.

[0198] Management information for welded steel pipes 1 based on measured physical quantities includes, as information related to changes in manufacturing conditions for welded steel pipes 1, information related to handling of welded steel pipes 1 in inventory. This information includes information related to the welded steel pipes' 1 serial number, name, lot number, warehouse where they are stored, location, scheduled shipping date, and destination. Furthermore, this information includes information related to warehouse specifications, the grade of each welded steel pipe 1 stored in the warehouse, the scheduled shipping date, and the storage location. Furthermore, management information for welded steel pipes 1 based on measured physical quantities includes, as information related to changes in manufacturing conditions for welded steel pipes 1, information related to changes in manufacturing conditions for welded steel pipes 1. This information includes information related to manufacturing processes and their conditions, implementation dates for each process, other measurement results during manufacturing, and the serial number, name, and lot number of the welded steel pipes 1 produced. Furthermore, the information on the manufacturing conditions of the changed object includes information including predetermined physical quantities, information characterizing the welded steel pipe 1 , and information related to judgment criteria for changing each process condition.

[0199] The method of utilizing management information will be described using the examples of inventory management and changes in manufacturing conditions. First, the third computing unit 273 extracts management information from the third storage unit 275, using some or all of the information related to the welded steel pipe 1 as a key. Next, the third computing unit 273 compares the extracted management information with the specified physical quantities and information characterizing the welded steel pipe 1 measured in measurement process S234. If the comparison result meets the judgment criteria, the third computing unit 273 decides not to perform operations such as moving inventory or changing manufacturing equipment conditions. On the other hand, if the results deviate from the judgment criteria, the third computing unit 273 determines to perform some operation on the welded steel pipe 1. For example, in the case of inventory management, the third computing unit 273 determines to relocate the welded steel pipe 1 according to a specified rule derived from the information characterizing the welded steel pipe 1. Specific methods of relocation include moving the welded steel pipe 1 to a location indicating an appropriate level or moving it to a predetermined disposal location. Furthermore, in the case of manufacturing equipment control, for example, the third computing unit 273 extracts manufacturing equipment conditions that affect the specified physical quantities from the information characterizing the welded steel pipe 1 and determines the operating procedures and quantities of the manufacturing equipment.

[0200] Next, if the management information on the welded steel pipe 1 includes information related to the pass / fail determination of the welded steel pipe 1, the second communication unit 65 of the imaging terminal 6 receives the information related to the pass / fail determination of the welded steel pipe 1 from the third communication unit 274 of the second information processing device 207 (receiving process S133). The second display unit 62 of the imaging terminal 6 then displays the information related to the pass / fail determination of the welded steel pipe 1 (display process S134). The measurement device 20 of this embodiment is not limited to transmitting the information related to the pass / fail determination of the welded steel pipe 1 determined by the second information processing device 207 to the imaging terminal 6 in real time after the pass / fail determination. The operator operates the second input unit of the imaging terminal 6 to transmit information such as the ID of the image data corresponding to the welded steel pipe 1 for which the pass / fail determination result is to be desired to the second information processing device 207, thereby requesting information related to the pass / fail determination of the welded steel pipe 1. Furthermore, based on information such as the ID of the received image data, the second information processing device 207 transmits information related to the pass / fail determination of the welded steel pipe 1 corresponding to the ID of the image data, which is stored in the third storage unit 275, to the imaging terminal 6. This allows for inspections such as pass / fail determination of the welded steel pipe 1, inventory management, and changes to manufacturing conditions to manage quality and manufacturing at any time between the time the operator uses the imaging terminal 6 to measure the physical quantity of the welded steel pipe 1 and the time the welded steel pipe 1 is shipped.

[0201] Furthermore, the present invention can be applied as a measurement step included in a method for manufacturing an object, and can measure physical quantities related to the cross-sectional shape of the object in a known or existing manufacturing step. According to this method for manufacturing an object, the manufacturing yield of the object can be improved.

[0202] Furthermore, the present invention can be applied to a method for quality control of objects, which can be performed by measuring physical quantities related to the cross-sectional shape of the object. Specifically, the present invention can measure physical quantities related to the cross-sectional shape of the object and perform quality control based on the measurement results. In the quality control step, the physical quantities are determined based on the measurement results to determine whether they meet pre-specified management criteria, thereby managing the quality of the object. This method for quality control of objects can provide high-quality welded steel pipes.

[0203] [Example]

[0204] Finally, an embodiment of the present invention is described. In this embodiment, the cross section of the welded portion of a welded steel pipe is selected as the object of measurement of the physical quantity. As the marking plate 10, a marking plate equipped with four AR markers is used. A smartphone is used as the photographing terminal 6, and a laptop computer equipped with a graphics board is used as the information processing device 7. Communication between the photographing terminal 6 and the information processing device 7 is achieved by transmitting and receiving data using a hard disk via a USB port. In the physical quantity measurement process, 10 different cross sections of the welded steel pipe are photographed to measure the pipe thickness. Measurements were performed at one point on the left and one point on the right with the weld pile height as the center. Figure 31 This is a schematic diagram showing a state where the position for measuring the pipe thickness is superimposed on a captured image. Figure 31 Reference numeral 11 denotes the outer surface of the welded steel pipe, reference numeral 12 denotes the inner surface of the welded steel pipe, reference numeral 13 denotes the cross section of the welded steel pipe, and reference numeral 14 denotes the weld height. Dashed lines 15L and 15R indicate the pipe thickness as measured using the measurement method of the present invention and are aligned with the pipe thickness direction. Figure 32 The measurement results and error range for pipe thickness are shown. To evaluate the error between the measured values ​​of the present invention and those obtained manually, the absolute value of the error was averaged and found to be 0.5 mm. This confirms that the present invention can measure physical quantities such as pipe thickness with high accuracy.

[0205] While the embodiments of the invention developed by the present inventors have been described above, the present invention is not limited to the description and drawings of these embodiments, which constitute part of the disclosure of the present invention. In other words, other embodiments, examples, and application techniques developed by those skilled in the art based on these embodiments are all within the scope of the present invention.

[0206] Industrial applicability

[0207] According to the present invention, a measurement method, a measurement device, a computing unit, a photographing terminal, and a photographing system can be provided that can measure a specified physical quantity at a specified position of an object in a short time and with high precision. In addition, according to the present invention, a method for manufacturing an object that can improve the manufacturing yield of an object can be provided. In addition, according to the present invention, a quality management method for an object that can provide high-quality objects can be provided. In addition, according to the present invention, a manufacturing device for an object that can improve the manufacturing yield of an object can be provided. In addition, according to the present invention, an information processing device that improves the manufacturing yield of an object and / or can provide high-quality objects can be provided.

[0208] Description of Reference Numerals

[0209] 1…welded steel pipe; 1a…weld bead; 1b…base metal; 1c…weld portion; 2…imaging unit; 3…calculation unit; 4…display unit; 5…storage unit; 6…imaging terminal; 7…information processing device; 8…network; 9…external storage device; 10…marker plate; 10a…pattern; 10b…opening; 11…outer surface; 12…inner surface; 13…cross section; 14…weld pile height; 19c…measurement position; 20…measuring device; 61…imaging unit; 62…second display unit; 63…second input unit; 64…second calculation unit; 65…second communication unit; 66… 6…second storage unit; 71…first display unit; 72…first input unit; 73…first operation unit; 74…first communication unit; 75…first storage unit; 101a…edge; 107…first information processing device; 171…first display unit; 172…first input unit; 173…first operation unit; 174…first communication unit; 175…first storage unit; 207…second information processing device; 271…third display unit; 272…third input unit; 273…third operation unit; 274…third communication unit; 275…third storage unit; L…imaginary circle.

Claims

1. A measurement method for measuring a predetermined physical quantity at a predetermined position of an object, characterized in that: include: The first step is to convert an image of the object captured together with a marker plate having a pattern serving as a length reference for length measurement into a facing image having a predetermined resolution using an image of the pattern in the image. The second step is to detect the edge of the object from the facing image; as well as The third step is to calculate the prescribed physical quantity based on the detected edge. The second step includes: A step of selecting a machine learning model that takes the facing image as input data and the edge in the facing image as output data according to the type of the edge to be detected; a step of detecting the edge in the facing image by inputting the facing image obtained through the first step into the selected machine learning model; and An edge identification step is performed to remove over-detection of edges and / or interpolate under-detection of edges based on the connectivity of the detected edges.

2. The measuring method according to claim 1, wherein include: A photographing step of photographing an image of the object together with the marking plate before the first step.

3. The measuring method according to claim 1 or 2, characterized in that: The edge identification step includes: A step of binarizing the edge image; a step of dividing the edge image obtained by binarization into a plurality of processing units to generate a plurality of segmented binary images; a step of determining validity or invalidity of the edge in each segmented binary image by taking into account the edge in the adjacent segmented binary image; and A step of interpolating the edges determined to be invalid based on the edges determined to be valid.

4. A measuring device for measuring a predetermined physical quantity at a predetermined position of an object, characterized in that: have: A marking plate having a pattern drawn thereon that serves as a length reference for length measurement; a photographing unit, which photographs the image of the object together with the marking plate; as well as a calculation unit that performs processing for converting the image into a facing image having a predetermined resolution using the image of the pattern in the captured image, processing for detecting an edge of the object from the facing image, and processing for calculating the predetermined physical quantity based on the detected edge, In the process of detecting the edge, the operation unit performs a process of selecting a machine learning model that uses the facing image as input data and the edge in the facing image as output data according to the type of the edge to be detected, a process of detecting the edge in the facing image by inputting the facing image into the selected machine learning model, and a process of removing over-detection of the edge and / or interpolating non-detection of the edge based on the connectivity of the detected edge.

5. A method for manufacturing an object, characterized in that: include: the steps involved in making the object; and The measuring step comprises measuring a predetermined physical quantity at a predetermined position of the object using the measuring method according to any one of claims 1 to 3.

6. A method for quality management of an object, characterized in that: include: A measuring step of measuring a predetermined physical quantity at a predetermined position of the object using the measuring method according to any one of claims 1 to 3; and The quality control step performs quality control on the object based on the measurement result of the predetermined physical quantity obtained in the measurement step.

7. A manufacturing device for an object, characterized in that: have: Manufacturing equipment, used to manufacture objects; and The measuring device according to claim 4 is used to measure a predetermined physical quantity at a predetermined position of the object manufactured by the manufacturing equipment.

8. A calculation unit for measuring a predetermined physical quantity at a predetermined position of an object, characterized in that: Perform the following processing: a process of detecting an edge of the object from a facing image calculated from an image of the object captured together with a marker plate having a pattern serving as a length reference for length measurement; and a process of calculating the predetermined physical quantity based on the detected edge, And in the process of detecting the edge, the following processing is performed: selecting, based on the type of edge to be detected, a machine learning model that uses the facing image as input data and the edge in the facing image as output data; detecting the edges in the facing image by inputting the facing image into the selected machine learning model; as well as The process of removing over-detection of the edges and / or interpolating under-detection of the edges is performed based on the connectivity of the detected edges.

9. The computing unit according to claim 8, wherein: In order to calculate the facing image from the image of the object, a process of converting the image into a facing image having a predetermined resolution using the pattern in the image is performed.

10. A photographing terminal for measuring a predetermined physical quantity at a predetermined position of an object, characterized in that: have: an imaging unit that captures an image of the object together with a marking plate on which a pattern serving as a length reference for length measurement is drawn; and The second computing unit processes the captured image. The second computing unit performs a process of outputting the captured image to an external first computing unit that is the computing unit according to claim 8 or 9, and / or the second computing unit is the computing unit according to claim 8 or 9.

11. A photographing system for measuring a predetermined physical quantity at a predetermined position of an object, characterized in that: have: A marking plate having a pattern drawn thereon that serves as a length reference for length measurement; and The photographing terminal described in claim 10 photographs the object on which the marking plate is provided.

12. An information processing device, characterized in that: have: The third operation unit performs one or more of a process of changing the manufacturing conditions of the object and a process of determining the degree of quality of the object based on a specified physical quantity at a specified position of the object calculated by the first operation unit or the second operation unit which is external to the operation unit as described in claim 8 or 9 and a plurality of information related to the object for which the physical quantity is calculated.

13. A manufacturing device for an object, characterized in that: have: Manufacturing equipment, used to manufacture objects; and The photographing terminal described in claim 10 is used to measure a specified physical quantity at a specified position of the object manufactured by the manufacturing equipment.

Images